Digital course evaluations too often turn into a cesspit in which lecturers are jeered at until they cry. In addition, women and lecturers of colour structurally receive lower evaluations. These are the reasons for Professor Henk van Houtum to say: stop using them. But the university doesn’t want to take that step.

Nasty sneers, abusive insults and sometimes even a tirade of swears. The open questions in course evaluations are often filled with reactions that would simply disgust you.

‘They sometimes really cross a line,’ said Pieter Zwaan, chair of the Public Administration programme committee. ‘Some remarks are completely unconstructive. That can be very painful, especially for a beginning lecturer.’ So some colleagues even skip the open questions completely. ‘They say: I want to improve my lessons, but this is no help whatsoever.’

Lecturers in tears

Some lecturers lie awake at night because of the evaluations, according to Professor of Political Geography Henk van Houtum. As head of the programme committee, he sometimes had weeping lecturers in his office. ‘They’d been confronted with negative remarks. One single line can have an enormous impact.’

It’s a well-known fact that angry students sometimes vent their frustrations at lecturers in the evaluation forms. But research shows that not every lecturer is measured by the same standards. Study after study reveals that women and lecturers of colour are criticised more often. In addition, the appearance, sexual preference and age of the lecturer play a role.

This discrimination isn’t restricted to the open questions but is evident throughout the evaluation, where the same racist and sexist pattern can be seen. Even in an experiment with a female avatar, the female variant received a considerably poorer evaluation from students than her male counterpart did.

‘The evaluations don’t measure quality at all but rather student satisfaction’

Ezra Delahaye, policy officer and chair of the Works Council, said that this problem also exists at the Faculty of Science (FNWI). ‘We too have noticed that women and colleagues of colour are sometimes heavily criticised in course evaluations.’

He pointed out that where a lecturer comes from can sometimes make a big difference. ‘English may be the native language of an Indian lecturer but, because students are not used to the accent, this lecturer receives lower evaluations on language skills.’

Discrimination and sexism

The findings touch on a broader social debate about themes such as racism and sexism, in which it is becoming increasingly evident that not everyone is looked at in the same way. In that sense, the university is a mirror of society.

Van Houtum finds it ‘extremely painful’ that the university’s own quality evaluation discriminates against lecturers. What bothers him is that the scores from the evaluations are used as an indicator of quality by, for example, the programme committee or the faculty board, in hiring and appointing. ‘There are even teaching prizes that are awarded partly on the basis of student evaluations.’

The question is whether you can actually compare scores between lecturers. In addition to discrimination, research has shown that a methodological course often receives lower scores than a content-based course. In addition, students show more appreciation for courses in which lecturers simplify the material and lead students by the hand. Van Houtum: ‘That manner of teaching is actually the opposite of what you want. Of course you want stimulating and challenging teaching.’

‘The evaluations don’t measure quality at all; at best, they just say something about student satisfaction,’ emphasised the professor. ‘The danger is that lecturers are going to try to please students in order to get higher scores.’

Stop

These points of criticism combined with the sometimes extremely low percentage of response to the questionnaires all lead to the question of how much value the course evaluations actually have. Should the university simply carry on as before? Make changes? Completely do away with these evaluations?

Van Houtum says it’s ‘incomprehensible’ that the evaluations still take place. ‘We’re talking about a research instrument that would never be allowed in a Bachelor’s thesis, that discriminates against your own employees and that fails to measure what it’s intended to measure. Is more proof needed to immediately stop these evaluations?’

Nevertheless, few policy officers share Van Houtum’s strong opinion. The general attitude: the evaluations have serious shortcomings, but it’s not a good idea to do away with them. Despite the objections, FNWI policy officer Delahaye doesn’t want to stop using the course evaluations. He emphasised that they’ve also made a positive contribution to the quality of education.

‘A low score can be of value to a lecturer’

Delahaye: ‘At times we’ve had lecturers whose teaching was mediocre and who hardly listened to student feedback. Partly thanks to the course evaluations, we gained insight into the problem and it hardly ever happens now. A lower score can also be of value to a lecturer. It offers a mutual starting point to see what can be improved.’

This is also the opinion of Jurian van der Waal, a Master’s student in Business Administration. He thinks that lecturers can retrieve a lot of valuable information from the evaluations. Primarily because they’re anonymous, which allows students to freely voice their opinions.

‘You shouldn’t forget the balance of power between a student and a lecturer. During the course, a student will never give an honest opinion if they still have to submit a paper. And even after this, evaluating a lecturer is a sensitive matter. This is why it’s important that the evaluations offer the opportunity to give feedback safely. The open questions in particular allow you to provide context to your answers.’

Tirade of swears

If the course evaluations are not done away with, then lecturers want to make them milder. At the FNWI the open questions are currently being rewritten to elicit more constructive feedback, and other changes are also being made. ‘This should improve the questionnaires and prevent students from lashing out at lecturers,’ said Delahaye.

The Faculty of Arts is also increasingly interested in qualitative feedback, for example in the form of group discussions. Policy officer Christa van Mourik: ‘They often offer the lecturer very good feedback.’

‘I think it’s great to talk with students, but then openly’

At the same time, the faculty doesn’t want to eliminate the traditional questionnaires. The percentage of responses is high compared with other faculties and the feedback is usually constructive. Nevertheless, the faculty did have trouble with abusive insults from students when the digital questionnaires were introduced in 2016.

‘We got the impression that many students failed to realise that lecturers get to read the evaluations. I can’t prove a causal connection but, as soon as student members of the programme committee began to explain how the evaluations worked, the number of ugly remarks dropped.’

Anonymous comments

At the Nijmegen School of Management, a ten-member committee of students, scientists and policy officers researched the evaluations. Professor Van Houtum was a member of this committee, as was lecturer Pieter Zwaan and student Jurian van der Waal. Van Houtum wants to make lecturers themselves responsible for the evaluations. For example, they could ask colleagues for feedback or start a discussion in the classroom.

He’s opposed to anonymous comments from students. ‘I think it’s great to talk with students, but then openly. If students want to say something anonymously, they can do so via the programme committees.’

The committee’s advice, which is confidential, has been sent to the faculty board. They haven’t yet made any decisions about the evaluation, but they don’t want to completely stop using the questionnaires. Policy officer Froukje van Zanten: ‘The board recognises the problems involved in student evaluations, but has no good alternative yet.’

‘The advice emphasises that the problem is not only about student evaluations but that it is part of many related factors, such as the quality of the teaching and the recognition and appreciation of teaching and lecturers. The faculty board is taking this message seriously and is going to contact other faculties to see how things could be done differently.’