Does ChatGPT herald a new breakthrough in higher education, or should the chatbot be ignored as much as possible? Three scientists at Radboud University share their view on the AI-powered software. ‘We can’t ignore ChatGPT; we have to engage with it critically.’

Disclaimer: This article was not written using ChatGPT. It could have been, however, because the chatbot contains millions of texts and is trained in joining them together. That is how the program is able to answer simple questions and requests like ‘write a text about the opportunities and threats of ChatGPT in higher education’ or ‘write a love song in the style of André Hazes.’ The program generates new texts that strongly resemble human-made works.

The chatbot hasn’t left the news cycle since the launch of its latest version, GPT-3.5, at the end of November. Inge Molenaar, director of the National Research Lab AI, as well as associate professor at the Behavioural Science Institute, understands why that is the case. ‘Every once in a while someone claims that AI is going to change the world, but for a lot of people those are distant concerns, until now. ChatGPT is one of the first examples of AI doing something that only humans could do before. For a lot of people that’s a shock.’

Philosopher and cognitive scientist Anco Peeters had to mull over the question whether the chatbot is a hype. ‘Hypes come and go’, he says. ‘I don’t think this is going to go.’ And yet, according to Peeters, the chatbot’s functions are not very innovative. ‘Computing Science students have been using this kind of software for some time already. But the new interface, especially the chat part, has suddenly made it accessible to people with little technological know-how.’

Use in lectures

The consequences of the new chatbot for higher education could be huge. Students have admitted they have used it to do homework without getting caught, leaving teachers worried. Eight prestigious universities in Australia have returned to using pen-and-paper exams due to the new software. Even Nijmegen students have been using the chatbot.

All of this raises the question of what university lecturers should do about it. According to Molenaar, tools like ChatGPT are a part of reality that cannot be ignored – not even in the lecture halls. ‘When pocket calculators entered the scene, people argued about it for years, but now those have become a regular part of our lives’, she says.

‘When pocket calculators entered the scene, people argued about it for years’

Molenaar sees a lot of possibilities for the chatbot, especially when it comes to education on writing. ‘You could ask the software to write an argumentative essay about AI in higher education and what structure ChatGPT uses for such essays. Then you could ask the chatbot to centralise a particular thought in the essay. This way you can learn how to write an argumentative essay in an interactive manner.’

Peeters, who also teaches Philosophy of Science to AI students, agrees that the genie has left the bottle. ‘Students will be using ChatGPT regardless’, he claims. He is not yet sure how to integrate the software into his lessons. ‘I want to teach students about the system, but I also want to teach them to think critically. Ignoring it is not an option, but I also don’t want to give students too much freedom.’

Molenaar is a little bit less cautious: she suggests you could ask students to have ChatGPT write the intro to a thesis. ‘It won’t result in a good intro, but it will produce a good first draft that can then be improved upon’, she says. Molenaar thinks that students may write theses in an entirely different way in the future because of the rise of new software. ‘I don’t mean that theses will be written by ChatGPT, but it could become a convenient aid in the endeavour.’

Parrot

However, Molenaar does state that the chatbot currently has too many issues to really cause a revolution in higher education. For example, ChatGPT can’t cite sources. Additionally, the quality of produced texts shows room for improvement. ‘They often sound plausible, but they’re not always correct’, she says.

‘ChatGPT is like that uncle with a lot of tall stories at birthdays’

The curious paradox, Peeters agrees, is that you almost have to be an expert in whatever subject you’re using ChatGPT for, before you can really estimate its usefulness. ‘I like to compare it to that uncle who, at every birthday party, can talk at length about subjects they are completely ignorant of. That is why Emily Bender (American linguist, ed.) and Iris van Rooij (professor of AI at Radboud University, ed.) refer to it as a parrot: using probability, ChatGPT combines things it has heard elsewhere and then presents a response it predicts you will like. It doesn’t care if its answer is factually incorrect.’

Plagiarism or no?

Schools and universities around the world are struggling with the fact that ChatGPT’s texts are unique, which means they mostly circumvent plagiarism detection – although one Princeton student has developed a program that can predict how likely it is a that a given text was written by the AI software.

Even so, according to Peeters, in the future it will be harder, if not impossible, to check if students used AI in their work. ‘And that does not necessarily amount to plagiarism. Perhaps students just used ChatGPT to brainstorm, rather than their peers. There’s no accounting for that.’ Peeters opines that, in the future, honesty between students and teachers will be even more important than it is today.

According to Jan Bransen, professor of Philosophy and academic leader for the Teaching and Learning Centre, students and teachers should avoid an arms race in which new software is followed by new plagiarism detection tools. ‘How does it happen that someone who comes to university to develop their critical thinking redefines themselves as someone who prospers by cheating using ChatGPT?’, Bransen wonders.

Despite this, Bransen does want to think about the potential applications for the chatbot in tests. ‘You could give students the exam questions and have ChatGPT answer them. In an oral exam you can then ask questions about the quality of the ChatGPT answers.’

Paywall

Even if you ignore the limitations of higher education, there are a lot of critical questions to be asked of the chatbot, according to scientists. Molenaar points out that OpenAI, the company that owns ChatGPT, is a commercial business and not an open source platform, as many people assume. ‘In the long term, they will probably start charging for the use of their chatbot’, she says. ‘Perhaps they will make a basic module that’s free to use and have a separate premium option that charges money.’

Peeters also thinks it’s likely that the chatbot will soon disappear behind a paywall. ‘As soon as a company has a large amount of users, they will explore monetization options’, he says. ‘The Netherlands is a rich country, so our institutions can likely afford such fees, but other countries may not. That would increase inequality.’

‘Every prejudice you see on the internet, is returned in the output of the program’

Another important thing to note: ChatGPT’s complete dataset comes from the internet. ‘That means that every prejudice you see on the internet, is returned in the output of the program’, according to Peeters. ‘If you ask it to describe a doctor, that will probably result in a middle-aged white male. If you ask for typical household chores you might see more examples of women.’

One final criticism: the software’s ecological footprint. ‘The American linguist Emily Bender wrote a paper on the amount of energy that’s spent on ChatGPT’, Peeters says. ‘These digital tools use more and more computing power and data, and that data needs to be stored somewhere. If more people start using this software, it will need more data centres. How will we justify that in an age where sustainability is a growing issue?’