A recent study published on scientific journal, Patterns, has identified a bias in GPT detecting software that misclassifies the work of students who speak English as a second language as AI generated.
Whilst the software was near-perfect in detecting AI generated content in native writers, the research paper states that “over half of the non-native English writing samples were misclassified as AI generated.”
Go deeper with GlobalData
The software works by examining the “text perplexity” of a piece, by ranking how “surprised” or “confused” an LLM is by trying to calculate the next word in a sentence.
The easier it is for an LLM to guess the next word, the lower the text perplexity.
The researchers behind the study hypothesise that this GPT detecting software may therefore “inadvertently penalise” non-native English speakers whose vocabulary may be more limited.
What might this paper mean for the adoption of AI in education?
At the beginning of this month (4th July) The Russell Group, including Oxford and Cambridge Universities, published principles to encourage the intigration of AI into their learning.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataThis move has been criticised due to the potential for students to use AI to cheat in exams and essays.
When asked if this study might lead UK universities to become hesitant to adopt AI into their syllabus, GlobalData analyst Maya Sherman answered that it may instead “lead to the development of enhanced AI screening tools” that are better equipped to accommodate non-native writers.
With the global specialised AI applications market set to be worth over $146bn by 2030, according to research analyst GlobalData, additional scrutiny around the efficacy and diversity of such applications will become more critical.
The Patterns study has also been published after the first month that ChatGPT’s website traffic declined since its initial release back in November 2022.
Reuters reports that there has been speculation that the dip in traffic correlates with the school holidays, meaning less pupils are accessing the technology to help them with homework.
Speaking on these claims, Sherman is sceptical of the link.
Stating that ChatGPT’s traffic remains “remarkable”, Sherman argues that more “research and data should be analysed to assume such speculation.”
She also advised against “assuming the user curve is linear” and remains optimistic that ChatGPT’s traffic could increase again soon.
