Since the launch of ChatGPT in November 2022, the chatbot’s developer OpenAI as well as ed-tech developers like Turnitin have launched new tools designed to catch instances of AI text plagiarism and address growing concerns among educators who say students have used AI text programs to cheat on writing prompts. Among other new AI text detection tools is a self-titled one from AI Detector Pro, described in a news release as being able to identify articles or papers written by most AI writing programs and gauge the probability that content was written by AI.
According to AI Detector Pro’s CEO and chief technologist Raj Dandage, the new SaaS tool works best on text created with GPT-3, which is used in most AI tools such as Jasper, as well as on ChatGPT and “rephrasers” like QuillBot. Similar to other AI text detection programs, he said the tool’s neural network was trained on hundreds of thousands of documents generated by GPT.
“Although to most readers GPT-generated text looks just like normal text, it does have a specific writing style. AI Detector Pro has learned that style and is able to detect the patterns that GPT exhibits,” Dandage wrote in an email to Government Technology. “Currently, AI Detector Pro is quite accurate at identifying AI-generated text. We continually train our systems with the latest GPT outputs.”
According to a recent study from the online magazine Intelligent, nearly a third of college students have used ChatGPT to complete written assignments, with 60 percent saying they use the program on more than half of their assignments. The study said about 75 percent of students believe that using the program for those purposes is cheating but do it anyway.
As professors continue noticing instances of cheating via the use of programs like ChatGPT, the problem has also raised alarms in K-12 districts, leading some of them in New York, Los Angeles and elsewhere to ban AI chatbot programs on district networks and devices. Dandage said “there’s no question” students are already using AI programs to complete assignments, adding that K-12 and higher ed students are among the earliest adopters of emerging technologies.
“Most people don’t realize this, but a shockingly large percentage of text written today is generated by AI,” he wrote in an email. “It’s up to educators to be aware of this trend and check assignments with a tool like AI Detector Pro.”
Due to concerns about how these programs can be used, teachers in Canada and the U.S. have already started making use of online tools like www.chatgptplagiarism.com, an AI detection resource created by 22-year-old Toronto tech developer Tomer Tarsky. He said the tool is mostly being used by K-12 educators, though he plans to pitch it to higher ed staff moving forward.
Tarsky said he believes his tool can compare students’ work with AI-generated text more accurately than other preliminary and beta-version AI text-detection programs now flooding the ed-tech market, adding that the challenge is checking for plagiarism without relying on existing data similar to other anti-plagiarism tools.
“If the teacher assigns students to write an essay about World War II, their students will write that essay, and all the teacher has to do is enter the topic in the teacher’s dashboard. My software on the back end will write 1,000 ChatGPT-generated essays on World War II and compare each student’s submission to the generated essay that ChatGPT wrote and does a standard plagiarism check on them for similarity, which gives the plagiarism rating to the teacher,” he explained. “It’s been really helpful for teachers.”
One of the earliest tools to detect ChatGPT content was GPTZero, created by Princeton student Edward Tian, also from Toronto. The tool has so far been used by over 6,000 educators from Harvard, Yale, the University of Rhode Island and elsewhere, according to a report in January from the New York Times. In addition, education technology nonprofits Quill.org and CommonLit also recently announced the launch of AIWritingCheck.org, a resource with similar functions to flag AI-generated text, as well as a toolkit for educators to learn how to use AI detection websites effectively.
According to Peter Gault, founder and executive director of Quill.org, the AI detection technology was trained on a set of 500,000 essays — 250,000 by humans and 250,000 from AI.
“The AI predicts whether a new piece of writing is more similar to AI writing or human writing,” he wrote in an email to Government Technology. “The difference is that AI writing is more formulaic. Our accuracy rate is 80 percent to 90 percent.”
Cody Tucker, a teacher from Sumner County Schools in Tennessee and Quill user, said in an email that catching AI-generated text can prove difficult without plagiarism tools designed to flag AI text.
“I have basically been having an existential crisis as an English teacher ever since I discovered [ChatGPT]. I am concerned that ChatGPT erases any need for writing instruction past the fourth or fifth grade, unless a student is explicitly going into the field of professional writing. The reason for this is that all of the typical, non-academic reasons which have historically justified teaching writing — you’ll need to be able to at least write an email, a cover letter, a job application, etc. — are things that ChatGPT can do in about seven seconds,” Tucker wrote.
According to Tarsky, educators need to embrace the fact that AI tools are not only not going away, but will improve as the field of artificial intelligence continues to advance. He said it’s important for educators to stay informed about the tech arms race between AI text programs and tools designed to identify their work, as well as potential use cases for tools like ChatGPT.
“Teachers can police what the kids are doing in schools, but when they get home, they have free reign to do what they want. I think [ChatGPT] is a great educational tool. … It’s fascinating how great it is at giving information, but I think it’s kind of a slippery slope,” he said. “I wouldn’t want to live in a society where there’s doctors, lawyers and important professionals who got through school using ChatGPT over actually learning the material.”
