AI Detection

This is an evolving space and we may yet see more shifts. At this time (Spring 2025), I still do not recommend AI detectors. Having said that, it seems worth the time to present a wide range of articles and sources related to this discussion.

AI detection tools/teacher skills – policy ideas

Promoting Academic Honesty in Your Courses. Temple University. “Here are some strategies for fostering academic integrity by tackling the four factors that Lang describes and employing practices that the research shows helps to promote academic integrity.”
Aug 16, 2023. Guidance on AI Detection and Why We’re Disabling Turnitin’s AI Detector. From Vanderbilt University. “After several months of using and testing this tool, meeting with Turnitin and other AI leaders, and talking to other universities who also have access, Vanderbilt has decided to disable Turnitin’s AI detection tool for the foreseeable future. This decision was not made lightly and was made in pursuit of the best interests of our students and faculty.”
Updated Aug 20, 2023. AI Writing Detectors Are Not Reliable and Often Generate Discriminatory False Positives. Teachers and schools are being tricked into wasting time and money on these tools that can be better invested in training faculty.
Updated Aug 31, 2023. How can educators respond to students presenting AI-generated content as their own? OpenAI. Big takeaway, AI detectors DO NOT WORK.
June 2024. Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays. Johanna Fleckenstein, Jennifer Meyer, Thorben Jansen, Stefan D. Keller, Olaf Köller, Jens Möller. “Here we show in two experimental studies that novice (N = 89) and experienced teachers (N = 200) could not identify texts generated by ChatGPT among student-written texts. However, there are some indications that more experienced teachers made more differentiated and more accurate judgments. Furthermore, both groups were overconfident in their judgments. Effects of real and assumed source on quality assessment were heterogeneous. Our findings demonstrate that with relatively little prompting, current AI can generate texts that are not detectable for teachers, which poses a challenge to schools and universities in grading student essays.”
Sept 18, 2024. Black teenagers twice as likely to be falsely accused of using AI tools in homework. Mizy Clifton. “Racial biases have been known to creep into artificial intelligence algorithms. Now teachers are bringing it into the classroom as they police students’ use of generative AI tools like ChatGPT to complete homework, according to a new study by children’s safety nonprofit Common Sense Media.”
September 2024. Phillip Dawson LinkedIn post on the “Swiss Cheese” method of AI detection and grading. “All approaches to addressing addressing a problem like cheating or inappropriate AI use have their holes, but many layers stacked (like layers of Swiss cheese) work better than any layer alone.”
Jan 30, 2025. A Big Picture Look at AI Detection Tools, with Christopher Ostro. Teaching in Higher Ed podcast.

AI detectors and learning to spot AI

RAID Benchmark Leaderboard. “These leaderboards contain the test-set scores of various detector models. To submit your own model’s predictions to the leaderboards, see Leaderboard Evaluation.“
Desklib. “Instantly identifying AI-generated text to ensure originality, credibility and plagiarism-free writing for students and professionals”
The Effects of Training on Teacher Ability to Assess Papers in the 21 st Century: Can We Learn to Detect AI-written Content Like ChatGPT? This study measured the effect of training on K-12 English and Social Studies teachers’ abilities to detect AI-written responses. In this study, teachers were given a pre-assessment containing human-written and AI-written samples and asked to identify which were plagiarized….The study also found the training resulted in a 10% increase in Google Translate plagiarism detection, a 2.66% decrease in false plagiarism accusations, and 16.66% decrease of false plagiarism accusations of human-written papers containing properly cited MLA format ChatGPT quotes (p>.05).
December 2024. 19 AI Detection: A Literature Review. Starting at 2:30 with Chris Ostro. Slides found here.
Jan 2025. LinkedIn post from Ethan Mollick. “Reminder for the new semester: you can’t detect AI writing.”
Jan 9, 2025. 13 Ways to Detect AI Writing Without Technology. Erik Ofgang. “As more of my students have submitted AI-generated work, I’ve gotten better at recognizing it.”
Jan 26, 2025. People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text. Jenna Russell, Marzena Karpinska, Mohit Iyyer. “In this paper, we study how well humans can detect text generated by commercial LLMs (GPT-4o, Claude, o1).”

On the potential use of AI detection tools

March 12, 2025. “Scarlet Cloak and the Forest Adventure”: a preliminary study of the impact of AI on commonly used writing tools. Barbara Bordalejo, Davide Pafumi, Frank Onuh, A. K. M. Iftekhar Khalid, Morgan Slayde Pearce & Daniel Paul O’Donnell. “Initially prompted by noticing how tools like Grammarly were being flagged by AI detection software, it examines how these popular tools such as Grammarly, EditPad, Writefull, and AI models such as ChatGPT and Microsoft Bing Copilot affect human-generated texts and how accurately current AI-detection systems, including Turnitin and GPTZero, can assess texts for use of these tools.”
March 2025. Why Universities are ABANDONING AI Detection. Tadhg Blommerde. “new research reveals why Cambridge, Durham, and other leading universities are dropping AI detection software”
Feb 20, 2025. Progress Towards Robust And Deployable AI Detectors In The Real World. Sergey Sanovich. “The seventh session of Challenges and Safeguards against AI-Generated Disinformation will discuss Progress Towards Robust and Deployable AI Detectors in the Real World with Liam Dugan and Sergey Sanovich”
Feb 16, 2025. AI detectors: A look at some arguments and where I ended up. Laura Dumin. “This piece looks at some of the arguments around AI detection tools and my deeper dive into what that means for me going forward.”
Feb 12, 2025. Why I’m using AI detection after all, alongside many other strategies. Anna Mills. “I argued against use of AI detection in college classrooms for two years, but my perspective has shifted. I ran into the limits of my current approaches last semester”
Feb 1, 2025. Try Mozilla’s New AI Detector Add-On for Firefox. Joey Sneddon. “Want to find out if the text you’re reading online was written by an real human or spat out by a large language model (LLM) trying to sound like one?”
Jan 2025. GenAI Content Detection Task 3: Cross-Domain Machine-Generated Text Detection Challenge. Liam Dugan, Andrew Zhu, Firoj Alam, Preslav Nakov, Marianna Apidianaki, Chris Callison-Burch. “We find that multiple participants were able to obtain accuracies of over 99% on machine-generated text from RAID while maintaining a 5% False Positive Rate — suggesting that detectors are able to robustly detect text from many domains and models simultaneously. We discuss potential interpretations of this result and provide directions for future research.”
Jan 2025. Beyond AI Detection: How Universities Are Redesigning Assessments That Work. Tadhg Blommerde.
Can AI be trained to detect plagiarism? The Learning Agency Lab. “A new dataset can now help learning scientists train AI to automatically detect and expose AI-generated essays and content.”
Dec 2024. AI detectors in universities: Time to turn them off and embrace AI for enhanced learning. Tadhg Blommerde, William Bright, Ellie Musgrave, Rosie Mitchell, Rebecca Heselton. “Though this article may be difficult reading for academic staff who unquestioningly accept AI detection tools (Roe et al., 2024), it recommends that universities worldwide discontinue their use.”

AI Detection

AI detection tools/teacher skills – policy ideas

AI detectors and learning to spot AI

On the potential use of AI detection tools

Other relevant sources from Tadhg Blommerde video.