Having the AI Detection Conversation with Your Students By Cheryl Colan Image from Teaching in Higher Ed Faculty often ask about what they should do when Turnitin's AI Detector reports a high percentage of AI-generated content in…

 By Cheryl Colan

Image from Teaching in Higher Ed

Faculty often ask about what they should do when Turnitin's AI Detector reports a high percentage of AI-generated content in student writing. It feels as though folks are seeking an easy-to-apply rule. Yet, we live in a time of an escalating "arms race" between generative AI tools designed to evade detection and generative AI detection tools. It's easy to over- or underestimate the number of students "cheating" on writing assignments. Faculty also differ in whether they allow generative AI use in their classes, so students struggle with knowing what is okay, and that standard may change from class to class.

In January, Bonni Stachowiak interviewed Christopher Ostro for the Teaching In Higher Ed podcast episode A Big Picture Look at AI Detection Tools. As part of the Learning Design Group at the University of Colorado, Boulder, Ostro conducted a literature review about AI detection, which he discusses with Stachowiak. If you are feeling uncertain as to the reliability of AI detection and are still grappling with your own approach in the classroom, I highly recommend listening to the podcast episode and checking out the linked resources. If you don't have 48 minutes to spare, here are my takeaways.

Students' Generative AI Use Is a Spectrum, Not a Dichotomy

Students may use generative AI to write an entire paper and copy/paste the whole thing into their submission without re-reading or fact-checking the output. That's clearly a maladaptive use of AI. But students may use it to brainstorm, or to get unstuck writing an introduction or a conclusion, and then edit the output in their own writing style and to better align with the ideas in their paper. And that might be a helpful use of generative AI. As a subject matter expert who understands and trusts that the struggle of working through the creation process can be difficult, but is invaluable to learning, you are best able to determine whether generative AI use is appropriate in conversation with your students. I'm talking about messy but authentic assessment, which takes care, time, effort, and communication designed around a set of values that are consistent with student learning and success.

We Must Adapt

Ostro talks about how people outside the classroom don't realize how much work has increased inside the classroom because of AI, and how people inside the classroom don't necessarily realize all of the complexities that come with suddenly enabling something like AI, and the need to adapt our processes as a result. He likens generative AI's impact on the writing process to the Internet's impact on the writing process. I learned to write research papers by collecting and organizing quotes and paraphrases from sources on index cards. When was the last time you did that in your writing process? (If you never have, we can be sure you're younger than me.) More than simply selecting a Generative AI statement from the Syllabus Template, different disciplines need to look at their learning outcomes and determine what makes sense regarding students' use of Generative AI. Consider wrestling this with your department colleagues if you haven't already.

Ostro claims one of the best things he's done in his classes is implement a policy and process for AI disclosure. You might include an AI disclosure students can use with discussion posts, video scripts, presentations, papers, or other writing assignments. Ostro allows students to copy/paste at the end of their papers and complete the disclosure so there is a single assignment submission. It would go something like: "On my honor as a Yavapai College student, I have used AI or AI tools in the following ways in this submission:" followed by a list of what tools were used and how they were used. Ostro asks students to include a few sample prompts and, if they used ChatGPT, a link to the chats. He also encourages students to provide a link to their Google Drive or Office 365 document because both keep a revision history that he can compare to the student's description of their AI use. He advises students to use these document formats because AI detection tools can still produce false positives.

A Process and Conversation with Students

Ostro's process is to begin by running assignments through an AI detection tool. He advises that if you use anything other than Turnitin, you remove any student-identifying data. Because you don't know what happens with that data, leaving it in could be a FERPA violation.

If a high AI detection score is shown, Ostro reaches out to the student for a conversation. He sends a friendly but vague email: "Hey, I was reading through your paper and I just have a few questions on how parts of it were written. Please send me the link to your Google/Word Document. Also, I'd love to schedule a short meeting." If teaching in person, he asks the student to stay for 5 minutes after class. If teaching online, he asks the student to meet during office hours or schedule a meeting.

Ostro admits online students often ghost his email and hope he forgets. He says he will not regrade their paper until they meet with him. He says he goes through this process with multiple students on the first writing assignment, but it's rarely necessary after that because students then understand what will happen. Unless the maladaptive AI use is a repeat offense, he gives the students a chance to redo the assignment. 

According to Ostro, these meetings are usually short. If the student used an AI disclosure, disciplining them for academic dishonesty is off the table, but if not, discipline is potentially in play. He starts by questioning and caring. He will ask students to explain how they put certain ideas together, why they phrased things a certain way, or to walk them through the parts of their paper that the AI detection has flagged. Note: you can save a PDF of Turnitin's AI detection report and share it with the student during this conversation.

Ostro contends an AI detection result should never be the sole reason a student is disciplined for academic dishonesty. Using AI detection in that way is effectively just as lazy as using generative AI to write a full paper instead of putting effort into the writing yourself. The intent is not to police students but rather to intervene when they show they are developing a maladapted process that won't serve them well and may even harm them in the future. The point is to help our students leverage their own skills, knowledge, expertise, background, and other uniquely human characteristics into checking and editing that AI output to create work of higher quality.