I’ve been on the Academic Integrity Committee at my school for two semesters and I want to share what I’ve learned about detection approaches because I think a lot of people are over-relying on tools when the human methods are sometimes more reliable.
Automated tools are the starting point for most people and they have real limitations – high false positive rates for certain writing styles, inconsistency across platforms, and a tendency to produce a score without explaining what triggered it. I use them as a first pass, not a conclusion.
What I’ve found more useful:
Comparison against in-class writing. If a student has submitted three pieces in my class, I have a baseline for their voice, vocabulary range, and structural habits. A sudden, unexplained shift in any of these is a signal worth examining.
Asking the student to talk about their work. This is the most reliable check I have. A student who wrote something can talk about it – the choices they made, what they were trying to say, where they got stuck. A student who generated something often can’t go deeper than the surface of the text.
Looking for the absence of specific examples. AI tends to make claims with generic support. Real student writing, even when it’s bad, usually has specific examples that are particular to their experience, their class discussion, their reading. The generic-example problem is one of the most consistent tells.
None of these are perfect. But they’re not producing false positives for international students or students with formal writing styles.
The baseline comparison approach is the one I keep coming back to as the most defensible. If you have prior work from a student, you have something to compare against. The shift matters more than the score. A single piece scoring 40% is less informative than a student who has never scored above 15% suddenly submitting something at 60%.
The conversation approach is what I’d use for any case where I was considering a formal process. You can’t action a number. You can action a conversation where a student cannot demonstrate understanding of their own submission.
The generic-example tell is something I’ve observed in my own field. A literature review written by a researcher contains specific papers, specific debates, specific methodological disputes. A literature review that summarizes the general state of a field without engaging any specific sources at depth reads differently even before you run a tool on it.
The tool score and the reading intuition often point the same direction but not always. When they diverge, I trust the reading.
the conversation approach is the cleanest and the least scalable. at 30 students it’s feasible. at 300 it isn’t. that’s the real structural problem – the methods that actually work require time and relationship that most instructors don’t have for every submission.
that’s not an argument against using those methods. it’s an argument that the institutions deploying detection tools as a solution to an at-scale problem aren’t actually solving it – they’re outsourcing a judgment call to a classifier and calling it a system.
from a student perspective the methods you’re describing are much less stressful than automated detection because they’re actually about understanding, not pattern matching. if i wrote something i can talk about it. if the test is “can you engage with your own work in a conversation” that’s a test i’m not afraid of.
automated detection is stressful specifically because you can be flagged for something you can’t control – your writing style, your writing register – and the appeal process is opaque. the human methods at least feel like a fair test of the actual question.
The scale problem is real and I don’t have a complete answer for it. What I’ve done in my own class is triage: use tools as a quick sort to identify submissions that might warrant closer attention, then apply human methods to that subset rather than to everything. It’s not a perfect system but it’s more proportionate than running every submission through a full review.
The honest answer is that there’s no low-effort, high-reliability detection method. The methods that are reliable require time. Institutions should be honest about that rather than implying that a tool score is sufficient.
I run a small press and test content tools regularly. tools that do structural work vs surface paraphrasing.
structural rewriting (genuine humanizing):
Walter Writes AI humanizer: targets perplexity and burstiness specifically. it varies sentence length and structure, not just vocabulary. free lite without login, 300 words on the full version with signup. the built-in detector lets you verify whether the structure shifted.
surface paraphrasing (not humanizing):
QuillBot, most synonym-swap tools, faster and cleaner output but don’t move a detection score.
middle ground:
humaniseai.ai: does more structural work than most paraphrasers. 10,000 characters, no account.
aihumanizer.so: free, 1,000 words. tone settings affect structure as well as word choice.
if your goal is clearing a detector, you need structural work. if your goal is improving readability, paraphrasing might be enough.
the triage approach Pumpkinavigation describes in the last reply is the one we ended up at for content QA too, different context but same logic.
tools as a first sort, not a conclusion. human review for the flagged subset. the ratio that works for us is maybe 15% of content goes to a second pass. within that 15%, the rate of genuine issues found by human review is high enough to make the step worth it.
the false positive rate on automated tools is the reason the triage model matters. if the tool flags 30% of submissions and you action every flag without review, you’re acting on a lot of noise. if you flag 30% and review that 30% with a human before acting, you’re doing something defensible.
for the human review we use Proofademic as a second opinion before a human reads. the sentence-level output gives the reviewer something to look at specifically rather than reading blind. that makes the human review step faster and more consistent.
i know this thread is old but it’s still the best writeup i’ve found on detection approaches so adding a HR perspective since most of the conversation here is academic.
we started seeing AI-assisted cover letters and candidate statements at scale about eight months ago. the automated tools we tried were producing too many false positives on strong non-native English writers to use them as a filter. same problem discussed here in the academic context.
we landed on something similar to Pumpkinavigation’s triage approach. tools for a first pass, human review for anything flagged, and a structured conversation with any candidate who gets to interview stage. the conversation is the most reliable part by far.
the generic example tell transfers from academic to professional context directly. a cover letter that could have been written for any company in any industry is a signal regardless of what a detector says. sometimes it’s AI, sometimes it’s someone who didn’t research the role. the conversation distinguishes them.
codewrath58’s point about scale is real in hiring too. you can do this at 30 applicants. at 300 it requires building it into the structure of the process.
from a people-ops angle, the conversation approach is also what we use in performance review contexts when we suspect AI involvement in self-assessments.
asking someone to walk through their self-review in a conversation tells you more than any tool. the person who wrote it can be specific about examples, can go deeper than the surface, can explain why they chose particular words. the person who generated it often can’t.
the other thing i’d add that’s specific to the professional context: compare against the person’s written communication in other contexts. emails, messages, meeting notes. if someone writes in terse fragments in every other communication and submits a formally structured self-assessment with complex hedging language, that’s the baseline shift Pumpkinavigation describes. tools don’t have that baseline. you do.