how AI detectors work is something I get asked about constantly by students and I realise I don’t have a great answer. I can explain that they look for patterns but I can’t explain what patterns specifically or why the same text scores differently on different tools. anyone who knows the actual mechanics want to help me explain this better?
the practical implication of the perplexity approach is that it’s measuring ‘how surprising is this text to a model’ rather than ‘was this written by a human.’ those aren’t the same thing. highly formulaic human writing, literature reviews, legal documents, technical summaries, scores low perplexity too. that’s where most false positives come from. the tools are doing what they were designed to do, the design just doesn’t map cleanly onto what people want to know.
the core signal most tools use is perplexity. a language model assigns probabilities to each word given the words before it. human writing is unpredictable in ways that AI writing isn’t. AI text tends to choose high-probability words consistently. perplexity measures how often the text surprises the model.
burstiness is the other main signal. human writers vary sentence length naturally. a short sentence. then a longer one that develops the thought further before coming back around. AI text tends to produce sentences of more consistent length. that consistency is measurable.
the different scores across tools come from different underlying models. GPTZero is measuring against its own language model. another tool is measuring against a different one. whether text is “surprising” depends on what you’re comparing it against. that’s why the same text can score 40% on one tool and 70% on another.
for a classroom explanation, I’d frame it this way.
AI text is predictable. not because it’s bad, but because it’s been trained to produce likely, coherent continuations. it chooses words that fit. it writes sentences that follow each other logically. that predictability is its strength as a tool and the signal that detectors use against it.
human writing is less predictable. we start sentences we don’t finish the way we planned. we choose unusual words. we write a very short sentence. then a long winding one that goes somewhere unexpected. that variation is hard to fake consistently.
detectors measure that variation (burstiness) and that predictability (perplexity). tools disagree because they’re each measuring against their own model of what “predictable” means. none of them are measuring the same thing in exactly the same way.
the practical lesson: a detection score is a probability estimate about writing patterns, not a determination of who wrote something.
the most useful framing for students: the detector is asking “does this text behave the way AI text behaves?” not “did an AI write this?” those are related questions but not identical ones.