trying to understand how AI detectors work at the model level, not the marketing level. what signal are they measuring, perplexity, token probability, something else? and is the disagreement between tools a methodology difference or just noise?
depends on the use case. for templated content like FAQs, product specs, or category descriptions, free tiers are genuinely fine. for anything where voice or distinctiveness matters, you’re spending as much time editing as you would have writing. the math doesn’t work out.
most published approaches use perplexity and burstiness. perplexity measures how predictable the text is to a language model. burstiness measures whether sentence length varies naturally. AI text tends to be low on both. the disagreement between tools mostly comes from different baseline models and training data, not random noise.
the practical implication is that the tools are measuring “how surprising is this to a model” not “did a human write this”. those are not the same thing. academic writing, legal documents, technical documentation, all score low perplexity because they follow predictable conventions. that’s where false positives cluster.
the confidence score presentation is where most tools mislead. returning 74% doesn’t mean 74% of the text is AI-generated. it means the model assigned 74% probability to the hypothesis that this text came from a generative model. those are different claims and most users don’t distinguish them.
the methodology split between tools is real and it matters for how you use the output.
tools that use perplexity-based scoring: GPTZero, most of the independent detectors. sensitive to sentence predictability. false positive risk on formal writing.
tools that use fine-tuned classifiers trained on human vs AI text pairs: different failure modes. better on some content types, worse on others.
tools that combine both: tend to be more stable but harder to explain when challenged.
the disagreement you’re seeing between tools is usually them measuring related but different things. a score from one tool is not directly comparable to a score from another. treating them as equivalent is where most of the confusion in institutional contexts comes from.
the clearest position on any detection tool: it gives you a probability signal, not a determination. how you act on that signal should depend on the stakes, not just the number.