i want to be clear about what this is and isn’t. this is not a thread about gaming a system. it’s a record of what editing changes actually affected detection scores on my own content, with the goal of understanding what the tools are measuring.
i write B2B content. i use AI for drafts. my clients’ internal review processes sometimes include running content through detection tools, so i have a practical need to understand this.
i ran the same 300-word draft through a detection tool before and after each of the following single edits, resetting to the original between each test:
- Broke 4 compound sentences into 6 shorter ones: score dropped 18 points
- Added 2 contractions where none existed: score dropped 4 points
- Replaced 3 generic transition phrases (“furthermore,” “in addition”) with no transition or a shorter one: score dropped 11 points
- Added one specific example with a real number: score dropped 9 points
- Cut the opening meta-sentence (“In this piece, I will…”): score dropped 7 points
- Added one intentional fragment for emphasis: score dropped 6 points
Combined edit applying all of the above: score dropped from 71% to 22%.
the most impactful single change was sentence fragmentation. the least impactful was contractions. the specific example was more impactful than i expected.
none of this tells you anything definitive about all tools or all content types. it’s one test on one tool on one piece. but it suggests the tools are heavily weighted toward sentence length variance and transition pattern recognition.
This is the most systematic breakdown of this I have seen shared publicly. The sentence fragmentation result makes sense – sentence length variance is one of the cleaner signals these tools use and it’s relatively easy to measure. The specific example result is interesting because it suggests the tools are picking up on something semantic, not just syntactic.
The combined edit result is the important one. A 49-point drop from editing changes that also improve the writing quality is not a hack. It’s a demonstration that good editing and lower scores move in the same direction.
The transition phrase result stands out to me. “Furthermore” and “in addition” are such strong AI tells at this point that removing them is almost table stakes for anyone doing this seriously. The question is what you replace them with – nothing, a shorter connector, a structural break, the next thought starting where the last one ended. Each of those choices affects both the score and the quality differently.
the contractions result being the weakest is interesting because that’s often the first thing people recommend for humanizing AI writing. your data suggests it matters much less than sentence structure changes.
to be fair, contractions might matter differently at different content types or for different tools. but the hierarchy you found – structure first, then specificity, then surface-level word choices – seems intuitively right to me.
the methodological approach here is actually clean – isolating single variables rather than making multiple changes at once. that’s how you’d design this as an experiment and it’s more rigorous than most of the “here’s what I did” posts on this topic.
the obvious caveat is single tool, single piece, single pass. would be interesting to replicate across different tools and content types. the sentence fragmentation finding might not hold the same way for technical or academic writing where fragments are stylistically unusual.
good caveat on the single-tool problem. i’d be curious whether the hierarchy holds across tools or whether different tools are weighted toward different signals. the tools i’ve tested informally do seem to vary on which edits move them most.
anyway. do with this what you will. the main takeaway i’d stand behind: the edits that moved the score most were also the edits that made the writing better. if those two goals are mostly aligned, the framing of “detection vs. quality” might be the wrong frame.
from a detection standpoint, the difference matters a lot. detectors aren’t looking at your word choices, they’re looking at your sentence rhythm and predictability patterns. swapping synonyms while keeping the same structure does almost nothing to a perplexity score.
this thread is months old but i’m sharing it with every writer i coach so figured i’d add something.
MrDoubtfire’s finding on sentence fragmentation being the biggest single mover matches what i’ve seen working with writers. burstiness, the variation in sentence length, is the signal detectors weight heavily because it’s one of the cleanest measurable differences between AI and human output.
the specific example result is the one i find most interesting. a 9-point drop from adding one concrete detail suggests the tools are doing more semantic analysis than most people assume. it’s not purely syntactic.
the edits that move the score most are also the edits a good line editor would make anyway. shorter sentences. cut the meta-sentence. add specificity. AI writing is flat because it’s calibrated for plausibility, not for communicating something particular. editing that makes it more human is editing that makes it say something specific.
one practical note: Walter Writes has a built-in detector that’s useful for running before/after tests without switching tabs. free, no login for the lite version.
i know this is an old thread but it’s still the most rigorous public test of this i’ve found so adding a publishing perspective.
Appartement’s caveat about single tool, single piece is the important one. the hierarchy MrDoubtfire found, structure first, then specificity, then surface choices, likely holds directionally across tools but the specific point values will differ. what drops a score 18 points on one tool might drop it 8 on another.
for publishing workflows where i need to verify before sending to press, i run two tools now. if both tools move in the same direction after an edit, i trust the edit. if they diverge, i look at the edit more carefully. the disagreement between tools is itself information.
the methodology point Appartement made is worth developing. the controlled variable approach, one edit at a time with a reset between each, is how you’d design this as an actual experiment. most “here’s what worked for me” posts don’t isolate variables and the results are correspondingly harder to interpret.
the academic writing caveat is the one i’d add. the sentence fragmentation finding might not hold the same way for technical or formal academic writing where fragments are stylistically unusual. a fragment in a literature review reads as an error, not an authorial choice. the edits that work for B2B content don’t map directly onto academic prose.
for academic writing specifically, varying sentence length within the formal register, without fragmenting, is the structural change that moves scores without creating style problems. that’s a harder edit to make but the right one for that context.