Noticed a shift in how clients and colleagues are approaching the is-this-AI-generated question over the last six months. Earlier, people were mostly asking me to confirm their suspicion about a specific piece. Now I’m getting more nuanced versions of the question.
Things I’m hearing more: ‘Is this AI-generated or just heavily AI-assisted?’ ‘Can you tell if this was originally AI and then edited, or written with AI suggestions throughout?’ ‘Does it matter if the ideas were human but the execution was AI?’
These are better questions but current tools can’t really answer them. Detection tools produce a binary or a probability, not an account of the process. The granularity people want isn’t available.
The fact that the questions have gotten more sophisticated while the tools haven’t kept up is interesting. It suggests people have internalized that the simple question doesn’t capture what they actually care about. What are others seeing in how this question is being asked?
In academia the process questions are starting to appear in integrity policy language. Some institutions are now asking not just ‘is this AI’ but ‘what was the student’s role in producing this.’ That’s a more useful framing and it shifts toward assessment design rather than detection technology.
The process-versus-product distinction you’re identifying is the frontier question in this space. Current tools can only assess the product. The process that produced it, including how much human judgment shaped it at each stage, is not recoverable from the text alone. That’s a fundamental limitation, not a calibration problem.
In publishing the question has evolved similarly. We’re no longer asking ‘did AI touch this’ because the answer is almost always yes in some form. We’re asking ‘what did the author actually contribute and is that contribution substantial.’ Detection tools aren’t equipped to answer that question.
I work with writers who use AI at very different points in their process. Some use it for a first draft and write everything else. Some use it only for feedback. Some use it to generate options and choose among them. The output from all of these processes can look similar. That’s exactly the problem with asking the text to tell you the process.
The ‘does it matter if ideas were human but execution was AI’ question is the one I think about most for my own content. My answer is that it depends on what was being evaluated. If the context is assessing ideas, the execution method shouldn’t matter. If it’s assessing communication skill, it does. The context defines what counts.