agentic AI is in production now. the announcements have been arriving for a year. I want to know what actual production deployments look like, not what the pitch says. what are people running in production and what are the real-world results?
the production deployments I’ve seen that work reliably: narrow, well-defined tasks with machine-readable outputs and automatic verification. code testing pipelines, structured data extraction, scheduled report generation. the common thread is that you can tell when it got it wrong without human review.
in marketing: we’re running an agent that monitors competitor pricing changes and generates a summary report on a daily schedule. it works reliably because the task is structured, the data sources are consistent, and the output format is fixed. anything requiring judgment has a human in the loop.
the gap between demo and production is larger for agentic AI than for any other category. demos show the best-case path. production means edge cases, API failures, ambiguous inputs, and compounding errors across steps. the demos almost never show what happens when step 2 fails.
the framing that’s helped us: agents are automated, not autonomous. automation follows a defined path and handles known cases. autonomy handles novel situations. current agents do the former well and fail unpredictably at the latter. deploying under the ‘automation’ frame means designing for failure modes explicitly.
from conversations and reading: the pattern in working production deployments.
works reliably: research synthesis from specified sources, structured data extraction from consistent formats, scheduled content generation for fixed templates, code quality checks on defined rule sets.
works unreliably: multi-step tasks where an early error compounds, anything requiring judgment on ambiguous inputs, tasks where the success criteria can’t be verified automatically.
the verification question is the gate: before deploying any agent, the question to answer is ‘how will we know when it got it wrong?’ if the answer is ‘we’ll check everything anyway,’ you haven’t automated anything. if the answer is ‘the output is machine-verifiable,’ you have a real production deployment.