The agent control loop is where production agents fail

Prompt engineering was never the hard part. You can rewrite a system prompt forty times before lunch and keep the best one. The agent control loop is not like that. It decides what the model perceives each turn, which actions it may take, how results re-enter context, what makes it stop and what a single step costs you. Almost every production agent failure I have taken apart was a loop failure. The weights were fine. The loop had no brakes.

Gartner's forecast from 25 June 2025 put it bluntly: more than 40% of agentic AI projects will be cancelled by the end of 2027, driven by escalating costs, unclear business value and inadequate risk controls. Read that list again. Model capability is not on it.

So call the discipline loop engineering. Perceive, decide, act, observe. Then two gates that most teams leave implicit: a budget check and a stop condition. Everything below is about making those two gates explicit and owned by code you wrote.

Termination conditions the agent control loop can actually check

"Until the task is done" is not a termination condition. It is a wish. It delegates the stop decision to the same component whose judgement is under question, and it produces the expensive failures: an agent that decides it is finished when it is not, or one that never decides at all.

Frameworks already concede this. The OpenAI Agents SDK ships with DEFAULT_MAX_TURNS set to 10 in run_config.py and raises MaxTurnsExceeded when you pass it. Ten. That default is not a recommendation for your workload, it is an admission that an uncapped loop is a defect.

A loop needs four independent stop classes, and at least three of them must be checkable without asking the model. First, a success predicate that something other than the agent evaluates. The file compiles. The row exists with the expected shape. The invoice total matches the ledger. If your only completion signal is the model announcing that it is done, you have a loop with no exit.

Second, budget exhaustion. Tokens, currency, wall clock and count of side-effecting calls, tracked separately because they run out at different times. Third, a no-progress detector: hash the tuple of tool name plus normalised arguments every turn and keep a sliding window. Three identical hashes inside eight turns means the agent is stuck, whatever the transcript sounds like. That one check kills a whole class of runaway spend. Fourth, escalation, where the loop stops and hands the decision to a person with the state attached.

"Until the task is done" delegates the stop condition to the same component whose judgement is under question.

What one step costs, in tokens and in money

Budget belongs in loop state, not in a dashboard you read the next morning.

Three numbers, decremented every turn and visible to the agent inside its own context: tokens remaining, money remaining, turns remaining. Telling a model it has four turns and 30,000 tokens left changes what it does with them. It compresses. It stops opening new lines of investigation. This is a twenty-line change and it works.

The public leaderboards make the cost argument for you if you read the model names carefully. On Artificial Analysis's Terminal-Bench v2.1 board in August 2026 the top configurations were GPT-5.6 Sol at xhigh effort with 89.5%, Claude Opus 5 in adaptive reasoning at max effort with 89.1% and Grok 4.6 at high with 88.4%. The suffixes are the story. Xhigh. Max effort. High. Leaderboard-topping agent performance is bought with reasoning tokens, which is precisely why that board publishes cost per task in the column next to the score.

Set ceilings in three places, because they fail differently: per run, per user per day and per tool. A search call that costs a fraction of a cent needs a different cap from one that provisions cloud resources or issues a refund. Then alarm on the ratio of cost per successful run to the value of the work done. An agent that spends more closing a ticket than a person would is not a productivity story, it is a subsidy.

Context is the loop's memory, and naive truncation lies

The loop's memory is the context window, and the default eviction policy in most homemade agents is to drop the oldest messages. The oldest messages are usually the task definition and the constraints. You are trimming the brief and keeping the noise, then wondering why turn nineteen ignores a rule stated at turn one.

Chroma's Context Rot report from July 2025, run across 18 models including GPT-4.1, Claude 4, Gemini 2.5 and Qwen3, found that performance degrades as input grows even on simple retrieval, and that the degradation is uneven. Needle-question similarity, distractors and haystack structure all bend the curve differently. Long context is not uniform quality that ends cleanly at the token limit. It thins out well before it.

Worse, the loop poisons itself. The 2025 paper The Illusion of Diminishing Returns found that models become more likely to make mistakes when their own earlier mistakes are sitting in context, an effect the authors name self-conditioning, and it does not disappear by scaling model size. The same work found reasoning matters enormously for execution length: without chain of thought DeepSeek-V3 failed to execute even two steps of a simple task, while its thinking version R1 managed 200 and GPT-5 with thinking exceeded 1,000.

So compaction, not truncation. Anthropic's published guidance is to tune the compaction prompt against real agent traces by maximising recall first, confirming nothing relevant is lost, then tightening precision to strip the filler. Keep the goal and the constraints verbatim. Keep the last few observations verbatim. Push the rest into a structured note that lives outside the window and gets re-injected on demand. A failed action deserves one line: what was attempted and why it failed. Not the stack trace, and not the model's three paragraphs of apology about it.

Retry-safe tools and the double-debit problem

Anyone who has used a Nigerian bank app knows this failure without needing the theory. You send a transfer, the app times out, you send it again and you are debited twice. Then you spend a week and several phone calls getting one leg reversed. That is a retry against a non-idempotent write, and it is the most common way an agent converts a transient network problem into a customer-facing incident.

Agents retry constantly. They retry because a tool timed out, because the orchestrator crashed and resumed from a checkpoint, because the model never saw the result and tried again. Every side-effecting tool therefore needs a deterministic idempotency key computed before execution from values that do not change on retry: run id, step index and action type. The server deduplicates on that key, and its response states plainly whether the action was newly executed or replayed. That last field matters, because an agent that cannot distinguish "sent" from "already sent" will reason itself into sending again.

Transport will not save you. The current MCP revision, 2026-07-28, removed the initialize handshake and the Mcp-Session-Id header, making the protocol stateless by default. A stateless transport is easier to retry. It says nothing whatsoever about whether your tool is safe to retry.

Temporal's model is the disciplined version of the same idea. Workflow code must be deterministic, because recovery works by replaying history, so every non-deterministic operation, every model call, timestamp, random draw and external request, moves into an activity, and the activity handler must be idempotent because it may run more than once. Writing against that constraint is irritating. It is also correct.

In Shenzhen the hardware people test this without ceremony. I have watched an integrator commission a motion controller by yanking the Ethernet cable mid-write, plugging it back in and checking whether the axis moved twice. That is the entire test. Teams building agents that touch money, inventory or physical equipment should be at least that rude to their own tools before a customer is.

Errors are inputs to the loop, not exits from it

An error surface is a prompt. Design it like one.

HTTP 500 plus a stack trace teaches an agent nothing it can act on. What it needs is three fields: what failed, why and what is available to do next. Something closer to: write rejected, field customer_id must be a UUID, received "cust-8812", call lookup_customer to resolve the external id. That error is a move. The agent takes it and recovers on the next turn. A stack trace is a wall, and against a wall the agent does the worst available thing, which is call the identical tool with identical arguments and burn a turn.

Make that impossible in the harness rather than hoping the model behaves. If the next turn proposes the same call that just failed with the same arguments, refuse it in code and return a message saying exactly that. Force a different action or force a stop.

Then decide, in code, which errors are terminal. The OpenAI Agents SDK exposes error_handlers keyed by kind, including max_turns, model_refusal and invalid_final_output, so a run can end with a controlled final output instead of an exception surfacing at your API boundary. Guardrails follow the same shape through input and output tripwires. The line you are drawing is between conditions the agent may act on and conditions the agent must never be allowed to act on.

Reactive loops, planned loops and where the human goes

A reactive loop takes one step at a time and decides afresh each turn. Cheap to build, adapts well to surprises and drifts badly: fifteen turns in, the agent is optimising a sub-goal it invented at turn six. A planned loop produces an explicit plan first, executes against it and replans only on deviation. More machinery, and the plan gives you an artefact to check the agent against, which is the actual benefit.

The evidence points at design rather than weights. The MAST work from Berkeley built a taxonomy of 14 failure modes in three categories from more than 1,600 annotated traces across seven multi-agent frameworks, with expert annotators reaching a kappa of 0.88. The three categories are specification and system design, inter-agent misalignment and task verification. Every one of those is a loop property. You do not fix them by swapping the model.

Consistency, not peak accuracy, is what production actually needs. Tau-bench introduced pass^k, the probability that all k independent attempts succeed, and reported GPT-4o-class function calling agents succeeding on under 50% of tasks with pass^8 below 25% in the retail domain. An agent that is right four times in five is an agent that is wrong roughly once a working day per user.

Human checkpoints are the pressure valve and the rule is narrow: interrupt on irreversible, high blast radius actions only, never on every step. LangGraph implements this with interrupt() to pause and surface a value, Command(resume=...) to continue and a checkpointer plus thread_id holding the state while a person thinks. Its durability setting is worth knowing by name. "exit" checkpoints only when the graph finishes, "async" persists while the next step runs, "sync" persists before each step starts. The first is fast and unrecoverable. The last is slower and survives a process crash mid-run. Choose per workload, not per project. And put a TTL sweep on paused threads, because a checkpoint waiting on an approval that never arrives is a leak with a database bill attached.

Record the trace, not the outcome

Outcome logging tells you a run failed. It will not tell you the run failed because a compaction pass dropped a constraint at turn nine and everything after that was confidently wrong.

Every turn should write the input context or a hash of it, the model and sampling parameters, the tool call with arguments, the raw result, tokens in and out, cost, latency, budget remaining and the stop reason when the loop ends. The reasoning trace matters as much as the answer, because a correct answer reached down a broken path will break next week on a slightly different input.

The vendor-neutral option is stabilising unevenly. OpenTelemetry's GenAI semantic conventions moved client spans out of experimental in early 2026 while agent and framework spans stayed experimental, and the June 2026 split into a separate semantic-conventions-genai repository says the work is mid-flight. Pin the version and put the attribute names behind a thin mapping layer of your own. Langfuse and Arize Phoenix both consume that shape, so the mapping layer is cheap insurance rather than wasted abstraction.

The most useful chart almost nobody builds is stop reasons over time, by category. Success predicate met. Budget exhausted. No progress. Human escalation. Hard error. If 30% of runs end on the turn cap, you do not have a model problem. You have a loop that never learned how to give up early and cleanly.

The strongest objection: the harness rots faster than the model

Here is the argument against everything above, and it is a good one. Better models genuinely dissolve loop complexity. A great deal of the scaffolding written in 2024 to babysit weaker models, the retry ladders, the plan validators, the elaborate decomposition trees, is dead code today. Building an intricate harness for a model that gets superseded in six months is a real waste of engineering time, and the people who over-engineered early were punished for it.

The measurements support the direction of travel. METR's Time Horizon 1.1 results, published 29 January 2026, put Claude Opus 4.5 at a 320 minute 50% time horizon with a confidence interval of 170 to 729 minutes, GPT-5 at 214 minutes, o3 at 121 and Claude Sonnet 3.7 at 60. The doubling time is 196 days across the full series, 131 days measured since 2023 and 89 days since 2024. That is a fast-rising floor underneath your scaffolding.

The answer is to separate two things that look identical in a codebase. Scaffolding that compensates for model weakness dies, and should be written so it is easy to delete: one module, one flag, no dependencies pointing into it. Loop machinery that encodes constraints the world imposes does not die, because the constraints have nothing to do with the model. Budgets exist because money is finite. Idempotency exists because networks drop packets and processes crash. Human checkpoints exist because some actions cannot be undone. Termination conditions exist because compute is not free. No frontier release changes any of that.

There is also a direction in which the objection runs backwards. Longer horizons raise the cost of loop bugs rather than lowering it. METR notes that measurements above 16 hours are unreliable with its current task suite, which is another way of saying we are arriving in the regime where a single run is long enough that losing it hurts. A twenty minute run that dies without a checkpoint is an annoyance. An eleven hour run that dies without one is a lost day and a very awkward status update.

Which leaves the part nobody has solved. A model that can work unattended for six hours is a model that can be wrong unattended for six hours, and the bill arrives either way. We have decent tools for stopping an agent that is looping and almost nothing for the agent at hour five that has just worked out, correctly, that hour two was a mistake.

Tools referenced

LangGraph, reviewed here: LangGraph review.

OpenAI Agents SDK, reviewed here: OpenAI Agents SDK review.

Temporal, reviewed here: Temporal review.

Langfuse, reviewed here: Langfuse review.

Arize Phoenix, reviewed here: Arize Phoenix review.

Sources

METR, Time Horizon 1.1 (29 January 2026): https://metr.org/blog/2026-1-29-time-horizon-1-1/

Cemri et al., Why Do Multi-Agent LLM Systems Fail? (MAST taxonomy): https://arxiv.org/abs/2503.13657

The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs: https://arxiv.org/abs/2509.09677

tau-bench: A Benchmark for Tool-Agent-User Interaction, introducing pass^k: https://arxiv.org/abs/2406.12045

Gartner, Over 40% of Agentic AI Projects Will Be Canceled by End of 2027: https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027

Anthropic, Effective context engineering for AI agents: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

OpenAI Agents SDK, Running agents (max_turns, MaxTurnsExceeded, guardrails): https://openai.github.io/openai-agents-python/running_agents/

LangGraph Durability reference (exit, async, sync): https://reference.langchain.com/python/langgraph/types/Durability