Context Engineering vs Prompt Engineering: One Task, Two Fixes, Two Bills

An agent fails a task. You have two ways to fix it. Rewrite the instruction, or change what the model can see when it reads that instruction. That choice is the whole of context engineering vs prompt engineering, and in 2026 the second option is where nearly all the remaining gains live. Prompt engineering optimises a string. Context engineering decides which tokens are in the window at all: what got retrieved, what a tool printed, how much history survived compaction and in what order the whole thing was assembled.

Andrej Karpathy pushed the term into general use in June 2025, arguing that people hear the word prompt and picture the short task description they type into a chat box, while every industrial LLM application is really doing something else entirely. Anthropic's engineering team formalised it that September, framing context as a finite resource with diminishing marginal returns, where the goal is the smallest set of high-signal tokens that produces the behaviour you want.

I have written separately about why the window is a budget rather than a bucket, and what the attention research says about paying for tokens the model cannot reach. This piece is the other half. Which lever to reach for, what each one costs and how to tell which one moved the number.

The same task, fixed two ways

Take a support agent that adjudicates warranty claims on industrial sensors. It reads a claim, decides coverage and cites the governing clause. It is wrong about a fifth of the time, mostly by declaring coverage on units that are out of term.

The prompt fix writes a better instruction. Number the steps. Force the model to state the purchase date and the warranty term before it states a verdict. Add a refusal path for missing dates. Add two worked examples, one covered and one not. This is real work and it does help, particularly on the failure mode where a model skipped a step it was perfectly capable of performing.

The context fix does not touch the instruction at all. Today the agent receives the full warranty document, all forty pages of it, on every single call, because that was the easy thing to build. Call it 31,000 tokens. The context fix retrieves the three clauses that govern this SKU, attaches the actual purchase record for this serial number, drops the other thirty-seven pages and places the purchase record last, where recency helps rather than in the middle where it does not. Call it 2,400 tokens.

Now the arithmetic, using published list prices. Claude Opus 5 bills input at $5 per million tokens. The pasted version costs $0.155 of input before the model writes a character. The retrieved version costs $0.012. An agent that takes 200 turns to close a difficult claim pays that difference 200 times: $31.00 against $2.40, on input alone, on one claim.

Both fixes can work. They fail differently, though, and that is the part worth internalising. The prompt fix is bounded by what the model can already do with the evidence in front of it. If the purchase record is not in the window, no instruction retrieves it. You can write check the purchase date in bold, in capitals, wrapped in XML tags and repeated in three languages, and the model will still produce a date, because a language model asked for a fact it does not have will manufacture a plausible one.

The context fix is bounded from the other direction. It cannot make the model reason better about evidence it now has. If the clause is present, correctly ordered and the model still misreads a nested exclusion, that is an instruction problem or a model problem, and no amount of retriever tuning touches it.

So the decision rule is boring on purpose. If the failure is that the model did not know, it is context. If the failure is that the model knew and did the wrong thing with it, it is the prompt. Most teams reach for the prompt regardless, because the prompt is a text file and the retriever is a system.

A prompt edit changes a string. A context edit changes your invoice, your p95 latency and your cache hit rate.

Context engineering vs prompt engineering, in one sentence

Prompt engineering optimises the instruction. Context engineering manages the budget the instruction is spent inside.

Chinese settled this faster than English did, and the reason is morphological. The Chinese term for prompt engineering is 提示词工程. The middle character, 词, means word. The discipline is named at the level of the word, explicitly, in its own name. Context engineering is 上下文工程, where 上下文 is literally above-below text, the material surrounding the thing you are looking at.

Reading Chinese developer writing from Shenzhen through the second half of 2025, I never once encountered the argument that consumed English-language discussion for months, the one about whether retrieved documents count as part of the prompt. The morphemes had already answered it. Chinese-language coverage from July 2025 onward treated 上下文工程 as the superset that contains 提示词工程, which is the correct relationship. English-language coverage mostly framed it as a replacement, with headlines announcing that prompt engineering was dead. It is not dead. It got smaller, and it got automated.

Automated quite literally. GEPA, from Agrawal and colleagues, accepted as an oral at ICLR 2026, evolves prompt instructions by reading execution traces and reflecting on them in natural language instead of collapsing feedback into a scalar reward. Across six tasks it beats GRPO by 6 percent on average and by up to 20 percent, using up to 35 times fewer rollouts, and it ships as dspy.GEPA. If your prompt engineering practice is a human reading outputs and rewording the instruction, a machine now runs that loop better than you and considerably cheaper.

Tokens are money, and they are also attention

The cost model is what moved the bottleneck. Two meters run at once, and they are not the same meter.

The money meter is published. Claude Opus 5 lists at $5 per million input tokens and $25 per million output. Sonnet 5 at $2 and $10. Haiku 4.5 at $1 and $5. Input is the cheap side per token and the expensive side in practice, because input is where agents put everything.

What actually fills the window is rarely prose. Declaring Anthropic's computer-use toolset adds about 4,500 input tokens to every request before you have written a word of instruction. The browser toolset adds about 6,600. Even a modest tool list drags in a tool-use system prompt: 286 tokens on Opus 5 with tool choice set to auto, 406 with it set to any. One web fetch of a 500 kB research PDF lands around 125,000 tokens in your window, per Anthropic's own sizing estimate. A single careless tool call can cost more than every prompt you have ever written, combined.

The budget also shifts under you. Anthropic's pricing documentation notes that Claude 4.7 and later use a newer tokenizer producing roughly 30 percent more tokens for the same text. Same document, same window size, thirty percent less of it available after a model upgrade. That is a context engineering problem with no prompt engineering equivalent.

The second meter is attention, and the evidence for it is covered in the companion piece. The short version here: Chroma tested 18 models in July 2025 and found accuracy degrading at every input-length increment, including on tasks a first-year student would find trivial. Both meters point the same direction. Tokens you do not need cost money, and they cost accuracy, so pruning is rarely a trade-off.

Cache-aware ordering is arithmetic, not taste

Prompt caching is a prefix match. That single mechanical fact turns the order of your context from an editorial preference into an arithmetic one.

Anthropic builds cache prefixes in a fixed order: tools, then system, then messages. Changes at any level invalidate that level and everything after it, so changing one tool definition invalidates all three. You get up to four cache breakpoints. The system checks at most 20 blocks per breakpoint when hunting for a match, counting the breakpoint itself as the first, and if it finds nothing in that window it stops looking.

The economics are symmetrical across vendors now. A five-minute cache write costs 1.25 times base input, a one-hour write costs 2 times and a read costs 0.1 times, which means the five-minute cache pays for itself after a single read and the one-hour cache after two. OpenAI converged on the same shape for GPT-5.6 and later, 1.25x on the write and 0.1x on the read, with a 1,024-token minimum and a 30-minute TTL measured from the most recent write or reuse.

That produces one ordering rule and one prohibition. The rule: most stable first, most volatile last. Frozen system instructions, then a deterministically serialised tool list, then retrieved evidence, then history, then the live turn. The prohibition: nothing above a breakpoint may change more often than the TTL. A timestamp in the system prompt, a tool list assembled from a Python set, a session identifier in the header block. Each of those is a full recompute on every call, billed at full input price, and none of them will ever appear in your eval, because your eval measures accuracy.

There is also a floor most teams never check. Anthropic will not cache below a per-model minimum: 512 tokens on Opus 5, 1,024 on Sonnet 5 and Opus 4.8, 4,096 on Haiku 4.5 and Opus 4.5. Requests under the floor are processed without caching and no error is returned. That is a quiet way to pay full price for a year.

What belongs in the system prompt and what gets retrieved

Three tests, applied in that order.

Frequency. Does every request need this? If yes, and it is stable, it belongs in the system prompt above a cache breakpoint, where you pay for it once per TTL rather than once per call. If only some requests need it, retrieve it. A tone-of-voice guide is system prompt material. A customer's account history is not, however much easier it is to paste.

Volatility. How often does it change relative to your cache TTL? Content that mutates hourly does not belong above a one-hour breakpoint. This is where careful teams get caught, because the content genuinely is needed on every call, so the frequency test says system prompt, but it changes faster than the cache and quietly costs more there than it would as a retrieved block.

Precision. Can you name the specific piece that matters? If yes, retrieve that piece rather than the container it happens to live in. The forty-page warranty document fails this test. Three clauses pass it.

The failure mode on the system prompt side has a name, or rather two. Researchers from Stanford and SambaNova published Agentic Context Engineering in October 2025, treating context as an evolving playbook rather than a paragraph, and they named the two ways hand-maintained context degrades: brevity bias, where iterative rewriting drops domain detail in favour of something that reads cleanly, and context collapse, where repeated rewriting erodes specifics until the playbook is generic. Their incremental-update approach beat strong baselines by 10.6 percent on agent benchmarks and 8.6 percent on finance, without labelled supervision, learning purely from execution feedback.

The lesson transfers even if you never touch their framework. A system prompt edited by hand every sprint drifts toward vague, because vague is what survives compression. Version it. Diff it. Treat deletions from it as changes requiring evidence, not as tidying up.

How to tell whether a context change helped

Most teams change three things at once and attribute the delta to whichever one they are proudest of. Four habits fix that.

Move one lever per run. Obvious, and almost never done, because context changes and prompt changes get bundled into the same pull request by the same person on the same afternoon.

Budget far more repetitions for prompt deltas than for context deltas. The Wharton team ran GPQA Diamond questions 100 times per condition on GPT-4o and GPT-4o-mini in March 2025 and found that switching from please to I order could shift performance by up to 60 percentage points in either direction, question by question. Prompt effects are real, and they are also enormously noisy. If you tested a new prompt on twenty examples and it went up four points, you measured nothing at all.

Report score and tokens as one number, never separately. A context change that gains two points while tripling input tokens is a regression wearing a win. Anthropic's context management release is the cleanest published example of reporting both halves together: on a 100-turn web search evaluation, clearing stale tool results let agents complete workflows that otherwise died of context exhaustion, while cutting token consumption by 84 percent. Context editing alone improved performance 29 percent over baseline, and combined with a memory tool, 39 percent. Both halves, same announcement. That is the standard.

Keep a frozen holdout of the exact failures that motivated the change. Six months from now somebody will propose deleting your retrieval layer because the new model has a bigger window, and the only honest way to evaluate that proposal is to rerun the cases that made retrieval necessary in the first place. Almost nobody keeps these. It is the cheapest artefact in the entire practice and the first one thrown away.

The strongest objection: the line does not exist

The best argument against this whole framing is that it describes a distinction without a difference, and it deserves a proper hearing rather than a strawman.

It goes like this. The system prompt is context. Few-shot examples are context. People were stuffing retrieved documents into prompts in 2021 and nobody called it a separate discipline. Everything that reaches the model arrives as one flat sequence of tokens, and the model has no idea which part a human typed and which part a retriever fetched. Drawing a boundary through a single undifferentiated sequence is rebranding, and it mostly benefits people selling courses on the new word. Beginners now believe there are two skills where there is one.

The ontology in that objection is correct. There is one sequence. The model cannot tell.

The practice is where it comes apart, and the failure is organisational rather than conceptual. A prompt change is a string edit in one file, reviewed by one person, deployed in minutes, with a cost impact of exactly zero. A context change touches the retriever, the chunking strategy, the tool output serialiser, the compaction policy and the cache breakpoint layout. It changes your bill. It changes your p95 latency. It can break in production for reasons that never surface in your eval set, because your eval set holds clean documents and production holds a 500 kB PDF. Two things that share an ontology but share no blast radius, no review process, no test strategy and no cost model are two disciplines in every sense that matters to whoever is on call.

One honest concession. If you are doing single-turn work over a document you already have, with no tools and no accumulated history, prompt engineering is the entire job and none of this applies to you. The distinction only starts earning its keep at the point where something other than a human decides what enters the window. That is the actual boundary, and it is architectural rather than conceptual. Prompt engineering is what you do when you control the input by hand. Context engineering is what you do when you do not.

Both halves are being eaten by the same machine

GEPA optimises the instruction by reading execution traces. Agentic Context Engineering optimises the assembled context by reading execution traces. Same input, opposite sides of a line only humans can see. Neither optimiser knows or cares which tokens a person typed. Run both against the same agent and the boundary this article spent two thousand words defending becomes a scheduling detail inside somebody's optimisation loop.

What does not automate is the input to both. Which traces. Which tools. Which corpus. Which of your internal systems the retriever is permitted to reach in the first place.

Every published gain in this field, the 84 percent token cut, the 10.6 percent on agent benchmarks, the 20 percent over GRPO, is measured against a corpus somebody already decided to include. Nobody publishes that decision. It is the one part of context engineering that is not an engineering decision at all, and on most teams it is sitting in a config file, unversioned, maintained by whoever wrote it first.

Tools referenced

LangGraph, reviewed here: LangGraph review.

LlamaIndex, reviewed here: LlamaIndex review.

RAGFlow, reviewed here: RAGFlow review.

pgvector, reviewed here: pgvector review.

Qdrant, reviewed here: Qdrant review.

Langfuse, reviewed here: Langfuse review.

Sources

Anthropic Engineering, Effective context engineering for AI agents (29 September 2025): https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents

Anthropic, Managing context on the Claude Developer Platform (29 September 2025) - context editing and memory tool benchmark figures: https://claude.com/blog/context-management

Anthropic, Prompt caching documentation - multipliers, breakpoints, 20-block lookback, per-model minimums: https://platform.claude.com/docs/en/build-with-claude/prompt-caching

Anthropic, Pricing documentation - model rates, tool token overheads, tokenizer note: https://platform.claude.com/docs/en/about-claude/pricing

OpenAI, Prompt caching guide - GPT-5.6 write and read multipliers, minimum tokens, TTL: https://developers.openai.com/api/docs/guides/prompt-caching

Meincke, Mollick, Mollick and Shapiro, Prompt Engineering is Complicated and Contingent (Wharton GAIL, 4 March 2025): https://gail.wharton.upenn.edu/research-and-insights/tech-report-prompt-engineering-is-complicated-and-contingent/

Agrawal et al., GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning (arXiv 2507.19457, ICLR 2026 Oral): https://arxiv.org/abs/2507.19457

Zhang et al., Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models (arXiv 2510.04618): https://arxiv.org/abs/2510.04618