Context Engineering

The Four Pillars

You are a financial advisor for retail investors.

Rules:
- Never recommend specific stocks
- Always include risk disclaimers
- Use simple language (8th grade reading level)
- If the user asks about crypto, redirect to our crypto specialist

Tone: Professional but approachable. Use analogies.

The honest read on "context engineering" as a newly-named discipline, tracked across r/MachineLearning, r/LocalLLaMA, and r/PromptEngineering: the four-pillars framing is useful as a checklist, and the community's sharper observation is that the bottleneck in production LLM systems is almost never "we didn't give the model enough context" — it's "we gave the model too much context, badly ordered, and it lost track of what mattered". The lost-in-the-middle paper (Liu et al., 2023), the Anthropic long-context benchmarks, and the LLMlingua prompt compression research all point to the same pattern: more tokens is not better, relevant-tokens-first is better.

Where the community correctly pushes back on the "200K context solves everything" pitch: large context windows make it easy to be lazy about retrieval. The teams getting good results are still doing the hard work of scoring, ranking, and pruning their context to the smallest set that lets the model answer — exactly as if the window were 8K. The RAG vs long-context ablations from the Chroma team are clear: curated 16K beats dumped 128K on most downstream metrics.

Pragmatic rule from people running real context pipelines: write a context budget per task (tokens for system, tokens for retrieval, tokens for examples, tokens for user input), enforce it in code, and when you exceed it, cut rather than upgrade to a bigger model. The discipline of cutting forces you to learn what the model actually needs to answer, which is worth more than the extra tokens.

Context Budget Management

Advanced Techniques

Test Your Understanding

Where to Go From Here

You now understand context architecture. Next, explore a specific challenge: the Lost-in-the-Middle problem, why models struggle with information buried in long contexts, and how to engineer around it.

Continue to Lost-in-the-Middle: Advanced RAG to learn about context position effects.

The Lost-in-the-Middle Effect

The honest read on lost-in-the-middle three years after the original Liu et al. paper, tracked across r/MachineLearning, r/LocalLLaMA, and r/LangChain: the effect is real, it has softened with newer models but not disappeared, and every vendor claim of "perfect recall across 1M tokens" is marketing until you verify it on your data. The NoLiMA benchmark from Anthropic, the RULER benchmark, and the Chroma context rot research all show the same picture: synthetic needle-in-a-haystack tests overstate real-world performance, because real documents contain distractors, partial matches, and semantically related noise that pure needle tests don't.

Where the community correctly pushes back on the "long context kills RAG" framing: long context and RAG are complementary, not competing. The teams with the best retrieval quality combine a 10-15K window of carefully ranked context with a long-context model that can hold the conversation history and user instructions. Dumping 128K of unranked chunks into the window performs worse than classic 8K RAG on most real queries; ranking matters more than window size.

Pragmatic rule from people who run production RAG: always do a reranking pass (Cohere Rerank, Jina Reranker, or a cross-encoder you host yourself), always put your highest-scored chunks at the start and the end of the context, and always measure recall on your own eval set — not on MTEB, not on BEIR, not on marketing slides. The position-sensitivity curve is subtle and model-specific, and you only learn yours by testing.

Advanced RAG Architecture

Reranking: The Key to Quality

Test Your Understanding

Next Steps

You understand how position affects AI context. The final article in this module covers prompt caching and MCP protocol, optimizing AI systems for production efficiency.

• →Contextual Retrieval and Advanced RAG, How contextual enrichment solves the "Lost in the Middle" problem

Continue to Prompt Caching & MCP Protocol to learn about production optimization.

The part of prompt caching nobody optimises for (until it bites)

The standard pitch is caching = save 80-90% on tokens. True. What gets left out of most write-ups, and what engineers on r/LangChain and r/OpenAI keep learning the hard way, is that caching is only a win when your cache hits. And whether it hits depends on architectural choices that look innocent until you measure them.

Three concrete traps worth naming:

• →TTL asymmetry is real. Anthropic's default TTL is 5 minutes, OpenAI's 1 hour, Google's in between. If your traffic is bursty with 10+ minute quiet periods, Anthropic's cache will evaporate between bursts and your "savings" will quietly vanish. Anthropic's prompt caching docs now offer 1-hour cache tiers at a premium — worth the math for quiet-but-steady workloads.
• →Cache boundaries must match change frequency. If you shuffle RAG chunks into a different order between requests, the cache breaks. Sort retrieved chunks by a stable key (document ID) before concatenating; this single change has single-handedly saved teams five-figure monthly bills.
• →Dynamic system prompts are a silent cache killer. Injecting the current timestamp or a user ID into the system prompt seems harmless. It invalidates the cache on every request. Move anything dynamic to the end of your prompt, always.

On the MCP side, the official Model Context Protocol spec is short and readable; if you're still writing bespoke function schemas per vendor in 2026, you're building tech debt. The MCP announcement from Anthropic is worth five minutes to understand why this standard won the race.

Prompt Caching: Stop Paying Twice for the Same Tokens

Every API call sends your system prompt + RAG context + conversation history. If your system prompt is 2,000 tokens and stays the same across all queries, you are paying for those 2,000 tokens every single time. Prompt caching tells the API: "I already sent this prefix, just reuse it."

Request 1: [8000 static tokens] + [2000 dynamic tokens] = 10,000 tokens billed
Request 2: [8000 static tokens] + [2000 dynamic tokens] = 10,000 tokens billed
Request 3: [8000 static tokens] + [2000 dynamic tokens] = 10,000 tokens billed

Total: 30,000 tokens billed for 6,000 tokens of unique content

MCP: The Model Context Protocol

MCP Server: 'Weather Service'
Tools:
  - get_weather(city, units)
  - get_forecast(city, days)
Resources:
  - weather://cities -> list of supported cities
  - weather://alerts -> active weather alerts

Production Optimization Checklist

Test Your Understanding

Congratulations!

You have completed Module 9 and the entire advanced AI curriculum. You now understand:

• →Context engineering, designing the information environment for AI
• →Lost-in-the-middle, position effects and optimization
• →Production optimization, caching, MCP, and cost management

These are the skills that separate prompt hobbyists from production AI engineers.

Return to the Module 9 overview to review your progress and explore next steps.