Context Engineering vs Prompt Engineering: What Changed When AI Agents Got Memory

Prompt engineering is dead. You've seen the headlines. They're wrong. Prompt engineering didn't get replaced — it got absorbed. The real story is about a discipline that grew up around it, one that matters more now that AI systems chain dozens of model calls together, pull in external data, and remember what happened three steps ago.

I run most of my day-to-day AI work through OpenCode on a terminal, not a chat window, and the difference between "writing a good prompt" and "managing what the agent actually sees at each step" is the single biggest lever I've found for making agentic workflows reliable instead of flaky. If you build anything with AI in 2026, this distinction matters.

The One-Liner

Prompt engineering improves how you ask. Context engineering improves what the model has to work with when you ask.

Short. Clean. Hold that frame.

What Prompt Engineering Is

Prompt engineering is the practice of crafting the wording, structure, and examples inside a single instruction to a language model.

Core question: "How should I phrase this?"

Classic techniques:

• Few-shot prompting — give examples, let the model follow the pattern
• Chain-of-thought — ask the model to "think step by step" before answering
• Role assignment — tell the model it's a security analyst, a poet, a senior dev
• Output format constraints — "respond in JSON," "use bullet points"

# Example prompt text (illustrative, not a file)

Classify the following customer message as Bug, Feature Request, or Question.

Examples:
Message: "The app crashes when I click Save"
Classification: Bug

Message: "Can you add dark mode?"
Classification: Feature Request

Message: "How do I reset my password?"
Classification: Question

Now classify this:
Message: "Why does the export button not work on mobile?"
Classification:

This works. For single-turn, stateless tasks — classification, extraction, rewriting, one-shot generation — prompt engineering is the whole job.

The limit is scope. A prompt only controls the text you typed. It doesn't decide what documents the model retrieved, what a tool call returned, or what the last ten turns of conversation contained.

Why It Broke

Three things happened between 2023 and 2025:

1. Context windows exploded. From 4K tokens to 200K, 1M, even 2M tokens. Suddenly models could hold entire codebases, document libraries, or hour-long conversations.

2. Agents started calling tools. Models stopped being stateless responders. They called APIs, read files, ran searches, executed code — and each tool output became new input for the next call.

3. Memory became real. Short-term (current session) and long-term (persisted facts across sessions). The same prompt now produces different outputs on different days because what surrounds it changed.

The moment those three things happened, the hand-written prompt became a small fraction of what the model actually processed.

Prompt engineering took the context window as given. Context engineering actively curates it.

What Context Engineering Is

Context engineering is the practice of designing, curating, and maintaining the full set of tokens a language model sees at inference time.

Core question: "What information does the model need to know right now?"

The term was popularized in June 2025. Shopify CEO Tobi Lütke wrote that it's "the art of providing all the context for the task to be plausibly solvable by the LLM." A few days later, Andrej Karpathy — who has since joined Anthropic — agreed on X, describing it as "the delicate art and science of filling the context window with just the right information for the next step."

Context engineering is not a single thing. It is a system:

  ┌─────────────────────────────────────────────────────┐
  │              WHAT THE MODEL SEES                    │
  │                                                     │
  │  ┌──────────┐  ┌──────────┐  ┌──────────────────┐   │ 
  │  │  Prompt  │  │ Documents│  │  History         │   │
  │  └──────────┘  └──────────┘  └──────────────────┘   │
  │                                                     │
  │  ┌──────────┐  ┌──────────┐  ┌──────────────────┐   │
  │  │  Tool    │  │  Memory  │  │  User Query      │   │
  │  │  Outputs │  │  (State) │  │                  │   │
  │  └──────────┘  └──────────┘  └──────────────────┘   │
  │                                                     │
  └─────────────────────────────────────────────────────┘
                         │
                         ▼
                    Model generates

Every box in that diagram is part of context. Prompt engineering controls the top-left box. Context engineering controls all of them.

The Four Strategies

LangChain formalized context engineering into four recurring moves. Most production agent architectures implement some variant of all four:

A Real Example From My Own Stack

Here's where this stops being theoretical. Say I ask an agent running in OpenCode to figure out why ratelimit.sandbox99.cc is returning 502s.

Prompt-engineering-only approach: I write one very well-crafted instruction — "You are an expert SRE. Diagnose the 502 error on this domain. Think step by step." — and paste in whatever logs I happen to have open. The model reasons well over what I gave it, but it never sees the actual Podman container status, the Caddy config, or the Cloudflare edge response, because none of that made it into the prompt. A great prompt over incomplete information still produces a wrong or incomplete diagnosis.

Context-engineering approach: the agent's harness assembles the context at runtime instead of relying on me to paste the right things:

# What the agent actually receives, assembled at runtime — not a file, illustrative only

- System prompt: "You are a sysadmin assistant. Diagnose using only the tool
  outputs provided. Cite which check revealed the issue."
- Tool output (SELECT): `podman ps -a` filtered to containers on that domain
- Tool output (SELECT): last 50 lines of `journalctl -u caddy`, not the full log
- Tool output (WRITE'd earlier, now recalled): "last incident on this domain
  was a stale Cloudflare DNS cache, resolved by proxy toggle"
- User query: "Why is ratelimit.sandbox99.cc returning 502?"

The model doesn't need a cleverer prompt to get this right — it needs the container status and the relevant log lines, filtered down instead of dumped whole. That's context engineering doing the actual work. The prompt engineering part (a clear system instruction, asking it to cite its source) still matters, but it's a small piece of why the diagnosis lands.

Before and After: Same Problem, Different Approach

Prompt Engineering Approach

You are a helpful assistant that answers questions about our product.
Use the following document to answer:

[Product documentation pasted here — all 50,000 tokens of it]

Question: How do I configure rate limiting on the API gateway?

Problem: you loaded everything. The model drowns in irrelevant tokens. Accuracy drops as context grows — a phenomenon Chroma's research team named "context rot" after testing 18 frontier models (including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3) and finding that every one of them got measurably less reliable as input length increased, even on simple tasks.

Context Engineering Approach

# System prompt (written once, stays stable)
You are a product support agent. Answer questions using only provided documents.
Cite the document section. If information is missing, say so.

# Context assembled at runtime:
  - User's current question: "How do I configure rate limiting?"
  - RAG retrieval: pulled 3 relevant chunks from a 100K-document corpus
  - User history: they asked about API authentication 2 turns ago
  - Tool available: search_docs, lookup_config

# Pseudocode illustrating context engineering — not a runnable file

def build_context(user_query, conversation_history):
    # Step 1: SELECT — retrieve only relevant docs
    relevant_chunks = vector_search(
        query=user_query,
        top_k=3,  # not 50, not 100 — just what fits
        filter={"section": "api-configuration"}
    )

    # Step 2: COMPRESS — trim conversation to key facts
    compressed_history = summarize_history(
        conversation_history,
        keep=["decisions", "open_questions", "user_preferences"]
    )

    # Step 3: ASSEMBLE — build the full context
    context = {
        "system_prompt": SUPPORT_AGENT_PROMPT,
        "relevant_docs": relevant_chunks,      # 3 chunks, not 50K tokens
        "history": compressed_history,          # summary, not raw log
        "user_query": user_query,
        "available_tools": ["search_docs", "lookup_config"]
    }

    return context  # model receives maybe 2K tokens, not 50K

Key difference: prompt engineering loads everything and hopes. Context engineering selects what matters and discards the rest.

The Three-Layer Stack

Prompt engineering isn't layer 1 of a two-layer system. It's layer 1 of three:

┌─────────────────────────────────────────────────────┐
│  Layer 3: SYSTEM LEVEL — Harness Engineering        │
│  Agent perceives own limits, recovers from tool     │
│  failures, verifies work, scopes sub-tasks          │
│                                                     │
│  ┌─────────────────────────────────────────────┐    │
│  │  Layer 2: SESSION LEVEL — Context Engineering│    │
│  │  What populates the context window each step │    │
│  │  Memory, retrieval, compaction, tool outputs │    │
│  │                                              │    │
│  │  ┌──────────────────────────────────────┐    │    │
│  │  │  Layer 1: MESSAGE LEVEL — Prompt Eng │    │    │
│  │  │  Wording, structure, examples,       │    │    │
│  │  │  output format constraints           │    │    │
│  │  └──────────────────────────────────────┘    │    │
│  └─────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────┘

Layer 1 still matters. A sloppy system prompt sabotages everything above it. But layer 1 is necessary, not sufficient. The ceiling of what you can fix without writing code is much lower than it was in 2023.

Side-by-Side: When to Use Which

Debugging: How to Tell Which Layer Failed

When output is wrong, check what the model was actually looking at:

Output is wrong
      │
      ├── Model HAD the right information → fix the prompt (Layer 1)
      │
      └── Model was MISSING or DROWNED in wrong info → fix context (Layer 2)

Concrete example:

Scenario: Support agent gives wrong refund policy answer

Check 1: What was in the context window?
  - Found: refund policy from 2024, not 2026 version
  - Diagnosis: stale document in retrieval
  - Fix: refresh vector store, re-index docs

Check 2: Was the prompt clear?
  - Found: prompt says "answer using provided documents"
  - But 8 documents were loaded, refund policy was token 45,000
  - Diagnosis: context rot — model attention degraded on long input
  - Fix: compress context, load only 2-3 relevant chunks

Check 3: Did the model have tools to find the answer?
  - Found: search_docs tool available but model did not use it
  - Diagnosis: tool description unclear
  - Fix: improve tool descriptions (still prompt engineering, but applied to the context layer)

I've hit variants of Check 1 myself with local RAG setups — the retrieval was working fine, the prompt was fine, and the answer was still wrong because the indexed document was simply out of date. No amount of prompt polish fixes stale data in the retrieval layer.

The Real Numbers

Context decisions dominate cost, too:

• Cached tokens cost roughly a tenth of uncached ones. Manus's engineering team found that with Claude Sonnet, cached input tokens run about $0.30 per million tokens versus $3.00 per million uncached — a 10x difference. If your context prefix stays byte-for-byte stable across requests, you save real money and get faster responses.
• Accuracy degrades before stated limits. Chroma's "Context Rot" report tested 18 frontier models and found performance dropping steadily as input grew — well before hitting any token ceiling. More tokens doesn't mean better answers.
• 82% of IT and data leaders say prompt engineering alone is no longer sufficient for production AI at scale, per DataHub's 2026 State of Context Management Report.

Practical Checklist

You only need prompt engineering when:

• The task is single-turn
• No tools or retrieval are involved
• The model already has everything it needs in the instruction
• Example: classify a ticket, rewrite a paragraph, extract fields

You need context engineering when:

• The agent chains multiple model calls
• External data feeds into responses
• Conversation history carries forward
• Tools return data that becomes the next input
• Anything where inputs are assembled at runtime

Start here:

1. Get the prompt clear (Layer 1 — fast, cheap, high leverage for simple tasks)
2. Get the context right (Layer 2 — correct information before the model is asked to use it)
3. Get the system resilient (Layer 3 — agent recovers from failures, verifies its own work)

The Bottom Line

Prompt engineering is table stakes. Context engineering is the multiplier.

Clear instructions are necessary and no longer sufficient. The teams whose AI systems actually ship reliably aren't the ones with the cleverest prompts. They're the ones who control what the model sees on every step.

Nothing replaced prompting. It got absorbed into a larger discipline. The system prompt is still there. The few-shot examples still work. The chain-of-thought scaffolding still helps.

It's simply no longer the only thing you control, or the only thing that can go wrong.