What Is Context Engineering? Tokens, Context Windows, and RAG in Plain English

Context engineering is the discipline of deciding exactly what an AI model sees at the moment it works: the instructions, the documents, the conversation history, the tool results. The model can only reason over what's in its context window — so the quality of your AI's output is mostly determined before the model writes a single word.

The glossary gives you the one-line definitions of token, context window, and RAG. This is the longer story — how those three terms lock together into the concept that quietly decides whether your AI systems are brilliant or useless, and why "context engineering" replaced "prompt engineering" as the phrase serious builders use.

Start with the token — the meter everything runs on

A token is the chunk a language model reads and writes in. It's usually a fragment of a word — on average, a token works out to roughly three-quarters of an English word. Every model is priced in tokens and limited in tokens. When a pricing page says "$3 per million input tokens," and when a spec sheet says "200K context window," they're both counting the same unit.

You don't need to count tokens yourself. You need to know the unit exists, because it explains two things buyers otherwise find mysterious: why AI costs scale with how much text you push through the system, and why there's a hard ceiling on how much the model can consider at once. That ceiling is the context window.

What is a context window, really?

The context window is the model's working memory — everything it can "see" while producing an answer, measured in tokens. Your instructions, the documents you pasted, the whole conversation so far, the results that came back from tools: all of it competes for the same finite space.

Two properties matter in practice:

• It's a window, not a warehouse. Nothing outside the window exists to the model in that moment. The model isn't "forgetting" your instructions from an hour ago out of laziness — those tokens may simply have fallen out of the window.
• Full isn't optimal. Stuffing a window to the brim with marginally-relevant text degrades output and inflates cost. Models do their best work when the window holds the right material, not the most material.

Which raises the obvious question: if the window is finite and your company's knowledge isn't, how does the right material get in there at the right time? That's RAG's job.

What is RAG and where does it fit?

RAG — retrieval-augmented generation — is the standard answer to the finite-window problem. Instead of hoping the model memorized your world, a RAG system retrieves the relevant documents at question time (your contracts, your help docs, your price list), places them in the context window, and only then asks the model to answer. Retrieval, then generation. The name is literal.

The critical thing RAG is not: training. A RAG system doesn't change the model at all — it changes what the model reads before answering. This distinction is worth real money in vendor conversations, because "our AI is trained on your data" gets said in demos constantly, and nine times out of ten the honest sentence is "our system retrieves your documents into the prompt." Same three words in the pitch, wildly different products under the hood — and wildly different costs, timelines, and data-handling implications. What that kind of ambiguity does to a contract negotiation is its own article: what jargon confusion costs in AI vendor negotiations.

So what does a context engineer actually do?

Context engineering is the umbrella practice over all of the above. The job, whether a human does it or an agent system does it automatically, is answering one question well: what should the model see right now to do this task at the standard I need? In practice that breaks into decisions like:

• Instructions — what standing rules, tone, and constraints ride along with every request.
• Retrieval — which documents get pulled in, how many, and how they're trimmed.
• History — how much of the running conversation or task log stays in the window, and what gets summarized to save space.
• Tool results — what an agent's own actions feed back into the window as it loops through a task.

Notice that "phrase the prompt cleverly" doesn't make the list. Wording matters far less than selection. A plainly-worded request sitting on top of the right fifty pages beats a masterfully-worded request sitting on top of nothing — which is the whole argument for plain-language briefing, made at length at plainenglishprompts.com.

Prompt engineering vs context engineering

The shift in vocabulary tracks a shift in how AI is used. When AI meant one-shot chat, the prompt was the whole game. Now that agents run multi-step work — looping, calling tools, accumulating results — the window's contents change dozens of times per task, and managing that flow is the engineering. If the stack around the model is fuzzy to you, how the agentic stack fits together draws the full picture.

Why should a non-technical buyer care?

Because context is where AI projects actually live or die, and it's invisible in a demo. Every vendor's demo runs on hand-curated context. Your deployment runs on whatever their retrieval actually finds in your messy real-world documents. When you evaluate a tool, the questions that predict success aren't about the model — they're about context: What does the system put in front of the model for my use case? Where does it come from? What happens when the right document doesn't exist? Buyers who can ask those questions sound like they built the thing. That fluency is the point — and it's a shorter climb than it looks. Start with the concepts that actually matter for a $5–50M business.

FAQ

Is context engineering the same as prompt engineering?

No. Prompt engineering tunes the wording of a single request. Context engineering manages everything the model sees when it works — instructions, retrieved documents, conversation history, tool results — across an entire task. Prompting is one sentence of the brief; context is the whole briefing packet.

What is a token, in plain English?

A token is the chunk of text a model reads and writes in — usually a word fragment, roughly three-quarters of an English word on average. Models are priced and limited in tokens, which is why "how many tokens" shows up in every AI pricing page and every context-window spec.

Does RAG mean the model was trained on my data?

No — and this is the most common confusion in vendor pitches. RAG retrieves your documents at question time and places them in the model's context window. The model itself is unchanged. Training or fine-tuning actually modifies the model, which is a far bigger, rarer undertaking. When a vendor says "trained on your data," ask which one they mean.

Do bigger context windows make RAG obsolete?

Not for most businesses. Even a very large window can't hold an entire company's documents, and stuffing a window with irrelevant text degrades quality and raises cost. Retrieval — picking the right material before the model works — stays valuable no matter how big windows get.