When context windows grew to hundreds of thousands of tokens, a common claim followed: just paste everything in, and retrieval is dead. Reality was more nuanced. Retrieval-augmented generation (RAG) didn't die — it grew up.
Why "just use long context" isn't enough
- Cost and latency. Stuffing a huge context into every request is expensive and slow. Retrieving the relevant slice is cheaper and faster.
- Freshness. Model weights are frozen at training time. Retrieval is how you answer questions about today.
- Attribution. Grounding an answer in retrieved sources lets you cite and verify — essential for trust.
- Signal. More tokens isn't more understanding; irrelevant context can actively distract a model ("lost in the middle").
What matured
Modern RAG is less "embed and hope" and more engineering: better chunking, hybrid keyword-plus-vector search, re-ranking, and query rewriting so the question the system searches with is a good one. The retrieval step is now treated as a first-class system to evaluate and tune, not an afterthought.
Long context and retrieval are not rivals. Retrieval decides what the model should look at; long context decides how much it can hold once it's there.
The takeaway
For anything that must be current, cited, or cost-controlled — which is most real applications — retrieval remains the backbone. The craft moved from "does it work at all?" to "how good is the thing you retrieve?"
