Beginner
What is Context Length?
How much a model can 'remember' at once — and why longer isn't always better.
Context length (or context window) is the maximum number of tokens a model can consider in a single request — your prompt plus its response. Modern models range from tens of thousands to over a million tokens.
A larger window lets you paste in whole documents, codebases, or long conversations. But attention gets expensive as context grows, and models can lose track of details buried in the middle of very long inputs (the 'lost in the middle' effect).
Techniques like RAG exist precisely because stuffing everything into context is costly and unreliable: retrieve only the relevant passages instead.
Key points
- Max tokens per request (input + output)
- Longer context = more cost and compute
- Models can miss details in very long inputs
- RAG is often better than giant prompts