A long context window tells you how much material an AI model can accept in one request. It does not promise that the model will use every part of that material equally well. A report may fit from cover to cover, yet the answer can still miss the one paragraph that settles the question.
Researchers call one version of this problem lost in the middle. In controlled tests published in 2024, they placed the same answer-bearing passage at different points among otherwise unchanged documents. Several models performed best when the useful passage appeared near the beginning or the end, and worse when it sat in the middle. The models and context sizes in that study are now dated, so the result is a warning to test the model you use, not a permanent score for every model.
The sketch below shows the pattern the researchers observed. It is qualitative: the depth of the curve is not a universal accuracy number.
Why fitting is not the same as finding
The context window is a capacity limit. Staying below it means the input can be processed; it says nothing by itself about retrieval or reasoning quality. In the study, the task was deliberately simple: one document contained the answer and the others were distractors. Moving that document changed performance even though the evidence and the correct answer stayed the same.
Adding more material can therefore create a trade-off. More pages increase the chance that the evidence is present, but they also give the model more material to sort through. The original study found that, in one open-domain question-answering test, increasing retrieval from 20 to 50 documents produced only a small improvement. Current Google guidance makes the same broader caution: a model that performs well when retrieving one “needle” may be less accurate when several pieces must be found.
Make the important page easier to find
You do not need to guess one perfect prompt layout. Use a few simple practices, then test the result on your own documents and model.
- Remove material that cannot help. A larger pile is not automatically a better source. Send the relevant chapters, records or search results when you can.
- Give the material visible structure. Label documents, sections and page numbers so the model can identify where an answer came from.
- Put the question after the documents. Anthropic’s current guidance also recommends placing the query at the end for long inputs. Keep the requested output and critical constraints there too.
- Ask for evidence before the conclusion. Have the model extract the relevant passage and name its document or page, then answer from that evidence. The same guidance recommends this quote-first pattern for long-document tasks.
- Test position, not just one happy path. Build a few examples where the decisive passage appears near the start, middle and end. A system that succeeds only when the evidence lands in a favoured position is not ready for consequential use.
Where you meet it
The problem appears whenever an assistant receives more material than a person would comfortably scan: a contract bundle, a research folder, a long support history, dozens of retrieved passages or a large codebase. It also explains an everyday frustration: “The answer was on page 47—why did it ignore it?”
The practical lesson is modest. Treat an advertised context length as a statement about what can enter the model, then verify what the model can reliably use. For an important decision, the proof is not that the page was included. The proof is that the system can find it, show it and use it when its position changes.