A long-context model's answer changed when the same relevant fact moved from the ends of its input to the middle.
Two retrieval tasks
Liu and colleagues tested multi-document question answering and key-value retrieval in Lost in the Middle. They varied the position of relevant information inside long contexts, isolating where a model must look while holding the information itself available.
The U-shaped result
Accuracy was often highest when the relevant material appeared near the beginning or end of the input and fell when it sat in the middle. The authors observed this pattern even in models built for long contexts. Position alone changed the measured performance, so a fixed placement can distort a context-use score.
The paper's evidence comes from controlled retrieval and question-answering tasks. Documents in use can scatter evidence across passages or require different reasoning, and the tested models and context lengths set the scope of the result. A follow-up evaluation of long-context systems should report scores at the beginning, middle, and end positions separately; the average can mask this pattern.
Bibliography
- Nelson F. Liu; Kevin Lin; John Hewitt; Ashwin Paranjape; Michele Bevilacqua; Fabio Petroni; Percy Liang. (2023). Lost in the Middle: How Language Models Use Long Contexts. arXiv:2307.03172v3. https://arxiv.org/abs/2307.03172v3
