What is the lost-in-the-middle problem in long contexts and how do you address it?
Reported at:Moonshot AI
Model answer
A concise answer you could give in the room. Try your own version first, then compare.
What the interviewer is probing
Whether you separate supporting a long context window from using that context well, and know the cost and quality trade-offs against retrieval.
How to structure your answer
- 1Define the problem: quality degrades for information in the middle and cost grows with length.
- 2Explain the causes: training distribution, attention dilution, positional extrapolation.
- 3List mitigations: ordering, chunked summarisation, retrieval, re-ranking, long-context fine-tuning.
- 4Decide between long context and RAG using cost, latency, freshness and citation needs.
Key points to hit
0/3 revealedTry to answer each one out loud before revealing it.
What is lost-in-the-middle?
Is a needle-in-a-haystack pass enough?
Long context or RAG?
Common pitfalls
- Equating the advertised window with effective capacity.
- Ignoring prefill cost and time-to-first-token.
- Not measuring on your own long-context tasks.
Likely follow-ups
- How would you build a long-context eval?
- How does prompt caching change the economics?
- How would you compress a 500k-token conversation?
Source: Outcome School · ai-engineering-interview-questions-company-wise. Question compiled by Outcome School from publicly reported interview experiences (Apache-2.0). The guidance on this page is original.