What are Multi-Query Attention (MQA) and Grouped-Query Attention (GQA), and what do they trade away?
Reported at:MetaMistral AI
Model answer
A concise answer you could give in the room. Try your own version first, then compare.
What the interviewer is probing
Whether you can do the memory arithmetic on the spot and reason about how the KV cache, not the weights, becomes the bottleneck for concurrency and long context.
How to structure your answer
- 1State why it exists: decode step t would otherwise recompute K and V for all previous tokens.
- 2Derive the size: 2 × layers × kv_heads × head_dim × seq_len × batch × bytes_per_element.
- 3Plug in real numbers for a known model to show the order of magnitude.
- 4List the levers: MQA/GQA/MLA, quantised cache, paging, eviction, prefix sharing.
Interactive · KV cache memory calculator
2 × layers × kv_heads × head_dim × seq × batch × bytes
share of one 80 GB GPU (cache only, excluding weights)
per token
320 KB
per sequence
2.5 GB
total
20.0 GB
Key points to hit
0/3 revealedTry to answer each one out loud before revealing it.
Typical size for a 70B-class model?
What do MQA and GQA trade?
What problem does PagedAttention solve?
Common pitfalls
- Forgetting the factor of 2 (keys and values).
- Using the number of query heads instead of KV heads after GQA.
- Ignoring that the cache grows linearly with batch × sequence length.
Likely follow-ups
- How many concurrent 8k-token users fit on one 80 GB GPU?
- How would you evict cache entries for very long chats?
- How does prefix caching interact with the KV cache?
Source: Outcome School · ai-engineering-interview-questions-company-wise. Question compiled by Outcome School from publicly reported interview experiences (Apache-2.0). The guidance on this page is original.