AI Engineering Interviews
598 real interview questions asked at 35 AI companies — each with an answer outline, key points, pitfalls and links to the related explainers on this site.
Questions
598
Companies
35
Core topics
10
Question set source. The questions catalogued here come from ai-engineering-interview-questions-company-wise, maintained by Outcome School and licensed Apache-2.0. They were compiled from publicly reported interview experiences — nothing confidential. The explanations, diagrams and code on this site are original work.
Common Questions Asked Across Companies
10 sections · 119 questions
LLM Internals and Architecture
16 questions- 001Explain scaled dot-product attention and why the 1/sqrt(d_k) scaling factor matters.
- 002What is the KV cache, and what are its memory implications at scale? Derive the formula.
- 003What are Multi-Query Attention (MQA) and Grouped-Query Attention (GQA), and what do they trade away?
- 004What is Multi-head Latent Attention (MLA) and why did DeepSeek introduce it?
- 005Explain FlashAttention. It does not reduce FLOPs, so why is it faster?
- 006How does Byte Pair Encoding work, and what are its failure modes (numbers, code, non-Latin scripts)?
- 007What is positional encoding in transformers, and how has it evolved (sinusoidal → learned → RoPE → ALiBi)?
- 008Explain RoPE and how position interpolation / YaRN extend context beyond the trained length.
- 009What do the Chinchilla scaling laws say, and how do they differ from earlier scaling intuitions?
- 010What is a mixture-of-experts architecture and how does it scale capacity without scaling FLOPs?
- 011Explain the difference between pre-training, supervised fine-tuning and preference optimisation.
- 012Compare greedy, beam search, top-k, top-p and temperature sampling. When does each fail?
- 013What is the lost-in-the-middle problem in long contexts and how do you address it?
- 014Why is LayerNorm placed pre-block in modern transformers, and what is RMSNorm?
- 015Explain SwiGLU and why gated activations replaced ReLU/GELU in modern LLM MLP blocks.
- 016Walk me through what happens, tensor by tensor, in one forward pass of a decoder-only transformer.
Inference, Serving and GPU Performance
15 questions- 017Explain the prefill and decode phases. Why is prefill compute-bound and decode memory-bandwidth-bound?
- 018What is continuous (in-flight) batching and why did it replace static batching?
- 019How does PagedAttention work, and what problem of KV-cache fragmentation does it solve?
- 020What is speculative decoding? Why is output quality preserved, and when does it not help?
- 021Explain prefix caching / prompt caching. When should you use it, and what invalidates a cached prefix?
- 022Compare FP16, BF16, FP8, INT8, INT4 and FP4 for serving. What breaks at each step down?
- 023Compare tensor, pipeline, data, sequence and expert parallelism. When do you combine them?
- 024Estimate the GPU memory needed to serve a 70B model: weights, KV cache, activations, fragmentation.
- 025What are TTFT, TPOT, ITL and throughput, and how do they trade against each other?
- 026Do the roofline maths: how many tokens/sec can one H100 produce for a 70B model at batch size 1?
- 027When would you choose vLLM vs SGLang vs TensorRT-LLM vs a custom stack?
- 028How would you cut LLM serving cost by 10x? Enumerate every lever and rank them.
- 029Your p99 latency doubled after a deploy with no model change. Walk through the diagnosis.
- 030What is chunked prefill, and why does it improve tail latency under mixed traffic?
- 031Explain disaggregated prefill/decode serving and when it pays for itself.
RAG and Retrieval
12 questions- 032What chunking strategy would you use for a large technical documentation corpus, and why?
- 033How do you choose between a sparse retriever (BM25) and a dense retriever? When do you need both?
- 034What is a reranker, when should you use one, and what does a cross-encoder cost you?
- 035How would you evaluate the quality of a RAG pipeline: retrieval and generation separately?
- 036What is HyDE (hypothetical document embeddings) and when does it outperform standard dense retrieval?
- 037How does agentic RAG differ from standard RAG, and when is the extra complexity justified?
- 038What causes semantic drift in embedding search and how do you detect it?
- 039Design permission-aware retrieval: users must never see content they can't access in the source system.
- 040Compare HNSW, IVF-PQ and flat indexes. How do you pick, and what does recall@k cost in latency?
- 041How do you handle tables, figures and multi-column PDFs in a retrieval pipeline?
- 042How do you keep an index fresh when the underlying corpus changes continuously?
- 043How do you attribute every claim in a generated answer to a specific retrieved span?
Agents and Tool Use
12 questions- 044Explain the ReAct pattern and what it solves over chain-of-thought alone.
- 045How do you handle tool-call errors, timeouts and retries in an agentic loop?
- 046What is the difference between structured output and function calling?
- 047What is MCP (Model Context Protocol) and how does it differ from traditional function calling?
- 048How many tools is too many? How do you design tool schemas an LLM can actually use correctly?
- 049How does multi-agent orchestration work, and when does it break down?
- 050Design memory for a long-running agent: what do you store, where, and how do you retrieve it?
- 051How does an agent decide when to call a tool versus answer from its own knowledge?
- 052What makes an agent loop terminate correctly? How do you bound cost and steps?
- 053How do you make an agent's actions reversible, or at least auditable, in a production system?
- 054Design human-in-the-loop approval for an agent that takes consequential actions.
- 055Your agent drifts after a long run and confidently works on the wrong thing. Diagnose it.
Fine-Tuning, Post-Training and Alignment
12 questions- 056Walk me through RLHF end to end: reward model, policy optimisation, KL penalty.
- 057What is DPO and why did it displace PPO-based RLHF at many labs? When is online RL still better?
- 058Explain GRPO and why dropping the value network matters at scale.
- 059Explain the LoRA decomposition mathematically. Why does it work, and how do you choose the rank r?
- 060How does QLoRA achieve its memory reduction, and what are the quantization trade-offs?
- 061Compare LoRA, prefix tuning, prompt tuning and full fine-tuning. When would you choose each?
- 062What is catastrophic forgetting and how do you mitigate it during fine-tuning?
- 063Prompting, RAG or fine-tuning: give me your decision framework with cost and latency attached.
- 064Do the GPU memory maths for full fine-tuning a 7B model in bf16 with Adam. Now with LoRA.
- 065What is RLVR (RL with verifiable rewards) and where does it beat a learned reward model?
- 066Explain reward hacking in RLHF and how labs address it.
- 067What is distillation, and how do you build a strong small model from a large one?
Evaluation and Observability
10 questions- 068Design an LLM-as-judge evaluation. What are its known biases and how do you correct for them?
- 069How do you build an eval set when there is no labelled ground truth and experts are expensive?
- 070How do you detect and measure hallucinations in a production RAG system?
- 071Design the regression gate that decides whether a prompt or model change ships.
- 072Why do benchmark scores improve while users say the system got worse? Enumerate the reasons.
- 073What is benchmark contamination and how do you guard against it?
- 074What observability does a production LLM system need: traces, spans, costs, feedback?
- 075How do you manage prompt versioning and rollbacks in production?
- 076Design online evaluation: what do you log, what do you sample, and what do you A/B?
- 077How would you evaluate an agent, as opposed to a single model response?
Safety, Security and Responsible AI
10 questions- 078What is prompt injection (direct and indirect), and what is your layered defence?
- 079Walk me through the OWASP Top 10 for LLM applications and which ones actually bite in practice.
- 080What is the difference between jailbreaking and adversarial prompting?
- 081Design guardrails for a consumer-facing assistant. Input filters, output filters, or both?
- 082What is Constitutional AI and how does it differ from RLHF? What is RLAIF?
- 083How do you prevent an agent with tool access from exfiltrating data via a malicious web page?
- 084How do you handle PII in prompts, logs and training data?
- 085What is mechanistic interpretability and why do labs invest in it?
- 086How would you audit a deployed model for differential performance across user groups?
- 087Design a red-teaming programme for a model you are about to release.
Multimodal, Speech and Voice AI
10 questions- 088How do vision-language models get images into an LLM: projector, cross-attention, or native tokens?
- 089What changes when you move from images to video?
- 090Budget the latency for a real-time voice agent: VAD, ASR, LLM, TTS, network. Where does the time go?
- 091Design barge-in / interruption handling for a voice agent.
- 092Cascaded ASR+LLM+TTS versus native speech-to-speech: argue both sides.
- 093How do you evaluate ASR quality beyond WER, and TTS quality when there is no single correct output?
- 094How do you handle code-switching and accents in a production ASR system?
- 095Explain streaming TTS chunking and jitter-buffer sizing.
- 096Design a diarisation system and explain how you attribute roles, not just clusters.
- 097How would you build multimodal retrieval over images, video and text in one index?
AI System Design
10 questions- 098Design an enterprise RAG assistant over 10M documents with per-user permissions.
- 099Design a code assistant: repo indexing, context assembly, edit application, evaluation.
- 100Design a customer-support agent that can take real actions, with escalation to humans.
- 101Design semantic search over a large product catalogue.
- 102Design a content-moderation system combining classifiers and LLMs.
- 103Design a document-intelligence pipeline: scanned PDFs in, structured fields out, at 10M documents.
- 104Design a Text-to-SQL system over a warehouse with thousands of tables.
- 105Design a meeting assistant: recording, diarisation, summary, action items, integrations.
- 106Design an LLM gateway: routing across providers, failover, caching, budgets and rate limits.
- 107Design the serving stack for a consumer chat assistant at hundreds of millions of users.
Coding and Data Structures
12 questions- 108Implement scaled dot-product attention with a causal mask, from scratch, in NumPy or PyTorch.
- 109Implement multi-head attention, then convert it to grouped-query attention.
- 110Implement a KV cache and single-step decode.
- 111Implement BPE training and encoding from scratch.
- 112Implement top-k, top-p and temperature sampling over a logits vector.
- 113Implement an LRU cache with O(1) get/put, then add TTL.
- 114Implement a token-bucket rate limiter, then make it distributed.
- 115Write an async batch processor over an API with concurrency limits, retries with jitter and error isolation.
- 116Write a streaming SSE/JSON parser that handles arbitrary chunk boundaries.
- 117Implement a text chunker with overlap that never splits a semantic unit.
- 118Implement cosine similarity search over embeddings, then explain why you would not ship it.
- 119Implement a minimal agent loop with tool dispatch, error handling and a step budget.
Frontier AI Labs
12 sections · 197 questions
Anthropic
37 questionsRoles: Member of Technical Staff (MTS), Software Engineer (product / infra / API-serving), Research Engineer, Research Scientist, Applied AI Engineer, Forward Deployed Engineer (Applied AI), Performance Engineer (inference & kernels), Product Engineer (Claude Code / Claude.ai).
Loop: Recruiter screen (~30 min, substantive and failable) → CodeSignal-style or live coding assessment (~70–90 min, one practical problem in ~4 progressive levels) → virtual onsite of roughly five rounds: project deep-dive, one or two coding rounds, system design, and a dedicated values/culture round. Reported end-to-end: 3 weeks to ~2 months. Some MLE loops now include an AI-collaboration round where Claude is provided and you are graded on how you direct and verify it.
Coding and Data Structures
- 120Build core business logic for a toy banking application: a spec that grows in four progressive levels against a black-box evaluator.
- 121Build an in-memory database: SET/GET/DELETE first, then filtered scans, then TTL with timestamps, then file compaction.
- 122Create a task scheduler.
- 123Build an OOP system for managing courses, grades and students.
- 124Given a helper method that crawls a URL, write a crawler over a domain: first synchronous, then make it async.
- 125Convert nested stack traces into discrete start and end events.
- 126Build a rate limiter. Every ten minutes I add a requirement: per-tenant limits, burst allowances, then a sliding window. How do you keep the code from collapsing?
- 127You need to run an LLM call over 50,000 documents. The API allows ~100 concurrent requests and occasionally returns 429s and timeouts. Write the Python.
- 128How would you parallelise this task? (Concurrency and data mutation come up repeatedly across rounds.)
- 129SQL: write a query to find the top five pairs of products most frequently purchased together.
- 130SQL: determine whether any user has overlapping subscription date ranges.
- 131SQL: return each employee's current salary after an ETL error inserted a new salary row every year.
LLM Internals and Architecture
- 132What are the key components of a Transformer model and why does each matter?
- 133Explain attention-free transformer architectures and their trade-offs.
- 134Walk me through matrix manipulations relevant to LLM architectures.
Inference, Serving and GPU Performance
- 135Design a batched inference system where 100 requests take the same time as 1.
- 136Design the serving stack for a Claude-scale LLM API. Maximise GPU utilisation without wrecking p99 latency.
Agents and Tool Use
- 137What matters more for an agentic coding tool like Claude Code: the model or the harness? Design the loop.
- 138Design the tool surface for a coding agent: which tools exist, what their schemas look like, and how results come back.
Fine-Tuning, Post-Training and Alignment
- 139Explain Constitutional AI. What does it buy you over vanilla RLHF, and what doesn't it solve?
- 140How do scaling laws influence the safety evaluation of large models?
AI System Design
- 141Design the Claude chat service.
- 142Design a system that enables a large language model to handle multiple questions in a single thread.
- 143Design a distributed search system for 1 billion documents at 1 million QPS.
- 144Design APIs for developers to access Anthropic's models securely and efficiently.
- 145Design a file-sharing / distribution system.
Evaluation and Observability
Safety, Security and Responsible AI
- 147Your agent reads inbound email and can send replies and search internal docs. Walk me through the prompt-injection attack surface and your defences.
- 148What do you see as the most pressing unsolved problem in AI alignment?
- 149How would you balance performance optimisation with model interpretability?
- 150How would you approach designing a system to ensure the safe deployment of AI models in production?
Applied and Forward-Deployed Scenarios
- 151An enterprise customer says “Claude hallucinates too much” in their RAG-based knowledge assistant. You're the applied engineer on the account. What happens in the first 48 hours?
- 152How would you make complex AI research findings accessible to a non-technical audience?
Behavioral and Culture
- 153Walk me through a project you owned end to end. What were the key technical decisions?
- 154Why Anthropic specifically, and where do you disagree with Anthropic?
- 155Tell me about a technical misjudgement that delayed a project.
- 156What are your thoughts on AI safety and the risks of advanced AI systems?
OpenAI
31 questionsRoles: Member of Technical Staff, Software Engineer, Machine Learning Engineer, Research Engineer, Research Scientist, Applied AI Engineer, Forward Deployed Engineer, Solutions Architect, Data Scientist.
Loop: Recruiter screen → technical screen (practical coding, often a build-something-real task rather than LeetCode) → onsite: two or three coding/practical rounds, a domain-depth or ML round, a system design round, and behavioural/mission-fit. Applied AI and FDE loops add a customer-scenario and solution-design round.
Coding and Data Structures
- 157Design and implement an in-memory key-value store supporting set, transactional begin, commit and abort.
- 158Create a database ORM, step by step.
- 159Code a trivial web crawler using Go.
- 160Implement a UI from a mockup with provided CSS and API.
- 161Refactor bad code: here are ~120 lines of working but messy code with passing tests. Improve the architecture without breaking them. What do you change first?
- 162Write a Python function that displays the first n Fibonacci numbers.
- 163Infection-spread simulation.
ML and DL Fundamentals
- 164Compute the KL divergence given different random variables.
- 165If the accuracy of a classifier is 1, what is the lower/upper bound on the loss function for a single training example?
- 166We have two models, 85% and 82% accuracy. Which do you pick?
- 167How do you handle missing data in Pandas?
LLM Internals and Architecture
- 168Explain self-attention. What is its computational complexity, and what are your options when contexts get long?
- 169What is the relationship between cross-entropy, KL divergence and perplexity, and why is cross-entropy the training loss for language models?
- 170What is the effect of adjusting an LLM's context window size?
Agents and Tool Use
AI System Design
- 172How would you build an LLM-powered enterprise search system?
- 173Design the serving stack for a ChatGPT-scale consumer assistant: hundreds of millions of weekly users, streaming chat, multiple model tiers.
- 174Design and build a webhook delivery system that reliably delivers events to customer-registered URLs.
- 175Design a system to schedule jobs in a distributed environment.
- 176Design an in-memory database. / Design Slack.
Evaluation and Observability
- 177A customer says “the model got worse” after you upgraded model versions in their deployment. How do you verify and respond?
- 178An enterprise customer reports that responses from your deployed system have gotten slow. Walk me through the diagnosis.
Safety, Security and Responsible AI
- 179How do you approach GenAI safety in consumer products?
- 180How would you design safeguards for an AI system that can take actions on behalf of a user?
Applied and Forward-Deployed Scenarios
- 181An enterprise customer says: “We want AI to automate our claims processing.” You're the engineer in the room. What do the first two weeks look like?
- 182Do you have experience working with APIs? Are you used to working with C-suite executives?
Behavioral and Culture
- 183What is your favourite product and why?
- 184Tell me about a time you made a mistake.
- 185Tell me about a time you had a conflict with someone. How did you resolve it and what did you learn?
- 186Tell me about a time you had conflicting priorities with stakeholders and how you secured alignment.
- 187What is the project you are most proud of?
Google DeepMind and Google AI
22 questionsRoles: Research Engineer, Research Scientist, Machine Learning Engineer, Software Engineer (ML), Forward Deployed Engineer, Applied AI Engineer (Google Cloud / Vertex AI), Data Scientist.
Loop: Recruiter screen → technical phone screen (coding, sometimes ML fundamentals) → onsite: two coding rounds, an ML domain/breadth round, an ML system design round, and Googleyness/leadership. DeepMind Research Engineer loops add a research deep-dive plus maths/probability and a from-scratch implementation round; hiring committee and team matching follow.
Coding and Data Structures
- 188You are receiving an unbounded stream of event IDs. Return the k most frequent IDs seen so far, at any point, with bounded memory.
- 189Closest key: given a dictionary with letter keys and lists of letters as values, find the closest key.
- 190Write a function to compute root-mean-square error given y_pred and y_true lists.
- 191Parse bigrams: extract two-word phrases from strings for NLP feature engineering.
ML and DL Fundamentals
- 192Define the bias-variance trade-off and discuss the relationship between the two.
- 193What are the assumptions of linear regression?
- 194Distinguish regularization from validation: when is each the right tool?
- 195Derive the gradient of cross-entropy loss with softmax inputs, and explain why we fuse them numerically.
- 196Explain the SVD and give two places it shows up in modern deep learning.
- 197On average, how many fair coin flips until you see two heads in a row? Walk me through it.
- 198When would you choose Q-learning over policy gradients, and vice versa?
- 199You have a binary loan-approval classifier and limited access to feature weights. How do you explain a rejection?
Fine-Tuning, Post-Training and Alignment
- 200Your pretraining loss suddenly diverges at step 300k of a long run. Diagnose and fix it.
- 201Design the training setup for a model that doesn't fit on one accelerator, say 70B parameters on a pod.
AI System Design
- 202Design the serving system for a multimodal assistant (text + image in, streaming text out) at hundreds of millions of users.
- 203Design a personalised recommendation system for rental listings using demographics, property metadata, amenities, price, reviews and location.
- 204Design a classifier that predicts the optimal moment to insert a commercial break in a video.
- 205How would you improve product search results, focusing on the fraction of relevant documents retrieved (recall)?
- 206Justify using a neural network for a given problem: what do you need to know about the network, dataset, timeline and business context?
Evaluation and Observability
- 207Build the evaluation harness for a new frontier model release. What does it need to do?
- 208Do 1 million Seattle ride trips suffice to build an accurate ETA prediction model? How would you decide?
Behavioral and Culture
Meta (Superintelligence Labs, FAIR, Llama)
26 questionsRoles: Machine Learning Engineer (E4–E7), Research Engineer, Research Scientist, AI Infrastructure Engineer, Software Engineer (ML), Applied Research Scientist.
Loop: Recruiter screen → technical screen (2 coding problems in 45 min) → onsite: two coding rounds, one ML system design round, one ML domain/breadth round, and a behavioural (“Jedi”) round. Some 2026 loops now include an AI-assisted coding round in three stages: explore and fix issues, implement new functionality, extend and improve the system.
Coding and Data Structures
- 210Given an array nums of n integers where n > 1, return an output array (product of array except self).
- 211Find the minimum window in S which will contain all the characters in T.
- 212Serialize and deserialize a binary tree.
- 213Convert a binary tree to a circular doubly linked list.
- 214Alien dictionary: determine character ordering from a sorted word list.
- 215K closest points to origin; top-k frequent elements; minimum number of conference rooms.
- 216Regular expression matching with '.' and '\*'.
- 217Two-part warm-up: given a stream of user actions, return the k most engaged-with items. Then: why might your heap solution be the wrong choice in production?
ML and DL Fundamentals
LLM Internals and Architecture
- 219Explain the architectural choices in a Llama-class model: why grouped-query attention, RoPE and SwiGLU instead of the vanilla 2017 Transformer?
- 220What breaks when you scale LLM training from 8 GPUs to thousands, and how do modern stacks deal with it?
Inference, Serving and GPU Performance
Agents and Tool Use
Fine-Tuning, Post-Training and Alignment
AI System Design
- 224Design the recommendation system for Instagram Reels.
- 225Design a personalised news-feed ranking system / the “next post” logic for Facebook's feed.
- 226Design a recommendation system for Facebook Ads, and an evaluation framework for ads ranking.
- 227Design the ML components behind an Instagram Story feature.
- 228Design an end-to-end classification pipeline for Marketplace listings.
- 229Design a language translation model / service.
Evaluation and Observability
Safety, Security and Responsible AI
Multimodal, Speech and Voice AI
Behavioral and Culture
xAI
10 questionsRoles: Member of Technical Staff, AI Engineer, Infrastructure Engineer, Research Engineer, Product Engineer (Grok), Data / RL environments engineer.
Loop: Fast, low-process loops. Typically a recruiter or hiring-manager screen → one or two live coding rounds heavy on data structures and systems → a practical build-something round (often a timed four-hour product build or a stubbed-out module in a real codebase) → founder/leadership conversation. Speed and raw shipping ability are weighted heavily.
Coding and Data Structures
- 236Build an in-memory key-value store with SET/GET/DELETE, then add transactions with BEGIN/COMMIT/ROLLBACK, including nested transactions.
- 237Write an iterator class that lazily flattens an arbitrarily nested list of lists/integers: no generators, explicit state.
- 238Here is a scheduler class from a small LLM inference engine. One method, \_admit_requests, is a stub: no spec, no docstring, no tests. Walk me through your first thirty minutes.
Inference, Serving and GPU Performance
- 239Estimate the KV-cache memory to serve a 70B-class model at 128k context. What do you do when it doesn't fit?
- 240Design a rate limiter for an LLM API where cost scales with tokens, not requests.
Fine-Tuning, Post-Training and Alignment
- 241You're training on tens of thousands of GPUs and hardware fails constantly. How do you keep goodput high?
- 242Loss spikes mid-run on a large pretraining job. Walk me through your debugging process.
- 243Design a deduplication pipeline for a web-scale pretraining corpus. It has to run as a streaming process.
AI System Design
- 244Design the serving stack for a consumer chatbot with real-time search over a social-media firehose.
Behavioral and Culture
Mistral AI
8 questionsRoles: Research Engineer, ML Engineer, Applied AI Engineer, Solutions Architect / Forward Deployed Engineer, Inference Engineer, Platform Engineer.
Loop: Recruiter screen → technical screen (Python + ML fundamentals) → pair-programming round building a small LLM-backed service → deep-dive on transformer/serving internals → customer-scenario round for applied roles → culture/founders round. European enterprise and on-prem deployment context shows up throughout.
Coding and Data Structures
LLM Internals and Architecture
- 247Mistral 7B shipped with grouped-query attention and sliding-window attention. What does each buy you, and what does each cost?
- 248Explain how a Mixtral-style sparse mixture-of-experts model works. Why does a ~47B-parameter model run at roughly the cost of a ~13B one?
Inference, Serving and GPU Performance
- 249Estimate the KV-cache memory for serving Mistral 7B, and design the rolling-buffer cache that sliding-window attention enables.
- 250You need to quantize a model for a customer's hardware. How do you choose a scheme, and how do you prove quality hasn't regressed?
Agents and Tool Use
Fine-Tuning, Post-Training and Alignment
AI System Design
Cohere
10 questionsRoles: Member of Technical Staff, ML Engineer, Applied AI Engineer, Solutions Architect / Forward Deployed Engineer, Platform & Inference Engineer.
Loop: Recruiter screen → technical screen (practical Python, streaming/API-shaped problems) → onsite: coding, retrieval/RAG depth, enterprise deployment design, customer-scenario round, and a values round. Remote-first; autonomy and ownership are explicitly tested.
Coding and Data Structures
LLM Internals and Architecture
RAG and Retrieval
- 256You have an embedding model and a reranker. Why sell both? Design the two-stage retrieval pipeline and tell me when the reranker earns its latency.
- 257An enterprise wants semantic search over ~100M documents but is balking at vector-index cost. Walk me through embedding compression options and the maths.
- 258How would you evaluate multilingual retrieval quality when employees query in French and Korean over mostly-English documents?
- 259A customer 10x'd their indexed documents and reports answer quality “got noticeably worse.” Drive the investigation.
Agents and Tool Use
AI System Design
Evaluation and Observability
Behavioral and Culture
DeepSeek
10 questionsRoles: Research Engineer, Infrastructure / Systems Engineer, Inference Engineer, Data Engineer, Algorithm Engineer.
Loop: Heavily research- and systems-weighted: paper deep-dive, from-scratch implementation rounds in PyTorch, distributed-training and low-precision depth, plus an algorithmic coding round. Expect direct questions about their published architecture and training papers.
LLM Internals and Architecture
- 264Walk me through DeepSeekMoE. How is it different from a standard top-2 MoE like Mixtral?
- 265DeepSeek-V3 uses auxiliary-loss-free load balancing. What was wrong with the auxiliary loss, and how does the bias trick work?
- 266What is multi-token prediction (MTP) and why train with it?
- 267Implement top-k MoE routing with a shared expert in PyTorch, and point out the efficiency and correctness traps.
Inference, Serving and GPU Performance
- 268Sketch how you would serve a 671B-parameter MoE model with low latency under GPU-memory constraints.
Fine-Tuning, Post-Training and Alignment
- 269R1-Zero was trained with RL and essentially no SFT first. What did that show, and why did full R1 add SFT back?
- 270FP8 training at 671B scale is hard. What actually breaks in low precision, and how do you make it stable?
- 271How do you build a training dataset without triggering model collapse when much of your data is synthetic?
- 272DualPipe overlaps computation and communication in training. Why is that overlap the whole game at this scale, and what is the trade-off?
Behavioral and Culture
Moonshot AI (Kimi)
10 questionsRoles: Research Engineer, Infrastructure Engineer, Inference / Serving Engineer, Agent Engineer.
Loop: Research and long-context systems focused: architecture deep-dive, distributed serving design, a PyTorch implementation round, and an agentic-evaluation discussion.
LLM Internals and Architecture
- 274Kimi's headline feature is very long context. When you push from 8K to hundreds of thousands of tokens, what actually breaks first, and why?
- 275Kimi K2 uses Multi-head Latent Attention (MLA). Explain what it does and how it compares to GQA for KV-cache reduction.
- 276Kimi K2 is a 1T-parameter MoE with ~32B active per token and hundreds of experts. Explain the routing and the systems cost of training it.
- 277How do you take a model trained at 8K–32K and make it work at 128K or more?
Inference, Serving and GPU Performance
- 278Walk me through why you would disaggregate prefill and decode onto separate machines, as Mooncake does. What does that buy you and what does it cost?
- 279A chat assistant re-sends a long conversation history on every turn. How do you avoid recomputing all of it, and what are the pitfalls?
RAG and Retrieval
Fine-Tuning, Post-Training and Alignment
- 281Training a trillion-parameter model, attention logits can blow up and destabilise the run. What is going on, and how does something like MuonClip address it?
- 282Kimi K1.5 scaled RL for reasoning without a process reward model or tree search. Why deliberately keep the RL recipe that simple?
Evaluation and Observability
Zhipu AI (GLM)
11 questionsRoles: Research Engineer, Agent Engineer, RL Infrastructure Engineer, Applied AI Engineer.
Loop: Architecture and post-training depth, an RL-infrastructure design round, a GUI-agent design round, and an implementation round in PyTorch.
LLM Internals and Architecture
- 284GLM's original pre-training objective is autoregressive blank infilling. How does it differ from BERT and GPT, and why did the team argue it unifies understanding and generation?
- 285GLM-4.5 is an MoE with 355B total but 32B active parameters. Explain the economics: what does that split buy you and what does it cost?
- 286Implement a top-k MoE router in PyTorch. Then contrast auxiliary-loss load balancing with a loss-free approach.
- 287What is Multi-Token Prediction (MTP), why add an MTP layer, and how does it help at inference time?
- 288GLM has been bilingual Chinese/English since GLM-130B. What changes in tokenization, data and evaluation when a model must serve both languages well?
Agents and Tool Use
Fine-Tuning, Post-Training and Alignment
- 290GLM-4.5 is a hybrid reasoning model with a thinking mode and a direct-response mode. How do you build one model that does both, and what are the training and serving implications?
- 291Why does long-horizon agentic RL need a disaggregated, asynchronous design (as in the slime framework) rather than colocated-synchronous?
- 292GLM-4.5's post-training trains expert models per domain then unifies with self-distillation. Walk through why you would train specialists and then merge them.
AI System Design
Evaluation and Observability
Alibaba (Qwen)
10 questionsRoles: Algorithm Engineer (LLM), Research Engineer, Inference Engineer, Multimodal Engineer, Applied AI Engineer (Alibaba Cloud / Model Studio).
Loop: Classic Alibaba structure: two or three technical rounds (algorithms + ML depth), a cross-examination round with a senior manager, and an HR round, with Qwen-specific architecture and multilingual questions layered on top.
Coding and Data Structures
LLM Internals and Architecture
- 296Qwen uses byte-level BPE with a ~151K vocabulary, augmented for multilingual coverage and with digits split into single characters. Why those choices, and what are the trade-offs?
- 297Qwen3 unifies a thinking mode and a non-thinking mode in one model with a caller-settable thinking budget. How would you train that, and how would you serve it?
- 298Qwen ships both dense and MoE models (30B with ~3B active; 235B with ~22B active). When would you pick the 30B-A3B MoE over a 32B dense?
- 299Qwen2.5 extends context to 128K (and ~1M for Turbo) using YaRN plus Dual Chunk Attention, mostly training-free. Explain how, and why post-hoc extension is attractive.
Fine-Tuning, Post-Training and Alignment
- 300Qwen3 uses strong-to-weak distillation, bootstrapping smaller models from flagship ones. How does that work and why is it cheaper?
- 301Qwen's reasoning models train with RL using verifiable rewards on maths and code. Why is that preferred over PPO with a learned reward model for these domains?
Evaluation and Observability
Multimodal, Speech and Voice AI
Behavioral and Culture
Sarvam AI
12 questionsRoles: Research Engineer (LLM / speech), ML Engineer, Applied AI / Forward Deployed Engineer, Speech Engineer, Edge / Inference Engineer.
Loop: Research and applied rounds side by side: Indic NLP and tokenizer depth, a speech/ASR round, an implementation round, a deployment-on-constrained-hardware round, and a government/enterprise deployment scenario.
Coding and Data Structures
LLM Internals and Architecture
Inference, Serving and GPU Performance
RAG and Retrieval
Fine-Tuning, Post-Training and Alignment
- 309Sarvam-M ships hybrid think/non-think modes and was post-trained with SFT then RLVR. How would you build that, and why RLVR over vanilla RLHF?
- 310A regional government wants an assistant in a low-resource language with only a few thousand sentences of clean text. How do you adapt a model to it?
AI System Design
Evaluation and Observability
Multimodal, Speech and Voice AI
- 313Build a Voice Activity Detector from scratch. How do you make it robust for phone-quality Indian-language audio?
- 314Whisper transcribes Hinglish poorly, often forcing output into one language or hallucinating. Why, and how would you build an ASR that handles code-mixed speech?
- 315Bulbul-style TTS has to speak code-mixed, mixed-script text naturally. What are the hard parts of text normalization and prosody for Indian-language TTS?
Applied and Forward-Deployed Scenarios
Big Tech AI Organizations
6 sections · 82 questions
Microsoft
10 questionsRoles: AI Engineer, Applied Scientist, Machine Learning Engineer, Software Engineer (AI Platform / Copilot), Azure AI Solutions Architect, Principal Applied AI Engineer.
Loop: Recruiter screen → technical phone screen (coding + a little ML) → onsite loop of 4–5 rounds: two coding, one ML/AI depth, one AI system design or low-level design, and an as-appropriate-hire round with a senior leader. Azure AI and Copilot roles add a customer-architecture round.
Coding and Data Structures
- 317Implement “top-k most frequent search queries” over a large query log, then tell me what breaks when the log becomes an unbounded stream across many machines.
- 318Low-level design: sketch the classes and interfaces for the tool-calling layer of an agent host, where tools can come from native code, an OpenAPI spec, or an MCP server.
Inference, Serving and GPU Performance
- 319A Copilot chat feature has a p95 budget of 3 seconds to first useful content. Where does the time go, and how do you cut it?
- 320Estimate the annual serving cost of adding an LLM summary feature for 100 million weekly active users, and how you'd cut it by 10x.
AI System Design
- 321Design a Copilot feature that answers questions over a user's work email, documents and meetings, without ever leaking content the user can't access.
- 322Design an agent that can take actions in a spreadsheet (“insert a pivot table of Q3 sales by region”): orchestration, tools and failure handling.
Evaluation and Observability
Safety, Security and Responsible AI
- 324Your Copilot summarises incoming email. An attacker emails a target user with hidden instructions addressed to the model. Walk me through the attack and your defence.
- 325A shipped Copilot feature that summarises job applicants for recruiters is accused of working worse for some groups. How do you establish whether that's true, and what do you do about it?
Behavioral and Culture
Amazon (AWS)
23 questionsRoles: Applied Scientist (I/II/III), Machine Learning Engineer, Data Scientist, Software Development Engineer (AI/ML), GenAI Specialist Solutions Architect, Applied AI Engineer (Bedrock, Q, SageMaker).
Loop: Online assessment or phone screen → onsite “loop” of 4–6 rounds, each anchored on Leadership Principles: two coding, one ML breadth, one ML depth / research deep-dive or ML system design, one bar-raiser. Applied Scientist loops add a research presentation.
Coding and Data Structures
- 327Find the top-K most frequent items in a high-volume event stream with bounded memory.
- 328Divide two integers without using multiplication, division or modulo. Find the number of connected components in a graph. Check balanced parentheses.
ML and DL Fundamentals
- 329What is the closed-form solution of linear regression, and when do you use gradient descent instead?
- 330What are the differences between L1 and L2 regularization in logistic regression?
- 331Write the loss function for logistic regression and prove it has a global minimum.
- 332How is KL divergence loss different from cross-entropy loss? And from contrastive loss?
- 333How do bagging and boosting differ? What is the computational difference between XGBoost and Random Forest?
- 334Explain the bias-variance trade-off, cross-validation, and the curse of dimensionality.
- 335How do GRU cells work, and how do they address the vanishing gradient problem? How does a BiLSTM work?
- 336What is Attention in machine learning models? What happens in a neural network if you remove all the hidden layers?
- 337Discuss precision, recall and F1: when would you prioritise one over the others?
- 338How do you handle data imbalance, collinearity, feature selection and regularization?
- 339Explain how you would design and evaluate an A/B test. What is a p-value and how do you interpret it here?
- 340What is Maximum Likelihood Estimation and how does it differ from Bayesian inference?
LLM Internals and Architecture
Inference, Serving and GPU Performance
Agents and Tool Use
AI System Design
- 344Design a multi-tenant inference platform that serves many foundation models to thousands of customers (Bedrock-shaped).
- 345How would you design a recommendation system to suggest books to users? How would you model a warehouse inventory problem?
Evaluation and Observability
Behavioral and Culture
- 347Tell me about a time you disagreed with your team's technical direction. What did you do? (Have Backbone; Disagree and Commit)
- 348Tell me about your most significant failure. What happened, and what did you change afterward?
- 349Tell me about a time you saw an opportunity to do something bigger than the initial scope. (Think Big)
Apple
11 questionsRoles: Machine Learning Engineer, AI/ML Research Engineer, On-device ML Engineer, Software Engineer (Apple Intelligence / Siri), Applied Scientist.
Loop: Recruiter screen → hiring-manager technical call → onsite of 4–6 rounds with the specific team: coding, ML depth, on-device/efficiency depth, system design, and behavioural. Secrecy means you may be interviewed for work you cannot be told about.
Inference, Serving and GPU Performance
- 350You need to run a ~3B-parameter language model on a phone with tight memory and power budgets. What changes versus serving the same model in a datacenter?
- 351Explain post-training quantization versus quantization-aware training. What breaks when you push weights to 2–4 bits, and how do you recover quality?
- 352Estimate the KV-cache memory for a 3B on-device model at 4k context, and name the levers that shrink it.
- 353Time-to-first-token for your on-device feature is 1.8 s. Walk me through diagnosing and fixing it.
Agents and Tool Use
Fine-Tuning, Post-Training and Alignment
- 355You have one on-device base model but a dozen features: summarization, rewriting, reply suggestions, tone adjustment. How do you specialise without shipping a dozen models?
- 356How would you improve an on-device model using signals from user devices without collecting user content?
AI System Design
- 357Design the routing layer that decides whether a user request is handled on-device, by a first-party server model, or by a third-party model.
- 358A user says “send Maya the photos from Saturday's hike.” Design the on-device path from that utterance to a structured app action with resolved parameters.
Evaluation and Observability
Behavioral and Culture
NVIDIA
13 questionsRoles: Deep Learning Software Engineer (Inference / LLM Performance), CUDA Kernel Engineer, Machine Learning Engineer, Solutions Architect, Applied Scientist, TensorRT-LLM / Dynamo engineer.
Loop: Recruiter screen → hiring-manager technical call → onsite of 4–6 rounds: CUDA/C++ or Python coding, GPU performance and roofline reasoning, LLM inference depth, ML fundamentals, and system/solution design. Solutions Architect loops replace kernel depth with customer-scenario rounds.
Coding and Data Structures
- 361Here's a CUDA kernel that's 10x slower than expected. Without running it, what are the usual suspects, and how do you confirm each?
- 362Implement the block manager for a paged KV cache: allocate, append, free, and copy-on-write prefix sharing.
- 363A model runs fine in FP32 but produces garbage after conversion to FP16. Debug it.
ML and DL Fundamentals
- 364Explain strategies to combat overfitting in tree-based classification models.
- 365Summarize the differences and benefits of the Adam optimizer compared with other methods for neural-network image classification.
- 366A network confuses pugs and pit bulls and some training labels are wrong. How do you modify the model and the data?
- 367How do you evaluate a clustering model's effectiveness without pre-labelled groups?
Inference, Serving and GPU Performance
- 368You want to serve a 70B-parameter model on a single 80 GB GPU. Walk me through whether it fits and what single-stream tokens/sec you'd expect.
- 369What does TensorRT / TensorRT-LLM actually do to a model to make it faster, and when will it not help?
- 370Design the parallelism strategy for serving a 405B-parameter dense model. TP, PP, EP: what goes where and why?
AI System Design
Applied and Forward-Deployed Scenarios
Behavioral and Culture
Tesla
11 questionsRoles: AI / ML Engineer (Autopilot, Optimus), Deep Learning Engineer, Computer Vision Engineer, Data Engineer (Autopilot), Inference / Silicon software engineer.
Loop: Recruiter screen → hiring-manager technical call → onsite: coding (often C++/Python), deep-learning depth focused on vision and training pipelines, a data/infra round, and a hands-on debugging round. Loops are fast and heavy on practical engineering.
Coding and Data Structures
- 374Implement non-maximum suppression. Then vectorise it.
- 375Write an efficient ring buffer for high-rate sensor data with a fixed memory budget.
ML and DL Fundamentals
- 376How would you design the neural network architecture for multi-camera 3D object detection without lidar?
- 377How do you handle extreme class imbalance in rare-event detection (e.g. a child running into the road)?
- 378Explain how you would auto-label a fleet dataset and what quality controls you would put on it.
- 379How would you detect and handle distribution shift between fleet data and your training set?
Inference, Serving and GPU Performance
AI System Design
Evaluation and Observability
Multimodal, Speech and Voice AI
Behavioral and Culture
Consumer-Scale ML Companies (Uber, Netflix, LinkedIn, Airbnb, Pinterest, Spotify)
14 questionsRoles: Machine Learning Engineer, Senior/Staff MLE, Applied Scientist, ML Platform Engineer, GenAI Engineer.
Loop: A consistent shape across all of them: coding screen → onsite with two coding rounds, one ML system design round (the differentiator), one ML breadth/depth round, and behavioural. GenAI rounds have been added to most of these loops since 2024.
Coding and Data Structures
- 385Implement a streaming top-k with a bounded-memory sketch; implement a sliding-window rate counter.
ML and DL Fundamentals
- 386Your offline metric improved but the online A/B did not. Enumerate the reasons this happens and how you would tell them apart.
- 387Explain position bias in ranking data and how you would debias training.
- 388How do you design a feature store, and what causes training/serving skew?
AI System Design
- 389Design the ETA prediction system for a ride-hailing marketplace. What features, what model, how do you serve it in <100 ms?
- 390Design a personalised feed ranking system with a two-stage candidate generation and ranking architecture.
- 391Design a content recommendation system for a streaming catalogue, including cold-start for new titles and new users.
- 392Design “people you may know” / job-recommendation ranking at a professional network's scale.
- 393Design dynamic pricing / surge for a two-sided marketplace and describe the feedback loops that can go wrong.
- 394Design a visual search system: user uploads an image, you return visually similar in-catalogue items.
- 395Design a fraud-detection system with heavy class imbalance and an adversarial opponent.
- 396Design an LLM-powered customer-support assistant on top of an existing help centre, with escalation to humans.
Evaluation and Observability
Behavioral and Culture
AI Infrastructure and Platform Companies
6 sections · 66 questions
Databricks
9 questionsRoles: Software Engineer (ML Platform / Mosaic AI), Machine Learning Engineer, GenAI Solutions Architect, Forward Deployed / Delivery Solutions Architect, Applied AI Engineer.
Loop: Recruiter screen → technical screen (practical coding, often concurrency or data-heavy) → onsite: two coding rounds, a distributed-systems or Spark-internals round, a GenAI/ML design round, and a customer-scenario round for field roles.
Coding and Data Structures
- 399Implement a thread-safe batching logger: many producer threads call log(msg); a background thread flushes batches of up to 100 messages every second or when full.
- 400You have a stream of billions of events and need the top-K most frequent keys with bounded memory. Exact is impossible: what do you do?
- 401Given allowed IP ranges as CIDR blocks plus explicit deny ranges, implement is_allowed(ip) efficiently for millions of checks per second.
- 402A Spark job joining a 2 TB fact table to a 50 GB dimension table has one straggler task running 100x longer than the rest. Diagnose and fix it.
- 403A Structured Streaming job reads Kafka and writes to a Delta table. The cluster is killed mid-batch and restarts. Does the customer get duplicate rows? Explain at the level of the checkpoint and the transaction log.
Fine-Tuning, Post-Training and Alignment
Evaluation and Observability
Applied and Forward-Deployed Scenarios
- 406A customer insists on fine-tuning an open model on their support tickets because “we want our own model.” You think RAG solves it. What do you do?
- 407An agent you shipped four months ago runs on a base model being deprecated in 60 days. How do you swap the model without regressing quality, and what had to be in place beforehand?
Groq
12 questionsRoles: Compiler Engineer, Runtime / Systems Engineer, Inference Engineer, Silicon Software Engineer, Solutions Architect.
Loop: Deep systems and compiler rounds: roofline and memory-hierarchy reasoning for an SRAM-only machine, compiler IR design, host-runtime design, a debugging round, and a customer/unit-economics round.
Coding and Data Structures
- 408Our compiler statically schedules every instruction and every chip-to-chip transfer. What does that compiler need to know that an NVCC-style compiler does not, and what breaks when it's wrong?
- 409Design the IR and pass pipeline for a compiler targeting a spatial dataflow accelerator. Where does the memory-residency decision live, and why?
- 410Write the host-side runtime that feeds a deterministic accelerator across many chips. What is genuinely hard about it?
- 411A model passes bit-exact against the functional simulator on one chip but produces wrong output at rack scale. How do you find it?
Inference, Serving and GPU Performance
- 412An LPU has no HBM at all, just on-die SRAM. Redo the decode roofline argument for that machine and tell me what changes.
- 413A 70B dense model at 8-bit weights, chips with ~230 MB of SRAM each. Walk me through the deployment and the unit economics.
- 414On a GPU you batch to amortise weight reads. What is the batching calculus on an SRAM-only machine, and how should that change how we price?
- 415Determinism is the headline claim. What does it actually buy at p99, and why does it matter especially for agentic workloads?
- 416How would you serve a large mixture-of-experts model on a statically scheduled fabric when expert selection is data-dependent?
AI System Design
Applied and Forward-Deployed Scenarios
Behavioral and Culture
Together AI
7 questionsRoles: Inference Engineer, Kernel Engineer, ML Systems Engineer, Solutions / Forward Deployed Engineer, Platform Engineer.
Loop: Inference-performance depth, a scheduler/serving design round, a practical streaming-server coding round, a distributed-training debugging round, and a customer-advisory round for field roles.
Coding and Data Structures
- 420Write the server-side handler for streaming token generation. Handle client disconnects correctly.
Inference, Serving and GPU Performance
- 421Design the scheduler for a continuous-batching inference engine.
- 422Explain speculative decoding. When does it help, when does it hurt, and why adapt the speculator to live traffic?
- 423Price a dedicated endpoint: estimate cost per million output tokens for a 70B model, and explain the throughput-latency trade.
Fine-Tuning, Post-Training and Alignment
AI System Design
Applied and Forward-Deployed Scenarios
Hugging Face
11 questionsRoles: ML Engineer (open-source maintainer), Research Engineer, Infrastructure Engineer, Developer Advocate Engineer, Inference Engineer.
Loop: Open-source-flavoured: a library-internals deep dive, a maintainer/code-review round, a practical fine-tuning or memory-budget round, a Hub/systems design round, and a culture round. Your public GitHub history is genuinely part of the evaluation.
Coding and Data Structures
- 427`transformers` famously repeats code: each model gets its own self-contained modeling file. Defend that decision, then critique it.
- 428Why did Hugging Face create safetensors when pickle-based checkpoints already worked everywhere?
- 429A user loads a 2 TB dataset with `datasets` on a 64 GB RAM machine and it works. How? And when does it stop working?
LLM Internals and Architecture
- 430Walk me through what actually happens when someone calls AutoModelForCausalLM.from_pretrained(…, device_map=“auto”, torch_dtype=“auto”).
- 431Compare BPE, WordPiece and Unigram tokenization. Why is `tokenizers` written in Rust, and what tokenizer bugs bite people in practice?
- 432What problem do chat templates solve, and what goes wrong when they're ignored?
Fine-Tuning, Post-Training and Alignment
- 433Fine-tune an 8B model on a single 24 GB GPU. Walk me through the memory maths and the exact stack you'd use.
- 434You're building a web-scale pretraining corpus (FineWeb-style). Walk me through the pipeline and how you decide whether each filter earns its place.
AI System Design
- 435Design the Hugging Face Hub: millions of git repos where individual files are tens to hundreds of GB.
- 436Design the serverless inference layer: any of thousands of Hub models can receive a request at any moment.
Behavioral and Culture
Scale AI
12 questionsRoles: Software Engineer, Machine Learning Engineer, Research Engineer (SEAL evals), Forward Deployed Engineer, Product Engineer (data engine, RL environments).
Loop: Recruiter screen → technical screen (practical coding) → onsite: coding with progressive requirements, a data-quality/annotation design round, an evaluation-design round, and a customer-scenario round for FDE.
Coding and Data Structures
- 438Build the task-lifecycle core of an annotation platform. Start simple; I'll add consensus of k annotators, then priority re-review, then annotator cooldowns.
- 439Given annotation sessions as (start, end) timestamps, return the peak number of concurrent annotators and the intervals at peak load.
Fine-Tuning, Post-Training and Alignment
- 440Compare SFT, RLHF, DPO and RLVR for improving an instruction-tuned model. What data does each need, and when would you pick which?
- 441We sell RL environments. Design one for “book a multi-city trip in a web travel app”, specify the reward, and tell me how you stop the policy hacking it.
AI System Design
- 442Design an end-to-end pipeline producing RLHF preference data for a frontier lab: 100k prompt-response comparisons a week, with quality guarantees.
- 443Design a private LLM benchmark and leaderboard (SEAL-style). How do you keep it trustworthy as labs optimise against it?
Evaluation and Observability
- 444Your annotators have no ground truth: the tasks are subjective preference judgments. How do you measure and improve label quality?
- 445How would you benchmark an LLM agent's tool use, say, for enterprise workflows composing 10+ APIs?
- 446An eval pipeline you own suddenly reports a 6-point drop for a customer's model between Tuesday and Wednesday. The model didn't change. Debug it.
Safety, Security and Responsible AI
Applied and Forward-Deployed Scenarios
- 448An enterprise wants a document-Q&A assistant over 2M internal documents, pilot in four weeks, and their security team forbids data leaving their VPC. Scope and design it.
- 449A robotics customer asks for 50,000 hours of manipulation demonstrations across 12 tasks and three embodiments. Design the collection pipeline, and tell me what makes one demonstration worth keeping.
Perplexity
15 questionsRoles: Software Engineer (search / infra), AI Engineer, Research Scientist, ML Engineer (ranking & retrieval), Product Engineer (Comet).
Loop: Recruiter screen → technical screen → onsite: a system design round on retrieval/ranking at web scale, an ML/search deep-dive, a practical coding round, a product/craft round, and behavioural. Product taste about their own app is explicitly evaluated.
Coding and Data Structures
- 450Implement a client pool over multiple LLM providers with failover: providers fail, time out, or rate-limit, and callers should just get a completion.
- 451You're ingesting millions of web pages a day. Detect near-duplicates (same article, different boilerplate) efficiently.
- 452You need to embed millions of text chunks. The embedding service takes batches with a max batch size and a max total-token limit. Write the batcher and make it fast.
- 453Implement beam search for an autoregressive model. When would an answer engine actually use it?
Inference, Serving and GPU Performance
RAG and Retrieval
- 455Discuss reranker architecture choices: cross-encoder, ColBERT, LLM-based.
- 456You retrieved 50 candidate passages but the model's useful context budget is ~10. How do you choose, and how do you know your choices are good?
AI System Design
- 457Design an answer engine: a user types a question and gets a cited, streamed answer. Your end-to-end budget is 3 seconds to a complete short answer.
- 458Design the retrieval pipeline pulling from 100B web pages with sub-second latency and freshness guarantees.
- 459Design the ranking system combining BM25, dense retrieval and LLM reranking across multiple indexes.
- 460Design Comet's hybrid browser architecture combining on-device privacy with cloud AI assistance.
- 461How does an answer engine handle breaking news: a query about something that happened 20 minutes ago?
Evaluation and Observability
Safety, Security and Responsible AI
Behavioral and Culture
AI-Native Product Companies
10 sections · 114 questions
Cursor (Anysphere)
17 questionsRoles: Software Engineer (product, infra, model serving), ML Engineer, Research Engineer, Infrastructure Engineer.
Loop: Recruiter/hiring-manager screen (~45 min) → one to three 60-minute technical screens where you build data structures against Cursor's actual codebase (AI-tool access varies by round) → a two-day in-person project onsite (or an ~8-hour remote version) where you design and ship a feature on real Cursor code. Scoping, autonomy and effective AI-tool use are explicitly graded.
Coding and Data Structures
- 465Build a hash tree to organise data in a repository.
- 466Given a repository snapshot (path → content), build a Merkle tree and write the function returning which files changed between two snapshots without comparing every file's content.
- 467Print the top view of nodes in a binary tree.
- 468Find duplicate files in a file system.
- 469Implement the core of an editor text buffer: efficient insert/delete at arbitrary positions and fast line lookup. What structure do you pick?
Inference, Serving and GPU Performance
RAG and Retrieval
Agents and Tool Use
- 472Design the harness for an agent that makes multi-file changes from a natural-language task. How do you keep it from wrecking a codebase?
- 473Design an agentic AI system that can autonomously adapt to new tasks.
AI System Design
- 474Design Cursor's tab (next-edit prediction) system: it must feel instant (sub-100 ms perceived latency) for millions of daily users.
- 475How would you index a 100k-file monorepo so an AI editor can retrieve relevant context, and keep the index fresh as the user edits?
- 476The model is streaming a multi-file edit while the user keeps typing in one of those files. How do you apply the edits without corrupting the buffer?
- 477Your agent model outputs an edited version of a 500-line file. Applying it verbatim is slow and error-prone. How do you make “apply” fast and reliable?
- 478An agent needs to iterate on code (run builds, tests, lints) without disturbing what the user sees in their editor. Architect that.
Evaluation and Observability
Behavioral and Culture
Cognition (Devin, Windsurf)
9 questionsRoles: Member of Technical Staff, Agent Engineer, Infrastructure Engineer, Deployed Engineer, Research Engineer (agent RL).
Loop: Fast loop weighted toward building: a practical agent-building or debugging round, an infrastructure design round, an agent-evaluation round, and a deployed-engineering scenario for customer-facing roles.
Agents and Tool Use
- 482You have eight hours to build a coding agent from scratch. Describe what you build and, more importantly, what you cut.
- 483Cognition published an argument against multi-agent systems and later published what actually works. Reconcile those two positions.
- 484Your agent spends over half its first turn just finding the relevant code. How do you fix that?
Fine-Tuning, Post-Training and Alignment
AI System Design
- 486Design the execution environment for thousands of concurrent cloud coding agents. It must survive the agent waiting forty minutes for CI.
- 487Devin runs asynchronously in the cloud; Windsurf's Cascade runs in the editor next to the user. What actually changes between those two products, technically?
Evaluation and Observability
Safety, Security and Responsible AI
Applied and Forward-Deployed Scenarios
Sierra
12 questionsRoles: Agent Engineer, Software Engineer, Forward Deployed Engineer, ML Engineer, Product Engineer.
Loop: Recruiter screen → technical screen → a two-hour build session where you can use any AI tools you like → agent-design and evaluation rounds → customer-scenario round → founders/values. Judgment about where to put guardrails is the core signal.
Coding and Data Structures
RAG and Retrieval
Agents and Tool Use
- 493Design a customer-facing agent for an airline that can cancel and rebook flights. How do you keep it from violating fare policy?
- 494LLMs are non-deterministic, but a refund over $200 must never be auto-approved. Where's the line between prompting and code?
- 495Design the human-handoff path for a customer-service agent. When should it escalate, and what does a good handoff look like?
Evaluation and Observability
- 496The space of possible conversations is effectively infinite. How do you evaluate a conversational agent before launch?
- 497Your agent passes 92% of eval tasks. Why might that number be misleading, and what would you measure instead?
- 498After a foundation-model version upgrade, your production agent's escalation rate doubles overnight. Walk me through your response.
Safety, Security and Responsible AI
Multimodal, Speech and Voice AI
Behavioral and Culture
Harvey
11 questionsRoles: Software Engineer, ML / Applied AI Engineer, Forward Deployed Engineer, Research Engineer (legal domain).
Loop: Recruiter screen → paired coding round → an architecture presentation round on a legal-workflow system → an evaluation-design round → a customer/partner-scenario round. Domain rigour and grounding matter more than algorithmic puzzles.
Coding and Data Structures
RAG and Retrieval
- 504A lawyer asks about a 200-page credit agreement where the operative clause on page 140 depends on a defined term on page 8. How do you build retrieval that gets this right?
- 505When would you put a whole contract in the context window instead of retrieving over it? Defend the answer with numbers.
Agents and Tool Use
AI System Design
Evaluation and Observability
Safety, Security and Responsible AI
- 509Every assertion in a Harvey answer needs to link back to a specific passage. Design the grounding system, and tell me how you would measure the unsupported-claim rate.
- 510An agentic research query returns a memo citing a case that was overruled. Where does that get caught?
- 511Two partners at the same firm are on opposite sides of a deal. Design the data isolation for that, on top of normal multi-tenancy.
Applied and Forward-Deployed Scenarios
Glean
10 questionsRoles: Software Engineer (search / ranking), ML Engineer, AI Engineer (agents), Forward Deployed Engineer, Infrastructure Engineer.
Loop: Recruiter screen → coding screen → onsite: a retrieval/ranking design round, a permissions-and-connectors systems round, a practical coding round, an evaluation round, and behavioural.
Coding and Data Structures
Inference, Serving and GPU Performance
RAG and Retrieval
- 516How would you chunk and embed heterogeneous enterprise content: Slack threads, Jira tickets, Google Docs, PDFs?
- 517Why is RAG the right architecture for an enterprise assistant instead of fine-tuning on the company's data? Where does RAG break?
Agents and Tool Use
- 518Design an agent that takes actions in enterprise tools (file a Jira ticket, draft an email) on a user's behalf. How do you handle permissions and evaluate it?
- 519Design agent orchestration across dozens of connected SaaS systems. Where is authorization enforced, and why can it not live in the model?
AI System Design
- 520Design a connector framework that syncs content and permissions from 100+ SaaS apps into one index.
- 521Glean's ranking leans on a knowledge graph of people, content and activity. How would you build that graph, and how does it improve retrieval beyond embedding similarity?
- 522You have dozens of ranking signals and a brand-new tenant with zero interaction data. How do you rank, and how do you improve?
Evaluation and Observability
Character.AI
12 questionsRoles: ML Engineer (inference), Research Engineer, Software Engineer (product / safety), Infrastructure Engineer.
Loop: Inference-economics and serving depth, a prompt-construction coding round, a safety-system design round, and product rounds on discovery and engagement.
Coding and Data Structures
LLM Internals and Architecture
Inference, Serving and GPU Performance
- 526Our serving cost is dominated by KV cache, not weights. Get it down by an order of magnitude and tell me what you give up.
- 527Dialogues here average around 180 messages. Design the cache that sits between turns.
- 528You train natively in int8 rather than doing post-training quantization. Defend that.
- 529Estimate what one message costs us to serve, and tell me which lever moves it most.
AI System Design
Evaluation and Observability
- 531Users complain that characters drift out of persona after a long session. Diagnose it.
- 532Engagement metrics and wellbeing metrics disagree. How do you build a system that resolves that?
Safety, Security and Responsible AI
ElevenLabs
8 questionsRoles: Research Engineer (speech), ML Engineer, Software Engineer (real-time audio), Forward Deployed Engineer, Infrastructure Engineer.
Loop: A real-time-audio systems round, a speech-model depth round, a practical streaming coding round, a safety round on voice cloning, and a customer-deployment round for FDE roles.
Coding and Data Structures
Inference, Serving and GPU Performance
Safety, Security and Responsible AI
Multimodal, Speech and Voice AI
- 539Budget the end-to-end latency for a real-time voice agent. Why is time-to-first-audio a different problem from an LLM's time-to-first-token?
- 540Text normalisation is where TTS quality actually dies in production. Walk me through it.
- 541Design the dubbing pipeline: an English video becomes Spanish, same speakers, same timing.
Applied and Forward-Deployed Scenarios
Abridge
11 questionsRoles: ML Engineer (ASR / NLP), Research Scientist, Software Engineer (clinical products), Forward Deployed / Implementation Engineer.
Loop: A clinical-ASR depth round, an evaluation-design round (there is rarely a single correct note), a healthcare-integration systems round, and a privacy/compliance round.
ML and DL Fundamentals
Inference, Serving and GPU Performance
RAG and Retrieval
Agents and Tool Use
AI System Design
Evaluation and Observability
- 549Two good clinicians write different notes for the same visit. So how do you evaluate note quality at all?
- 550Edit rate is the obvious measure of clinician trust. What does it hide, and what would you instrument instead?
Safety, Security and Responsible AI
- 551A generated note contains a medication the patient never mentioned. Treat that as a safety incident: how do you detect it before a clinician sees it?
- 552Clinicians will not sign what they cannot verify. How would you build span-level provenance from every line of the note back to the conversation?
- 553PHI is in every audio file, transcript and note you touch. How does that shape the architecture, and what can you send to a third-party model API?
Multimodal, Speech and Voice AI
Figure AI
12 questionsRoles: Robotics AI Engineer, Research Engineer (VLA / manipulation), Controls Engineer, Data Engineer (teleoperation), Deployment Engineer.
Loop: Robot-learning depth (VLA, imitation, RL), a data-pipeline design round, a sim-to-real round, a safety-architecture round, and a hands-on numerical/implementation round.
ML and DL Fundamentals
- 555Behaviour cloning on teleoperation data has a well-known failure mode. What is it, and what do you do about it on a real humanoid?
- 556A whole-body controller trained entirely in simulation has to run on real hardware. What transfers, what does not, and how do you close the gap?
- 557Where does reinforcement learning fit on top of imitation learning for manipulation, and what makes the reward the hard part?
Inference, Serving and GPU Performance
AI System Design
- 559Design the teleoperation data pipeline. Why is data collection the bottleneck in robotics rather than compute?
- 560You have 10 hours of demonstrations for a new task and budget for 50 more. How do you decide what to collect, and what return do you expect?
Evaluation and Observability
- 561How do you evaluate a manipulation policy when every trial costs robot time and every failure has physical consequences?
- 562You ship a policy to 300 robots. It works in the lab and degrades in the field. Debug it.
Safety, Security and Responsible AI
Multimodal, Speech and Voice AI
Waymo
12 questionsRoles: Software Engineer (perception, prediction, planning), Research Scientist, ML Infrastructure Engineer, Simulation Engineer, Safety Engineer.
Loop: Coding screen → onsite: two coding rounds (often numerical/vectorised), a perception or planning depth round, an ML-infrastructure or simulation design round, and behavioural. Safety-case reasoning is weighted heavily for senior roles.
Coding and Data Structures
ML and DL Fundamentals
- 568Modular perception, prediction and planning, or end-to-end learned driving? Make the case, then tell me what you would actually build.
- 569Design the output representation for a behaviour prediction model. What metrics would you gate it on?
- 570Where do vision-language models and foundation models genuinely help in an autonomy stack, and where are they a liability?
Inference, Serving and GPU Performance
AI System Design
- 572You have hundreds of millions of fleet miles. How do you find and use the rare scenarios that matter?
- 573Design a system that finds driving segments similar to a given one across the entire fleet archive.
- 574You are opening in a new city. Structure the safety case.
Evaluation and Observability
- 575Disengagement rate is a weak safety proxy. How would you actually measure whether the Driver is safe enough to ship?
- 576How do you build a simulator you would trust to gate a release?
- 577Two days before a release decision, simulation shows a 15% increase in hard-braking events in one scenario cluster. Walk me through what you do.
Multimodal, Speech and Voice AI
Forward-Deployed and Enterprise AI
1 section · 20 questions
Palantir
20 questionsRoles: Forward Deployed Engineer, Forward Deployed Software Engineer, Software Engineer, Data Engineer, Deployment Strategist, AIP engineer.
Loop: Recruiter call (~30 min) → technical screen (live coding or HackerRank: a coding challenge, a SQL query and an API task) → onsite of three 60-minute rounds drawn from decomposition, learning, coding, re-engineering and system design → hiring-manager round that revisits weaker areas. Decomposition is the signature round.
Coding and Data Structures
- 579Implement a set of shape classes that compute area, then extend them to handle a new shape.
- 580Write a SQL query that joins and aggregates across tables to answer a business question.
- 581Build a function that fetches paginated data from a REST API, handling page size and total-page logic.
- 582Find and fix a double-counting bug in a function that tallies values in a HashMap.
- 583Debug a program that models infection spread across a social graph.
- 584You inherit an 800-line pipeline script from a previous deployment. It's slow and occasionally produces wrong numbers. The original author is gone. Go.
- 585Given exports from three customer systems, each with its own customer records, write code to produce one deduplicated set of entities and explain your design.
RAG and Retrieval
Agents and Tool Use
- 587What is an ontology in the Palantir sense, and why put LLM agents on top of one instead of on raw tables and documents?
- 588Design an LLM agent that files and updates work orders in a customer's ERP: real writes to a production system. How do you make that safe?
AI System Design
Evaluation and Observability
Applied and Forward-Deployed Scenarios
- 591A freight rail operator loses tens of millions a year to unplanned locomotive downtime. Decompose this into an engineering plan.
- 592Design a system to improve traffic in NYC.
- 593Design a sync system between two employee record systems.
- 594Design a system that lets multiple teams query a shared dataset without exposing raw data.
- 595Design an application to catalog and log species while exploring an unfamiliar environment.
- 596A customer executive says “the AI keeps getting things wrong” and wants to cancel the pilot. Walk me through your next 48 hours.
Behavioral and Culture