All categories
AI / ML Interview Questions
Transformers, RAG and training internals — the questions behind modern AI roles.
3 of 3 questions
Topic
Difficulty
Frequency
Round
Type
IntermediateAsked very oftenRAGSystem design · Design
Design a RAG pipeline end to end
Two pipelines, one grounded answer: ingestion makes knowledge retrievable, query turns a question into a cited response.
ragvector-searchembeddings
IntermediateAsked very oftenTrainingConcept round · Concept
How does backpropagation compute every gradient?
One forward pass computes the loss; one backward pass applies the chain rule to every weight; one step moves them downhill.
backpropagationchain-rulegradients
AdvancedAsked sometimesTransformersConcept round · Concept
Why does attention scale the logits by √d_k?
The 1/√d_k factor looks like a footnote, but it decides whether attention learns or saturates into one-hot weights.
transformersattentionsoftmax