1. Why is chunking called the highest-leverage decision in a RAG pipeline?
A Because chunk size fixes the token budget each retrieved passage consumes, and prompt-budget arithmetic is what ultimately bounds how much evidence the generator can readB Because smaller chunks always embed more crisply, so chunking reduces to minimizing size — once every vector captures a single idea, overlap, structure, and boundary placement stop mattering downstreamC Because chunks are the unit of embedding, retrieval, and grounding — if an answer is sliced across a boundary or buried in noise at index time, no reranker or fusion downstream can recover itD Because the chunker decides which retrieval modes apply: fixed-size chunks can only be searched densely, while structural chunks are what makes BM25 and hybrid search possible
2. Which two query types does dense (embedding) retrieval handle poorly while BM25 handles them well?
A Exact identifiers (error codes, SKUs, config keys) and rare proper nouns/acronyms the embedding model never learnedB Paraphrased questions and synonym-heavy queries, since BM25's IDF weighting captures what a rare word means while embeddings only match on surface token overlapC Long multi-turn conversational questions and multi-hop comparisons, because a single query embedding can't span several sub-questions while BM25 scores each term independentlyD Misspelled queries and non-English questions, because embedding models are trained per language while BM25's tokenization is language- and typo-agnostic
3. How does Reciprocal Rank Fusion (RRF) combine BM25 and dense results, and why ranks instead of scores?
A It min-max normalizes BM25 scores and cosine similarities onto [0, 1], then sorts by a tuned weighted average — normalization is what makes the two score distributions directly comparable across corporaB Each document receives 1/(k + rank) from every ranking that contains it, summed and sorted; ranks are used because BM25 scores and cosine similarities live on incomparable, corpus-dependent scalesC It keeps only documents that appear in both top-k lists — the intersection filters out each retriever's false positives — and orders the survivors by their dense scoreD It interleaves the two lists round-robin, one from BM25 then one from dense, so each retriever contributes equally regardless of how its scores are scaled
4. Why are bi-encoders used for first-stage retrieval but cross-encoders for reranking?
A Bi-encoders and cross-encoders are equally accurate; the split exists because vector databases can only index vector outputs, so the cross-encoder is relegated to the stage that doesn't need an indexB Bi-encoders attend over the query and document jointly, making them more accurate but too slow for the full corpus, so they're saved for the small reranking set while cross-encoders handle first-stage scaleC Cross-encoders must score all the candidates against each other in a single forward pass, and roughly fifty pairs is what fits in one context window alongside the queryD Bi-encoders encode query and document independently, so document vectors can be precomputed and searched at corpus scale; cross-encoders attend over each query–document pair jointly — more accurate, but requiring a fresh forward pass per pair, affordable only for a few dozen candidates
5. Define precision@5, recall@5, and MRR.
A Precision@5 = fraction of all relevant chunks that appear in the top 5; recall@5 = fraction of the top 5 retrieved chunks that are relevant; MRR = the mean rank position of the first relevant chunk across queriesB Precision@5 = fraction of the top 5 retrieved chunks that are relevant; recall@5 = fraction of all relevant chunks that appear in the top 5; MRR = mean over queries of 1/rank of the first relevant chunkC All three are the same hit-rate at different cutoffs — precision over the top 5, recall over the whole ranking, MRR over the top 1 — so reporting more than one of them is redundantD Precision@5 = fraction of 5 generated answers the judge marks correct; recall@5 = fraction of eval questions the system attempts to answer; MRR = the judge's mean relevance rating
6. Your recall@5 is high but final answers are frequently wrong. Where do you look first?
A At generation: the evidence is reaching the prompt, so the failure is downstream — check faithfulness (is the model using the context or answering from its weights?) and precision (is junk alongside the good chunks poisoning it?)B At the embedding model: recall@5 only counts nearest-neighbor hits, so a high score with wrong answers usually means the embeddings surface topically similar but factually wrong chunks — swap in a stronger embedding model and re-index before touching the promptC At chunking: wrong answers despite high recall mean the evidence is straddling chunk boundaries, so the model only ever sees fragments — add overlap and re-index firstD At the ANN index: HNSW's greedy walk is approximate, so high measured recall can coexist with stale or unreachable vectors — rebuild the index and diff it against brute force first
7. What's the difference between faithfulness and answer relevance, and how does LLM-as-judge measure faithfulness?
A Faithfulness measures retrieval quality (did the right chunks arrive) while answer relevance measures generation quality; both are computed by counting citation markers against the passage listB They're two views of one property — an answer can't be faithful without being relevant, and vice versa — so RAGAS folds them into a single groundedness scoreC Faithfulness = every claim in the answer is supported by the retrieved context; answer relevance = the answer actually addresses the question. A judge model decomposes the answer into atomic claims and checks each for entailment by the context — faithfulness is the supported fractionD Faithfulness = the answer string matches the eval set's gold answer; relevance = a 1–5 fluency rating; both use exact-match scoring, which is why neither needs a judge model
8. How do you build a retrieval eval set cheaply but credibly?
A Adapt a public QA benchmark like Natural Questions — professionally labeled data beats anything you can generate, and retrieval metrics transfer across corpora anywayB Sample chunks, have an LLM draft a question each chunk answers (recording the source chunk as the relevance label), then human-verify every item and add some questions the corpus cannot answerC Have the system draft the questions, run its own retrieval, and record whichever chunks come back as the relevance labels — the pipeline's retrievals are the best available proxy for ground truth at zero labeling costD Skip synthetic data entirely and wait for six months of production queries — questions users never actually asked can't measure anything real
9. What is query decomposition, and when is single-shot RAG structurally unable to answer?
A Splitting a long query into windows that fit the embedding model's sequence limit and averaging the window vectors; needed whenever a query runs past the encoder's ~512-token capB Resolving pronouns and ellipsis from chat history to turn a conversational fragment into a standalone search query; needed from the second turn of any multi-turn RAG chatC Having an LLM expand the query into a hypothetical documentation-style answer passage and embedding that as the search probe, landing it nearer the document region of embedding space; needed whenever queries are short, vague, and information-poor rather than genuinely multi-partD Breaking a complex question into independently searchable sub-questions, retrieving for each, and answering over the union — needed for multi-hop/comparison questions where no single chunk contains the answer, so no single retrieval can ever surface sufficient evidence
10. Fixed single-shot RAG pipeline vs. agentic retrieval-as-a-tool: what are the trade-offs?
A Fixed pipeline: predictable latency and cost, easy to eval and debug, but fails on queries needing reformulation or multiple hops. Agentic: the model reformulates and retries retrieval until satisfied — higher quality ceiling on hard queries, but variable latency/cost and much harder to evaluate and debugB Agentic retrieval strictly dominates: because the model can always choose to run just one query, it's never worse than a fixed pipeline, and prompt caching makes the extra tool calls effectively freeC Fixed pipelines have the higher quality ceiling since every stage is tuned offline against the eval set; agentic retrieval mainly cuts cost by stopping early once results look good enoughD The trade-off is only latency: both are equally easy to evaluate offline, because either way you can replay a query and compare the retrieved chunks against the labeled relevant set
11. Your RAG system confidently answers questions the corpus can't support. Name three layered mitigations.
A Retrieve more chunks so the evidence is less likely to be missing, raise k until recall@k saturates, and move to a larger generator model — hallucination is a capacity problem that more evidence and a stronger model reliably solve togetherB Add a cross-encoder reranker so the best evidence rises to the top, widen chunk overlap so answers arrive intact, and embed headings so chunks are self-describingC Prompt an explicit exact refusal string for insufficient context; abstain when retrieval/rerank scores fall below a threshold; measure faithfulness and citation accuracy in the eval loop, with unanswerable questions in the eval setD Fine-tune the generator on the corpus so the facts live in its weights, decompose every incoming query, and lower temperature to zero for deterministic answers
12. How does HyDE improve retrieval, and what does it actually embed?
A It re-embeds the query with a larger, higher-dimensional embedding model whose vectors carry more nuance, so the unchanged question lands closer to relevant documents; what's embedded is still the query itselfB It has an LLM generate a hypothetical answer passage to the query, then embeds that passage as the search probe — because answer-shaped text lands nearer to real documents in embedding space than question-shaped text doesC It appends the top BM25 result's keywords to the query before embedding, anchoring the probe in the corpus's own vocabulary; what's embedded is the query plus retrieved termsD It generates a hypothetical summary of every document at index time and embeds those instead of raw chunks, so at query time the plain question is matched against answer-shaped summaries sitting on the document side of the index
13. "Context windows are a million tokens — why not paste the whole corpus into every prompt instead of building RAG?" Which answer covers the senior objections?
A Mostly habit and sunk cost: with prompt caching the corpus prefix is paid for once, so long context now matches RAG on price while beating it on recall — new systems should stuff the window firstB Cost (re-processing the corpus per query, and any corpus edit invalidates the cached prefix), degraded attention over huge mostly-irrelevant contexts, no per-user access control or citations, and corpora that outgrow any window — though a single small document that fits is legitimately better served without a pipelineC Stuffed prompts overflow the KV cache, so providers silently truncate from the middle of the context and the model never actually sees most of the corpusD Only latency: a million-token prompt takes minutes to prefill today, but once providers reach sub-second long-context inference there is no remaining reason to retrieve
14. When should you fine-tune a model versus building RAG, for a product that must answer questions about internal documentation?
A Fine-tune on the documentation first: facts stored in the weights answer with zero retrieval latency and no pipeline to maintain, and RAG is only worth bolting on as a fallback when training costs or corpus size make re-tuning per release impracticalB Both inject domain knowledge, so pick by operations: a team with GPU experience should tune while a team with search experience should build RAG — the outcomes converge either wayC Fine-tune for behavior (style, format, domain vocabulary); use RAG for knowledge — tuned-in facts are slow to update, impossible to cite, and make hallucination more fluent, while an index updates per docs release and yields checkable citationsD Always do both from day one: tuning bakes the facts into the weights and RAG double-checks them at query time, and that redundancy is what finally eliminates hallucination
15. What problem do contextual retrieval and small-to-big (parent-document) retrieval both solve, and what's the shared principle?
A They shrink the index: contextual retrieval collapses related chunks under one situating vector and small-to-big stores a single parent-level embedding per section — fewer vectors, less RAM, cheaper ANN searchB Chunks are read out of context and can't be both crisp to embed and rich enough to answer from — contextual retrieval prepends situating text before embedding, small-to-big embeds small units but hands the model their parent section; both decouple the retrieval representation from the generation payloadC Both make the reranker unnecessary — once chunks carry their own context, first-stage rankings are precise enough to feed the generator directlyD They fix dense retrieval's exact-identifier blind spot: the prepended blurbs and parent sections reintroduce the rare tokens and error codes that embedding models squash toward noise, which is why systems adopting them can drop BM25 and go dense-only
16. Your multi-tenant vector search returns almost nothing for small tenants, though their documents are indexed. What's the likely cause?
A Small tenants' vectors form weakly linked islands in the shared HNSW graph, so the greedy walk descends toward regions dominated by large tenants and never visits them — raise the search-breadth parameter (ef)B Post-filtering: the ANN search retrieves top-k over the whole corpus first, then applies the tenant filter — a tenant owning 1% of the corpus gets ~0 of the k results; the fix is pre-filtering (the filter constrains the graph walk) or, crudely, oversampling before filteringC Their upserts land in small unmerged segments the query path skips until compaction runs, so recently imported tenants stay invisible — force a segment merge after each bulk importD A global similarity-score cutoff: small tenants' narrower corpora produce lower best-match cosine scores that fall under a threshold tuned on the biggest tenants, so nearly all their hits are silently dropped — fix it with per-tenant thresholds or by normalizing scores per tenant before the cutoff
17. Before trusting an LLM judge's faithfulness scores, what validation does a senior engineer run?
A None beyond spot checks — a pinned judge model run at temperature 0 is deterministic and self-consistent by construction, which is exactly the reliability property human raters lackB Run the judge twice per item and keep only the verdicts where it agrees with itself — self-consistency filtering is far cheaper than hand-labeling and measures the same thing human agreement wouldC Hand-label 30–50 answers, measure judge-vs-human agreement, and iterate on the judge prompt until agreement is high; de-bias by construction (different model family than the generator, claim-level scoring, order-randomized comparisons, pinned judge version); keep known-good/known-bad canaries in every run to catch judge driftD Have the judge attach a confidence score to every verdict and discard the low-confidence ones — a judge that knows when it is unsure doesn't need external calibration against humans