AI system design interview: evaluate a RAG system before launch
Build a test set with a domain expert: answerable questions linked to the passages that answer them, plus unanswerable ones. Measure retrieval (recall@k, MRR) and generation (faithfulness, relevance, abstention) separately, check any LLM judge against expert labels, add permission and red-team tests, and set launch gates in advance. After launch, sample production traces every week and add new failures to the set.
Question
You built a RAG assistant. How do you know it's ready to launch?
Clarify
- Who uses it and for what decisions: a wrong answer in HR policy costs more than one in a product FAQ.
- Whether real user questions exist (from search logs, tickets, emails).
- Who the domain expert is.
Build the test set
| Set | Size to start | Source |
|---|---|---|
| Answerable questions with gold passages | 100 to 200 | Real questions from logs or tickets, labelled by an expert |
| Unanswerable questions | 30 to 50 | Plausible questions the documents don't answer |
| Permission tests | 20+ | Questions whose answers only some users may see |
| Adversarial inputs | 20+ | Injections, out-of-scope requests |
Use real questions where you can, because generated ones tend to copy the wording of the documents and make retrieval look better than it is.
Measure in two halves
Retrieval: recall@k (did the gold passage appear in the top k?) and MRR (how high?). Generation, given good retrieval: faithfulness (every claim supported by the context), relevance, and correct abstention on unanswerable questions. Splitting them tells you where to fix: if recall is low, no prompt change will help.
Validate the judge
If an LLM judges faithfulness, have the expert label 100 or so answers, then compare: true positive rate and true negative rate on the expert's labels. A judge that agrees 90% of the time can still miss most of the failures if failures are rare.
Launch gates
Set them before running the final evaluation, for example:
- Recall@5 above an agreed target.
- Zero permission leaks.
- Faithfulness above an agreed target on sampled answers.
- Correct abstention on most unanswerable questions.
- No critical failures in the adversarial set.
After launch
Sample production traces weekly, read them, label failures, and add them to the test set, so the set reflects real use.
What interviewers listen for
Two halves, a validated judge, unanswerable questions, and launch gates.
From the floor
In AI Engineer HQ, the question I keep asking builders about any RAG project is the same: show me one question it got wrong, whether the right passage was retrieved, and what you changed, because that tells me whether they evaluated retrieval and generation separately.