AI system design interview: Q&A over annual financial reports
Parse filings with their section structure and extract tables as tables, chunk by section with company, year and section as metadata, and retrieve with hybrid search filtered by company and year. Send numeric questions to a query tool over the extracted tables, so numbers come from data, and cite page and section in every answer. Evaluate with analyst-written questions, checking numeric answers exactly.
Question
Analysts want to ask questions over hundreds of annual reports (10-K filings). Design it.
Clarify
- Question types: single facts, numbers, comparisons across years or companies, summaries of risk factors.
- How many companies and years.
- Whether answers need exact page citations for compliance.
Architecture
| Stage | Component |
|---|---|
| Parsing | Section-aware parsing (risk factors, MD&A, financial statements), tables extracted as structured data |
| Chunking | By section and subsection, section titles kept in each chunk |
| Metadata | Company, ticker, fiscal year, section, page |
| Retrieval | Hybrid search with company and year filters, reranking |
| Numbers | A query tool over extracted tables |
| Comparison | Retrieve the same section for both years, compare side by side |
| Answers | Citations to page and section, abstain when not found |
Deep dive: numbers
A model reading a table in a text chunk can pick the wrong column or year. Extract financial tables into structured data during ingestion, and let the model call a tool that returns the exact value, then cite where it came from.
Failure modes
| Failure | Mitigation |
|---|---|
| Wrong year or company | Metadata filters from the parsed question |
| Wrong number from a table | Structured table extraction and a query tool |
| Missed answer in a long section | Section-aware chunks, reranking |
| Comparison mixes up years | Retrieve each year separately and label them |
Evals
Analyst-written questions with gold answers and source pages, numeric answers checked exactly, and retrieval checked by whether the right page appears in the top k.
What interviewers listen for
Table extraction, metadata filters, numbers from tools, and citations to page.
From the floor
In the financial RAG session in AI Engineer HQ, we built a small version on the first 10 pages of Tesla's 10-K: LangChain's PDF loader and recursive text splitter, sentence-transformers for embeddings, Chroma as the vector database, a retrieval QA chain and a Flask app, run against that sample report before moving toward production.