AI system design interview: design a meeting summariser with action items
Transcribe with speaker labels, split the transcript by topic or time, summarise each part and combine the parts (map-reduce), and extract action items into a schema with task, owner, due date and the quote that supports it. Check that owners are attendees and dates are valid, show items for confirmation before they go to a task tracker, and handle consent and retention. Evaluate with human-labelled meetings for action item recall, precision and owner accuracy.
Question
Summarise 60-minute meetings and extract action items with owners and dates.
Clarify
- Platforms (Zoom, Teams, Meet) and whether recordings or live audio.
- Languages and accents.
- Where action items go: email, a task tracker, a CRM?
- Consent and retention rules.
Estimate
60 minutes of speech is roughly 8,000 to 10,000 words, around 12,000 tokens, which fits in many context windows. Map-reduce is still useful because models summarise long inputs unevenly, and it lets you cite each item back to a part of the meeting.
Architecture
| Stage | Component |
|---|---|
| Transcription | Speech-to-text with speaker diarisation, names mapped from the invite |
| Segmentation | Split by topic or by 10-minute windows |
| Map | Summarise each segment, extract candidate action items |
| Reduce | Merge summaries, deduplicate action items |
| Validation | Owner must be an attendee, dates resolved and valid, quote must exist in the transcript |
| Review | Organiser confirms or edits before items are sent |
| Export | Task tracker or email |
Deep dive: action items
The schema forces the model to commit: task, owner, due date, evidence quote. A quote check (does this text appear in the transcript?) catches invented items, and the owner check catches names that weren't in the meeting.
Failure modes
| Failure | Mitigation |
|---|---|
| Wrong speaker attribution | Diarisation mapped to attendees, owner shown for review |
| Invented action items | Evidence quote required and checked |
| Missed action items | Map step per segment, recall measured |
| Sensitive content shared too widely | Access follows meeting attendees, retention limits |
Evals
Human-labelled meetings: action item recall and precision, owner accuracy, date accuracy, and a sample of summaries rated for faithfulness.
What interviewers listen for
Structured extraction with evidence, validation against attendees, and review before anything leaves the system.
From the floor
In the portfolio session, I described how I turn transcripts of my own sessions into blogs and other content, which works because a transcript with clear speakers can be summarised and searched, and the same applies to meetings: the transcript is the source of truth, and every summary line should point back to it.
Related
- Choose chunk size with evals
- Structured outputs and function calling
- Data leakage and PII
- Human in the loop