Non-functional requirements for AI systems
Non-functional requirements describe how well a system must work, not what it does. For AI systems, write them as testable targets: p95 latency (time to first token and total), cost per request or per task, a minimum pass rate on a named eval set, availability, data residency and retention, safety thresholds, and auditability. "It should be fast and accurate" can't be tested, so it can't be met, and for AI, accuracy is a target on a specific eval set, not a property of a model.
Functional and non-functional
Functional requirements say what the system does: "answer questions about our refund policy", "extract invoice totals". Non-functional requirements say how well: how fast, how often correct, how available, how private, at what cost.
Most AI projects write the first kind and assume the second. Then the system works in a demo, and fails on cost, latency or a privacy review in production.
Classic qualities
Every system design starts with the same qualities:
- Scalability: handling growth in users and data.
- Reliability: working correctly even when parts fail.
- Performance: responding fast enough.
- Maintainability: easy to understand, change and fix.
Think of a bridge: it has to hold whatever crowd walks on it, stand through every season for years, and be inspected and repaired without being rebuilt. Software systems need the same.
What AI systems add
| Requirement | Why it's different for AI | Example target |
|---|---|---|
| Latency | Two numbers matter: first token and total | p95 time to first token under 1.5 s, total under 6 s |
| Cost | Changes with every prompt edit and every extra step | Under a set amount per resolved conversation |
| Quality | Outputs vary, so quality is a pass rate on an eval set | 95% pass on the policy eval set v3 |
| Availability | Depends on external model providers | 99.5% with a fallback provider |
| Privacy | Data may be sent to a third party | No PII sent to external APIs, data stays in region |
| Safety | New failure types: injection, leakage, harmful content | Zero leaked system prompts in the red-team suite |
| Audit | Answers must be traceable | Every answer traceable to its sources for 1 year |
| Freshness | Answers depend on the documents behind them | Policy changes reflected within 1 hour |
How to write them
Each requirement needs:
- A metric ("p95 time to first token").
- A target ("under 1.5 seconds").
- Where it's measured ("at the API gateway, for the chat route").
- How it's tested (a dashboard, an eval run, a load test).
- An owner.
Name the eval set and its version in quality requirements. "95% accurate" means nothing, and "95% pass on refund-policy-evals v3" can be tested on every release.
Trade-offs
Requirements pull against each other:
- Higher quality often means a bigger model or more steps, which costs more and takes longer.
- Lower latency may mean a smaller model, or fewer checks.
- Stronger privacy may mean self-hosting, which affects cost and quality.
Write down which one wins when they conflict, per feature. A fraud check needs speed, and a legal research answer can take longer if it's correct.
Template
| Requirement | Metric | Target | Measured at | Test | Owner |
|---|---|---|---|---|---|
| Latency | p95 TTFT, chat route | < 1.5 s | Gateway | Dashboard, load test | Platform |
| Quality | Pass rate, policy evals v3 | ≥ 95% | CI and weekly sample | Eval run | Support lead |
| Cost | Cost per resolved conversation | < budget | Cost tracker | Weekly review | Product |
| Privacy | PII to external APIs | 0 | Guardrail logs | Red-team suite | Security |
From the floor
In the first AI system design session, I explained the key qualities of any system as scalability, handling many requests at the same time, and reliability and performance over years, using a structure that has to hold up in every season and under any crowd as the picture. For AI systems, I add cost per request and quality on an eval set to that list.
In the ML system design session, one requirement question I asked for every system was whether accuracy or latency has priority, because the answer changes the model, the compute and the design.