Reference architecture: multi-agent platform
A production agent platform has a front end, an API that accepts goals and returns run IDs, an orchestrator that manages agent state as a graph, workers that execute steps from a queue, a tool registry with sandboxes, a model gateway for every LLM call, an approval service for risky actions, persistent checkpoints, and tracing, metrics and budgets for every run. The agents are a small part of the code. Most of the platform is what keeps them observable, limited and recoverable.
Architecture
front end ──> API (goal in, run ID out)
│
▼
queue ──> workers
│
▼
orchestrator (state graph)
planner → coder → tester → reviewer
│ │ │
▼ ▼ ▼
model gateway tool registry + sandbox approval service
│ │ │
└──────────┴──────────┬──────────────┘
▼
checkpoints (Postgres), traces, metrics, budgets
Components
| Component | Example choices |
|---|---|
| Front end | Next.js with a component library, on a managed platform |
| API | FastAPI |
| Orchestrator | LangGraph state graphs |
| Persistence | Postgres for state and checkpoints |
| Queue and workers | Celery, Redis queues or a cloud queue |
| Model gateway | LiteLLM or similar |
| Tools | Registry plus sandboxed execution in containers |
| Approvals | Web UI or Slack interrupts |
| Observability | Langfuse for traces, Prometheus and Grafana for metrics |
Key decisions
- Agent roles and contracts: what each agent receives and must return. For a software factory: planner, coder, tester, reviewer.
- Where people approve: merges, deployments, anything irreversible.
- Per-run budgets for steps, tokens, money and time. See agent cost control.
- What happens on failure: replan, retry with a hint, or stop and report. See agent state and recovery.
- Whether you need several agents at all. See multi-agent systems.
Failure modes to test
- Agents looping without progress.
- An agent passing a bad result that the next agent trusts.
- Tool calls outside the agent's permissions.
- Lost runs after a worker restart.
- Cost per run growing with each version.
Metrics
- Run success rate.
- Steps and cost per successful run.
- Approval rejection rate.
- Time to resume after a failure.
From the floor
Agent Forge, the second artifact of AI Engineer HQ cohort 2, was a multi-agent software factory with planner, coder, reviewer and tester agents, each with its own versioned prompt, plus a metrics configuration for judging the system and test cases. Its production version used Vercel for the front end, a back end with a list of workers, LiteLLM for model calls, Docker, Prometheus and Grafana for monitoring, Langfuse for tracing every LLM call, and GitHub Actions for CI/CD.
In the demo, when the tester found an issue in the generated code, the run paused and offered three options, replan, stop, or retry with a hint from the user, and choosing replan sent it back to the planner.