Structuring an AI team: roles and ratios
A first AI team needs AI engineers who build and ship, at least one data engineer, a product owner who owns the metric, time from domain experts who define what correct means, and platform and security support. Start with a small cross-functional team on one use case, and add specialists, ML engineers for fine-tuning or MLOps engineers for infrastructure, as the work demands.
When one person is the whole AI team
Companies often hire one "AI person" and expect them to cover data engineering, model work, software, deployment and governance. That person spends most of their time waiting for data access and doing work outside their strengths.
Data engineers first
Without pipelines and access to data, AI engineers spend their time waiting or building on samples that don't match production. Hire or assign a data engineer before, or with, the first AI engineer.
Roles
| Role | Owns | When you need it |
|---|---|---|
| AI engineer | Building the system: retrieval, agents, evals, deployment | From the start |
| Data engineer | Pipelines, data access, quality | From the start |
| AI product manager | Problem, metric, users, priorities | From the start, can be part-time |
| Domain expert | Definition of correct, labels, review | From the start, a few hours a week |
| Platform and MLOps | Infrastructure, monitoring, cost | When you run more than one system |
| Security and governance | Risk review, policies | Before the first launch |
| ML engineer | Fine-tuning, custom models | When a use case needs it |
Team shapes
| Stage | Shape |
|---|---|
| First use case | One squad: 2 AI engineers, 1 data engineer, a product owner, a domain expert's time |
| Several use cases | One squad per business area, shared platform team |
| Many use cases | Platform team plus embedded AI engineers in product teams, central governance |
These are starting shapes, not ratios to copy: the number of engineers depends on the use cases and how much you buy versus build.
Where the team sits
A central team builds consistency and shared infrastructure, and embedded engineers stay close to the business. Most companies start central and move toward embedded engineers with a central platform as use cases grow.
From the floor
In the AI hype vs reality session, I listed the people a data and AI team needs most: data engineers first, who can create pipelines, then AI product managers and MLOps engineers, because in most cases the hype makes companies expect one person to do everything.