Measuring AI adoption inside the company
Track adoption in layers: access (who has the tool), activity (who uses it weekly in the target workflow), outcomes (cycle time, throughput, quality of the process) and economics (cost per task, value delivered). Logins and prompts sent are activity, not adoption, and self-reported time savings need checking against sampled measurements of the process itself.
Activity isn't adoption
A dashboard shows 10,000 prompts a week and leadership calls it success, while the process it was meant to improve hasn't changed. Adoption means the target workflow now runs differently, and the numbers for that workflow moved.
Layers
| Layer | Metric | Source |
|---|---|---|
| Access | Licensed or enabled users | Admin console |
| Activity | Weekly active users in the target workflow | Product analytics |
| Outcome | Cycle time, throughput, error rate for the process | Process data, sampled |
| Quality | Pass rate on sampled outputs | Expert review, evals |
| Economics | Cost per task, value per month | Finance and usage data |
Validate time savings
Surveys asking "how much time did AI save you?" can be far off. In METR's 2025 study of experienced open-source developers working on their own repositories, developers expected AI tools to speed them up, and were measured as taking 19% longer, which is one study in one setting, and a good reason to measure instead of asking. Sample real tasks before and after, and time them, or compare cycle times in the system of record, like ticket resolution time or documents processed per day.
Unused tools are a signal
If a tool sees little use, find out why before pushing harder: it may sit outside the workflow, give answers people don't trust, or solve a problem people don't have. Reports that nobody reads are a sign to stop the tool, not to send more reports.
North Star per use case
Each use case gets one business metric, owned by a business leader, with a baseline measured before launch and reviewed monthly with the owner in the room. Model metrics like accuracy matter to engineers, and the board cares about the business number.
From the floor
In the founding session of the AI leadership cohort, one of the ideas we stopped was an AI training content generator, because it had no usage data and no learner outcome metric, so there was no way to know whether anyone was helped by it.