Analyzing coding agent transcripts to upper bound productivity gains from AI agents
METR (AI lab research) · 17 Feb 2026
Read original sourceOpen accessNot peer reviewed
Where this sits in the debate
- Will AI actually boost productivity and growth? · its finding: it depends (why studies differ)
Direction of the finding as read from the abstract by an AI model (AI-written).
Source excerpt
Introduction Human uplift studies like the one we did in 2025 are becoming more expensive as working without AI becomes increasingly costly. In this post, I investigate whether coding agent transcripts could serve as a cheaper alternative for estimating uplift. I prototyped this using 5305 Claude Code transcripts generated in January 2026 by 7 METR technical staff 1 . I used an LLM judge to estimate how long each task would have taken an experienced software engineer without AI tools, then compared that to the time people actually spent on these tasks to calculate a time savings factor . Takeaways This method estimates a time savings factor of ~1.5x to ~13x on Claude Code-assisted tasks for 7 METR technical staff in January 2026 – though this result comes with substantial caveats. I believe the true productivity multiplier is substantially lower, and the time savings factor is a soft upper bound for the true uplift that the individuals experienced. Increased agent concurrency may contribute to a higher time savings factor on the Claude Code-assisted task distributions. Limitations The time savings factor on the coding agent-assisted task distributions does not equal the productivity multiplier. People likely do not create 10x as much value with AI, even if we observe a 10x time savings factor on tasks that people do with AI. I believe the time savings factor overestimates AI-enabled productivity gains for reasons including: Task Substitution. With AI assistance, people sometimes complete tasks they wouldn’t do otherwise, such as making nice-to-have tools. These tasks tend to deliver lower counterfactual value, even if the observed time savings factor on these tasks might be high. Task Selection Effects. People don’t spend their entire workday using AI—they use it only on tasks where they expect it to be helpful. Since coding transcripts exclude tasks people do without AI, the time savings factor likely overestimates total AI-enabled uplift. Pre-existing Worker Speci
About this publication
Lab or institute research, not peer reviewed. A report or blog post published by an AI lab or research institute on its own site. It has not been through independent peer review, and the publisher may have a stake in the findings.
- Published
- 2026 (8 months ago)
- Citations
- Not recorded for this source
Citation counts come from OpenAlex; none are recorded for this source — absence of a count is not absence of citations.
Related by theme, not necessarily by method or conclusion.
- Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality
SSRN · 2023 - Generative AI at Work
The Quarterly Journal of Economics · 2025 - The Impact of AI on Developer Productivity: Evidence from GitHub Copilot
arXiv · 2023 - The Productivity J-Curve: How Intangibles Complement General Purpose Technologies
American Economic Journal Macroeconomics · 2020
Overview assembled automatically from collected metadata. How this site is built.