The Impact of AI on Developer Productivity: Evidence from GitHub Copilot

Sida Peng, Eirini Kalliamvakou, Peter Cihon, Mert Demirer · arXiv (Cornell University) · 13 Feb 2023

Where this sits in the debate

Will AI actually boost productivity and growth? · its finding supports the claim (why studies differ)

Direction of the finding as read from the abstract by an AI model (AI-written).

Key takeaway AI-written

A controlled experiment with recruited developers building a JavaScript HTTP server found that those with GitHub Copilot access finished the task 55.8% faster than a control group without it, with especially strong effects for less experienced programmers. It remains one of the most-cited direct measurements of an AI coding assistant's productivity effect and is frequently used as a benchmark figure in broader estimates of AI-driven productivity growth.

The paper in three steps AI-written

Written by an AI agent from the paper's abstract, which is the primary source and is quoted below.

The question

Does access to an AI pair programmer (GitHub Copilot) make software developers complete a coding task faster?

The approach

A controlled experiment: recruited developers were asked to implement an HTTP server in JavaScript as quickly as possible, with the treatment group given access to GitHub Copilot and the control group working without it.

The findings

The group with Copilot completed the task 55.8% faster than the control group. The authors also observe heterogeneous effects that suggest AI pair programmers could help people move into software development careers.

How to read it: experiment. Measures a causal effect in a controlled or randomized setting — strong internal validity; ask how far it generalizes.

Limitations and counterpoints AI-written

A single, well-defined programming task under time pressure; speed on this task may not reflect productivity on larger, real-world codebases or code quality over time.

Abstract

Generative AI tools hold promise to increase human productivity. This paper presents results from a controlled experiment with GitHub Copilot, an AI pair programmer. Recruited software developers were asked to implement an HTTP server in JavaScript as quickly as possible. The treatment group, with access to the AI pair programmer, completed the task 55.8% faster than the control group. Observed heterogenous effects show promise for AI pair programmers to help people transition into software development careers.

About this publication

Preprint. A preprint — posted publicly before or without journal peer review; findings may still be revised.

Published: 2023 (45 months ago)
Citations: 251 (OpenAlex)

251 citations recorded by OpenAlex — too few 2023 papers in this collection for a fair comparison.

□ added to this collection in 2026.