Contents
Claims about artificial intelligence and productivity tend to come from people selling it or people afraid of it. A better guide is the small set of controlled studies in which some workers were given an AI tool, others were not, and the difference was measured. Most of them find a gain. One well-known study found a loss. Read together, they say that the answer depends on who is doing the work and what the work is.
The short answer
Usually, yes, but not for everyone. In controlled trials, AI tools made people between about 12% and 56% more productive on the main measure each study used, and the least experienced workers gained the most. The exception is experienced developers working on code they knew well: in one 2025 trial they were 19% slower with AI. And so far the gains haven’t shown up in pay or working hours.
How these studies work
The strongest evidence comes from randomised controlled trials. Workers or tasks are assigned to AI or no AI by chance, so any difference in output can be put down to the tool and not to the kind of person who chooses to use it. The tools tested are generative AI systems, meaning software such as ChatGPT or GitHub Copilot that writes text or computer code on request.
| Study | Who took part | Sample | Main result |
|---|---|---|---|
| Brynjolfsson, Li and Raymond | Customer support agents | 5,172 | 15% more issues resolved per hour |
| Noy and Zhang, 2023 | Professionals on writing tasks | 453 | 40% less time, quality up 18% |
| Dell’Acqua and others, 2023 | Management consultants | 758 | 12.2% more tasks completed |
| Peng and others, 2023 | Programmers on one set task | 95 | 55.8% faster |
| Cui and others, 2025 | Developers at three companies | 4,867 | 26.08% more completed tasks |
| METR, 2025 | Experienced open-source developers | 16 | 19% slower |
Customer support: the least experienced gained most
The largest study followed 5,172 customer support agents at a large software company as it introduced an AI assistant in stages. Most agents were not assigned the tool at random, but the staggered rollout let the economists Erik Brynjolfsson, Danielle Li and Lindsey Raymond compare agents with and without it. Agents with access resolved 15% more issues per hour on average. The least experienced and least skilled improved by about 30%. The most skilled gained little speed, and the quality of their conversations fell slightly. Customers also wrote to agents in a more positive tone, and fewer new staff left.
The pattern of larger gains for weaker performers recurs. In an experiment published in the journal Science, 453 college-educated professionals were given two short writing tasks, and half could use ChatGPT for the second one. They finished in 40% less time, and assessors who did not know which pieces had used AI rated their work 18% higher. Those who had scored worst on the first task gained most. The tasks took 20 to 30 minutes and came with clear instructions, and nobody checked the output for factual accuracy, so the gain in real work may be smaller.
Consulting: a boost, and a trap
A trial with 758 consultants at Boston Consulting Group, using the GPT-4 model, tested two kinds of work. On a set of 18 tasks the researchers judged to be within the AI’s abilities, consultants with AI completed 12.2% more of them and worked about 25% faster. Consultants in the bottom half for skill improved their scores by 43%, against 17% for the top half.
The second task was a business problem designed so that the AI would make an error in its analysis. Without AI, consultants reached the right answer about 84.5% of the time. With it, the figure fell to between 60% and 70%, an average drop of 19 percentage points. The authors call the boundary between the two kinds of task a “jagged technological frontier”, because tasks that look equally hard can fall on either side of it.
Without AI, consultants reached the right answer about 84.5% of the time. With it, the figure fell to between 60% and 70%.”
Coding: the widest range of results
Software development has produced both the largest gain and the only measured loss.
In an early experiment, 95 programmers recruited through a freelance site were asked to build a simple web server in the JavaScript language. Among the 70 who finished, those given GitHub Copilot were 55.8% faster, averaging about 71 minutes against 161. The margin of uncertainty was wide: the 95% confidence interval, the range the true effect probably lies in, ran from 21% to 89%. It was one standardised task, and code quality was not examined.
A larger analysis combined three company-run trials covering 4,867 developers at Microsoft, Accenture and an unnamed Fortune 100 firm. Developers using the assistant completed 26.08% more tasks. The reported standard error of 10.3%, a measure of how much the estimate could vary by chance, means the true figure could plausibly be anywhere from about 6% to 46%. According to MIT Sloan’s account of the study, junior developers and recent hires gained 27% to 39% and senior developers 8% to 13%. The researchers could not see the code, so they could not judge its quality.
| Item | % improvement on each study’s main measure |
|---|---|
| Programmers on one set task (speed) | 55.8% |
| Developers at three firms (tasks done) | 26.1% |
| Customer support agents (issues per hour) | 15% |
| Consultants (tasks completed) | 12.2% |
Each trial measured something different. A fifth trial, by METR, found experienced developers were 19% slower.Source: Studies cited in this article. Chart: The Sidelight.
The study where AI slowed people down
In 2025 the research group METR ran a randomised trial with 16 experienced open-source developers. They worked on 246 real tasks in large software projects they had contributed to for years, and each task was randomly assigned to allow or forbid AI tools. With AI allowed, tasks took 19% longer. Before starting, the developers had forecast that AI would make them 24% faster. Afterwards they still believed it had made them 20% faster.
The sample was small, and the confidence interval ran from a 2% to a 39% slowdown. The developers were experts on their own code, which is where the earlier studies found the least benefit. METR does not claim the result applies to most developers, and now describes it as out of date.
Its follow-up, published in February 2026, recruited 57 developers working on more than 800 tasks with newer tools. The central estimates pointed to a speed-up of 18% for returning developers and 4% for new ones, but both ranges included zero. METR called the data “an unreliable signal”. Many developers declined to take part because they did not want to work without AI, and 30% to 50% of participants said they held back some tasks for the same reason. METR thinks developers are probably faster with AI now than in early 2025, but says its data is only very weak evidence of how much.
What the evidence means
Three conclusions hold across these studies.
- AI helps most where the worker knows least. Novice agents, weaker writers, lower-scoring consultants and junior developers all gained more than their experienced colleagues.
- Gains are largest on short, well-defined tasks. The biggest effect came from a single set exercise. Effects measured in everyday work were smaller, and for experts on complex and familiar projects the effect was negative in one trial.
- People judge their own speed badly. The METR developers were wrong about the direction of the effect, which is a reason to distrust surveys that ask workers how much time AI saves them.
Faster tasks have not yet become visible in pay. A study linking Danish adoption surveys to official employment records found no effect of AI chatbots on earnings or recorded hours, and ruled out changes larger than 2% two years after ChatGPT’s launch.
What would change the conclusion
These trials tested tools available between 2022 and 2025, and the tools have changed since. The evidence is also narrow. It covers support, writing, consulting and coding, and it measures speed more often than quality.
The picture would shift with large randomised trials of current tools that measure the quality of the finished work as well as its speed, and with wage or output data showing gains across whole firms. METR’s experience suggests such trials are getting harder to run, because fewer workers are willing to be assigned to the group without AI.
Frequently asked questions
Does AI actually improve productivity?
In most controlled studies, yes. Support agents resolved 15% more issues an hour, consultants completed 12.2% more tasks and developers at three large companies completed about 26% more. The gains were largest on short, clearly defined tasks and for less experienced workers.
Does AI make experienced workers more productive?
Less so. Senior developers gained 8% to 13% against 27% to 39% for junior staff, and the most skilled support agents gained little speed. In METR’s 2025 trial, experienced developers working on their own projects were 19% slower.
Do people know whether AI is saving them time?
Not reliably. The developers in METR’s trial believed AI had made them 20% faster when it had made them 19% slower. That is a good reason to be wary of surveys that simply ask workers how much time AI saves.
Are AI agents making people more productive?
The trials in this article tested AI assistants, which suggest text or code for a person to accept, not agents that carry out tasks on their own. None of the trials covered here tested agents, so this evidence doesn’t answer that question yet.
References and sources
- 01Generative AI at Work · Brynjolfsson, Li and Raymond, arXiv (version published in The Quarterly Journal of Economics) · 23 April 2023, revised 6 November 2024
- 02Study finds ChatGPT boosts worker productivity for some writing tasks · MIT News · 14 July 2023
- 03Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality · Dell’Acqua and co-authors, Harvard Business School Working Paper 24-013, hosted by MIT Sloan · September 2023
- 04The Impact of AI on Developer Productivity: Evidence from GitHub Copilot · Peng, Kalliamvakou, Cihon and Demirer, arXiv · 13 February 2023
- 05The Effects of Generative AI on High-Skilled Work: Evidence from Three Field Experiments with Software Developers · Microsoft Research · June 2025
- 06How generative AI affects highly skilled workers · MIT Sloan School of Management · 4 November 2024
- 07Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity · METR · 10 July 2025
- 08We are Changing our Developer Productivity Experiment Design · METR · 24 February 2026
- 09Still Waters, Rapid Currents: Early Labor Market Transformation under Generative AI (Working Paper 33777) · National Bureau of Economic Research · May 2025, revised March 2026
- 10Lead image: Coworking space 2026-09-27.jpg · JustWritingHere, via Wikimedia Commons (CC0) · Accessed 13 November 2026
House byline
Articles under this byline are researched and drafted with AI assistance, with every figure checked against its source before publication. The editor, Pankaj Guleria, reviews them and corrects them where needed.
Edited by Pankaj Guleria, Founder and Editor



