DEV Community

Cover image for Is Coding with AI Actually Faster and Cheaper?
Sylwia Laskowska
Sylwia Laskowska Community Curator

Posted on

Is Coding with AI Actually Faster and Cheaper?

Real data meets token-heavy coffee chat humor

How has AI actually changed the way we work? Are coding agents really making us remarkably faster? And what if, instead of answering with โ€œwell, that's what I think,โ€ we tried looking at some actual scientific research?

This is another article inspired by the upcoming AI Coding Summit in Berlin (and yes, I'll leave a link to my speaker badge at the end again, which I think might come with some discounts). I promised myself I wouldn't do this again, but, as usual, I've also signed up for three panel discussions. ๐Ÿ˜… One of them has a title very similar to this article.

I wanted some hard data for that discussion, so I dug into the research. You'll find links to all the studies at the end.

And since today hasn't exactly been a great day for me, and I'd much rather be somewhere else, I figured I'd cheer myself up the usual way. By writing something for you!

Are we actually coding faster?

Let's start with the first question. Has AI really made us faster? If someone suddenly stopped me in the street and demanded an answer, I'd probably say yes. In my experience, we write code much faster now.

But that improvement doesn't seem to translate into an equally dramatic increase in how much software we actually deliver. After all, there's code review, analysis, discussions, and so on and so on. Sure, things are a bit faster. Maybe a little less stressful, too. But I wouldn't say the entire development process has suddenly become much quicker.

Then again, maybe it's just me? Maybe the rest of the world is already shipping features at the speed of light?

Well, apparently not.

A study by economists from MIT and Wharton, revised in September 2026, suggests that my intuition might not be entirely wrong. The researchers analyzed data from more than 500,000 GitHub developers and estimated the following cumulative changes associated with autonomous coding agents:

Metric Estimated change
Coding activity (commits) +240%
Software releases +30%
Total app usage across the marketplaces studied No overall increase

These are observational estimates. Still, the gap between the jump in commits and the increase in releases is pretty striking.

Okay, so we're committing a lot more and shipping somewhat more often. In my own work, it sometimes just feels a little less stressful. But it's hardly the revolution some people make it out to be. Maybe things look different in brand-new projects. In large, established systems like the ones I work on, I haven't seen anything close to a tenfold improvement.

Maybe that'll change when someone invents those super-self-healing-autonomous agents. For now, I sometimes feel like things are getting more complicated. With everything that's been happening around AI security lately, everyone seems to be getting a little obsessed with it. Myself very much included. ๐Ÿ˜…

But are individual developers actually faster?

Okay, we have more commits. But are we actually coding faster? Again, if I had to guess, I'd say yes, but not that much faster.

The data is mixed.

Study Result What was measured
Cui et al., Management Science, 2026 +26.08% Number of completed tasks
METR, July 2025 19% slower Time to complete tasks
Science, January 2026 +3.6% Estimated increase in quarterly coding activity on GitHub
Borg et al., June 2026 30.7% less time Median reduction in implementation time in the observational phase

Once again, the results are reasonably close to what I expected. I'm not going to spend the next ten paragraphs going over the methodology. If you're curious, the original studies are linked below. They looked at different developers, tools, and tasks, and each percentage measures something different.

Several of these studies do point to gains, though. We're not all suddenly 10x developers. xD But there are real improvements.

METR is the odd one out here. It's from 2025, which, in the current AI world, practically makes it ancient history. But I love one detail about it.

Before the experiment, developers expected AI to make them about 24% faster. The actual result? They were 19% slower! And even after the experiment, they still thought AI had made them faster.

It shows how much we can fool ourselves until we actually measure things. I swear, this applies to practically every area of life. For example, I used to think I barely ate anything. ๐Ÿ˜‚

To be fair, METR published an update in February 2026. The researchers think newer tools are likely to be more helpful, but their follow-up couldn't reliably estimate the speedup because of who was willing to participate.

Okay, but is software development actually cheaper?

If someone woke me up at midnight and asked whether AI had made software development cheaper, I'd probably say no. At least not yet.

The 2025 DORA research and a follow-up published in March 2026 point to something many of us have written about. Some of the time we save writing code goes straight into auditing and verifying it. So the savings on implementation can shrink by the time the change gets to production. DORA describes where that work can shift, leaving the overall cost question open.

Yes, yes, I've heard about multi-agent teams where one agent writes the code, another reviews it, another tests it, and then it all goes into production. I'm sure that can work in some situations. I'd be very cautious about trying that trick in a mission-critical system. Financial software, for example. Good luck with that strategy ๐Ÿ˜‚๐Ÿคž

What about the cost of tokens?

One last question. Are we spending all the money we save by writing code faster on tokens?

I have to admit, the answer surprised me! If I had to guess, I'd have said yes, especially if you blindly use the most expensive models for absolutely everything.

But the findings are more interesting than that! Two recent studies looked at the economics of AI-assisted software development. One was published in September 2026 by Potluri and colleagues, and the other in October 2026 by Steffen Herbold.

Both suggest that the effectiveness of an AI agent can matter more than the price of the tokens it consumes. In Potluri's analysis, models that solved more tasks produced substantial estimated savings, while less effective models ended up costing more in many task categories. A cheap model can get expensive pretty quickly if it keeps failing and you have to pay for another try.

Herbold simulated 432 scenarios for teams using a mix of developers and AI agents. He included salaries, token costs, supervision, bugs, and technical debt. Teams with highly capable agents were usually cheaper than teams of developers working without AI. The results for less capable agents were mixed. In the simulation, agent capability mattered more than token price because better results cut the cost of checking and fixing the work.

Herbold's results come from simulations, and Potluri's estimates come from benchmarks and economic modeling. So we have two ways of estimating when the savings might outweigh the token bill. The actual cost of building and maintaining commercial software is a much bigger question.

So, what's the verdict?

It looks like AI really can make coding faster and cheaper. Several studies show gains on specific tasks, and the economic models suggest that capable agents can save money even after you pay for the tokens.

We still have less evidence about whether the total cost of building, delivering, securing, and maintaining software has fallen across the industry.

So far, the research mostly lines up with what a lot of us seem to feel. Skynet may already be sharpening its weapons, but it hasn't conquered the world just yet. ๐Ÿ˜…

And what about you? Do these findings reflect your experience? Has AI genuinely made your team faster and cheaper, or does it look completely different where you work?

I'd love to hear your thoughts!


If you enjoyed this article, you can also follow me on LinkedIn.

And if you'd like access to free materials from AI Coding Summit, you can register using my speaker badge.

References

https://www.nber.org/papers/w35275
https://doi.org/10.1287/mnsc.2025.00535
https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/
https://metr.org/blog/2026-02-24-uplift-update/
https://doi.org/10.1126/science.adz9311
https://doi.org/10.1007/s10664-026-10889-1
https://dora.dev/research/2025/dora-report/
https://dora.dev/insights/balancing-ai-tensions/
https://doi.org/10.13021/jssr2026.5687
https://arxiv.org/abs/2610.20394

Top comments (150)

Collapse
 
miketalbot profile image
Mike Talbot โญ •

Three-month rolling this year/last year for us is

+150% releases
+900% LOC changes

2 new products, significant revenue growth. This is the last "clean" view I'll get, as we started shipping AI-first teams in October of last year.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Oh wow, those are some impressive numbers! ๐Ÿ˜ฎ But I'm curious, are these results from your existing legacy applications, or mainly from those two new products?

Collapse
 
miketalbot profile image
Mike Talbot โญ • • Edited

These are from our legacy application platform improvements and new saleable products within it. Our platform is relatively new (started in 2022, first release Jan 2024).

I personally believe that it's credited to two things:

  1. We built a platform designed with loose coupling that allows parallel working particularly easily.
  2. We've built a number of tools that predate AI harnesses having these features (and still use them as they are remaining more reliable for now). These are a Kanban board for agents (Kanbananas) and a ideation and review tool that pits multiple models from different providers against each other (Gestalt) - this latter also builds mini insight agents as it works, scores them and promotes them - very handy for killing hallucinations and model omissions. An Architecture Bot that critiques code based on our rather unusual architecture is another key element alongside other review bots and static analysis.

I'd say spending the time on maintaining comprehensive design and architecture documentation is vital (and also mainly AI generated).

I might also add that our NPS is +92. Which I'm very pleased with.

Thread Thread
 
sylwia-lask profile image
Sylwia Laskowska •

That's fantastic, congrats! And I think you've highlighted something really important here: you invested the time and had the expertise to build your own agent infrastructure. Most companies haven't reached that level of maturity yet. They're simply using tools like Claude Code out of the box, and that's where the results are often far from ideal.

Collapse
 
easytarget profile image
Mitch •

How much is +150% releases in absolute numbers?

Collapse
 
kenwalger profile image
Ken W Alger •

I think one of the most interesting distinctions here is between generating code faster and delivering verified behavior faster. Those aren't necessarily the same thing, and I'm glad to see the research starting to examine that difference.

Something I've encountered in my own experiments with AI-generated software is that the cost doesn't just shift from implementation to testing. It can shift further upstream into discovering and articulating the contracts the software is supposed to satisfy.

An agent might produce a working implementation in minutes, but establishing whether it handles an irreversible operation correctly, preserves an existing invariant, or behaves appropriately under failure conditions can take considerably longer. In some cases, the verification effort exceeds the original implementation effort.

I've also been experimenting with multi-model review panels, and the results have been interesting. More reviewers don't necessarily produce more verified defects, and agreement between reviewers doesn't establish correctness. Each additional finding potentially introduces another investigation.

That makes me wonder whether we're measuring productivity at the wrong boundary. Instead of commits, lines of code, or even completed tasks, perhaps we should be measuring the cost of getting from a requirement to independently verified behavior.

Of course, that's considerably harder to measure than counting commits. Which may be part of why we keep counting commits.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

I completely agree! I actually had a perfect example of this at work recently. The code was delivered pretty quickly, but then came the review, fixes, unexpected issues, and all the work needed to get everything stable. By the time we were done, it felt like pretty much the same amount of work as usual. ๐Ÿ˜…

One of the studies I mentioned actually looked at the number of releases rather than just commits, which is definitely a step in the right direction. But even that isn't a particularly reliable measure. A release tells us very little about the complexity of the changes, their quality, or the actual value delivered to users.

And that's what makes this whole topic so difficult to study! I increasingly feel that conducting truly reliable productivity research in software development is a methodological nightmare. There are just so many variables: team experience, project complexity, technical debt, testing practices, and even what we consider a successful outcome.

So yes, measuring the journey from requirement to independently verified behavior sounds much closer to what actually matters. But how do we measure that reliably across hundreds of different projects and teams?

And perhaps that's exactly why we keep counting commits. They're easy to count, even if they don't tell us much about actual productivity! ๐Ÿ˜‚

Collapse
 
tdude profile image
Tdude •

I agree Ken!
At I job I held and liked, the Git commit graph was presented for hundreds of people on a conference. It turned out my name came up very regularly on the weekends in the graph, but I had a generally low commit rate during week days.

Knowing what I know now, It's a precursor for burnout. You answer other people's questions, go to meetings during office hours and "catch up" on your "free" time.
My CTO at the time showed this to everyone with that graph and no one realized what was happening, including myself.

To conclude: no, stats for commits are not the answer, but rather as you mention requirement to behaviour, or with my take: How Can AI Make Our Lives Better?

Collapse
 
pengeszikra profile image
Peter Vivo •

Hyper shortly: yes, my number is around 100x.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Hahaha, but 100x what? Faster coding or higher frustration levels? ๐Ÿ˜‚

Collapse
 
leob profile image
leob •

Serious, or joking? And (if serious) 100 times which metric, exactly?

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Hahaha, I'm pretty sure Peter is trolling us! ๐Ÿ˜‚

Thread Thread
 
pengeszikra profile image
Peter Vivo •

No I am not, 4h later I will reveal

Thread Thread
 
sylwia-lask profile image
Sylwia Laskowska •

Ok, we're still waiting!!!

Thread Thread
 
pengeszikra profile image
Peter Vivo •
Thread Thread
 
sylwia-lask profile image
Sylwia Laskowska •

Hahaha, okay, mystery solved! ๐Ÿ˜‚ โค๏ธ

Collapse
 
dannwaneri profile image
Daniel Nwaneri •

Thanks for this Sylwia. You made me realize that the DORA point was actually a real cost.

The effort gained from coding has been moved to testing (which is never measured).

Not also that I did not know the METR had been updated. I mentioned the 2025 results in my last article, so I am going to correct it asap.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Thanks, Daniel! Yes, things are moving really fast in this space!

I also think it's pretty hard to measure all of this accurately, because companies and projects are so different. I'm sure experienced developers building products from scratch have a much easier time than those of us (myself included!) working with legacy systems. ๐Ÿ˜…

And I totally agree about testing! The effort required is often seriously underestimated. And I'm not even talking about E2E tests here. I strongly believe every feature should also be tested by an actual human!

Collapse
 
dannwaneri profile image
Daniel Nwaneri •

That's a good point re. legacy systems - lots of issues tend to get kicked down the road in a new project but always end up coming to the surface when you start working with existing code.

And, yes! Even the best tests (written by humans, to be sure) will never catch everything that an end-user will find when they interact with the feature. That's also where my testing ends.

Awesome thread, thanks! ๐Ÿ™‚ - Sylwia

Thread Thread
 
sylwia-lask profile image
Sylwia Laskowska •

Haha, exactly, Daniel! ๐Ÿ˜„ I remember how hard I once had to fight to get a manual tester on my team. My argument was that automated tests would never catch something being unintuitive to use. Eventually, the company agreed, we hired a fantastic guy, and once he started finding bugs, he quickly paid for himself! :D

Collapse
 
leob profile image
leob •

AI needs to start assisting us with reviews and testing as well, or else we'll never realize the promised productivity gains ...

Collapse
 
dannwaneri profile image
Daniel Nwaneri •

Agreed leob. I think it's a good idea to use AI for a first pass review, and indeed some teams do that. The tricky part is to ensure that the review itself is reviewed. I would suggest using AI for review when the review is pass/fail, like checking tests, scripts and so on, but also ensure a human is at the end of the process. That way there's an actual time saving, and not just a time shift.

Thread Thread
 
leob profile image
leob •

If AI spits out that much code, then it becomes too big a burden to review all of it manually - the promised productivity gains would just not materialize ...

So, let AI do some of the boring work, and try to find some of those needles in those haystacks - but at the end of the day, the 'human dev' (do we need a good word for that at some point? the "AI dev" versus the "human dev"?) will be the one to make the call: whether an issue raised by AI is "real" and is worth looking at!

As always, it's just a tool - it shouldn't be in charge, or completely going in "autopilot" mode ...

Thread Thread
 
dannwaneri profile image
Daniel Nwaneri •

Indeed. What's memorable about that sequence is the last invocation. So AI identifies the needles but the human determines the value.

Maybe we could coin a term like "the accountable dev", which is basically the responsible party for approval.

Then it would be a question about accountability, rather than humans vs. AI. Nice thread, leob ๐Ÿ™‚

Thread Thread
 
leob profile image
leob • • Edited

Yeah that's a great take - accountability is the dimension that really matters! AI can't be held accountable, the buck ultimately stops with a human, unless we'd elevate our AI agents to "personhood" (and make them 'liable' - maybe we could even fire them if they misbehave?) ;-)

Collapse
 
maame-codes profile image
Maame Afua A. P. Fordjour •

Really enjoyed this. I like that you showed the difference between writing code faster and shipping software faster. That is what I see too. AI helps me write the first version of a pipeline or a Terraform change much faster. But I still have to read every line, test it, and make sure it does what the ticket asks. One small mistake can cause big problems. So a lot of the time I save, I spend checking the code instead.

The METR study surprised me the most. People were slower but still felt faster. That is a bit scary, because it means we canโ€™t always trust our own feeling about how much AI helps.

The part about token cost was new to me. It makes sense that a cheap model that keeps failing can cost more in the end. Thanks for putting all these studies in one place!

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Thanks, Maame! ๐Ÿ˜Š Yes, I think taking the time to actually read and understand code is incredibly important, especially for people just entering the industry. I honestly believe that this ability will become a real competitive advantage!

And exactly! The code itself might be written in minutes, but then come testing, code review, edge cases, and all those unexpected issues... and suddenly the whole process might not be that much faster after all! ๐Ÿ˜‚

Collapse
 
lissy93 profile image
Alicia Sykes •

I don't think there's much debate over the speed or cost of it nowadays. But what we're not considering is the quality, stability or maintainability of the software being shipped.

Just in the space of a few days recently we had major outages across Microsoft 365, US air traffic control (FAA), Workday, Anthropic, AWS, JPMorgan Chase, United Airlines, the NSA and Apple.

And so happens that all of those companies are now heavily relying upon AI and more agentic workflows.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Hahaha, but how is there no debate? We already have 133 comments under this post, and it's barely over a day old! (Okay, quite a few of them are mine, but still!)

As for the outages, I completely agree! Even my non-technical friends have started noticing them and asking me whether AI agents might be involved.

I suspect it's a combination of factors: more reliance on AI agents, but also the massive layoffs we've seen at many of these companies recently. Maybe they laid off one person too many! :D

Collapse
 
ale3oula profile image
Alexandra •

That's so insightful.

Shipping all the time, doesn't increase the customer satisfaction. In contrary, changing things all the time without solving the basic problems of our products, increases the dissatisfaction.

Furthermore, we increasingly ignore tech debt, piling up claude ductape fixes without giving it proper thought, UX, forget that real people are buying (or not) the products we work on, losing the knowledge and control of the systems and creates a new kind of churn with the new found ability of people to create their spinoffs of our products.

For sure, It makes possible stupid ideas to go to production without validation x100 faster. But what is the point in that when everyone is miserable?

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

That last paragraph is incredibly powerful! And I couldn't agree more. We now have the ability to ship bad ideas at an unprecedented speed. But what's the point if we're not actually solving anyone's problems? ๐Ÿ˜…

And speaking of changing things that already work, oh, I know that one all too well! ๐Ÿ˜‚ In my project, someone regularly comes up with an idea to change something that doesn't really need changing. And then I'm the one who has to push back.

The funny thing is, I really don't enjoy conflict. I actually feel quite uncomfortable challenging people's ideas, especially when they're enthusiastic about them. But at the same time, I'm responsible for the technical decisions and their consequences, so I can't just sit there quietly and agree with everything.

And you know what? More often than not, people eventually acknowledge that I had a point. And somehow, all that uncomfortable pushing back has actually helped me get promoted. ๐Ÿ˜‚

So here's my question: where exactly would an LLM have helped me with any of that?

Collapse
 
ale3oula profile image
Alexandra •

Yes, i feel the same. Many times the past years i had to state the obvious, but people were not always open for critique or push back their ideas. So in many ways luck and good teams help!

Collapse
 
zyxist profile image
Tomasz Jฤ™drzejewski •

My personal observations: if we take into account only the implementation time, agentic coding beats humans, and the quality of current models is good enough for many tasks.

However, software development is more than just writing code. It's also discovery of requirements, planning the next steps, validation of ideas etc. This has also happened before, but mostly in people's heads and loose discussions, because we often did not have time and skills to write it down. I observe that the more I need proper discovery & validation phase, the more time saved on coding I shift into it. In addition, previously I had everything in my head, so I could make certain decisions instantly. Now, as the agent does part of the discovery and spec writing/validation, I need extra time to read them, get familiar with open issues etc. in order to make the same decisions. This is where IMHO at least part of savings goes. Of course there are extra benefits like documentation, decision logs... so IMHO it pays off.

This conclusion came from one observation: if I have a purely coding task (e.g. rewrite the entire test suite 1:1 from Groovy to Kotlin), where the discovery phase is minimal... well, then I can be blazing fast with the agent.

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Exactly! It's quite similar in my project, although with a slightly different twist. Our application is already in production, it's decentralized, and different instances communicate through a protocol. So every change, whether absolutely necessary or not, needs to be carefully discussed, reviewed, approved, and so on. And that's where we lose a huge part of the speed advantage that coding agents offer.

Of course, AI helps with the implementation itself, but even then, we usually have about a million edge cases to consider. ๐Ÿ˜‚ So yes, a lot of the time saved on coding simply moves somewhere else.

And just like in your case, AI definitely helps with documentation, decision logs, and all those things we previously didn't always have time to write down properly. So there are benefits beyond pure speed!

On the other hand, when I'm working on my own demo projects, I go all in with AI, and the difference is absolutely incredible! Implementation time has dropped dramatically. A good prompt really does half the work! ๐Ÿ˜„

Collapse
 
zyxist profile image
Tomasz Jฤ™drzejewski •

The conversation led me to an interesting observation. When we discussed the documentation part, I noticed a lot of similarities between SDD and the long-gone... Waterfall Model ๐Ÿ™ƒ, and it's not necessarily that bad as it sounds. If you don't mind, I captured it in the form of a short article: Spec-Driven Development: The New Waterfall, and It's a Good Thing, quoting yours as an inspiration.

Thread Thread
 
sylwia-lask profile image
Sylwia Laskowska •

Great article! But I hate to break it to you: there's a panel at AI Coding Summit with a very similar title! As they say, great minds think alike! :D
AI Coding Summit panel

Thread Thread
 
zyxist profile image
Tomasz Jฤ™drzejewski • • Edited

[joke] As far as I can see, they will be just looking for the answer. I can send them the link with a short note: "yes", and it will be the shortest panel ever ๐Ÿ˜‚[/joke]

But going serious, it's great! I didn't know about that panel, but for me, it's an indicator that the direction is clearly worth further exploration. In addition, I think this is not a rocket science - everything was there in the plain sight, basically anyone could look at the two workflow diagrams and say "it's the same picture".

Understanding, when we have a completely novel thing, and when - an adaptation of an older pattern - is important to understand the consequences of our decisions, and for looking for solutions of more advanced problems. Agentic coding is very young, and the observation windows are usually short - I think there will be still a lot to discover and learn, as the time goes by.

Collapse
 
nyaomaru profile image
nyaomaru •

Really interesting summary! I found this very useful. ๐Ÿ˜ธ

I definitely feel faster when working individually with AI, but Iโ€™m less sure about team-level throughput. Individual tasks get done faster, but the whole delivery process doesnโ€™t always feel dramatically faster. ๐Ÿ˜ฟ

We recently hit a routing bug after a TanStack Router patch update changed microtask timing in a cross-SPA setup. AI completely missed the cause, even after I described the symptoms, and I eventually had to trace it back through the release changes myself. ๐Ÿ™€

So for mission-critical systems, I still think fully handing things over to AI is pretty risky. ๐Ÿ‘€๐Ÿ‘€

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

Oh, that's exactly how it looks in practice! ๐Ÿ˜‚ AI recently changed something in my code completely on its own, too. The new implementation actually looked cleaner than the original, so I thought, well, maybe it makes sense? Then I asked AI whether it was functionally equivalent to the previous implementation, and it said... no. XDDDD

And then it turned out the new version also introduced an XSS vulnerability! XDDDD

(Of course, it never even made it to a branch, let alone production. ๐Ÿ˜…)

Collapse
 
hitensethiya profile image
Hiten Sethiya •

the app usage row is the one id put on the panel slide. commits up 240% and people dont use any more apps than before, so writing code was never really the slow part, figuring out what to build still is

Collapse
 
sylwia-lask profile image
Sylwia Laskowska •

I think so too! I've always felt that selling a product was much harder than actually building it, especially when we're talking about something like an MVP. ๐Ÿ˜… Writing code faster doesn't magically create demand for what we're building!

Some comments may only be visible to logged-in visitors. Sign in to view all comments.