In general, AI is slightly improving the developer experience, though teams may spend more time fixing generated code than drafting it. Drafting code remains a particularly useful part of the process, so it remains to be seen whether AI will ultimately improve or hurt long-term effectiveness.
Right now, we’ve seen a 20–30% improvement across processes, but the tail end of repairs and fixes seems to be growing a little. Since we always get post-build adjustments, it is difficult to tell how much this is actually eating into that 20–30%.
Upsides
- Some very tedious boilerplate processes have been made much simpler.
- Large forms and similar repetitive development tasks are easier to draft.
- Scaffolding and testing the viability of an idea have become easier.
- Recent LLMs are getting better at finding real security issues.
Downsides
- We still frequently see generated code ignoring access control rules that undermine security. More specifically, part of the generated code may do proper checks while another part does not, which makes it easy to be lulled into a false sense of security.
- There can be a lower overall understanding of the entire code base.
- Security is a comprehensive process, and a lack of familiarity with the code undermines it, even if we eventually get to the point where the produced code is more secure.
- The overall developer learning process can be slower.
What We Are Doing to Mitigate the Minuses
- Reviewing all generated code — not just important pieces, not some of it, but all of it.
- Reviewing generated tests to make sure they are performing useful tests.
- Adding tests where needed to ensure our access controls are functional across the board.
- Depending on the project, alternating between human-drafted and AI-drafted code.
So far, AI is accelerating our draft development process. It still needs adjustments, and under most circumstances it is frequently much easier to change the code ourselves.
We have also found that the more experience a developer has, the better the overall product tends to be at the end of a cycle of working with AI.
