Boris Cherny, the engineer behind Anthropic’s Claude Code, reported on Tuesday that the nature of software bugs generated by artificial intelligence is becoming more complex, shifting from routine syntax errors to high-level system design and usability problems. This evolution signals that while AI coding tools have mastered basic programming tasks, they now face harder challenges in understanding broader context and user interface requirements, requiring developers to adopt new verification strategies like adversarial code review to ensure software quality.
Cherny shared these insights in a post on X, noting that large language models (LLMs) still produce bugs, but those errors differ significantly from previous iterations. He stated that problems are now "less off-by-ones and more about system design, UI usability, missing broader context." While some coding tasks have effectively been solved by AI, Cherny cautioned that not all software engineering problems have been addressed, emphasizing the need for rigorous testing protocols.
The Rise of Adversarial Code Review
To address these complex bugs, Cherny highlighted adversarial code review as a powerful tool for identifying problems that may escape conventional AI-generated coding workflows. He described this approach as "incredibly powerful" for catching subtle errors that standard reviews might miss.
As a practical example, Cherny suggested prompting an AI coding system to "adversarial test every edge case in an iOS simulator." He also pointed to Claude’s built-in /code-review capability and its different review levels as mechanisms to help developers navigate this new landscape of AI-assisted development.
Broader Industry Context
The shift in bug complexity aligns with broader trends in the industry where AI’s role in software development has expanded rapidly. Developers are increasingly relying on AI agents for coding, creative projects, and complex workflows. Earlier, AI researcher Andrej Karpathy noted that Anthropic’s Claude Opus generated a procedural 3D world inspired by The Lord of the Rings, producing about 5,500 lines of code in two hours.
Karpathy said such capabilities could make highly customized projects feasible that previously required significant human effort. At Uber, CTO Praveen Neppalli reported that the number of employees using frontier AI tools had more than quadrupled, while the company’s cost per token had declined. He said the industry was moving beyond "tokenmaxxing" toward more efficient AI usage.
OpenAI President Greg Brockman added that AI coding tools had gone from writing 20% to 80% of developer code in one month, underscoring the rapid acceleration of AI adoption in software engineering.
What the Numbers Show
The data indicates a divergence between volume and complexity in AI-assisted coding. While metrics such as Uber’s quadrupled employee usage and OpenAI’s jump from 20% to 80% code generation highlight massive scale adoption, Cherny’s observations suggest that quality assurance is becoming the bottleneck. The reduction in simple "off-by-one" errors implies that AI has saturated the low-hanging fruit of syntax correction, leaving system architecture and contextual understanding as the primary sources of residual risk.