Software Verification: AI Proves Code Safety at Scale

A new AI system just changed how developers might verify software safety.

Anthropic has demonstrated a method that allows its Claude model to automatically prove that existing code is correct. This approach addresses a significant bottleneck in software development, where mathematical verification was previously too expensive and time-consuming for widespread use. The system works by translating software behavior into mathematical problems, which the AI then solves to create formal proofs of correctness.

The project was a collaboration between Anthropic’s Frontier Red Team and a partner company called Theorem. Over a few weeks, Claude processed test programs from 123 open-source libraries within Google’s OSS-Fuzz security project. The AI generated approximately 180 million lines of mathematical proof code. This volume of work would have taken human specialists decades to complete manually. An independent verification program named Rocq checks these proofs, reducing the need for human reviewers in the final stages.

The primary value of this system is its ability to find bugs that standard testing overlooks. Claude identified hundreds of defects during the proof generation process and was authorized to fix them. Many of these issues only appear under very specific conditions, such as unusually large input files or specific memory access sequences. In one notable case, the team found as many security bugs in a single week for a popular PDF reader as are typically reported in an entire year. In another instance, the AI caught a logical error that had remained undetected for more than a decade.

Despite the impressive results, the method has strict limitations. Anthropic notes that a "formally verified" status does not guarantee the software is completely error-free. The proofs only hold under specific conditions, including a particular processor architecture, single-threaded execution, and input sizes under 64 MB. Real-world applications often run on diverse hardware and in more complex configurations, meaning the verification covers a specific slice of potential behavior.

The computational cost remains a factor. Individual proofs can require hundreds of millions of tokens to process. While Anthropic has not disclosed the total project costs, the efficiency gains are evident in the models themselves. Newer versions of the AI, such as Opus 5.5, solve half the proofs using about 40 percent fewer tokens than the previous Opus 5 model. This trend suggests that the process will become more efficient as the underlying technology improves.

This development shifts the focus of AI coding from simply generating code to proving its safety. If AI can verify its own work under defined conditions, developers can trust automated software more and spend less time on manual review. This could reduce costs across the entire development lifecycle. Theorem’s roadmap already targets critical infrastructure components like glibc, OpenSSL, and cURL, indicating a move toward verifying the foundational software that powers the internet.

For the tech industry, this represents a significant step toward automation. The question is no longer whether AI can write code, but whether it can guarantee that code is safe. As the models become more efficient and the process more routine, formal verification with AI may become as standard as basic testing. This could lead to more reliable software in critical systems, from avionics to web servers, without the prohibitive costs that previously limited such methods to extreme cases.