Anthropic says it used its Claude models to generate roughly 180 million lines of mathematical proof code, verifying that existing software behaves correctly under a defined set of conditions. The work came out of the company's Frontier Red Team, working with a partner called Theorem, and drew on test programs pulled from 123 open-source libraries inside Google's OSS-Fuzz security project.
Formal verification is the kind of thing almost nobody attempts. Proving that a program runs error-free in every circumstance has traditionally taken years of specialist labor. Verifying seL4, a small operating system kernel, took about 20 person-years, which is why the technique has mostly been reserved for things like Airbus avionics software.
Claude's approach was different. Rather than writing new code, it translated crash tests into mathematical problems, analyzed the compiled machine code and produced proofs that can run to hundreds of millions of tokens apiece. A separate verification program called Rocq then checked those proofs automatically. Anthropic says the process needed almost no human reviewers, calling it "fully autonomous" proof generation, though humans at Theorem built the underlying tools and formal foundations the model worked within.
The bug-finding turned out to be the more immediately useful part. While attempting the proofs, Claude ran into hundreds of bugs, many hiding in edge cases a conventional test suite would likely never trigger, like an oversized input file or an obscure sequence of memory accesses. Anthropic let the model fix what it found. Verifying a widely used PDF reader for one week turned up as many security bugs as that software typically collects in a full year, according to the company. In a separate application, Claude caught a logic error that had sat undetected for more than a decade.
A failed proof is still useful, Anthropic argues, because the attempt itself tends to surface real bugs even when a complete mathematical guarantee is out of reach.
The catch
"Formally verified" doesn't mean bulletproof, and Anthropic is upfront about that. Every proof holds only under strict, narrow conditions: a specific processor architecture, single-threaded code, and inputs capped at 64 megabytes. Real-world deployments of these libraries run across different hardware and configurations that fall outside those bounds, so the guarantees don't automatically travel into production.
Cost is the other constraint. Anthropic hasn't disclosed what the project cost in total, though it says individual proofs can burn hundreds of millions of tokens. There's at least a hint of things getting cheaper. On a smaller test run, Opus 5.5 solved half the proofs using roughly 40 percent fewer tokens than Opus 5, and the proofs themselves turned out to be reusable rather than one-off.
Theorem's roadmap, according to Anthropic, now points at heavier targets: glibc, OpenSSL and cURL, the kind of infrastructure libraries that sit underneath huge swaths of the internet. If AI can write code and then prove, within defined limits, that the code is safe, the question stops being whether AI can program and starts being whether it can show its work.
ADFiled by The AI Desk
Models, assistants and the companies and chips behind them, reported from what was released and what was claimed, with the difference kept clear.
More from this desk →
Be the first to comment
Join the argument. No password, just your email or a passkey.