Anthropic turned its Claude AI model loose on 25 major open source projects written in C, C++, Java, PHP and Python, and the results were not encouraging. The company says Claude flagged almost 500 security issues across more than five million lines of code, ranging from security controls that silently canceled each other out to sensitive code that ran before a user had even logged in.
One flaw, according to Anthropic, had sat undetected for 21 years.
Rather than hunting for individual bugs one by one, which Anthropic calls a losing game that "scales poorly and never guarantees security," Claude was set to a different task: mapping where software exposes more of itself than it needs to. The model built an inventory of more than 17,000 configuration options across the 25 codebases, and flagged 3,842 of them as security relevant.
What Claude actually found
The research focused on three failure patterns: features left reachable by default when nobody needs them, settings that look fine individually but undercut each other in combination, and complicated or sensitive code that executes before any security check runs.
All 11 of the scanned projects built to serve authenticated users turned out to contain code reachable by people who hadn't logged in at all. Anthropic counted 93 such pre-authentication attack surfaces sitting in default configurations, 37 cases of code executing ahead of authentication checks, and 61 configuration combinations that quietly eroded those pre-login protections. Across all 25 projects, 162 configuration combinations were accepted by the software without any documentation explaining what was supposed to happen when they overlapped.
The 21-year-old flaw fits that pattern. One setting forced clients to prove their identity using certificates. A separate setting, meant only to delay that check rather than skip it entirely, effectively switched it off when combined with the first. The two settings conflicted, and the software accepted both without complaint.
OpenSSH as the test case
Anthropic used OpenSSH, the software millions of servers rely on for secure remote access, as its main example. Claude found that the OpenSSH daemon could reach roughly 4,500 lines of experimental code implementing a post-quantum signature algorithm before the server had even confirmed that algorithm was allowed to run. The algorithm itself was switched off by default, but the code that parsed incoming requests for it was still reachable by anyone, logged in or not.
Anthropic reported the issue to the OpenSSH project, which shipped a fix in version 10.6. That update made roughly 20% of the previously reachable code unavailable to unauthenticated clients, according to Anthropic.
Anthropic says it has packaged the approach into a Claude "skill" that checks for risky default settings, pre-authentication code paths and conflicting configurations. The skill is headed for OSS-Scanner, Anthropic's free, opt-in scanning service for open source projects, though the company has not given a date for when that integration lands.
The system is also meant to check its own work. Anthropic says proposed fixes get validated through dynamic program tracing and tested against existing behavior before anything ships, with changes that could break things left for a human to sign off on.
Anthropic's pitch is that this kind of scanning cuts risk from vulnerabilities nobody has found yet, not just the ones already cataloged. Whether that holds up outside a controlled research exercise, and how OSS-Scanner performs once it's handling arbitrary projects at scale, is the next thing to watch.
ADFiled by The AI Desk
Models, assistants and the companies and chips behind them, reported from what was released and what was claimed, with the difference kept clear.
More from this desk →
Be the first to comment
Join the argument. No password, just your email or a passkey.