Ever since Anthropic announced Mythos in April, anxiety has been high over the model’s ability to discover vulnerabilities in every major operating system and web browser. At the same time, defenders have been experimenting with how to use Mythos defensively as part of Project Glasswing. However, new research suggests that getting defensive value from tools like Mythos might be easier said than done.

Today, software supply chain security company Echo released its Mythos Readiness report , studying nearly 40,000 CVE lifecycles across 250 widely used open-source container projects, alongside survey responses from more than 80 security leaders across the United States to conduct an independent analysis of Mythos.

The study found that of the 27 vulnerabilities Anthropic has publicly disclosed, only one of the eight findings Mythos originally rated “Critical” held up under independent review. Similarly, fewer than 10% of the model’s 23,019 candidate findings have undergone any external validation at all.

This indicates that Mythos might not be as effective at assessing the severity of vulnerabilities, despite its ability to discover and chain exploits. It also highlights the importance of having a human-in-the-loop when triaging vulnerabilities.

After Anthropic announced Mythos and Project Glasswing in April, selected organizations have been experimenting with the tool to prepare for an era of AI-driven vulnerability exploitation. In July, Michael Sentonas, President of cybersecurity vendor CrowdStrike, a partner of Project Glasswing, told Observer to “imagine a world where 200 vulnerabilities are discovered every day,” warning that such a reality wasn’t far away thanks to tools like Mythos.

Mythos is adept at discovering and chaining vulnerabilities. For instance, the UK’s AI Security Institute conducted evaluations of Mythos Preview and found that the model could execute multi-stage attacks on vulnerable networks, discovering and exploiting vulnerabilities autonomously, though the researchers noted they “cannot say for sure” whether Mythos Preview could attack well-defended systems.

While the offensive capabilities of Mythos raise concerns for defenders, its defensive capabilities also appear to be significantly limited. Eylam Milner, cofounder and CTO of Echo, who leads the company’s product and research function, told me in a video interview that “it finds a lot of problems. It doesn’t know how serious they are."

Milner added that it is an effective tool for finding vulnerabilities, “but you still need to review it independently to understand if it’s really exploitable.” He recommends that maintainers and users validate each and every vulnerability Mythos finds very carefully and highlights the need for a human in the loop due to the amount of noise the tools produces.

Criticism over Mythos is growing. “While I certainly think Mythos is impressive, I’d stop short of saying that it has completely lived up to the hype,” Erik Avakian, technical counsellor at Info-Tech Research Group and former state CISO for the Commonwealth of Pennsylvania, told me via email.

“What jumps out to me in this report is that AI is clearly changing everything we’ve known about vulnerability discovery. But discovery may no longer be the problem we need to solve. The Mythos severity findings are especially important. If only one of eight vulnerabilities initially rated as “Critical” held up under independent review, that tells me AI can dramatically increase the speed and volume of findings but without necessarily increasing our understanding of what actually matters," Avakian said.

Avakian adds that this creates a new problem for defenders, in that Mythos creates more findings, but more noise increases the need for human reviews and judgement to determine how the findings relate to the actual business.

More broadly, Echo’s study also found that the gap between vulnerability discovery and exploitation is closing, with the average time between vulnerability disclosure and first observed exploitation falling from approximately 1,216 days in 2018 to just 15 days in 2026. It also found significant risk in open source software, with 75% of exploited open-source vulnerabilities weaponized after the first day of disclosure.

From this perspective, defenders have a very small window to respond to vulnerabilities before threat actors have a chance to exploit them. Organizations thus need a high degree of readiness in terms of discovering and patching vulnerabilities if they want to keep up.

Milner notes there is a readiness gap between company expectations and their real world security postures. “Their perception is that they are more mature in their security than they really are,” adding that just a latency of one or two days can be the difference between downloading a malicious package or staying secure.

Elsewhere in the industry, other providers have outperformed Mythos in vulnerability discovery. In July, cybersecurity startup Wiz , acquired by Google for $32 billion in March 2026, announced that Project Atlas, a multi-model system had beaten Mythos on the CyberGym, a benchmark measuring how AI agents perform on real-world vulnerability analysis tasks. During Wiz’s testing, Atlas scored 90.9% compared to Mythos 83%, indicating more effective vulnerability discovery capabilities.

Beyond Mythos, AI vulnerability discovery remains useful for helping defenders to surface vulnerabilities faster, but it still requires oversight. “AI is helping defenders discover more vulnerabilities. Like any tool, AI can help with certain types of vulnerabilities in specific scenarios. LLM’s are good at writing code and that also makes them good at understanding code, finding vulnerabilities and writing exploits. In that sense, it is compressing the time between code creation and vulnerability discovery,” Jeff Pollard, Forrester VP and principal analyst, told me via email. That being said, Pollard notes that “AI slop is becoming a growing challenge across security operations, including vulnerability management and application security. Security teams are already drowning in vulnerabilities and it’s only going to get worse. Some of this is from low effort use of models which can lead to tons of bugs being discovered only to find the severity is low or mitigating controls exist. Or the AI was wrong,” Pollard said.

Part of the challenge, Pollard says, is that AI sounds authoritative in its outputs until it is questioned, and then its “built-in sycophancy has it doubting everything it confidently told you moments ago.” Considering this, he says some AI skepticism is healthy, but too much can be an issue, noting that CISOs have to equip their defenders with AI so that employees can learn to work with AI.

From this perspective, defenders must be cautious about overreliance on AI tools. This was also reflected in the recent Hugging Face breach , during which Hugging Face noted that multiple frontier AI models’ guardrails had refused to respond to prompts during the remediation process. In this sense, defenders need to build automated workflows with AI’s limitations in mind.