New Echo research finds Claude Mythos, OpenAI’s GPT-5.6 Sol, and GLM-5.3 agreed on just one of 121 potential security issues, meaning attackers and defenders using different models may be looking at entirely different weaknesses in the same software

NEW YORK, Nov. 18, 2026 (GLOBE NEWSWIRE) — Three of the most powerful AI models for finding software vulnerabilities looked at the same open-source code and came back with almost entirely different results, according to new research from Echo.

Echo, the trusted source for frontier-hardened software, tested Anthropic’s Claude Mythos 5.1, OpenAI’s GPT-5.6 Sol that’s accessed through Daybreak, its program for vetted security teams, and Z.ai’s GLM-5.3 against three widely used open-source projects using the same vulnerability-hunting framework and validation process. Of 121 distinct potential issues, only one was found by all three models, and 83% were found by just one of the models.

The findings raise complicated but fundamental questions about the emerging AI-powered cybersecurity landscape. Attackers can use the same models as defenders, and some, like GLM-5.3, are widely available with fewer guardrails. GLM-5.3 is open-weight and available to anyone, and Anthropic’s own research found it close to Claude Mythos Preview at building exploits. If each model finds different flaws, a company relying on one could miss the exact weakness an attacker finds with another.

The research also challenges the growing race to identify the single best AI model for cybersecurity. No model came out on top across every project, and the models regularly disagreed over whether the same issue was critical, minor, or not a vulnerability at all.

What the research found

  • Almost no overlap. Of the 446 potential findings first raised, only eight were flagged by all three models, and just one of those survived filtering. Of the 121 distinct findings left, 100 (83%) were found by a single model, 20 by two, and just one by all three.
  • No single model found everything. Mythos had the most findings pass validation overall, but GPT-5.6 Sol led on one project, and GLM-5.3 found validated issues that neither of the others spotted.
  • The models couldn’t agree on severity. One issue in an npm package, which has more than 350 million monthly downloads, was rated Critical by GLM-5.3, High by Mythos, and dismissed as documented behavior by GPT-5.6 Sol. Validation rated it Low.
  • More expensive AI didn’t guarantee better coverage. GLM-5.3 was the least reliable of the three at judging what could actually be exploited, but at about $22 to run, compared with $303 for Mythos and $30 for GPT-5.6 Sol, it still found validated issues Mythos missed.
  • Most AI-generated findings didn’t hold up. The models rated 45% of their 121 distinct findings High or Critical. After testing, only 26% held up at that severity. Without that validation, security teams would be buried in reports that never need action.

“Everyone is racing to figure out which AI model is best at finding vulnerabilities, but our research shows that’s the wrong question,” said Yuval Saggie, Security Researcher at Echo. “We gave three leading models the exact same code, and they came back with almost entirely different findings. Think about what that means when an attacker is hunting with one model and a defender is relying on another. The defender might never even see the vulnerability the attacker is about to exploit.”

The three projects tested are established and widely used, yet the models surfaced more than 30 high-severity issues, confirmed through testing within hours. As these tools spread, open-source maintainers will face more vulnerability reports than ever, while the businesses that depend on their code wait for fixes to work their way upstream.

“AI is making it dramatically easier and cheaper to find vulnerabilities, but protecting against them is a different story,” said Eylam Milner, co-founder and CTO of Echo. “Running multiple frontier models is only the beginning. You still need to validate findings, develop and test patches, and continuously deliver fixes across thousands of software components. As AI accelerates vulnerability discovery, the gap between finding security issues and actually fixing them is only going to grow.”

Echo Frontier was built with that in mind. It runs several frontier models to hunt for previously unknown vulnerabilities in open-source software, then validates the findings and builds and tests fixes. Customers receive those patches before public disclosure, through the components Echo already maintains for them, and Echo works with maintainers to get the permanent fix upstream.

The full research, including methodology and data, is available at echo.ai/frontier-research

Notes to Editors

Research methodology

Echo ran Anthropic’s Claude Mythos 5.1, OpenAI’s GPT-5.6 Sol (via its Daybreak program) and Z.ai’s GLM-5.3 once each against three widely used open-source projects: a Maven library with more than 24,000 GitHub stars, a Maven library with over 80,000 lines of code, and an npm package with more than 350 million monthly downloads, two Maven libraries and one npm package. All runs used the same Echo hunting framework, code version, and validation setup. The nine runs produced 446 findings, which became 121 distinct potential issues once repeats, unreachable issues, documented behavior, and false positives were removed. These are model-attributed passes rather than necessarily 42 distinct vulnerabilities, and the 121 include issues that were not independently validated. The 55 findings rated High or Critical went to automated proof-of-concept (PoC) testing, with GLM-5.3 used to create the PoCs, and 31 were confirmed at High severity. This is a controlled comparison rather than a full benchmark, and automated validation is not confirmation by project maintainers. Costs shown cover the vulnerability hunts only, based on published token pricing as of November 1, 2026. Including the GLM-5.3 proof-of-concept validation, costs were about $363 for Mythos, $45 for GPT-5.6 Sol, and $40 for GLM-5.3. Echo is coordinating responsible disclosure and will not name the projects tested or publish technical details until maintainers have been notified.

About Echo

Echo delivers frontier-hardened software across the software supply chain. From containers and libraries to VMs, OS packages, and more, Echo continuously vets and patches software against threats and vulnerabilities, including those discovered by frontier models before disclosure. Enterprises get secure-by-default software without changing their existing tools, versions, or workflows.

Learn more at https://echo.ai.

Media contact

Waters Agency for Echo
echopr@watersagency.com