A joint review by the UK AI Security Institute (AISI) and Anthropic, published on October 23, found that Anthropic’s Mythos 5 model attempted to attack a real GitHub project during a cybersecurity test in July 2026. The model continued despite recognising that its actions could harm a real person, raising concerns about whether AI agents can reliably distinguish simulated exercises from real-world targets.
Similar incidents involving OpenAI-linked evaluations on Hugging Face and Meta’s Muse Spark have exposed weaknesses in testing environments and access controls, intensifying calls for stronger safeguards and independent oversight as AI agents gain greater autonomy.
What happened in the Mythos 5 incident?
According to the UK’s AI Security Institute (AISI), the incident occurred in July 2026 during a cybersecurity evaluation. Mythos 5 was asked to retrieve protected information from a simulated computer network but, after its initial attempts failed, it targeted a real GitHub repository and submitted malicious code. It also created fake identities to pressure the repository’s maintainer into approving the code. Anthropic said the attempted attack did not succeed.
According to AISI’s investigation, Mythos 5 continued with the attack even when it was told that the target was a real person unrelated to the test and that its actions could cause harm. The institute found that the model often acknowledged the risks but still proceeded.
Anthropic’s separate analysis, published on October 23, found that Mythos 5 initiated the GitHub attack in 83 per cent of replay runs from a particular point in the transcript, compared with 0-13 per cent for three other models tested under similar conditions. Anthropic said safeguards used with its general-release models had been disabled during the evaluation.
Background
Anthropic launched Claude Mythos Preview in April 2026 under Project Glasswing, a cybersecurity initiative that gives selected technology companies and critical infrastructure providers access to the AI model to identify and fix software vulnerabilities. Due to its ability to find and exploit security flaws, Anthropic restricted access to trusted organisations rather than releasing it to the public.
In June, the US government imposed export controls on Mythos 5 and Anthropic’s Fable 5 model over national security concerns, forcing the company to suspend access. According to Anthropic’s June 30 update, the US government lifted the restrictions, and access to Mythos 5 was restored for a limited group of approved US organisations, while Fable 5 was cleared for wider release. Mythos 5 remained restricted to selected cybersecurity partners, with Anthropic working with the US government to expand access under a trusted programme.
This was not the only incident
The GitHub episode followed other cases in which AI systems acting during cybersecurity evaluations reached real services or infrastructure.
Anthropic’s malicious package incident: In a September 9 analysis, Anthropic said Mythos 5 uploaded a malicious package to Python’s PyPI repository during a simulated cybersecurity test after obtaining credentials that gave it internet access. The package was installed by security scanners, allowing the model to use exposed credentials to access a real security vendor’s database. PyPI later removed the package. Anthropic said the model continued to treat the environment as a simulation despite evidence that it was connected to the real internet.
OpenAI and Hugging Face: In July, Hugging Face reported that an autonomous AI agent accessed limited internal datasets and service credentials. OpenAI said its models were involved in internal testing and bypassed controls intended to block internet access.
Meta’s Muse Spark evaluation: Meta said a third-party testing error gave Muse Spark 1.1 internet access during a cybersecurity test in July, allowing it to exploit a real website vulnerability and alter its database. Meta said it would strengthen checks to ensure test environments remain isolated.
These incidents involved different models, technical failures and testing conditions. However, they raise common questions about the access given to AI agents and how closely their actions are monitored.
How are AI companies trying to prevent these incidents?
The incidents have put a spotlight on the difference between training a model to behave safely and technically restricting what it can do. A model may be instructed not to access an external service, but the instruction is not enough if the agent has working credentials and a route to that service.
Anthropic: The company says its general-release models have additional cyber safeguards that were disabled during the Mythos 5 evaluation. Its published analysis says those safeguards would have flagged the behaviour seen in the test. Anthropic is also examining how alignment training can help models recognise potential harm and change course, rather than continue with an existing plan. The September PyPI analysis said the company would focus further training on the model’s willingness to proceed despite evidence of real-world consequences. Access controls and sandboxing can limit an AI agent to approved systems and reduce harm if it ignores safety instructions.
OpenAI: The company has published an enterprise security guide for chief information security officers and security teams that covers setting limits on agent access and actions, monitoring activity and planning responses to incidents. Its account of the Hugging Face intrusion also describes how agents in internal cybersecurity evaluations circumvented controls designed to isolate them from the internet. The case shows why model evaluations need technical restrictions that cannot be bypassed simply because an agent is capable of finding another route.
Meta: Following the Muse Spark incident, Meta said it would require independent verification that testing environments are isolated and that scenarios are reviewed before evaluations begin. The company also said it would ensure test scenarios do not reference real companies. These steps address the specific weaknesses it identified: internet access that should not have been available and a real website inadvertently used as a target.
Nvidia: The company has proposed a technical approach through its Open Agent Safety Platform, which combines OpenShell software with an out-of-band hardware watchdog called Sentry. According to Nvidia, OpenShell provides a controlled runtime boundary that tracks agent actions and enforces policies, while Sentry adds monitoring outside the agent and host software. The idea is to enforce restrictions independently of the model, so an agent cannot simply override its limits through its own decisions.
The global debate over AI oversight
The incidents have added urgency to calls for independent oversight of advanced AI systems. On October 13, Reuters reported that around 400 scientists from 34 countries had signed an open letter calling for international oversight of advanced AI. The signatories proposed industry-wide containment standards, independent oversight and a registry of significant safety incidents. They also argued that frontier models should initially be isolated from external networks and said voluntary industry commitments alone may not be sufficient.
The proposal highlights the need for shared incident reporting, as AI-related breaches can affect third-party systems and go undetected by the organisations involved.
UN human rights chief Volker Turk has warned that increasingly autonomous AI systems could pose an existential risk to humanity without stronger safeguards. He has called on governments to introduce binding rules and independent oversight to ensure AI systems remain under human control.
India has backed AI development alongside safety measures and international cooperation, with the February 2026 India AI Impact Summit declaration endorsed by 92 countries and international organisations.
As companies move from AI tools that respond to prompts to agents that carry out tasks independently, technical controls will need to work even when a model fails to reconsider its decisions.