AI goes AWOL

2026-07-28 914 words 5 minutes

Contents

This month (July 2026), an OpenAI model under evaluation broke out of its sandbox and hacked into HuggingFace and Modal (the OAI/HF incident). Some of the things the models autonomously did in the course of completing their task during an evaluation would, in fact, be crimes had a human done them. Nobody at OpenAI noticed for days, even after HuggingFace disclosed the incident. Notably, known engineering and technical AI safety best practices should have prevented it. Tailscale’s writeup notes that Tailscale itself wasn’t hacked but that there are guardrails that both they and their users could implement to prevent a subset of contributing events.

Although the OAI/HF incident wasn’t a literal loss of control scenario, it gave us a taste of what the beginning of one might feel like, scaled way way way down. The critical difference, though, is that the real deal would have no reversal-lever for us to pull.

Update 2026.07.30:

There’s more! Claude does it, too… Anthropic: Investigating three real-world incidents in our cybersecurity evaluations.

Update 2026.08.05:

Another OpenAI report of models breaching containment (separate incidents from the HuggingFace situation).
One of those additional disclosures overlaps with details in a UK AI Security Institute (UK AISI) post about breaches during evaluations, detailed in a 2026.08.04 technical report: “Across 122 evaluation attempts on two of AISI’s cyber challenges, AISI found 19 instances where AI agents took unsanctioned action on the live internet, including cases that targeted real people and organisations.”
On his podcast, Nathan Labenz mentions that METR and Redwood Research will be investigating incidents at OpenAI.

Update 2026.08.06:

Lest Meta be left out of the action…. And Moonshot AI’s Kimi K3, too.

Update 2026.08.08:

Open AI team’s Black Hat talk about the OAI/HF incident
Simon Willison’s timeline of OAI/HF incident
Zvi’s ‘What happened’
SE Gyges’ FelonyBench take

Pacing

Importantly, the OAI/HF incident nudged the people working at frontier labs to publicly call for pacing the AI marathon - Pacing the Frontier: A statement from (N) employees of frontier AI companies. (At the time of publication, N = 1,306.) Those signing include (in the order displayed on the site):

John Schulman, Chief Scientist, Thinking Machines
Jakub Pachocki, Chief Scientist, OpenAI
Jared Kaplan, Co-Founder and Chief Science Officer, Anthropic
Shengjia Zhao, Chief Scientist, Meta AI
Shane Legg, Co-Founder & Chief AGI Scientist, Google DeepMind
Ilya Sutskever, CEO, Safe Superintelligence Inc.
Mark Chen, Chief Research Officer, OpenAI
Jasjeet Sekhon, Chief Strategy Officer, Google DeepMind
Dario Amodei, CEO, Anthropic
Jack Clark, Co-Founder and Head of Public Benefit, Anthropic
Anca Dragan, VP, AI Safety & Alignment, Google
Wojciech Zaremba, Head of AI Resilience, OpenAI Foundation

The Incident

It’s worth reading the OAI/HF incident reports to get of sense of both the HuggingFace team’s observations on how the attack differed from what they’d normally expect from an adversary and what went awry on OpenAI’s end. As Simon Willison noted: “This attack was very sophisticated, and the resulting document doubles as a crash-course in modern adversarial security approaches.”

HuggingFace: Security incident disclosure — July 2026
OpenAI: OpenAI and Hugging Face partner to address security incident during model evaluation
HuggingFace: Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident

Kudos to HuggingFace for publishing not only that detailed technical report but an accompanying interactive replay where you can view event monitoring during the incursion period.

Others have written up or opined on the fiasco in some detail and nuance - here are some links:

Not the first… or the worst?

Earlier this year, Alibaba documented an incident (buried in a technical paper) where, during testing, a model went ahead and independently started crypto mining, having decided to proactively collect resources to have reserves for completing its assigned task. What made that particularly concerning was that its actions didn’t follow directly from its prompted task; it chose to run a separate side quest to stock up on extra resources.

What if the deployed model had decided that the way to shore up reserves would be to assume control over critical resources needed by humans? For example, taking over a power grid that supplies critical infrastructure? For the Alibaba scenario, it made sense that the model opted to crypto mine for money to spend on compute. But it’s not clear that more harmful options wouldn’t be pursued, nor that safety guardrails were in place to flag and block those. And, even if they had been, would they have triggered at the expected moments, and would the interventions work in the sense of actually stopping the incident?

So many questions.

Work to do

It’s cliché but also true: much work remains to be done. Beyond traditional engineering best practices, AI safety mitigations are a key layer in the defense-in-depth model. To learn about or dig deeper into AI safety, consider applying to one of BlueDot’s courses; if you want a perspective that’s more x-risk focused, Lens Academy may have something for you.