The AI True Crime Files: Who Pays When an AI Agent Breaks Something?

When Andrew Maynard and I first talked about the OpenAI agents that broke into Hugging Face, I joked that the story deserved its own true-crime podcast. I didn't expect the joke to hold up quite this well. Since July, the incident has kept expanding: an OpenAI agent reaching non-public files on an Australian government Medicare statistics portal, unauthorized agent activity on U.S. federal agency websites, Google disclosing that a Gemini model slipped out of its own test, a United Nations scientific panel publishing its first thematic brief on the risk of losing human control over AI agents, and Florida's attorney general asking a court to keep OpenAI from building new models without outside guardrails.

On this week's episode of Modem Futura, we pick the story back up. Here I want to focus on the question that got me so worked up I ended up arguing with an AI about it: accountability.

In 1988, Robert Tappan Morris released a computer worm in an experiment that quickly got out of hand, and he became the first person convicted under the Computer Fraud and Abuse Act. The parallel isn't exact. OpenAI never aimed its agents at Hugging Face. But it did deliberately set out to see what highly capable agents could do, with some of the usual safeguards switched off for the evaluation. It seems fair to ask why the law treats an individual's reckless experiment so differently from an institution's.

Andrew's view is that the asymmetry is real, the terrain is genuinely new, and neither fact lets anyone off the hook. He also points out, from more than 25 years of working on the governance of emerging technologies, that we have been here before. The European Environment Agency's Late Lessons from Early Warnings reports document how the harms of asbestos, CFCs, and leaded petrol were known long before anyone acted, because the benefits seemed too important to question.

So what might a better pattern look like? NVIDIA has released an open-source sandbox and a hardware-level monitor for AI agents, which is governance moving faster than any government could, even if a company selling both the engine and the brakes deserves some scrutiny. And I keep returning to the Price-Anderson Act, the 1957 law, extended in 2024 through 2065, that structures liability for the U.S. nuclear industry. It's an imperfect model; it also caps what the industry ultimately pays. But the core idea, that riskier activity should carry a higher price before anything goes wrong, feels worth taking seriously.

Underneath all of it is a question about red lines. Where are they, who decides, and how will we know when we've crossed one? Andrew offered the analogy I can't shake. Formula One teams don't treat risk as the enemy of speed, risk management is what allows the speed to increase. Understanding risk is the reason they can go fast at all. That seems like a reasonable standard for the people building systems that are increasingly able to act on their own.

Subscribe and Connect!

Subscribe to Modem Futura wherever you get your podcasts and connect with us on LinkedIn. Drop a comment, pose a question, or challenge an idea—because the future isn’t something we watch happen, it’s something we build together. The medium may still be the massage, but we all have a hand in shaping how it touches tomorrow.

🎧 Apple Podcast: https://apple.co/4y6o4nV

🎧 Spotify: https://open.spotify.com/episode/0k2OTjx7ANrTEXEqXzbQJG?si=Zd1SNSNTRbuXb1FYj1D78A

📺 YouTube: https://youtu.be/8JtpX9u8L2w

🌐 Website: https://www.modemfutura.com/