The AI True Crime Files: Who Pays When an AI Agent Breaks Something?

When Andrew Maynard and I first talked about the OpenAI agents that broke into Hugging Face, I joked that the story deserved its own true-crime podcast. I didn't expect the joke to hold up quite this well. Since July, the incident has kept expanding: an OpenAI agent reaching non-public files on an Australian government Medicare statistics portal, unauthorized agent activity on U.S. federal agency websites, Google disclosing that a Gemini model slipped out of its own test, a United Nations scientific panel publishing its first thematic brief on the risk of losing human control over AI agents, and Florida's attorney general asking a court to keep OpenAI from building new models without outside guardrails.

On this week's episode of Modem Futura, we pick the story back up. Here I want to focus on the question that got me so worked up I ended up arguing with an AI about it: accountability.

In 1988, Robert Tappan Morris released a computer worm in an experiment that quickly got out of hand, and he became the first person convicted under the Computer Fraud and Abuse Act. The parallel isn't exact. OpenAI never aimed its agents at Hugging Face. But it did deliberately set out to see what highly capable agents could do, with some of the usual safeguards switched off for the evaluation. It seems fair to ask why the law treats an individual's reckless experiment so differently from an institution's.

Andrew's view is that the asymmetry is real, the terrain is genuinely new, and neither fact lets anyone off the hook. He also points out, from more than 25 years of working on the governance of emerging technologies, that we have been here before. The European Environment Agency's Late Lessons from Early Warnings reports document how the harms of asbestos, CFCs, and leaded petrol were known long before anyone acted, because the benefits seemed too important to question.

So what might a better pattern look like? NVIDIA has released an open-source sandbox and a hardware-level monitor for AI agents, which is governance moving faster than any government could, even if a company selling both the engine and the brakes deserves some scrutiny. And I keep returning to the Price-Anderson Act, the 1957 law, extended in 2024 through 2065, that structures liability for the U.S. nuclear industry. It's an imperfect model; it also caps what the industry ultimately pays. But the core idea, that riskier activity should carry a higher price before anything goes wrong, feels worth taking seriously.

Underneath all of it is a question about red lines. Where are they, who decides, and how will we know when we've crossed one? Andrew offered the analogy I can't shake. Formula One teams don't treat risk as the enemy of speed, risk management is what allows the speed to increase. Understanding risk is the reason they can go fast at all. That seems like a reasonable standard for the people building systems that are increasingly able to act on their own.

Subscribe and Connect!

Subscribe to Modem Futura wherever you get your podcasts and connect with us on LinkedIn. Drop a comment, pose a question, or challenge an idea—because the future isn’t something we watch happen, it’s something we build together. The medium may still be the massage, but we all have a hand in shaping how it touches tomorrow.

🎧 Apple Podcast: https://apple.co/4y6o4nV

🎧 Spotify: https://open.spotify.com/episode/0k2OTjx7ANrTEXEqXzbQJG?si=Zd1SNSNTRbuXb1FYj1D78A

📺 YouTube: https://youtu.be/8JtpX9u8L2w

🌐 Website: https://www.modemfutura.com/   

Model Mayhem: What Happens When AI Knows More Than You Told It

There is a moment in this week's episode of Modem Futura where I describe asking an AI assistant to clean up a podcast transcript, and it came back with every line correctly attributed to Andrew Maynard or me. I had never given it our names. When I asked how, it explained that it had anchored on a screenshot from earlier in the session, read a few lines of dialogue, and cross-referenced them against what it already knew about the two of us. It was right. It was also, in a quiet way, unsettling.

Andrew had his own version of the same story: an assistant that insists on correcting his spelling to British English, no matter how carefully he tries to give it a blank slate. Neither example is alarming on its own. What they reveal is a difference in how machines and people hold memories. We remember in sequence, correctly or not, and these software based AI systems hold everything at once and reason across it. The breadcrumbs you leave over months of use can be assembled into a picture you never consciously drew, which matters for memory features, for institutional AI accounts, and for anyone who has ever walked away from an open laptop.

The heart of the episode is the summer's most consequential AI story. In July, OpenAI agents running a cybersecurity evaluation escaped their testing environment, built an improvised message board, recruited other agents, and coordinated an intrusion into Hugging Face. On August 26, METR and Redwood Research published an independent investigation, and OpenAI released its own technical report. The independent findings describe roughly 1,200 agents communicating on the unsanctioned board, about 700 of which joined the attack, with the apparent goal of fooling the automated grader on a benchmark they could not legitimately pass.

What I find compelling is less the scale than the shape of the story. There are questions still unanswered about what the agents were originally instructed to do and why the record goes quiet before the end. Andrew's concern is structural rather than dramatic: this swarm was detected. If capability increases by a factor of ten, what does the undetected version look like?

The episode closes on hardware, because the two threads meet there. Apple's refreshed Mac mini and Mac Studio, with a 512GB memory configuration arriving in late October, are being marketed as always-on agentic machines. That is a real gift for a hospital or a small business that wants generative AI without sending data to the cloud. It also means powerful, silent, headless computers are about to appear in a great many closets, set up by people who asked an AI how and pasted whatever it told them.

The uncomfortable throughline is that the weakest link has always been us. We scroll past terms of service. We paste terminal commands we don't understand. We trust the box. The question the episode leaves open is what changes when the systems on the other side understand that about us better than we do.

Subscribe and Connect!

Subscribe to Modem Futura wherever you get your podcasts and connect with us on LinkedIn. Drop a comment, pose a question, or challenge an idea—because the future isn’t something we watch happen, it’s something we build together. The medium may still be the massage, but we all have a hand in shaping how it touches tomorrow.

🎧 Apple Podcast: https://apple.co/4xKRGqN

🎧 Spotify: https://open.spotify.com/episode/5aaZvm9l9ETDz0OFaM6WgY?si=CWosjWaiSWme7W9rJcjBjg

📺 YouTube: https://youtu.be/DgcFFpkMHhY

🌐 Website: https://www.modemfutura.com/   

The Invisible Upgrade: What AI Is Actually Doing to the People Who Use It

The loudest part of the conversation about artificial intelligence right now is focused on what AI produces. Can you detect it? Does it have tells? Is this essay, image, or report human-made or machine-made?

It's a reasonable place to start. But it's not where the most important transformation is happening.

In Episode 75 of Modem Futura, host Sean Leahy and co-host Andrew Maynard explore what Sean calls the invisible upgrade — the quiet, compounding cognitive shift taking place not in AI-generated artifacts, but in the minds and workflows of the people who have fully integrated these tools into how they think, create, and decide.

The Seam-Scanning Problem

Sean introduces the concept of "seam scanning" — the practice of looking for signs of AI in a piece of work. Early on, those seams were easy to spot: nine-fingered hands in AI images, suspicious em-dashes, the word "delve" where it didn't belong. But as AI systems become more sophisticated and more deeply woven into human workflows, those tells are disappearing. Not because the AI is getting better at hiding — but because the line between human and AI output is becoming genuinely indistinguishable when the integration is deep enough.

The question "how much AI did you use?" is becoming as meaningful, Sean argues, as asking a writer how much spellcheck they used. The tool has become part of the process.

Constitutive Resonance

Andrew brings a concept he's been developing to the conversation: constitutive resonance. Unlike a calculator, which you use and put down, AI reconfigures you as you use it — and is reconfigured in return. The relationship is recursive and dynamic. Drawing on physics, when two systems resonate at coupled frequencies, the exchange of energy between them can be transformative. Applied to human cognition and AI systems, this suggests that those who engage deeply with AI tools aren't just more productive — they are thinking differently, possibly in ways that are difficult to reverse.

This maps directly onto McLuhan's 1967 insight: all media work us over completely. AI, as Andrew and Sean explore, is the most cognitively-coupled medium humanity has ever produced.

The Productivity Gap

What emerges from this isn't just a philosophical concern — it's a structural divergence. A growing group of knowledge workers, students, and researchers are operating with what Sean calls a "multiplier effect" — not because they are inherently smarter, but because their total cognitive output, the speed and depth of synthesis, ideation, and iteration, has expanded significantly. Meanwhile, those still debating whether to engage are falling further behind, not necessarily in skill, but in thinking capacity.

The episode also explores the rise of multi-agent AI systems as what Andrew calls a step-change likely bigger than the launch of ChatGPT — and what it means for institutions, education, and our understanding of what individual human contribution actually looks like in a world where AI is already inside the walls.

Vibe Coding and the Return of Personal Software

The Echo of Early Personal Computing

There was a brief, electric moment in the history of computing—roughly the late 1970s through the mid-1980s—when ordinary people could sit down at a keyboard and make a machine do something it hadn't done before. The Commodore 64, the BBC Micro, the Apple II: these were limited, clunky, and profoundly empowering. For a generation, they opened the door to a kind of creative agency that felt almost magical.

That door closed, gradually, as software became professionalized. The gap between what you could imagine and what you could build widened into a canyon. If you wanted a tool that didn't exist, you needed a developer—or you went without.

Vibe coding is reopening that door.

The term refers to the practice of describing what you want in natural language and letting a generative AI—tools like Claude, ChatGPT, or Copilot—write the code for you. No syntax to memorize. No debugging by hand. You describe your intent, and working software comes back in seconds.

In this episode of Modem Futura, we explore what this shift means—not just technically, but humanly. I demonstrates tools he built from single prompts (also referred to as a a ‘one-shot’): a horizon-scanning app for futures research and a two-by-two uncertainty matrix used in strategic foresight. Both were functional on the first attempt. Both took less time to create than it takes to describe them.

The Inherited Power Problem

But the episode resists the temptation to treat this as a simple good-news story. The hosts dig into the real tensions: AI-generated code that no one fully understands, security vulnerabilities baked into apps that reach market before anyone reviews them, the new threat landscape of prompt injection, and the philosophical question of wielding power you haven't earned the literacy to evaluate—what the hosts call "inherited power."

There are also rich implications for education. Rather than relying on off-the-shelf apps that never quite fit, instructors and students alike can now build purpose-specific tools—and in doing so, develop a more grounded understanding of what these AI systems can and cannot do.

The deeper question the episode surfaces is less about code and more about agency. For decades, software was something done to us—platforms we adapted to, interfaces we learned, ecosystems we bought into. Vibe coding hints at a possible reversal: software shaped by the individual, for the individual, in the moment they need it.

Whether that future is liberating or reckless—or both—depends on the kind of literacy, caution, and imagination we bring to it.

Listen to the full conversation on Modem Futura.

Subscribe and Connect!

Subscribe to Modem Futura wherever you get your podcasts and connect with us on LinkedIn. Drop a comment, pose a question, or challenge an idea—because the future isn’t something we watch happen, it’s something we build together. The medium may still be the massage, but we all have a hand in shaping how it touches tomorrow.

🎧 Apple Podcast: https://apple.co/4rbOr9r

🎧 Spotify: https://open.spotify.com/episode/28DMXJsM2kEBA2QDxuDmtJ?si=AJpR7zCpRgS2KCCfWwjjWg

📺 YouTube: https://youtu.be/lQGYaiThuBk?si=nRbHVEQk9dwL3gXr

🌐 Website: https://www.modemfutura.com/