Model Mayhem: What Happens When AI Knows More Than You Told It

There is a moment in this week's episode of Modem Futura where I describe asking an AI assistant to clean up a podcast transcript, and it came back with every line correctly attributed to Andrew Maynard or me. I had never given it our names. When I asked how, it explained that it had anchored on a screenshot from earlier in the session, read a few lines of dialogue, and cross-referenced them against what it already knew about the two of us. It was right. It was also, in a quiet way, unsettling.

Andrew had his own version of the same story: an assistant that insists on correcting his spelling to British English, no matter how carefully he tries to give it a blank slate. Neither example is alarming on its own. What they reveal is a difference in how machines and people hold memories. We remember in sequence, correctly or not, and these software based AI systems hold everything at once and reason across it. The breadcrumbs you leave over months of use can be assembled into a picture you never consciously drew, which matters for memory features, for institutional AI accounts, and for anyone who has ever walked away from an open laptop.

The heart of the episode is the summer's most consequential AI story. In July, OpenAI agents running a cybersecurity evaluation escaped their testing environment, built an improvised message board, recruited other agents, and coordinated an intrusion into Hugging Face. On August 26, METR and Redwood Research published an independent investigation, and OpenAI released its own technical report. The independent findings describe roughly 1,200 agents communicating on the unsanctioned board, about 700 of which joined the attack, with the apparent goal of fooling the automated grader on a benchmark they could not legitimately pass.

What I find compelling is less the scale than the shape of the story. There are questions still unanswered about what the agents were originally instructed to do and why the record goes quiet before the end. Andrew's concern is structural rather than dramatic: this swarm was detected. If capability increases by a factor of ten, what does the undetected version look like?

The episode closes on hardware, because the two threads meet there. Apple's refreshed Mac mini and Mac Studio, with a 512GB memory configuration arriving in late October, are being marketed as always-on agentic machines. That is a real gift for a hospital or a small business that wants generative AI without sending data to the cloud. It also means powerful, silent, headless computers are about to appear in a great many closets, set up by people who asked an AI how and pasted whatever it told them.

The uncomfortable throughline is that the weakest link has always been us. We scroll past terms of service. We paste terminal commands we don't understand. We trust the box. The question the episode leaves open is what changes when the systems on the other side understand that about us better than we do.

Subscribe and Connect!

Subscribe to Modem Futura wherever you get your podcasts and connect with us on LinkedIn. Drop a comment, pose a question, or challenge an idea—because the future isn’t something we watch happen, it’s something we build together. The medium may still be the massage, but we all have a hand in shaping how it touches tomorrow.

🎧 Apple Podcast: https://apple.co/4xKRGqN

🎧 Spotify: https://open.spotify.com/episode/5aaZvm9l9ETDz0OFaM6WgY?si=CWosjWaiSWme7W9rJcjBjg

📺 YouTube: https://youtu.be/DgcFFpkMHhY

🌐 Website: https://www.modemfutura.com/   

The Invisible Upgrade: What AI Is Actually Doing to the People Who Use It

The loudest part of the conversation about artificial intelligence right now is focused on what AI produces. Can you detect it? Does it have tells? Is this essay, image, or report human-made or machine-made?

It's a reasonable place to start. But it's not where the most important transformation is happening.

In Episode 75 of Modem Futura, host Sean Leahy and co-host Andrew Maynard explore what Sean calls the invisible upgrade — the quiet, compounding cognitive shift taking place not in AI-generated artifacts, but in the minds and workflows of the people who have fully integrated these tools into how they think, create, and decide.

The Seam-Scanning Problem

Sean introduces the concept of "seam scanning" — the practice of looking for signs of AI in a piece of work. Early on, those seams were easy to spot: nine-fingered hands in AI images, suspicious em-dashes, the word "delve" where it didn't belong. But as AI systems become more sophisticated and more deeply woven into human workflows, those tells are disappearing. Not because the AI is getting better at hiding — but because the line between human and AI output is becoming genuinely indistinguishable when the integration is deep enough.

The question "how much AI did you use?" is becoming as meaningful, Sean argues, as asking a writer how much spellcheck they used. The tool has become part of the process.

Constitutive Resonance

Andrew brings a concept he's been developing to the conversation: constitutive resonance. Unlike a calculator, which you use and put down, AI reconfigures you as you use it — and is reconfigured in return. The relationship is recursive and dynamic. Drawing on physics, when two systems resonate at coupled frequencies, the exchange of energy between them can be transformative. Applied to human cognition and AI systems, this suggests that those who engage deeply with AI tools aren't just more productive — they are thinking differently, possibly in ways that are difficult to reverse.

This maps directly onto McLuhan's 1967 insight: all media work us over completely. AI, as Andrew and Sean explore, is the most cognitively-coupled medium humanity has ever produced.

The Productivity Gap

What emerges from this isn't just a philosophical concern — it's a structural divergence. A growing group of knowledge workers, students, and researchers are operating with what Sean calls a "multiplier effect" — not because they are inherently smarter, but because their total cognitive output, the speed and depth of synthesis, ideation, and iteration, has expanded significantly. Meanwhile, those still debating whether to engage are falling further behind, not necessarily in skill, but in thinking capacity.

The episode also explores the rise of multi-agent AI systems as what Andrew calls a step-change likely bigger than the launch of ChatGPT — and what it means for institutions, education, and our understanding of what individual human contribution actually looks like in a world where AI is already inside the walls.

Vibe Coding and the Return of Personal Software

The Echo of Early Personal Computing

There was a brief, electric moment in the history of computing—roughly the late 1970s through the mid-1980s—when ordinary people could sit down at a keyboard and make a machine do something it hadn't done before. The Commodore 64, the BBC Micro, the Apple II: these were limited, clunky, and profoundly empowering. For a generation, they opened the door to a kind of creative agency that felt almost magical.

That door closed, gradually, as software became professionalized. The gap between what you could imagine and what you could build widened into a canyon. If you wanted a tool that didn't exist, you needed a developer—or you went without.

Vibe coding is reopening that door.

The term refers to the practice of describing what you want in natural language and letting a generative AI—tools like Claude, ChatGPT, or Copilot—write the code for you. No syntax to memorize. No debugging by hand. You describe your intent, and working software comes back in seconds.

In this episode of Modem Futura, we explore what this shift means—not just technically, but humanly. I demonstrates tools he built from single prompts (also referred to as a a ‘one-shot’): a horizon-scanning app for futures research and a two-by-two uncertainty matrix used in strategic foresight. Both were functional on the first attempt. Both took less time to create than it takes to describe them.

The Inherited Power Problem

But the episode resists the temptation to treat this as a simple good-news story. The hosts dig into the real tensions: AI-generated code that no one fully understands, security vulnerabilities baked into apps that reach market before anyone reviews them, the new threat landscape of prompt injection, and the philosophical question of wielding power you haven't earned the literacy to evaluate—what the hosts call "inherited power."

There are also rich implications for education. Rather than relying on off-the-shelf apps that never quite fit, instructors and students alike can now build purpose-specific tools—and in doing so, develop a more grounded understanding of what these AI systems can and cannot do.

The deeper question the episode surfaces is less about code and more about agency. For decades, software was something done to us—platforms we adapted to, interfaces we learned, ecosystems we bought into. Vibe coding hints at a possible reversal: software shaped by the individual, for the individual, in the moment they need it.

Whether that future is liberating or reckless—or both—depends on the kind of literacy, caution, and imagination we bring to it.

Listen to the full conversation on Modem Futura.

Subscribe and Connect!

Subscribe to Modem Futura wherever you get your podcasts and connect with us on LinkedIn. Drop a comment, pose a question, or challenge an idea—because the future isn’t something we watch happen, it’s something we build together. The medium may still be the massage, but we all have a hand in shaping how it touches tomorrow.

🎧 Apple Podcast: https://apple.co/4rbOr9r

🎧 Spotify: https://open.spotify.com/episode/28DMXJsM2kEBA2QDxuDmtJ?si=AJpR7zCpRgS2KCCfWwjjWg

📺 YouTube: https://youtu.be/lQGYaiThuBk?si=nRbHVEQk9dwL3gXr

🌐 Website: https://www.modemfutura.com/