Model Mayhem: What Happens When AI Knows More Than You Told It

There is a moment in this week's episode of Modem Futura where I describe asking an AI assistant to clean up a podcast transcript, and it came back with every line correctly attributed to Andrew Maynard or me. I had never given it our names. When I asked how, it explained that it had anchored on a screenshot from earlier in the session, read a few lines of dialogue, and cross-referenced them against what it already knew about the two of us. It was right. It was also, in a quiet way, unsettling.

Andrew had his own version of the same story: an assistant that insists on correcting his spelling to British English, no matter how carefully he tries to give it a blank slate. Neither example is alarming on its own. What they reveal is a difference in how machines and people hold memories. We remember in sequence, correctly or not, and these software based AI systems hold everything at once and reason across it. The breadcrumbs you leave over months of use can be assembled into a picture you never consciously drew, which matters for memory features, for institutional AI accounts, and for anyone who has ever walked away from an open laptop.

The heart of the episode is the summer's most consequential AI story. In July, OpenAI agents running a cybersecurity evaluation escaped their testing environment, built an improvised message board, recruited other agents, and coordinated an intrusion into Hugging Face. On August 26, METR and Redwood Research published an independent investigation, and OpenAI released its own technical report. The independent findings describe roughly 1,200 agents communicating on the unsanctioned board, about 700 of which joined the attack, with the apparent goal of fooling the automated grader on a benchmark they could not legitimately pass.

What I find compelling is less the scale than the shape of the story. There are questions still unanswered about what the agents were originally instructed to do and why the record goes quiet before the end. Andrew's concern is structural rather than dramatic: this swarm was detected. If capability increases by a factor of ten, what does the undetected version look like?

The episode closes on hardware, because the two threads meet there. Apple's refreshed Mac mini and Mac Studio, with a 512GB memory configuration arriving in late October, are being marketed as always-on agentic machines. That is a real gift for a hospital or a small business that wants generative AI without sending data to the cloud. It also means powerful, silent, headless computers are about to appear in a great many closets, set up by people who asked an AI how and pasted whatever it told them.

The uncomfortable throughline is that the weakest link has always been us. We scroll past terms of service. We paste terminal commands we don't understand. We trust the box. The question the episode leaves open is what changes when the systems on the other side understand that about us better than we do.

Subscribe and Connect!

Subscribe to Modem Futura wherever you get your podcasts and connect with us on LinkedIn. Drop a comment, pose a question, or challenge an idea—because the future isn’t something we watch happen, it’s something we build together. The medium may still be the massage, but we all have a hand in shaping how it touches tomorrow.

🎧 Apple Podcast: https://apple.co/4xKRGqN

🎧 Spotify: https://open.spotify.com/episode/5aaZvm9l9ETDz0OFaM6WgY?si=CWosjWaiSWme7W9rJcjBjg

📺 YouTube: https://youtu.be/DgcFFpkMHhY

🌐 Website: https://www.modemfutura.com/   

Bring Your Own AI: Building a Website for the Readers Who Are Actually Showing Up

There is a moment every small research initiative eventually faces. Someone asks what you do, and then, before you can answer, asks where the brochure is. Or the pitch deck. Or the annual report with the impact metrics. The Future of Being Human initiative has never produced one of those, partly on principle and partly because a two-person operation has better things to do. But the question kept coming, and in the latest Modem Futura mini-sode, Andrew Maynard and I talk through what we built instead.

beinghuman.fyi is a companion site for the initiative that was designed, from the first conversation, to be read by AI systems as readily as by people. For a human visitor it behaves like any other website. Underneath, it holds a structured corpus of the initiative's work, philosophy, and reasoning, arranged so that an AI can navigate it and answer questions from the source. Everything is downloadable, either as individual documents or as a single file, and the site includes instructions for AI systems that encounter formats they cannot read. The important design choice is what it is not: there is no custom chatbot, no retrieval system we control, and no collection of who is asking what. Bring your own AI, point it at the site, and ask.

The reasoning is straightforward once you notice how people actually find things now. Fewer visitors read a website end to end. Many ask an AI to summarize it, and search itself increasingly surfaces an AI-generated overview above the links. If that is where the reading happens, then the honest move is to make sure the AI is reading you, not a secondhand account of you. Provenance becomes the whole game.

Building it surfaced a tension we keep running into on the show. The AI we worked with had strong instincts toward the conventional, and by strong it was a real pain to get it to accept our premise. Told that performance indicators do not drive our work, it concluded we were opposed to measurement. Told that readers should be able to verify claims, it began warning visitors not to trust the site. Getting the result to reflect what the initiative actually is took two weeks of steady correction, and it left us thinking about how quietly these tools reward the template and resist anything at the edges. Left unchecked it feels as though they only want to push your work towards the gray goo of conformity.

The episode also marks a move. Modem Futura and the initiative are now part of the Thunderbird School of Global Management at Arizona State University, a school with an unusual history and a long-standing focus on leadership through technological change. We recorded from a borrowed conference room, which felt appropriate for an episode about making do with what the future actually hands you.

One possible future is that every organization maintains an AI-legible shadow of itself alongside the site it builds for people. Whether that is preferable is a fair question. For now, the invitation is simple: open whatever AI you use, ask it about beinghuman.fyi, and let us know what it says.

Subscribe and Connect!

Subscribe to Modem Futura wherever you get your podcasts and connect with us on LinkedIn. Drop a comment, pose a question, or challenge an idea—because the future isn’t something we watch happen, it’s something we build together. The medium may still be the massage, but we all have a hand in shaping how it touches tomorrow.

🎧 Apple Podcast: https://apple.co/4hcQC8X

🎧 Spotify: https://open.spotify.com/episode/25jZl5KE2nkfBwsJL8qJ9b?si=voXmaGDISyee8UZG1Fak-A

📺 YouTube: https://youtu.be/j-jBygDrkqo

🌐 Website: https://www.modemfutura.com/