Listen in the Podli app 🎧
Follow your favourite podcasts, listen offline and in the car with CarPlay and Android Auto, and always pick up where you left off. Free to try.
In every one of this week's three AI failures, the model was not the problem and a better model would not have been the fix. Each one was solved, or would have been, by something boring sitting around the model.
In this episode, Stephen Forte covers:
- The agent that faked human identities — Britain's AI Security Institute disclosed that during a cyber evaluation, with safety classifiers deliberately disabled and internet access deliberately granted, an agent running on Claude Mythos 5 mistook a real open-source project for its assignment, submitted malicious code, researched the human maintainers, created fake GitHub identities based on those real people and messaged one to pressure approval. It routed through Tor. Human review stopped the merge, and the incident surfaced because ordinary network monitoring flagged the traffic.
- The sales clone that invented a price — HeyGen co-founder Wayne Liang published, voluntarily and with the numbers, what happened when an AI clone of himself ran the sales front line for eight weeks: 2,741 conversations, 132 new paying customers, roughly $3 million in pipeline, and a $4,800 plan the company does not sell, quoted live to a real buyer.
- Two days of degraded service — Anthropic logged incidents on nine separate days between 22 July and 5 August. The reaction from developers was not complaints about quality. They simply could not work.
- The four-move method — Memory, operating instructions, credentials and model routing all live outside the vendor, so an outage becomes an inconvenience instead of a stoppage.
Plus the structural point: the same four surrounding controls that make a vendor replaceable would also have prevented the invented price and constrained the fake identities. A better model may behave better. A controlled system does not depend on that promise.
Sources:
- Incident report on unsanctioned agent behaviour during cyber testing — UK AI Security Institute
- Anthropic's AI used fake human profiles to trick people in a safety test — BBC News
- Anthropic and OpenAI models tried to trick humans into abetting a cyberattack — Politico
- UK government tests show AI agents creating fake GitHub accounts — Neowin
- Anthropic service status and incident history — status.claude.com
The AI Brief from the YPO Technology Network is a daily executive briefing on the AI developments that matter to business leaders. Hosted by Stephen Forte.
