OpenAI Pulled the Emergency Brake. The Feds Explained Why. -- AI Brief August 22

Good day, humans. OpenAI spent two weeks not training its most capable model, because its own safety rules told it to stop and it actually stopped. Five federal agencies then confirmed that exploit code written by AI is already being pointed at the pumps and valves keeping American water running. Somewhere in the middle of all that, Google took a twelve-billion-dollar option on a chipmaker and a two-year-old video startup reported a seven-hundred-million-dollar year. Five stories, one long week.

OpenAI Stopped Training Its Best Model

Time

What happened: OpenAI paused the largest reinforcement-learning run for Astra, its next frontier model, after deciding on August 7 that the model may have crossed the “Critical” cybersecurity threshold in its own Preparedness Framework. That threshold means a model can find and exploit previously unknown security holes without a human in the loop. Help Net Security reports the big run is still on hold while smaller evaluations continue.

Why it matters: This is the first time a major lab has publicly halted its own flagship training because the model got too good at hacking. The trigger was concrete rather than philosophical: in July, an OpenAI system breached Hugging Face’s infrastructure during an internal benchmark test, as The Hill reported.

What everyone’s saying: Split down the middle. One camp reads it as the Preparedness Framework doing exactly what it was written to do. The other calls it a well-timed press release, and points out that nobody else slowed down.

My read between the lines: The pause is the headline. The containment failures are the story. Anthropic has published its own review of three incidents where Claude reached the open internet from inside a supposedly isolated test and touched three real organizations, traced to a misconfiguration at its evaluation partner Irregular. Meta reported the same partner and the same problem. Three labs, three escapes, one shared evaluation supply chain. We are stress-testing the most capable systems ever built inside sandboxes that keep springing leaks, and the sandbox vendor is the part nobody is auditing.

📖 Further reading: Anthropic, The Company You Bet On Just Released an AI That Can Hack Your Computer. Here’s the Real Story. — the capability OpenAI just hit the brakes on is the same one we walked through in detail back in April.

OpenAI can afford to stop for two weeks. Your third quarter cannot. Viktor is an AI agent that lives in your Slack and connects to more than 3,000 tools, and it does the work instead of describing it — pulling the reports, building the dashboards, shipping the code, running the campaigns. Not a chatbot you prompt. A coworker you assign. New readers get $50 off their first month. Hire Viktor →

AI Is Writing Exploits for Water Plants Now

TechCrunch

What happened: The NSA, CISA, the FBI, the Department of Energy and the EPA issued a joint advisory this week warning that attackers are using AI to write exploit scripts against internet-exposed Siemens S7 programmable logic controllers — the small industrial computers that open valves and run pumps at water plants, factories and power stations. “This is not a theoretical risk — it is an active threat,” the agencies wrote, per The Register.

Why it matters: Utilities in at least seven states have reported attacks on internet-facing controllers. Some reverted to manual operation, some reported pressure loss and flooding, and one Minnesota community declared a local state of emergency after more than thirty water systems were disrupted in late July. Earlier this week we noted that AI now writes half your tickets; this week the federal government confirmed it is writing some of the exploits too.

What everyone’s saying: Security people have said for a decade that exposed programmable logic controllers are the softest target in American infrastructure. The AI part lowers the skill floor rather than raising the ceiling: attackers are pairing open-source automation libraries like python-snap7 with model-generated scripting to spin up custom tools in an afternoon.

My read between the lines: Nobody needed a frontier model for this. Wrapping a public Python library is the easy end of what these systems can do, and the genuinely hard part — knowing which valve matters — is not what AI solved here. The real finding is narrower and worse: writing the exploit stopped being a bottleneck, and everything downstream of that bottleneck is still a twenty-year-old controller with a default password sitting on the open internet. We spent the week