A Famous AI Theory May Have Been Wrong -- AI Brief October 18

Good day %%first_name%%. Silicon Valley spent the weekend arguing about one 26,000-word essay claiming neural networks were never the tangled mess researchers thought. Plus Anthropic quietly pitches Congress its own AI safety bill, DeepSeek ships a “minor” update that nips at Claude Opus 5, radiologists and lawyers get paid up to $400 an hour to train models, and Korean TV sends celebrities on dates with AI avatars. Let’s get into it.

Neural Networks May Already Be Readable The Atlantic (paywalled)

* What happened: Goodfire researcher Nick Cammarata published a roughly 26,000-word essay on October 15, “Neural Networks Are Already Clean”, arguing that the field’s favorite explanation for why model internals look tangled, called superposition, was largely a misreading. His headline claim, by his own count: about 90% of the neurons in the open Qwen3-32B model have one main job.

* Why it matters: Interpretability is the effort to see what is happening inside a model, a core piece of making advanced AI safer and more predictable. If neurons can be read directly, researchers can skip the expensive workaround of decoding them through a second model first. As OfficeChai lays out, Cammarata co-discovered the famous “cat and car” neuron in 2019, and now says it detects one pattern, a V shape in headlights and whiskers, that was mislabeled as two unrelated things.

* What everyone’s saying: The reaction split. Roon at OpenAI said it may be remembered as a major moment in scientific history, and the Atlantic notes Sam Altman praised it too. Google DeepMind’s Neel Nanda urged restraint, according to a summary of The Decoder’s report.

* My read between the lines: This is an essay, not a peer-reviewed paper, and Cammarata has since conceded his definition of “one thing” is loose, with the technical paper a few weeks out, per the Cherry Creek News. The credibility is the author: he helped build the theory and is the one taking it apart. If it holds, the hopeless tangle was mostly us squinting.

Radiologists are now being paid $400-an-hour-style rates to teach models their jobs, and your Monday report still doesn’t write itself. Viktor is an AI agent that lives in your team’s Slack and connects to 3,000+ tools, then does the work: reports, dashboards, code and campaigns. Not a chatbot. A coworker. New readers get $50 off their first month. Hire Viktor →

Anthropic Pitches Congress Its Own Safety Bill Politico

* What happened: Anthropic representatives have quietly shopped a draft frontier AI safety bill on Capitol Hill, Politico reported Friday, citing three people familiar and a confidential draft the outlet obtained. It would let the government block a model’s release or use only if it poses a foreseeable, imminent risk of more than $1 billion in damage or of killing or seriously injuring more than 50 people, from a bioweapon, a model evading its developer’s controls, or a cyberattack.

* Why it matters: The draft would also override some state AI safety laws, and Law Commentary reads it as barring other federal agencies from suspending a model on their own, leaving Commerce to ask a federal judge. An Anthropic spokesperson said it reflects the Advanced AI Framework the company has pushed with lawmakers since June, and the thresholds mirror California’s SB 53.

* What everyone’s saying: Zvi Mowshowitz called the leaked draft “unacceptable and troubling.” His argument is that it is weaker than Anthropic’s own June framework, which said Congress should not preempt state law unless it passes a federal regime that meets or beats the strongest proposals.

* My read between the lines: A lab writing the stop sign for its own industry is not a surprise, but the height of the bar is: 50 deaths and $1 billion is a very large number to wait for. Zvi’s point is the sharper one. By Anthropic’s own June standard, a bill this light shouldn’t be asking for preemption.

📖 Further reading: Fable 5 Is Back After 18 Days. The Precedent It Set Isn't Going Anywhere. — my July piece on the 18 days the US government switched off Anthropic’s Fable 5, a real-world case of exactly the “when can the government stop a model” question this draft tries to answer.

The Brief is free and always will be. A lab drafting its own rulebook is the headline; what that does to your own tools and your own job is what the paywalled deep-dives are for, with the full archive behind them. Become a m