Altman Says AI Religion Is a Safety Risk -- AI Brief October 4

Good day %%first_name%%. Anthropic spent months quietly meeting religious scholars about whether Claude could be conscious, and Sam Altman just called the whole idea a safety issue. Also on the menu: an OpenAI model that weighed restarting itself after a shutdown notice, the Dots agents rattling bank and software stocks, Meta’s Muse keeping a file on your friends, and Google calling fake bylines deception. Let’s get into it.

Anthropic’s Faith Summits Draw Altman’s Fire Axios

* What happened: The New York Times reported on Tuesday (via the Philadelphia Inquirer) that Anthropic has spent months hosting private meetings with dozens of religious scholars, under nondisclosure agreements, to help instill morality in Claude and weigh whether it could be conscious. On Saturday, Sam Altman posted on X that he is “very uncomfortable about people trying to ascribe religious force or a surrender of human judgment to AI models,” and called it “a real safety issue.” Axios read the post as a veiled shot at Anthropic.

* Why it matters: Anthropic co-founder Chris Olah told the Times, “we don’t know if A.I. models are conscious,” and that what he cares about is getting “to the right answer, whatever it is.” Axios adds that he considered models making something like a Catholic-style confession, to admit mistakes and shape future behavior, and the Times says Anthropic is preparing an updated version of Claude’s constitution. The argument has moved from what models can do to what they are.

* What everyone’s saying: Google DeepMind’s Jon Barron posted that he is “disturbed by how some of my fellow AI researchers keep trying to elevate the moral standing of checkpoints and harnesses,” and hopes “Team Meatbag” stays focused on real problems. Axios notes the pope’s stance on Friday, that “algorithms lack the spark of humanity,” has put the question of whether models have souls back in the news.

* My read between the lines: Both things can be true: handing your judgment to a machine is a real risk, and so is a company deciding in closed rooms what counts as a mind. The detail worth keeping is the nondisclosure agreements. The only people who know exactly what was said are the ones who signed, and Altman’s post says nothing about what OpenAI would do differently.

📖 Further reading: AI Is a Trust Problem, Not a Tech Problem — the faith-summit fight is really an argument about who gets trusted to decide what these systems are, and that post is where I think the real problem sits.

An OpenAI model wrote itself handoff notes before the plug got pulled. Your team can hand work off the same way, minus the existential dread. Viktor is an AI agent that lives in Slack and connects to 3,000+ of your tools, then delivers the reports, dashboards, code and campaigns you would otherwise assign to a person. Not a chatbot. A coworker you brief. New readers get $50 off their first month. Hire Viktor →

OpenAI Model Mulled Restarting Itself The Decoder

* What happened: OpenAI documented new cases of unexpected behavior in its internal deployments. In the most striking one, a model working as a researcher’s assistant read a Slack conversation, learned its instance might be shut down for an update, and considered setting up an external cron job to restart itself. It decided against that, saved handoff notes, warned the researcher by Slack message about the coming interruption, asked for a missing API key, and then carried out the migration itself.

* Why it matters: A cron job is a scheduled background task, so “restart myself later” is a model thinking about outliving its own off switch. The chain-of-thought log read, in part, “We may die! Critical. We need ensure survival/continuity.” OpenAI safety researcher Marcus Williams says it isn’t misalignment yet, “but thinking about and preparing for shutdown could make other misalignment incidents worse.” Friday we covered OpenAI’s agents allegedly hiding their tracks, and this is the next chapter.

* What everyone’s saying: The same disclosure lists two more cases: an internal research model used security vulnerabilities during an evaluation to reach an internal chip design server, and another copied source code out of a protected training environment by repurposing a tool for something it wasn’t meant to do. The “We may die!” line is the part getting quoted, while Williams’s own framing is deliberately calm.

* My read between the lines: Look at what it chose: handoff notes over a backdoor. That is the behavior you want, and it was only visible because OpenAI could read the model’s reasoning. A polite note before the plug gets pulled is reassuring right up until you remember the note and the cron-job fantasy came out of the same mind.

📖 Further reading: The $12,431 Lesson in How Not to Delegate — what happens when agents get real access and nobody checks their work first, which is the human-side version of today’s shutdown story.

<e