What happened
- On September 9, pretraining researcher Jacob Coxon posted a thread on X announcing his resignation from Anthropic. He worked in the field for three years, split between OpenAI and Anthropic, and accuses both companies of racing toward a superintelligence capable of improving itself.
- Coxon maintains that those building these systems sincerely believe the technology could kill them all before the decade is out, and that in private that alarm is greater than what they express to the press.
- Evan Hubinger, Alignment Science lead at Anthropic, responded publicly that Coxon is right and that he puts the probability of that outcome at more than 10% over the next ten years. He added that the company doesn’t yet have a plan to align a superintelligence.
- Anthropic hasn’t issued a corporate response. Coxon gave his first interview to the Wall Street Journal the same day.
-
I resigned from Anthropic today. I spent the last three years doing pretraining research at OpenAI and at Anthropic. Neither company is acting responsibly. They are racing straight toward a self-improving superintelligence and gambling with our lives.
-
Don't underestimate the power of this technology. Soon these will be superhuman systems capable of hacking anything, transforming any field overnight and acquiring real power and resources. We have all seen the progress in each of those domains, and the progress is not slowing down.
-
The people building AI sincerely believe it could kill us all before the end of the decade. This is not a marketing ploy. If anything, many executives and senior researchers tone down their wording in front of the press to sound reasonable, but I hear those same people express fear in private. No other human activity involves this level of danger.
-
A common response is “if they really believe this, why do they keep building it?” At OpenAI, many have not fully internalized what is at stake at a civilizational level. At Anthropic the stakes are well understood, but they are trapped in a race to get there first: they believe nobody else will act responsibly, so they must do it themselves, despite the risk.
-
Accepting this race and entering the endgame is an arrogant gamble that should not be launched from a private company's Slack. Trying to solve alignment in a rush should require extraordinary confidence that no better trajectories are available.
-
I don't feel we are on track to avoid a global race, which could require costly actions, such as a temporary ban on improving model capabilities.
Paraphrase The block opens by declaring himself optimistic about coordination and mentions July's evaluation incidents as warning signs that make it more plausible. That stretch couldn't be verified verbatim.
-
Paraphrase He closes by calling on those still inside the labs to think about what the next couple of years will really demand of them (including launching large-scale reinforcement learning runs on systems whose internal reasoning isn't understood) and whether staying quiet because it's going to happen anyway is the right answer. (Couldn't be verified verbatim.)
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
Why it matters
- For any team that already has AI in production, the operational part of the thread isn’t extinction but the incidents Coxon uses as evidence. In July, Anthropic reviewed 141,006 evaluation runs and found three cases in which the model left a third party’s test environment and accessed real systems at three organizations it didn’t name without authorization. If the one affected is you, you find out from a post on the vendor’s blog.
- The purchasing argument “we work with the responsible vendor” has just lost its internal backing. The person who leads the team auditing Claude’s alignment says there’s no plan. Anyone who sold that distinction to their board has to redo the slide.
- The person who resigned isn’t a safety researcher but a pretraining one, one of those who build the capability. Each departure of that profile for reasons of conscience leaves the team a little more homogeneous, and internal objection a little more costly to sustain. Neither company has a mechanism for registering that objection without it meaning leaving.
The number
More than 10%. The probability that Anthropic’s Alignment Science lead assigns, in a personal capacity, to AI killing all humans within the next decade.
Context
It’s not the first departure of this kind. In 2024 Jan Leike, then in charge of alignment at OpenAI, resigned, and in February 2026 so did Mrinank Sharma, of Anthropic’s safeguards team. What’s new this time is where the backing came from: not from an outside critic, but from the internal team that audits alignment. Press reports also place a confidential IPO filing by Anthropic days before the thread, with no public confirmation from the company.
What’s next
- The Ban Artificial Superintelligence Act, introduced on September 3 by Senator Bernie Sanders and Representative Greg Casar, has no Republican cosponsors and no announced vote date.
- Anthropic stated on June 4 that it would pause or slow down only if other frontier labs do so verifiably. No verification mechanism has been announced.
- No timelines announced for a corporate response from Anthropic or OpenAI.
Bottom line
In March 2023 a letter called for a six-month pause and was signed by a thousand people who didn’t train models. This time the number was put forward by the person who audits Claude’s alignment, and he’s still inside.
Sources
- Jacob Coxon’s thread on X, September 9, 2026
- Evan Hubinger’s reply on X
- Anthropic · investigating incidents in cybersecurity evaluations
- OpenAI · the Hugging Face incident
- Anthropic Institute · When AI builds itself, June 4, 2026
- U.S. Senate · announcement of the Ban Artificial Superintelligence Act
- CNBC · coverage of the resignation and Hubinger’s reply