Anthropic Insiders Say AI Could Kill Humanity as Regulation Struggles to Catch Up

Anthropic Insiders Say AI Could Kill Humanity as Regulation Struggles to Catch Up
The race to build increasingly capable artificial intelligence is producing an unusual consensus among insiders: the technology may become dangerously difficult to control, even as experts sharply disagree over how soon that could happen and whether safeguards are keeping pace.

Former Anthropic researcher Jacob Coxon says companies pursuing superintelligence are “gambling with our lives,” while Anthropic alignment researcher Evan Hubinger has put the chance of AI killing all humans within the next decade at more than 10%. Their warnings focus not on today’s systems, which both describe as posing a relatively limited immediate threat, but on recursive self-improvement—the possibility that AI systems could autonomously build more capable successors.

The more alarmed interpretation is that this transition could arrive within years. Critics say laboratories are effectively trying to automate AI research, potentially accelerating development faster than humans can understand or supervise it. The concern is not merely malfunction, but a loss of control concentrated in a handful of companies or decision-makers.

Coxon’s account is similarly severe but more overtly political. He told Fox News that recursive self-improvement “could happen as soon as next year,” while insisting the crisis remains “eminently solvable” through regulation and international cooperation. He described AI as a potential “superweapon” in an arms race and argued that governments must prevent companies from racing into an irreversible disaster.

The two perspectives therefore converge on the core risk but diverge in emphasis. Hubinger’s warning underscores uncertainty and the absence of a reliable alignment plan; Coxon stresses that the danger is a policy failure, not an unavoidable technological destiny. Other experts cited in the debate believe superintelligence could remain decades away, while acknowledging that autonomous learning would be a decisive warning sign.

That disagreement leaves the central tension unresolved: governments may have time to regulate advanced AI—or may already be behind the curve.