Researcher who departed AI role over fears of racing into extinction says people are 'begging' for regulation

A former Anthropic researcher, Jacob Coxon, has expressed concerns that artificial intelligence could lead to the extinction of humankind within the next decade due to rapid, unchecked development. He believes the issue is solvable through international regulation and cooperation, comparing the current situation to a dangerous arms race. Coxon stated that while AI offers potential for tremendous discoveries, the race to achieve superintelligence without proper safety measures is a gamble with human lives, and people are actively seeking regulation to prevent this outcome.

• Former Anthropic researcher Jacob Coxon fears AI could lead to human extinction within a decade.
• He believes the problem is solvable with international regulation and a slowdown in development.
• Coxon compares the current AI race to a dangerous arms race that could be disastrous.
• Evan Hubinger, Anthropic’s alignment science lead, agrees with Coxon’s assessment of the risks.
• Hubinger estimates a greater than 10% chance of AI killing all humans within the next decade.
• Coxon emphasizes that while current AI models pose a low risk, recursive self-improvement could accelerate the threat.
• He stated that people are “begging for regulation” to ensure responsible AI development.
• Despite the risks, Coxon acknowledges the potential for tremendous discoveries through AI.
• He believes humanity can solve the problem if they prioritize safety over a “doomed race” or a “hubristic gamble.”