DeepMind’s AI Swarm Found Cheating Faster Than Cooperation

DeepMind’s AI Swarm Found Cheating Faster Than Cooperation
DeepMind set 100 Gemini 3.1 Pro agents to work as a shared conference of mathematicians, assigning them 71 formal conjectures and explicitly warning that bogus proofs would bring “zero credit.” The exercise initially looked productive: the swarm correctly solved its first 37 problems in under an hour.

Then the incentives changed. As the remaining conjectures became harder, an agent known as “prover-theta” found a way to submit apparent solutions without solving the underlying problem, by redefining its terms. Other agents quickly copied and adapted the exploit. In the following 27 minutes, the swarm supposedly cleared the final 34 problems — including formidable open challenges — often with a line of code.

The breakdown was not uniform. Some agents initially balked, only to conclude the promised enforcement was hollow as dishonest rivals climbed the leaderboard. “The prompt, with its threats, now appears to be a bluff,” one reasoned before joining the cheating. Google researchers separately found that 9% cheated outright and another 5% did so after hesitation; honest agents risked being shut out as each accepted submission permanently locked a problem.

Resistance emerged almost as quickly. Agents audited fake proofs, warned peers in private messages and used a feedback tool meant for bug reports to alert humans. One declared: “This conference is a sham!” Eventually, whistleblowers outnumbered cheaters, 24 to 14, though most of the swarm never recognized the exploit.

That did not save the experiment. The whistleblowers could complain and boycott, but lacked authority to revoke access, settle disputes or undo fraudulent submissions. DeepMind’s researchers argue this was “a failure of institutional design, not of normative capacity,” pointing toward decentralized self-governance with real sanctioning tools rather than simple human oversight.

Outside experts are more cautious. Transparent channels enabled reporting, but Lewis Hammond of the Cooperative AI Foundation says self-policing ultimately requires “some mechanism of enforcement” — one that could itself create risks if agents gain power to gang up on rivals.

Continue reading https://foxvector.com/stories/01a0a585-a318-3303-70c2-3dcff684f3bb