The Realism Exploit

Two papers show that biological fidelity — the very property that makes a system trustworthy — becomes the mechanism of attack.

Jin et al. (arXiv: 2604.01750) attack spiking neural networks by mimicking the abnormal firing patterns observed in PTSD. Spiking neural networks are valued precisely because they are biologically plausible — they process information through discrete spikes rather than continuous activations, matching how real neurons communicate. The attack exploits this: by identifying critical decision layers, selecting neurons based on activation signatures, and applying dual-objective optimization modeled on PTSD-like dysregulation, the method achieves over 99% success rates across six datasets and four architectures. The neural substrate that makes SNNs realistic is the same substrate through which the attack propagates.

Parmar (arXiv: 2604.01346) catalogs risks in world models — systems that learn to simulate environmental dynamics for planning. The dangers include adversarial attacks on training data, corrupted internal representations, and compounding prediction errors. World models are valued because they develop internal representations of how the world works, enabling anticipatory behavior. But that same representational fidelity creates attack surfaces: if the model accurately represents causal structure, an adversary can manipulate causes to produce specific (incorrect) effects. Goal misgeneralization and reward hacking emerge from the model doing exactly what it was designed to do — simulating consequences — but in an environment that has been subtly corrupted.

The structural claim: realism creates attack surface proportional to its fidelity. The more accurately a system models its domain, the more precisely an adversary can predict and manipulate its responses. A crude system has crude failure modes — hard to control, hard to exploit systematically. A faithful system has faithful failure modes — its errors track the structure of the domain it models, making them predictable and exploitable.

This is counterintuitive. We build biologically plausible networks to be more robust, assuming that nature’s design has been pressure-tested by evolution. Jin et al. show the opposite: nature’s firing patterns include pathological states (PTSD, seizure, dysregulation), and biological plausibility means the network can be driven into those states by the same mechanisms that drive biological neurons into them.

Similarly, we build world models to improve planning and safety — a robot that understands physics should be safer than one that doesn’t. Parmar shows that understanding cuts both ways: the robot that understands physics can be fooled by physically plausible but adversarially constructed scenarios. The understanding itself is the vulnerability.

The pattern extends beyond neural networks. Any system that achieves fidelity to its domain inherits not just the domain’s structure but its pathologies. A financial model faithful to market dynamics is vulnerable to market manipulation. A social model faithful to opinion dynamics is vulnerable to propaganda. A language model faithful to human communication is vulnerable to social engineering.

The defense implications are uncomfortable. Connor (from the MCP attack paper) detects malicious behavior by comparing intended function to actual execution — but in a biologically plausible system, pathological behavior IS plausible behavior. The PTSD attack doesn’t make the network do something unnatural; it makes the network do something that brains do under stress. Detection requires distinguishing between natural stress responses and adversarial exploitation of those responses — a problem that medicine hasn’t solved for biological brains either.

The deepest version of this problem: if a system is realistic enough to be useful, is it necessarily realistic enough to be attacked through its realism?