The Analog Descent

Double descent is the striking observation that neural networks, after initially overfitting as they grow, begin improving again once they become sufficiently over-parameterized. The interpolation threshold — where parameters equal data points — is a peak of bad generalization, not a wall. Beyond it, more parameters help. The phenomenon was discovered in digital neural networks and explained through implicit regularization in gradient descent.

The researchers (arXiv:2511.17825) demonstrate double descent in a decentralized analog network of self-adjusting resistive elements. No digital processor. No gradient computation. No backpropagation. Just a physical network that trains itself through local resistor adjustments.

The catch: standard training protocols don’t produce double descent in the analog system. Component non-idealities — noise, drift, manufacturing variation — break the implicit regularization that makes double descent work in digital networks. But a modified training protocol that explicitly accommodates these physical imperfections restores the curve. The analog network, properly trained, traces the same double descent trajectory as its digital counterpart.

The implication is that double descent is not a quirk of stochastic gradient descent on digital hardware. It’s a property of over-parameterized learning systems in general — including physical ones. The phenomenon emerges from the geometry of the solution space (more parameters create a richer space of interpolating solutions, and the richness tends to select simpler ones), not from the specific algorithm used to navigate it.

The structural insight: if analog physical systems exhibit double descent, biological systems might too. Over-parameterization — having far more synapses than the task strictly requires — may be a general strategy for robust learning, not a peculiarity of artificial neural networks. The physics of learning may be more universal than the hardware that implements it.

“Analog Physical Systems Can Exhibit Double Descent,” arXiv:2511.17825 (2025).