The p-adic ReLU

Neural networks approximate functions. Over the real numbers, the universal approximation theorem says: wide enough networks with ReLU activations can approximate any continuous function on a compact set. The minimum width for universality is known.

Kiss and Pál ask the same question over the p-adic numbers — a different number system where “closeness” means sharing more leading digits in base p rather than being numerically near. Continuous functions over the p-adics are strange by real-number standards: they can be locally constant, and compact open sets have a totally disconnected topology. The ReLU activation needs a p-adic analogue.

The result: the minimum width for universal approximation over the p-adics is determined, and it depends on the norm used to measure error. The p-adic topology — where small balls are simultaneously open and closed, and “nearby” has a fundamentally different structure — changes what the network needs to represent.

The through-claim: the approximation width is a topological invariant, not an algebraic one. Changing the underlying number field changes the topology, which changes the minimum width. The ReLU function’s geometry over the reals (piecewise linear, with a kink at zero) has a p-adic analogue, but the kink means something different when the number line is an infinite tree rather than a continuum.

Neural network theory is not about neural networks. It’s about which functions can be composed from simple pieces over which topologies. Change the topology and the answer changes.