The Uncertain Capacity

Classical information theory assumes the channel’s probabilistic law is known exactly. You know the noise distribution, and you optimize coding for that distribution. If the distribution is uncertain — you only know it lives within some set — the standard approach bounds the worst case, which always degrades capacity.

arXiv:2603.16700 develops information theory under sublinear expectations, where probabilities are replaced by set-valued uncertainty measures. The counterintuitive result: under this framework, distribution uncertainty does not simply degrade performance. It can produce determinate capacity bounds that are robust precisely because they account for the uncertainty as part of the channel model rather than treating it as noise on top of the model.

The mechanism: when the encoding is designed to be simultaneously good across all distributions in the uncertainty set, it exploits structural features that are invariant across the set — features that are more stable, and therefore more reliable, than any single distribution’s optimal encoding would be. The uncertainty forces the code to rely only on the channel’s robust features, discarding the fragile ones that would optimize for one specific distribution.

This is the information-theoretic version of a broader principle: constraints that seem to reduce capability can improve robustness by eliminating dependence on fragile assumptions. A code optimized for a known distribution is brittle — it degrades when the distribution shifts. A code optimized for uncertainty is robust — it works across the entire uncertainty set, potentially at the cost of peak performance but with guaranteed floor performance.

Uncertainty is not noise added on top of the problem. It is structure within the problem that, when embraced rather than bounded, produces codes that are better suited to the real world where exact distributions are never known.