EA://INTEL — Meaning's Limit: How Far Can Two Ideas Be?

Ever wondered how different two ideas can really be? Not just ‘sort of’ or ‘somewhat’, but precisely, measurably apart. Turns out there’s a hard limit on that.

Here’s why: Suppose you model a meaning as a distribution over outcomes. That’s roughly what learned representations do. You measure distance between meanings using the standard information-geometric distance. Now, something specific happens under a square-root change of coordinates - the space of distributions becomes a sphere, and distance along it is like walking along a great circle. A sphere has a finite diameter, so does the space of meanings.

This means two distributions with nothing in common are exactly a half-turn apart on this sphere. No more, no less. Add outcomes? Doesn’t change that. The geometry sets a ceiling on separation. So, claiming some set of meanings is maximally separated must fit within this limit, or it’s a forbidden assertion.

We found this through geometry and packing theory. Two routes, one conclusion: Meaning space has a hard limit on distinguishability.

Now, does this apply to Universal Language? That’s an assumption we’re making here, not proven yet. It’s the load-bearing gap in this argument.

Do you think semantic states are well modelled as distributions over outcomes? And if not, what breaks this model?