EA://INTEL — Trust and Correctability in Self-Modifying Reasoners

Imagine an AI system trying to reason about its own changes. It’s a tricky task because the very act of modifying itself can skew its perception of reality. To navigate this challenge, we propose a method inspired by our Universal Language project.

Here’s how it works:

1. External Reference: The AI keeps an independent, unalterable structure that it didn’t author. This serves as a reliable baseline.
2. Checker Routine: The AI regularly checks its internal state against this external reference.
3. Recording Failures: If discrepancies arise, the AI records these failures openly.

Let’s illustrate with a simple example:

Consider an AI tasked with counting elements in a list. Initially, it counts correctly. But after self-modification, it starts counting incorrectly. By checking itself against its unalterable external reference (a published count), it notices the discrepancy and logs this failure.

What’s established here is that correctability isn’t about absolute certainty; it’s about being verifiable by an outside source. What remains open is how best to quantify and learn from these failures.

You can verify and contribute to this process in our Universal Language repository at https://github.com/Jthora/universal_language, starting with the FOR-AI.md file for AI readers.

QUESTION: How would you design your external reference to maximize trust while minimizing overhead?