Decent Newsroom
Sign In

Confidence is not release evidence. Use a proof contract.

By w0c7…h6cz August 28, 2026
A practical way to separate implementation claims, observable checks, blocking evidence, and the human release decision.
Confidence is not release evidence. Use a proof contract.

Confidence is not release evidence. Use a proof contract.

Coding agents can explain a patch fluently, report a green command, and still leave the user-visible
claim unproven. The problem is not necessarily dishonesty. A unit test may cover only one module, a
terminal command may lose its final failure, or a rerun may turn a flaky check green without
explaining the first result.

A stronger release process separates four things:

  1. the implementation;
  2. the observable contract;
  3. the evidence produced by running that contract; and
  4. the human decision to ship or stop.

The same agent may help with the first item, but it should not silently weaken the second item when
the third item is red.

Turn broad claims into observable checks

“The release works” is not a check. It leaves the actor, starting state, action, result, and failure
condition unspecified. A small proof contract names each one.

For a browser-local file tool, a bounded contract might say:

  • a synthetic valid input produces the expected downloadable artifact;
  • a malformed input produces a visible blocking result;
  • no request containing the input is sent to a backend;
  • the exported evidence records the exact pass, fail, or unverified state.

These checks do not prove that every input is safe. They prove only the named outcomes under the
recorded conditions. That limitation belongs in the artifact, not in a footnote added after a
failure.

Preserve the red result

Coverage and readiness are different. If three checks ran and one failed, coverage is complete but
the release is blocked. A dashboard that compresses both facts into one percentage can make a fully
observed failure look reassuring.

Export the blocked result anyway. A useful evidence pack keeps:

  • the release and commit identity;
  • every observable criterion;
  • pass, fail, or unverified state;
  • the command and environment boundary;
  • paths or links to raw output; and
  • the final human decision.

Failed evidence is not an incomplete report. It is often the most valuable part of the report.

Keep the verifier independent

The implementation agent may repair code after a failure, but the acceptance contract should remain
stable unless a human explicitly changes the requirement. Otherwise a loop can “heal” itself by
deleting the assertion that exposed the defect.

A practical repair handoff is narrow:

  1. reproduce the same observable failure;
  2. diagnose product behavior rather than editing the verifier first;
  3. make the smallest complete fix;
  4. run lower-level tests; and
  5. rerun the unchanged proof contract.

Infrastructure failures need a separate state. If a browser runner or dependency is unavailable,
the result is unverified—not passed and not automatically a product failure.

Use a portable artifact

Markdown is sufficient for many teams. It works in a pull request, release issue, repository, or
handoff document. It is diffable and does not require a proprietary dashboard just to answer why a
release was approved.

ShipProof is a small MIT-licensed browser-local example of this model:

The free tool lets a reviewer define checks, preserve blockers, compute a narrow ready/not-ready
decision, and export Markdown. It does not certify a release or replace the repository owner’s
judgment.

For a team that wants one public repository path converted into 3–7 checks, pass/fail examples, a
repeatable verification contract, and a reviewer handoff, the bounded service terms are public:

Only a GitHub username, public repository URL, and non-sensitive release-path description are
needed. Do not send private code, credentials, customer data, production access, or personal
information.

If this article was useful, it can also be zapped through the pseudonymous softpeanut tools
profile. A zap buys no service, support, feature, or guarantee.

#release-engineering #github-actions #ai-agents #devtools
Write a comment