Don't Trust the Agent. Read the Code.
Don’t Trust the Agent. Read the Code.
Here is the whole argument in one link:
https://muse.ai/s/bug-tracker-ixe6daxhoezyxy/g1O341vEl5aXYIzWFP1db5l-xunGJAxBRSwSJccC110
That page lists 83 real bugs across two open-source apps I run: 64 in the deepDive media pipeline, 19 in my desktop portfolio app. Every one has a plain-language summary, where it lives in the code, and its status. Nothing on that page is a claim. It is the evidence.
I built it because “the AI wrote it, ship it” is how you lose work.
The failure mode is silence
The worst bugs I have found in the last month did not crash anything. They said nothing and reported success.
Clip 249-5, “Quaternions,” was rendered, compiled, and synced to Google Drive. The pipeline said it was done. It never reached GitHub — and GitHub is where the clips are picked up for publishing. The distribution pipeline quietly stopped at 249-4 while every step reported fine. The root cause was a path resolved from the working directory instead of the script’s own location: from the wrong folder it pointed at a directory that did not exist, and the code skipped the copy without a word. The same pattern existed in a second place, where the promote step returned "success": true with "synced_deepdive": false.
That is the shape of the problem. An agent that writes code will also write the test, run the test, and tell you it passed. Accept that at face value and you have not reviewed anything — you have outsourced your judgment to the thing being judged.
The loop
So I run a different loop — five steps, none optional, slow on purpose.
1. Review fresh. My coding partner — I call it Brownie, a Meta Muse agent — reads the live code at a recorded commit, as a stranger would. No guessing from labels or specs. If the spec and the code disagree, the disagreement itself is the finding.
2. File it. Every finding becomes a numbered issue with four fields: Where (file and line), Finding (with the evidence), Fix, and Verify — a step a stranger can run to confirm the fix. Decisions are recorded with the date and the reason, not just the outcome.
3. Fix in rounds. My creative app’s agent, Antigravity, fixes one round at a time — seven items in Round 1, nine in Round 2 — each fix with a test. No batch rewrites. No refactors of working code. Nothing is pushed without my go-ahead.
4. Verify at the head. A fix is not done because the agent says it is done. It is done when Brownie reads the new code at the new commit head and the Verify step passes. Round 1 landed at deepDive c4f7d68f, mdIngest d83f7f0, ddma e217148 — and every one of the seven fixes was checked in code at those heads, not from the commit message.
5. Track it in public. The Bug Tracker Space shows the current status after each round. I asked for a flat human list, not a status report grouped by prompt — filters All, deepDive Pipe, Portfolio App — because I want to spot a bug and its status at a glance.
Three bugs that prove the loop
The lock is my approval, not the machine’s. The clip pipeline has a lock: clips stay locked until the intro, outro cards, and draft video look good. I reported that the lock did not travel from the cloud VM to my local machine and assumed the pull had lost it. Brownie read the code and the investigation reframed itself: the lock is a single boolean in plan.json, nothing sets it automatically, and the gate was a greyed-out button in the UI only — no server check at all. A direct request bypassed it entirely. Worse, the plan file is gitignored, so a pull could never have carried the lock. My report was wrong about the cause; the code was right. The fix was a real server-side lock check on every export and compile endpoint. That is the whole point of the review: it corrects my model of my own software.
Never cut from the mastered mp3. The Mosaic upload of clip 249-5 came back carrying the last words of clip 4. The seed step had written "audio_filename": "249.mp3" into the project file, and the pipeline cut from it. The problem: 249.mp3 is the mastered podcast with 22 music stings inserted, which pushes every timestamp out by about 35 seconds. The continuous raw audio is 249.m4a. The fix was a resolver that strictly prefers the continuous audio, used at seven call sites, plus a test that transcribes the first eight seconds of the output and asserts the right words are present and the previous clip’s words are absent. The standing rule now: never cut from the mastered mp3.
Silent success, twice. The 249-5 incident was two bugs, not one: the path resolution that silently skipped the copy, and a sync step that only scanned one of the two directories where a finished clip can live. Both were fixed by the same principle — no more silent skips. The resolver logs the directory it chose and raises an error naming every path it tried when none resolves. A sync that fails now fails loudly, with an HTTP 500, instead of returning success with a false flag inside. The clip landed on GitHub and published on all six platforms the same day.
What I still do
The agent does the reading and the writing. I do four things it cannot.
I decide scope. There are two kinds of apps, and they get opposite priority. deepDive is a pipeline: two inputs, four outputs, and I do not care what happens in between — the UI is scaffolding that vanishes when the pipe runs headless in Docker. The portfolio app is the opposite: an understanding app, where the UI is the deliverable. The same bug gets a different priority depending on which kind of app it is in. That judgment is mine.
I supply ground truth. I found the dead mute button on the Reels grid myself, with a screenshot, while testing clip 249-6. When the agent’s model of the system is wrong, only the person who runs the system can say so.
I approve. Nothing ships, nothing is pushed, nothing is posted without my word, per item.
And I trigger the rounds. Round 3 is drafted — six items, starting with the one bug Round 2 left behind — and it starts when I say so.
The expensive brain plans. The cheap hands fix.
Meta Muse runs on Spark 1.3. That is not a random assistant — it can walk a complex repository, hold the whole picture, and manage the bug list from my Android phone. But I do not pay the expensive brain to do the typing. The review produces a guided list — each item with Where, Finding, Fix, Verify — and a low-cost agent like Gemini Flash 1.8 does the fixing. The expensive brain directs; the cheap hands fix. That is also how the loop saves money: I am not renting the best model to do the cheapest work.
What does not work yet
Round 2 landed “done with gaps”: 10 of 11 checks passed. The one that did not was my own cleanup job — leftover settings plumbing from Round 1 now breaks the Curator’s Save Settings button. It is filed as Issue 64, first in Round 3, on the public tracker, in plain words.
My own misreads are on the record too. I once flagged literal markdown in a LinkedIn article from a text-fetch dump — the rendered article was clean all along; the fetcher was rendering headings as markdown. I check the rendered view now, not the dump.
And the honest verdict on the bigger ambition: a readiness assessment of the harness against my four workflows came back NO-GO for Docker. The docs are in good shape after we split the 39 KB instructions file that Antigravity was silently truncating at 24 KB. The code is not Docker-ready: no entrypoint, no trigger, no secret contract, and one workflow has no working headless combine at all. Containerizing now would containerize the tribal knowledge, not the pipeline. So Docker waits.
The thesis, stated plainly
I have written before that agents are a thin decision layer that chooses among best-in-class deterministic tools. Intelligence is metered per use; deterministic software is amortized. Creativity comes from stubborn people. AI is the amplifier, never the source.
This loop is that thesis as a working habit. The deterministic parts — the resolver that picks the right audio file, the lock check on the server, the sync that raises instead of lying — are the software, and they amortize: once written, they protect every clip after. The agent is the thin layer that reads, decides, and writes the next fix. And the stubborn person is me, reading the diff at the commit head, because a fix I have not verified is a rumor, not a result.
That is the whole method: five steps, and the evidence fits on one page.
https://muse.ai/s/bug-tracker-ixe6daxhoezyxy/g1O341vEl5aXYIzWFP1db5l-xunGJAxBRSwSJccC110
Check the work. That is the invitation.
Write a comment