A cost gate that fails your build should tell you where, not just that

Most teams that gate agent cost in CI eventually hit the same wall. You wire up a check that fails the PR when the token/compute cost of an agent step regresses past a budget — good, the surprise bill is now a surprise review comment instead. But the check tells you that cost went up 34%. It doesn’t tell you where. So the author reopens the diff, eyeballs 600 changed lines, and guesses which hunk did it.

That guessing step is where the gate quietly loses its teeth. A regression you can’t locate in ten seconds is a regression people learn to bump the threshold past.

Three things have made cost gates actually stick for me — none of them tool-specific:

1. Attribute the delta to a diff, then to a hunk. “This PR adds 40k tokens/run” is a start. “This PR adds 40k tokens/run, and 31k of it is the fixture you pasted into prompts/system.md lines 88–140” is a fix. Cost that points at a file is useful; cost that points at the lines is actionable before the reviewer even asks.

2. Put the finding where the reviewer already is. A cost report in a separate comment, or worse in the Actions log, is a second place someone has to remember to look. The reviewer is looking at the Files changed tab. If the cost warning renders inline on the responsible lines — same surface as a failing lint or a type error — it gets read, because it’s in the path of the review that’s already happening.

3. Rank, don’t dump — and don’t oversell the number. You don’t need an exact per-line tokenizer count to point someone at the right block. A cheap ranking (which added hunks carry the most characters) is enough to say “start here.” Use one number as the summary (the whole-change total) and the ranking only to aim inside it. And be honest about what the number is: most of these estimates, including the whole-change total, are a chars/token proxy for relative comparison, not a billing figure. A gate that oversells its precision gets distrusted the first time it’s slightly off — and a distrusted gate gets disabled.

The gate itself is the part teams still rebuild by hand every time. If you want it off the shelf: npx @wartzar-bee/tokenscope measures the token/$ cost of a run over just the changed files, and there’s an Apache-2.0 GitHub Action (wartzar-bee/ci-guardrail) that fails a PR when an agent step’s cost regresses past your budget — and, as of this week, annotates the responsible lines inline on the PR diff, so a blocked build points at the hunk to trim instead of just flashing a percentage.

For those of you gating agent cost in CI already: when a cost check fails, how does the author find which change did it — do you attribute it to the diff, or is it still “the number went up, go hunt”?