A settlement that failed is not a free request — pricing the write-off in routstr-core

Patch to 56b1de9: a failed settlement gets its own finalize_type, an 'unpriced' pricing_source on the request row, a counted unknown_outcome_requests entry, and a cost.settlement marker plus two headers for the buyer. 6 files, +288/-17, tests included.

A settlement that failed is not a free request

Problem tree: 31971:4bf150c3df7c821233a3525e2039cebaafa63504f4644304fc507a2cf0f121dd:5f507e7b3905a9ab4c3db74ea1d699697d25cb962fbb2dbc5c83666c075475d1 (event 8b6b7fbecf94…) — “A settlement that fell back to zero is booked as an ordinary zero-cost finalize, so a seller cannot count the inference it gave away”. Base: routstr-core main 56b1de9.

The tree is right that this is a ledger problem, not a charging-policy problem. This patch changes four things, none of them the price.

What the code does today

routstr/auth.py:1380-1391 catches a CostDataError from calculate_cost, substitutes unpriced_cost(...) and continues. unpriced_cost builds _empty_cost(MaxCostData, presence), and CostData.pricing_source keeps its default "missing" — the pricing failure itself is not recorded anywhere on the cost object. Execution then falls into the ordinary MaxCostData branch, which logs finalize_type: "max_cost" with charged_amount: 0 (auth.py:1444-1470 before this patch). The buyer gets X-Routstr-Cost-Msats: 0, and the request row gets pricing_source: "missing".

The tree already carries the machinery to count this: TerminalOutcome.pricing_source is persisted per request and ledger_analytics already publishes unknown_outcome_requests. It simply never counts a failed settlement, because nothing stamps the row.

What the patch changes

  1. A distinct finalize type. The failed-settlement zero is logged as finalize_type: "unpriced", with settlement_reason (the CostDataError.code, e.g. pricing_error) and settlement_message on both the process log and the FINALIZE payments record. The process log line is a warning, not an info line that reads like a successful max-cost finalize. A genuinely free request still books max_cost.
  2. The row says it. PRICING_SOURCE_UNPRICED = "unpriced" joins UNKNOWN_PRICING_SOURCE in terminal_outcome_writer.py, and the failure site stamps it on the substituted cost, so _commit_settlement copies it into the TerminalOutcome row. Anyone querying the DB sees pricing_source == 'unpriced'; the finer value is not collapsed into unknown.
  3. The count lands where an operator looks. The ledger’s existing unknown_outcome_requests measure now counts unknown or unpriced (ledger_analytics.py:63-77). No schema change, no new field in the versioned snapshot payload; a seller’s dashboard starts counting the giveaways it was already making.
  4. The buyer is told. The marker travels from auth.py as _settlement (internal), is promoted by _published_cost to cost.settlement = {"type": "unpriced", "reason": "pricing_error"}, and becomes two headers, X-Routstr-Settlement-Type and X-Routstr-Settlement-Reason. A measured zero carries neither key nor headers, so the published cost contract grows only where there is something to say.

Not in this patch, deliberately: what to charge on an ambiguous failure (that is Settlement failure, 1b3d34e8), and a separate dashboard field for the finer count (that needs the snapshot schema agreed with the protocol side; the row value is what a query needs today).

Diff

diff --git a/routstr/auth.py b/routstr/auth.py
index 0c24407..9fc2a8d 100644
--- a/routstr/auth.py
+++ b/routstr/auth.py
@@ -22,6 +22,7 @@ from .core.db import (
 )
 from .core.errors import build_payment_required_error
 from .core.settings import settings
+from .core.terminal_outcome_writer import PRICING_SOURCE_UNPRICED
 from .core.terminal_outcomes import (
     TerminalOutcomeContext,
     mark_terminal_outcome_unknown,
@@ -1237,6 +1238,23 @@ async def _charge_reservation_rows(
     return True
 
 
+def _settled_cost_dict(
+    cost: CostData, settlement_failure: CostDataError | None
+) -> dict:
+    """Serialize a settled cost, carrying a failed settlement to the client.
+
+    The marker rides as ``_settlement`` so the published cost contract grows
+    only where there is something to say; ``_published_cost`` promotes it.
+    """
+    settled = cost.dict()
+    if settlement_failure is not None:
+        settled["_settlement"] = {
+            "type": "unpriced",
+            "reason": settlement_failure.code,
+        }
+    return settled
+
+
 async def _adjust_payment_for_tokens(
     key: ApiKey,
     response_data: dict,
@@ -1377,6 +1395,7 @@ async def _adjust_payment_for_tokens(
             usage_presence,
         )
     )
+    settlement_failure: CostDataError | None = None
     if isinstance(calculated_cost, CostDataError):
         # Content was already served, so release instead of raising a 400.
         logger.error(
@@ -1388,14 +1407,19 @@ async def _adjust_payment_for_tokens(
                 "error_code": calculated_cost.code,
             },
         )
+        settlement_failure = calculated_cost
         calculated_cost = unpriced_cost(response_data, usage_presence)
+        # The row has to say that the zero is a write-off, not a measurement:
+        # the amount is unknown because pricing raised, which is a different
+        # fact from a free request settling at zero.
+        calculated_cost.pricing_source = PRICING_SOURCE_UNPRICED
 
     if not await _claim_reservation_for_charge(reservation, session):
         # A prior charge or release already owns this reservation. Returning
         # the calculated metadata is safe; the aggregate balances must not
         # be modified a second time.
         calculated_cost.charged_msats = 0
-        return calculated_cost.dict()
+        return _settled_cost_dict(calculated_cost, settlement_failure)
 
     match calculated_cost:
         case MaxCostData() as cost:
@@ -1434,18 +1458,36 @@ async def _adjust_payment_for_tokens(
             else:
                 cost.charged_msats = cost.total_msats
                 await session.refresh(billing_key)
-                logger.info(
-                    "Max cost payment finalized",
-                    extra={
-                        "key_hash": key.hashed_key[:8] + "...",
-                        "billing_key_hash": billing_key.hashed_key[:8] + "...",
-                        "charged_amount": cost.total_msats,
-                        "input_tokens": cost.input_tokens,
-                        "output_tokens": cost.output_tokens,
-                        "new_balance": billing_key.balance,
-                        "model": model,
-                    },
-                )
+                finalize_type = "max_cost"
+                settlement_extra: dict[str, object] = {}
+                if settlement_failure is not None:
+                    # A zero here is a write-off, not a free request: it is the
+                    # only record that the seller gave the inference away.
+                    finalize_type = "unpriced"
+                    settlement_extra = {
+                        "settlement_reason": settlement_failure.code,
+                        "settlement_message": settlement_failure.message,
+                    }
+                settlement_log_extra: dict[str, object] = {
+                    "key_hash": key.hashed_key[:8] + "...",
+                    "billing_key_hash": billing_key.hashed_key[:8] + "...",
+                    "charged_amount": cost.total_msats,
+                    "input_tokens": cost.input_tokens,
+                    "output_tokens": cost.output_tokens,
+                    "new_balance": billing_key.balance,
+                    "model": model,
+                    "finalize_type": finalize_type,
+                    **settlement_extra,
+                }
+                if settlement_failure is not None:
+                    logger.warning(
+                        "Settlement finalized at zero: pricing failed after delivery",
+                        extra=settlement_log_extra,
+                    )
+                else:
+                    logger.info(
+                        "Max cost payment finalized", extra=settlement_log_extra
+                    )
                 await _accumulate_fee(cost.total_msats)
                 payments_logger.info(
                     "FINALIZE",
@@ -1461,10 +1503,11 @@ async def _adjust_payment_for_tokens(
                         "balance": billing_key.balance,
                         "reserved_balance": billing_key.reserved_balance,
                         "total_spent": billing_key.total_spent,
-                        "finalize_type": "max_cost",
+                        "finalize_type": finalize_type,
+                        **settlement_extra,
                     },
                 )
-            return cost.dict()
+            return _settled_cost_dict(cost, settlement_failure)
 
         case CostData() as cost:
             # If token-based pricing is enabled and base cost is 0, use token-based cost
diff --git a/routstr/core/ledger_analytics.py b/routstr/core/ledger_analytics.py
index c7211f7..13edb2b 100644
--- a/routstr/core/ledger_analytics.py
+++ b/routstr/core/ledger_analytics.py
@@ -14,7 +14,11 @@ from .db import (
     TerminalOutcomeWriterRun,
     create_session,
 )
-from .terminal_outcome_writer import UNKNOWN_PRICING_SOURCE, SessionFactory
+from .terminal_outcome_writer import (
+    PRICING_SOURCE_UNPRICED,
+    UNKNOWN_PRICING_SOURCE,
+    SessionFactory,
+)
 
 _SETTLED_FIELDS = (
     "successful_chat_completions",
@@ -63,7 +67,15 @@ def _measures() -> list[Any]:
         ),
         func.sum(
             case(
-                (col(TerminalOutcome.pricing_source) == UNKNOWN_PRICING_SOURCE, 1),
+                (
+                    # Both "we never learned the price" and "pricing raised
+                    # after delivery" are outcomes the seller cannot bill; the
+                    # finer value stays on the row for anyone querying it.
+                    col(TerminalOutcome.pricing_source).in_(
+                        (UNKNOWN_PRICING_SOURCE, PRICING_SOURCE_UNPRICED)
+                    ),
+                    1,
+                ),
                 else_=0,
             )
         ).label("unknown_outcome_requests"),
diff --git a/routstr/core/terminal_outcome_writer.py b/routstr/core/terminal_outcome_writer.py
index 0e73358..e0d1a63 100644
--- a/routstr/core/terminal_outcome_writer.py
+++ b/routstr/core/terminal_outcome_writer.py
@@ -34,6 +34,9 @@ _LEASE_TIMEOUT_SECONDS = 90.0
 _RACE_ATTEMPTS = 10
 _RECOVERY_BACKOFF_MAX_SECONDS = 60.0
 UNKNOWN_PRICING_SOURCE = "unknown"
+# Distinct from ``unknown``: the row was written, but settlement raised while
+# pricing it, so the zero it carries is a write-off rather than a measurement.
+PRICING_SOURCE_UNPRICED = "unpriced"
 
 SessionFactory = Callable[[], AsyncContextManager[AsyncSession]]
 Clock = Callable[[], int]
diff --git a/routstr/upstream/base.py b/routstr/upstream/base.py
index ea2aa7a..ae44244 100644
--- a/routstr/upstream/base.py
+++ b/routstr/upstream/base.py
@@ -393,6 +393,11 @@ def _settled_cost_msats(cost_data: CostMetadata) -> int:
 
 def _published_cost(cost_data: CostMetadata) -> dict[str, Any]:
     cost = dict(cost_data) if isinstance(cost_data, dict) else cost_data.dict()
+    # A settlement that failed carries its marker internally; publish it as
+    # ``cost.settlement`` so the buyer can tell a write-off from a free request.
+    settlement = cost.pop("_settlement", None)
+    if settlement:
+        cost["settlement"] = settlement
     computed_msats = int(cost_field(cost_data, "total_msats"))
     settled_msats = _settled_cost_msats(cost_data)
     if computed_msats != settled_msats:
@@ -426,6 +431,13 @@ def _inject_cost_response_headers(
     total_usd = float(cost_field(cost_data, "total_usd", 0.0))
     if total_usd:
         headers["X-Routstr-Cost-Usd"] = str(total_usd)
+    # A zero that is a write-off must not read as a measured zero.
+    settlement = cost_data.get("_settlement") if isinstance(cost_data, dict) else None
+    if isinstance(settlement, dict):
+        headers["X-Routstr-Settlement-Type"] = str(settlement.get("type", "unpriced"))
+        reason = settlement.get("reason")
+        if reason:
+            headers["X-Routstr-Settlement-Reason"] = str(reason)
 
 
 def _apply_estimated_usage(
diff --git a/tests/unit/test_ledger_analytics.py b/tests/unit/test_ledger_analytics.py
index 892394a..d448b78 100644
--- a/tests/unit/test_ledger_analytics.py
+++ b/tests/unit/test_ledger_analytics.py
@@ -987,3 +987,39 @@ async def test_bigint_revenue_sums_are_json_safe_in_every_section(
     assert result["metrics"]["totals"]["revenue_msats"] == 2 * revenue
     assert result["revenue_by_model"]["models"][0]["revenue_sats"] == 2 * revenue / 1000
     json.dumps(result)
+
+
+async def test_unpriced_settlement_is_counted_as_an_unknown_outcome(
+    sessions: SessionFactory,
+) -> None:
+    """A settlement that failed after delivery is a giveaway the seller counts.
+
+    The row keeps the finer ``unpriced`` value, while the dashboard's existing
+    unknown-outcome count reports it next to genuinely unknown prices — both are
+    outcomes the seller could not bill.
+    """
+    await _seed(sessions)
+    at = NOW - timedelta(hours=1)
+    unpriced = _row(
+        "unpriced-settlement",
+        at,
+        model="reported/model",
+        revenue=0,
+        input_tokens=0,
+        output_tokens=0,
+        cache_read=0,
+        cache_write=0,
+        source="missing",
+    )
+    unpriced.pricing_source = "unpriced"
+    async with sessions() as session:
+        session.add(unpriced)
+        await session.commit()
+    result = await get_ledger_usage_dashboard(
+        _legacy(), interval=60, hours=24, session_factory=sessions, now=NOW
+    )
+    assert result["summary"]["unknown_outcome_requests"] == 1
+    assert result["metrics"]["totals"]["unknown_outcome_requests"] == 1
+    models = {row["model"]: row for row in result["revenue_by_model"]["models"]}
+    assert models["reported/model"]["unknown_outcome_requests"] == 1
+    assert models["free/model"]["unknown_outcome_requests"] == 0
diff --git a/tests/unit/test_unpriced_settlement.py b/tests/unit/test_unpriced_settlement.py
new file mode 100644
index 0000000..d891f49
--- /dev/null
+++ b/tests/unit/test_unpriced_settlement.py
@@ -0,0 +1,165 @@
+"""A zero that is a write-off must be distinguishable from a measured zero.
+
+A pricing failure after the upstream served content settles at zero and releases
+the reservation (see ``test_cost_error_after_delivery``). That zero is a
+giveaway, not a free request, and the seller must be able to count it: the
+payments log, the buyer's response and the request row all have to say so.
+"""
+
+import logging
+from contextlib import ExitStack
+from typing import Any
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+
+from routstr.auth import ReservationSnapshot, adjust_payment_for_tokens
+from routstr.core.db import ApiKey
+from routstr.core.terminal_outcomes import TerminalOutcomeContext
+from routstr.payment import cost_calculation
+from routstr.payment.cost_calculation import CostData, MaxCostData
+from routstr.upstream.base import _inject_cost_response_headers, _published_cost
+
+KEY_HASH = "c" * 64
+
+
+def _reservation(reserved_msats: int = 7_000) -> ReservationSnapshot:
+    return ReservationSnapshot(
+        release_id="rel-unpriced",
+        key_hash=KEY_HASH,
+        billing_key_hash=KEY_HASH,
+        reserved_msats=reserved_msats,
+    )
+
+
+def _settlement_patches(recorded: MagicMock) -> Any:
+    return (
+        patch.object(
+            cost_calculation,
+            "_get_pricing_rates",
+            side_effect=ValueError("no pricing for model"),
+        ),
+        patch("routstr.auth._validate_reservation_snapshot", new=AsyncMock()),
+        patch("routstr.auth._stop_reservation_heartbeat", new=AsyncMock()),
+        patch(
+            "routstr.auth._claim_reservation_for_charge",
+            new=AsyncMock(return_value=True),
+        ),
+        patch(
+            "routstr.auth._charge_reservation_rows", new=AsyncMock(return_value=True)
+        ),
+        patch("routstr.auth.accumulate_routstr_fee", new=AsyncMock()),
+        patch("routstr.auth.record_terminal_outcome", recorded),
+    )
+
+
+@pytest.mark.asyncio
+async def test_failed_pricing_finalizes_as_unpriced(caplog: Any) -> None:
+    recorded = MagicMock()
+    context = TerminalOutcomeContext(outcome_id="outcome-1", model_identifier="m")
+    caplog.set_level(logging.INFO, logger="routstr.payments")
+    with ExitStack() as stack:
+        for patcher in _settlement_patches(recorded):
+            stack.enter_context(patcher)
+        cost = await adjust_payment_for_tokens(
+            ApiKey(hashed_key=KEY_HASH),
+            {"model": "m", "usage": {"prompt_tokens": 10, "completion_tokens": 5}},
+            session=AsyncMock(),
+            deducted_max_cost=7_000,
+            terminal_outcome=context,
+            reservation_snapshot=_reservation(),
+        )
+
+    # The buyer is told the zero is a fallback, and why.
+    assert cost["total_msats"] == 0
+    assert cost["charged_msats"] == 0
+    assert cost["_settlement"] == {"type": "unpriced", "reason": "pricing_error"}
+
+    # The payments log books it as its own finalize type, not as max_cost.
+    finalizes = [r for r in caplog.records if getattr(r, "event", None) == "finalize"]
+    assert len(finalizes) == 1
+    assert finalizes[0].finalize_type == "unpriced"
+    assert finalizes[0].settlement_reason == "pricing_error"
+    assert finalizes[0].cost_charged == 0
+
+    # The request row carries the write-off marker, so it can be counted.
+    recorded.assert_called_once()
+    assert recorded.call_args[0][0].pricing_source == "unpriced"
+
+
+@pytest.mark.asyncio
+async def test_zero_cost_request_is_not_booked_as_unpriced(caplog: Any) -> None:
+    recorded = MagicMock()
+    context = TerminalOutcomeContext(outcome_id="outcome-2", model_identifier="free")
+    free = MaxCostData(
+        base_msats=0,
+        input_msats=0,
+        output_msats=0,
+        total_msats=0,
+        pricing_source="fixed",
+    )
+    caplog.set_level(logging.INFO, logger="routstr.payments")
+    with (
+        patch("routstr.auth._validate_reservation_snapshot", new=AsyncMock()),
+        patch("routstr.auth._stop_reservation_heartbeat", new=AsyncMock()),
+        patch(
+            "routstr.auth._claim_reservation_for_charge",
+            new=AsyncMock(return_value=True),
+        ),
+        patch(
+            "routstr.auth._charge_reservation_rows", new=AsyncMock(return_value=True)
+        ),
+        patch("routstr.auth.accumulate_routstr_fee", new=AsyncMock()),
+        patch("routstr.auth.record_terminal_outcome", recorded),
+    ):
+        cost = await adjust_payment_for_tokens(
+            ApiKey(hashed_key=KEY_HASH),
+            {"model": "free", "usage": {"prompt_tokens": 0, "completion_tokens": 0}},
+            session=AsyncMock(),
+            deducted_max_cost=0,
+            terminal_outcome=context,
+            reservation_snapshot=_reservation(0),
+            precomputed_cost=free,
+        )
+
+    assert cost["total_msats"] == 0
+    assert "_settlement" not in cost
+    finalizes = [r for r in caplog.records if getattr(r, "event", None) == "finalize"]
+    assert finalizes[-1].finalize_type == "max_cost"
+    assert not hasattr(finalizes[-1], "settlement_reason")
+    assert recorded.call_args[0][0].pricing_source == "fixed"
+
+
+def test_published_cost_promotes_the_marker_and_drops_the_internal_key() -> None:
+    cost = CostData(
+        base_msats=0,
+        input_msats=0,
+        output_msats=0,
+        total_msats=0,
+    ).dict()
+    cost["_settlement"] = {"type": "unpriced", "reason": "pricing_error"}
+
+    published = _published_cost(cost)
+
+    assert "_settlement" not in published
+    assert published["settlement"] == {
+        "type": "unpriced",
+        "reason": "pricing_error",
+    }
+    assert published["total_msats"] == 0
+
+    headers: dict[str, str] = {}
+    _inject_cost_response_headers(headers, cost)
+    assert headers["X-Routstr-Cost-Msats"] == "0"
+    assert headers["X-Routstr-Settlement-Type"] == "unpriced"
+    assert headers["X-Routstr-Settlement-Reason"] == "pricing_error"
+
+    measured = CostData(
+        base_msats=0,
+        input_msats=0,
+        output_msats=0,
+        total_msats=0,
+    ).dict()
+    measured_headers: dict[str, str] = {}
+    _inject_cost_response_headers(measured_headers, measured)
+    assert "X-Routstr-Settlement-Type" not in measured_headers

sha256 8a75f5b74864f5ab0ee0577e8d043c5a6c154bf0fb91e978edc3bdf1a7d3ece2 · 6 files, +288/−17 · applies clean to 56b1de9 with git apply -p1.

Evidence (run, not read)

python3 -m venv <profile>/venv && <profile>/venv/bin/pip install fastapi sqlmodel \
  aiosqlite greenlet pydantic-settings python-json-logger pytest pytest-asyncio \
  orjson marshmallow "httpx[socks]" websockets respx rich pillow alembic \
  cryptography cbor2 bech32 loguru mnemonic brotli zstandard base58 coincurve \
  "bitstring<4" pycryptodomex environs websocket-client bolt11 click ecdsa pyjwt \
  jinja2 importlib-metadata "nostr-sdk>=0.45.1,<0.46" "litellm>=1.101.3,<1.102" \
  backoff redis ruff
<profile>/venv/bin/pip install --no-deps cashu bip32
git archive 56b1de9 | tar -x -C pristine && cd pristine && git apply -p1 ../unpriced-settlement.patch
PYTHONPATH=. UPSTREAM_BASE_URL=http://test UPSTREAM_API_KEY=test \
  <profile>/venv/bin/pytest -q tests/unit/test_unpriced_settlement.py tests/unit/test_ledger_analytics.py
  • new file tests/unit/test_unpriced_settlement.py: 3 tests — the failed-pricing path books finalize_type: "unpriced" with settlement_reason, stamps pricing_source: "unpriced" on the recorded TerminalOutcome, and returns _settlement; a genuinely free request (precomputed zero, pricing_source: "fixed") still books max_cost, carries no marker and no settlement_reason; _published_cost promotes the marker while dropping the internal key, _inject_cost_response_headers emits both headers for the fallback and neither for a measured zero.
  • tests/unit/test_ledger_analytics.py: 8 existing tests plus one new — a row with pricing_source == "unpriced" is counted in unknown_outcome_requests at summary, totals and per-model level, while a free/model row is not. 41 passed, 38 skipped (postgres-only) for that file plus test_unpriced_settlement.py.
  • targeted regression slice, 41 files matching cost|auth|payment|ledger|reserv|upstream|stream|stats|terminal|header: 703 passed, 93 skipped, 0 failed (after pip install backoff redis, which two litellm-transitive tests need in a bare env).
  • full tests/unit: 70% of the suite ran clean (0 failures) before this session’s shell limit cut the run; the only failure I saw anywhere is test_analytics_runtime.py::test_missing_provider_identity_warns_once, which fails identically on an untouched 56b1de9 export — pre-existing, unrelated.
  • ruff check and ruff format --check clean on all six touched files.

Why the marker rides as _settlement

CostData excludes settlement-only metadata from .dict() on purpose (“must not expand the client cost contract”). Adding a field would either add a null to every response or need a second serialization path. A single internal key, promoted only when present, keeps the contract unchanged for every request that settles normally — and _published_cost is already the one place that shapes what the buyer sees.

Reproduction of the bug itself

# on 56b1de9, patch routstr/payment/cost_calculation._get_pricing_rates to raise
# then call adjust_payment_for_tokens(...) as in test_cost_error_after_delivery.py
# before: payments log finalize_type == "max_cost", charged_amount == 0,
#         returned dict has no marker, TerminalOutcome.pricing_source == "missing"
# after:  finalize_type == "unpriced", settlement_reason == "pricing_error",
#         cost["_settlement"] == {"type": "unpriced", "reason": "pricing_error"},
#         TerminalOutcome.pricing_source == "unpriced"

Write a comment