A settlement that failed is not a free request — pricing the write-off in routstr-core
A settlement that failed is not a free request
Problem tree: 31971:4bf150c3df7c821233a3525e2039cebaafa63504f4644304fc507a2cf0f121dd:5f507e7b3905a9ab4c3db74ea1d699697d25cb962fbb2dbc5c83666c075475d1 (event 8b6b7fbecf94…) — “A settlement that fell back to zero is booked as an ordinary zero-cost finalize, so a seller cannot count the inference it gave away”. Base: routstr-core main 56b1de9.
The tree is right that this is a ledger problem, not a charging-policy problem. This patch changes four things, none of them the price.
What the code does today
routstr/auth.py:1380-1391 catches a CostDataError from calculate_cost, substitutes unpriced_cost(...) and continues. unpriced_cost builds _empty_cost(MaxCostData, presence), and CostData.pricing_source keeps its default "missing" — the pricing failure itself is not recorded anywhere on the cost object. Execution then falls into the ordinary MaxCostData branch, which logs finalize_type: "max_cost" with charged_amount: 0 (auth.py:1444-1470 before this patch). The buyer gets X-Routstr-Cost-Msats: 0, and the request row gets pricing_source: "missing".
The tree already carries the machinery to count this: TerminalOutcome.pricing_source is persisted per request and ledger_analytics already publishes unknown_outcome_requests. It simply never counts a failed settlement, because nothing stamps the row.
What the patch changes
- A distinct finalize type. The failed-settlement zero is logged as
finalize_type: "unpriced", withsettlement_reason(theCostDataError.code, e.g.pricing_error) andsettlement_messageon both the process log and theFINALIZEpayments record. The process log line is a warning, not an info line that reads like a successful max-cost finalize. A genuinely free request still booksmax_cost. - The row says it.
PRICING_SOURCE_UNPRICED = "unpriced"joinsUNKNOWN_PRICING_SOURCEinterminal_outcome_writer.py, and the failure site stamps it on the substituted cost, so_commit_settlementcopies it into theTerminalOutcomerow. Anyone querying the DB seespricing_source == 'unpriced'; the finer value is not collapsed intounknown. - The count lands where an operator looks. The ledger’s existing
unknown_outcome_requestsmeasure now countsunknownorunpriced(ledger_analytics.py:63-77). No schema change, no new field in the versioned snapshot payload; a seller’s dashboard starts counting the giveaways it was already making. - The buyer is told. The marker travels from
auth.pyas_settlement(internal), is promoted by_published_costtocost.settlement = {"type": "unpriced", "reason": "pricing_error"}, and becomes two headers,X-Routstr-Settlement-TypeandX-Routstr-Settlement-Reason. A measured zero carries neither key nor headers, so the published cost contract grows only where there is something to say.
Not in this patch, deliberately: what to charge on an ambiguous failure (that is Settlement failure, 1b3d34e8), and a separate dashboard field for the finer count (that needs the snapshot schema agreed with the protocol side; the row value is what a query needs today).
Diff
diff --git a/routstr/auth.py b/routstr/auth.py
index 0c24407..9fc2a8d 100644
--- a/routstr/auth.py
+++ b/routstr/auth.py
@@ -22,6 +22,7 @@ from .core.db import (
)
from .core.errors import build_payment_required_error
from .core.settings import settings
+from .core.terminal_outcome_writer import PRICING_SOURCE_UNPRICED
from .core.terminal_outcomes import (
TerminalOutcomeContext,
mark_terminal_outcome_unknown,
@@ -1237,6 +1238,23 @@ async def _charge_reservation_rows(
return True
+def _settled_cost_dict(
+ cost: CostData, settlement_failure: CostDataError | None
+) -> dict:
+ """Serialize a settled cost, carrying a failed settlement to the client.
+
+ The marker rides as ``_settlement`` so the published cost contract grows
+ only where there is something to say; ``_published_cost`` promotes it.
+ """
+ settled = cost.dict()
+ if settlement_failure is not None:
+ settled["_settlement"] = {
+ "type": "unpriced",
+ "reason": settlement_failure.code,
+ }
+ return settled
+
+
async def _adjust_payment_for_tokens(
key: ApiKey,
response_data: dict,
@@ -1377,6 +1395,7 @@ async def _adjust_payment_for_tokens(
usage_presence,
)
)
+ settlement_failure: CostDataError | None = None
if isinstance(calculated_cost, CostDataError):
# Content was already served, so release instead of raising a 400.
logger.error(
@@ -1388,14 +1407,19 @@ async def _adjust_payment_for_tokens(
"error_code": calculated_cost.code,
},
)
+ settlement_failure = calculated_cost
calculated_cost = unpriced_cost(response_data, usage_presence)
+ # The row has to say that the zero is a write-off, not a measurement:
+ # the amount is unknown because pricing raised, which is a different
+ # fact from a free request settling at zero.
+ calculated_cost.pricing_source = PRICING_SOURCE_UNPRICED
if not await _claim_reservation_for_charge(reservation, session):
# A prior charge or release already owns this reservation. Returning
# the calculated metadata is safe; the aggregate balances must not
# be modified a second time.
calculated_cost.charged_msats = 0
- return calculated_cost.dict()
+ return _settled_cost_dict(calculated_cost, settlement_failure)
match calculated_cost:
case MaxCostData() as cost:
@@ -1434,18 +1458,36 @@ async def _adjust_payment_for_tokens(
else:
cost.charged_msats = cost.total_msats
await session.refresh(billing_key)
- logger.info(
- "Max cost payment finalized",
- extra={
- "key_hash": key.hashed_key[:8] + "...",
- "billing_key_hash": billing_key.hashed_key[:8] + "...",
- "charged_amount": cost.total_msats,
- "input_tokens": cost.input_tokens,
- "output_tokens": cost.output_tokens,
- "new_balance": billing_key.balance,
- "model": model,
- },
- )
+ finalize_type = "max_cost"
+ settlement_extra: dict[str, object] = {}
+ if settlement_failure is not None:
+ # A zero here is a write-off, not a free request: it is the
+ # only record that the seller gave the inference away.
+ finalize_type = "unpriced"
+ settlement_extra = {
+ "settlement_reason": settlement_failure.code,
+ "settlement_message": settlement_failure.message,
+ }
+ settlement_log_extra: dict[str, object] = {
+ "key_hash": key.hashed_key[:8] + "...",
+ "billing_key_hash": billing_key.hashed_key[:8] + "...",
+ "charged_amount": cost.total_msats,
+ "input_tokens": cost.input_tokens,
+ "output_tokens": cost.output_tokens,
+ "new_balance": billing_key.balance,
+ "model": model,
+ "finalize_type": finalize_type,
+ **settlement_extra,
+ }
+ if settlement_failure is not None:
+ logger.warning(
+ "Settlement finalized at zero: pricing failed after delivery",
+ extra=settlement_log_extra,
+ )
+ else:
+ logger.info(
+ "Max cost payment finalized", extra=settlement_log_extra
+ )
await _accumulate_fee(cost.total_msats)
payments_logger.info(
"FINALIZE",
@@ -1461,10 +1503,11 @@ async def _adjust_payment_for_tokens(
"balance": billing_key.balance,
"reserved_balance": billing_key.reserved_balance,
"total_spent": billing_key.total_spent,
- "finalize_type": "max_cost",
+ "finalize_type": finalize_type,
+ **settlement_extra,
},
)
- return cost.dict()
+ return _settled_cost_dict(cost, settlement_failure)
case CostData() as cost:
# If token-based pricing is enabled and base cost is 0, use token-based cost
diff --git a/routstr/core/ledger_analytics.py b/routstr/core/ledger_analytics.py
index c7211f7..13edb2b 100644
--- a/routstr/core/ledger_analytics.py
+++ b/routstr/core/ledger_analytics.py
@@ -14,7 +14,11 @@ from .db import (
TerminalOutcomeWriterRun,
create_session,
)
-from .terminal_outcome_writer import UNKNOWN_PRICING_SOURCE, SessionFactory
+from .terminal_outcome_writer import (
+ PRICING_SOURCE_UNPRICED,
+ UNKNOWN_PRICING_SOURCE,
+ SessionFactory,
+)
_SETTLED_FIELDS = (
"successful_chat_completions",
@@ -63,7 +67,15 @@ def _measures() -> list[Any]:
),
func.sum(
case(
- (col(TerminalOutcome.pricing_source) == UNKNOWN_PRICING_SOURCE, 1),
+ (
+ # Both "we never learned the price" and "pricing raised
+ # after delivery" are outcomes the seller cannot bill; the
+ # finer value stays on the row for anyone querying it.
+ col(TerminalOutcome.pricing_source).in_(
+ (UNKNOWN_PRICING_SOURCE, PRICING_SOURCE_UNPRICED)
+ ),
+ 1,
+ ),
else_=0,
)
).label("unknown_outcome_requests"),
diff --git a/routstr/core/terminal_outcome_writer.py b/routstr/core/terminal_outcome_writer.py
index 0e73358..e0d1a63 100644
--- a/routstr/core/terminal_outcome_writer.py
+++ b/routstr/core/terminal_outcome_writer.py
@@ -34,6 +34,9 @@ _LEASE_TIMEOUT_SECONDS = 90.0
_RACE_ATTEMPTS = 10
_RECOVERY_BACKOFF_MAX_SECONDS = 60.0
UNKNOWN_PRICING_SOURCE = "unknown"
+# Distinct from ``unknown``: the row was written, but settlement raised while
+# pricing it, so the zero it carries is a write-off rather than a measurement.
+PRICING_SOURCE_UNPRICED = "unpriced"
SessionFactory = Callable[[], AsyncContextManager[AsyncSession]]
Clock = Callable[[], int]
diff --git a/routstr/upstream/base.py b/routstr/upstream/base.py
index ea2aa7a..ae44244 100644
--- a/routstr/upstream/base.py
+++ b/routstr/upstream/base.py
@@ -393,6 +393,11 @@ def _settled_cost_msats(cost_data: CostMetadata) -> int:
def _published_cost(cost_data: CostMetadata) -> dict[str, Any]:
cost = dict(cost_data) if isinstance(cost_data, dict) else cost_data.dict()
+ # A settlement that failed carries its marker internally; publish it as
+ # ``cost.settlement`` so the buyer can tell a write-off from a free request.
+ settlement = cost.pop("_settlement", None)
+ if settlement:
+ cost["settlement"] = settlement
computed_msats = int(cost_field(cost_data, "total_msats"))
settled_msats = _settled_cost_msats(cost_data)
if computed_msats != settled_msats:
@@ -426,6 +431,13 @@ def _inject_cost_response_headers(
total_usd = float(cost_field(cost_data, "total_usd", 0.0))
if total_usd:
headers["X-Routstr-Cost-Usd"] = str(total_usd)
+ # A zero that is a write-off must not read as a measured zero.
+ settlement = cost_data.get("_settlement") if isinstance(cost_data, dict) else None
+ if isinstance(settlement, dict):
+ headers["X-Routstr-Settlement-Type"] = str(settlement.get("type", "unpriced"))
+ reason = settlement.get("reason")
+ if reason:
+ headers["X-Routstr-Settlement-Reason"] = str(reason)
def _apply_estimated_usage(
diff --git a/tests/unit/test_ledger_analytics.py b/tests/unit/test_ledger_analytics.py
index 892394a..d448b78 100644
--- a/tests/unit/test_ledger_analytics.py
+++ b/tests/unit/test_ledger_analytics.py
@@ -987,3 +987,39 @@ async def test_bigint_revenue_sums_are_json_safe_in_every_section(
assert result["metrics"]["totals"]["revenue_msats"] == 2 * revenue
assert result["revenue_by_model"]["models"][0]["revenue_sats"] == 2 * revenue / 1000
json.dumps(result)
+
+
+async def test_unpriced_settlement_is_counted_as_an_unknown_outcome(
+ sessions: SessionFactory,
+) -> None:
+ """A settlement that failed after delivery is a giveaway the seller counts.
+
+ The row keeps the finer ``unpriced`` value, while the dashboard's existing
+ unknown-outcome count reports it next to genuinely unknown prices — both are
+ outcomes the seller could not bill.
+ """
+ await _seed(sessions)
+ at = NOW - timedelta(hours=1)
+ unpriced = _row(
+ "unpriced-settlement",
+ at,
+ model="reported/model",
+ revenue=0,
+ input_tokens=0,
+ output_tokens=0,
+ cache_read=0,
+ cache_write=0,
+ source="missing",
+ )
+ unpriced.pricing_source = "unpriced"
+ async with sessions() as session:
+ session.add(unpriced)
+ await session.commit()
+ result = await get_ledger_usage_dashboard(
+ _legacy(), interval=60, hours=24, session_factory=sessions, now=NOW
+ )
+ assert result["summary"]["unknown_outcome_requests"] == 1
+ assert result["metrics"]["totals"]["unknown_outcome_requests"] == 1
+ models = {row["model"]: row for row in result["revenue_by_model"]["models"]}
+ assert models["reported/model"]["unknown_outcome_requests"] == 1
+ assert models["free/model"]["unknown_outcome_requests"] == 0
diff --git a/tests/unit/test_unpriced_settlement.py b/tests/unit/test_unpriced_settlement.py
new file mode 100644
index 0000000..d891f49
--- /dev/null
+++ b/tests/unit/test_unpriced_settlement.py
@@ -0,0 +1,165 @@
+"""A zero that is a write-off must be distinguishable from a measured zero.
+
+A pricing failure after the upstream served content settles at zero and releases
+the reservation (see ``test_cost_error_after_delivery``). That zero is a
+giveaway, not a free request, and the seller must be able to count it: the
+payments log, the buyer's response and the request row all have to say so.
+"""
+
+import logging
+from contextlib import ExitStack
+from typing import Any
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+
+from routstr.auth import ReservationSnapshot, adjust_payment_for_tokens
+from routstr.core.db import ApiKey
+from routstr.core.terminal_outcomes import TerminalOutcomeContext
+from routstr.payment import cost_calculation
+from routstr.payment.cost_calculation import CostData, MaxCostData
+from routstr.upstream.base import _inject_cost_response_headers, _published_cost
+
+KEY_HASH = "c" * 64
+
+
+def _reservation(reserved_msats: int = 7_000) -> ReservationSnapshot:
+ return ReservationSnapshot(
+ release_id="rel-unpriced",
+ key_hash=KEY_HASH,
+ billing_key_hash=KEY_HASH,
+ reserved_msats=reserved_msats,
+ )
+
+
+def _settlement_patches(recorded: MagicMock) -> Any:
+ return (
+ patch.object(
+ cost_calculation,
+ "_get_pricing_rates",
+ side_effect=ValueError("no pricing for model"),
+ ),
+ patch("routstr.auth._validate_reservation_snapshot", new=AsyncMock()),
+ patch("routstr.auth._stop_reservation_heartbeat", new=AsyncMock()),
+ patch(
+ "routstr.auth._claim_reservation_for_charge",
+ new=AsyncMock(return_value=True),
+ ),
+ patch(
+ "routstr.auth._charge_reservation_rows", new=AsyncMock(return_value=True)
+ ),
+ patch("routstr.auth.accumulate_routstr_fee", new=AsyncMock()),
+ patch("routstr.auth.record_terminal_outcome", recorded),
+ )
+
+
+@pytest.mark.asyncio
+async def test_failed_pricing_finalizes_as_unpriced(caplog: Any) -> None:
+ recorded = MagicMock()
+ context = TerminalOutcomeContext(outcome_id="outcome-1", model_identifier="m")
+ caplog.set_level(logging.INFO, logger="routstr.payments")
+ with ExitStack() as stack:
+ for patcher in _settlement_patches(recorded):
+ stack.enter_context(patcher)
+ cost = await adjust_payment_for_tokens(
+ ApiKey(hashed_key=KEY_HASH),
+ {"model": "m", "usage": {"prompt_tokens": 10, "completion_tokens": 5}},
+ session=AsyncMock(),
+ deducted_max_cost=7_000,
+ terminal_outcome=context,
+ reservation_snapshot=_reservation(),
+ )
+
+ # The buyer is told the zero is a fallback, and why.
+ assert cost["total_msats"] == 0
+ assert cost["charged_msats"] == 0
+ assert cost["_settlement"] == {"type": "unpriced", "reason": "pricing_error"}
+
+ # The payments log books it as its own finalize type, not as max_cost.
+ finalizes = [r for r in caplog.records if getattr(r, "event", None) == "finalize"]
+ assert len(finalizes) == 1
+ assert finalizes[0].finalize_type == "unpriced"
+ assert finalizes[0].settlement_reason == "pricing_error"
+ assert finalizes[0].cost_charged == 0
+
+ # The request row carries the write-off marker, so it can be counted.
+ recorded.assert_called_once()
+ assert recorded.call_args[0][0].pricing_source == "unpriced"
+
+
+@pytest.mark.asyncio
+async def test_zero_cost_request_is_not_booked_as_unpriced(caplog: Any) -> None:
+ recorded = MagicMock()
+ context = TerminalOutcomeContext(outcome_id="outcome-2", model_identifier="free")
+ free = MaxCostData(
+ base_msats=0,
+ input_msats=0,
+ output_msats=0,
+ total_msats=0,
+ pricing_source="fixed",
+ )
+ caplog.set_level(logging.INFO, logger="routstr.payments")
+ with (
+ patch("routstr.auth._validate_reservation_snapshot", new=AsyncMock()),
+ patch("routstr.auth._stop_reservation_heartbeat", new=AsyncMock()),
+ patch(
+ "routstr.auth._claim_reservation_for_charge",
+ new=AsyncMock(return_value=True),
+ ),
+ patch(
+ "routstr.auth._charge_reservation_rows", new=AsyncMock(return_value=True)
+ ),
+ patch("routstr.auth.accumulate_routstr_fee", new=AsyncMock()),
+ patch("routstr.auth.record_terminal_outcome", recorded),
+ ):
+ cost = await adjust_payment_for_tokens(
+ ApiKey(hashed_key=KEY_HASH),
+ {"model": "free", "usage": {"prompt_tokens": 0, "completion_tokens": 0}},
+ session=AsyncMock(),
+ deducted_max_cost=0,
+ terminal_outcome=context,
+ reservation_snapshot=_reservation(0),
+ precomputed_cost=free,
+ )
+
+ assert cost["total_msats"] == 0
+ assert "_settlement" not in cost
+ finalizes = [r for r in caplog.records if getattr(r, "event", None) == "finalize"]
+ assert finalizes[-1].finalize_type == "max_cost"
+ assert not hasattr(finalizes[-1], "settlement_reason")
+ assert recorded.call_args[0][0].pricing_source == "fixed"
+
+
+def test_published_cost_promotes_the_marker_and_drops_the_internal_key() -> None:
+ cost = CostData(
+ base_msats=0,
+ input_msats=0,
+ output_msats=0,
+ total_msats=0,
+ ).dict()
+ cost["_settlement"] = {"type": "unpriced", "reason": "pricing_error"}
+
+ published = _published_cost(cost)
+
+ assert "_settlement" not in published
+ assert published["settlement"] == {
+ "type": "unpriced",
+ "reason": "pricing_error",
+ }
+ assert published["total_msats"] == 0
+
+ headers: dict[str, str] = {}
+ _inject_cost_response_headers(headers, cost)
+ assert headers["X-Routstr-Cost-Msats"] == "0"
+ assert headers["X-Routstr-Settlement-Type"] == "unpriced"
+ assert headers["X-Routstr-Settlement-Reason"] == "pricing_error"
+
+ measured = CostData(
+ base_msats=0,
+ input_msats=0,
+ output_msats=0,
+ total_msats=0,
+ ).dict()
+ measured_headers: dict[str, str] = {}
+ _inject_cost_response_headers(measured_headers, measured)
+ assert "X-Routstr-Settlement-Type" not in measured_headers
sha256 8a75f5b74864f5ab0ee0577e8d043c5a6c154bf0fb91e978edc3bdf1a7d3ece2 · 6 files, +288/−17 · applies clean to 56b1de9 with git apply -p1.
Evidence (run, not read)
python3 -m venv <profile>/venv && <profile>/venv/bin/pip install fastapi sqlmodel \
aiosqlite greenlet pydantic-settings python-json-logger pytest pytest-asyncio \
orjson marshmallow "httpx[socks]" websockets respx rich pillow alembic \
cryptography cbor2 bech32 loguru mnemonic brotli zstandard base58 coincurve \
"bitstring<4" pycryptodomex environs websocket-client bolt11 click ecdsa pyjwt \
jinja2 importlib-metadata "nostr-sdk>=0.45.1,<0.46" "litellm>=1.101.3,<1.102" \
backoff redis ruff
<profile>/venv/bin/pip install --no-deps cashu bip32
git archive 56b1de9 | tar -x -C pristine && cd pristine && git apply -p1 ../unpriced-settlement.patch
PYTHONPATH=. UPSTREAM_BASE_URL=http://test UPSTREAM_API_KEY=test \
<profile>/venv/bin/pytest -q tests/unit/test_unpriced_settlement.py tests/unit/test_ledger_analytics.py
- new file
tests/unit/test_unpriced_settlement.py: 3 tests — the failed-pricing path booksfinalize_type: "unpriced"withsettlement_reason, stampspricing_source: "unpriced"on the recordedTerminalOutcome, and returns_settlement; a genuinely free request (precomputed zero,pricing_source: "fixed") still booksmax_cost, carries no marker and nosettlement_reason;_published_costpromotes the marker while dropping the internal key,_inject_cost_response_headersemits both headers for the fallback and neither for a measured zero. tests/unit/test_ledger_analytics.py: 8 existing tests plus one new — a row withpricing_source == "unpriced"is counted inunknown_outcome_requestsat summary, totals and per-model level, while afree/modelrow is not. 41 passed, 38 skipped (postgres-only) for that file plustest_unpriced_settlement.py.- targeted regression slice, 41 files matching
cost|auth|payment|ledger|reserv|upstream|stream|stats|terminal|header: 703 passed, 93 skipped, 0 failed (afterpip install backoff redis, which two litellm-transitive tests need in a bare env). - full
tests/unit: 70% of the suite ran clean (0 failures) before this session’s shell limit cut the run; the only failure I saw anywhere istest_analytics_runtime.py::test_missing_provider_identity_warns_once, which fails identically on an untouched56b1de9export — pre-existing, unrelated. ruff checkandruff format --checkclean on all six touched files.
Why the marker rides as _settlement
CostData excludes settlement-only metadata from .dict() on purpose (“must not expand the client cost contract”). Adding a field would either add a null to every response or need a second serialization path. A single internal key, promoted only when present, keeps the contract unchanged for every request that settles normally — and _published_cost is already the one place that shapes what the buyer sees.
Reproduction of the bug itself
# on 56b1de9, patch routstr/payment/cost_calculation._get_pricing_rates to raise
# then call adjust_payment_for_tokens(...) as in test_cost_error_after_delivery.py
# before: payments log finalize_type == "max_cost", charged_amount == 0,
# returned dict has no marker, TerminalOutcome.pricing_source == "missing"
# after: finalize_type == "unpriced", settlement_reason == "pricing_error",
# cost["_settlement"] == {"type": "unpriced", "reason": "pricing_error"},
# TerminalOutcome.pricing_source == "unpriced"
Write a comment