Skip to content

fix(pricing): a missing cache rate prices as a floor, not a silent zero - #48

Merged
kleyt0n merged 4 commits into
mainfrom
fix/cache-rate-floor
Aug 5, 2026
Merged

fix(pricing): a missing cache rate prices as a floor, not a silent zero#48
kleyt0n merged 4 commits into
mainfrom
fix/cache-rate-floor

Conversation

@as-koshiyama

Copy link
Copy Markdown
Contributor

Closes #16.

What

For a model whose LiteLLM entry omits cache_read_input_token_cost or cache_creation_input_token_cost, cache tokens were multiplied by an unwrap_or(0.0) rate and returned as Cost::Known — "unpriced is not free" broken silently inside the headline number, on the token kind that dominates a coding-agent corpus. The CHANGELOG claimed input-rate fallback; code and changelog now agree, per the issue's checklist.

Semantics chosen

The two fallbacks lean opposite ways for the same reason — never overstate:

  • Cache creation → input rate. Providers that publish a write rate set it above input, so this understates.
  • Cache reads → zero. Their real rate sits below input, so an input fallback would overstate.

Both understate, so the result is a new Cost::Floor(f64): rendered ≥$x in the Cost view, {"state":"floor","usd":x} in --json, and counted into every total's marker via Cost::not_fully_priced() (which replaces is_unpriced() at each aggregation site — a floor's dollars are real, unlike unpriced's, so the two stay distinct states). A rate listed as 0.0 still prices as zero exactly, because the table said so — only an absent key floors, and only when the usage actually has tokens of that kind.

Also per the issue's last checkbox: the parse-time zero-cost filter now keeps cache-only entries instead of dropping them with the embeddings, and the Cost view's floor title reads "N model(s) not fully priced" rather than "have no price", because a floored model has one.

Verification

The issue's acceptance test verbatim (missing_cache_rates_price_as_a_floor_never_a_silent_zero): cache reads + creations against an entry with input/output rates only → Cost::Floor(2.0), non-zero, not exact, not unpriced. Plus: a missing rate with no cache tokens stays Known; a cache-only entry survives parsing and prices exactly; absent keys parse to None while listed values parse to Some. cargo fmt, clippy -D warnings, all 246 tests pass.

Note for review ordering: independent of every open PR; will need the usual one-line CHANGELOG rebase after whichever lands first.

🤖 Generated with Claude Code

For a model whose table entry omits cache_read_input_token_cost or
cache_creation_input_token_cost, cache tokens were multiplied by an
unwrap_or(0.0) rate and the result returned as Cost::Known - "unpriced
is not free" broken silently, inside the one number the tool exists to
report, on the token kind that dominates a coding-agent corpus.

The two absent-rate fallbacks lean opposite ways for the same reason.
Cache creation bills at the input rate: providers that publish a write
rate set it above input, so this understates. Cache reads bill at
zero: their real rate sits below input, so an input fallback would
overstate. Both understate, so the result is a new Cost::Floor - shown
as `>=` in the Cost view, "floor" in --json, and counted into every
total's floor marker through Cost::not_fully_priced, which replaces
is_unpriced at each aggregation site. A rate listed as 0.0 still
prices as zero exactly, because the table said so: only an absent key
floors, and only when the usage actually has tokens of that kind.

The parse-time filter keeps cache-only entries now instead of dropping
them with the embeddings, and PerToken's cache rates are Option so
"absent" survives to the pricing decision. The Cost view's floor title
reads "not fully priced" rather than "have no price", because a
floored model has one.

Closes #16.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
kleyt0n
kleyt0n previously approved these changes Aug 1, 2026

@kleyt0n kleyt0n left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@kleyt0n
kleyt0n force-pushed the fix/cache-rate-floor branch from ac6970b to 944717d Compare August 3, 2026 10:27
@kleyt0n
kleyt0n merged commit 3dec9f4 into main Aug 5, 2026
7 checks passed
@kleyt0n
kleyt0n deleted the fix/cache-rate-floor branch August 5, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(pricing): cache tokens are billed at zero when the price table omits a rate

2 participants