feat(cost): config-level rates override for models missing from _RATES - #65
Open
Joi Ito (Joi) wants to merge 1 commit into
Open
feat(cost): config-level rates override for models missing from _RATES#65Joi Ito (Joi) wants to merge 1 commit into
Joi Ito (Joi) wants to merge 1 commit into
Conversation
Models absent from the hardcoded _RATES table make compute_cost() return
None, so every usage.cost_usd on such models is null. A new 'rates'
provider config key maps model ids (exact or trailing-* glob) to
{input, output, cache_read, cache_write} in USD per 1M tokens. Overrides
take precedence over _RATES; lookup order is exact override, longest
matching glob override, then the built-in table. Missing cache fields
default to 10% / 125% of input, the ratios every _RATES row uses.
Values are parsed via Decimal(str(x)) so no float arithmetic touches
the cost path; unknown models still yield None. Invalid entries are
skipped with a warning, matching the provider's lenient config parsing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Models absent from the hardcoded
_RATEStable in_cost.pymakecompute_cost()returnNone, so everyusage.cost_usdfor those models is null. Today this affects all claude-fable-* sessions: in my logs, 129 of 129 fable responses havecost_usd: null. microsoft/amplifier#303 documents the family-detection side of the same gap. Any private, fine-tuned, or newly released model hits this until a code change ships.Solution
A new
ratesprovider config key supplies rates for models the built-in table does not know:*glob override, then_RATES. Config rates take precedence over the built-in table._RATESrow uses.Decimal(str(x)). No float arithmetic touches the cost path.Nonestill means unknown model with no rate anywhere. Unknown is never silently zero._config_int/_config_floatstyle). They never fail the mount.speed='fast'2x multiplier logic is unchanged and still applies only to_FAST_ELIGIBLE_MODELS._RATESitself is untouched.Relationship to #58
Complements it. #58 adds fable/mythos rows to
_RATESand should still land for out-of-box fable pricing. This PR covers the general case: any model missing from the table, without waiting for a release. The override row shape and the fable example rates here match #58.Tests
tests/test_rates_override.pyadds 21 tests: parse validation (missing/invalid/unknown fields, non-mapping input), derived cache defaults, Decimal precision (no float artifacts), override hit for an unlisted model, precedence over_RATES, unknown model stillNone, glob matching (exact beats glob, longest prefix wins), fast-multiplier interaction, and three integration tests through_convert_to_chat_responsewith config-supplied rates.Local run (uv, amplifier-core 1.6.0 from PyPI): 499 passed, 1 skipped, and 10 pre-existing failures/errors that reproduce identically on clean
mainin the same environment (test_tool_repair,test_validation,test_behavioral,test_cost_usd_json_serializable::test_usage_model_stores_decimal_internally). No regressions from this change; all cost tests (test_cost.py+ new file, 41 tests) pass.🤖 Generated with Claude Code