feat(provider-anthropic): populate ModelInfo.pricing from _RATES table - #63
Draft
manojp99 wants to merge 3 commits into
Draft
feat(provider-anthropic): populate ModelInfo.pricing from _RATES table#63manojp99 wants to merge 3 commits into
manojp99 wants to merge 3 commits into
Conversation
list_models() now surfaces pricing data that was previously only used internally for cost accounting (compute_cost() in _cost.py). A new _build_pricing(model_id) helper reads the existing _RATES dict and builds a Pricing object (input/output per-million rates, cache-read and cache-write rates, currency), passed through as ModelInfo(..., pricing=_build_pricing(model_id)). Models with no _RATES entry get pricing=None, matching the existing None-means-unknown convention used by compute_cost(). This lets HTTP-bridge applications (e.g. amplifier-app-opencode) read pricing from /v1/models instead of maintaining their own hardcoded pricing table. Fixes: microsoft-amplifier/amplifier-support#295 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
…alization and drop as_of Triage feedback on this PR identified two problems in _build_pricing(): 1. Three _RATES entries are asymmetric between bare-alias and dated-snapshot id shapes: claude-sonnet-4-6 and claude-opus-4-8 are alias-only (no dated row), while claude-haiku-3-5 is dated-only (claude-haiku-3-5-20250929, no bare alias). _build_pricing() did a plain _RATES.get(model_id), so whichever shape wasn't in the table produced a silent pricing=None if the Anthropic Models API happened to return the other shape. 2. The companion amplifier-core PR dropped Pricing.as_of entirely (all Pricing fields are now float/str, no dates) and added ISO 4217 currency validation. _build_pricing() was still passing as_of=None, which no longer exists as a constructor parameter. Fix: - Added _find_rates() to _cost.py: tries an exact _RATES match first, then falls back to comparing normalized ids (Anthropic's "-YYYYMMDD" dated snapshot suffix stripped from both the query and each _RATES key) so either shape -- bare alias or dated snapshot -- resolves to the same rate entry regardless of which shape happens to be populated in _RATES. - _build_pricing() now calls _find_rates(model_id) instead of _RATES.get(model_id) directly. - Removed as_of=None from the Pricing(...) construction to match the updated core schema. Per triage guidance, no entries were added to _RATES itself -- the fix is purely the lookup-normalization layer, since Anthropic can introduce new dated snapshots at any time and hand-enumerating them doesn't scale. Tests: added two cases to tests/test_model_pricing.py covering both asymmetry directions (dated snapshot of a bare-alias-only model, and bare alias of a dated-only model), both resolving correctly through the new _find_rates() normalization. No `from datetime import date` import existed in this module prior to this change (verified via grep), so there was nothing to remove on that front. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
… rate-table invariants The existing test_model_pricing.py tests only exercised _build_pricing() in isolation. If someone deleted `pricing=_build_pricing(model_id)` from the ModelInfo(...) construction in list_models(), none of those tests would fail -- the wiring itself was untested. Added TestListModelsPricingWiring, which mocks client.models.list() (via AsyncMock on a MagicMock client, matching the pattern already used in tests/test_close.py) and calls the real list_models() end to end. It asserts that a model present in _RATES ends up with a populated ModelInfo.pricing, and a fabricated model absent from _RATES ends up with pricing=None -- exercising list_models()'s family grouping/filtering plus the pricing wiring together, not _build_pricing() directly. Also, per triage: - Added a module-load assertion (_validate_rates_table() in _cost.py) that every _RATES entry carries all four required rate keys (input_per_m/output_per_m/cache_read_per_m/cache_write_per_m). This makes the invariant explicit and fails fast at import time instead of relying on convention. With the invariant enforced by the loader, the optional-key guards in _build_pricing() (`if "cache_read_per_m" in rates else None`, same for cache_write) were dead code -- every current entry already has all four keys -- so they're removed in favor of direct unconditional access. - Added a comment above the deprecated-models block in _RATES (claude-3-haiku-20240307, claude-sonnet-4-20250514, claude-opus-4-20250514) noting they're retained for historical cost accounting and not expected from list_models() post-retirement. 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔄 Updated per triage feedback (2026-06-30)
Two commits added addressing the "Updated Triage — PR Branch Review" from issue microsoft-amplifier/amplifier-support#295:
refactor(provider-anthropic): use _find_rates for snapshot-alias normalization and drop as_of(56dfeef)_find_rates(model_id)in_cost.py— two-level lookup: exact match, then strip-YYYYMMDDsuffix and re-lookup against normalized keys. Symmetric normalization handles both directions (dated-query vs bare-alias-only entry, and bare-alias-query vs dated-only entry)._build_pricing()to use_find_rates()— closes the three asymmetry gaps the triage flagged:claude-sonnet-4-6,claude-opus-4-8,claude-haiku-3-5no longer silently returnpricing=Nonewhen the API returns the counterpart form.as_of=Nonefrom thePricing()call site (companionamplifier-core#92removed the field).Note: the triage's assertion that
_find_rates()already existed in_cost.py:118-137was inaccurate — it did not exist and was introduced by this commit. The design intent was unambiguous; the function follows the pattern described.test(provider-anthropic): add list_models() wiring integration test + rate-table invariants(c64ebeb)client.models.list()and assertslist_models()returnsModelInfoobjects with correctly populatedPricing— addresses triage Medium Fix test_tool_repair tests and add streaming mode coverage #5 (the wiring at thepricing=call site was previously covered only by code inspection)._validate_rates_table()assertion in_cost.pythat enforces every_RATESentry has all four expected keys — makes the invariant explicit at import time and eliminates the dead-code optional-key guards the triage flagged as LOW Fix infinite loop when synthetic tool results aren't persisted #3._RATESentries (claude-3-haiku-20240307,claude-sonnet-4-20250514,claude-opus-4-20250514) noting they're retained for historical cost accounting — addresses triage LOW Fix beta headers being overwritten when extended thinking enabled #4.Verification results
test_usage_model_stores_decimal_internallyand three intest_tool_repair.py) are pre-existing (confirmed against the branch's prior HEAD4b270fdwith the same locally-installedamplifier-core). They surface a pre-existing mock/test-infra incompatibility unrelated to this work._RATEStable.Pre-existing bug surfaced (out of scope for this PR)
compute_cost()in_cost.pyuses the same plain_RATES.get(model)lookup and has the same asymmetric-miss behaviour — a real API call billed againstclaude-haiku-3-5(bare) or aclaude-sonnet-4-6-*/claude-opus-4-8-*dated snapshot would silently returncost=None. This affects live cost accounting, not just/v1/modelsdisplay. Left untouched — happy to file a follow-up PR (or issue) to routecompute_cost()through_find_rates()as well.What
AnthropicProvider.list_models()now populates the optionalModelInfo.pricingfield by reading from the existing_RATESdict in_cost.pyand buildingPricingobjects.Why
Surfaces the pricing data this module already maintains internally (used
today for per-turn cost accounting) through the public
ModelInfocontract, so
/v1/modelscarries it automatically. This is the secondhalf of the fix for
microsoft-amplifier/amplifier-support#295.
What's in this PR
_build_pricing(model_id)inamplifier_module_provider_anthropic/__init__.pythat reads from_RATESand returns aPricing | None(None when the model has norate entry)
list_models()now passespricing=_build_pricing(model_id)to eachModelInfo(...)constructortests/test_model_pricing.pycovering the helperCompatibility
Backwards-compatible. Models without a
_RATESentry returnpricing=None,the existing wire behavior. The change is opt-in for consumers — anyone
reading
ModelInfo.pricinggets data; anyone ignoring the field isunaffected.
Dependency
Requires
amplifier-core≥ the version that ships thePricingfield (see partner PR in
microsoft/amplifier-core). The partner PRmust merge first: microsoft/amplifier-core#92
Validation
Validated end-to-end in a Digital Twin Universe environment with the
matching
amplifier-corepatch active:_build_pricingandpricing=argument confirmed present inthe installed
__init__.pylist_models()returns models with populatedPricingfor entries in_RATES, andpricing=Nonefor entries without (claude-sonnet-5inthe current catalog, by design —
_RATESdoesn't include it yet)ModelInfo.model_dump(mode="json")emits the expected wire shape witha
pricingobject on entries that have rates_RATEStable exactly
Known follow-up
A handful of models in the current
list_models()catalog have no_RATESentry and will surfacepricing=None. Filling those in is afollow-up — keeping it out of this PR so the schema fix is reviewable
in isolation.