feat: add context_budget_cap to provider defaults for cost-optimal budgeting - #27
Open
Salil Das (sadlilas) wants to merge 1 commit into
Open
feat: add context_budget_cap to provider defaults for cost-optimal budgeting#27Salil Das (sadlilas) wants to merge 1 commit into
Salil Das (sadlilas) wants to merge 1 commit into
Conversation
…dgeting Anthropic charges 2x input / 1.5x output above 200k tokens. Set context_budget_cap in ProviderInfo.defaults when enable_1m_context is active so context modules can cap their working budget at the pricing threshold. The full 1M context_window remains as safety net for overflow. Related: microsoft-amplifier/amplifier-support#57 🤖 Generated with [Amplifier](https://github.com/microsoft/amplifier) Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
context_budget_cap: 200_000to Anthropic'sProviderInfo.defaultswhenenable_1m_contextis active and the model supports 1M context.Why
Anthropic charges 2x input / 1.5x output above 200k tokens. The context module (
context-simple) needs a signal from the provider to cap its working budget at the cost-optimal threshold, rather than using the full 1M context window.What changes
enable_1m_contextis active and the model supports 1M:defaults["context_budget_cap"] = 200_000context_budget_capisNone(no cap needed -- the window itself is 200k)Design
This is part of a provider-driven budget cap approach that replaces the fraction-based approach in context-simple PR #5. See amplifier-support#57 analysis for the full rationale.
The key insight: pricing knowledge belongs in the provider, not the context module. Providers with tiered pricing emit a cap; providers with flat pricing (OpenAI) emit nothing.
Related PRs
context_budget_cap: 200_000(Gemini has the same pricing cliff)context_budget_capfrom provider defaults and appliesbudget = min(raw_budget, cap - safety_margin)Related issues
🤖 Generated with Amplifier