fix: only advertise beta-gated 1M context window when account is entitled - #60
Conversation
|
Gentle reminder — this has been open ~5 days with no review yet and no reviewers assigned. Why it matters: on Anthropic accounts not entitled to the beta 1M context window, the provider still advertises The companion safety-net PR on the adapter side ( Scope is small and low-risk: new One thing I could not verify locally: the full pytest suite needs Could a maintainer take a look or point me at the right reviewer? Happy to adjust the default or split anything out. Thanks! |
…tled The provider advertised a 1,000,000-token context_window for any model with supports_1m (Sonnet 4.6/4.7, Opus 4.6/4.7) whenever enable_1m_context was on (the default), and correctly sent the context-1m-2025-08-07 beta header per request. But the beta header is necessary, not sufficient: the 1M window is also Anthropic usage-tier gated. A non-entitled account still has its prompt enforced at 200K, yet list_models()/get_info advertised 1,000,000. Downstream clients (e.g. opencode) budget context compaction against the advertised window, so a 1M advertisement on a non-entitled account makes them skip compaction in the 200K danger zone and overflow with a hard `prompt is too long: N > 200000 maximum` 400. Introduce _advertised_context_window() and a context_1m_entitled config flag (default False). Beta-gated 1M (i.e. models where _should_add_context_1m_beta is True) is only advertised when the operator confirms entitlement. GA 1M windows (Opus 4.8+, which need no beta header) are always advertised, so this is a no-op for them. Request-time beta-header behavior is unchanged. Adds tests/test_advertised_context_window.py covering beta-gated/GA/no-1m/ disabled cases. Generated with Amplifier Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
3aa3904 to
8b6bce8
Compare
Problem
provider-anthropicadvertises a 1,000,000-tokencontext_windowfor Sonnet 4.6/4.7 and Opus 4.6/4.7 wheneverenable_1m_contextis on (the default), and sends thecontext-1m-2025-08-07beta header per request. But that beta is usage-tier gated — the header is necessary, not sufficient. On an account not entitled to 1M, Anthropic still enforces 200K, whilelist_models()/get_infokeep advertising 1,000,000.Downstream clients budget context compaction against the advertised window. A 1M advertisement on a non-entitled account makes them skip compaction in the 200K danger zone and overflow with a hard 400:
(Observed via
amplifier-app-opencode→ opencode onclaude-sonnet-4-6.)Change
_advertised_context_window(model_id, caps)centralizes the decision. Beta-gated 1M (models where_should_add_context_1m_beta()is True) is advertised only when the operator asserts entitlement via the newcontext_1m_entitledconfig flag (defaultFalse). GA 1M (Opus 4.8+, no beta header needed) is always advertised — no-op for those models.list_models()andget_info()defaults; added acontext_1m_entitledConfigFieldfor discoverability.context_1m_entitled: trueand get the 1M window advertised + honored as before.Behavior change
Default-config advertised window for Sonnet 4.6/4.7 and Opus 4.6/4.7 goes 1,000,000 → 200,000 unless
context_1m_entitled: trueis set. This is the safe default: advertising a window the API may reject causes hard downstream failures, whereas advertising the guaranteed floor only costs unused headroom for entitled accounts (recoverable with one flag). Maintainers: flag if you'd prefer a different default.Test plan
tests/test_advertised_context_window.py(beta-gated not-entitled → 200K; entitled → 1M; GA Opus 4.8 → 1M regardless; haiku/Sonnet 4.5 → 200K;enable_1m_context=False→ 200K).ruff checkclean;python -c "import ast"parses.amplifier_core, whose Rust_enginenative extension isn't built in my sandbox (ModuleNotFoundError: amplifier_core._engine). Please rely on CI to run the test suite.