Skip to content

fix: only advertise beta-gated 1M context window when account is entitled - #60

Open
Michael J. Jabbour (michaeljabbour) wants to merge 1 commit into
mainfrom
fix/advertise-1m-context-only-when-entitled
Open

fix: only advertise beta-gated 1M context window when account is entitled#60
Michael J. Jabbour (michaeljabbour) wants to merge 1 commit into
mainfrom
fix/advertise-1m-context-only-when-entitled

Conversation

@michaeljabbour

Copy link
Copy Markdown

Problem

provider-anthropic advertises a 1,000,000-token context_window for Sonnet 4.6/4.7 and Opus 4.6/4.7 whenever enable_1m_context is on (the default), and sends the context-1m-2025-08-07 beta header per request. But that beta is usage-tier gated — the header is necessary, not sufficient. On an account not entitled to 1M, Anthropic still enforces 200K, while list_models() / get_info keep advertising 1,000,000.

Downstream clients budget context compaction against the advertised window. A 1M advertisement on a non-entitled account makes them skip compaction in the 200K danger zone and overflow with a hard 400:

prompt is too long: 207537 tokens > 200000 maximum

(Observed via amplifier-app-opencode → opencode on claude-sonnet-4-6.)

Change

  • New _advertised_context_window(model_id, caps) centralizes the decision. Beta-gated 1M (models where _should_add_context_1m_beta() is True) is advertised only when the operator asserts entitlement via the new context_1m_entitled config flag (default False). GA 1M (Opus 4.8+, no beta header needed) is always advertised — no-op for those models.
  • Used by both list_models() and get_info() defaults; added a context_1m_entitled ConfigField for discoverability.
  • Request-time beta-header behavior is unchanged — entitled operators set context_1m_entitled: true and get the 1M window advertised + honored as before.

Behavior change

Default-config advertised window for Sonnet 4.6/4.7 and Opus 4.6/4.7 goes 1,000,000 → 200,000 unless context_1m_entitled: true is set. This is the safe default: advertising a window the API may reject causes hard downstream failures, whereas advertising the guaranteed floor only costs unused headroom for entitled accounts (recoverable with one flag). Maintainers: flag if you'd prefer a different default.

Test plan

  • Added tests/test_advertised_context_window.py (beta-gated not-entitled → 200K; entitled → 1M; GA Opus 4.8 → 1M regardless; haiku/Sonnet 4.5 → 200K; enable_1m_context=False → 200K).
  • ruff check clean; python -c "import ast" parses.
  • NOTE: I could not execute the full pytest suite locally — this repo's tests import amplifier_core, whose Rust _engine native extension isn't built in my sandbox (ModuleNotFoundError: amplifier_core._engine). Please rely on CI to run the test suite.

@michaeljabbour

Copy link
Copy Markdown
Author

Gentle reminder — this has been open ~5 days with no review yet and no reviewers assigned.

Why it matters: on Anthropic accounts not entitled to the beta 1M context window, the provider still advertises context_window: 1000000 for Sonnet 4.6/4.7 and Opus 4.6/4.7. Downstream clients budget context compaction against that advertised number, skip compaction in the 200K danger zone, and the request dies with a hard prompt is too long: 207537 tokens > 200000 maximum 400. This is actively biting real sessions (surfaced via amplifier-app-opencode → opencode on claude-sonnet-4-6).

The companion safety-net PR on the adapter side (microsoft/amplifier-app-opencode#7, an opt-in --max-context clamp) is already merged, but that only masks the symptom for opencode users who opt in. This PR is the actual root-cause fix.

Scope is small and low-risk: new _advertised_context_window() + a context_1m_entitled flag (default False); GA 1M (Opus 4.8+) is unaffected; request-time beta-header behavior is unchanged. Only open question for reviewers is the intended default (this makes the safe 200K floor the default unless entitlement is asserted).

One thing I could not verify locally: the full pytest suite needs amplifier_cores Rust _engine, which is not built in my environment — please confirm CI runs the added tests/test_advertised_context_window.py.

Could a maintainer take a look or point me at the right reviewer? Happy to adjust the default or split anything out. Thanks!

…tled

The provider advertised a 1,000,000-token context_window for any model with
supports_1m (Sonnet 4.6/4.7, Opus 4.6/4.7) whenever enable_1m_context was on
(the default), and correctly sent the context-1m-2025-08-07 beta header per
request. But the beta header is necessary, not sufficient: the 1M window is
also Anthropic usage-tier gated. A non-entitled account still has its prompt
enforced at 200K, yet list_models()/get_info advertised 1,000,000.

Downstream clients (e.g. opencode) budget context compaction against the
advertised window, so a 1M advertisement on a non-entitled account makes them
skip compaction in the 200K danger zone and overflow with a hard
`prompt is too long: N > 200000 maximum` 400.

Introduce _advertised_context_window() and a context_1m_entitled config flag
(default False). Beta-gated 1M (i.e. models where _should_add_context_1m_beta
is True) is only advertised when the operator confirms entitlement. GA 1M
windows (Opus 4.8+, which need no beta header) are always advertised, so this
is a no-op for them. Request-time beta-header behavior is unchanged.

Adds tests/test_advertised_context_window.py covering beta-gated/GA/no-1m/
disabled cases.

Generated with Amplifier

Co-Authored-By: Amplifier <240397093+microsoft-amplifier@users.noreply.github.com>
@michaeljabbour
Michael J. Jabbour (michaeljabbour) force-pushed the fix/advertise-1m-context-only-when-entitled branch from 3aa3904 to 8b6bce8 Compare July 7, 2026 00:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant