Repository navigation
Run the live checks from the provider-limits research #229
Description
Question
Which of the open live checks listed at the end of the provider-limits research hold on the pinned runtime against Meta? Run them and record the facts the telemetry snapshot contract waits on:
- Log
ResultMessage.model_usage[...]["contextWindow"],maxOutputTokens,costBasisandget_context_usage()(maxTokens,rawMaxTokens,autoCompactThreshold) on SDK 0.2.163 against Meta, with and withoutCLAUDE_CODE_MAX_CONTEXT_TOKENS. - Confirm Meta's context window for the Contributor variant equals 1,048,576.
- Capture raw response headers from the LiteLLM gateway for
/v1/messages(x-ratelimit-*,llm_provider-*,x-litellm-*) with plaincurl. Decides whether provider rate-limit headroom can appear in v1. - Confirm
compact_metadatafields at runtime from a forced/compact. - Measure the request cost of
get_context_usage()against the 150 RPM Contributor limit. Decides whether the snapshot may call it at all.
Research note: branch research/provider-limits-and-context-window, file docs/research/provider-limits-and-context-window.md. AFK where credentials allow; otherwise hand the human a checklist. If check 1 confirms the CLI assumes a 200,000-token window for the Meta model while Meta documents 1,048,576, also open a separate needs-triage issue: that affects the agent's own compaction and handoff reserve, not only the dashboard.
Part of map #222.
Activity
lbacik commentedon Oct 14, 2026
Resolution
All five checks ran AFK in throwaway containers from the agent image (SDK 0.2.163, CLI 2.1.286, managed settings present), never inside a running agent. Details, tables and the probe scripts: branch
research/provider-limits-and-context-window, section "Live check results (2026-10-14)" ofdocs/research/provider-limits-and-context-window.md, scripts underdocs/research/provider-limits-live-checks/.contextWindow/maxTokens/rawMaxTokens= 200000 andautoCompactThreshold= 167000; withCLAUDE_CODE_MAX_CONTEXT_TOKENS=1048576all follow (1048576, threshold 1015576).maxOutputTokens= 32000 either way.costBasis=managed, andtotal_cost_usdmatches themanaged-settings.jsonprices exactly.GET /v1/models/{model}exposes no window. Overflow is a generic400 invalid_request_error, not a context-length error.x-ratelimit-{limit,remaining}-{requests,tokens}(150 RPM, 3,000,000 TPM) andx-request-idon every response, team-wide and lagging. Through the LiteLLM gateway (v1.104.0): nox-ratelimit-*and nollm_provider-*, onlyx-litellm-*with cost headers at0.0;/model/infohas every limit fieldnull;/key/infois 404. Provider rate-limit headroom staysunavailablein v1.compact_metadata. Confirmed at runtime:{trigger, pre_tokens, post_tokens, cumulative_dropped_tokens, duration_ms}(duration_msis extra to the CLI schema), preceded bystatusmessages (compacting→compact_result: success) and a freshinit.get_context_usage()cost. 5count_tokensrequests on the first call of a session (~1.7 s), 2 after a compaction, 0 otherwise (~10 ms, also after new turns). They count against RPM, not TPM. Negligible against 150 RPM, so the snapshot may call it; note it is an async SDK call, so it belongs on the attempt's event loop (where the snapshot is published), never in the socket thread.Facts the snapshot contract can rely on
model_usage.contextWindoworget_context_usage().rawMaxTokens) with provenance; it issdk_assumed(200000) unlessCLAUDE_CODE_MAX_CONTEXT_TOKENSis set, thenconfigured. Meta's 1,048,576 is a documented-and-verified constant, not a runtime value.autoCompactThreshold), and each compaction yields measuredpre_tokens/post_tokens.costBasis: managed); USD is already cut from v1 UI.unavailable.Follow-up outside this map
Check 1 confirmed the window mismatch, so it is filed separately as Claude Code assumes a 200,000-token window for the Meta model; Meta accepts ~1,048,576 (
needs-triage): it affects the agent's own compaction point and handoff reserve, not only the dashboard.Spend: about $1.7, mostly the two large window probes.