You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
cloud-api: modelRateLimits publishes a 1M window for qwen3.8-flash while the proxy enforces 262144 #53
// qwen3.8-flash: external model served via DashScope. YaRN extends the// native window to 1M, which is what we expose to members."qwen3.8-flash": {ContextWindow: 1_048_576, Concurrency: 5, FillsPerMin: 6},
And the model card on /docs/models: "262K token context, the model native window", with { label: Context, value: 262K tokens }.
So two of three sources say 262K and the rate table says 1M.
Why it is not cosmetic
modelRateLimits.ContextWindow is not only descriptive — the file says the effective ITPM is ContextWindow * FillsPerMin. At 1_048_576 x 6 the model is granted ~6.3M tokens/min; at its
enforced 262_144 x 6 it would be ~1.6M. The member cannot actually spend the difference,
because the proxy rejects anything over 262_144 input, so the generosity is unreachable — but
any future reasoning that treats that ITPM as real (a comparison across models, a capacity
estimate) starts from a 4x overstatement.
It also makes the comment above it false: we do NOT expose the YaRN-extended window; the
deployment caps at native.
What to decide
If the intent is to serve 1M, the deployment needs max_input_tokens raised (and the model
card updated) — DashScope has to actually accept it.
If the intent is native 262K, modelRateLimits should say 262_144 and the comment should go.
Option 2 matches what is served today.
Not changed here
/docs/examples now publishes 262144 for this model, following the proxy, and the test that
pins it carries this issue number as the reason it disagrees with modelRateLimits. Also
noted: mimo-v2.5 is 1_050_000 there against a 1M model card and no enforced cap — same
direction, 1424 tokens, not worth a change on its own.
Activity
fix(docs): address the #52 blocking review