Skip to content

fix(chat): route agent chat to the account's remote-agent node - #48

Merged
simonsmh merged 2 commits into
simonsmh:mainfrom
Eridanus117:fix/chat-endpoint-discovery
Nov 1, 2026
Merged

simonsmh merged 2 commits into
simonsmh:mainfrom
Eridanus117:fix/chat-endpoint-discovery

Conversation

@Eridanus117

Copy link
Copy Markdown
Contributor

Problem

On some accounts every qoder/* model fails immediately with

Upstream status 403: {"code":"110","message":"Billing daily count exceeded"}

while the same account and the same model work from qodercli, and the account's
Credit balance is untouched (quota/usage reports isQuotaExceeded: false, org package
at 42%). Error code 110 is documented by Qoder as the daily Chat allowance, i.e. a
per-account, per-node counter — not a Credit balance.

Root cause

Chat is pinned to the fixed inference gateway:

baseUrl: "https://api3.qoder.sh/"

Qoder's node map, published per account by region/endpoints, separates node classes:

{
  "inferNodes": ["https://api3.qoder.sh"],
  "dataNodeMap": { "remote_agent": ["https://api2.qoder.sh"], "codebase": ["https://api2.qoder.sh"] },
  "security": ["https://api2.qoder.sh"]
}

The official CLI drives agent_chat_generation against the remote-agent node, not
the inference node. Captured from qodercli 1.1.74 on the affected account:

POST https://api2.qoder.sh/algo/api/v2/service/pro/sse/agent_chat_generation

The extension sends the identical request (same path, same Cosy-ClientType: 5, same
business.product: "cli", same machine id) to api3.qoder.sh, so the daily chat counter
for that node is consumed by extension traffic and never shared with the CLI. Accounts
whose daily allowance on the inference node is exhausted get 110 forever, while the CLI
keeps working.

Change

  1. qoderDecodeBody — inverse of qoderEncodeBody. region/endpoints is returned in
    the same custom Base64 alphabet/block order, so it was previously opaque.
  2. src/endpoints.ts — fetch and parse the account node map at startup, cache the
    resolved chat node in ~/.pi/agent/qoder-endpoints-cache.json (1 h), and route chat
    to the remote-agent node while the model catalog keeps using the inference node.
  3. 403 code 110 during a stream now rotates to the next candidate (remote-agent →
    inference → security) once per node and retries; the node that actually served the
    request is persisted. A rejection after partial output is not retried.

Discovery failures, missing credentials, and non-parseable responses all fall back to
the previous fixed gateway, so accounts that already work are unaffected.

Verification

  • npm run check (tsc), npm run lint (biome), npm test — 436 tests pass, including
    new coverage for decode round-trips, node-map parsing, candidate ordering, cache TTL,
    discovery fallbacks, rotation, and a stream-level test that replays a 110 envelope and
    asserts the retry lands on the next node.
  • End-to-end on the reporting account with the built extension: startup discovery selects
    api2.qoder.sh, chat and tool calls succeed, and the node is cached. With the cache
    removed and the node map served fresh, the same result reproduces.
  • The real region/endpoints response captured from that account is committed as
    src/__fixtures__/region-endpoints.json (public node names and IPs only).

Notes

  • CN (qoder-cn) keeps gateway.qoder.com.cn: its region has no equivalent node split
    in the responses observed, and discovery falls back to the configured gateway.
  • I did not add a models.yml-style override: the host resolves baseUrl per model
    from the provider registration, so the provider itself has to resolve it.

Eridanus117 and others added 2 commits November 2, 2026 03:40
Qoder tracks the daily chat allowance per node. For accounts whose CLI node is
the remote-agent node, every chat request to the fixed inference node is
rejected with `403 code 110 "Billing daily count exceeded"` while the same
account and model succeed from qodercli.

- decode Qoder's custom Base64 responses (`qoderDecodeBody`)
- resolve the account node map from `region/endpoints` at startup, cache it for
  one hour in `~/.pi/agent/qoder-endpoints-cache.json`, and route chat to the
  remote-agent node while the catalog stays on the inference node
- on code 110, switch through the remaining nodes once each and persist the
  node that actually served the request

Discovery failures fall back to the previous fixed gateway, so accounts that
already work are unaffected.
- stream: the 110 daily-limit rejection can also arrive as a plain
  HTTP error instead of an SSE envelope; rotate nodes for it the same
  way (no model output has been seen when headers arrive, so the
  retry is safe). Covered by a stream-level test.
- endpoints: let QODER_ENDPOINTS_CACHE_PATH override the cache file
  location, and point the endpoints/stream tests at a temp file.
  The tests previously wrote the user's real cache file and raced on
  it when run in parallel.
- providers tests: raise the file timeout to 30s. Registration pulls
  in the full pi-ai compat layer; under parallel workers that can
  pass vitest's 5s default, and a timeout here also lets the late
  registration pollute the next test's registry counts.
@simonsmh
simonsmh force-pushed the fix/chat-endpoint-discovery branch from 899a6e1 to 30c993f Compare November 1, 2026 23:21
@simonsmh

simonsmh commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Verified this end to end, and pushed one follow-up commit (as maintainer, since edits from maintainers are allowed on this branch) — please take a look and say if you'd rather shape it differently:

What I checked

  • The decode is the exact inverse of qoderEncodeBody (tail + middle + head block order), and the committed fixture decodes to the node map you describe.
  • Live, on my own account: region/endpoints parses to infer api1, remote-agent api2, security api2, and discovery selects api2 and caches it. (Note my account's infer node is api1, not api3 — the per-account map really does vary, which supports routing per account rather than hardcoding.)
  • The PR build installed normally starts and chats fine on my account (same ~9s as v0.6.7; an earlier slow linked-install run turned out to be a linking artifact, not the code).
  • Suite: 437/437 locally, CI green.

What the follow-up fixes

  1. The 110 rejection can also arrive as a plain HTTP error instead of an SSE envelope — the original code only rotated on the envelope path, so that shape would have surfaced raw. The !response.ok branch now throws the same daily-limit error (with a stream-level test replaying a 403 JSON body, rotating to api2 and persisting it).
  2. The endpoints/stream tests wrote the user's real ~/.pi/agent/qoder-endpoints-cache.json and raced on it under parallel workers. QODER_ENDPOINTS_CACHE_PATH now overrides the location, and both test files point at a temp file.
  3. The providers tests get a 30s timeout: registration pulls in the full pi-ai compat layer, and under parallel workers that occasionally passed vitest's 5s default — a timeout there also lets the late registration pollute the next test's registry counts (the same flake showed up twice in my runs, never with a stable repro).

Nothing in the routing logic itself needed changing — the candidate order, once-per-node rotation, partial-output guard, and fallback to the fixed gateway all check out. Thanks again for the fix!

@simonsmh
simonsmh merged commit a45e2dd into simonsmh:main Nov 1, 2026
1 check passed
@simonsmh

simonsmh commented Nov 1, 2026

Copy link
Copy Markdown
Owner

Merged — thank you! Squash-merged as a45e2dd and shipped in v0.6.8: https://github.com/simonsmh/pi-provider-qoder/releases/tag/v0.6.8

Verification recap (beyond the suite): the decode is the exact inverse of the request encoding, the live node map on my account parses to infer api1 / remote-agent api2 / security api2 with discovery selecting api2, and the PR build installed normally chats fine at the same speed as v0.6.7. Thanks again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants