Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion claude/aqe-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ called first**, e.g. `fleet_init({ topology:"hierarchical", maxAgents:15, memory
shows an "LLM Billing" section saying who pays. `aqe init` never writes these — but
**`ak x provider pick` now manages `AQE_LLM_PROVIDER` for you** (into
`.claude/settings.local.json` `env`, reversibly), and can write an ordered **fallback chain**
into `.agentic-qe/llm-config.json` from `kit.json` (`--aqe-fallback 'claude-code:claude-opus-4-8; openai:gpt-5.6'`) —
into `.agentic-qe/llm-config.json` from `kit.json` (`--aqe-fallback 'claude-code:claude-opus-5; openai:gpt-5.6'`) —
keys stay in the env, never the file. aqe is NOT limited to claude-code: codex the *CLI* isn't a
provider type, but its OpenAI models are reachable via `AQE_LLM_PROVIDER=openai`.

Expand Down
2 changes: 1 addition & 1 deletion claude/providers-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ For **deterministic** ordering (rather than relying on env auto-enable), `ak` wr

```bash
ak x provider pick --aqe-provider claude-code \
--aqe-fallback 'claude-code:claude-opus-4-8; openai:gpt-5.6; gemini:gemini-3.5-flash'
--aqe-fallback 'claude-code:claude-opus-5; openai:gpt-5.6; gemini:gemini-3.5-flash'
```

Each `provider:model,model` entry becomes an ordered `fallbackChain` entry (first = highest
Expand Down
22 changes: 16 additions & 6 deletions docs/PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,14 +118,14 @@ When you want explicit ordering rather than env auto-enable, `ak` manages agenti
```bash
ak x provider pick \
--aqe-provider claude-code \
--aqe-fallback 'claude-code:claude-opus-4-8; openai:gpt-5.6; gemini:gemini-3.5-flash'
--aqe-fallback 'claude-code:claude-opus-5; openai:gpt-5.6; gemini:gemini-3.5-flash'
```

Each `provider:model,model` becomes an ordered chain entry (first = highest priority). `ak`
writes a complete, schema-correct chain, tags it `_managedBy: agentic-kit`, and **never**
writes your API keys.

> Model IDs above are examples current as of July 2026 (Claude Opus 4.8, OpenAI GPT-5.6 —
> Model IDs above are examples current as of July 2026 (Claude Opus 5, OpenAI GPT-5.6 —
> or `gpt-5.3-codex` for agentic coding — Google Gemini 3.5 Flash). Use whatever IDs your
> provider currently offers; `ak` writes the strings you give it verbatim.

Expand All @@ -135,7 +135,7 @@ provider — add them to the chain and put `OPENROUTER_API_KEY` in your env:
```bash
ak x provider pick \
--aqe-provider claude-code \
--aqe-fallback 'claude-code:claude-opus-4-8; openrouter:z-ai/glm-5.2'
--aqe-fallback 'claude-code:claude-opus-5; openrouter:z-ai/glm-5.2'
```

Curated picks (verified July 2026): `z-ai/glm-5.2` (flagship — 1M context, strong
Expand Down Expand Up @@ -171,7 +171,7 @@ Defaults (all overridable; your edits are marked `custom` and never re-seeded):
| Activity | Host | Default model |
|---|---|---|
| specification, review, release | claude | `claude-sonnet-5` |
| architecture, design, debugging, security-analysis | claude | `claude-opus-4-8` |
| architecture, design, debugging, security-analysis | claude | `claude-opus-5` |
| implementation, testing, security-scan | codex | `gpt-5.4` |
| documentation, packaging | codex | `gpt-5.3-codex` |

Expand All @@ -181,15 +181,25 @@ Defaults (all overridable; your edits are marked `custom` and never re-seeded):

| Host | Model | When to use |
|---|---|---|
| claude | `claude-opus-4-8` | deep reasoning — architecture, design, hard debugging |
| claude | `claude-opus-5` | new top Opus — ~2× Opus 4.8 at the same price; premium reasoning default |
| claude | `claude-sonnet-5` | near-Opus at lower cost — review, spec, release |
| claude | `claude-fable-5` | top capability (above Opus), premium — hardest problems |
| claude | `claude-fable-5` | top capability (Mythos-class, above Opus 5) — hardest problems |
| claude | `claude-haiku-4-5-20251001` | cheap/fast — high-volume mechanical |
| claude | `claude-opus-4-8` | prior Opus generation — same price as opus-5, kept for pinned configs |
| codex | `gpt-5.4` | coding + reasoning + agentic — recommended execution default |
| codex | `gpt-5.6-sol` | newest line; first-class max reasoning effort |
| codex | `gpt-5.3-codex` | pure coding-tuned — mechanical implementation & docs |
| codex | `gpt-5-codex-mini` | smallest/cheapest — escalation floor, high volume |

> **Where Opus 5 sits** ([announcement](https://www.anthropic.com/news/claude-opus-5), July 2026):
> same $5/$25 per-Mtok pricing as Opus 4.8 with roughly double the Frontier-Bench
> performance, so it strictly supersedes 4.8 as the reasoning-tier default — a capability
> tier above Opus 4.8 at no added cost. It is **not** Mythos-class: `claude-fable-5`
> remains the flagship tier. Opus 5 lands within ~0.5% of Fable on coding/agentic
> benchmarks at about half the cost per task, but stays behind the Mythos-class models on
> frontier domains. Rule of thumb: `claude-opus-5` is the premium default;
> `claude-fable-5` is the escalation ceiling.

`ak x provider pick --help` prints this list too. Tuning is per-route and reversible: hand-edit
`kit.json` `providers.dualRouting`, pass `--route`, or `ak x provider off` to clear it entirely.

Expand Down
6 changes: 3 additions & 3 deletions src/commands/x/provider.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -73,13 +73,13 @@ Options (pick, all optional — omit for interactive):
billing: claude-code = Claude sub ($0),
ollama/onnx = local ($0), all others = metered key
--aqe-fallback '<chain>' ordered aqe chain, e.g.
'claude-code:claude-opus-4-8; openai:gpt-5.6'
'claude-code:claude-opus-5; openai:gpt-5.6'
(metered providers work too, e.g. add
'openrouter:z-ai/glm-5.2' — GLM via OpenRouter,
needs OPENROUTER_API_KEY in the env)
--provider <csv> register ruflo API providers (e.g. openai:gpt-5.6)
--route 'act:host[:model]' override one activity's routing (repeatable), e.g.
--route 'implementation:claude:claude-opus-4-8'
--route 'implementation:claude:claude-opus-5'
activities: specification, architecture, design,
implementation, testing, review, security-scan,
security-analysis, documentation, debugging,
Expand Down Expand Up @@ -296,7 +296,7 @@ async function pick({ flags, cwd }) {
aqeProvider = aAns ? aAns : null;
const suggestion = suggestedFallbackFor(enabled);
const fAns = (await rl.question(
`aqe fallback chain, ordered (e.g. "claude-code:claude-opus-4-8; openai:gpt-5.6"${suggestion ? `, blank = use suggested [${suggestion}]` : ', blank = none'}): `,
`aqe fallback chain, ordered (e.g. "claude-code:claude-opus-5; openai:gpt-5.6"${suggestion ? `, blank = use suggested [${suggestion}]` : ', blank = none'}): `,
)).trim().toLowerCase();
aqeFallback = fAns ? parseFallback(fAns) : (suggestion ? parseFallback(suggestion.toLowerCase()) : []);
const provAns = (await rl.question('ruflo API-key providers to register (e.g. openai:gpt-5.6, blank to skip): ')).trim();
Expand Down
2 changes: 1 addition & 1 deletion src/lib/providers.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ export const QE_COURT_TIP = 'agentic-qe ≥ 3.13.0 ships qe-court (adversarial r
* itself), so pairing claude-code + openai is a direct inference from the
* hosts already chosen in the same session. Literal reused from
* docs/PROVIDERS.md's own example rather than inventing new model ids. */
export const AQE_FALLBACK_CODEX_SUGGESTION = 'claude-code:claude-opus-4-8; openai:gpt-5.6';
export const AQE_FALLBACK_CODEX_SUGGESTION = 'claude-code:claude-opus-5; openai:gpt-5.6';
export const suggestedFallbackFor = (enabledHosts) => (enabledHosts.includes('codex') ? AQE_FALLBACK_CODEX_SUGGESTION : null);

// ── agentic-qe router config (.agentic-qe/llm-config.json) ──────────────────
Expand Down
19 changes: 10 additions & 9 deletions src/lib/routing.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -40,13 +40,14 @@ export const SUBSCRIPTION_PROVIDERS = new Set(['claude-code', 'codex', 'ollama',
// `--help`, and docs/PROVIDERS.md. NOT a hard allow-list: any model your host CLI
// accepts also works — these are ak's curated picks. Web-verified on the date
// below; model lines move fast, so re-check and let users override (ADR-0002/0003).
export const MODEL_CATALOG_VERIFIED = '2026-07-23';
export const MODEL_CATALOG_VERIFIED = '2026-07-24';
export const MODEL_CATALOG = {
claude: [
{ id: 'claude-opus-4-8', tier: 'reasoning', note: 'deep reasoning — architecture, design, hard debugging' },
{ id: 'claude-opus-5', tier: 'reasoning', note: 'new top Opus — ~2× Opus 4.8 at the same price, near-Fable on coding/agentic; premium reasoning default' },
{ id: 'claude-sonnet-5', tier: 'balanced', note: 'near-Opus at lower cost — review, spec, release' },
{ id: 'claude-fable-5', tier: 'flagship', note: 'top capability (above Opus), premium — hardest problems' },
{ id: 'claude-fable-5', tier: 'flagship', note: 'top capability (Mythos-class, above Opus 5) — hardest problems' },
{ id: 'claude-haiku-4-5-20251001', tier: 'fast', note: 'cheap/fast — high-volume mechanical work' },
{ id: 'claude-opus-4-8', tier: 'prior', note: 'prior Opus generation — same price as opus-5, kept for pinned configs' },
],
codex: [
{ id: 'gpt-5.4', tier: 'flagship', note: 'coding + reasoning + agentic — recommended execution default' },
Expand Down Expand Up @@ -138,15 +139,15 @@ export function swapRoute(route) {
const R = (host, model, escalate) => ({ host, model, ...(escalate ? { escalate } : {}) });
export const DEFAULT_ROUTES = {
specification: R('claude', 'claude-sonnet-5'),
architecture: R('claude', 'claude-opus-4-8'),
design: R('claude', 'claude-opus-4-8'),
implementation: R('codex', 'gpt-5.4', [{ host: 'claude', model: 'claude-opus-4-8' }]),
testing: R('codex', 'gpt-5.4', [{ host: 'claude', model: 'claude-opus-4-8' }]),
architecture: R('claude', 'claude-opus-5'),
design: R('claude', 'claude-opus-5'),
implementation: R('codex', 'gpt-5.4', [{ host: 'claude', model: 'claude-opus-5' }]),
testing: R('codex', 'gpt-5.4', [{ host: 'claude', model: 'claude-opus-5' }]),
review: R('claude', 'claude-sonnet-5'),
'security-scan': R('codex', 'gpt-5.4'),
'security-analysis': R('claude', 'claude-opus-4-8'),
'security-analysis': R('claude', 'claude-opus-5'),
documentation: R('codex', 'gpt-5.3-codex'),
debugging: R('claude', 'claude-opus-4-8'),
debugging: R('claude', 'claude-opus-5'),
packaging: R('codex', 'gpt-5.3-codex'),
release: R('claude', 'claude-sonnet-5'),
};
Expand Down
3 changes: 2 additions & 1 deletion tests/kit/routing.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,8 @@ test('escalatePolicy bumps ladder activities to their next (cross-vendor) rung',

test('escalatePolicy skips a rung that equals the current route (no same-model retry)', () => {
// a user override to the ladder rung itself must not "escalate" to the same thing
const policy = { implementation: { host: 'claude', model: 'claude-opus-4-8', source: 'user' } };
const rung = DEFAULT_ROUTES.implementation.escalate[0];
const policy = { implementation: { host: rung.host, model: rung.model, source: 'user' } };
assert.ok(!('implementation' in escalatePolicy(policy)));
});

Expand Down
Loading