Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/MODEL-PRICING-AUDIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,9 @@ which the estimator does not model.
GPT-6 models first and labels GPT-5.6 "Older"; OpenAI's Codex docs make Sol the
default preset ([Codex models](https://learn.chatgpt.com/docs/models)). GPT-5.6
models remain listed for pins.
- Cross-host tiers pair by role and per-token price: GPT-6 Sol ↔ Sonnet 5
($2/$10), GPT-6 Astra ↔ Fable 5.1 ($10/$50), GPT-6 Luna ↔ Haiku 4.5.
- Cross-host tiers pair by role: GPT-6 Sol ↔ Sonnet 5 (balanced, $2/$10),
GPT-6 Astra ↔ Opus 5.5 (reasoning; Astra also pairs with Fable 5.1), GPT-6
Luna ↔ Haiku 4.5 (fast). A Codex-driven seed runs reasoning work on Astra.
- Routes seeded before this change are reported as diverged and keep their model
until `ak x host refresh`.
- Not added as retirements: OpenAI's Codex docs retire GPT-5.4 / 5.4 Mini
Expand Down
15 changes: 13 additions & 2 deletions docs/PROVIDERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -460,8 +460,19 @@ ak host pick --primary-host codex # make codex the lead; claude becomes th
**Which host leads.** `--primary-host claude|codex` (default `claude`) chooses the primary.
Codex-primary **mirrors** the default table below — codex takes the reasoning/review lead
and claude becomes the alternate/escalation target — so the experience is ambidextrous
regardless of which CLI drives. `ak status` marks the primary and fails (not warns) if the
primary host is missing.
regardless of which CLI drives. Models pair by tier, so each host leads reasoning work on its
reasoning model and escalation steps up a tier either way:

| Tier | Claude | Codex |
|---|---|---|
| reasoning | `claude-opus-5-5` | `gpt-6-astra` |
| balanced | `claude-sonnet-5` | `gpt-6-sol` |
| fast | `claude-haiku-4-5-20251001` | `gpt-6-luna` |

With Codex leading, architecture, design, security-analysis and debugging run on `gpt-6-astra`,
and implementation and testing run on `claude-sonnet-5`, escalating to `gpt-6-astra`. Astra costs
more per token than Opus 5.5 ($10/$50 vs $4/$20), so Codex-led reasoning uses more Codex allowance.
`ak status` marks the primary and fails (not warns) if the primary host is missing.

**OpenCode is explicit, not seeded or AQE-projected.** Enable it, then use `ak run` with either
a persisted route or a run-local override:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,13 +48,14 @@ as such** wherever surfaced (an `ak` tag in the UI, a comment in the defaults ta
Default *models* map to the host's appropriate tier (Opus for deep reasoning, Sonnet for review, a Codex
model for execution) and are treated as **soft defaults** — see the "open question" on pinning live model IDs.

Tier is the pairing key, not the model id: `MODEL_CATALOG` spells `flagship`/`balanced`/`fast`
Tier is the pairing key, not the model id: `MODEL_CATALOG` spells `reasoning`/`balanced`/`fast`
identically on both hosts so primary-host mirroring can map a route to its counterpart's equivalent.
As of 2026-09-23 execution routes to `gpt-6-sol` (balanced) and mechanical work to `gpt-6-luna`
(fast), and deep reasoning routes to `claude-opus-5-5`. These are agentic-kit default preferences,
following each vendor's recommended starting model, not evidence that earlier models were withdrawn.
Cross-host tiers pair by role and per-token price: `gpt-6-sol` ↔ `claude-sonnet-5`,
`gpt-6-astra` ↔ `claude-fable-5-1`, `gpt-6-luna` ↔ Haiku. Withdrawn ids are handled by the retirement
`gpt-6-astra` ↔ `claude-opus-5-5` (reasoning; Astra also pairs with `claude-fable-5-1`),
`gpt-6-luna` ↔ Haiku. See [ADR-0006](0006-primary-host-and-ambidextrous-mirroring.md) for tier parity. Withdrawn ids are handled by the retirement
mechanism in [ADR-0003](0003-auto-seed-dual-host-provenance.md), not by editing this table alone.

## Consequences
Expand Down
20 changes: 14 additions & 6 deletions docs/adr/0006-primary-host-and-ambidextrous-mirroring.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,12 @@

- **Status:** Amended by [ADR-0020](0020-ga-stable-surfaces.md)
- **Date:** 2026-07-23
- **Updated:** 2026-07-30
- **Update note:** Preserved primary-host mirroring while removing the compatibility executor and
moving persisted intent to `routing.primaryHost`.
- **Updated:** 2026-09-23
- **Update note:** Tier parity: both hosts now have a `reasoning` tier (`claude-opus-5-5` ↔
`gpt-6-astra`), so a Codex-driven seed gets reasoning-tier models and an escalation that steps up a
tier, as a Claude-driven seed does. A catalog entry may `pairs` with an extra tier.
- **Earlier update (2026-07-30):** Preserved primary-host mirroring while removing the compatibility
executor and moving persisted intent to `routing.primaryHost`.
- **Deciders:** agentic-kit maintainers

> **GA amendment:** primary-host mirroring remains. The old command spelling, adapter dependency,
Expand Down Expand Up @@ -51,9 +54,14 @@ orchestrator stays symmetric underneath.
- Codex-primary is expressible in one flag at setup or pick time, with the whole experience
(routing, escalation, status severity, dashboard indicator) following symmetrically.
- The default (claude-primary) is unchanged, so existing repos see no difference.
- The mirror is coarse where host model tiers don't line up (claude tiers ≠ codex tiers): a
swapped route falls back to the counterpart host's recommended model. Users tune per activity
with `--route`.
- The mirror pairs models by tier. Both hosts carry `reasoning`, `balanced` and `fast` tiers
(`claude-opus-5-5` ↔ `gpt-6-astra`, `claude-sonnet-5` ↔ `gpt-6-sol`, Haiku ↔ `gpt-6-luna`), so
reasoning work and escalation land on the same tier whichever host drives. A model with no
same-tier twin uses an explicit `pairs` entry (`claude-fable-5-1` → `gpt-6-astra`), else the
counterpart host's recommended model. Users tune per activity with `--route`.
- Tier parity is not price parity: `gpt-6-astra` costs more per token than `claude-opus-5-5`
($10/$50 vs $4/$20). Codex-driven reasoning therefore costs more at API rates, and uses more
Codex plan allowance, than Claude-driven reasoning.
- `status`/dashboard must read `primaryHost` to render the correct severity + primary marker
(done: `status.mjs` host rows, `dashboard-server.mjs` routing matrix).

Expand Down
27 changes: 17 additions & 10 deletions src/lib/routing.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,10 @@ export const MODEL_CATALOG_VERIFIED = '2026-09-23';
export const COST_AXIS_NOTE = 'per-token price ≠ per-task cost — compare total tokens, cache use and agentic turns on representative tasks';
// Tier names are the pairing key for swapHostModel(): a codex tier only mirrors
// to a claude model (and back) when BOTH catalogs use the same tier string.
// Keep `flagship`/`balanced`/`fast` spelled identically on both hosts —
// renaming one side silently degrades every mirrored route to cat[0].
// Keep `reasoning`/`balanced`/`fast` spelled identically on both hosts —
// renaming one side silently degrades every mirrored route to cat[0]. A model
// may also list `pairs`: extra tiers it answers for when the other host has no
// model of that exact tier (Astra stands in for claude's `flagship`).
export const MODEL_CATALOG = {
claude: [
// Opus 5.5 (released 2026-09-22) is the reasoning preset: Anthropic's
Expand All @@ -105,12 +107,17 @@ export const MODEL_CATALOG = {
// catalog ranks these first and labels every GPT-5.6 model "Older", and
// OpenAI's Codex docs make Sol the default preset
// (learn.chatgpt.com/docs/models, verified 2026-09-23). Tier pairing with the
// claude catalog follows role and per-token price: Sol ↔ Sonnet 5 ($2/$10
// both), Astra ↔ Fable 5.1 ($10/$50 both), Luna ↔ Haiku. Sol is first, so a
// claude tier with no codex twin (reasoning) mirrors to the workhorse.
// claude catalog follows role: Sol ↔ Sonnet 5 (balanced, $2/$10 both),
// Luna ↔ Haiku (fast), and Astra ↔ Opus 5.5 (reasoning). Astra is the
// reasoning tier so a Codex-driven seed gets what a Claude-driven one does:
// reasoning work on the host's reasoning model and an escalation that steps
// UP a tier (Sonnet → Astra), not sideways to an equal-priced workhorse.
// Astra is priced above Opus 5.5 ($10/$50 vs $4/$20); OpenAI positions it for
// the hardest end-to-end work, and Anthropic reports Opus 5.5 at Fable 5.1
// level, which is Astra's class. It also answers for claude's `flagship`.
{ id: 'gpt-6-sol', tier: 'balanced', note: 'workhorse preset for coding, testing and everyday work' },
{ id: 'gpt-6-luna', tier: 'fast', note: 'fast-tier preset for mechanical work, documentation and packaging' },
{ id: 'gpt-6-astra', tier: 'flagship', note: 'frontier preset for the hardest end-to-end work; compare per-task cost before selection' },
{ id: 'gpt-6-astra', tier: 'reasoning', pairs: ['flagship'], note: 'reasoning preset for the hardest end-to-end work; compare per-task cost before selection' },
// Still served, retained for user pins. Codex labels them "Older".
{ id: 'gpt-5.6-sol', tier: 'prior', note: 'prior coding preset retained for user pins; compare measured per-task results before switching' },
{ id: 'gpt-5.6-terra', tier: 'prior', note: 'prior balanced preset retained for user pins' },
Expand Down Expand Up @@ -224,14 +231,14 @@ const MODEL_TIER = Object.fromEntries(
HOSTS.flatMap((h) => (MODEL_CATALOG[h] ?? []).map((m) => [m.id, m.tier])),
);

/** The opposite host's best model when swapping. Tier names differ between claude
* and codex, so an exact-tier match is best-effort; else fall back to that host's
* first (recommended) model. Pure. */
/** The opposite host's best model when swapping: the model of the same tier,
* else one that `pairs` with that tier, else that host's first (recommended)
* model. Pure. */
export function swapHostModel(host, model) {
const other = host === 'claude' ? 'codex' : 'claude';
const cat = MODEL_CATALOG[other] ?? [];
const tier = MODEL_TIER[model];
const pick = (tier && cat.find((m) => m.tier === tier)) || cat[0];
const pick = (tier && (cat.find((m) => m.tier === tier) || cat.find((m) => m.pairs?.includes(tier)))) || cat[0];
return { host: other, model: pick?.id };
}

Expand Down
34 changes: 34 additions & 0 deletions tests/kit/routing-primary.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,40 @@ test('seedActivityRoutes with codex primary mirrors host assignments', () => {
assert.equal(policy.implementation.host, 'claude'); // claude becomes the alternate
});

// Tier parity: a Codex-driven seed must get the same tier benefits a
// Claude-driven one does — reasoning work on each host's reasoning-tier model,
// and an escalation that steps UP a tier rather than sideways.
const REASONING_ACTIVITIES = ['architecture', 'design', 'security-analysis', 'debugging'];

test('each host leads reasoning work on its own reasoning-tier model', () => {
const claudeLed = seedActivityRoutes({ hosts: ['claude', 'codex'] });
const codexLed = seedActivityRoutes({ hosts: ['claude', 'codex'], primary: 'codex' });
for (const act of REASONING_ACTIVITIES) {
assert.deepEqual([claudeLed[act].host, claudeLed[act].model], ['claude', 'claude-opus-5-5'], act);
assert.deepEqual([codexLed[act].host, codexLed[act].model], ['codex', 'gpt-6-astra'], act);
}
});

test('escalation steps up to the reasoning tier whichever host drives', () => {
const claudeLed = seedActivityRoutes({ hosts: ['claude', 'codex'] });
const codexLed = seedActivityRoutes({ hosts: ['claude', 'codex'], primary: 'codex' });
for (const act of ['implementation', 'testing']) {
assert.deepEqual(claudeLed[act].escalation, [{ host: 'claude', model: 'claude-opus-5-5' }], act);
assert.equal(codexLed[act].model, 'claude-sonnet-5', act);
assert.deepEqual(codexLed[act].escalation, [{ host: 'codex', model: 'gpt-6-astra' }], act);
}
});

test('tier pairs mirror both ways, and flagship pins reach Astra through an explicit pair', () => {
assert.equal(swapHostModel('claude', 'claude-opus-5-5').model, 'gpt-6-astra');
assert.equal(swapHostModel('codex', 'gpt-6-astra').model, 'claude-opus-5-5');
assert.equal(swapHostModel('claude', 'claude-fable-5-1').model, 'gpt-6-astra');
assert.equal(swapHostModel('claude', 'claude-sonnet-5').model, 'gpt-6-sol');
assert.equal(swapHostModel('codex', 'gpt-6-sol').model, 'claude-sonnet-5');
assert.equal(swapHostModel('claude', 'claude-haiku-4-5-20251001').model, 'gpt-6-luna');
assert.equal(swapHostModel('codex', 'gpt-6-luna').model, 'claude-haiku-4-5-20251001');
});

test('seedActivityRoutes stamps every seeded entry with provenance:seeded', () => {
const policy = seedActivityRoutes({ hosts: ['claude', 'codex'], primary: 'codex' });
assert.ok(Object.values(policy).every((r) => r.provenance === 'seeded'));
Expand Down
Loading