Skip to content

Support newer GPT models with isolated text-only execution - #15

Merged
ethanplusai merged 2 commits into
ethanplusai:mainfrom
ikemHood:fix/newer-gpt-models
Oct 15, 2026
Merged

ethanplusai merged 2 commits into
ethanplusai:mainfrom
ikemHood:fix/newer-gpt-models

Conversation

@ikemHood

Copy link
Copy Markdown
Contributor

Fixes #14

Harvey can now use newer GPT models, including Luna, Sol, Terra, and Astra, while keeping model calls text-only. Users can set their preferred model in config or leave it unset and let Harvey select an eligible listed model.

What was happening

The restriction came from the model's tool configuration. Newer models advertise code_mode_only, and some also advertise experimental tools. Those settings can expose execution tools even when Harvey disables the usual tool features. Harvey rejected these models before sending a prompt because its inputs include scraped pages and prospect replies, and those calls must have no tools available.

That also broke automatic selection on accounts where every listed model used code mode. A user could have access to newer models and still be unable to run Harvey with either their preferred model or a blank model setting.

How we handle it

We read the runtime's model catalog and create a private copy inside Harvey's temporary home. For recognized tool modes, we set tool_mode to direct and clear experimental_supported_tools. The rest of each model entry stays intact, including its ID, context window, reasoning capabilities, visibility, and priority.

The runtime restarts with that private catalog. We check that its effective configuration points to the exact catalog Harvey created, and pass the same path into the thread configuration. If another configuration layer overrides the catalog, Harvey stops before inference.

Direct mode respects the existing lockdown: no execution environments, dynamic tools, enabled tool features, inherited instructions, active MCP tools, or hooks. Runtime requests against the local mock server confirm that the model receives an empty tool list. Attempted tool calls still abort the turn. The private catalog is deleted with the temporary runtime, and the user's own configuration and login remain untouched.

Model selection

Users can choose an exact model ID:

brain:
  provider: codex
  model: gpt-6.1-sol

Or allow automatic selection:

brain:
  provider: codex
  • An explicit ID must exist in the account's catalog. Harvey uses that exact model or fails, with no silent model substitution.
  • Hidden models can be selected explicitly, but are never picked automatically.
  • An omitted, empty, or null model selects the highest-priority eligible listed model. Equal priorities are ordered by model ID.
  • Unknown model IDs and unrecognized tool modes are rejected before inference.

There is no model-name allowlist or GPT version ceiling. Future catalog entries can use the same path when their tool metadata is compatible with the verified runtime. Model access still depends on the user's account. The CLI remains pinned to 0.159.3; this change does not claim compatibility with unverified CLI versions.

The README and dashboard FAQ now explain the supported model selection behavior.

What we tested

The full local suite passed: 446 tests. The updated pinned-CLI suite passed 21 checks, including every model in the bundled catalog, real YAML config loading, exact model selection, omitted/empty/null defaults, hidden-model behavior, empty tool lists, and isolation failures.

The installed-wheel check also passed, exercising the packaged adapter outside the checkout.

We also ran live inference through Harvey's config loader, Brain, and subscription adapter. Every request below returned the expected HARVEY_MODEL_OK response using the requested model:

Model Live result
gpt-6.1-sol Passed
gpt-6-astra Passed
gpt-6-sol Passed
gpt-6-luna Passed
gpt-reserve Passed
gpt-5.6-sol Passed
gpt-5.6-terra Passed
gpt-5.6-luna Passed
codex-auto-review Passed

Omitted, empty, and null model settings each selected gpt-6.1-sol and returned the expected response. An unknown ID was rejected before inference. That gives us 12 successful live calls and one expected rejection, covering all nine entries in the test account's catalog. The native login file was unchanged after testing.

Live results establish behavior for that account and catalog; they do not guarantee access to every model on every ChatGPT plan. CI uses scripted and localhost mock runtimes and needs no live credentials.

@ethanplusai
ethanplusai merged commit d05ce58 into ethanplusai:main Oct 15, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

it only supports gpt 5.5

2 participants