Repository navigation
Support newer GPT models with isolated text-only execution - #15
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #14
Harvey can now use newer GPT models, including Luna, Sol, Terra, and Astra, while keeping model calls text-only. Users can set their preferred model in config or leave it unset and let Harvey select an eligible listed model.
What was happening
The restriction came from the model's tool configuration. Newer models advertise
code_mode_only, and some also advertise experimental tools. Those settings can expose execution tools even when Harvey disables the usual tool features. Harvey rejected these models before sending a prompt because its inputs include scraped pages and prospect replies, and those calls must have no tools available.That also broke automatic selection on accounts where every listed model used code mode. A user could have access to newer models and still be unable to run Harvey with either their preferred model or a blank model setting.
How we handle it
We read the runtime's model catalog and create a private copy inside Harvey's temporary home. For recognized tool modes, we set
tool_modetodirectand clearexperimental_supported_tools. The rest of each model entry stays intact, including its ID, context window, reasoning capabilities, visibility, and priority.The runtime restarts with that private catalog. We check that its effective configuration points to the exact catalog Harvey created, and pass the same path into the thread configuration. If another configuration layer overrides the catalog, Harvey stops before inference.
Direct mode respects the existing lockdown: no execution environments, dynamic tools, enabled tool features, inherited instructions, active MCP tools, or hooks. Runtime requests against the local mock server confirm that the model receives an empty tool list. Attempted tool calls still abort the turn. The private catalog is deleted with the temporary runtime, and the user's own configuration and login remain untouched.
Model selection
Users can choose an exact model ID:
Or allow automatic selection:
nullmodel selects the highest-priority eligible listed model. Equal priorities are ordered by model ID.There is no model-name allowlist or GPT version ceiling. Future catalog entries can use the same path when their tool metadata is compatible with the verified runtime. Model access still depends on the user's account. The CLI remains pinned to
0.159.3; this change does not claim compatibility with unverified CLI versions.The README and dashboard FAQ now explain the supported model selection behavior.
What we tested
The full local suite passed: 446 tests. The updated pinned-CLI suite passed 21 checks, including every model in the bundled catalog, real YAML config loading, exact model selection, omitted/empty/null defaults, hidden-model behavior, empty tool lists, and isolation failures.
The installed-wheel check also passed, exercising the packaged adapter outside the checkout.
We also ran live inference through Harvey's config loader, Brain, and subscription adapter. Every request below returned the expected
HARVEY_MODEL_OKresponse using the requested model:gpt-6.1-solgpt-6-astragpt-6-solgpt-6-lunagpt-reservegpt-5.6-solgpt-5.6-terragpt-5.6-lunacodex-auto-reviewOmitted, empty, and
nullmodel settings each selectedgpt-6.1-soland returned the expected response. An unknown ID was rejected before inference. That gives us 12 successful live calls and one expected rejection, covering all nine entries in the test account's catalog. The native login file was unchanged after testing.Live results establish behavior for that account and catalog; they do not guarantee access to every model on every ChatGPT plan. CI uses scripted and localhost mock runtimes and needs no live credentials.