Repository navigation
Nous v2: one session, enforced validation, light method (#330) - #333
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…guard - Pin claude-agent-sdk==0.2.172 (CLI 2.1.308) in pyproject.toml - orchestrator/v2/session.py: Session with client_factory, builds ClaudeAgentOptions per spec 3.1 (preset system prompt with append, max_buffer_size 32MB, CLAUDE_CODE_MAX_OUTPUT_TOKENS env, hooks passthrough) - orchestrator/v2/errors.py: TransientError hierarchy, is_transient() classifies ProcessError exit 143, "Fatal error in message reader", API 5xx, and timeouts as transient (B9) - tests/fakes/sdk_client.py: FakeClient plays scripted message sequences, invokes hook callbacks, writes files — no live LLM calls - tests/conftest.py: ClaudeSDKClient.connect blocked in all tests - 39 new tests (1393 total), all green Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
FakeClient.connect/query/interrupt/disconnect are now async def, receive_response is an async generator yielding each message. All test methods in test_session.py and test_fake_client.py converted to async def (pytest-asyncio auto mode). Added signature tests verifying FakeClient methods are coroutines/async generators. Corrected progress.txt SDK API note. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Session.connect now awaits client.connect(), turn() awaits client.query() and iterates receive_response() as async for, disconnect() awaits client.disconnect(). Completes the async mismatch fix started in 2f2f9fd. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
8 JSON schemas (baseline, problem, bundle, plan, findings, principle_updates, answer, ledger_entry) with B5 extra-field handling, evidence rules for JSON/JSONL/CSV/TSV, noise honesty validation, paired-seed check, locked_parameters/workload checks, task_limits (B11), bundle hash, secret masking helper (3.14). 222 tests across test_schemas_v2, test_validators_v2, test_evidence_rules, test_secrets. Full suite: 1617 passed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Fixes from pr-review-toolkit review: - Path traversal guard in check_evidence_item (is_relative_to check) - RFC 6901 JSON Pointer "/" now resolves to doc[""] not root - Paired-seed check compares all run pairs, not just first - Baseline schema noise field wired to oneOf $defs - Silent JSONL corruption in _check_task_limits now reported - Regex no-capture-group now errors instead of silent pass - Empty path/key in evidence items rejected early - IsADirectoryError added to except clause Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|effect.value| <= 2 * noise_sd now triggers repair message for CONFIRMED/REFUTED hypotheses. Added effect field to findings schema. 6 new tests covering: effect inside noise rejected, outside passes, deterministic zero sd, unmeasured exempt, no effect field skips check. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Add _get_baseline_noise_sd() helper that reads baseline.json and returns noise sd (0.0 for deterministic, sd for measured, None otherwise). CONFIRMED/REFUTED hypotheses without noise_sd now fall back to baseline noise before requiring noise_unmeasured. Missing effect.value with noise_sd present is now an error, not a silent pass. Four new tests cover the fallback paths. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PostToolUse records Bash commands to runs.jsonl with secret masking.
PostToolUseFailure records failed commands with exit code parsing.
Stop hook validates phase artifacts, blocks up to 6 consecutive times.
PreCompact hook writes state/digest.json for context recovery.
build_hooks() returns the hooks dict with per-event timeouts.
39 tests covering all four hooks, experiment command matching,
consecutive block escalation, digest content, and secret masking.
Review fixes (code-reviewer + silent-failure-hunter):
- Fixed output_bytes in failure hook: .encode("utf-8") for byte count
- Widened stop hook exception catch to include OSError/UnicodeDecodeError
- Injected deterministic now=lambda in test helper per tests/CLAUDE.md
- cache key uses tuple(patterns) not id() (pre-review fix)
- _append_runs_jsonl wrapped in try/except OSError with logging
- digest write wrapped in try/except with logging
- Skipped: runs.jsonl write propagation (logging is the fix per prior
review; propagating would crash the hook framework)
- Skipped: eager regex validation (patterns from campaign config, crash
at first use is acceptable fail-fast)
- Skipped: error field in failure rows (exit_code is sufficient;
error text may contain secrets)
- Skipped: _PHASE_ARTIFACTS plan in EXECUTE (plan belongs to DESIGN)
- Skipped: corrupt-JSON markers in digest (best-effort, warning is fine)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
B1 recurred: build_hooks() returned dicts instead of HookMatcher objects, causing _hooks_to_internal_format to produce empty hook lists. Also: callbacks must be async def returning dict, tool_response must be stringified before masking, and FakeClient now uses the real SDK contract (_hooks_to_internal_format + _convert_hook_output_for_cli + TypeError on sync callbacks). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
New modules: - orchestrator/v2/stop_rule.py (139 lines): check_stop() with halt > stop_sentinel > answered > max_iterations > stall priority, claims_needed gating, claims_changed/best_changed/principles_changed helpers. - orchestrator/v2/loop.py (285 lines): CampaignConfig dataclass, parse_campaign_config(), Campaign class driving BASELINE → (DESIGN → EXECUTE)* → REPORT lifecycle, bundle hash lock with pre-registration violation detection, per-iteration ledger writes, principle accumulation. Tests: 42 new tests (22 stop_rule + 17 loop + 3 review-fix tests). Review fixes applied: 9 accepted (None guards, baseline validation, ledger-per-iteration, claims_needed gating, stall uses current_* params). 5 skipped with rationale (no-ops or over-engineering). Runtime: 1,694 lines (target ~2,500). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reviewer findings addressed (from cross-session review): 1. Hooks wired into loop: HookState built in Campaign.run(), phase/iteration set before each turn. build_hooks() provides the hooks dict to Session options. 2. Campaign worktree commit per phase: _commit_phase() runs git add -A && git commit after each phase turn. Skips if no .git or no changes. 3. Phase messages completed per spec 3.2: BASELINE asks for task_type when not configured, DESIGN says to read digest and baseline and frame predictions relative to baseline, EXECUTE mentions wait_for placeholder. 4. Bundle lock raises on missing bundle.json after DESIGN (was silently returning ""). 5. question_answered moved to findings.json top level per spec 3.4 (was per-hypothesis). Schema, stop_rule, and all tests updated. 6. hooks.py _iter_dir fixed: removed erroneous "runs/" subdir to match loop.py's iter-N path. Code review fix: - _commit_phase guard simplified from wrong conjunction to single .git existence check. - experiment_commands sourced from CampaignConfig field (was incorrectly reading from prompts dict). Runtime: 1,758 lines (target ~2,500). Tests: 1692 passed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
SPEC.md line 271: "iter-N/... is short for <work_dir>/runs/iter-N/..." Added iter_dir() helper in loop.py. Updated hooks.py _iter_dir and phase messages. All tests updated to use runs/iter-N/ layout. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Repair: 2 tries then status: partial, never drops results. Failure breaker: 3 identical fingerprints → halt.json → REPORT. wait_for: polls file/command every 60s in executor (no event loop block). Preflight: validates hooks + validators on fixtures at campaign start. Review fixes: poll_wait_for logs command errors and timeout warnings, wait_for polling runs via run_in_executor to avoid blocking async loop. 1712 tests pass, 2086 runtime lines. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
smoke/1.json missing. See BLOCKED.md for instructions. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1. Agent receives "Runs X finished; continue EXECUTE." after poll. 2. Stop hook allows stop when pending_wait_for is set in HookState. 3. All wait_for conditions polled together in one 60s loop. 4. Timeout counts as a breaker failure. 3 new integration tests. 1715 tests pass, 2159 runtime lines. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ation, simplify - pending_wait_for changed from bool to list[str]; Stop hook checks conditions live via check_wait_condition - Campaign passes cwd=target_system.path to Session per spec 3.1 - Secrets read from os.environ instead of literal "***" - plan.json added to DESIGN phase artifacts in Stop hook - Consolidated check_wait_condition into robustness.py (was duplicated in loop.py and hooks.py) - Extracted _build_row helper in hooks.py - Removed dead _EVIDENCE_EXTENSIONS, FailureBreaker.reset(), unused imports Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
loop.py:171 and loop.py:474 read target_system["path"] but the campaign schema uses repo_path. cwd was None for every real campaign, and _commit_phase silently did nothing. Tests used "path" which masked it. Also adds parse_campaign_config validation: missing repo_path fails early with a clear error. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CostMeter deduplicates by message_id, tracks per-phase totals, prices from defaults.yaml. Wired into Campaign.run via session.turn(on_message=). Ledger entries record cumulative cost_meter_usd and token counts. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…er 2 cuts EffortManager tracks consecutive and total cuts, persists cut_offs to state.json. Ladder: max > xhigh > high > medium > low. Phase reset restores configured effort. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p-rule requirements, safe phase commits Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dge on this Mac) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…c 3.5) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ode (orchestrator 6104->3076 lines) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…B12) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… crash) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…wer evidence exported into the repo Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ady validate Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t as evidence_detail Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… evidence without crashing Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… repo root, schema-exact method prompt, absolute artifact paths Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ric-string match, append-only claims_needed, repro must run as written, export path fix Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rk done Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… loop; drop v1 wiki tooling Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… --max-cli-retries, --sandbox) They were accepted, warned about and ignored; nothing in v2 read them. Passing one is now an argparse error. README and pyproject comment updated. 653 tests pass. The s390x deployment still passes these flags; tracked in a separate issue. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…esume iter-1 prev, campaign worktree, persisted breaker, fake fires Stop/PreCompact/PostToolUseFailure) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR delivers Nous v2 from issue #330 into the
nous-v2branch. It does not changemain.mainkeeps old Nous until a separate, later decision.Nous v2 is Claude Code plus a light scientific method plus checks enforced by code:
claude-agent-sdk==0.2.172) runs BASELINE → DESIGN → EXECUTE → REPORT. Nous switches the phases.runs.jsonl, run the validators when a phase ends (Stop), and write a digest before compaction. Nous never blocks or judges a command.nous resume, and it handles effort when output hits the cap.<repo>/.nous-experiments/<run_id>/on branchnous/<run_id>, so the user's checkout is never touched.Old Nous is deleted: the planner/executor split, the dispatchers, the shell hooks, gate summaries, the critic and complexity tiers, and the five v1 CLI flags.
orchestrator/went from about 15,900 lines to about 3,400; the v2 runtime is about 2,700.Closes nothing yet. Refs #330.
How it was built and checked
/goalsession, with a reviewer session alongside. Each item went test-first, then/pr-review-toolkit:review-pr, then commit and push. The plan and full log are indocs/plans/nous-v2/(SPEC, plan.json, progress.txt, LEAN_AUDIT.md).conftest.pyguard is kept and now extended toClaudeSDKClient.connect.nous/<run_id>in the campaign worktree.Merge bar (R13): NOT met after 3 improvement rounds
See
docs/plans/nous-v2/EVAL_REPORT.md. These are 3 oracle tasks compared against the recorded Claude + 4-line prompt (L2). Frontier-CS p5 and p225 were deferred because there is no Docker judge on the build machine.Across every run: 0 lost rounds, 0 crashes, 0 pre-registration violations. The merge bar runs found and fixed 7 real Nous bugs. This PR delivers a working, tested base. Closing the remaining accuracy and cost gaps is follow-up work in PRs into
nous-v2.Known follow-ups
nous-v2merges intomain.runs.jsonlfields;Test plan
.venv/bin/python -m pytest -q: 669 passedtests/v2/test_docs_examples.py🤖 Generated with Claude Code