Issues

• Helper lint false positives: sheet_name read as a task id, answer_position read as a gold path (refuses legitimate spreadsheet helpers)

  algorithmbugpriority-p1
  skillberry-ai/cap-evolve#912 · Filter by author bcarmeli opened on Oct 23, 2026
  1

• AppWorld ablations: harness vs instructions, edit surface, optimizer feedback — gpt-oss-120b agent, Sonnet 5.5 optimizer, 3 seeds

  algorithmhonestyobservabilitypaper-experimentpriority-p1
  skillberry-ai/cap-evolve#908 · Filter by author OsherElhadad opened on Oct 22, 2026
  1

• Experiment plan: CapEvolve (HC, stock gate k=0.5) vs 9 optimizer arms on SpreadsheetBench verified_400 — Gemma-4-31B agent, GLM-5.3 optimizer, k=3, 3 seeds

  algorithmhonestypaper-experiment
  skillberry-ai/cap-evolve#895 · Filter by author OsherElhadad opened on Oct 20, 2026
  3

  

• v6 g2_e1: cap-evolve config review request (for Osher)

  algorithmhonestypaper-experimentpriority-p1question
  skillberry-ai/cap-evolve#891 · Filter by author bcarmeli opened on Oct 20, 2026
  1

• cap-evolve estimate underprices agentic optimizer iterations by ~1000x

  bughonesty
  skillberry-ai/cap-evolve#878 · Filter by author erezh16 opened on Oct 15, 2026
  1

• hill-climb has no pre-rollout candidate check (lint/precheck hook) for adapters

  algorithmenhancement
  skillberry-ai/cap-evolve#877 · Filter by author erezh16 opened on Oct 15, 2026
  2

• hill-climb: evaluation-error steps still consume max_iterations and trigger a new optimizer call each

  algorithmbug
  skillberry-ai/cap-evolve#876 · Filter by author erezh16 opened on Oct 15, 2026
  1

• Optimizer can read the whole host: no way to keep benchmark gold data away from claude-code and its subagents

  bugenhancementhonestypriority-p0
  skillberry-ai/cap-evolve#875 · Filter by author erezh16 opened on Oct 15, 2026
  1

• No visible signal when a reused optimizer_instructions_file doesn't match the project's actual task cardinality (silent single-task/multi-task mismatch)

  bugenhancementhonestyobservabilitypriority-p1
  skillberry-ai/cap-evolve#868 · Filter by author bcarmeli opened on Oct 15, 2026
  1

• harness._algorithm_brief tells the optimizer it is scored on "the SAME held-out val tasks" even under split_mode: pooled, which is false

  algorithmbughonestypriority-p1
  skillberry-ai/cap-evolve#867 · Filter by author bcarmeli opened on Oct 15, 2026
  1

• [WIP] benchmark: cap-evolve GLM 5.3 skill optimization on DeepSWE 1.1

  algorithm
  skillberry-ai/cap-evolve#864 · Filter by author AIMikav opened on Oct 14, 2026
  1

• P2 evaluator logs confirm-stratum chunk means to the run's events.jsonl (should be sealed-only)

  bughonestypriority-p0
  skillberry-ai/cap-evolve#847 · Filter by author OsherElhadad opened on Oct 13, 2026
  2

• optimizer_cost: conductor runs under-bill jobs ~8x (stream-json usage) and keep an operator session registered before conductor.json existed

  bughonestyobservabilitypriority-p1
  skillberry-ai/cap-evolve#837 · Filter by author OsherElhadad opened on Oct 13, 2026
  1

• Pooled finalize must not score the confirm stratum C, or must seal it

  bughonestypriority-p2
  skillberry-ai/cap-evolve#828 · Filter by author OsherElhadad opened on Oct 13, 2026
  1

• CapEvolve v2 (epic): gate and budget fixes, pooled splits, SH-C evaluation, AO-2 graph conductor, deterministic helpers

  algorithmenhancementpaper-experimentpriority-p1
  skillberry-ai/cap-evolve#808 · Filter by author OsherElhadad opened on Oct 12, 2026
  1

• [experiment] SWE-bench Verified with GLM 5.3 as agent and optimizer: clean design (Suite G) and what the 482/500 claim needs

  algorithmenhancementhonestypaper-experimentpriority-p0
  skillberry-ai/cap-evolve#805 · Filter by author OsherElhadad opened on Oct 12, 2026
  1

• Defect: test-seal guard advertises its own override (CAPEVOLVE_ALLOW_TEST_RESCORE) to the optimizer agent

  bughonestypriority-p0
  skillberry-ai/cap-evolve#799 · Filter by author OsherElhadad opened on Oct 12, 2026
  1

• [design] Screen accounting for selective evaluation: stratified screens with inclusion probabilities + equal-size random control

  algorithmenhancementhonestypaper-experiment
  skillberry-ai/cap-evolve#789 · Filter by author OsherElhadad opened on Oct 11, 2026
  1

• [design] Interaction-aware merge verification for agent-optimize (fresh common-block parent/branch/merge evaluation + ddmin)

  algorithmenhancementpaper-experiment
  skillberry-ai/cap-evolve#787 · Filter by author OsherElhadad opened on Oct 11, 2026
  1

• [design] Promotion rule P as one shared function for hill-climb and agent-optimize (gate parity)

  algorithmenhancementhonestypaper-experiment
  skillberry-ai/cap-evolve#785 · Filter by author OsherElhadad opened on Oct 11, 2026
  1

• [provenance] Historical optimizer sessions read gold workbooks; run 36887243479 read a prior sealed-test evaluation

  algorithmbughonestyobservabilitypriority-p0
  skillberry-ai/cap-evolve#784 · Filter by author OsherElhadad opened on Oct 11, 2026
  1

• [audit] SpreadsheetBench: gold answer workbooks are visible (read-write) inside the agent sandbox

  bughonestypriority-p0
  skillberry-ai/cap-evolve#782 · Filter by author OsherElhadad opened on Oct 11, 2026
  1

• Engine defects from the first real run: no branching from unpromoted tips, fill-probe overspend, superseded prunes counted as refuted, stale clusters, auto-added session billing

  algorithmbughonestypriority-p1
  skillberry-ai/cap-evolve#781 · Filter by author OsherElhadad opened on Oct 11, 2026
  1

• reward_gated blocks real gains: unrequested cost cap and z>2.73 bar (found by first real tau2 run)

  algorithmbughonestypriority-p1
  skillberry-ai/cap-evolve#780 · Filter by author OsherElhadad opened on Oct 11, 2026
  1

• [provenance] Latest agent-optimize tau2 candidate (c_win) fails full-val confirmation; evidence not committed

  documentationhonestypaper-experimentpriority-p0
  skillberry-ai/cap-evolve#778 · Filter by author OsherElhadad opened on Oct 11, 2026
  1