Epic: Feedback-loop hardening (PRD E1)

[sungjunlee]

Tracking issue for PRD workstream E1 — docs/prd-2026-07-consolidation.md §5.

Make the improvement loop trustworthy: changes kept by craft-autoresearch should generalize, and convergence claims by craft-tune should carry the right caveat. Highest-priority workstream (Goal G1).

Tasks

• craft-autoresearch: add train/holdout split to the experiment contract (E1.1) #104 — craft-autoresearch: train/holdout split (E1.1, P0)
• craft-tune: add independent-second-pass caveat to Self-LGTM convergence (E1.2) #105 — craft-tune: Self-LGTM independent-second-pass caveat (E1.2, P0)
• SkillOpt pilot: run one machine-scorable optimization pass on craft-critique (E1.3) #106 — SkillOpt pilot on craft-critique (E1.3, P2)
• craft-autoresearch: document dormant explicit-only experiment posture #107 — craft-autoresearch: methodology-vs-executor repositioning (E1.4, P2)

Done when

• Autoresearch contract template shows the train/holdout split; eval-guide + contract-example updated in the same commits.
• craft-tune final output names the independent-second-pass recommendation.
• SkillOpt pilot has a dated adopt/fork/defer decision note, regardless of outcome.