test(turn-driver): pin the controller rule sequence and prove it is load bearing - #4580
Conversation
…oad bearing The controller evaluates `rules` in order and stops at the first match, so a rule that refines another must stay in front of it. M2 made the JSON contract the single source of that program, but nothing pinned the sequence: the suite asserted id uniqueness, domain membership and disposition validity, and a separate test proved key order *inside* a rule is immaterial. Reordering two rules during a regeneration would silently change first-match semantics. Pin the 29 ids as `_RULE_SEQUENCE`, and prove the pin protects real behaviour: for five refining/general pairs, moving the specific rule behind the general one must change at least one cell of the full decision matrix. Compare the operator-visible decision, not the disposition alone. Two rules can share a disposition and explain it differently: moving `host_exhausted` behind `failed_receipt` keeps `repair` but silently replaces "retryable host failure ... exhausted its bounded attempt budget" with "turn receipt ended in host_failure". A disposition-only comparison reports no change there, so the reason is part of the observed decision. Refs loopx-project#4447 (M2 hardening). Signed-off-by: song <22676124+songoow@users.noreply.github.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
huangruiteng
left a comment
There was a problem hiding this comment.
动机
评审 head:61f9d881bd70e21869106245d221cf918faa3393;base:main(merge-base 81f435d6b)。
交付判定(policy 6):goal_achieved。本 PR 的目标是"把 controller 规则顺序钉住,并证明该顺序是承重的",head 上这一目标成立,且我没有找到 off-goal 或碎片化的证据。
为什么值得钉:decide_loop_disposition 按顺序求值规则、命中第一条 return 即返回,因此规则顺序本身就是被交付的决策契约。M2 把规则从手写 if/elif 变成数据后,任何一次重排/重新生成都可能静默改变 disposition——而现有测试只覆盖"注册合法性"(未知 disposition、臆造 key、重复行、缺行)和"同一规则内 JSON key 顺序无关",恰恰没有覆盖顺序语义。
更小的修法(只钉一个顺序元组)会退化成"把当前文件抄一遍":即使某条优先级是装饰性的,测试也照样通过。作者补上了承重证明,这是这个 PR 真正的价值。
改动思路
入口是 tests/test_loop_turn_controller_contract.py(CI),权威输入是 checked-in 的 turn_loop_controller_contract_v0.json(经生成绑定暴露为 controller._LOOP_CONTROLLER_CONTRACT),决策权仍在真实 decide_loop_disposition 手里。
两层保护:一层把 29 条 rule id 的顺序钉成元组并比对;另一层对 5 组"细化规则 / 被细化规则"做变异式证明——把细化规则移到它细化的规则之后,断言可观测决策(disposition + reason)确实改变。后者使前者不再是自证。
具体改动
1 个测试文件、+97/-0:_RULE_SEQUENCE(29 条 id)、test_contract_rule_sequence_is_pinned、_decisions_for(记录每个 cell 的 disposition 与 reason)、以及参数化的 test_a_refining_rule_must_precede_the_rule_it_refines。
关键代码讲解
_decisions_for 把"可观测决策"定义为 disposition + reason,而不是只看 disposition——理由是对的:两条规则可以给出相同 disposition 却解释不同的原因,把 reason 计入才能捕获"语义被悄悄换掉"的重排。
test_a_refining_rule_must_precede_the_rule_it_refines 用 deepcopy 复制契约、把特定规则移到一般规则之后、再对同一批 cell 重算决策并断言与 shipped 决策不同。它证明的是"这条优先级是承重的",而不是"文件里是这么写的"——正是仓库要求的"不要从被测实现反推期望"。
test_contract_rule_sequence_is_pinned 则挡住任何整体重排,包括 future 生成器/契约编辑带来的顺序漂移。
对主干的风险
最强回归是"future 契约编辑重排两条规则、Turn disposition 静默改变而所有检查仍绿"。我用真实变异验证了防线有效:在源契约 JSON 里交换 progress_exhausted 与 progress_route 并重新生成绑定后,该文件 7 failed / 143 passed,其中包含新的顺序钉子与对应的承重用例;还原两个文件后回到 150 passed。(该次变异同时改写了 JSON 的键序,因此同批失败里也包含既有的 key-order 无关用例,那是我的变异方式导致的,不是本 PR 的问题。)
测试对象是真函数:被替换的只有作为被测对象的契约对象,决策路径没有 mock。改动只碰测试,运行期风险为零;durable_smoke_value 判定:守卫的是已交付的决策前置顺序(真实不变量),现有覆盖扫描显示生成套件不覆盖顺序,同作者当日其他 PR 形状各不相同、非同形刷量,97 行薄而聚焦,无需合并或裁剪。
P3(非阻塞):承重清单覆盖作者能演示的 5 组;其他被钉住的优先级只由顺序钉子保护。若将来有意调整某条优先级,记得同时更新钉子与清单(文件里已有注释说明这一点)。
我的整体评价
baseline(无顺序保护)与 head(顺序钉住 + 承重证明)对比:新增的是一个可证伪的不变量检查,而且没有引入新 harness(复用既有 cell 构造器与 monkeypatch)。对测试型改动而言,这正是"真实、持久、不是一次性脚手架"的形态:会失败(我用变异证明了)、确定性、公开安全,并且守住的是真实契约。
结论 APPROVE,一条 P3 非阻塞。复评只需在 head 变化时重跑该文件与上面的一次契约重排变异。
English verdict: APPROVE - exact head 61f9d88; the file pins the 29-rule evaluation order that decide_loop_disposition actually depends on and proves five pinned precedences are load bearing by moving each refining rule behind the rule it refines and asserting the observable decision (disposition and reason) changes. Independently verified: 150 passed at the head, and swapping two rules in the source contract JSON plus regeneration produces 7 failures including both new tests while restoring returns the file to green. One non-blocking P3: only the demonstrated pairs are load-bearing-checked; the rest rely on the sequence pin.
Summary
rulesin order and stops at the first match, so a rule that refines another must precede it. M2 (feat(semantics): generate shared Turn contracts and controller rules (M2) #4499) madeturn_loop_controller_contract_v0.jsonthe single source of that program, but nothing pinned the sequence: the suite asserted id uniqueness, domain membership and disposition validity, and a separate test proved key order inside a rule is immaterial. A reorder during regeneration would silently change first-match semantics._RULE_SEQUENCE, and prove the pin protects real behaviour: for five refining/general pairs, moving the specific rule behind the general one must change at least one cell of the full decision matrix. Without that, the pin would only restate the shipped file.host_exhaustedbehindfailed_receiptkeepsrepairbut silently replaces "retryable host failure … exhausted its bounded attempt budget" with "turn receipt ended in host_failure". A disposition-only comparison reports no change for that pair — found while writing this test — so the reason is part of the observed decision.Issue Or Task
Validation
unitpassed(replayed)tests/test_loop_turn_controller_contract.py: sequence pin plus 5 parametrised precedence cases. Control assertion: with the shipped order every cell still matches the hand-written_ROWSexpectations. Each swap changed 3–5 cells of the 7×20 matrix. pytest is not installable in the authoring sandbox, so the cases were replayed by drivingdecide_loop_dispositiondirectly; CItest-shardis authoritative.regression_paritypassedhost_exhausted, which changes only the reason.staticpassedscripts/generate_turn_contract.py --checkandscripts/generate_semantic_bindings.py --check: up to date;py_compile.integrationpassedloopx canary premerge --from-git-diff --git-diff-base origin/main(tierstandard):ok: true, 0 failures (test-only diff selects no catalog canary).initial_terminal/initial_routeis a sixth structural refinement that the current fixtures cannot drive (noterminal_actioncell), so it is covered by the sequence pin only and named here rather than asserted.Frontend / Visual Evidence
N/A — no user-visible change.
🤖 Generated with Claude Code