Skip to content

test(turn-driver): pin the controller rule sequence and prove it is load bearing - #4580

Merged
huangruiteng merged 1 commit into
loopx-project:mainfrom
songoow:codex/m2-pin-controller-rule-order
Sep 16, 2026
Merged

huangruiteng merged 1 commit into
loopx-project:mainfrom
songoow:codex/m2-pin-controller-rule-order

Conversation

@songoow

@songoow songoow commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • The controller evaluates rules in order and stops at the first match, so a rule that refines another must precede it. M2 (feat(semantics): generate shared Turn contracts and controller rules (M2) #4499) made turn_loop_controller_contract_v0.json the single source of that program, but nothing pinned the sequence: the suite asserted id uniqueness, domain membership and disposition validity, and a separate test proved key order inside a rule is immaterial. A reorder during regeneration would silently change first-match semantics.
  • Pin the 29 ids as _RULE_SEQUENCE, and prove the pin protects real behaviour: for five refining/general pairs, moving the specific rule behind the general one must change at least one cell of the full decision matrix. Without that, the pin would only restate the shipped file.
  • Compare the operator-visible decision, not the disposition alone. Two rules can share a disposition and explain it differently: moving host_exhausted behind failed_receipt keeps repair but silently replaces "retryable host failure … exhausted its bounded attempt budget" with "turn receipt ended in host_failure". A disposition-only comparison reports no change for that pair — found while writing this test — so the reason is part of the observed decision.

Issue Or Task

Validation

  • Tested revision: 61f9d88
  • Run state: finished
  • Input classes: public_fixture
Check kind Result Public-safe evidence / limitation
unit passed (replayed) tests/test_loop_turn_controller_contract.py: sequence pin plus 5 parametrised precedence cases. Control assertion: with the shipped order every cell still matches the hand-written _ROWS expectations. Each swap changed 3–5 cells of the 7×20 matrix. pytest is not installable in the authoring sandbox, so the cases were replayed by driving decide_loop_disposition directly; CI test-shard is authoritative.
regression_parity passed Baseline: shipped contract order reproduces the existing expectations exactly. Mutation: each of the 5 pairs changes the decision matrix — including host_exhausted, which changes only the reason.
static passed scripts/generate_turn_contract.py --check and scripts/generate_semantic_bindings.py --check: up to date; py_compile.
integration passed loopx canary premerge --from-git-diff --git-diff-base origin/main (tier standard): ok: true, 0 failures (test-only diff selects no catalog canary).
  • Coverage and gaps: test-only change; no production code, contract data or registry value is touched. The five pinned precedences are the pairs whose shadowing is reachable from the existing fixture matrix; initial_terminal / initial_route is a sixth structural refinement that the current fixtures cannot drive (no terminal_action cell), so it is covered by the sequence pin only and named here rather than asserted.

Frontend / Visual Evidence

N/A — no user-visible change.

🤖 Generated with Claude Code

…oad bearing

The controller evaluates `rules` in order and stops at the first match, so a
rule that refines another must stay in front of it. M2 made the JSON contract
the single source of that program, but nothing pinned the sequence: the suite
asserted id uniqueness, domain membership and disposition validity, and a
separate test proved key order *inside* a rule is immaterial. Reordering two
rules during a regeneration would silently change first-match semantics.

Pin the 29 ids as `_RULE_SEQUENCE`, and prove the pin protects real behaviour:
for five refining/general pairs, moving the specific rule behind the general
one must change at least one cell of the full decision matrix.

Compare the operator-visible decision, not the disposition alone. Two rules can
share a disposition and explain it differently: moving `host_exhausted` behind
`failed_receipt` keeps `repair` but silently replaces "retryable host failure
... exhausted its bounded attempt budget" with "turn receipt ended in
host_failure". A disposition-only comparison reports no change there, so the
reason is part of the observed decision.

Refs loopx-project#4447 (M2 hardening).

Signed-off-by: song <22676124+songoow@users.noreply.github.com>

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

动机

评审 head:61f9d881bd70e21869106245d221cf918faa3393;base:main(merge-base 81f435d6b)。

交付判定(policy 6):goal_achieved。本 PR 的目标是"把 controller 规则顺序钉住,并证明该顺序是承重的",head 上这一目标成立,且我没有找到 off-goal 或碎片化的证据。

为什么值得钉:decide_loop_disposition 按顺序求值规则、命中第一条 return 即返回,因此规则顺序本身就是被交付的决策契约。M2 把规则从手写 if/elif 变成数据后,任何一次重排/重新生成都可能静默改变 disposition——而现有测试只覆盖"注册合法性"(未知 disposition、臆造 key、重复行、缺行)和"同一规则内 JSON key 顺序无关",恰恰没有覆盖顺序语义。

更小的修法(只钉一个顺序元组)会退化成"把当前文件抄一遍":即使某条优先级是装饰性的,测试也照样通过。作者补上了承重证明,这是这个 PR 真正的价值。

改动思路

入口是 tests/test_loop_turn_controller_contract.py(CI),权威输入是 checked-in 的 turn_loop_controller_contract_v0.json(经生成绑定暴露为 controller._LOOP_CONTROLLER_CONTRACT),决策权仍在真实 decide_loop_disposition 手里。

两层保护:一层把 29 条 rule id 的顺序钉成元组并比对;另一层对 5 组"细化规则 / 被细化规则"做变异式证明——把细化规则移到它细化的规则之后,断言可观测决策(disposition + reason)确实改变。后者使前者不再是自证。

具体改动

1 个测试文件、+97/-0:_RULE_SEQUENCE(29 条 id)、test_contract_rule_sequence_is_pinned、_decisions_for(记录每个 cell 的 disposition 与 reason)、以及参数化的 test_a_refining_rule_must_precede_the_rule_it_refines。

关键代码讲解

_decisions_for 把"可观测决策"定义为 disposition + reason,而不是只看 disposition——理由是对的:两条规则可以给出相同 disposition 却解释不同的原因,把 reason 计入才能捕获"语义被悄悄换掉"的重排。

test_a_refining_rule_must_precede_the_rule_it_refines 用 deepcopy 复制契约、把特定规则移到一般规则之后、再对同一批 cell 重算决策并断言与 shipped 决策不同。它证明的是"这条优先级是承重的",而不是"文件里是这么写的"——正是仓库要求的"不要从被测实现反推期望"。

test_contract_rule_sequence_is_pinned 则挡住任何整体重排,包括 future 生成器/契约编辑带来的顺序漂移。

对主干的风险

最强回归是"future 契约编辑重排两条规则、Turn disposition 静默改变而所有检查仍绿"。我用真实变异验证了防线有效:在源契约 JSON 里交换 progress_exhausted 与 progress_route 并重新生成绑定后,该文件 7 failed / 143 passed,其中包含新的顺序钉子与对应的承重用例;还原两个文件后回到 150 passed。(该次变异同时改写了 JSON 的键序,因此同批失败里也包含既有的 key-order 无关用例,那是我的变异方式导致的,不是本 PR 的问题。)

测试对象是真函数:被替换的只有作为被测对象的契约对象,决策路径没有 mock。改动只碰测试,运行期风险为零;durable_smoke_value 判定:守卫的是已交付的决策前置顺序(真实不变量),现有覆盖扫描显示生成套件不覆盖顺序,同作者当日其他 PR 形状各不相同、非同形刷量,97 行薄而聚焦,无需合并或裁剪。

P3(非阻塞):承重清单覆盖作者能演示的 5 组;其他被钉住的优先级只由顺序钉子保护。若将来有意调整某条优先级,记得同时更新钉子与清单(文件里已有注释说明这一点)。

我的整体评价

baseline(无顺序保护)与 head(顺序钉住 + 承重证明)对比:新增的是一个可证伪的不变量检查,而且没有引入新 harness(复用既有 cell 构造器与 monkeypatch)。对测试型改动而言,这正是"真实、持久、不是一次性脚手架"的形态:会失败(我用变异证明了)、确定性、公开安全,并且守住的是真实契约。

结论 APPROVE,一条 P3 非阻塞。复评只需在 head 变化时重跑该文件与上面的一次契约重排变异。

English verdict: APPROVE - exact head 61f9d88; the file pins the 29-rule evaluation order that decide_loop_disposition actually depends on and proves five pinned precedences are load bearing by moving each refining rule behind the rule it refines and asserting the observable decision (disposition and reason) changes. Independently verified: 150 passed at the head, and swapping two rules in the source contract JSON plus regeneration produces 7 failures including both new tests while restoring returns the file to green. One non-blocking P3: only the demonstrated pairs are load-bearing-checked; the rest rely on the sequence pin.

@huangruiteng
huangruiteng merged commit 3536c50 into loopx-project:main Sep 16, 2026
18 of 22 checks passed
@songoow
songoow deleted the codex/m2-pin-controller-rule-order branch September 28, 2026 03:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants