Skip to content

Merge test/node-conformance into tests/, and gate the partition that keeps the two suites apart #55

Description

@Wahbeh-Mohammad

Problem

The repository has two top-level test trees. Their names differ by one character.

  • test/node-conformance/ — 14 files named *.test.mjs. node --test runs them, through bun run test:node. They run against the built dist/. Phase 3 added this tree in commit e3ba885, to close checkpoint §5.9.
  • tests/conformance/xcut/ — files named *.conformance.test.ts. Bun runs them, inside bun run test. Phase 9 added this tree in commit d8217af. The name follows styleguide 11-testing:48 (docs/knowledge/testing.md:8).

Both trees are necessary. The split between them is also necessary. The names are the problem.

Phase 9 took the name tests/ from the styleguide. It did not compare that name against the test/ tree that already existed. No file records that the two names differ by one character. CLAUDE.md, docs/open-items.md, and 10-deliberate-deviations-from-the-reference-contract.md are all silent on it.

The same audit found two more gaps:

  • test/node-conformance/*.mjs gets only the gts format rules (eslint.config.js:38). It gets no type-aware rules. tsc does not read it at all, because the repository sets no allowJs or checkJs option. tests/**/*.ts gets strictTypeChecked and tsc -p tests/tsconfig.json --noEmit. The tree with fewer checks is the tree that tests the shipped artifact.
  • The 80% coverage floor applies only to the Bun run. test:node adds nothing to it.

Proposal

Move test/node-conformance/ to tests/node-conformance/. Then one top-level tree holds every test that crosses a process, a network, or a runtime boundary. Each subfolder holds one runner.

tests/
  conformance/xcut/     # Bun runner, part of `bun run test`
  node-conformance/     # node --test, run by `bun run test:node`

Feasibility

I built the target layout in a scratchpad. I ran the Bun version pinned in .bun-version, which is 1.3.14.

Case Setup Result
A tests/node-conformance/node.test.mjs present, no exclusion Bun collected it. Ran 3 tests across 3 files. Bun reported pass on a node:test file.
B CLI flag --path-ignore-patterns '**/node-conformance/**' Works. 2 files.
C bunfig key testPathIgnorePatterns Not a valid key. Bun ignored it and gave no warning. 3 files ran.
D bunfig key pathIgnorePatterns Works. 2 files.
E Narrowed script: bun test ./packages ./tests/conformance Works. 2 files.
F Case D, on Bun 1.4.0 Works. Same result.

Case A shows the failure that this change must prevent. Bun does not fail on a node:test import. Bun reports success on a file that tests nothing. The comment at bunfig.toml:3-8 describes this same risk.

Case C shows a second hazard. The key name testPathIgnorePatterns looks correct, but Bun does not read it. Bun gives no warning. The correct key is pathIgnorePatterns.

Use the bunfig key from case D. Do not use the CLI flag from case B. Do not use the narrowed path from case E. The bunfig key also protects a command that a person types by hand, such as bun test ./tests. Today that command is safe only because the Node files sit outside tests/.

Trade-off

Today the file system holds the partition. The Node files do not sit under tests/. Therefore no Bun command can reach them. Nothing can fail.

After this change, one config line and one glob string hold the partition. If a person changes either one incorrectly, Bun runs the Node suite, and the run reports success. For this reason, a gate must check both strings.

Work

  1. Run git mv test/node-conformance tests/node-conformance.
  2. In bunfig.toml, add pathIgnorePatterns = ["tests/node-conformance/**"] under [test]. Rewrite the comment on root = "packages". It must now describe two mechanisms, not one.
  3. In package.json:56, change the test:node glob to tests/node-conformance/*.test.mjs.
  4. In eslint.config.js:38, change the .mjs override glob to tests/node-conformance/*.mjs.
  5. In .claude/skills/ci-preflight/run-ci.mjs, change the three globs at lines 315, 319, and 323. They are in the --node-floor path.
  6. In tests/node-conformance/README.md, change the paths that name the tree itself.
  7. In CLAUDE.md, update the section on the two test trees. Add the hard rule below.
  8. Add scripts/verify-test-partition.mjs. Add it to .github/workflows/ci.yml as a blocking step.

Do not change the test steps in .github/workflows/ci.yml. They call bun run test:node, not a path.

Do not rewrite the files under docs/superpowers/plans/. They are dated phase records. Their test/node-conformance/ paths were correct on the day they were written.

Do not change tests/tsconfig.json. It sets no allowJs, so the .mjs files stay outside the typecheck. Add a comment there. The comment must state that types: ["bun"] does not apply to the node-conformance/ subtree.

The gate

scripts/verify-test-partition.mjs reads files only. It does not run either suite. It must check five things:

  1. bunfig.toml contains a [test] pathIgnorePatterns key. The script must fail if the string testPathIgnorePatterns appears anywhere. That string is case C.
  2. Every file under tests/node-conformance/ matches that glob.
  3. Every file under tests/node-conformance/ matches the test:node glob in package.json.
  4. No file under tests/conformance/ matches the ignore glob.
  5. Every glob in .claude/skills/ci-preflight/run-ci.mjs and in eslint.config.js matches at least one real file.

Add the script to bun run test:scripts as well. Note that test:scripts is still absent from CI. See docs/open-items.md H13.

Acceptance criteria

  • bun run test collects zero files from tests/node-conformance/. Check the file count. Do not check the exit code alone.
  • bun run test:node collects all 14 files, on the pinned Bun and on the Node floor.
  • node scripts/verify-test-partition.mjs fails if the bunfig key is renamed to testPathIgnorePatterns.
  • node .claude/skills/ci-preflight/run-ci.mjs --clean passes.
  • CLAUDE.md contains the hard rule below.
  • No changeset. This change is not visible to consumers.

CLAUDE.md addition

Add this text to the section on the two test trees.

HARD RULE — the tests/ partition

tests/ holds two suites. They must never run together. tests/conformance/ runs on Bun, as part of
bun run test. tests/node-conformance/ runs on node --test, through bun run test:node, against the
built dist/. It must not run on Bun. That is the only reason the tree exists.

Before Phase 10, the file system held this separation. The Node tree was at test/, and no Bun command
could reach it. Two strings now hold it instead. Those strings appear in five files, and they must agree:

File What it holds
bunfig.toml [test] pathIgnorePatterns — keeps the Node tree out of bun test
package.json the test:node glob — the only command that runs the Node tree
eslint.config.js the .mjs override — without it, console, URL, and the Web Streams globals fail no-undef
.claude/skills/ci-preflight/run-ci.mjs three globs in the --node-floor path
tests/node-conformance/README.md the membership rule, and the paths that name the tree

The key is pathIgnorePatterns. The key is not testPathIgnorePatterns. Bun accepts an unknown
[test] key without complaint. A wrong key gives no warning, does not fail, and does not stop the run. Bun
then collects the Node suite. Bun runs node:test files without an error and reports them as passing. The
run reports success over a suite that proves nothing about Node. Measured on Bun 1.3.14: the correct key
gives 2 files, the wrong key gives 3 files, and both exit with code 0.

Never change one of these five strings alone. Change one, then change all of them. Then run
node scripts/verify-test-partition.mjs. That gate catches the wrong key name, and CI blocks on it.

Do not remove the bunfig key and narrow the root script to bun test ./packages ./tests/conformance
instead. That protects the root script only. A command typed by hand, such as bun test ./tests, would
still collect the Node suite.

Keep [test] root = "packages". It controls discovery for a bare bun test, and it keeps
scripts/*.test.mjs out of the coverage floor. It is a second mechanism, and it is independent. It does
not replace the ignore glob. The ignore glob controls the explicit ./tests path that the root script
passes.

Metadata

Metadata

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions