Skip to content

conformance: forth measured, and the RRECT/RRECTB the measuring found missing - #3093

Merged
nesbox merged 2 commits into
mainfrom
conformance-forth
Oct 16, 2026
Merged

nesbox merged 2 commits into
mainfrom
conformance-forth

Conversation

@nesbox

@nesbox nesbox commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

The conformance table watched twelve of the fourteen script languages. forth was the one left over — not a translation like the other ports but a hand table of 61 stack shapes, which is why it moved to its own change. This lands it, and fixes the one defect the measuring found.

The table itself is in the Checks run — Actions → this PR's Checks workflow → the tests job → Summary (the runner prints markdown to the step summary). forth reads 🔴 1 divergence, coverage 91/127, and the not-measured line is wasm alone now.

The port

tests/conformance/probes/forth.fth, 813 lines: one word per probe, a PROBE helper that is the forth spelling of the reference's own probe(id, f), and TIC as one line per section, in the reference's order. Each call is written in full — forth has no optional arguments, so the values lua's binding would default are pushed explicitly, and each word's stack shape is quoted from src/api/forth.c's own // name ( ... -- ... ) comment.

Why 36 of the 127 sections are out

  • 19 the reference refuses. CATCH does not catch what a binding refuses on a stack language, and the port's first task is what established it: pfPopFromStack is POP_DATA_STACK, a raw pointer read with no bounds check, and no C glue word raises a THROW — so 3 CLS PIX DROP read the two cells above the empty stack and failed the runner's balance check for the whole language. "Leave an argument out" has no forth spelling, so those sections cannot be written at all.
  • 10 with no word — the fftr and vqt readers.
  • 5 whose argument is a string or a float — the stack holds integers.
  • 2 that make no call — print() and vbank(), where a forth word always does.

The defect, and the fix

RRECT and RRECTB were not in the dictionary at all. CustomFunctionTable carries tic_forth_rrect and tic_forth_rrectb at 52 and 53, and CompileCustomFunctions() stopped registering at FFTS — so neither word existed, naming one in a cart ended the whole interpretation with THROW -13, and both wrappers were dead code. Six of the reference's rrect/rrectb sections could not be written for it.

Fixed here: two CreateGlueToC calls in the table's own order, plus the two words in ForthAPIKeywords, which the editor's highlighter reads. All three lists — table, registrations, keywords — carry 54 entries now, so the invariant the file's own comment states holds completely.

The port measures and does not fix as a rule; this one the owner called. The two commits keep the story: measured at 85/127 with the defect named, then 91/127 with it closed.

The one divergence

reset/call: lua's reset() sets core->state.initialized = false and makes no API call, while forth has a RESET word and calls api.reset. The same divergence gravity, scheme and python carry.

Checks

  • api-conformance tests/conformance/probes: exit 0, forth 🔴 1, coverage 91/127; js 🔴 25, moon/yue/fennel clean, gravity 109/127 — all unchanged.
  • The same 91 probes run one to a file, each against the reference on its own: 90 match, 1 diverges (reset/call), none unbalanced. The one-to-a-file pass is what guards the wrong-stack-order class, which records a different call rather than crashing.
  • ctest: 16/16.
  • A fresh-agent review of the table against forth.c's comments, and of the bodies against the reference's calls, ran against the diff. It found no wrong order and no wrong value (52 wrappers, 61 rows, 30+ bodies), and three things to fix, all fixed: the one-to-a-file pass could not read a divergence at all (it broke on a blank line, so its earlier result was not evidence), two claims in my own notes about the omit sets (a third no-call section, and a budget that double-counted two lines), and a latent trap in the ttri row's parameter list.

🤖 Generated with Claude Code

nesbox and others added 2 commits October 16, 2026 18:10
tests/conformance/probes/forth.fth: the last text-portable language of the
build, 85 of the reference's 127 sections, one word per probe and TIC as one
PROBE line each, in the reference's order. Generated from lua.lua by
openspec/changes/conformance-forth-probes/generate.py, whose 61-row table
carries each word's stack shape as src/api/forth.c quotes it.

The suite in the checks job's configuration, exit 0:

    forth: 1 divergence, coverage 85/127
    Not measured: wasm (no probe file)

Decisions, each measured rather than assumed (tasks 1.1, 1.2, 2.1, 2.2, 3.1):

- CATCH does not catch what a binding refuses. pfPopFromStack is
  POP_DATA_STACK, a raw pointer read, and no C glue word raises a THROW, so
  `3 CLS PIX DROP` read the cells above the empty stack and failed the
  runner's balance check for the whole language. The 19 sections whose call
  the reference refuses are omitted, and the coverage line carries them.
- The same 85 probes run one to a file: 84 match the reference call for call,
  1 diverges and none is left unbalanced — and the one is reset/call, the same
  section the whole-file run reports, so the sweep measures what the suite
  measures (sweep.py).
- The divergence is reset/call, which gravity, scheme and python carry too:
  lua's reset() sets core->state.initialized and makes no API call, while
  forth has a RESET word. A disagreement to read, not a gap in the file.
- rrect and rrectb are not in the dictionary at all — CustomFunctionTable
  carries them at 52 and 53 and CompileCustomFunctions never registers them —
  so naming either word stops the file loading. Their 8 sections are omitted
  with that reason, and the run is what found it.
- Two forms the design expected to be out are in: clip() is CLIP0, and key(),
  keyp(), sync() and btn() take the value the binding defaults to.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Measuring forth for the conformance suite found the two words unreachable:
CustomFunctionTable carries tic_forth_rrect and tic_forth_rrectb at 52 and 53,
and CompileCustomFunctions() stopped registering at FFTS, so neither word
existed and naming one in a cart ended the whole interpretation with
THROW -13. Six of the reference's rrect/rrectb sections could not be written
for it, and are written now — the probe file's coverage goes 85/127 to
91/127, its one divergence unchanged:

    forth: 1 divergence, coverage 91/127
    Not measured: wasm (no probe file)

The two CreateGlueToC calls close the gap in the table's own order, C_RETURNS_
VOID like the wrappers they name. RRECT and RRECTB join ForthAPIKeywords with
them, which the editor's highlighter reads.

The sweep agrees one probe at a time: 91 probes, 90 match the reference,
1 diverges (reset/call), none unbalanced.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@nesbox
nesbox merged commit 692e7b8 into main Oct 16, 2026
16 checks passed
@nesbox
nesbox deleted the conformance-forth branch October 16, 2026 16:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant