Repository navigation
conformance: forth measured, and the RRECT/RRECTB the measuring found missing - #3093
Merged
Merged
Conversation
tests/conformance/probes/forth.fth: the last text-portable language of the
build, 85 of the reference's 127 sections, one word per probe and TIC as one
PROBE line each, in the reference's order. Generated from lua.lua by
openspec/changes/conformance-forth-probes/generate.py, whose 61-row table
carries each word's stack shape as src/api/forth.c quotes it.
The suite in the checks job's configuration, exit 0:
forth: 1 divergence, coverage 85/127
Not measured: wasm (no probe file)
Decisions, each measured rather than assumed (tasks 1.1, 1.2, 2.1, 2.2, 3.1):
- CATCH does not catch what a binding refuses. pfPopFromStack is
POP_DATA_STACK, a raw pointer read, and no C glue word raises a THROW, so
`3 CLS PIX DROP` read the cells above the empty stack and failed the
runner's balance check for the whole language. The 19 sections whose call
the reference refuses are omitted, and the coverage line carries them.
- The same 85 probes run one to a file: 84 match the reference call for call,
1 diverges and none is left unbalanced — and the one is reset/call, the same
section the whole-file run reports, so the sweep measures what the suite
measures (sweep.py).
- The divergence is reset/call, which gravity, scheme and python carry too:
lua's reset() sets core->state.initialized and makes no API call, while
forth has a RESET word. A disagreement to read, not a gap in the file.
- rrect and rrectb are not in the dictionary at all — CustomFunctionTable
carries them at 52 and 53 and CompileCustomFunctions never registers them —
so naming either word stops the file loading. Their 8 sections are omitted
with that reason, and the run is what found it.
- Two forms the design expected to be out are in: clip() is CLIP0, and key(),
keyp(), sync() and btn() take the value the binding defaults to.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Measuring forth for the conformance suite found the two words unreachable:
CustomFunctionTable carries tic_forth_rrect and tic_forth_rrectb at 52 and 53,
and CompileCustomFunctions() stopped registering at FFTS, so neither word
existed and naming one in a cart ended the whole interpretation with
THROW -13. Six of the reference's rrect/rrectb sections could not be written
for it, and are written now — the probe file's coverage goes 85/127 to
91/127, its one divergence unchanged:
forth: 1 divergence, coverage 91/127
Not measured: wasm (no probe file)
The two CreateGlueToC calls close the gap in the table's own order, C_RETURNS_
VOID like the wrappers they name. RRECT and RRECTB join ForthAPIKeywords with
them, which the editor's highlighter reads.
The sweep agrees one probe at a time: 91 probes, 90 match the reference,
1 diverges (reset/call), none unbalanced.
Co-Authored-By: Claude Code <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The conformance table watched twelve of the fourteen script languages.
forthwas the one left over — not a translation like the other ports but a hand table of 61 stack shapes, which is why it moved to its own change. This lands it, and fixes the one defect the measuring found.The table itself is in the Checks run — Actions → this PR's Checks workflow → the
testsjob → Summary (the runner prints markdown to the step summary).forthreads 🔴 1 divergence, coverage 91/127, and the not-measured line iswasmalone now.The port
tests/conformance/probes/forth.fth, 813 lines: one word per probe, aPROBEhelper that is the forth spelling of the reference's ownprobe(id, f), andTICas one line per section, in the reference's order. Each call is written in full — forth has no optional arguments, so the values lua's binding would default are pushed explicitly, and each word's stack shape is quoted fromsrc/api/forth.c's own// name ( ... -- ... )comment.Why 36 of the 127 sections are out
CATCHdoes not catch what a binding refuses on a stack language, and the port's first task is what established it:pfPopFromStackisPOP_DATA_STACK, a raw pointer read with no bounds check, and no C glue word raises a THROW — so3 CLS PIX DROPread the two cells above the empty stack and failed the runner's balance check for the whole language. "Leave an argument out" has no forth spelling, so those sections cannot be written at all.fftrandvqtreaders.print()andvbank(), where a forth word always does.The defect, and the fix
RRECTandRRECTBwere not in the dictionary at all.CustomFunctionTablecarriestic_forth_rrectandtic_forth_rrectbat 52 and 53, andCompileCustomFunctions()stopped registering atFFTS— so neither word existed, naming one in a cart ended the whole interpretation withTHROW -13, and both wrappers were dead code. Six of the reference'srrect/rrectbsections could not be written for it.Fixed here: two
CreateGlueToCcalls in the table's own order, plus the two words inForthAPIKeywords, which the editor's highlighter reads. All three lists — table, registrations, keywords — carry 54 entries now, so the invariant the file's own comment states holds completely.The port measures and does not fix as a rule; this one the owner called. The two commits keep the story: measured at 85/127 with the defect named, then 91/127 with it closed.
The one divergence
reset/call: lua'sreset()setscore->state.initialized = falseand makes no API call, while forth has aRESETword and callsapi.reset. The same divergencegravity,schemeandpythoncarry.Checks
api-conformance tests/conformance/probes: exit 0,forth🔴 1, coverage 91/127;js🔴 25,moon/yue/fennelclean,gravity109/127 — all unchanged.reset/call), none unbalanced. The one-to-a-file pass is what guards the wrong-stack-order class, which records a different call rather than crashing.ctest: 16/16.forth.c's comments, and of the bodies against the reference's calls, ran against the diff. It found no wrong order and no wrong value (52 wrappers, 61 rows, 30+ bodies), and three things to fix, all fixed: the one-to-a-file pass could not read a divergence at all (it broke on a blank line, so its earlier result was not evidence), two claims in my own notes about the omit sets (a third no-call section, and a budget that double-counted two lines), and a latent trap in thettrirow's parameter list.🤖 Generated with Claude Code