Formalize AST format - #11
Merged
Merged
Conversation
…hema validation - meta.schema.json now requires 'grammar', 'stages', 'exit' properties - Create test/validate_schemas.py — validates config.json and all meta.json files against their schemas, with clear error messages - Add schema validation step to both CI workflows (ci.yml, tests.yml) before running tests, ensuring invalid meta.json is caught early
Add tokens.schema.json to validate test tokens.json files. Mandatory fields: kind (string), line (integer >=1), column (integer >=1). Optional field: value (string).
The validate_schemas.py script now also loads tokens.schema.json and validates every tokens.json found under test/. The schema requires: - kind (string), line (integer >=1), column (integer >=1) — mandatory - value (string) — optional This runs in CI as part of the 'Validate JSON schemas' step in both ci.yml and tests.yml workflows.
- Create ast.schema.json for AST tree validation: each node requires line, column, kind; allows any other fields (value, elems, body, …). - Update tokens.schema.json: remove 'value' from properties (no longer tracked in schema), remove 'additionalProperties: false' so any token fields are accepted. Only 'kind', 'line', 'column' remain required.
Each AST node now has: - elems: ordered list of child nodes (previously in named fields like body, left, right, cond, then, else, init, etc.) - value: scalar payload (name, op, message, field, type, literal value) or null for nodes without one - Extra fields (mut, type, returnType, etc.) preserved
All ast.json golden files regenerated using the fixed compiler that: - Removes AST_EXPR_STMT wrapper from expression statements - Produces correct elems arrays per documented format (Declare: [name, init], FuncDecl: [name, params..., body], ExternDecl: [name, params...], StructDecl: [name, fields...], Param: [name], FieldAccess: [base, name]) - Reports correct line/column positions (first token of each node) - Drops the value field (not compared during golden validation)
Verify that top-level declarations (extern, def, struct) can appear in any order at the top level: - grammar3: def and extern interleaved - grammar4: def and extern with typed params/return types - grammar5: struct, extern, and def all interleaved
The run_tests.py output format recently changed: - list command now groups tests under === STAGE === headers with indented names (no stage/ prefix) - test/update commands now group results under === STAGE === headers and print names without stage prefix - config stage key is 'compiler' not 'compile' Also fix test_compiler_update_regenerates_and_exit_warning to produce non-empty stdout so the golden file is written. All 54 self-tests now pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tokens.jsonandast.json