Skip to content

Add load-time bytecode validation pass - #23

Merged
vbergeron merged 1 commit into
mainfrom
claude/relaxed-pasteur-ljcdsf
Sep 23, 2026
Merged

vbergeron merged 1 commit into
mainfrom
claude/relaxed-pasteur-ljcdsf

Conversation

@vbergeron

@vbergeron vbergeron commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

Closes #17. Adds a load-time validation pass (Program::validate). The VM runs it before executing any code, so the interpreter can read the code stream without bounds checks: every operand and jump target has already been checked.

Key Changes

  • New validate.rs module: a single-pass, no_std, allocation-free validator that checks:

    • All opcodes are known and their operands fit within the code (including GLOBAL_W and the new integer ops from Add integer overflow detection to VM and compiler #21/Add division, bitwise, shift ops and support >256 globals #22)
    • The last instruction never falls through (it must be FIN, ENCORE, MATCH, or BRANCH)
    • All static jump targets (MATCH tables, BRANCH, CLOSURE/FUNCTION code pointers, global entry points) are inside the code and land on instruction boundaries
    • PACK/UNPACK tags are in the arity table, and UNPACK doesn't write past the register file
    • GLOBAL/GLOBAL_W indices are < n_globals
    • EXTERN slots are < 32
    • Code length is ≤ 0xFFFF (code pointers are 16-bit)
  • Program: parse() still checks only the header, so the disassembler can open malformed files. Added a validate() method.

  • Vm:

    • load() calls prog.validate() before carving globals or running anything.
    • resolve_code_ptr() range-checks dynamic call targets at run time, which catches calls through the NULL continuation.
    • call_global_raw() returns an error for an out-of-range global index instead of panicking.
  • Code: now crate-private. Code::new() is unsafe and requires validated bytecode, Code::empty() covers an uninitialized VM, and a SAFETY comment documents the invariant.

  • Errors: new VmError::Invalid { pc, reason } variant.

  • Docs: VM.md gains a "Load-time validation" section, including the remaining trust assumption: values are not type-checked at run time.

  • Tests:

    • validate_tests.rs: accepted and rejected programs for each rule, plus deterministic fuzzing of parse + validate (random bytes, random code, mutated programs). The fuzz tests don't run code, so they don't cover full Vm::load. Miri reports no problems on these tests.
    • validate_examples.rs: every example validates, with and without the optimizer.

Implementation Details

Boundary checks avoid an 8 KB bitset, which would be a lot of stack on firmware. Jump targets go into a fixed 128-entry buffer. When it fills, it's sorted and checked against one walk over the instruction starts.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WELBLatovBrmXgkbHCMBgR

The interpreter reads the code stream with get_unchecked, so a truncated
stream, a bad jump target or an out-of-range operand in a .encr file
caused undefined behaviour or a panic.

Vm::load now runs Program::validate before carving globals or running
anything. It is a no_std, allocation-free pass that returns
VmError::Invalid { pc, reason } (or InvalidOpcode) when:
- an opcode is unknown or its operands run past the code;
- the last instruction can fall through past the end;
- a MATCH/BRANCH/CLOSURE/FUNCTION target or global entry point is outside
  the code or not on an instruction boundary (targets are batched, sorted
  and merged against a walk of the instruction starts);
- a PACK/UNPACK tag is outside the arity table, or UNPACK writes past the
  register file;
- a GLOBAL/GLOBAL_W index is >= n_globals;
- an EXTERN slot is >= MAX_EXTERN.

Program::parse still checks only the header, so the disassembler can open
broken files.

ENCORE range-checks dynamic call targets, which catches calls to the
NULL continuation (0xFFFF). call_global_raw rejects out-of-range global
indices instead of panicking. Code is now crate-private, Code::new is
unsafe, and a SAFETY comment states the invariant. VM.md documents the
rules and the remaining trust assumption: values are not type-checked at
run time.

Tests: rule-by-rule cases, deterministic fuzzing of parse + validate
(random bytes, random code, mutated programs), and a check that every
example compiles to code that validates, with and without the optimizer.

Closes #17

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WELBLatovBrmXgkbHCMBgR
@vbergeron
vbergeron force-pushed the claude/relaxed-pasteur-ljcdsf branch from 2aac56f to 4b88557 Compare September 23, 2026 20:36
@vbergeron
vbergeron merged commit ba55ebb into main Sep 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VM: validate bytecode at load time so malformed .encr cannot cause UB or panics

2 participants