Skip to content

RFC-028: Runtime user-defined language plugins (Phase A) #796

Description

@Abhishek5517

Summary

Add a runtime path for user-defined Phase A (tree-sitter) languages: a user
drops a <lang>.lang.toml plus a grammar artifact, runs travsr lang add, and
the daemon parses that language without a recompile. Full design in
docs/rfcs/RFC-028-runtime-language-plugins.md (PR incoming).

This is the feature the sealed npm binary cannot offer today: adding a language
currently means editing the Language enum, wiring registry.rs, adding a
grammar crate, and rebuilding. An npm user (who gets a prebuilt binary that never
compiles) has no path at all.

Grounded by two spikes

Before writing the RFC, both transports were prototyped end to end. Measured
results:

  • WASM (recommended default): the tree-sitter wasm feature builds
    in-workspace; a correctly built .wasm (zig, ruby with a static scanner) loads
    and parses at abi 15 in our tree-sitter 0.26 runtime. Isolation is intrinsic to
    wasmtime, so a downloaded artifact runs safely with no OS sandbox and no
    unsafe.
  • Native (opt-in only): dlopen of a .dylib compiled with the system cc
    loads, parses, and runs queries. But it is in-process with full graph access,
    it is unsafe FFI, and a grammar bug crashes the whole daemon.
  • Artifact compatibility is the real hurdle for WASM: old emscripten prebuilts
    are rejected (legacy dylink vs dylink.0, and scanner-as-import vs static).
    Producing a correct .wasm needs emscripten or Docker. A correctly built
    third-party set exists (tree-sitter-wasm, 112 grammars, dylink.0,
    cosign-signed) and loads.
  • Enum blast radius: adding a Language variant compiled the whole workspace
    after fixing one match arm, but the catch-all dispatch arms mean the compiler
    will not flag missing wiring. A conformance list is required.

Proposed decision

  • Ship WASM as the default, safe path; offer native only as an opt-in
    --trust-native power-user path with Security sign-off.
  • Registration is a validating gate (artifact, symbol, ABI, query compile),
    never trust-and-run, and never a panic.
  • Distribution: CI-built, version-pinned, cosign-signed prebuilt .wasm assets,
    plus a lang add guardrail that translates raw wasmtime errors into actionable
    messages. No catalog service.

Scope

  • In scope: Phase A structural parsing (defs, imports, references) for a
    runtime-loaded grammar.
  • Out of scope: Phase B semantic resolution for custom languages.

Open questions (see RFC section "Unresolved Questions")

  1. Native on the sealed npm binary at all, or WASM-only for end users?
  2. Binary-size and cold-start cost of compiling in wasmtime plus cranelift.
  3. Signature policy for our own and third-party artifacts.
  4. travsr lang build helper vs docs-only for artifact production.

Affected crates

travsr-core, travsr-analysis, travsr-plugin-host, travsr-indexer,
travsr-store, travsr-mcp, travsr-cli.

Metadata

Metadata

Assignees

No one assigned

    Labels

    architectureArchitectural decision or contractenhancementNew feature or requestrfcRequest for CommentssecuritySecurity gap or hardening

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions