Summary
Add a runtime path for user-defined Phase A (tree-sitter) languages: a user
drops a <lang>.lang.toml plus a grammar artifact, runs travsr lang add, and
the daemon parses that language without a recompile. Full design in
docs/rfcs/RFC-028-runtime-language-plugins.md (PR incoming).
This is the feature the sealed npm binary cannot offer today: adding a language
currently means editing the Language enum, wiring registry.rs, adding a
grammar crate, and rebuilding. An npm user (who gets a prebuilt binary that never
compiles) has no path at all.
Grounded by two spikes
Before writing the RFC, both transports were prototyped end to end. Measured
results:
- WASM (recommended default): the
tree-sitter wasm feature builds
in-workspace; a correctly built .wasm (zig, ruby with a static scanner) loads
and parses at abi 15 in our tree-sitter 0.26 runtime. Isolation is intrinsic to
wasmtime, so a downloaded artifact runs safely with no OS sandbox and no
unsafe.
- Native (opt-in only):
dlopen of a .dylib compiled with the system cc
loads, parses, and runs queries. But it is in-process with full graph access,
it is unsafe FFI, and a grammar bug crashes the whole daemon.
- Artifact compatibility is the real hurdle for WASM: old emscripten prebuilts
are rejected (legacy dylink vs dylink.0, and scanner-as-import vs static).
Producing a correct .wasm needs emscripten or Docker. A correctly built
third-party set exists (tree-sitter-wasm, 112 grammars, dylink.0,
cosign-signed) and loads.
- Enum blast radius: adding a
Language variant compiled the whole workspace
after fixing one match arm, but the catch-all dispatch arms mean the compiler
will not flag missing wiring. A conformance list is required.
Proposed decision
- Ship WASM as the default, safe path; offer native only as an opt-in
--trust-native power-user path with Security sign-off.
- Registration is a validating gate (artifact, symbol, ABI, query compile),
never trust-and-run, and never a panic.
- Distribution: CI-built, version-pinned, cosign-signed prebuilt
.wasm assets,
plus a lang add guardrail that translates raw wasmtime errors into actionable
messages. No catalog service.
Scope
- In scope: Phase A structural parsing (defs, imports, references) for a
runtime-loaded grammar.
- Out of scope: Phase B semantic resolution for custom languages.
Open questions (see RFC section "Unresolved Questions")
- Native on the sealed npm binary at all, or WASM-only for end users?
- Binary-size and cold-start cost of compiling in wasmtime plus cranelift.
- Signature policy for our own and third-party artifacts.
travsr lang build helper vs docs-only for artifact production.
Affected crates
travsr-core, travsr-analysis, travsr-plugin-host, travsr-indexer,
travsr-store, travsr-mcp, travsr-cli.
Summary
Add a runtime path for user-defined Phase A (tree-sitter) languages: a user
drops a
<lang>.lang.tomlplus a grammar artifact, runstravsr lang add, andthe daemon parses that language without a recompile. Full design in
docs/rfcs/RFC-028-runtime-language-plugins.md(PR incoming).This is the feature the sealed npm binary cannot offer today: adding a language
currently means editing the
Languageenum, wiringregistry.rs, adding agrammar crate, and rebuilding. An npm user (who gets a prebuilt binary that never
compiles) has no path at all.
Grounded by two spikes
Before writing the RFC, both transports were prototyped end to end. Measured
results:
tree-sitterwasmfeature buildsin-workspace; a correctly built
.wasm(zig, ruby with a static scanner) loadsand parses at abi 15 in our tree-sitter 0.26 runtime. Isolation is intrinsic to
wasmtime, so a downloaded artifact runs safely with no OS sandbox and no
unsafe.dlopenof a.dylibcompiled with the systemccloads, parses, and runs queries. But it is in-process with full graph access,
it is
unsafeFFI, and a grammar bug crashes the whole daemon.are rejected (legacy
dylinkvsdylink.0, and scanner-as-import vs static).Producing a correct
.wasmneeds emscripten or Docker. A correctly builtthird-party set exists (
tree-sitter-wasm, 112 grammars,dylink.0,cosign-signed) and loads.
Languagevariant compiled the whole workspaceafter fixing one match arm, but the catch-all dispatch arms mean the compiler
will not flag missing wiring. A conformance list is required.
Proposed decision
--trust-nativepower-user path with Security sign-off.never trust-and-run, and never a panic.
.wasmassets,plus a
lang addguardrail that translates raw wasmtime errors into actionablemessages. No catalog service.
Scope
runtime-loaded grammar.
Open questions (see RFC section "Unresolved Questions")
travsr lang buildhelper vs docs-only for artifact production.Affected crates
travsr-core,travsr-analysis,travsr-plugin-host,travsr-indexer,travsr-store,travsr-mcp,travsr-cli.