✨ feat: TypeScript SCIP indexing for find_impact + call-graph - #167
Merged
Conversation
added 10 commits
July 27, 2026 11:12
Add scip + protobuf crates and src/symbols/scip_proto.rs, parsing standard SCIP protobuf (.scip) files emitted by Sourcegraph indexers (e.g. scip-typescript) into the same ScipIndex shape the C# JSON parser produces, so downstream storage/resolution code is reusable. - parse_scip_protobuf(): iterates documents/occurrences, skips empty symbols and malformed ranges - decode_range(): SCIP compact range (3-elem single-line / 4-elem multi-line, 0-based) -> 1-based (start_line, end_line) - role_to_kind(): maps standard SCIP SymbolRole bitmask (distinct from the C# helper's custom JSON role encoding) to definition/ import/write/call/reference 7 unit tests cover round-trip parsing (1 def + 3 calls across 2 files), range decoding edge cases, role priority, and malformed input handling. cargo clippy -D warnings clean. Part of TypeScript SCIP indexing (stage 1/6, MVP plan in PLAN_TYPESCRIPT_SCIP.md).
- Add TypeScriptSymbolIndexer (src/symbols/typescript.rs) implementing the SymbolIndexer trait, mirroring csharp.rs but simplified for the single-pass SCIP protobuf model (no lazy ref resolution, no ref cache table - scip-typescript emits defs+refs in one pass). - RebuildScope::Files falls back to Full for TS (scip-typescript has no file filter) - documented decision. - LMDB table-sharing-with-C#-if-same-db_path documented as an MVP limitation in a rebuild() comment. - Register TypeScriptSymbolIndexer in SymbolIndexerRegistry::new(). - Add LANG_TYPESCRIPT, SCIP_TYPESCRIPT_HELPER_ENV, SCIP_TYPESCRIPT_REBUILD_TIMESTAMP_KEY constants. - Remove stage-1 #![allow(dead_code)] from scip_proto.rs now that parse_scip_protobuf is wired in. - 6 new unit tests, all passing.
Map ts/tsx/mts/cts file extensions to LANG_TYPESCRIPT in find_impact's language auto-detect logic, mirroring the existing cs -> LANG_CSHARP mapping. Update the find_impact tool description (doc comment + MCP description string) and the no-indexer-installed message to mention TypeScript/scip-typescript alongside C#/scip-csharp.
Add a parallel .ts/.tsx/.mts/.cts file-tracking branch in start_file_watcher (src/index/manager.rs), mirroring the existing hardcoded C# dispatch (Option B design decision from PLAN_TYPESCRIPT_SCIP.md $8: a parallel branch, not a generic registry loop). - New is_ts_extension() helper checks ts/tsx/mts/cts extensions. - Modified/Deleted/Renamed events now also populate ts_files_modified / ts_files_deleted / ts_last_event_time, cleared on branch-change refresh alongside the existing cs_* state. - New debounce-flush block (SCIP_TYPESCRIPT_DEBOUNCE_MS, new constant mirroring SCIP_CSHARP_DEBOUNCE_MS = 60s) dispatches to registry.get(LANG_TYPESCRIPT). Unlike C#, there is no per-.csproj grouping (TypeScript MVP only supports a single root tsconfig.json), so any tracked change triggers one full rebuild (RebuildScope::Full) directly instead of RebuildScope::Files -- this is more honest than passing Files, since TypeScriptSymbolIndexer::rebuild() falls back to Full internally anyway. - No CSharpRebuildNotifier equivalent is threaded through for TS (that type is C#-specific); the TUI indexing-active callback (indexing_cb) is still signaled around the rebuild. Validation: cargo clippy --all-targets -D warnings clean; cargo test --lib --bins: 1214 passed, 36 ignored.
- New tests/fixtures/ts-sample/: root tsconfig.json + src/math.ts (1
definition: `add`) + src/consumer.ts + src/other.ts (3 call-sites of
`add` across 2 files), mirroring the C# SmallSolution fixture shape.
- New tests/symbols_typescript_test.rs mirroring symbols_csharp_test.rs:
- test_indexer_returns_empty_when_db_missing: LMDB empty-DB path never
panics, returns Ok(empty) or a clean Err.
- test_applies_to_requires_root_tsconfig: applies_to() gating on a
root tsconfig.json.
- test_fixture_directory_shape: sanity-checks the fixture's shape used
by the gated integration test.
- test_typescript_pipeline_ts_sample_roundtrip (gated behind new
`typescript_helper_integration` feature, requires npx/scip-typescript
or CODESEARCH_SCIP_TYPESCRIPT): full pipeline round-trip — rebuild()
on the fixture, then find_references("add") asserts exactly 1
definition in math.ts and >=3 call-sites spanning consumer.ts +
other.ts. This is the acceptance test for find_impact on a TS symbol
returning all call-sites, per PLAN_TYPESCRIPT_SCIP.md §9.
- Cargo.toml: new `typescript_helper_integration` feature flag, mirroring
the existing `csharp_helper_integration` flag.
Validated: cargo clippy --all-targets -D warnings clean; cargo test
--test symbols_typescript_test -> 3 passed, 1 ignored (gated test
correctly skipped without scip-typescript); cargo test --lib --bins ->
1214 passed, 36 ignored (no regression).
This is the final stage (6/6) of the TypeScript SCIP indexing MVP.
Final cross-stage review (Phase 4) found the TypeScript SCIP pipeline
non-functional: Command::new("npx") is never resolvable on Windows
because std::process::Command does not consult PATHEXT the way cmd.exe
does (npx only exists as npx.cmd/npx.ps1). Additionally the unscoped npm
name "scip-typescript" is a squatted security placeholder with no
functionality; the real Sourcegraph package is the scoped package
@sourcegraph/scip-typescript (bin name scip-typescript).
Fix: route the npx invocation through "cmd /C" on Windows, and invoke
npx -y @sourcegraph/scip-typescript instead of the bare unscoped name.
Verified: the previously-ignored gated integration test
(test_typescript_pipeline_ts_sample_roundtrip, --features
typescript_helper_integration) now passes end-to-end: 1 definition +
3 call-sites across 2 files, confirming find_impact on a TS symbol
returns all call-sites as required by the acceptance criterion.
cargo clippy --all-targets -- -D warnings: clean.
cargo test --lib --bins: 605 passed, 0 failed, 18 ignored.
Final review flagged fuzzy_symbol_match/open_scip_env duplication between csharp.rs and typescript.rs as an Important, non-blocking finding. Tracking as T5 in the Open TODOs backlog rather than refactoring stable, already-tested csharp.rs at the tail end of this branch — matches the reviewer's own accepted resolution path.
…eline Opt-in via CODESEARCH_TS_TEST_REAL env var + typescript_helper_integration feature flag. Validates the full pipeline (rebuild + find_references) on a non-trivial real-world TS codebase. Never runs in normal CI.
Add per-repo TypeScript index status to the TUI and /status JSON: - RepoRow + RepoStatusInfo gain a typescript_index field - Alias column shows ' TS·' / ' TS!' / ' TS…' alongside the C# indicator - Footer shows TS helper availability (green/dark-gray) next to C# - /status JSON emits typescript_index per repo + ts_helper flag - Remote TUI deserializes the new fields (serde default for backward compat) TS status is probed directly (helper available + index dir exists → Ready) since there is no live status cache populated during TS rebuilds yet; C# status_cell embedding is left C#-only — the alias column is the canonical multi-language indicator.
…-scip-indexing # Conflicts: # AGENTS.md # src/mcp/mod.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds TypeScript SCIP indexing so
find_impactand the symbol/call-graph tooling work for.ts/.tsx/.mts/.ctsexactly as they already do for C#. The pipeline mirrors the C# adapter but uses Sourcegraph'sscip-typescriptnpm CLI and a single-pass defs+refs model (vs C#'s two-phase lazy resolution).This closes the open TODO
T5/PLAN_TYPESCRIPT_SCIP.md(MVP scope: stages 1–6).Changes
Core indexer
src/symbols/typescript.rs(new, ~780 lines) —TypeScriptSymbolIndexerimplementing theSymbolIndexertrait:tsconfig.jsonentrypoint detection,npx scip-typescript indexinvocation, single-pass LMDB write of defs+refs,find_referencesvia pure LMDB read (no subprocess), fuzzy symbol matching.src/symbols/scip_proto.rs(new, ~280 lines) — standard SCIP protobuf decoder (documents → occurrences → roles). Used by the TS adapter; C# keeps its custom JSON path.src/symbols/mod.rs— registry now registersTypeScriptSymbolIndexeralongsideCSharpSymbolIndexer.Dispatch / wiring (generalized off hardcoded C#)
src/mcp/mod.rs—find_impactroutes.ts/.tsx/.mts/.ctsto the TS indexer.src/index/manager.rs— file-watcher tracks TypeScript file changes (modified/deleted/rename) and triggers a debounced TS rebuild.src/serve/mod.rs— Phase-3 pre-warm loop iterates all registry languages instead of hardcoding C#.src/serve/tui*.rs— TUI shows the TS symbol-index indicator alongside the C# one.Config & deps
Cargo.toml/Cargo.lock— addprost(protobuf decode) +bytes.src/constants.rs— TS-related constants (helper override env var, extensions).Tests & fixtures
tests/symbols_typescript_test.rs(new) +tests/fixtures/ts-sample/— unit tests for protobuf decode, LMDB round-trip, fuzzy matching, and a real-project gated smoke test (requiresnpx/scip-typescripton PATH; auto-skipped otherwise).PLAN_TYPESCRIPT_SCIP.md— the 8-stage implementation plan (kept for reference).AGENTS.md— tracks the SCIP adapter dedup (fuzzy-match + LMDB-env boilerplate copied between C# and TS) as follow-up TODO.Key design decisions
rebuild()) — scip-typescript emits both in its.scipoutput, so no lazy on-demand resolution is needed.find_referencesis a pure LMDB read with no subprocess.npx scip-typescriptruntime — no binary shipped in the release bundle;npxresolves it on the host. The indexer reportsis_available() = falsewhennpxis absent so MCP degrades gracefully.scip_proto.rs) is TS-only for now; deliberately did not refactorcsharp.rsto use it (flagged as follow-up TODO T5 to avoid touching stable, already-tested code at the tail of a large feature).Testing
cargo fmt --check— passcargo check --all-targets— passcargo clippy -D warnings— passcargo test --lib— 607 passed, 18 ignored, 2 pre-existing flaky Windows env tests fail under parallel load (both pass in isolation; unrelated to this branch — filessrc/db_discovery/repos.rs&src/watch/mod.rsare untouched here)npx scip-typescriptis availableChecklist
develop(repo gitflow)