diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0bad5a2..64222f4 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -38,7 +38,7 @@ jobs: echo "GROVE_TEST_TYPESCRIPT_ROOT=$RUNNER_TEMP/ts" >> "$GITHUB_ENV" echo "GROVE_TEST_REQUIRE_TS=1" >> "$GITHUB_ENV" - run: go vet ./... - - run: go test ./... -count=1 -short -timeout 120s + - run: go test ./... -count=1 -short -timeout 300s - run: go build ./... verify-self: diff --git a/CHANGELOG.md b/CHANGELOG.md index a2a7231..06f34e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,52 @@ # Changelog +## v0.63.25 - 2026-10-21 + +Index correctness fixes from the 2026-10-20 stack audit. + +**Fixes (results)** +- **Unreadable changed files are pruned**: a file whose stat changed but + whose content can no longer be hashed (over the 10 MiB cap, not a regular + file, unreadable) kept the symbols of its last readable version, served as + current. Its stored symbols are now dropped, as for a changed file that + fails to parse. The error stays in `IndexResult.Errors` and is reported on + every run until the file indexes. Unreadable directories are unchanged: + their files are not seen, so their stored state is kept. +- **Compiler inputs are fingerprinted per analyzer**: editing only build + configuration (`go.mod`/`go.sum`/`go.work`, `tsconfig*.json`, + `jsconfig*.json`, `package.json`, Maven/Gradle files, `Cargo.toml`/ + `Cargo.lock`, `compile_commands.json`, `*.csproj`/`Directory.Build.props`, + `composer.json`, Python project files) used to leave the graph resolved + under the old configuration. These files are now content-hashed (even when + gitignored); a change gives the analyzer that reads them a full pass and + drops only that analyzer's resident worker state. A touch without a content + change is not a change. +- **cgo files in project packages**: a local Go package whose files all + import `"C"` loaded as an empty package. cgo files are now parsed and + type-checked with the package (`FakeImportC`), keep positions in the + user's source, and no longer mark the package as partially checked. +- **Bounded retries for incomplete native analysis**: a no-change index + returned before giving an analyzer without a completed pass another run, + so installing a missing compiler had no effect until `--force`. An + unavailable analyzer is now rechecked after 15s, doubling to at most + 10 min; a pass that ran and failed is retried after 1 min, doubling to at + most 6 h. The no-change message says which analyzer is pending, why, and + when the next retry is due. +- **One current native verdict per analyzer**: carried-forward verdict lines + were re-carried on every run, so one diagnostic grew by two lines per + index. Each analyzer's last pass that produced edges is now kept in its + own meta key; the stored verdict is that last good pass once plus this + run's lines. Line formats are unchanged. + +**Dependencies** +- astkit v0.16.10: call columns count characters when multibyte text + precedes a call, `Engine.Validate` rejects syntax errors, and `.mts`/`.cts` + are detected as TypeScript. Call-edge scores on gin, cobra, socket.io and + flask are identical with it. + +Entries for v0.62.3 through v0.63.24 are not written up here; see the git +history for those releases. + ## v0.62.2 - 2026-10-05 Python receivers that hold objects built elsewhere now resolve. Flask's diff --git a/README.md b/README.md index 91d6477..7534cb5 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ Grove is the graph engine embedded by [Prism](https://github.com/provasign/prism ## What it provides -- Tree-sitter extraction across sixteen languages, with native semantic enrichment when the local toolchain is available +- Tree-sitter extraction across fourteen languages (the table under Language support), with native semantic enrichment when the local toolchain is available - Type-resolved calls, imports, inheritance, implementation, type-use, and test relationships - Incremental indexing by content hash - Deterministic change-impact, rename, missing-implementation, and dead-code operations @@ -25,7 +25,7 @@ curl -fsSL https://raw.githubusercontent.com/provasign/grove/main/install.sh | b irm https://raw.githubusercontent.com/provasign/grove/main/install.ps1 | iex # Pin the current release -VERSION=v0.45.0 curl -fsSL https://raw.githubusercontent.com/provasign/grove/main/install.sh | bash +VERSION=v0.63.25 curl -fsSL https://raw.githubusercontent.com/provasign/grove/main/install.sh | bash ``` The installer writes to `~/bin` by default. Set `INSTALL_DIR` to choose another location. @@ -151,19 +151,19 @@ The package also exposes snapshots and structural diffs for integrations that ne | Language | Extensions | |---|---| | Go | `.go` | -| TypeScript / TSX | `.ts`, `.tsx` | +| TypeScript / TSX | `.ts`, `.mts`, `.cts`, `.tsx` | | JavaScript / JSX | `.js`, `.jsx`, `.mjs`, `.cjs` | | Python | `.py` | | Java | `.java` | | Rust | `.rs` | -| C / C++ | `.c`, `.h`, `.cc`, `.cpp`, `.cxx`, `.hh`, `.hpp` | +| C / C++ | `.c`, `.h`, `.cc`, `.cpp`, `.cxx`, `.c++`, `.hh`, `.hpp`, `.hxx` | | C# | `.cs` | -| PHP | `.php`, `.phtml` | +| PHP | `.php`, `.php3`, `.php4`, `.php5`, `.phtml` | | Swift | `.swift` | | Kotlin | `.kt`, `.kts` | | Objective-C | `.m`, `.mm`, `.h` (content-sniffed) | -| COBOL | `.cbl`, `.cob`, `.cpy` and copybook variants | -| JCL | `.jcl`, `.prc` | +| COBOL | `.cbl`, `.cob`, `.cobol`; copybooks `.cpy`, `.ccp`, `.cpb`, `.copy` (any case) | +| JCL | `.jcl`, `.prc` (any case) | Common non-code files are indexed as document symbols for lexical retrieval. Native analyzers can enrich the graph when the relevant language tooling is available. `grove doctor` reports each language's resolution tier and known limitations. @@ -177,36 +177,41 @@ go test ./internal/parser/... -v ``` Current call-edge accuracy against each language's compiler or runtime oracle -(pinned corpora, CI-gated floors in `eval/baseline.json`; 2026-09-19): +(pinned corpora, CI-gated floors in `eval/baseline.json`; 2026-10-22, Grove v0.63.25): | Language | Corpus | Oracle | Precision | Recall | |---|---|---|---|---| -| Go | gin | SSA + VTA | 0.952 | 0.951 | -| Java | commons-lang | javac + javap | 0.938 | 0.952 | -| C# | Newtonsoft.Json | Roslyn | 0.935 | 0.947 | -| TypeScript | socket.io | TypeScript checker | 0.903 | 0.992 | -| JavaScript | express | TypeScript checker (`checkJs`) | 0.840 | 1.000 | -| Rust | ripgrep | rust-analyzer SCIP | 0.936 | 0.905 | -| C | jansson | clang AST | 0.999 | 0.984 | -| Swift | SwiftyJSON | SourceKit index | 0.936 | 1.000 | -| Kotlin | turtle | kotlinc + javap | 1.000 | 0.988 | -| Objective-C | SBJson | clang AST | 1.000 | 0.991 | -| Python | flask | pytest trace (dynamic) | 0.852 | 0.716 | -| PHP | PHP-Parser | Xdebug trace (dynamic) | 0.918 | 0.647 | +| Go | gin | SSA + VTA | 0.979 | 0.997 | +| Java | commons-lang | javac + javap | 1.000 | 1.000 | +| C# | Newtonsoft.Json | Roslyn | 0.971 | 0.977 | +| TypeScript | socket.io | TypeScript checker | 0.937 | 0.992 | +| JavaScript | express | TypeScript checker (`checkJs`) | 0.885 | 1.000 | +| Rust | ripgrep | rust-analyzer SCIP | 0.965 | 0.906 | +| C | jansson | clang AST | 1.000 | 0.998 | +| Swift | SwiftyJSON | SourceKit index | 1.000 | 1.000 | +| Kotlin | turtle | kotlinc + javap | 1.000 | 1.000 | +| Objective-C | SBJson (json-framework) | clang AST | 1.000 | 1.000 | +| Python | flask | pytest trace (dynamic) | 0.896 | 0.787 | +| PHP | PHP-Parser | Xdebug trace (dynamic) | 0.922 | 0.683 | A second, untuned corpus per language guards against rules that only fit the first (also gated): | Language | Second corpus | Precision | Recall | |---|---|---|---| -| Go | cobra | 0.983 | 0.952 | -| Java | commons-io | 0.893 | 0.912 | +| Go | cobra | 0.995 | 1.000 | +| Java | commons-io | 1.000 | 1.000 | | C | cJSON | 0.998 | 0.999 | | Rust | fd | 0.952 | 0.913 | | TypeScript | p-queue | 0.959 | 1.000 | -| Swift | Files | 0.988 | 0.922 | -| Objective-C | CocoaLumberjack | 0.956 | 0.799 (property reads) | -| PHP | league/csv (dynamic) | 0.889 | 0.702 | +| Swift | Files | 1.000 | 1.000 | +| Objective-C | CocoaLumberjack | 0.975 | 0.957 | +| PHP | league/csv (dynamic) | 0.947 | 0.837 | + +C#, JavaScript, Kotlin and Python have one scored corpus each. + +COBOL and JCL are supported with a regression-tested graph; their accuracy is +not yet independently scored. The dynamic oracles record only paths the test suites execute and count reflection-driven dispatch a static graph cannot name, so their recall is a diff --git a/eval/README.md b/eval/README.md index 2b30a57..98f816e 100644 --- a/eval/README.md +++ b/eval/README.md @@ -45,27 +45,32 @@ capped false-positive/false-negative examples for debugging (`GROVE_EVAL_MAX_EXAMPLES=100000` lists every one; `GROVE_TRACE_CALLS=1` traces candidate narrowing per call site to stderr). -## Current scores (2026-10-10, grove v0.63.0, calls edges) +## Current scores (2026-10-22, grove v0.63.25, calls edges) | Corpus (pin) | Language | Oracle (`--lang`) | Universe | P | R | F1 | |---|---|---|---|---|---|---| -| gin (`d75fcd4`) | Go | SSA + VTA (default) | 99.7% | 0.9599 | 0.9735 | 0.9667 | -| commons-lang (`44298fe`) | Java | javac + javap (`java`) | 97.0% | 0.9631 | 0.9807 | 0.9718 | -| newtonsoft (`0a2e291`) | C# | Roslyn (`csharp`) | 99.7% | 0.9350 | 0.9467 | 0.9408 | -| socket.io (`3ad4e1f2`) | TypeScript | tsc checker (`tstruth/gen_truth.mjs`) | 98.3% | 0.9156 | 0.9917 | 0.9521 | -| express (`dae209ae`) | JavaScript | tsc `checkJs` (`tstruth/gen_truth.mjs`) | 90.3% | 0.8400 | 1.0000 | 0.9130 | -| ripgrep (`82313cf`) | Rust | rust-analyzer SCIP (`rust`) | 100% | 0.9366 | 0.9047 | 0.9204 | -| jansson (`684e18c`) | C | clang AST (`clang`) | 97.7% | 0.9991 | 0.9837 | 0.9913 | -| SwiftyJSON (`3d25441`) | Swift | SourceKit index (`swift`) | 100% | 0.9355 | 1.0000 | 0.9667 | -| turtle (`3cfc963`) | Kotlin | kotlinc + javap (`kotlin`) | 76.8% | 1.0000 | 0.9881 | 0.9940 | -| json-framework (`93e4ca5`) | Objective-C | clang AST (`objc`) | 98.7% | 1.0000 | 0.9914 | 0.9957 | -| flask (`36e4a824`) | Python | pytest trace (`pytruth`, dynamic) | 97.9% | 0.8544 | 0.7288 | 0.7866 | -| php-parser (`8eea230`) | PHP | Xdebug trace (`php`, dynamic) | 100% | 0.9177 | 0.6487 | 0.7601 | +| gin (`d75fcd4`) | Go | SSA + VTA (default) | 99.7% | 0.9792 | 0.9965 | 0.9878 | +| commons-lang (`44298fe`) | Java | javac + javap (`java`) | 99.3% | 1.0000 | 1.0000 | 1.0000 | +| newtonsoft (`0a2e291`) | C# | Roslyn (`csharp`) | 99.7% | 0.9709 | 0.9774 | 0.9741 | +| socket.io (`3ad4e1f2`) | TypeScript | tsc checker (`tstruth/gen_truth.mjs`) | 99.5% | 0.9367 | 0.9918 | 0.9635 | +| express (`dae209ae`) | JavaScript | tsc `checkJs` (`tstruth/gen_truth.mjs`) | 100% | 0.8846 | 1.0000 | 0.9388 | +| ripgrep (`82313cf`) | Rust | rust-analyzer SCIP (`rust`) | 100% | 0.9650 | 0.9055 | 0.9343 | +| jansson (`684e18c`) | C | clang AST (`clang`) | 100% | 1.0000 | 0.9983 | 0.9991 | +| SwiftyJSON (`3d25441`) | Swift | SourceKit index (`swift`) | 100% | 1.0000 | 1.0000 | 1.0000 | +| turtle (`3cfc963`) | Kotlin | kotlinc + javap (`kotlin`) | 76.8% | 1.0000 | 1.0000 | 1.0000 | +| json-framework (`93e4ca5`) | Objective-C | clang AST (`objc`) | 98.7% | 1.0000 | 1.0000 | 1.0000 | +| flask (`36e4a824`) | Python | pytest trace (`pytruth`, dynamic) | 97.9% | 0.8962 | 0.7867 | 0.8379 | +| php-parser (`8eea230`) | PHP | Xdebug trace (`php`, dynamic) | 100% | 0.9218 | 0.6825 | 0.7843 | + +Rows for gin through jansson, flask and php-parser come from the v0.63.25 +release CI ("Edge accuracy"). SwiftyJSON, turtle and json-framework need +macOS toolchains, so they are not in CI; they were scored locally with the +same binary against the committed truth snapshots in `testdata/`. Every row is gated in `baseline.json`; the per-language sections below hold the progression that produced each number and what remains. -### Second corpus per language (2026-09-19) +### Second corpus per language (2026-10-22, grove v0.63.25) Every rule above was tuned against one repository per language. A second pin per language is the overfitting check: the same binary, a repository @@ -74,14 +79,17 @@ sit beside the first corpus's under `testdata/`. | Corpus (pin) | Language | Oracle | Universe | P | R | First corpus P / R | |---|---|---|---|---|---|---| -| cobra (`adbc881`) | Go | SSA + VTA | 100% | 0.9834 | 0.9870 | 0.9599 / 0.9735 | -| commons-io (`8ad9867d`) | Java | javac + javap | 94.6% | 0.9317 | 0.9776 | 0.9631 / 0.9807 | -| cJSON (`6d9f244`) | C | clang AST | 100% | 0.9982 | 0.9991 | 0.9991 / 0.9837 | -| fd (`5bbfa3e`) | Rust | rust-analyzer SCIP | 100% | 0.9516 | 0.9130 | 0.9366 / 0.9047 | -| p-queue (`180ab9e`) | TypeScript | tsc checker | 100% | 0.9592 | 1.0000 | 0.9156 / 0.9917 | -| Files (`e85f2b4`) | Swift | SourceKit index | 97.0% | 0.9881 | 0.9222 | 0.9355 / 1.0000 | -| CocoaLumberjack (`f54de25f`) | Objective-C | clang AST | 99.6% | 0.9414 | 0.5679 | 1.0000 / 0.9914 | -| csv (`89ac08c`) | PHP | Xdebug trace (dynamic) | 100% | 0.8888 | 0.7068 | 0.9177 / 0.6487 | +| cobra (`adbc881`) | Go | SSA + VTA | 100% | 0.9945 | 1.0000 | 0.9792 / 0.9965 | +| commons-io (`8ad9867d`) | Java | javac + javap | 99.6% | 1.0000 | 1.0000 | 1.0000 / 1.0000 | +| cJSON (`6d9f244`) | C | clang AST | 100% | 0.9982 | 0.9991 | 1.0000 / 0.9983 | +| fd (`5bbfa3e`) | Rust | rust-analyzer SCIP | 100% | 0.9593 | 0.9130 | 0.9650 / 0.9055 | +| p-queue (`180ab9e`) | TypeScript | tsc checker | 100% | 0.9592 | 1.0000 | 0.9367 / 0.9918 | +| Files (`e85f2b4`) | Swift | SourceKit index | 97.0% | 1.0000 | 1.0000 | 1.0000 / 1.0000 | +| CocoaLumberjack (`f54de25f`) | Objective-C | clang AST | 99.6% | 0.9751 | 0.9565 | 1.0000 / 1.0000 | +| csv (`89ac08c`) | PHP | Xdebug trace (dynamic) | 100% | 0.9472 | 0.8367 | 0.9218 / 0.6825 | + +Scores as of 2026-10-22, grove v0.63.25. Files and CocoaLumberjack were +scored locally (macOS toolchains); the rest come from the release CI. CocoaLumberjack, 2026-10-10 (grove v0.63.0): the universe went from 79.2% to 99.6% because Objective-C containers inside `#if` blocks and files whose