Fix/cgo line directives - #282
Open
JBacchelli wants to merge 2 commits into
Open
Conversation
added 2 commits
July 30, 2026 07:55
scip-go emitted one SCIP Document per *compiled* Go file, keyed by pkg.Fset.File(f.Package).Name(). For cgo packages that physical file is cgo's generated output under the build cache (foo.cgo1.go, _cgo_gotypes.go), so every occurrence in a cgo file -- including each `C.<fn>` call site -- was attributed to an ephemeral GOCACHE path instead of the user's real .go source. Occurrence *ranges* were already computed with Fset.Position(), which honors the //line directives cgo emits, so line/column were correct -- only the document they were attached to was wrong. Fix: key documents (and route each occurrence) by the //line-adjusted origin file, and drop occurrences that don't map to real source: - loader: request NeedFiles so pkg.GoFiles is populated. - visitors.OriginFile(pkg, pos): the cleaned, //line-adjusted path for a position. visitors.RealGoFiles(pkg): the package's on-disk source set. - VisitPackageSyntax / index.Index / ListMissing: key documents by OriginFile instead of the physical compiled path, and skip files whose origin is not real source (e.g. cgo's _cgo_gotypes.go glue). - fileVisitor: drop occurrences whose //line-adjusted position resolves to another file, or to a line/column outside the origin. cgo rewrites such as `defer C.f(x)` and inserted thunks like _cgoCheckPointer otherwise yield out-of-bounds ranges that downstream SCIP consumers reject. Non-cgo packages are unaffected (origin == physical file), so existing snapshots are unchanged. For cgo, `C.<fn>` references now resolve to the real <file>.go at the call site.
The previous commit attributed each document to its file's //line origin and dropped any occurrence that didn't fit that one file. That is safe for cgo and for all //line-free code, but a generated file whose //line points at a *different* real source file would lose those occurrences. Generalize it: accumulate occurrences on the document of each occurrence's own //line-resolved origin (via a shared path->document map), emitting one document per source file at the end. An occurrence is dropped only when its origin is not a real source document (cgo's _cgo_gotypes.go glue, a yacc .y, a build-cache path) or its range does not fit that source. - Paths are compared symlink-resolved (CleanResolve), so a module reached through a symlinked directory no longer drops real files as "not a GoFile". - InBounds now also rejects negative line/column (a "//line file:N" directive with no column collapses to column 0, i.e. scip -1) and reversed ranges, so routing can never emit a malformed occurrence. - Document owns occurrence/symbol accumulation, bounds checking, and ToScip rendering; the file visitor routes via targetDoc and attaches symbols in Finish. Verified: existing snapshots are byte-identical (no regression); protobuf/normal Go output is byte-identical to upstream; cgo Go->C refs still anchor to the real .go; a //line-to-another-real-.go file now routes there; malformed (negative column) occurrences are dropped rather than emitted. InBounds unit-tested.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Anchor cgo occurrences to real
.gosource via//linedirectivesProblem
scip-goemits one SCIPDocumentper compiled Go file, keyed bypkg.Fset.File(f.Package).Name()— the physical path the compiler parsed. Fora cgo package that physical file isn't your
foo.go; it's cgo's generatedoutput under the build cache (
foo.cgo1.go,_cgo_gotypes.go). As a result everyoccurrence in a cgo file — including each
C.<fn>call site — was attributed toan ephemeral
GOCACHEpath that isn't in the repo, so those references resolvedto nothing for downstream consumers.
The occurrence ranges were already correct: they're computed with
Fset.Position(), which honors the//linedirectives cgo emits. Only thedocument each occurrence was attached to was wrong.
Fix
Key documents (and route each occurrence) by the
//line-adjusted originfile rather than the physical compiled path, and drop occurrences that can't be
faithfully placed. Non-cgo packages are unaffected (origin == physical file), so
existing output is byte-identical.
Delivered in two commits:
1.
Anchor cgo occurrences to real .go source via //line directivesloader: addpackages.NeedFilessopkg.GoFiles(the real on-disk sources)is populated.
visitors.OriginFile(pkg, pos)— the cleaned,//line-adjusted path for aposition;
visitors.RealGoFiles(pkg)— the package's on-disk source set.OriginFileinstead of the physical path, and skip fileswhose origin is not real source (e.g. cgo's
_cgo_gotypes.goglue).fileVisitordrops occurrences whose//line-adjusted position lands inanother file or outside the origin's bounds.
2.
Route occurrences to their //line origin instead of droppingfile's single origin, accumulate each occurrence on the document of its own
//line-resolved origin (via a sharedpath -> documentmap), then emit onedocument per source file. This correctly handles a generated file whose
//linepoints at a different real source file.(cgo glue, a yacc
.y, a build-cache path) or its range doesn't fit thatsource.
CleanResolvecompares paths symlink-resolved, so a module reached through asymlinked directory no longer drops real files as "not a
GoFile".InBoundsrejects negative line/column (a//line file:Ndirective with nocolumn collapses to column 0 → scip
-1), reversed ranges, and ranges pastEOF/EOL, so routing can never emit a malformed occurrence.
Documentnow owns occurrence/symbol accumulation, bounds checking, andToSciprendering; the file visitor routes viatargetDocand attachessymbols in
Finish.Behavior
byte-identical.
C.<fn>references now resolve to the real<file>.goat the callsite; compiler glue (
_cgo_gotypes.go) and out-of-bounds cgo rewrites (e.g.defer C.f(x), inserted_cgoCheckPointerthunks) are dropped rather thanemitted with a bogus location.
Testing
go test ./...:document(incl. newinbounds_test.go),index(the goldensnapshot suite),
loader,symbols, andoutputall pass. The snapshot suiteconfirms existing output is byte-identical — no regression — and its
assertTypedOccurrenceRangesguard confirms no malformed range is emitted.InBoundsunit test covers EOL columns, past-EOF, negative columns,reversed ranges, and the unreadable-source fail-open policy.
.go;//line-to-another-realfile routing there; negative-column occurrences dropped) verified manually.
Follow-up
There is currently no cgo fixture in
internal/testdata/snapshots/input/, sothe routing/drop behavior is not yet covered end-to-end by the automated suite
(it requires a C toolchain in CI). Adding one is the natural next step for
regression coverage.