Skip to content

Fix/cgo line directives - #282

Open
JBacchelli wants to merge 2 commits into
scip-code:mainfrom
JBacchelli:fix/cgo-line-directives
Open

Fix/cgo line directives#282
JBacchelli wants to merge 2 commits into
scip-code:mainfrom
JBacchelli:fix/cgo-line-directives

Conversation

@JBacchelli

Copy link
Copy Markdown

Anchor cgo occurrences to real .go source via //line directives

Problem

scip-go emits one SCIP Document per compiled Go file, keyed by
pkg.Fset.File(f.Package).Name() — the physical path the compiler parsed. For
a cgo package that physical file isn't your foo.go; it's cgo's generated
output under the build cache (foo.cgo1.go, _cgo_gotypes.go). As a result every
occurrence in a cgo file — including each C.<fn> call site — was attributed to
an ephemeral GOCACHE path that isn't in the repo, so those references resolved
to nothing for downstream consumers.

The occurrence ranges were already correct: they're computed with
Fset.Position(), which honors the //line directives cgo emits. Only the
document each occurrence was attached to was wrong.

Fix

Key documents (and route each occurrence) by the //line-adjusted origin
file rather than the physical compiled path, and drop occurrences that can't be
faithfully placed. Non-cgo packages are unaffected (origin == physical file), so
existing output is byte-identical.

Delivered in two commits:

1. Anchor cgo occurrences to real .go source via //line directives

  • loader: add packages.NeedFiles so pkg.GoFiles (the real on-disk sources)
    is populated.
  • visitors.OriginFile(pkg, pos) — the cleaned, //line-adjusted path for a
    position; visitors.RealGoFiles(pkg) — the package's on-disk source set.
  • Key documents by OriginFile instead of the physical path, and skip files
    whose origin is not real source (e.g. cgo's _cgo_gotypes.go glue).
  • fileVisitor drops occurrences whose //line-adjusted position lands in
    another file or outside the origin's bounds.

2. Route occurrences to their //line origin instead of dropping

  • Generalizes the above: instead of dropping any occurrence that doesn't fit the
    file's single origin, accumulate each occurrence on the document of its own
    //line-resolved origin (via a shared path -> document map), then emit one
    document per source file. This correctly handles a generated file whose
    //line points at a different real source file.
  • An occurrence is dropped only when its origin is not a real source document
    (cgo glue, a yacc .y, a build-cache path) or its range doesn't fit that
    source.
  • CleanResolve compares paths symlink-resolved, so a module reached through a
    symlinked directory no longer drops real files as "not a GoFile".
  • InBounds rejects negative line/column (a //line file:N directive with no
    column collapses to column 0 → scip -1), reversed ranges, and ranges past
    EOF/EOL, so routing can never emit a malformed occurrence.
  • Document now owns occurrence/symbol accumulation, bounds checking, and
    ToScip rendering; the file visitor routes via targetDoc and attaches
    symbols in Finish.

Behavior

  • Non-cgo code: unchanged — origin equals the physical file, output is
    byte-identical.
  • cgo: C.<fn> references now resolve to the real <file>.go at the call
    site; compiler glue (_cgo_gotypes.go) and out-of-bounds cgo rewrites (e.g.
    defer C.f(x), inserted _cgoCheckPointer thunks) are dropped rather than
    emitted with a bogus location.

Testing

  • go test ./...: document (incl. new inbounds_test.go), index (the golden
    snapshot suite), loader, symbols, and output all pass. The snapshot suite
    confirms existing output is byte-identical — no regression — and its
    assertTypedOccurrenceRanges guard confirms no malformed range is emitted.
  • New InBounds unit test covers EOL columns, past-EOF, negative columns,
    reversed ranges, and the unreadable-source fail-open policy.
  • cgo behavior (Go→C refs anchoring to the real .go; //line-to-another-real
    file routing there; negative-column occurrences dropped) verified manually.

Follow-up

There is currently no cgo fixture in internal/testdata/snapshots/input/, so
the routing/drop behavior is not yet covered end-to-end by the automated suite
(it requires a C toolchain in CI). Adding one is the natural next step for
regression coverage.

Jacopo Bacchelli added 2 commits July 30, 2026 07:55
scip-go emitted one SCIP Document per *compiled* Go file, keyed by
pkg.Fset.File(f.Package).Name(). For cgo packages that physical file is
cgo's generated output under the build cache (foo.cgo1.go,
_cgo_gotypes.go), so every occurrence in a cgo file -- including each
`C.<fn>` call site -- was attributed to an ephemeral GOCACHE path
instead of the user's real .go source.

Occurrence *ranges* were already computed with Fset.Position(), which
honors the //line directives cgo emits, so line/column were correct --
only the document they were attached to was wrong.

Fix: key documents (and route each occurrence) by the //line-adjusted
origin file, and drop occurrences that don't map to real source:

- loader: request NeedFiles so pkg.GoFiles is populated.
- visitors.OriginFile(pkg, pos): the cleaned, //line-adjusted path for a
  position. visitors.RealGoFiles(pkg): the package's on-disk source set.
- VisitPackageSyntax / index.Index / ListMissing: key documents by
  OriginFile instead of the physical compiled path, and skip files whose
  origin is not real source (e.g. cgo's _cgo_gotypes.go glue).
- fileVisitor: drop occurrences whose //line-adjusted position resolves
  to another file, or to a line/column outside the origin. cgo rewrites
  such as `defer C.f(x)` and inserted thunks like _cgoCheckPointer
  otherwise yield out-of-bounds ranges that downstream SCIP consumers
  reject.

Non-cgo packages are unaffected (origin == physical file), so existing
snapshots are unchanged. For cgo, `C.<fn>` references now resolve to
the real <file>.go at the call site.
The previous commit attributed each document to its file's //line origin and
dropped any occurrence that didn't fit that one file. That is safe for cgo and
for all //line-free code, but a generated file whose //line points at a
*different* real source file would lose those occurrences.

Generalize it: accumulate occurrences on the document of each occurrence's own
//line-resolved origin (via a shared path->document map), emitting one document
per source file at the end. An occurrence is dropped only when its origin is not
a real source document (cgo's _cgo_gotypes.go glue, a yacc .y, a build-cache
path) or its range does not fit that source.

- Paths are compared symlink-resolved (CleanResolve), so a module reached
  through a symlinked directory no longer drops real files as "not a GoFile".
- InBounds now also rejects negative line/column (a "//line file:N" directive
  with no column collapses to column 0, i.e. scip -1) and reversed ranges, so
  routing can never emit a malformed occurrence.
- Document owns occurrence/symbol accumulation, bounds checking, and ToScip
  rendering; the file visitor routes via targetDoc and attaches symbols in Finish.

Verified: existing snapshots are byte-identical (no regression); protobuf/normal
Go output is byte-identical to upstream; cgo Go->C refs still anchor to the real
.go; a //line-to-another-real-.go file now routes there; malformed (negative
column) occurrences are dropped rather than emitted. InBounds unit-tested.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant