Skip to content

[travsr-analysis] #780 reconcile scip-ruby accessor/operator/DSL symbols - #793

Open
Abhishek5517 wants to merge 2 commits into
masterfrom
fix/travsr-store-780-scip-ruby-symbol-reconciliation
Open

[travsr-analysis] #780 reconcile scip-ruby accessor/operator/DSL symbols#793
Abhishek5517 wants to merge 2 commits into
masterfrom
fix/travsr-store-780-scip-ruby-symbol-reconciliation

Conversation

@Abhishek5517

Copy link
Copy Markdown
Collaborator

Issue

#780: on Ruby corpora, travsr status warned that ~52% of semantic definitions did not reconcile with the tree-sitter parse (fastlane @ 426146e: 8572/16527, 51.9%), so references could resolve to duplicate orphan nodes.

The 52% was three unrelated causes wearing one figure. This PR fixes all three, plus the tokenization and measurement issues that let them escape.

Result (fastlane, clean semantic reindex)

attempted missed rate
before 16527 8572 51.9%
after 10697 36 0.34%

travsr fsck clean (no ghost nodes). Proof case supply/lib/supply/generated_universal_apk.rb: the 4 attr_accessor accessors (reader+writer) and == now have real method: twins, the scip: twins are gone, and travsr references package_name / travsr graph == resolve to the real node.

What changed

Prong A (measurement honesty + precision) - travsr-indexer, travsr-daemon, travsr-store:

  • scip_name_kind parses the descriptor chain backtick-aware: the four metadata fields split on the 4th space (not split_whitespace().last()), and the leaf/container/paren splits skip separators inside backticks. RSpec DSL scopes whose descriptions contain spaces, # or / now parse intact instead of as garbage that still counted as a miss.
  • is_synthetic_dsl_scope recognizes Sorbet DSL meta-scopes (describe/context/it blocks) that have no tree-sitter twin, distinguishing them from real <Class:>/<Module:> singleton scopes and from operator methods that start with <.
  • unify_all drops synthetic DSL defs and defs in files tree-sitter never indexed (gitignored vendored code scip-ruby indexes anyway), excludes both from the attempted/missed counters, and drops their inbound refs/edges so they stop stealing spec-file references. Adds Store::phase_a_indexed_paths.
  • A <Class:>/<Module:> leaf (a metaclass) unwraps to the real type so it unifies onto class:X.

Prong B (real reconciliation) - travsr-analysis:

  • Capture setter (def x=) and operator (def ==, <=>, [], []=, ...) methods, whose name node is not an identifier.
  • New post_parse hook on LanguageConfig (Ruby only) expands attr_accessor/attr_reader/attr_writer into reader/writer method nodes, and Struct.new into class:X plus member accessors, block methods, and the implicit initialize. Guards against double-emission when an explicit def also exists.

Prong C - travsr-analysis:

  • Emit constants (const:X) and instance variables (field:C.@x) as def nodes so scip const/ivar references stop resolving to duplicates.

Notes

  • Schema unchanged (v22, additive). No travsr-plugin-* src touched, so no plugin-hashes.lock regeneration.
  • The other 9 config-driven languages get only post_parse: None.
  • Unit tests added in ruby.rs, generic.rs path, indexer scip_unifier.rs, daemon scip_unifier.rs, and store lock each behavior in.

Out of scope (residual 36)

A handful of edge cases remain: methods added by reopening core classes (Hash), a few metaprogramming-defined methods, and classes defined inside describe blocks. These are distinct constructs, not part of this issue's root causes.

@Abhishek5517
Abhishek5517 requested a review from raj-rkv as a code owner August 24, 2026 12:01
@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown

Benchmark comparison (master → PR)

group             master                                 pr
-----             ------                                 --
bfs/chain/50      1.12   325.5±18.43µs        ? ?/sec    1.00    291.4±6.57µs        ? ?/sec
ppr/chain/100     1.14     32.6±0.71µs        ? ?/sec    1.00     28.5±0.21µs        ? ?/sec
ppr/chain/1000    1.12     32.7±0.38µs        ? ?/sec    1.00     29.3±0.19µs        ? ?/sec

Threshold: p95 regression ≥ +10% fails this check.

@Abhishek5517
Abhishek5517 force-pushed the fix/travsr-store-780-scip-ruby-symbol-reconciliation branch from 64be1d0 to 0527899 Compare August 24, 2026 12:11
scip-ruby reported ~52% of semantic definitions as unreconciled on Ruby
corpora (fastlane: 8572/16527). The number was three unrelated causes wearing
one figure. This fixes all three plus the tokenization and measurement issues
that let them escape, taking fastlane's rate to 36/10697 (0.34%).

Prong A (measurement honesty + precision):
- scip_name_kind: parse the descriptor chain backtick-aware. The 4 metadata
  fields split on the 4th space (not split_whitespace().last()), and the
  leaf/container/paren splits skip separators inside backticks, so RSpec DSL
  scopes whose descriptions contain spaces, '#' or '/' parse intact instead of
  as garbage that still counted as a miss.
- is_synthetic_dsl_scope: recognize Sorbet DSL meta-scopes (describe/context/it
  blocks) that have no tree-sitter twin, distinguishing them from real
  Class:/Module: singleton scopes and from operator methods that start with '<'.
- unify_all: drop synthetic DSL defs and defs in files tree-sitter never indexed
  (gitignored vendored code scip-ruby indexes anyway); exclude both from the
  attempted/missed counters and drop their inbound refs/edges so they stop
  stealing spec-file references. Adds Store::phase_a_indexed_paths.
- Unwrap a Class:/Module: leaf (a metaclass) to the real type so it unifies
  onto class:X instead of orphaning.

Prong B (real reconciliation, Ruby parser):
- capture setter (def x=) and operator (def ==, <=>, [], []=, ...) methods,
  whose name node is not an identifier.
- new post_parse hook expands attr_accessor/attr_reader/attr_writer into
  reader/writer method nodes, and Struct.new into class:X plus member accessors,
  block methods, and the implicit initialize. Guards against double-emission
  when an explicit def also exists.

Prong C: emit constants (const:X) and instance variables (field:C.@x) as def
nodes so scip const/ivar references stop resolving to duplicates.

Schema unchanged (v22); no plugin-* src touched. Unit tests lock each behavior.
@Abhishek5517
Abhishek5517 force-pushed the fix/travsr-store-780-scip-ruby-symbol-reconciliation branch from 0527899 to 8619614 Compare August 24, 2026 12:12
Ad-hoc probe_*/*_probe/calib_* example binaries under crates/*/examples were
untracked scratch that could be swept into a commit by a broad `git add`. Ignore
them by pattern; the one real tracked example (rerank_bench.rs) does not match.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant