Skip to content

[#175 architectural follow-up] Tokenizer-level fix for Variable :-naivete (compound-label-in-USE-args) #176

Description

@msarson

Surfaced at #175 Phase A. Deferred to architectural review.

Context

#175 shipped option (b) diagnostic-side guard in AttributeDiagnostics.validateAttributeApplicability to suppress the bare-identifier Variable:Attribute false-positive surface (e.g. USE(RCFilter_SL_Clients:External)). The diagnostic-side fix is narrow + targeted; works correctly. But it leaves the 3-token split intact in the token stream (`Variable(MyVar) + Delimiter(:) + Attribute(External)`), which is grammatically incorrect for compound USE-label names per Clarion convention.

Architectural question

Is the diagnostic-side workaround the right long-term shape, or do we want to invest the 30-file cascade audit + type-keyword-list synchronization burden to land the tokenizer-side fix?

Phase A analysis (from #175)

Three candidate fix layers

(a) Tokenizer-level Variable extension with negative-lookahead

Sketch from #175 issue body:
```ts
[TokenType.Variable]: /\b(?!(?:IF|LOOP|...))[A-Za-z_][A-Za-z0-9_](?::(?!\s(?:byte|short|ushort|long|ulong|real|sreal|decimal|pdecimal|string|cstring|pstring|date|time)\b)[A-Za-z_][A-Za-z0-9_]*)?\b/i
```

Root cause fix. Requires:

  • Full 30-file cascade audit (any consumer that walks neighboring tokens around Variable could break)
  • Type-keyword list synchronization between Variable regex + TypeAnnotation regex (likely factor into shared constant)
  • Comprehensive test coverage on type-annotation edge cases

(b) Already shipped at #175

Diagnostic-side guard in AttributeDiagnostics. Narrow scope; 0 other Attribute consumers affected per audit. Adequate for the immediate user-visible surface.

(c) New CompoundLabel TokenType

Cleanest separation but adds new TokenType. Tier-walker + switch-case-by-type cascade across all consumers. Over-engineered for current surface.

Recommendation

Defer indefinitely unless one of these triggers:

  • A new compound-label-consuming feature surfaces that requires the single-token shape (e.g. a refactoring tool that needs to rename `PrefixGroup:Field` atomically)
  • The diagnostic-side guard pattern repeats in other diagnostics (would suggest the workaround is becoming structural debt)
  • Mark calls option (a) the right architectural direction during a deeper review

Current option (b) is fully adequate per the #175 consumer audit; no other Attribute consumer has the false-positive surface.

Cross-references

Priority: low. Not a user-visible blocker.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions