analyze.py: make optional Leiden/import-community signal explicit, historical, and explainable

[hailcpy]

Finding

The optional leiden_signal.py module builds a static import graph from the current checkout and returns a Leiden community ID per source file. analyze.py then uses shared community membership as a fourth clustering signal.

There are two problems:

• The preflight flag leiden_available only says the local Python module imported, not whether optional dependencies were installed or communities were actually produced.
• The community graph reflects the current checkout, not the historical code at the time each commit happened. Deleted files, renamed files, and pre-migration dependency structure are invisible.

Impact

The signal is useful as a weak structural prior, but it can be over-trusted:

• old commits may be clustered using today’s import structure,
• deleted/replaced modules cannot participate accurately,
• users cannot tell whether Leiden was disabled, failed, empty, or active,
• assisted mode cannot explain which community edge caused a merge.

Suggested solution

Treat static import communities as an optional, explainable prior rather than historical evidence.

Possible implementation:

1. Return a structured status object, for example:

{
  "status": "active | disabled | missing_deps | no_sources | no_edges | failed",
  "nodes": 123,
  "edges": 456,
  "communities": 12
}

2. Include the status in analyzer preflight.
3. Store community match as an edge reason when it contributes to clustering.
4. Rename wording from “Leiden available” to “import community signal” unless actual graph deps and communities are active.
5. Consider a future historical mode that builds graphs per tag/window or from commit snapshots, but keep current-checkout mode clearly labeled.

Acceptance criteria

• Analyzer JSON distinguishes module importability from active community detection.
• Missing optional dependencies are reported clearly and do not look like success.
• Candidate merge reasons identify when an import-community edge contributed.
• Documentation explains that current-checkout communities are a weak prior, not historical evidence.