From 9fe3c2448958c6b8cead2452758077fef093cf4e Mon Sep 17 00:00:00 2001 From: mohammed naji Date: Fri, 31 Jul 2026 08:00:37 +0400 Subject: [PATCH 1/5] feat(index): add authenticated execution semantics --- docs/core-reset/removal-manifest.yml | 206 +- docs/core-reset/scorecard.md | 33 +- docs/roadmap.md | 22 +- package.json | 2 - src/adapters/filesystem/graph-artifact.ts | 8 + src/adapters/mcp/server.ts | 49 +- src/adapters/typescript/execution.ts | 2033 +++++++++++++++++ src/adapters/typescript/index.ts | 80 +- src/application/retrieve-context.ts | 295 ++- src/domain/index/build-state.ts | 4 +- src/domain/index/model.ts | 690 +++++- src/domain/query/index-status.ts | 630 ++++- src/domain/query/rank.ts | 4 + ...anonical-index-execution-hardening.test.ts | 771 +++++++ tests/unit/canonical-index-execution.test.ts | 408 ++++ tests/unit/core-reset-governance.test.ts | 372 ++- tests/unit/graph-artifact.test.ts | 10 +- .../query-index-execution-validation.test.ts | 637 ++++++ tests/unit/retrieve-context.test.ts | 4 + ...ieve-evidence-skeleton-adversarial.test.ts | 4 + ...rieve-evidence-skeleton-regression.test.ts | 6 +- 21 files changed, 5957 insertions(+), 311 deletions(-) create mode 100644 src/adapters/typescript/execution.ts create mode 100644 tests/unit/canonical-index-execution-hardening.test.ts create mode 100644 tests/unit/canonical-index-execution.test.ts create mode 100644 tests/unit/query-index-execution-validation.test.ts diff --git a/docs/core-reset/removal-manifest.yml b/docs/core-reset/removal-manifest.yml index 3d3f30e3..0aa82846 100644 --- a/docs/core-reset/removal-manifest.yml +++ b/docs/core-reset/removal-manifest.yml @@ -23,39 +23,49 @@ review: disposition_changes: 11 amendment: 'Source lists remain complete and de-overlapped. Issue #588 moved four guaranteed extraction orphans into the completed delete contract. Issue #592 transferred stage.ts, freshness.ts, and source-discovery.ts to evidence-path-query and doctor.ts to thin-delivery. Approved issues #596 and #599 combined the original 54-file / 29,441-LOC query closure with nine finalizer files / 3,590 LOC, yielding one completed 63-file / 33,031-LOC predecessor contract and 22 ownership transfers. proof-report.ts plus review-compare.ts remain move-to-delete changes; serve.ts changed from rebuild to delete, raising disposition_changes from 4 to 7. Owner-approved issue #602 removed stale thin-delivery ownership of deleted serve.ts, transferred package-metadata.ts and shell.ts from rebuild to evaluation-tooling move ownership, raising disposition_changes from 7 to 9, and absorbed the remaining non-core-graph-products and activation-and-extra-integrations production owners into one exact 16-file / 7,277-LOC thin-delivery deletion contract. PR #604 completed that contract without further ownership change. Owner-approved issue #606 transferred graph-source-root.ts and workspace-copy.ts from safe-workspace-primitives to evaluation-tooling, raising disposition_changes from 9 to 11, and activated the exact 20-file / 4,698-LOC move contract from protected base 317dda89f2ea5c75e7626a26b104ceca1bd04ce5. Governance activation merged at 452ad84890c012392c5e6af613e8bfeb17de45db without production source changes. PR #608 completed the exact move without changing any surviving production TypeScript or dependency. First-stage owner-approved issue #610 governance activation merged at dcb52596a3efa89f9ef5d372231ce97a91ae5f9f, then independent review stopped its uncommitted implementation under conditions 7, 8, and 13 before any implementation PR, campaign lock, provider request, or spend. First-stage owner-approved issue #612 authorizes only an eight-path governance-only v2 candidate from that exact merge; its separate activation merge approval remains required. It changes no production ownership or disposition. Issue #625 modifies five existing evidence-path-query production paths and changes no ownership or disposition. Graph/index generation, schemas, CLI, MCP, package dependencies, publication surfaces, and main remain frozen.' cancellation_amendment: 'On 2026-07-28 the owner closed Capability Validation issues #610, #612, #614, #615, and #616 as not planned and revoked every unconsumed preparation, activation, implementation, campaign, provider, spend, and target-execution authority. No campaign ran, no comparative result exists, provider requests remain zero, and spend remains USD 0. The governance-only v2 activation remains immutable history. Issue #618 is a separate bounded retrieval repair and does not revive Capability Validation or Graphify.' - release_amendment: 'Historical release receipt: @lubab/madar@0.40.0-beta.3 was published under npm dist-tag next and GitHub prerelease v0.40.0-beta.3 from exact protected-next commit ece7d0d02643ecec08bd91aa904a4514aa845f42. Issue #625 and PR #626 subsequently completed the generic evidence-skeleton repair on protected next at b6562b715133304bd46e537b6f39008bc1e02095. The owner separately authorized the next beta, tracked by https://github.com/mohanagy/madar/issues/627; release preparation is limited to @lubab/madar@0.40.0-beta.4 under npm dist-tag next and a matching exact-commit GitHub prerelease. npm latest remains 0.32.0. Stable 0.40.0, MCP Registry publication, comparative claims, and main remain outside this release.' + release_amendment: 'Historical release receipt: @lubab/madar@0.40.0-beta.3 was published under npm dist-tag next and GitHub prerelease v0.40.0-beta.3 from exact protected-next commit ece7d0d02643ecec08bd91aa904a4514aa845f42. Issue #625 and PR #626 subsequently completed the generic evidence-skeleton repair on protected next at b6562b715133304bd46e537b6f39008bc1e02095. Issue #627 then published @lubab/madar@0.40.0-beta.4 under npm dist-tag next and the matching GitHub prerelease from exact protected-next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. npm latest remains 0.32.0. Stable 0.40.0, MCP Registry publication, comparative claims, and main remain outside this release. Issue #632 authorizes no publication, release, Registry metadata, tag, or main action.' current: - updated_at: 2026-07-30 + updated_at: 2026-07-31 completed_phase: retrieval-regression-625 - active_phase: null - ready_phase: release-beta - base_commit: ece7d0d02643ecec08bd91aa904a4514aa845f42 + active_phase: semantic-execution-index-632 + ready_phase: null + base_commit: 9043320cfa08370e5cdd3911bfb9283005aa9912 completed_phase_commit: b6562b715133304bd46e537b6f39008bc1e02095 - production_typescript_files: 43 - production_typescript_loc: 12454 - production_loc_added: 1409 - production_loc_removed: 1102 - production_loc_net: 307 + production_typescript_files: 44 + production_typescript_loc: 15873 + production_loc_added: 3606 + production_loc_removed: 187 + production_loc_net: 3419 npm_files: 102 - npm_packed_bytes: 159937 - npm_unpacked_bytes: 639875 - npm_shasum: c5250a0d308b3d6df374851154ddb393a678a992 - npm_integrity: sha512-772P+n4Cx55nqC+CAx8A1aTJ2rY4yk1hUH45lAlxNMMw4YRj8hhswgDiCwczS5hx1S3a+Z+KUv2jma/zWjQZ6w== - npm_artifact_sha256: 8bd8d501b8cd3546e16a5a1ddac1f7649434e685517e1171fbd5897515e76e6b + npm_packed_bytes: 143170 + npm_unpacked_bytes: 638803 + npm_shasum: 8ed94633cdf6f63dfabe12845e5e77644a89dbf1 + npm_integrity: sha512-Nq3m/a7D/iZHYjjRd9b7ENth95qz4zssOPnBqmQEsy+aiebRp68UtMMWArYWFRTftdWwNfTcIvxns0JVdahrdw== + npm_artifact_sha256: df9bb97b425db5063ab178920c7fb824c19c681378744c6cfc7094e99926f1eb measurement_state: source_and_package_exact - snapshot_scope: release_candidate_source_and_package + snapshot_scope: semantic_execution_index_632_candidate release_candidate: version: 0.40.0-beta.4 - protected_anchor_commit: b6562b715133304bd46e537b6f39008bc1e02095 - protected_anchor_tree: 15565d356d947b2b195e7e0ec4871094b60b8493 + protected_anchor_commit: 9043320cfa08370e5cdd3911bfb9283005aa9912 + protected_anchor_tree: f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc target_branch: next npm_dist_tag: next stable_release: false comparator_result: none provider_requests: 0 spend_usd: 0 - publication_state: preparation + publication_state: published + tag: v0.40.0-beta.4 + github_prerelease: https://github.com/mohanagy/madar/releases/tag/v0.40.0-beta.4 + published_at: 2026-07-30T13:07:45.103Z + published_package: + npm_files: 102 + npm_packed_bytes: 159937 + npm_unpacked_bytes: 639875 + npm_shasum: c5250a0d308b3d6df374851154ddb393a678a992 + npm_integrity: sha512-772P+n4Cx55nqC+CAx8A1aTJ2rY4yk1hUH45lAlxNMMw4YRj8hhswgDiCwczS5hx1S3a+Z+KUv2jma/zWjQZ6w== + npm_artifact_sha256: 8bd8d501b8cd3546e16a5a1ddac1f7649434e685517e1171fbd5897515e76e6b targets: production_typescript_files_max: 80 @@ -2206,6 +2216,164 @@ items: notes: 'Issue #625 is a separate architecture repair after the immutable #618 and #622 merges and the published beta.3 receipt. It replaces phrase-gated full-flow recovery with one generic deterministic pipeline: concept groups -> bounded candidate subgraph -> graph-coherent evidence skeleton or forest -> atomic evidence-bundle packing -> structural coverage check. The immutable first candidate stopped before PR under receipt 5122801278 at 104 files / 168,719 packed / 697,521 unpacked bytes with five correctness blockers and a material retained-heap regression. PR #626 first head eb664013a399bd4ef970915ce9382f841395ebfa then stopped honestly when all six hosted jobs rejected cold single-sample timing assertions and CodeRabbit opened five threads; no merge was attempted. Corrective source/test snapshot 4285bdf3f335a31b9c5176cb4215db3aabc5e787 fixed sentence-boundary casing and locator formatting, proved exact boundary-heavy budget accounting plus the maximum-question/minimum-budget envelope, and passed the unchanged package gate at 102 files / 159,980 packed / 639,930 unpacked bytes. The strict warmed loaded-ready-graph reference p95 remained below 500 ms and was not claimed by hosted CI. A corrected post-yield forced-GC comparison proved no material retained-heap regression. Final head 3d194b62aba3324e93bcf01c77cf508edea70a6f passed all six CI jobs, independent review, CodeRabbit, and zero unresolved threads; protected squash merge b6562b715133304bd46e537b6f39008bc1e02095 preserved reviewed tree 15565d356d947b2b195e7e0ec4871094b60b8493. No ceiling was widened. Issue #625 itself authorized no publication; the separate beta.4 release is tracked by #627.' exit_gate: The released broad, focused, and control report-flow prompts, accepted paraphrases, alpha-renamed and distractor-heavy variants, and independent synchronous, queued, fan-out/fan-in, cyclic, and disconnected topologies pass within the frozen retrieval and performance ceilings, with zero false-ready results and no repository-specific production rule. + - id: semantic-execution-index-632 + disposition: keep + status: in_progress + destination: canonical JavaScript/TypeScript semantic execution index + modified_sources: + - src/adapters/filesystem/graph-artifact.ts + - src/adapters/mcp/server.ts + - src/adapters/typescript/execution.ts + - src/adapters/typescript/index.ts + - src/application/retrieve-context.ts + - src/domain/index/build-state.ts + - src/domain/index/model.ts + - src/domain/query/index-status.ts + - src/domain/query/rank.ts + verification: + - tests/unit/canonical-index-execution.test.ts + - tests/unit/canonical-index-execution-hardening.test.ts + - tests/unit/query-index-execution-validation.test.ts + - tests/unit/retrieve-context.test.ts + - tests/unit/graph-artifact.test.ts + - tests/unit/core-reset-governance.test.ts + activation: + issue: https://github.com/mohanagy/madar/issues/632 + protected_base: 9043320cfa08370e5cdd3911bfb9283005aa9912 + protected_base_tree: f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc + target_branch: next + delivery_limits: + new_production_files_max: 4 + net_production_loc_max: 3500 + graph_artifact_size_ratio_max: 1.5 + indexing_median_ratio_max: 1.25 + warm_retrieval_p95_ms_less_than: 500 + npm_package_budget: + files_max: 102 + packed_bytes_max: 165000 + unpacked_bytes_max: 640000 + package_metadata_contract: + path: package.json + removed_files_entries: + - examples/why-madar.md + - CHANGELOG.md + repository_files_deleted: false + version_change: forbidden + script_change: forbidden + dependency_change: forbidden + package_lock_change: forbidden + candidate: + source_measurement: + production_typescript_files: 44 + production_typescript_loc: 15873 + added: 3606 + removed: 187 + net: 3419 + diff_sha256: 1916499d8e195e20c47cc05868a50f3ca408759e932596c16388da6e66e41ec6 + package_measurement: + files: 102 + packed_bytes: 143170 + unpacked_bytes: 638803 + shasum: 8ed94633cdf6f63dfabe12845e5e77644a89dbf1 + integrity: sha512-Nq3m/a7D/iZHYjjRd9b7ENth95qz4zssOPnBqmQEsy+aiebRp68UtMMWArYWFRTftdWwNfTcIvxns0JVdahrdw== + artifact_sha256: df9bb97b425db5063ab178920c7fb824c19c681378744c6cfc7094e99926f1eb + local_verification: + focused_tests_passed: 149 + last_pre_cache_full_tests_passed: 722 + candidate_full_suite_status: pending_clean_runner_after_local_host_memory_pressure + candidate_full_suite_local_attempts: 3 + graph_nodes: 12350 + graph_edges: 32839 + exact_queue_channels: 6 + exact_channel_edges: 51 + graph_artifact_size_ratio: 1.2344886849638093 + beta4_indexing_trials_seconds: + - 20.32 + - 21.33 + - 22.16 + - 22.51 + - 22.74 + beta4_indexing_median_seconds: 22.16 + candidate_indexing_trials_seconds: + - 23.31 + - 24.16 + - 24.41 + - 24.87 + - 25.42 + candidate_indexing_median_seconds: 24.41 + indexing_median_ratio: 1.1015342960288808 + warm_retrieval_samples: 100 + warm_retrieval_median_ms: 165.343 + warm_retrieval_p95_ms: 216.599 + warm_retrieval_max_ms: 257.814 + deterministic_graph_sha256: 478e1ed93fcef5a7592dfa0f884318713128e01857dc706a28f2e0d6142937d0 + beta4_retrieval_output_byte_identical: true + exact_head_ci: pending + independent_review: pending + retrieval_budget: + files_max: 12 + snippets_max: 25 + closure_passes_max: 1 + serialized_tokens_max: 4000 + constraints: + repository_specific_rule: forbidden + manual_channel_map: forbidden + dependency_change: forbidden + retrieval_result_v2_cutover: forbidden + compatibility_reader_or_alias: forbidden + provider_activity: forbidden + npm_publication: forbidden + github_release: forbidden + registry_metadata_publication: forbidden + tag: forbidden + main_target: forbidden + notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. Exact local source, package, focused-test, real-corpus graph-size, channel-topology, beta.4 retrieval-compatibility, alternating indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The pre-cache candidate passed all 722 tests; the exact final candidate full suite remains pending a clean hosted runner because three local attempts were stopped by fork-start or per-test timeouts while an unrelated user process held approximately 19 GB of resident memory. Exact-head CI, independent review, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' + exit_gate: Every retained fact and exact channel edge is deterministic, source-authenticated and mutation-sensitive; false persistence/channel matches remain absent; source, graph-size, indexing, warm-retrieval, package, full-test, exact-head CI, independent-review, and zero-thread gates pass without a v2 result cutover, new dependency, publication, tag, release, Registry metadata, or main target. + + - id: obligation-driven-retrieval-630 + disposition: keep + status: planned + destination: strict obligation-driven workflow dossier retrieval + depends_on: + - semantic-execution-index-632 + activation: + issue: https://github.com/mohanagy/madar/issues/630 + protected_base: 9043320cfa08370e5cdd3911bfb9283005aa9912 + protected_base_tree: f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc + target_branch: next + constraints: + work_before_dependency_completion: forbidden + publication: forbidden + tag: forbidden + registry_metadata_publication: forbidden + main_target: forbidden + notes: 'Issue #630 is pending behind #632. It owns question obligations, bounded recovery, answerability and the retrieve-result v2 dossier cutover; none of that work is activated by #632.' + exit_gate: Every mandatory question obligation is proven and packed into one non-truncated ready dossier, or the result returns the exact non-ready state and missing obligations within the unchanged file, excerpt, token, recovery, package, and latency ceilings. + + - id: no-fallback-qualification-631 + disposition: keep + status: planned + destination: installed-package no-fallback qualification against frozen comparators + depends_on: + - semantic-execution-index-632 + - obligation-driven-retrieval-630 + activation: + issue: https://github.com/mohanagy/madar/issues/631 + protected_base: 9043320cfa08370e5cdd3911bfb9283005aa9912 + protected_base_tree: f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc + target_branch: next + constraints: + work_before_dependency_completion: forbidden + provider_traffic_or_spend: requires_separate_owner_authorization + beta_publication: requires_separate_owner_authorization + stable_or_latest_publication: forbidden + github_release: forbidden + registry_metadata_publication: forbidden + tag: forbidden + main_target: forbidden + notes: 'Issue #631 is pending behind #632 and #630. It will compare an installed reviewed candidate with 0.32.0, 0.40.0-beta.4, one pinned Graphify commit and a no-tool control. No comparative claim or provider campaign exists yet.' + exit_gate: The installed exact-head package matches or beats the strongest frozen baseline, scores at least 90 mean with no run below 85 or critical error, makes one Madar retrieval with zero repository-tool fallback in natural-client runs, and passes closed-book, parity, budget, CI, independent-review, and zero-thread gates. + - id: non-core-graph-products disposition: delete status: complete diff --git a/docs/core-reset/scorecard.md b/docs/core-reset/scorecard.md index 3b5c6d5b..bbb28c85 100644 --- a/docs/core-reset/scorecard.md +++ b/docs/core-reset/scorecard.md @@ -2,7 +2,7 @@ > **RFC:** [#577](https://github.com/mohanagy/madar/issues/577) > **Milestone:** [`v0.40.0 — Core Reset`](https://github.com/mohanagy/madar/milestone/7) -> **Status:** accepted; the product vertical slice through Evaluation Tooling Isolation and retrieval regressions #618, #622, and #625 passed; `0.40.0-beta.3` remains immutable published history; `0.40.0-beta.4` release preparation is authorized against protected `next`; Capability Validation remains cancelled; npm `latest`, stable release, MCP Registry publication, and `main` remain unauthorized +> **Status:** accepted; the product vertical slice through Evaluation Tooling Isolation and retrieval regressions #618, #622, and #625 passed; `0.40.0-beta.4` is published from exact protected-`next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912`; semantic execution index #632 is active, #630 and #631 are pending in dependency order; npm `latest` remains `0.32.0`; another beta, stable release, MCP Registry publication, and `main` remain unauthorized This is the phase-gate evidence ledger. An issue or PR link is not evidence by itself; each gate needs a reproducible test, receipt, measurement, or external-user record. @@ -43,11 +43,14 @@ The schema-validated, share-safe receipt was recorded at tooling checkout `250a6 | Retrieval regression #618 | **Passed** | Restore grounded natural-flow retrieval in one call or at most one bounded recovery without repository-specific rules, graph/index changes, dependencies, or fallback engines | [#618](https://github.com/mohanagy/madar/issues/618) completed through [PR #620](https://github.com/mohanagy/madar/pull/620), merged at `eaa1a8781eda28dad5395d6da378a2cc40bf81fe`; all six exact-head CI jobs, two independent no-blocker reviews, and zero review threads passed | | Retrieval regression #622 | **Passed** | Stabilize equivalent end-to-end report-flow prompts and expose honest asynchronous handoff targets within the unchanged retrieval and package ceilings | [#622](https://github.com/mohanagy/madar/issues/622) completed through [PR #623](https://github.com/mohanagy/madar/pull/623), merged at `6416dbc02cefb3bd79157cf440e420b30dda8cf0`; [six-job CI](https://github.com/mohanagy/madar/actions/runs/30452883659), two exact-head no-blocker reviews, CodeRabbit PASS, and zero unresolved threads | | Retrieval regression #625 | **Passed** | Replace phrase-gated recovery with a generic bounded, graph-coherent evidence skeleton/forest without exceeding the inherited package ceilings | [#625](https://github.com/mohanagy/madar/issues/625) completed through [PR #626](https://github.com/mohanagy/madar/pull/626), merged at `b6562b715133304bd46e537b6f39008bc1e02095`; [six-job CI](https://github.com/mohanagy/madar/actions/runs/30533140531), independent exact-head review, CodeRabbit PASS, and zero unresolved threads | +| Semantic execution index #632 | **In progress** | Authenticated ordered body facts, exact async channels and receiver/type-proven persistence pass every source, graph, indexing, latency, package, CI, review and zero-thread gate | [#632](https://github.com/mohanagy/madar/issues/632); protected base `9043320cfa08370e5cdd3911bfb9283005aa9912`; no candidate, CI, review or merge receipt yet | +| Obligation-driven retrieval #630 | **Pending** | Return a complete authenticated workflow dossier or exact missing obligations within unchanged budgets | [#630](https://github.com/mohanagy/madar/issues/630); blocked on #632 | +| No-fallback qualification #631 | **Pending** | Installed exact-head package matches or beats the strongest frozen baseline and requires zero repository-tool fallback | [#631](https://github.com/mohanagy/madar/issues/631); blocked on #632 and #630 | | External validation | **Deferred** | Activation, retention, and paid-intent evidence remains required for later stable claims, not this beta | No external-validation claim in `0.40.0-beta.4` | -| Beta release | **Ready** | Publish only the exact protected-`next` beta.4 candidate under npm `next` plus a matching exact-commit GitHub prerelease | [#627](https://github.com/mohanagy/madar/issues/627); candidate package 102 / 159,937 / 639,875 against fixed 102 / 165,000 / 640,000 ceilings; npm `latest` remains `0.32.0` | +| Beta release | **Published** | Preserve exact beta.4 npm/GitHub release history; any later beta requires separate authorization after #632, #630 and #631 gates | [#627](https://github.com/mohanagy/madar/issues/627); exact commit `9043320cfa08370e5cdd3911bfb9283005aa9912`; package 102 / 159,937 / 639,875; npm `next` is `0.40.0-beta.4`, npm `latest` is `0.32.0` | | Stable release | Not started | Every separately retained stable gate passed; old core absent; migration docs ready | Pending; the beta does not satisfy this gate | -Issues `#622` and `#625` are complete on `next`. Evaluation Tooling Isolation completed through #606 and PR #608 at 43 production files / 11,956 LOC; #618 completed at 43 production files / 12,008 LOC with `+69/-17/net +52`; #622 completed at 43 production files / 12,147 LOC with `+164/-25/net +139`; and #625 completed at 43 production files / 12,454 LOC with `+1,409/-1,102/net +307` against its protected base. The beta.3 release is immutable published history at exact protected-`next` commit `ece7d0d02643ecec08bd91aa904a4514aa845f42`. Capability Validation issues #610, #612, #614, #615, and #616 are closed not planned: no campaign ran, no Native/Graphify/Madar result exists, provider requests remain zero, and paid spend remains USD 0. Their `Deferred` project disposition means “not a beta gate and not passed.” Issue #625 itself authorized no publication; the separately authorized beta.4 release is tracked by #627. +Issues `#622` and `#625` are complete on `next`. Evaluation Tooling Isolation completed through #606 and PR #608 at 43 production files / 11,956 LOC; #618 completed at 43 production files / 12,008 LOC with `+69/-17/net +52`; #622 completed at 43 production files / 12,147 LOC with `+164/-25/net +139`; and #625 completed at 43 production files / 12,454 LOC with `+1,409/-1,102/net +307` against its protected base. Beta.4 is immutable published history at exact protected-`next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912`. Capability Validation issues #610, #612, #614, #615, and #616 are closed not planned: no campaign ran, provider requests remain zero, and paid spend remains USD 0. The new work is dependency ordered: #632 active, then #630 pending, then #631 pending. ### Directed multigraph phase evidence (passed) @@ -187,12 +190,26 @@ The following contract facts are historical. Issues #610 and #612, together with - Local focused retrieval tests pass 107/107, the exact five-file candidate gate passes 128/128, and the complete V8 coverage suite passes 76 files / 684 tests with 83.74% statement, 75.41% branch, 91.00% function, and 87.56% line coverage. - Final reviewed head `3d194b62aba3324e93bcf01c77cf508edea70a6f` and tree `15565d356d947b2b195e7e0ec4871094b60b8493` passed all six [exact-head CI jobs](https://github.com/mohanagy/madar/actions/runs/30533140531), [independent review](https://github.com/mohanagy/madar/pull/626#pullrequestreview-4817647772), CodeRabbit, and zero unresolved threads. Protected squash merge `b6562b715133304bd46e537b6f39008bc1e02095` preserved that exact reviewed tree. [Issue completion](https://github.com/mohanagy/madar/issues/625#issuecomment-5129571164) and [PR completion](https://github.com/mohanagy/madar/pull/626#issuecomment-5129571401) receipts close the phase. -### Beta.4 release (ready) +### Beta.4 release (published) -- Release preparation starts from exact protected `next` commit `b6562b715133304bd46e537b6f39008bc1e02095` and tree `15565d356d947b2b195e7e0ec4871094b60b8493`. -- Scope is release-only: governance completion, version/package metadata, changelog and release documentation, SBOM/Registry manifest alignment, release-workflow pins, and matching tests. No production source, dependency, graph/index schema, CLI/MCP surface, or retrieval budget changes. -- The exact npm 12.0.1 candidate is 102 files / 159,937 packed / 639,875 unpacked bytes with shasum `c5250a0d308b3d6df374851154ddb393a678a992`, integrity `sha512-772P+n4Cx55nqC+CAx8A1aTJ2rY4yk1hUH45lAlxNMMw4YRj8hhswgDiCwczS5hx1S3a+Z+KUv2jma/zWjQZ6w==`, and tarball SHA-256 `8bd8d501b8cd3546e16a5a1ddac1f7649434e685517e1171fbd5897515e76e6b`. -- The release PR targets protected `next`. Merge requires the exact head to pass all six CI jobs, independent no-blocker review, honest CodeRabbit disposition, and zero unresolved threads. Only the exact merged `next` tip may be tagged `v0.40.0-beta.4` and published as npm `next` plus a matching GitHub prerelease. npm `latest`, stable release, MCP Registry publication, comparative claims, and `main` remain out of scope. +- `@lubab/madar@0.40.0-beta.4` is published under npm `next` from exact protected-`next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912` and tree `f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc`; tag `v0.40.0-beta.4` and the matching [GitHub prerelease](https://github.com/mohanagy/madar/releases/tag/v0.40.0-beta.4) target that commit. +- The published npm 12.0.1 artifact is 102 files / 159,937 packed / 639,875 unpacked bytes with shasum `c5250a0d308b3d6df374851154ddb393a678a992`, integrity `sha512-772P+n4Cx55nqC+CAx8A1aTJ2rY4yk1hUH45lAlxNMMw4YRj8hhswgDiCwczS5hx1S3a+Z+KUv2jma/zWjQZ6w==`, and tarball SHA-256 `8bd8d501b8cd3546e16a5a1ddac1f7649434e685517e1171fbd5897515e76e6b`. +- npm `latest` remains `0.32.0`. Another beta, stable release, MCP Registry publication, new comparative claims, and `main` remain out of scope without separate authority. + +### Semantic execution index #632 (in progress) + +- Protected base and target are exact `next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912`, tree `f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc`, and protected branch `next`; the issue is [#632](https://github.com/mohanagy/madar/issues/632). +- The exact allowed production paths are `src/adapters/filesystem/graph-artifact.ts`, `src/adapters/mcp/server.ts`, `src/adapters/typescript/execution.ts`, `src/adapters/typescript/index.ts`, `src/application/retrieve-context.ts`, `src/domain/index/build-state.ts`, `src/domain/index/model.ts`, `src/domain/query/index-status.ts`, and `src/domain/query/rank.ts`. +- Delivery is blocked above four new production files, 3,500 net new production lines, 1.5x the beta.4 GoValidate graph size, 1.25x the beta.4 same-machine indexing median, or warm retrieval p95 greater than or equal to 500 ms. Package ceilings remain 102 files / 165,000 packed / 640,000 unpacked bytes. +- The package whitelist may remove only `examples/why-madar.md` and `CHANGELOG.md`; the repository files remain present, and version, scripts, dependencies, package lock, publication and public surface cannot change. +- #632 owns authenticated ordered body facts, exact queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. Retrieval-result v2, obligation planning, response dossier generation, comparator claims, provider activity, npm publication, GitHub Release, Registry metadata, tags, and `main` are outside this phase. +- No candidate final receipt, CI result, review result, merge commit, package result, performance result, or publication is claimed here. Those fields remain open until the exact head passes its gates. + +### Successors #630 and #631 (pending) + +- [#630](https://github.com/mohanagy/madar/issues/630) is blocked on #632 and owns explicit obligations, bounded recovery, strict answerability and the `madar.retrieve` v2 dossier. +- [#631](https://github.com/mohanagy/madar/issues/631) is blocked on #632 and #630 and owns installed-package parity plus the no-fallback comparison against `0.32.0`, `0.40.0-beta.4`, one pinned Graphify commit and a no-tool control. +- Neither pending issue authorizes provider traffic or spend, npm publication, GitHub Release, Registry metadata, tags, stable/`latest`, or `main`; any real campaign or beta publication requires separate owner authorization. ## Graph gates diff --git a/docs/roadmap.md b/docs/roadmap.md index 2690b6a5..36fb4275 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -11,7 +11,7 @@ Madar is executing an accepted Core Reset. The roadmap is outcome-driven: work a - [Removal manifest](core-reset/removal-manifest.yml) — keep, rebuild, move, delete, and defer decisions - [Scorecard](core-reset/scorecard.md) — technical and business evidence gates -The RFC is **accepted**. Scope and baseline, Directed multigraph, Canonical TypeScript/JavaScript index, the combined legacy/non-code deletion, Generation and reconciliation, Evidence-path query, Thin Delivery, Evaluation Tooling Isolation, and the bounded retrieval repairs in [#618](https://github.com/mohanagy/madar/issues/618), [#622](https://github.com/mohanagy/madar/issues/622), and [#625](https://github.com/mohanagy/madar/issues/625) have passed. Capability Validation and the Native-vs-Graphify comparator are cancelled as release gates. `0.40.0-beta.3` remains immutable published history; `0.40.0-beta.4` release preparation is authorized against protected `next` under [#627](https://github.com/mohanagy/madar/issues/627). npm `latest`, stable release, MCP Registry publication, and `main` remain unauthorized. +The RFC is **accepted**. Scope and baseline, Directed multigraph, Canonical TypeScript/JavaScript index, the combined legacy/non-code deletion, Generation and reconciliation, Evidence-path query, Thin Delivery, Evaluation Tooling Isolation, and the bounded retrieval repairs in [#618](https://github.com/mohanagy/madar/issues/618), [#622](https://github.com/mohanagy/madar/issues/622), and [#625](https://github.com/mohanagy/madar/issues/625) have passed. Capability Validation and the earlier Native-vs-Graphify campaign are cancelled history. `0.40.0-beta.4` is published from exact protected-`next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912`; npm `latest` remains `0.32.0`. The semantic execution program is now dependency-ordered as active [#632](https://github.com/mohanagy/madar/issues/632), pending [#630](https://github.com/mohanagy/madar/issues/630), then pending [#631](https://github.com/mohanagy/madar/issues/631). Another beta, stable release, MCP Registry publication, and `main` remain unauthorized. ## Passed — directed multigraph @@ -157,11 +157,25 @@ First PR head `eb664013a399bd4ef970915ce9382f841395ebfa` stopped in [run 3052779 Final reviewed head `3d194b62aba3324e93bcf01c77cf508edea70a6f` and tree `15565d356d947b2b195e7e0ec4871094b60b8493` passed all six [exact-head CI jobs](https://github.com/mohanagy/madar/actions/runs/30533140531), [independent review](https://github.com/mohanagy/madar/pull/626#pullrequestreview-4817647772), CodeRabbit, and zero unresolved threads. Protected squash merge `b6562b715133304bd46e537b6f39008bc1e02095` preserved the reviewed tree. Issue #625 itself authorized no publication. -## Ready — `0.40.0-beta.4` +## Published — `0.40.0-beta.4` -Release preparation starts from exact protected `next` commit `b6562b715133304bd46e537b6f39008bc1e02095` and tree `15565d356d947b2b195e7e0ec4871094b60b8493`. Scope is limited to governance completion, version/package metadata, changelog and release documentation, SBOM/Registry manifest alignment, release-workflow pins, and tests; it changes no production source or dependency. +The release completed from exact protected `next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912` and tree `f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc`. `@lubab/madar@0.40.0-beta.4` is published under npm dist-tag `next`; tag `v0.40.0-beta.4` and the matching [GitHub prerelease](https://github.com/mohanagy/madar/releases/tag/v0.40.0-beta.4) target that exact commit. -The exact npm 12.0.1 candidate is 102 files / 159,937 packed / 639,875 unpacked bytes under the unchanged 102 / 165,000 / 640,000 ceilings. The release PR targets protected `next`, never `main`. Merge requires all six exact-head CI jobs, independent no-blocker review, honest CodeRabbit disposition, and zero unresolved threads. Only the exact merged `next` tip may be tagged `v0.40.0-beta.4` and published as npm `next` plus a matching GitHub prerelease. npm `latest`, stable release, MCP Registry publication, comparative claims, and `main` remain out of scope. +The published npm artifact is 102 files / 159,937 packed / 639,875 unpacked bytes under the unchanged 102 / 165,000 / 640,000 ceilings. npm `latest` remains `0.32.0`; stable release, MCP Registry publication, new comparative claims, and `main` remain out of scope. + +## In progress — semantic execution index #632 + +[#632](https://github.com/mohanagy/madar/issues/632) starts from exact protected `next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912` and tree `f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc`, and its PR target is `next`. It adds compact authenticated body facts, numeric order and control, exact queue/job/event channel topology, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It does not cut retrieval output to v2. + +The active delivery limits are no more than four new production files, no more than 3,500 net new production lines, no more than 1.5x the beta.4 GoValidate graph size, indexing median no slower than 1.25x beta.4 on the same machine, warm retrieval p95 strictly below 500 ms, and the unchanged package ceilings of 102 files / 165,000 packed / 640,000 unpacked bytes. No dependency, provider activity, publication, GitHub Release, Registry metadata, tag, or `main` action is authorized. Candidate, CI, review, merge, and publication receipts remain pending and must not be inferred from local work. + +## Pending — obligation-driven retrieval #630 + +[#630](https://github.com/mohanagy/madar/issues/630) starts only after #632 completes. It owns explicit question obligations, graph-coherent workflow construction, at most two bounded recovery passes, exact-range hydration, strict answerability, and the `madar.retrieve` v2 dossier. It retains the 4,000-token / 12-file / 25-excerpt ceilings and cannot publish. + +## Pending — installed no-fallback qualification #631 + +[#631](https://github.com/mohanagy/madar/issues/631) starts only after #632 and #630 complete. It owns installed-package parity and the frozen comparison against `0.32.0`, `0.40.0-beta.4`, one pinned Graphify commit, and a no-tool control. It requires one Madar retrieval and zero repository Read/Grep/Glob/Bash fallback calls in natural-client candidate runs. No provider traffic, spend, beta publication, release, Registry metadata, tag, stable/`latest`, or `main` authority exists without a separate owner authorization. ## Validation — release decision diff --git a/package.json b/package.json index 778563eb..1678343d 100644 --- a/package.json +++ b/package.json @@ -12,8 +12,6 @@ "files": [ "dist/", "examples/sample-workspace/", - "examples/why-madar.md", - "CHANGELOG.md", "README.md", "LICENSE" ], diff --git a/src/adapters/filesystem/graph-artifact.ts b/src/adapters/filesystem/graph-artifact.ts index 4ef9a934..bcda4d37 100644 --- a/src/adapters/filesystem/graph-artifact.ts +++ b/src/adapters/filesystem/graph-artifact.ts @@ -7,6 +7,14 @@ import { writeTextFileAtomically } from '../../shared/atomic-file.js' import { validateGraphPath } from '../../shared/security.js' const MAX_GRAPH_BYTES = 100 * 1024 * 1024 const descriptorIdentity = (stats: ReturnType) => `${stats.dev}:${stats.ino}:${stats.ctimeMs}:${stats.mtimeMs}:${stats.size}` +export function graphArtifactIdentity(graphPath: string): string { + const descriptor = openSync(validateGraphPath(graphPath), 'r') + try { + const stats = fstatSync(descriptor) + if (stats.size > MAX_GRAPH_BYTES) throw new Error(`Graph file too large: ${graphPath}`) + return descriptorIdentity(stats) + } finally { closeSync(descriptor) } +} export function readBoundedUtf8(descriptor: number, maxBytes: number, tooLarge: string): string { const chunks: Buffer[] = []; let total = 0 while (total <= maxBytes) { diff --git a/src/adapters/mcp/server.ts b/src/adapters/mcp/server.ts index 4471047d..c94e5719 100644 --- a/src/adapters/mcp/server.ts +++ b/src/adapters/mcp/server.ts @@ -3,7 +3,10 @@ import { resolve } from 'node:path' import { createInterface } from 'node:readline' import { Transform, type Readable, type Writable } from 'node:stream' import { setTimeout as delay } from 'node:timers/promises' -import { readGraphArtifactReceipt } from '../filesystem/graph-artifact.js' +import { + graphArtifactIdentity, + readGraphArtifactReceipt, +} from '../filesystem/graph-artifact.js' import { readBuildState } from '../../domain/index/build-state.js' import { failedQueryIndex, @@ -107,28 +110,43 @@ function boundedLineInput(): Transform { function unavailableIndex(): QueryIndex { return failedQueryIndex('unavailable', UNAVAILABLE_SUBJECT) } +type QueryIndexCache = { + identity: string + acceptedBuildId: string + index: QueryIndex +} function readAcceptedQueryIndex( workspace: MadarWorkspace, controller: ReconciliationController, -): QueryIndex { + cached: QueryIndexCache | null, +): QueryIndexCache | { index: QueryIndex } { try { + const acceptedBuildId = controller.acceptedBuildId() + if ( + cached + && cached.acceptedBuildId === acceptedBuildId + && cached.identity === graphArtifactIdentity(workspace.graphPath) + ) return cached const receipt = readGraphArtifactReceipt(workspace.graphPath) const index = inspectQueryIndex(receipt.graph) - if (index.state !== 'ready') return index + if (index.state !== 'ready') return { index } const build = readBuildState(receipt.graph) if (!build - || build.build_id !== controller.acceptedBuildId() + || !acceptedBuildId + || build.build_id !== acceptedBuildId || !samePath(index.root_path, workspace.rootPath)) { - return unavailableIndex() + return { index: unavailableIndex() } } - return index + return { identity: receipt.identity, acceptedBuildId, index } } catch (error) { - return failedQueryIndex( - unavailableGraphError(error) ? 'unavailable' : 'corrupt', - unavailableGraphError(error) - ? UNAVAILABLE_SUBJECT - : 'canonical graph artifact', - ) + return { + index: failedQueryIndex( + unavailableGraphError(error) ? 'unavailable' : 'corrupt', + unavailableGraphError(error) + ? UNAVAILABLE_SUBJECT + : 'canonical graph artifact', + ), + } } } function settleBefore( @@ -181,6 +199,7 @@ class McpRuntime { #initialized = false #toolsListed = false #controller: ReconciliationController | null = null + #queryIndexCache: QueryIndexCache | null = null #startFailure: string | null = null #startPromise: Promise | null = null constructor( @@ -238,7 +257,11 @@ class McpRuntime { if (controller.failureReason()) return unavailableIndex() const state = controller.state() if (controller.startupComplete() && state === 'idle') { - return readAcceptedQueryIndex(this.#workspace, controller) + const result = readAcceptedQueryIndex( + this.#workspace, controller, this.#queryIndexCache, + ) + this.#queryIndexCache = 'identity' in result ? result : null + return result.index } if (state === 'failed' || state === 'stopped') return unavailableIndex() await delay(Math.min( diff --git a/src/adapters/typescript/execution.ts b/src/adapters/typescript/execution.ts new file mode 100644 index 00000000..cfbf2eca --- /dev/null +++ b/src/adapters/typescript/execution.ts @@ -0,0 +1,2033 @@ +import { createHash } from 'node:crypto' +import ts from 'typescript' +const { + isArrowFunction: isArrow, isBinaryExpression: isBinary, + isCallExpression: isCall, isFunctionExpression: isFunction, + isIdentifier, isIfStatement: isIf, isNewExpression: isNew, + isNumericLiteral: isNumeric, isParameter, isPropertyAccessExpression: isAccess, + isPropertyDeclaration: isPropertyDecl, isTypeReferenceNode: isTypeReference, + isVariableDeclaration: isVariable, +} = ts +const K = ts.SyntaxKind +import { + decodeIndexBodyFactTable, + encodeIndexBodyFactTable, + INDEX_BODY_FACT_CONTROL_LIMIT, + IndexBodyFactBoundsError, + indexBodyFactId, + indexChannelId, +} from '../../domain/index/model.js' +import type { + IndexBodyFact, IndexCallFact, IndexChannelNode, IndexControlFrame, + IndexDiagnostic, IndexEdge, IndexFactEvidence, IndexFactSource, + IndexChannelTransport, IndexPersistenceOperation, IndexRange, IndexSymbol, + IndexValue, +} from '../../domain/index/model.js' +export type CollectExecutionInput = { + program: ts.Program + sourceFiles: readonly ts.SourceFile[] + checker: ts.TypeChecker + pathToFileId: ReadonlyMap + symbols: IndexSymbol[] + symbolsByFile: ReadonlyMap +} +export type CollectExecutionResult = { + channels: readonly IndexChannelNode[]; edges: readonly IndexEdge[] + diagnostics: readonly IndexDiagnostic[] +} +type Confidence = 'high' | 'medium' | 'low' +type OwnerSpan = { symbol: IndexSymbol; start: number; end: number } +type ImportBinding = { imported: string; module: string; namespace: boolean } +type CallSite = readonly [ + targetId: string, arguments: readonly IndexValue[], node: EffectWitness, +] +type QueueTransport = Extract +type QueueOrigin = readonly [key: IndexValue, transport: QueueTransport] +type EmitterScope = readonly [ + scope: string, + transport: 'node-event-emitter' | 'nestjs-event-emitter', +] +type EffectWitness = ts.CallExpression | ts.NewExpression +type BullEffect = readonly [ + kind: 'bull-publish' | 'bull-consume', queue: IndexValue, + endpoint: IndexValue, transport: QueueTransport, scope: undefined, + witness: EffectWitness, confidence: Confidence, source: IndexFactSource, +] +type EventEffect = readonly [ + kind: 'event-publish' | 'event-consume', event: IndexValue, + handler: IndexValue | undefined, + transport: 'node-event-emitter' | 'nestjs-event-emitter', scope: string, + witness: EffectWitness, confidence: Confidence, source: IndexFactSource, +] +type PersistenceEffect = readonly [ + kind: 'persistence', operation: IndexPersistenceOperation, + resource: IndexValue | undefined, receiverType: string, scope: undefined, + witness: EffectWitness, confidence: Confidence, source: IndexFactSource, +] +type ExecutionEffect = BullEffect | EventEffect | PersistenceEffect +type FileContext = { + sf: ts.SourceFile; fileId: string + imports: ReadonlyMap; owners: readonly OwnerSpan[] +} +type CollectionState = { + input: CollectExecutionInput + symbolsById: ReadonlyMap + facts: Map + overflow: Set + effects: Map + calls: Map + callIds: Map + channels: Map + edges: IndexEdge[] + diagnostics: IndexDiagnostic[] + seenDiagnostics: Set + queues: Map> + emitters: Map + nestQueues: Map> + files: Map +} +const MAX_VALUE_DEPTH = 5, MAX_VALUE_ELEMENTS = 32 +const MAX_STRING_BYTES = 512, MAX_TEXT_BYTES = 256 +const MAX_WRAPPER_HOPS = 2 +const MAX_FACTS_PER_OWNER = 8_192, MAX_EFFECTS_PER_OWNER = 8_192 +const FS_MODULES = new Set(['node:fs', 'node:fs/promises', 'fs', 'fs/promises']) +const FS_OPERATIONS = { + readFile: 'file_read', readFileSync: 'file_read', open: 'file_read', + opendir: 'file_read', readdir: 'file_read', appendFile: 'file_write', + appendFileSync: 'file_write', copyFile: 'file_write', copyFileSync: 'file_write', + rename: 'file_write', writeFile: 'file_write', writeFileSync: 'file_write', + rm: 'delete', rmSync: 'delete', unlink: 'delete', unlinkSync: 'delete', +} as const satisfies Record +const REPOSITORY_OPERATIONS = { + find: 'read', findOne: 'read', findOneBy: 'read', findMany: 'read', + findUnique: 'read', count: 'read', aggregate: 'read', create: 'create', + insert: 'create', save: 'create', createMany: 'create', update: 'update', + updateOne: 'update', updateMany: 'update', delete: 'delete', + deleteOne: 'delete', deleteMany: 'delete', remove: 'delete', upsert: 'upsert', + transaction: 'transaction', $transaction: 'transaction', +} as const satisfies Record +const PROMISE_COMPLETIONS = { + all: 'all_or_first_rejection', allSettled: 'all_settled', + any: 'first_fulfilled', race: 'first_settled', +} as const +const LOGICAL_FLOW = new Map([ + [K.AmpersandAmpersandToken, ['logical_and', 'truthy']], + [K.BarBarToken, ['logical_or', 'falsy']], + [K.QuestionQuestionToken, ['nullish', 'nullish']], +]) +const ARRAY_MUTATIONS = new Map([ + ['push', 'append'], ['unshift', 'append'], ['pop', 'remove'], + ['shift', 'remove'], ['splice', 'remove'], +]) +const FACT_ORDER: Readonly> = { + condition: 0, loop: 1, parallel: 2, call: 3, literal: 4, + mutation: 5, persistence: 6, return: 7, throw: 8, +} +const ASSIGNMENT_OPERATORS = new Set([ + K.EqualsToken, K.PlusEqualsToken, + K.MinusEqualsToken, K.AsteriskEqualsToken, + K.AsteriskAsteriskEqualsToken, K.SlashEqualsToken, + K.PercentEqualsToken, K.LessThanLessThanEqualsToken, + K.GreaterThanGreaterThanEqualsToken, K.GreaterThanGreaterThanGreaterThanEqualsToken, + K.AmpersandEqualsToken, K.BarEqualsToken, K.CaretEqualsToken, + K.BarBarEqualsToken, K.AmpersandAmpersandEqualsToken, + K.QuestionQuestionEqualsToken, +]) +const ARRAY_ITERATION_METHODS = new Set([ + 'every', 'filter', 'find', 'findIndex', 'flatMap', + 'forEach', 'map', 'reduce', 'reduceRight', 'some', +]) +const STRUCTURAL_LITERAL_TOKENS = new Set([ + K.StringLiteral, K.NumericLiteral, + K.BigIntLiteral, K.RegularExpressionLiteral, + K.NoSubstitutionTemplateLiteral, K.TemplateHead, + K.TemplateMiddle, K.TemplateTail, +]) +const SECRET_NAME = /(?:api[_-]?key|authorization|cookie|credential|passwd|password|private[_-]?key|secret|token)/i +const SECRET_VALUE = /^(?:bearer\s+|gh[pousr]_|github_pat_|sk-(?:live|test|proj)-|xox[baprs]-)/i +function sha256(value: string): string { + return createHash('sha256').update(value, 'utf8').digest('hex') +} +function boundedText(value: string, maxBytes = MAX_TEXT_BYTES): string { + if (Buffer.byteLength(value, 'utf8') <= maxBytes) return value + let result = '' + for (const character of value) { + if (Buffer.byteLength(result + character, 'utf8') > maxBytes) break + result += character + } + return result +} +function structuralText(node: ts.Node, sf: ts.SourceFile): string { + const scanner = ts.createScanner(ts.ScriptTarget.Latest, true, sf.languageVariant, node.getText(sf)); + const tokens: string[] = []; + for (let token = scanner.scan(); token !== K.EndOfFileToken; token = scanner.scan()) { + tokens.push(STRUCTURAL_LITERAL_TOKENS.has(token) ? '' : scanner.getTokenText()); + } + return boundedText(tokens.join(' ')); +} +function compareText(left: string, right: string): number { + return left < right ? -1 : left > right ? 1 : 0 +} +function compareOrder(left: readonly number[], right: readonly number[]): number { + const length = Math.min(left.length, right.length) + for (let index = 0; index < length; index += 1) { + const difference = (left[index] ?? 0) - (right[index] ?? 0) + if (difference !== 0) return difference + } + return left.length - right.length +} +function rangeOf(node: ts.Node, sf: ts.SourceFile): IndexRange { + return rangeFromOffsets(sf, node.getStart(sf, false), node.getEnd()) +} +function rangeFromOffsets(sf: ts.SourceFile, start: number, end: number): IndexRange { + const startPosition = sf.getLineAndCharacterOfPosition(start) + const endPosition = sf.getLineAndCharacterOfPosition(end) + return { + start: { line: startPosition.line + 1, column: startPosition.character + 1 }, + end: { line: endPosition.line + 1, column: endPosition.character + 1 }, + } +} +function statementFor(node: ts.Node): ts.Node { + let current: ts.Node = node; + while (current.parent) { + if (ts.isStatement(current) + || isVariable(current) + || isPropertyDecl(current) + || isParameter(current)) { + return current; + } + if (ts.isSourceFile(current.parent)) + return current; + current = current.parent; + } + return current; +} +function evidenceFor(node: ts.Node, sf: ts.SourceFile, fileId: string, stmtNode: ts.Node = statementFor(node), bounds?: { + start: number; + end: number; +}): IndexFactEvidence { + const rawStatementStart = stmtNode.getStart(sf, false); + const rawStatementEnd = stmtNode.getEnd(); + const statementStart = bounds + ? Math.max(rawStatementStart, bounds.start) + : rawStatementStart; + const statementEnd = bounds + ? Math.min(rawStatementEnd, bounds.end) + : rawStatementEnd; + return { + file_id: fileId, + range: rangeOf(node, sf), + statement_range: rangeFromOffsets(sf, statementStart, statementEnd), + excerpt_sha256: sha256(sf.text.slice(statementStart, statementEnd)), + }; +} +function factOrder(kind: IndexBodyFact['kind'], node: ts.Node, suffix = 0): readonly number[] { + const sf = node.getSourceFile() + return [ + node.getStart(sf, false), + FACT_ORDER[kind], + node.getEnd(), + suffix, + ] +} +function factBase(ownerId: string, kind: IndexBodyFact['kind'], node: ts.Node, file: FileContext, control: readonly IndexControlFrame[], opts: { + confidence?: Confidence; + source?: IndexFactSource; + statementNode?: ts.Node; + orderSuffix?: number; +} = {}): Pick { + const ownerBounds = file.owners.find((span) => span.symbol.id === ownerId); + const order = factOrder(kind, node, opts.orderSuffix); + const evidence = evidenceFor(node, file.sf, file.fileId, opts.statementNode, ownerBounds); + return { + id: indexBodyFactId(ownerId, kind, order, evidence.excerpt_sha256), + owner_symbol_id: ownerId, + order, + evidence, + control: [...control], + confidence: opts.confidence ?? 'high', + source: opts.source ?? 'typescript-syntactic', + }; +} +type ConditionKind = Extract['condition_kind'] +type BranchArm = Extract['arm'] +type MutationOperation = Extract['operation'] +function addCondition( + ownerId: string, + conditionKind: ConditionKind, + expr: ts.Expression, + file: FileContext, + ctx: CollectionState, + control: readonly IndexControlFrame[], + stmtNode: ts.Node, +): ReturnType { + const base = factBase(ownerId, 'condition', expr, file, control, { statementNode: stmtNode }) + addFact(ctx, { + ...base, + kind: 'condition', + condition_kind: conditionKind, + test: readValue(expr, file, ctx, { constants: true }), + }) + return base +} +function branchFrame(control: readonly IndexControlFrame[], controllerFactId: string, arm: BranchArm): IndexControlFrame[] { + return [...control, { kind: 'branch', controller_fact_id: controllerFactId, arm }]; +} +function addMutation( + ownerId: string, operationNode: ts.Node, operation: MutationOperation, + targetNode: ts.Node, file: FileContext, ctx: CollectionState, + control: readonly IndexControlFrame[], + value?: ts.Expression, orderSuffix = 0, +): void { + const target = structuralText(targetNode, file.sf) + addFact(ctx, { + ...factBase(ownerId, 'mutation', operationNode, file, control, { orderSuffix }), + kind: 'mutation', + operation, + target: boundedText(target), + ...(value ? { + value: readValue(value, file, ctx, { + constants: true, + secret: SECRET_NAME.test(target), + }), + } : {}), + }) +} +function addIteration(ownerId: string, node: ts.Node, file: FileContext, ctx: CollectionState, control: readonly IndexControlFrame[]): string { + const base = factBase(ownerId, 'loop', node, file, control, { orderSuffix: 1 }); + addFact(ctx, { ...base, kind: 'loop', loop_kind: 'array_iteration' }); + return base.id; +} +function addFact(ctx: CollectionState, fact: IndexBodyFact): void { + if (ctx.overflow.has(fact.owner_symbol_id)) return + const facts = ctx.facts.get(fact.owner_symbol_id) + if (!facts) { ctx.facts.set(fact.owner_symbol_id, [fact]); return } + if (facts.length >= MAX_FACTS_PER_OWNER) { + ctx.overflow.add(fact.owner_symbol_id); return + } + facts.push(fact) +} +function addBounded( + ctx: CollectionState, map: Map, key: string, value: T, +): void { + const values = map.get(key) + if (!values) { map.set(key, [value]); return } + if (values.length >= MAX_EFFECTS_PER_OWNER) { + ctx.overflow.add(key) + } else { + values.push(value) + } +} +function addEffect(ctx: CollectionState, ownerId: string, fx: ExecutionEffect): void { + addBounded(ctx, ctx.effects, ownerId, fx) +} +function addCall(ctx: CollectionState, ownerId: string, callSite: CallSite): void { + addBounded(ctx, ctx.calls, ownerId, callSite); +} +function isOwnerSymbol(symbol: IndexSymbol): boolean { + if (!['function', 'method', 'constant', 'variable'].includes(symbol.kind)) return false + // Execution facts require an authenticated owner span. Framework-only + // synthetic nodes without declaration/definition ranges remain topology + // nodes and must not become evidence owners. + if (!symbol.declaration_range) return false + if (symbol.framework_metadata?.external_call === true) return false + if (typeof symbol.framework_metadata?.storage_operation === 'string') return false + return true +} +function offsetOf(sf: ts.SourceFile, position: IndexRange['start']): number { + return sf.getPositionOfLineAndCharacter(position.line - 1, position.column - 1) +} +function owners(sf: ts.SourceFile, symbols: readonly IndexSymbol[]): OwnerSpan[] { + return symbols + .filter(isOwnerSymbol) + .map((symbol) => ({ + symbol, + start: offsetOf(sf, symbol.range.start), + end: offsetOf(sf, symbol.range.end), + })) + .sort((left, right) => (left.end - left.start) - (right.end - right.start) + || left.start - right.start + || compareText(left.symbol.id, right.symbol.id)); +} +function ownerFor(node: ts.Node, file: FileContext): IndexSymbol | null { + const start = node.getStart(file.sf, false) + const end = node.getEnd() + return file.owners.find((span) => span.start <= start && span.end >= end)?.symbol ?? null +} +function importsFor(sf: ts.SourceFile): ReadonlyMap { + const bindings = new Map() + for (const stmt of sf.statements) { + if (!ts.isImportDeclaration(stmt) || !ts.isStringLiteral(stmt.moduleSpecifier)) continue + const module = stmt.moduleSpecifier.text + const clause = stmt.importClause + if (!clause) continue + if (clause.name) { + bindings.set(clause.name.text, { imported: 'default', module, namespace: false }) + } + const named = clause.namedBindings + if (!named) continue + if (ts.isNamespaceImport(named)) { + bindings.set(named.name.text, { imported: '*', module, namespace: true }) + continue + } + for (const element of named.elements) { + bindings.set(element.name.text, { + imported: element.propertyName?.text ?? element.name.text, + module, + namespace: false, + }) + } + } + return bindings +} +function importedBinding(expr: ts.Expression, file: FileContext): ImportBinding | null { + if (isIdentifier(expr)) + return file.imports.get(expr.text) ?? null; + if (isAccess(expr) + && isIdentifier(expr.expression)) { + const namespace = file.imports.get(expr.expression.text); + if (namespace?.namespace) { + return { + imported: expr.name.text, + module: namespace.module, + namespace: false, + }; + } + } + return null; +} +function isImport(expr: ts.Expression, file: FileContext, modules: readonly string[], names: readonly string[]): boolean { + const binding = importedBinding(expr, file); + return binding !== null + && modules.includes(binding.module) + && names.includes(binding.imported); +} +function followAlias(symbol: ts.Symbol | undefined, checker: ts.TypeChecker): ts.Symbol | undefined { + if (!symbol || (symbol.flags & ts.SymbolFlags.Alias) === 0) return symbol + try { + return checker.getAliasedSymbol(symbol) + } catch { + return symbol + } +} +function symbolAt(node: ts.Node, ctx: CollectionState): ts.Symbol | undefined { + const checker = ctx.input.checker + return followAlias(checker.getSymbolAtLocation(node), checker) +} +function declSymbol(node: ts.Node, file: FileContext, ctx: CollectionState): IndexSymbol | null { + const sf = node.getSourceFile(); + const fileId = ctx.input.pathToFileId.get(sf.fileName); + if (!fileId) + return null; + const spans = sf === file.sf + ? file.owners + : owners(sf, ctx.input.symbolsByFile.get(fileId) ?? []); + const start = node.getStart(sf, false); + const end = node.getEnd(); + return spans.find((span) => span.start <= start && span.end >= end)?.symbol ?? null; +} +function symbolFor(expr: ts.Expression, file: FileContext, ctx: CollectionState): IndexSymbol | null { + const symbol = symbolAt(isAccess(expr) ? expr.name : expr, ctx); + const declarations = symbol?.declarations ?? []; + for (const decl of declarations) { + const indexed = declSymbol(decl, file, ctx); + if (indexed) + return indexed; + } + return null; +} +function callSymbol(call: ts.CallExpression | ts.NewExpression, file: FileContext, ctx: CollectionState): IndexSymbol | null { + const signature = ctx.input.checker.getResolvedSignature(call); + const decl = signature?.getDeclaration(); + if (decl && !decl.getSourceFile().isDeclarationFile) { + const indexed = declSymbol(decl, file, ctx); + if (indexed) + return indexed; + } + return symbolFor(call.expression, file, ctx); +} +function callableOwner(node: ts.SignatureDeclaration, file: FileContext, ctx: CollectionState): IndexSymbol | null { + if (isArrow(node) || isFunction(node)) { + const parent = node.parent; + if (isVariable(parent) && parent.initializer === node) { + const stmt = parent.parent.parent; + return ts.isVariableStatement(stmt) + && ts.isSourceFile(stmt.parent) + ? declSymbol(parent, file, ctx) + : null; + } + return isBinary(parent) + ? declSymbol(node, file, ctx) + : null; + } + return ts.isFunctionDeclaration(node) + || ts.isMethodDeclaration(node) + || ts.isConstructorDeclaration(node) + || ts.isGetAccessorDeclaration(node) + || ts.isSetAccessorDeclaration(node) + ? declSymbol(node, file, ctx) + : null; +} +function parameterValue(identifier: ts.Identifier, file: FileContext, ctx: CollectionState): IndexValue | null { + const symbol = symbolAt(identifier, ctx); + for (const decl of symbol?.declarations ?? []) { + if (!isParameter(decl)) + continue; + const parent = decl.parent; + if (!ts.isFunctionLike(parent)) + continue; + const position = parent.parameters.indexOf(decl); + if (position >= 0) { + return callableOwner(parent, file, ctx) + ? { kind: 'parameter', position } + : { kind: 'parameter', position, scope: 'iteration' }; + } + } + return null; +} +function redactedValue(value: string): IndexValue { + return { + kind: 'redacted', + sha256: sha256(value), + byte_length: Buffer.byteLength(value, 'utf8'), + } +} +function literalString(value: string, secret = false): IndexValue { + const byteLength = Buffer.byteLength(value, 'utf8') + if (secret || SECRET_VALUE.test(value) || byteLength > MAX_STRING_BYTES) { + return redactedValue(value) + } + return { kind: 'literal', value } +} +function unknown(reason: 'dynamic' | 'ambiguous' | 'unsupported' = 'dynamic'): IndexValue { + return { kind: 'unknown', reason } +} +function unwrap(node: ts.Expression): ts.Expression { + let current = node; + while (ts.isAsExpression(current) + || ts.isTypeAssertionExpression(current) + || ts.isNonNullExpression(current) + || ts.isParenthesizedExpression(current) + || ts.isSatisfiesExpression(current)) { + current = current.expression; + } + return current; +} +type ValueOptions = { + constants?: boolean + secret?: boolean + depth?: number + seen?: ReadonlySet +} +function readValue( + expr: ts.Expression, + file: FileContext, + ctx: CollectionState, + opts: ValueOptions = {}, +): IndexValue { + const depth = opts.depth ?? 0 + if (depth >= MAX_VALUE_DEPTH) return unknown('unsupported') + const seen = new Set(opts.seen ?? []) + const node = unwrap(expr) + if (seen.has(node)) return unknown('ambiguous') + seen.add(node) + const nested = (value: ts.Expression, extra: Partial = {}): IndexValue => + readValue(value, file, ctx, { + ...opts, + ...extra, + depth: depth + 1, + seen, + }) + if (ts.isStringLiteralLike(node)) return literalString(node.text, opts.secret) + if (isNumeric(node)) { + const value = Number(node.text) + return Number.isFinite(value) && !Object.is(value, -0) + ? { kind: 'literal', value } + : unknown('unsupported') + } + if (node.kind === K.TrueKeyword) return { kind: 'literal', value: true } + if (node.kind === K.FalseKeyword) return { kind: 'literal', value: false } + if (node.kind === K.NullKeyword) return { kind: 'literal', value: null } + if (ts.isPrefixUnaryExpression(node) && isNumeric(node.operand)) { + const value = Number(node.operand.text) + const signed = node.operator === K.MinusToken ? -value : value + if ((node.operator === K.MinusToken + || node.operator === K.PlusToken) + && Number.isFinite(signed) && !Object.is(signed, -0)) { + return { kind: 'literal', value: signed } + } + } + if (isIdentifier(node)) { + const parameter = parameterValue(node, file, ctx) + if (parameter) return parameter + const symbol = symbolAt(node, ctx) + const decl = symbol?.valueDeclaration + ?? symbol?.declarations?.find((candidate) => isVariable(candidate)) + if ( + opts.constants + && decl + && isVariable(decl) + && decl.initializer + ) { + return nested(decl.initializer, { + secret: opts.secret || SECRET_NAME.test(node.text), + }) + } + const indexed = decl + ? declSymbol(decl, file, ctx) + : symbolFor(node, file, ctx) + return indexed ? { kind: 'symbol', symbol_id: indexed.id } : unknown() + } + if (ts.isArrayLiteralExpression(node)) { + if (node.elements.length > MAX_VALUE_ELEMENTS) return unknown('unsupported') + const elements: IndexValue[] = [] + for (const element of node.elements) { + if (ts.isSpreadElement(element) || ts.isOmittedExpression(element)) return unknown('unsupported') + elements.push(nested(element, { constants: true })) + } + return { kind: 'array', elements } + } + if (ts.isObjectLiteralExpression(node)) { + if (node.properties.length > MAX_VALUE_ELEMENTS) return unknown('unsupported') + const entries = new Map() + for (const property of node.properties) { + if (ts.isPropertyAssignment(property)) { + const key = propertyName(property.name) + if (key === null || key.includes('\0') + || Buffer.byteLength(key, 'utf8') > MAX_STRING_BYTES) { + return unknown('unsupported') + } + entries.set( + key, + nested(property.initializer, { + constants: true, + secret: SECRET_NAME.test(key), + }), + ) + } else if (ts.isShorthandPropertyAssignment(property)) { + const key = property.name.text + if (Buffer.byteLength(key, 'utf8') > MAX_STRING_BYTES) { + return unknown('unsupported') + } + entries.set( + key, + nested(property.name, { + constants: true, + secret: SECRET_NAME.test(key), + }), + ) + } else { + return unknown('unsupported') + } + } + return { + kind: 'object', + entries: [...entries].map(([key, value]) => ({ key, value })), + } + } + if (ts.isNoSubstitutionTemplateLiteral(node)) return literalString(node.text, opts.secret) + if (ts.isTemplateExpression(node)) { + if (1 + (2 * node.templateSpans.length) > MAX_VALUE_ELEMENTS) { + return unknown('unsupported') + } + const parts: IndexValue[] = [literalString(node.head.text, opts.secret)] + for (const span of node.templateSpans) { + parts.push(nested(span.expression, { constants: true })) + parts.push(literalString(span.literal.text, opts.secret)) + } + return { kind: 'template', parts } + } + if (isCall(node) && isAccess(node.expression)) { + const method = node.expression.name.text + const receiver = node.expression.expression + if (method === 'slice') { + const value = nested(receiver, { constants: true }) + if (value.kind !== 'array') return unknown() + const start = numericIndex(node.arguments[0], file, ctx) + const end = numericIndex(node.arguments[1], file, ctx) + if (start === null || (node.arguments[1] && end === null)) return unknown() + return { kind: 'array', elements: value.elements.slice(start, end ?? undefined) } + } + if (method === 'map') { + return nested(receiver, { constants: false }) + } + } + if (ts.isElementAccessExpression(node)) { + const collection = nested(node.expression, { constants: true }) + const index = node.argumentExpression + ? numericIndex(node.argumentExpression, file, ctx) + : null + if (collection.kind === 'array' && index !== null) { + return collection.elements[index] ?? unknown() + } + } + const target = symbolFor(node, file, ctx) + return target ? { kind: 'symbol', symbol_id: target.id } : unknown() +} +function propertyName(name: ts.PropertyName): string | null { + if (isIdentifier(name) || ts.isStringLiteralLike(name) || isNumeric(name)) { + return name.text + } + return null +} +function numericIndex(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): number | null { + if (!expr) + return 0; + const value = readValue(expr, file, ctx, { constants: true }); + return value.kind === 'literal' + && typeof value.value === 'number' + && Number.isSafeInteger(value.value) + ? value.value + : null; +} +function staticString(value: IndexValue): string | null { + if (value.kind === 'literal' && typeof value.value === 'string') { + return value.value.length > 0 && Buffer.byteLength(value.value, 'utf8') <= MAX_TEXT_BYTES + ? value.value + : null + } + if (value.kind !== 'template') return null + let result = '' + for (const part of value.parts) { + if (part.kind !== 'literal' + || !['string', 'number', 'boolean'].includes(typeof part.value)) return null + result += String(part.value) + } + return result.length > 0 && Buffer.byteLength(result, 'utf8') <= MAX_TEXT_BYTES + ? result + : null +} +function symbolId(value: IndexValue): string | null { + return value.kind === 'symbol' ? value.symbol_id : null +} +function mapValue(value: IndexValue, depth: number, resolve?: (position: number) => IndexValue): IndexValue { + if (value.kind === 'parameter' && value.scope !== 'iteration' && resolve) { + return mapValue(resolve(value.position), depth); + } + if (value.kind === 'array') { + if (depth >= MAX_VALUE_DEPTH && value.elements.length > 0) + return unknown('unsupported'); + return { + kind: 'array', + elements: value.elements.map((entry) => mapValue(entry, depth + 1, resolve)), + }; + } + if (value.kind === 'object') { + if (depth >= MAX_VALUE_DEPTH && value.entries.length > 0) + return unknown('unsupported'); + return { + kind: 'object', + entries: value.entries.map((entry) => ({ + key: entry.key, + value: mapValue(entry.value, depth + 1, resolve), + })), + }; + } + if (value.kind === 'template') { + if (depth >= MAX_VALUE_DEPTH && value.parts.length > 0) + return unknown('unsupported'); + return { + kind: 'template', + parts: value.parts.map((entry) => mapValue(entry, depth + 1, resolve)), + }; + } + return value; +} +function substitute(value: IndexValue, args: readonly IndexValue[]): IndexValue { + return mapValue(value, 0, (position) => args[position] ?? unknown()) +} +function instantiateEffect(fx: ExecutionEffect, args: readonly IndexValue[], witness: EffectWitness): ExecutionEffect { + switch (fx[0]) { + case 'bull-publish': + case 'bull-consume': + return [ + fx[0], substitute(fx[1], args), substitute(fx[2], args), + fx[3], undefined, witness, fx[6], 'wrapper-summary', + ]; + case 'event-publish': + return [ + fx[0], substitute(fx[1], args), undefined, fx[3], fx[4], + witness, fx[6], 'wrapper-summary', + ]; + case 'event-consume': + return [ + fx[0], substitute(fx[1], args), substitute(fx[2]!, args), + fx[3], fx[4], witness, fx[6], 'wrapper-summary', + ]; + case 'persistence': + return [ + fx[0], fx[1], fx[2] ? substitute(fx[2], args) : undefined, + fx[3], undefined, witness, fx[6], 'wrapper-summary', + ]; + } +} +function callName(call: ts.CallExpression | ts.NewExpression): string { + const sf = call.getSourceFile() + const text = structuralText(call.expression, sf) + return boundedText(isNew(call) ? `new ${text}` : text) +} +function callScheduling(call: ts.CallExpression | ts.NewExpression): IndexCallFact['scheduling'] { + let current: ts.Node = call; + while (ts.isParenthesizedExpression(current.parent) + || ts.isAsExpression(current.parent) + || ts.isNonNullExpression(current.parent)) { + current = current.parent; + } + if (ts.isAwaitExpression(current.parent)) + return 'awaited'; + if (ts.isVoidExpression(current.parent) + || ts.isExpressionStatement(current.parent)) { + return 'fire_and_forget'; + } + return 'sync'; +} +function callFact( + call: ts.CallExpression | ts.NewExpression, + sym: IndexSymbol, + file: FileContext, + ctx: CollectionState, + control: readonly IndexControlFrame[], +): IndexCallFact { + const target = callSymbol(call, file, ctx) + const args = (call.arguments ?? []).map((argument) => { + const value = unwrap(argument) + return isArrow(value) || isFunction(value) + ? handlerValue(value, file, ctx) + : readValue(value, file, ctx, { + constants: true, + secret: SECRET_NAME.test(value.getText(file.sf)), + }) + }) + const fact: IndexCallFact = { + ...factBase(sym.id, 'call', call, file, control, { + confidence: target ? 'high' : 'medium', + source: target ? 'typescript-semantic' : 'typescript-syntactic', + }), + kind: 'call', + callee: callName(call), + ...(target ? { target_symbol_id: target.id } : {}), + arguments: args, + scheduling: callScheduling(call), + } + addFact(ctx, fact) + ctx.callIds.set(call, fact.id) + if (target) { + addCall(ctx, sym.id, [target.id, fact.arguments, call]) + } + return fact +} +function receiverType( + expr: ts.Expression, + ctx: CollectionState, +): string { + try { + return boundedText( + ctx.input.checker.typeToString( + ctx.input.checker.getTypeAtLocation(expr), + undefined, + ts.TypeFormatFlags.NoTruncation, + ).replace(/(["'`])(?:\\[\s\S]|(?!\1)[^\\])*\1/gu, ''), + ) + } catch { + return '' + } +} +function typeImport(expr: ts.Expression, file: FileContext, ctx: CollectionState): ImportBinding | null { + const symbol = symbolAt(expr, ctx); + for (const decl of symbol?.declarations ?? []) { + const typeNode = isParameter(decl) + || isPropertyDecl(decl) + || isVariable(decl) + ? decl.type + : undefined; + if (!typeNode) + continue; + const root = isTypeReference(typeNode) + ? (ts.isQualifiedName(typeNode.typeName) ? typeNode.typeName.left : typeNode.typeName) + : null; + if (root && isIdentifier(root)) { + const binding = file.imports.get(root.text); + if (binding) + return binding; + } + } + return null; +} +function bullTransport(binding: ImportBinding | null): QueueTransport | null { + if (!binding || !['bull', 'bullmq'].includes(binding.module) + || !['Queue', 'default'].includes(binding.imported)) return null + return binding.module as QueueTransport +} +function queueTypeTransport( + node: ts.TypeNode | undefined, + file: FileContext, +): QueueTransport | null { + if (!node || !isTypeReference(node)) return null + const name = node.typeName + if (isIdentifier(name)) return bullTransport(file.imports.get(name.text) ?? null) + if (!isIdentifier(name.left) || name.right.text !== 'Queue') return null + const binding = file.imports.get(name.left.text) + return binding?.namespace ? bullTransport({ ...binding, imported: 'Queue' }) : null +} +function mapQueueTransport(expr: ts.Expression, file: FileContext, ctx: CollectionState): QueueTransport | null { + const target = isAccess(expr) + ? expr.name + : expr; + const symbol = symbolAt(target, ctx); + const decl = symbol?.valueDeclaration; + const declFile = decl + ? contextFor(decl.getSourceFile(), ctx) + : null; + if (!decl || !declFile) + return null; + const type = (isVariable(decl) + || isPropertyDecl(decl) + || isParameter(decl)) + ? decl.type + : undefined; + const init = (isVariable(decl) + || isPropertyDecl(decl)) + ? decl.initializer + : undefined; + const mapType = type && isTypeReference(type) + ? type.typeArguments?.[1] + : init && isNew(unwrap(init)) + ? (unwrap(init) as ts.NewExpression).typeArguments?.[1] + : undefined; + return queueTypeTransport(mapType, declFile); +} +function contextFor(sf: ts.SourceFile, ctx: CollectionState): FileContext | null { + return ctx.files.get(sf) ?? null; +} +function queueOrigin(expr: ts.Expression, sym: IndexSymbol, file: FileContext, ctx: CollectionState, seen: ReadonlySet = new Set()): QueueOrigin | null { + const node = unwrap(expr); + if (seen.has(node)) + return null; + const nextSeen = new Set(seen); + nextSeen.add(node); + if (isNew(node) && node.arguments?.[0]) { + const transport = bullTransport(importedBinding(node.expression, file)); + if (transport) { + return [ + readValue(node.arguments[0], file, ctx, { constants: true }), + transport, + ]; + } + } + if (isIdentifier(node)) { + const local = ctx.queues.get(sym.id)?.get(node.text); + if (local) + return local; + const symbol = symbolAt(node, ctx); + const decl = symbol?.valueDeclaration; + if (decl + && isVariable(decl) + && decl.initializer) { + const declFile = contextFor(decl.getSourceFile(), ctx); + return declFile + ? queueOrigin(decl.initializer, sym, declFile, ctx, nextSeen) + : null; + } + } + if (isAccess(node) + && node.expression.kind === K.ThisKeyword) { + const className = sym.kind === 'method' + ? sym.name.slice(0, sym.name.lastIndexOf('.')) + : sym.name; + const injected = ctx.nestQueues + .get(`${sym.file_id}\0${className}`)?.get(node.name.text); + if (injected) + return injected; + const symbol = symbolAt(node.name, ctx); + const decl = symbol?.valueDeclaration; + if (decl && isPropertyDecl(decl) + && decl.initializer) { + const declFile = contextFor(decl.getSourceFile(), ctx); + if (declFile) { + return queueOrigin(decl.initializer, sym, declFile, ctx, nextSeen); + } + } + } + if (isCall(node) + && isAccess(node.expression) + && node.expression.name.text === 'get' + && node.arguments[0]) { + const transport = mapQueueTransport(node.expression.expression, file, ctx); + if (transport) { + return [ + readValue(node.arguments[0], file, ctx, { constants: true }), + transport, + ]; + } + } + return null; +} +function emitterScope(expr: ts.Expression, sym: IndexSymbol, file: FileContext, ctx: CollectionState, seen: ReadonlySet = new Set()): EmitterScope | null { + const node = unwrap(expr); + if (seen.has(node)) + return null; + const nextSeen = new Set(seen); + nextSeen.add(node); + if (isIdentifier(node)) { + const symbol = symbolAt(node, ctx); + const decl = symbol?.valueDeclaration; + if (decl) { + const indexed = declSymbol(decl, file, ctx); + if (indexed) { + const cached = ctx.emitters.get(indexed.id); + if (cached) + return cached; + } + if (isVariable(decl) + && decl.initializer + && isNew(unwrap(decl.initializer))) { + const init = unwrap(decl.initializer) as ts.NewExpression; + const declFile = contextFor(decl.getSourceFile(), ctx); + if (!declFile) + return null; + const transport = emitterTransport(init.expression, declFile); + if (transport) { + const scope = indexed?.id + ?? `${sym.id}:local:${node.text}:${decl.getStart(declFile.sf, false)}`; + const value: EmitterScope = [scope, transport]; + if (indexed) + ctx.emitters.set(indexed.id, value); + return value; + } + } + } + } + if (isAccess(node) + && node.expression.kind === K.ThisKeyword) { + const binding = typeImport(node, file, ctx); + const transport = binding?.module === '@nestjs/event-emitter' + ? 'nestjs-event-emitter' + : binding?.module === 'node:events' + || binding?.module === 'events' + ? 'node-event-emitter' + : null; + const decl = symbolAt(node.name, ctx)?.valueDeclaration; + const inferred = decl && isPropertyDecl(decl) + && decl.initializer + && isNew(unwrap(decl.initializer)) + ? emitterTransport((unwrap(decl.initializer) as ts.NewExpression).expression, contextFor(decl.getSourceFile(), ctx) ?? file) + : null; + if (transport || inferred) { + const className = sym.kind === 'method' + ? sym.name.slice(0, sym.name.lastIndexOf('.')) + : sym.name; + return [ + `${file.fileId}:${className}.${node.name.text}`, + transport ?? inferred!, + ]; + } + } + return null; +} +function emitterTransport(expr: ts.Expression, file: FileContext): 'node-event-emitter' | 'nestjs-event-emitter' | null { + if (isImport(expr, file, ['node:events', 'events'], ['EventEmitter'])) { + return 'node-event-emitter'; + } + if (isImport(expr, file, ['@nestjs/event-emitter'], ['EventEmitter2'])) { + return 'nestjs-event-emitter'; + } + return null; +} +function handlerValue(expr: ts.Expression, file: FileContext, ctx: CollectionState): IndexValue { + const node = unwrap(expr); + if (isArrow(node) || isFunction(node)) { + const targets = new Set(); + const visit = (child: ts.Node): void => { + if (child !== node && (isArrow(child) || isFunction(child))) + return; + if (isCall(child)) { + const target = callSymbol(child, file, ctx); + if (target) + targets.add(target.id); + } + ts.forEachChild(child, visit); + }; + visit(node.body); + return targets.size === 1 + ? { kind: 'symbol', symbol_id: [...targets][0]! } + : unknown(targets.size > 1 ? 'ambiguous' : 'dynamic'); + } + return readValue(node, file, ctx, { constants: false }); +} +type PersistenceSummary = readonly [ + operation: IndexPersistenceOperation, + resource: IndexValue | undefined, + receiverType: string, +] +function storageEffect( + operation: IndexPersistenceOperation | null, + receiverType: string, + resource: ts.Expression | undefined, + file: FileContext, + ctx: CollectionState, +): PersistenceSummary | null { + return operation ? [ + operation, + resource ? readValue(resource, file, ctx, { constants: true }) : undefined, + receiverType, + ] : null +} +function persistenceOperation(call: ts.CallExpression, file: FileContext, ctx: CollectionState): PersistenceSummary | null { + const expr = call.expression; + if (isIdentifier(expr)) { + const binding = file.imports.get(expr.text); + if (binding && FS_MODULES.has(binding.module)) { + return storageEffect(fsOperation(binding.imported), `${binding.module}:${binding.imported}`, call.arguments[0], file, ctx); + } + } + if (!isAccess(expr)) + return null; + const method = expr.name.text; + const receiver = expr.expression; + const root = leftmostIdentifier(receiver); + const namespace = root ? file.imports.get(root.text) : undefined; + if (namespace?.namespace && FS_MODULES.has(namespace.module)) { + const summary = storageEffect(fsOperation(method), `${namespace.module}:namespace`, call.arguments[0], file, ctx); + if (summary) + return summary; + } + const type = receiverType(receiver, ctx); + const typeBinding = typeImport(receiver, file, ctx); + if (typeBinding?.module === 'typeorm' + && ['Repository', 'MongoRepository'].includes(typeBinding.imported) + || packageDecl(receiver, ctx, '/node_modules/typeorm/')) { + const summary = storageEffect(repoOperation(method), type || `${typeBinding?.module}:${typeBinding?.imported}`, call.arguments[0], file, ctx); + if (summary) + return summary; + } + if (packageDecl(receiver, ctx, '/node_modules/@prisma/client/', '/node_modules/.prisma/client/')) { + const summary = storageEffect(repoOperation(method), type || 'PrismaClient', call.arguments[0], file, ctx); + if (summary) + return summary; + } + if (method !== 'send' + || !(typeBinding?.module === '@aws-sdk/client-s3' + && typeBinding.imported === 'S3Client' + || packageDecl(receiver, ctx, '/node_modules/@aws-sdk/client-s3/'))) + return null; + const command = call.arguments[0]; + if (command && isNew(unwrap(command))) { + const constructor = unwrap(command) as ts.NewExpression; + const binding = importedBinding(constructor.expression, file); + if (binding?.module === '@aws-sdk/client-s3') { + const writes = ['PutObjectCommand', 'UploadPartCommand', 'CompleteMultipartUploadCommand']; + const reads = ['GetObjectCommand', 'HeadObjectCommand']; + return storageEffect(writes.includes(binding.imported) ? 'object_write' + : reads.includes(binding.imported) ? 'object_read' : null, type, constructor.arguments?.[0], file, ctx); + } + } + return null; +} +function leftmostIdentifier(expr: ts.Expression): ts.Identifier | null { + let current = expr + while (isAccess(current)) current = current.expression + return isIdentifier(current) ? current : null +} +function packageDecl(expr: ts.Expression, ctx: CollectionState, ...packagePaths: readonly string[]): boolean { + const checker = ctx.input.checker; + let current: ts.Expression = expr; + while (true) { + try { + const type = checker.getTypeAtLocation(current); + const symbols = [type.aliasSymbol, type.getSymbol()]; + if (symbols.some((symbol) => symbol?.declarations?.some((decl) => { + const path = decl.getSourceFile().fileName.replaceAll('\\', '/'); + return packagePaths.some((packagePath) => path.includes(packagePath)); + }))) { + return true; + } + } + catch { + return false; + } + if (!isAccess(current)) + return false; + current = current.expression; + } +} +function namedOperation(method: string, operations: Readonly>): IndexPersistenceOperation | null { + return Object.hasOwn(operations, method) ? operations[method]! : null; +} +const fsOperation = (method: string): IndexPersistenceOperation | null => namedOperation(method, FS_OPERATIONS); +const repoOperation = (method: string): IndexPersistenceOperation | null => namedOperation(method, REPOSITORY_OPERATIONS); +function recordEffects(call: ts.CallExpression | ts.NewExpression, sym: IndexSymbol, file: FileContext, ctx: CollectionState): void { + const proof = [call, 'high', 'framework'] as const; + if (isNew(call)) { + if (isImport(call.expression, file, ['bullmq'], ['Worker']) + && call.arguments?.[0] + && call.arguments[1]) { + addEffect(ctx, sym.id, [ + 'bull-consume', + readValue(call.arguments[0], file, ctx, { constants: true }), + handlerValue(call.arguments[1], file, ctx), + 'bullmq', undefined, ...proof, + ]); + } + return; + } + if (isAccess(call.expression)) { + const method = call.expression.name.text; + const receiver = call.expression.expression; + if (method === 'add' && call.arguments[0]) { + const queue = queueOrigin(receiver, sym, file, ctx); + if (queue) { + addEffect(ctx, sym.id, [ + 'bull-publish', queue[0], + readValue(call.arguments[0], file, ctx, { constants: true }), + queue[1], undefined, ...proof, + ]); + } + } + const emitter = emitterScope(receiver, sym, file, ctx); + if (emitter && method === 'emit' && call.arguments[0]) { + addEffect(ctx, sym.id, [ + 'event-publish', + readValue(call.arguments[0], file, ctx, { constants: true }), + undefined, emitter[1], emitter[0], ...proof, + ]); + } + else if (emitter + && ['addListener', 'on', 'once', 'prependListener'].includes(method) + && call.arguments[0] + && call.arguments[1]) { + addEffect(ctx, sym.id, [ + 'event-consume', + readValue(call.arguments[0], file, ctx, { constants: true }), + handlerValue(call.arguments[1], file, ctx), + emitter[1], emitter[0], ...proof, + ]); + } + } + const persistence = persistenceOperation(call, file, ctx); + if (persistence) { + addEffect(ctx, sym.id, ['persistence', ...persistence, undefined, ...proof]); + } +} +function promiseCombinator(call: ts.CallExpression, ctx: CollectionState): { + combinator: 'all' | 'allSettled' | 'any' | 'race'; + completion: 'all_or_first_rejection' | 'all_settled' | 'first_fulfilled' | 'first_settled'; +} | null { + if (!isAccess(call.expression) + || !isIdentifier(call.expression.expression) + || call.expression.expression.text !== 'Promise') { + return null; + } + const symbol = symbolAt(call.expression.expression, ctx); + if (!symbol?.declarations?.some((decl) => decl.getSourceFile().isDeclarationFile + && /\/typescript\/lib\/lib\..+\.d\.ts$/u.test(decl.getSourceFile().fileName.replaceAll('\\', '/')))) + return null; + const combinator = call.expression.name.text; + return Object.hasOwn(PROMISE_COMPLETIONS, combinator) + ? { + combinator: combinator as keyof typeof PROMISE_COMPLETIONS, + completion: PROMISE_COMPLETIONS[combinator as keyof typeof PROMISE_COMPLETIONS], + } + : null; +} +function parallelSource(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): IndexValue | undefined { + if (!expr) + return undefined; + const node = unwrap(expr); + if (isCall(node) + && isAccess(node.expression) + && node.expression.name.text === 'map') { + return readValue(node.expression.expression, file, ctx, { constants: true }); + } + return readValue(node, file, ctx, { constants: true }); +} +function directParallelLanes(expr: ts.Expression | undefined): readonly ts.Expression[] { + if (!expr) + return []; + const node = unwrap(expr); + if (!ts.isArrayLiteralExpression(node)) + return []; + return node.elements.flatMap((element) => ts.isOmittedExpression(element) || ts.isSpreadElement(element) + ? [] + : [element]); +} +function mappedInput(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): { + call: ts.CallExpression; + input: IndexValue; + receiver: ts.Expression; +} | null { + if (!expr) + return null; + const node = unwrap(expr); + if (!isCall(node) + || !isAccess(node.expression) + || node.expression.name.text !== 'map') + return null; + const input = readValue(node.expression.expression, file, ctx, { constants: true }); + return input.kind === 'array' + ? { call: node, input, receiver: node.expression.expression } + : null; +} +function exits(stmt: ts.Statement): boolean { + if (ts.isReturnStatement(stmt) || ts.isThrowStatement(stmt)) return true + if (ts.isBlock(stmt)) { + const last = stmt.statements.at(-1) + return last ? exits(last) : false + } + if (isIf(stmt) && stmt.elseStatement) { + return exits(stmt.thenStatement) && exits(stmt.elseStatement) + } + return false +} +function guardContinuation( + stmt: ts.IfStatement, +): BranchArm | 'unreachable' | null { + const thenExits = exits(stmt.thenStatement) + const elseExits = stmt.elseStatement + ? exits(stmt.elseStatement) + : false + if (thenExits && elseExits) return 'unreachable' + if (thenExits) return 'else' + return elseExits ? 'then' : null +} +function topLevelValue(decl: ts.VariableDeclaration, sym: IndexSymbol, file: FileContext, ctx: CollectionState): IndexValue | null { + if (!decl.initializer || decl.parent.parent.parent !== file.sf) + return null; + const value = readValue(decl.initializer, file, ctx, { + constants: true, + secret: isIdentifier(decl.name) && SECRET_NAME.test(decl.name.text), + }); + if (value.kind === 'unknown' || value.kind === 'symbol' || value.kind === 'parameter') + return null; + if (value.kind === 'literal' + && typeof value.value === 'string' + && value.value.length === 0) { + return null; + } + return sym.id === declSymbol(decl, file, ctx)?.id ? value : null; +} +function collectFacts(file: FileContext, ctx: CollectionState): void { + const visit = ( + node: ts.Node, + control: readonly IndexControlFrame[], + executeCallable = false, + ): void => { + if (ts.isDecorator(node)) return + const sym = ownerFor(node, file) + if (ts.isFunctionLike(node) + && callableOwner(node, file, ctx)?.id !== sym?.id + && !executeCallable) return + if (sym && control.length > INDEX_BODY_FACT_CONTROL_LIMIT) { + ctx.overflow.add(sym.id) + return + } + if (sym && ts.isBlock(node)) { + let nextControl = control + for (const stmt of node.statements) { + visit(stmt, nextControl) + if (!isIf(stmt)) { + if (exits(stmt)) break + continue + } + const continuation = guardContinuation(stmt) + if (continuation === 'unreachable') break + if (continuation) { + const base = factBase( + sym.id, + 'condition', + stmt.expression, + file, + nextControl, + { statementNode: stmt }, + ) + nextControl = branchFrame(nextControl, base.id, continuation) + } + } + return + } + if (sym && isVariable(node) && isIdentifier(node.name)) { + const value = topLevelValue(node, sym, file, ctx) + if (value) { + addFact(ctx, { + ...factBase(sym.id, 'literal', node.initializer!, file, control, { + statementNode: statementFor(node), + }), + kind: 'literal', + value, + role: 'initializer', + }) + } + if (node.initializer) { + const queue = queueOrigin(node.initializer, sym, file, ctx) + if (queue) { + const origins = ctx.queues.get(sym.id) ?? new Map() + origins.set(node.name.text, queue) + ctx.queues.set(sym.id, origins) + } + } + } + if (sym && isIf(node)) { + const continuation = guardContinuation(node) + const base = addCondition( + sym.id, continuation ? 'guard' : 'if', + node.expression, file, ctx, control, node, + ) + visit(node.expression, control) + visit(node.thenStatement, branchFrame(control, base.id, 'then')) + if (node.elseStatement) { + visit(node.elseStatement, branchFrame(control, base.id, 'else')) + } + return + } + if (sym && ts.isSwitchStatement(node)) { + const base = addCondition( + sym.id, 'switch', node.expression, file, ctx, control, node, + ) + visit(node.expression, control) + for (const clause of node.caseBlock.clauses) { + const arm = ts.isDefaultClause(clause) + ? 'default' as const + : boundedText( + `case:${structuralText(clause.expression, file.sf)}`, + 96, + ).replaceAll('\0', '') as `case:${string}` + if (ts.isCaseClause(clause)) visit(clause.expression, control) + for (const stmt of clause.statements) { + visit(stmt, branchFrame(control, base.id, arm)) + } + } + return + } + if (sym && ts.isConditionalExpression(node)) { + const base = addCondition( + sym.id, 'ternary', node.condition, file, ctx, control, statementFor(node), + ) + visit(node.condition, control) + visit(node.whenTrue, branchFrame(control, base.id, 'truthy')) + visit(node.whenFalse, branchFrame(control, base.id, 'falsy')) + return + } + const logical = isBinary(node) ? LOGICAL_FLOW.get(node.operatorToken.kind) : undefined + if (sym && isBinary(node) && logical) { + const base = addCondition( + sym.id, logical[0], node.left, file, ctx, control, statementFor(node), + ) + visit(node.left, control) + visit(node.right, branchFrame(control, base.id, logical[1])) + return + } + const loop = loopDescriptor(node) + if (sym && loop) { + const base = factBase(sym.id, 'loop', node, file, control) + const repeatedControl = [...control, { + kind: 'loop' as const, + controller_fact_id: base.id, + }] + addFact(ctx, { + ...base, + kind: 'loop', + loop_kind: loop.kind, + ...(loop.test + ? { test: readValue(loop.test, file, ctx, { constants: true }) } + : {}), + }) + for (const setup of loop.once) visit(setup, control) + for (const repeated of loop.repeated) visit(repeated, repeatedControl) + visit(loop.body, repeatedControl) + return + } + if (sym && ts.isTryStatement(node)) { + visit(node.tryBlock, [...control, { kind: 'exception', arm: 'try' }]) + if (node.catchClause) { + visit(node.catchClause, [...control, { kind: 'exception', arm: 'catch' }]) + } + if (node.finallyBlock) { + visit(node.finallyBlock, [...control, { kind: 'exception', arm: 'finally' }]) + } + return + } + if (sym && ts.isReturnStatement(node)) { + addFact(ctx, { + ...factBase(sym.id, 'return', node, file, control), + kind: 'return', + ...(node.expression + ? { value: readValue(node.expression, file, ctx, { constants: true }) } + : {}), + }) + if (node.expression) visit(node.expression, control) + return + } + if (sym && ts.isThrowStatement(node)) { + addFact(ctx, { + ...factBase(sym.id, 'throw', node, file, control), + kind: 'throw', + value: readValue(node.expression, file, ctx, { constants: true }), + }) + visit(node.expression, control) + return + } + if (sym && isBinary(node) && ASSIGNMENT_OPERATORS.has(node.operatorToken.kind)) { + addMutation(sym.id, node, 'assign', node.left, file, ctx, control, node.right) + } else if ( + sym + && (ts.isPrefixUnaryExpression(node) || ts.isPostfixUnaryExpression(node)) + && [K.PlusPlusToken, K.MinusMinusToken].includes(node.operator) + ) { + addMutation( + sym.id, node, + node.operator === K.PlusPlusToken ? 'increment' : 'decrement', + node.operand, file, ctx, control, + ) + } else if (sym && ts.isDeleteExpression(node)) { + addMutation(sym.id, node, 'delete', node.expression, file, ctx, control) + } + if (sym && (isCall(node) || isNew(node))) { + const call = callFact(node, sym, file, ctx, control) + recordEffects(node, sym, file, ctx) + if (isCall(node)) { + const promise = promiseCombinator(node, ctx) + if (promise) { + const base = factBase(sym.id, 'parallel', node, file, control) + const before = ctx.facts.get(sym.id)?.length ?? 0 + const input = parallelSource(node.arguments[0], file, ctx) + const mapped = mappedInput(node.arguments[0], file, ctx) + if (mapped) { + callFact(mapped.call, sym, file, ctx, control) + recordEffects(mapped.call, sym, file, ctx) + const loopId = addIteration(sym.id, mapped.call, file, ctx, control) + visit(mapped.receiver, control) + for (const argument of mapped.call.arguments) { + const callback = unwrap(argument) + const executes = isArrow(callback) + || isFunction(callback) + visit( + argument, + executes + ? [ + ...control, + { kind: 'loop', controller_fact_id: loopId }, + { + kind: 'parallel', + controller_fact_id: base.id, + lane: 'each', + }, + ] + : control, + executes, + ) + } + } else { + for (const [lane, expr] of directParallelLanes( + node.arguments[0], + ).entries()) { + visit(expr, [...control, { + kind: 'parallel', + controller_fact_id: base.id, + lane, + }]) + } + } + const memberFactIds = (ctx.facts.get(sym.id) ?? []) + .slice(before) + .filter((fact) => fact.kind === 'call' + && fact.control.some((frame) => + frame.kind === 'parallel' + && frame.controller_fact_id === base.id)) + .map((fact) => fact.id) + addFact(ctx, { + ...base, + kind: 'parallel', + ...promise, + lane_count: input?.kind === 'array' ? input.elements.length : 0, + ...(input ? { input } : {}), + member_fact_ids: memberFactIds, + }) + return + } + if ( + isAccess(node.expression) + && ARRAY_ITERATION_METHODS.has(node.expression.name.text) + ) { + const input = readValue( + node.expression.expression, + file, + ctx, + { constants: true }, + ) + visit(node.expression.expression, control) + if (input.kind !== 'array') { + for (const argument of node.arguments) visit(argument, control) + return + } + const loopId = addIteration(sym.id, node, file, ctx, control) + for (const argument of node.arguments) { + const callback = unwrap(argument) + visit(argument, [...control, { + kind: 'loop', + controller_fact_id: loopId, + }], isArrow(callback) || isFunction(callback)) + } + return + } + const arrayMutation = isAccess(node.expression) + ? ARRAY_MUTATIONS.get(node.expression.name.text) + : undefined + if (arrayMutation && isAccess(node.expression)) { + addMutation( + sym.id, node, arrayMutation, node.expression.expression, file, ctx, + control, arrayMutation === 'append' ? node.arguments[0] : undefined, 1, + ) + } + } + } + ts.forEachChild(node, (child) => visit(child, control)) + } + visit(file.sf, []) +} +function loopDescriptor(node: ts.Node): { + kind: 'for' | 'for_in' | 'for_of' | 'while' | 'do_while' + test?: ts.Expression + once: readonly ts.Node[] + repeated: readonly ts.Node[] + body: ts.Statement +} | null { + if (ts.isForStatement(node)) { + const once: ts.Node[] = [] + const repeated: ts.Node[] = [] + if (node.initializer) once.push(node.initializer) + if (node.condition) repeated.push(node.condition) + if (node.incrementor) repeated.push(node.incrementor) + return { + kind: 'for', + ...(node.condition ? { test: node.condition } : {}), + once, + repeated, + body: node.statement, + } + } + if (ts.isForInStatement(node) || ts.isForOfStatement(node)) { + return { + kind: ts.isForInStatement(node) ? 'for_in' : 'for_of', + test: node.expression, + once: [node.expression], + repeated: [node.initializer], + body: node.statement, + } + } + if (ts.isWhileStatement(node) || ts.isDoStatement(node)) { + return { + kind: ts.isWhileStatement(node) ? 'while' : 'do_while', + test: node.expression, + once: [], + repeated: [node.expression], + body: node.statement, + } + } + return null +} +function decorators(node: ts.Node): readonly ts.Decorator[] { + return ts.canHaveDecorators(node) ? ts.getDecorators(node) ?? [] : [] +} +function bullDecoratorValue( + node: ts.Node, + name: 'InjectQueue' | 'Processor' | 'Process', + file: FileContext, + ctx: CollectionState, +): { value: IndexValue; transport: QueueTransport } | null { + let result: { value: IndexValue; transport: QueueTransport } | null = null + for (const decorator of decorators(node)) { + if (!isCall(decorator.expression)) continue + const call = decorator.expression + if (!call.arguments[0]) continue + const binding = importedBinding(call.expression, file) + if (binding?.imported === name + && ['@nestjs/bull', '@nestjs/bullmq'].includes(binding.module)) { + result = { + value: readValue(call.arguments[0], file, ctx, { constants: true }), + transport: binding.module === '@nestjs/bull' ? 'bull' : 'bullmq', + } + } + } + return result +} +function collectNest(file: FileContext, ctx: CollectionState): void { + for (const stmt of file.sf.statements) { + if (!ts.isClassDeclaration(stmt) || !stmt.name) continue + const classKey = `${file.fileId}\0${stmt.name.text}` + const properties = ctx.nestQueues.get(classKey) + ?? new Map() + for (const member of stmt.members) { + if (!ts.isConstructorDeclaration(member)) continue + for (const parameter of member.parameters) { + if (!isIdentifier(parameter.name)) continue + const queue = bullDecoratorValue(parameter, 'InjectQueue', file, ctx) + if (queue) { + properties.set(parameter.name.text, [queue.value, queue.transport]) + } + } + } + if (properties.size > 0) ctx.nestQueues.set(classKey, properties) + } +} +function collectNestConsumers(file: FileContext, ctx: CollectionState): void { + for (const stmt of file.sf.statements) { + if (!ts.isClassDeclaration(stmt) || !stmt.name) continue + const queue = bullDecoratorValue(stmt, 'Processor', file, ctx) + if (!queue) continue + for (const member of stmt.members) { + if (!ts.isMethodDeclaration(member) || !member.name || !isIdentifier(member.name)) continue + const symbol = declSymbol(member, file, ctx) + if (!symbol) continue + const job = bullDecoratorValue(member, 'Process', file, ctx) + if (job?.transport === queue.transport) { + const queueKey = staticString(queue.value) + const jobKey = staticString(job.value) + if (queueKey && jobKey) { + const queueNode = channel(ctx, { + channel_kind: 'queue', + transport: queue.transport, + key: queueKey, + }) + const jobNode = channel(ctx, { + channel_kind: 'job', + transport: queue.transport, + key: jobKey, + parent_channel_id: queueNode.id, + }) + appendChannelEdge(ctx, symbol.id, jobNode.id, symbol.id, 'consumed_by', member, file, 'framework-decorator') + appendChannelEdge(ctx, symbol.id, jobNode.id, queueNode.id, 'routes_through', member, file, 'framework-decorator') + } + } else if (!job && member.name.text === 'process') { + const queueKey = staticString(queue.value) + if (queueKey) { + const queueNode = channel(ctx, { + channel_kind: 'queue', + transport: queue.transport, + key: queueKey, + }) + appendChannelEdge(ctx, symbol.id, queueNode.id, symbol.id, 'consumed_by', member, file, 'framework-decorator') + } + } + } + } +} +function expandedEffects(ownerId: string, ctx: CollectionState, depth: number, stack: ReadonlySet): ExecutionEffect[] { + const direct = [...(ctx.effects.get(ownerId) ?? [])]; + if (depth >= MAX_WRAPPER_HOPS || stack.has(ownerId)) + return direct; + const nextStack = new Set(stack); + nextStack.add(ownerId); + for (const call of ctx.calls.get(ownerId) ?? []) { + if (nextStack.has(call[0])) + continue; + const nested = expandedEffects(call[0], ctx, depth + 1, nextStack); + for (const fx of nested) { + direct.push(instantiateEffect(fx, call[1], call[2])); + } + } + return dedupeEffects(direct); +} +function dedupeEffects(effects: readonly ExecutionEffect[]): ExecutionEffect[] { + const values = new Map() + for (const fx of effects) { + const witness = fx[5] + const key = JSON.stringify([ + ...fx.slice(0, 5), + witness.getSourceFile().fileName, + witness.getStart(witness.getSourceFile(), false), + witness.getEnd(), + ]) + if (!values.has(key)) values.set(key, fx) + } + return [...values.values()] +} +type ChannelDescriptor = Omit +function channel(ctx: CollectionState, descriptor: ChannelDescriptor): IndexChannelNode { + const id = indexChannelId(descriptor) + const node: IndexChannelNode = { + id, + node_kind: 'channel', + ...descriptor, + } + const prior = ctx.channels.get(id) + if (prior && JSON.stringify(prior) !== JSON.stringify(node)) { + throw new Error(`Conflicting execution channel identity ${id}`) + } + ctx.channels.set(id, node) + return node +} +function appendChannelEdge( + ctx: CollectionState, + ownerId: string, + from: string, + to: string, + kind: Extract, + witness: ts.Node, + file: FileContext, + source: IndexEdge['source'], + confidence: Confidence = 'high', +): void { + const evidence = evidenceFor(witness, file.sf, file.fileId) + ctx.edges.push({ + from, + to, + kind, + confidence, + source, + evidence, + metadata: { execution_owner_id: ownerId }, + }) +} +function edgeSource(source: IndexFactSource): IndexEdge['source'] { + if (source === 'framework') return 'framework-decorator' + return source +} +function fileContextForNode(node: ts.Node, ctx: CollectionState, filesById: ReadonlyMap): FileContext | null { + const fileId = ctx.input.pathToFileId.get(node.getSourceFile().fileName); + return fileId ? filesById.get(fileId) ?? null : null; +} +function unresolved(ctx: CollectionState, ownerId: string, fx: ExecutionEffect, file: FileContext): void { + const witness = fx[5]; + const id = `canonical-index.execution.unresolved.${sha256([ + ownerId, + fx[0], + file.fileId, + witness.getStart(file.sf, false), + witness.getEnd(), + ].join(':')).slice(0, 16)}`; + if (ctx.seenDiagnostics.has(id)) + return; + ctx.seenDiagnostics.add(id); + ctx.diagnostics.push({ + id, + level: 'info', + message: `Dynamic or ambiguous ${fx[0]} identity; unresolved channel parts were omitted`, + evidence: { + file_id: file.fileId, + range: rangeOf(witness, file.sf), + }, + }); +} +function projectEffects(ctx: CollectionState, filesById: ReadonlyMap): void { + for (const sym of ctx.input.symbols.filter(isOwnerSymbol)) { + if (ctx.overflow.has(sym.id)) continue + for (const fx of expandedEffects(sym.id, ctx, 0, new Set())) { + if (ctx.overflow.has(sym.id)) break + const [kind, primary, endpoint, qualifier, scope, witness, confidence, source] = fx + const file = fileContextForNode(witness, ctx, filesById) + if (!file) continue + const emit = ( + fromId: string, + toId: string, + relation: 'publishes_to' | 'consumed_by' | 'routes_through', + ): void => appendChannelEdge( + ctx, sym.id, fromId, toId, relation, witness, file, + edgeSource(source), confidence, + ) + if (kind === 'bull-publish') { + const queueKey = staticString(primary) + const jobKey = staticString(endpoint) + if (!queueKey) { + unresolved(ctx, sym.id, fx, file) + continue + } + const queueNode = channel(ctx, { + channel_kind: 'queue', + transport: qualifier, + key: queueKey, + }) + if (!jobKey) { + emit(sym.id, queueNode.id, 'publishes_to') + unresolved(ctx, sym.id, fx, file) + continue + } + const jobNode = channel(ctx, { + channel_kind: 'job', + transport: qualifier, + key: jobKey, + parent_channel_id: queueNode.id, + }) + emit(sym.id, jobNode.id, 'publishes_to') + emit(jobNode.id, queueNode.id, 'routes_through') + } else if (kind === 'bull-consume') { + const queueKey = staticString(primary) + const handlerId = symbolId(endpoint) + if (!queueKey || !handlerId || !ctx.symbolsById.has(handlerId)) { + unresolved(ctx, sym.id, fx, file) + continue + } + const queueNode = channel(ctx, { + channel_kind: 'queue', + transport: qualifier, + key: queueKey, + }) + emit(queueNode.id, handlerId, 'consumed_by') + } else if (kind === 'event-publish' || kind === 'event-consume') { + const eventKey = staticString(primary) + const handlerId = kind === 'event-consume' ? symbolId(endpoint!) : null + if (!eventKey || (kind === 'event-consume' + && (!handlerId || !ctx.symbolsById.has(handlerId)))) { + unresolved(ctx, sym.id, fx, file) + continue + } + const eventNode = channel(ctx, { + channel_kind: 'event', + transport: qualifier, + key: eventKey, + scope, + }) + if (kind === 'event-publish') { + emit(sym.id, eventNode.id, 'publishes_to') + } else { + emit(eventNode.id, handlerId!, 'consumed_by') + } + } else { + const persistence = fx as PersistenceEffect + const operation = persistence[1] + const resource = persistence[2] + const receiverType = persistence[3] + const callFactId = ctx.callIds.get(witness) + ?? findCallId(sym.id, witness, ctx) + if (!callFactId || !receiverType) continue + const ownerFacts = ctx.facts.get(sym.id) ?? [] + const existing = ownerFacts.some((fact) => + fact.kind === 'persistence' + && fact.call_fact_id === callFactId + && fact.operation === operation) + if (existing) continue + const callControl = ownerFacts.find((fact) => + fact.id === callFactId)?.control ?? [] + addFact(ctx, { + ...factBase( + sym.id, + 'persistence', + witness, + file, + callControl, + { + confidence, + source, + }, + ), + kind: 'persistence', + operation, + call_fact_id: callFactId, + ...(resource ? { resource } : {}), + receiver_type: receiverType, + }) + } + } + } +} +function findCallId(ownerId: string, witness: ts.Node, ctx: CollectionState): string | null { + const sf = witness.getSourceFile(); + const range = rangeOf(witness, sf); + return ctx.facts.get(ownerId)?.find((fact) => fact.kind === 'call' + && fact.evidence.range.start.line === range.start.line + && fact.evidence.range.start.column === range.start.column + && fact.evidence.range.end.line === range.end.line + && fact.evidence.range.end.column === range.end.column)?.id ?? null; +} +function attachFacts(ctx: CollectionState): void { + for (const symbol of ctx.input.symbols) { + if (ctx.overflow.has(symbol.id)) { + ctx.diagnostics.push({ + id: `canonical-index.execution.owner-bound.${sha256(symbol.id).slice(0, 16)}`, + level: 'error', + message: `Execution facts exceeded a per-owner safety bound for ${symbol.name}; body facts were omitted`, + evidence: { file_id: symbol.file_id, range: symbol.range }, + }); + continue; + } + const facts = ctx.facts.get(symbol.id); + if (!facts || facts.length === 0) + continue; + const byId = new Map(); + for (const fact of facts) + byId.set(fact.id, fact); + const sorted = [...byId.values()].sort((left, right) => compareOrder(left.order, right.order) || compareText(left.id, right.id)); + try { + const encoded = encodeIndexBodyFactTable(sorted); + const normalized = decodeIndexBodyFactTable(encoded, symbol.id, symbol.file_id); + if (!normalized) + throw new Error('execution fact codec rejected its output'); + symbol.body_facts = normalized; + } + catch (error) { + if (!(error instanceof IndexBodyFactBoundsError)) { + throw new Error(`Invalid execution facts for ${symbol.name}: ${error instanceof Error ? error.message : String(error)}`, { cause: error }); + } + ctx.diagnostics.push({ + id: `canonical-index.execution.owner-bound.${sha256(symbol.id).slice(0, 16)}`, + level: 'error', + message: `Execution facts exceeded a per-owner safety bound for ${symbol.name}; body facts were omitted`, + evidence: { file_id: symbol.file_id, range: symbol.range }, + }); + } + } +} +function sortEdges(edges: readonly IndexEdge[]): IndexEdge[] { + const structuralRoutes = new Map(); + const retained: IndexEdge[] = []; + for (const edge of edges) { + if (edge.kind !== 'routes_through') { + retained.push(edge); + continue; + } + const key = `${edge.from}\u0000${edge.to}\u0000${edge.kind}`; + const existing = structuralRoutes.get(key); + if (!existing + || compareText(JSON.stringify(edge), JSON.stringify(existing)) < 0) { + structuralRoutes.set(key, edge); + } + } + return [...retained, ...structuralRoutes.values()].sort((left, right) => compareText(JSON.stringify(left), JSON.stringify(right))); +} +export function collectExecutionSemantics(input: CollectExecutionInput): CollectExecutionResult { + const symbolsById = new Map(input.symbols.map((symbol) => [symbol.id, symbol])) + const ctx: CollectionState = { + input, + symbolsById, + facts: new Map(), + overflow: new Set(), + effects: new Map(), + calls: new Map(), + callIds: new Map(), + channels: new Map(), + edges: [], + diagnostics: [], + seenDiagnostics: new Set(), + queues: new Map(), + emitters: new Map(), + nestQueues: new Map(), + files: new Map(), + } + const filesById = new Map() + for (const sf of input.sourceFiles) { + const fileId = input.pathToFileId.get(sf.fileName) + if (!fileId) continue + const file: FileContext = { + sf, + fileId, + imports: importsFor(sf), + owners: owners(sf, input.symbolsByFile.get(fileId) ?? []), + } + filesById.set(fileId, file) + ctx.files.set(sf, file) + } + for (const file of filesById.values()) { + collectNest(file, ctx) + } + for (const file of filesById.values()) { + collectFacts(file, ctx) + collectNestConsumers(file, ctx) + } + projectEffects(ctx, filesById) + attachFacts(ctx) + return { + channels: [...ctx.channels.values()].sort((left, right) => + compareText(left.id, right.id)), + edges: sortEdges(ctx.edges), + diagnostics: [...ctx.diagnostics].sort((left, right) => + compareText(left.id, right.id)), + } +} diff --git a/src/adapters/typescript/index.ts b/src/adapters/typescript/index.ts index c1aee6c8..02f3b8ea 100644 --- a/src/adapters/typescript/index.ts +++ b/src/adapters/typescript/index.ts @@ -11,11 +11,13 @@ import ts from 'typescript' import { KnowledgeGraph } from '../../domain/graph/directed-multigraph.js' import { CANONICAL_INDEX_FORMAT_VERSION } from '../../domain/index/build-state.js' +import { encodeIndexBodyFactTable } from '../../domain/index/model.js' import type { IndexDiagnostic, IndexEdge, IndexEdgeEvidence, IndexFile, + IndexChannelNode, IndexFrameworkRole, IndexLanguage, IndexRange, @@ -31,6 +33,7 @@ import { detectHonoFramework } from './framework-hono.js' import { detectFastifyFramework } from './framework-fastify.js' import { detectTrpcFramework } from './framework-trpc.js' import { detectPrismaFramework } from './framework-prisma.js' +import { collectExecutionSemantics } from './execution.js' export interface BuildCanonicalTypeScriptIndexOptions { root: string @@ -119,6 +122,7 @@ export function buildCanonicalTypeScriptIndex(opts: BuildCanonicalTypeScriptInde const files: IndexFile[] = [] const symbols: IndexSymbol[] = [] const symbolById = new Map() + const channels: IndexChannelNode[] = [] const edges: IndexEdge[] = [] const diagnostics: IndexDiagnostic[] = [] @@ -152,15 +156,29 @@ export function buildCanonicalTypeScriptIndex(opts: BuildCanonicalTypeScriptInde visitFile(sourceFile, file, root, pathToFileId, compiler.resolveModule, symbols, symbolById, edges, diagnostics) } } - addTypeCheckerEdges({ files, root, pathToFileId, symbols, edges, diagnostics, program: compiler.program }) + addTypeCheckerEdges({ + files, + root, + pathToFileId, + symbols, + channels, + edges, + diagnostics, + program: compiler.program, + }) } files.sort((a, b) => compareCodeUnits(a.path, b.path)) symbols.sort((a, b) => compareCodeUnits(symbolSortKey(a), symbolSortKey(b))) + channels.sort((a, b) => compareCodeUnits(a.id, b.id)) edges.sort((a, b) => compareCodeUnits(edgeSortKey(a), edgeSortKey(b))) diagnostics.sort((a, b) => compareCodeUnits(diagnosticSortKey(a), diagnosticSortKey(b))) - return { graph: writeCanonicalGraph(root, files, symbols, edges), files, diagnostics } + return { + graph: writeCanonicalGraph(root, files, symbols, channels, edges), + files, + diagnostics, + } } const confidence = { @@ -173,6 +191,7 @@ function writeCanonicalGraph( root: string, files: readonly IndexFile[], symbols: readonly IndexSymbol[], + channels: readonly IndexChannelNode[], edges: readonly IndexEdge[], ): KnowledgeGraph { const graph = new KnowledgeGraph({ @@ -225,10 +244,28 @@ function writeCanonicalGraph( framework_role: symbol.framework_role, } : {}), ...(Object.keys(metadata).length > 0 ? { framework_metadata: metadata, ...metadata } : {}), + ...(symbol.body_facts && symbol.body_facts.length > 0 + ? { body_facts: encodeIndexBodyFactTable(symbol.body_facts) } + : {}), provenance: [provenance(file.path, location)], }) } + for (const channel of channels) { + graph.addNode(channel.id, { + label: channel.key, + node_kind: 'channel', + channel_kind: channel.channel_kind, + transport: channel.transport, + key: channel.key, + ...(channel.scope ? { scope: channel.scope } : {}), + ...(channel.parent_channel_id + ? { parent_channel_id: channel.parent_channel_id } + : {}), + layer: 'semantic', + }) + } + for (const edge of edges) { let source = edge.from const target = edge.to @@ -265,7 +302,16 @@ function writeCanonicalGraph( source_file: sourceFile, source_location: location, layer: 'semantic', - evidence: { source: edge.source, ...(range ? { range } : {}) }, + evidence: { + source: edge.source, + ...(range ? { range } : {}), + ...(edge.evidence?.statement_range + ? { statement_range: edge.evidence.statement_range } + : {}), + ...(edge.evidence?.excerpt_sha256 + ? { excerpt_sha256: edge.evidence.excerpt_sha256 } + : {}), + }, ...(edge.metadata ?? {}), provenance: [provenance(sourceFile, location)], } @@ -1039,7 +1085,10 @@ function makeFileId(relPath: string): string { } function makeSymbolId(fileId: string, kind: IndexSymbolKind, name: string): string { - return `symbol:${fileId}/${kind}/${name}` + const id = `symbol:${fileId}/${kind}/${name}` + return Buffer.byteLength(id, 'utf8') <= 1_024 + ? id + : `symbol:${fileId}/${kind}/hashed:${sha256(name).slice(0, 32)}` } function sha256(text: string): string { @@ -1113,13 +1162,23 @@ type TypeCheckerEdgeContext = { root: string pathToFileId: Map symbols: IndexSymbol[] + channels: IndexChannelNode[] edges: IndexEdge[] diagnostics: IndexDiagnostic[] program: ts.Program } function addTypeCheckerEdges(ctx: TypeCheckerEdgeContext): void { - const { files, root, pathToFileId, symbols, edges, diagnostics, program } = ctx + const { + files, + root, + pathToFileId, + symbols, + channels, + edges, + diagnostics, + program, + } = ctx const checker = program.getTypeChecker() const seenCalls = new Set() const seenTypeEdges = new Set() @@ -1221,6 +1280,17 @@ function addTypeCheckerEdges(ctx: TypeCheckerEdgeContext): void { } finalizeExpressMountPrefixes({ symbols, edges }) + const execution = collectExecutionSemantics({ + program, + sourceFiles: programSourceFiles, + checker, + pathToFileId, + symbols, + symbolsByFile, + }) + channels.push(...execution.channels) + edges.push(...execution.edges) + diagnostics.push(...execution.diagnostics) } function walkCallExpressions( diff --git a/src/application/retrieve-context.ts b/src/application/retrieve-context.ts index affaae65..51021e1a 100644 --- a/src/application/retrieve-context.ts +++ b/src/application/retrieve-context.ts @@ -16,60 +16,77 @@ import { type RetrieveContextResult, type RetrieveOutcome, } from '../domain/query/types.js' -type AuthenticatedSource = { state: 'ready'; text: string } +type AuthenticatedSource = { + state: 'ready' + text: string + lineStarts: readonly number[] + lineEnds: readonly number[] + proofHashes: Map +} | { state: 'stale' | 'unavailable'; subject: string } type AuthenticatedNode = { state: 'ready'; node: EvidenceNode } | { state: 'corrupt' | 'stale' | 'unavailable'; subject: string } +type ChannelProof = readonly [edgeId: string, attributes: GraphAttributes] const utf8 = new TextDecoder('utf-8', { fatal: true, ignoreBOM: true }) +const proofCache = new WeakMap< +ReadyQueryIndex, +Map +>() -function isPositiveLine(value: unknown): value is number { +function validLine(value: unknown): value is number { return typeof value === 'number' && Number.isSafeInteger(value) && value > 0 } -function stringFact(attributes: GraphAttributes, key: string): string | null { - const value = attributes[key] +function stringFact(attrs: GraphAttributes, key: string): string | null { + const value = attrs[key] return typeof value === 'string' && value.length > 0 ? value : null } -function sourceIsBeneathRoot(root: string, source: string): boolean { +function insideRoot(root: string, source: string): boolean { const path = relative(root, source) return path !== '..' && !path.startsWith(`..${sep}`) && !isAbsolute(path) } -function readAuthenticatedSource( - index: ReadyQueryIndex, sourceFile: string, cache: Map, +function readSource( + index: ReadyQueryIndex, file: string, cache: Map, ): AuthenticatedSource { - const cached = cache.get(sourceFile) + const cached = cache.get(file) if (cached) return cached const remember = (result: AuthenticatedSource): AuthenticatedSource => { - cache.set(sourceFile, result) + cache.set(file, result) return result } - const expectedHash = index.file_hashes.get(sourceFile) - if (!expectedHash) return remember({ state: 'stale', subject: sourceFile }) + const expected = index.file_hashes.get(file) + if (!expected) return remember({ state: 'stale', subject: file }) try { const root = realpathSync(index.root_path) - const candidate = realpathSync(resolve(root, sourceFile)) - if (isAbsolute(sourceFile) || !sourceIsBeneathRoot(root, candidate)) { - return remember({ state: 'unavailable', subject: sourceFile }) + const candidate = realpathSync(resolve(root, file)) + if (isAbsolute(file) || !insideRoot(root, candidate)) { + return remember({ state: 'unavailable', subject: file }) } const bytes = readFileSync(candidate) + const actual = createHash('sha256').update(bytes).digest('hex') + if (actual !== expected) { + return remember({ state: 'stale', subject: file }) + } const text = utf8.decode(bytes) - const hash = createHash('sha256').update(bytes).digest('hex') - return remember(hash === expectedHash - ? { state: 'ready', text } - : { state: 'stale', subject: sourceFile }) + const lines = lineOffsets(text) + return remember({ + state: 'ready', text, lineStarts: lines.starts, lineEnds: lines.ends, + proofHashes: new Map(), + }) } catch { - return remember({ state: 'unavailable', subject: sourceFile }) + return remember({ state: 'unavailable', subject: file }) } } -function offsetOf(text: string, position: IndexRange['start']): number | null { - if (!Number.isSafeInteger(position.line) || position.line < 1 - || !Number.isSafeInteger(position.column) || position.column < 1) return null +function lineOffsets(text: string): { + starts: readonly number[] + ends: readonly number[] +} { const starts = [0], ends: number[] = [] for (let index = 0; index < text.length; index += 1) { const code = text.charCodeAt(index) @@ -79,93 +96,191 @@ function offsetOf(text: string, position: IndexRange['start']): number | null { starts.push(index + 1) } ends.push(text.length) - const start = starts[position.line - 1], end = ends[position.line - 1] + return { starts, ends } +} + +function offset( + source: Extract, + pos: IndexRange['start'], +): number | null { + if (!Number.isSafeInteger(pos.line) || pos.line < 1 + || !Number.isSafeInteger(pos.column) || pos.column < 1) return null + const start = source.lineStarts[pos.line - 1] + const end = source.lineEnds[pos.line - 1] if (start === undefined || end === undefined) return null - const offset = start + position.column - 1 + const offset = start + pos.column - 1 return offset <= end ? offset : null } function validRange(value: unknown): value is IndexRange { if (!value || typeof value !== 'object') return false const range = value as IndexRange - return offsetPosition(range.start) <= offsetPosition(range.end) + return positionKey(range.start) <= positionKey(range.end) } -function offsetPosition(position: IndexRange['start'] | undefined): number { - return position && Number.isSafeInteger(position.line) && position.line > 0 - && Number.isSafeInteger(position.column) && position.column > 0 - ? position.line * 0x1_0000_0000 + position.column +function positionKey(pos: IndexRange['start'] | undefined): number { + return pos && Number.isSafeInteger(pos.line) && pos.line > 0 + && Number.isSafeInteger(pos.column) && pos.column > 0 + ? pos.line * 0x1_0000_0000 + pos.column : Number.NaN } -function exactRange(text: string, range: IndexRange): string | null { - const start = offsetOf(text, range.start), end = offsetOf(text, range.end) - return start === null || end === null || end < start ? null : text.slice(start, end) +function excerpt( + source: Extract, + range: IndexRange, +): string | null { + const start = offset(source, range.start) + const end = offset(source, range.end) + return start === null || end === null || end < start + ? null + : source.text.slice(start, end) +} + +function checkFactProofs( + index: ReadyQueryIndex, + ownerId: string, + source: Extract, +): boolean { + for (const fact of index.operations_by_owner.get(ownerId) ?? []) { + if (!proofMatches( + source, + fact.evidence.statement_range, + fact.evidence.excerpt_sha256, + )) return false + } + return true +} + +function proofMatches( + source: Extract, + range: IndexRange, + expected: string, +): boolean { + const key = `${range.start.line}:${range.start.column}:${ + range.end.line}:${range.end.column}` + let actual = source.proofHashes.get(key) + if (!actual) { + const proofText = excerpt(source, range) + if (proofText === null) return false + actual = createHash('sha256').update(proofText, 'utf8').digest('hex') + source.proofHashes.set(key, actual) + } + return actual === expected +} + +function channelProofs(index: ReadyQueryIndex, ownerId: string): readonly ChannelProof[] { + let byOwner = proofCache.get(index) + if (!byOwner) { + byOwner = new Map() + for (const [, , attrs, edgeId] of index.graph.edgeEntries()) { + const owner = attrs.execution_owner_id + if (typeof owner !== 'string' + || !['publishes_to', 'routes_through', 'consumed_by'] + .includes(String(attrs.relation))) continue + const proofs = byOwner.get(owner) ?? [] + proofs.push([edgeId, attrs]) + byOwner.set(owner, proofs) + } + proofCache.set(index, byOwner) + } + return byOwner.get(ownerId) ?? [] +} + +function checkChannelProofs( + index: ReadyQueryIndex, + ownerId: string, + sources: Map, +): { state: 'ready' } | { state: 'corrupt' | 'stale' | 'unavailable'; subject: string } { + for (const [edgeId, attrs] of channelProofs(index, ownerId)) { + const file = attrs.source_file + const evidence = attrs.evidence as Record | undefined + const range = evidence?.statement_range + const expected = evidence?.excerpt_sha256 + if (typeof file !== 'string' || !validRange(range) + || typeof expected !== 'string') { + return { state: 'corrupt', subject: edgeId } + } + const source = readSource(index, file, sources) + if (source.state !== 'ready') return source + if (!proofMatches(source, range, expected)) { + return { state: 'corrupt', subject: edgeId } + } + } + return { state: 'ready' } } function authenticateNode( - index: ReadyQueryIndex, nodeId: string, sourceCache: Map, + index: ReadyQueryIndex, nodeId: string, sources: Map, ): AuthenticatedNode { if (!index.graph.hasNode(nodeId)) return { state: 'corrupt', subject: nodeId } - const attributes = index.graph.nodeAttributes(nodeId) - const label = stringFact(attributes, 'label') - const nodeKind = stringFact(attributes, 'node_kind') - const sourceFile = stringFact(attributes, 'source_file') - const sourceLocation = stringFact(attributes, 'source_location') - const provenance = attributes.provenance - const contentHash = sourceFile ? index.file_hashes.get(sourceFile) : undefined - - if (!label || !nodeKind || !sourceFile + const attrs = index.graph.nodeAttributes(nodeId) + const label = stringFact(attrs, 'label') + const nodeKind = stringFact(attrs, 'node_kind') + const file = stringFact(attrs, 'source_file') + const location = stringFact(attrs, 'source_location') + const provenance = attrs.provenance + const contentHash = file ? index.file_hashes.get(file) : undefined + + if (!label || !nodeKind || !file || !Array.isArray(provenance) || provenance.length === 0 || !contentHash) { return { state: 'corrupt', subject: nodeId } } - const source = readAuthenticatedSource(index, sourceFile, sourceCache) + const source = readSource(index, file, sources) if (source.state !== 'ready') return source - const sourceDomain = stringFact(attributes, 'source_domain') + if (!checkFactProofs(index, nodeId, source)) { + return { state: 'corrupt', subject: nodeId } + } + const channelProof = checkChannelProofs( + index, + nodeId, + sources, + ) + if (channelProof.state !== 'ready') return channelProof + const domain = stringFact(attrs, 'source_domain') const common = { - node_id: nodeId, label, source_file: sourceFile, provenance, + node_id: nodeId, label, source_file: file, provenance, content_hash: contentHash, - ...(sourceDomain ? { source_domain: sourceDomain } : {}), + ...(domain ? { source_domain: domain } : {}), } if (nodeKind === 'file') { return { state: 'ready', node: { ...common, evidence_kind: 'structural_file', node_kind: 'file' } } } - const startLine = attributes.line_number - const endLine = attributes.end_line_number - const definitionRange = attributes.definition_range - const declarationRange = attributes.declaration_range - if (!sourceLocation || !isPositiveLine(startLine) || !isPositiveLine(endLine) + const startLine = attrs.line_number + const endLine = attrs.end_line_number + const definition = attrs.definition_range + const declaration = attrs.declaration_range + if (!location || !validLine(startLine) || !validLine(endLine) ) return { state: 'corrupt', subject: nodeId } - if (!validRange(definitionRange) || !validRange(declarationRange) - || offsetPosition(declarationRange.start) < offsetPosition(definitionRange.start) - || offsetPosition(declarationRange.end) > offsetPosition(definitionRange.end)) { - return { state: 'stale', subject: sourceFile } + if (!validRange(definition) || !validRange(declaration) + || positionKey(declaration.start) < positionKey(definition.start) + || positionKey(declaration.end) > positionKey(definition.end)) { + return { state: 'stale', subject: file } } - const expectedLocation = definitionRange.end.line > definitionRange.start.line - ? `L${definitionRange.start.line}-L${definitionRange.end.line}` - : `L${definitionRange.start.line}` - if (startLine !== definitionRange.start.line || endLine !== definitionRange.end.line - || sourceLocation !== expectedLocation) return { state: 'stale', subject: sourceFile } - const snippet = exactRange(source.text, declarationRange) - if (snippet === null || exactRange(source.text, definitionRange) === null) { - return { state: 'stale', subject: sourceFile } + const expectedLocation = definition.end.line > definition.start.line + ? `L${definition.start.line}-L${definition.end.line}` + : `L${definition.start.line}` + if (startLine !== definition.start.line || endLine !== definition.end.line + || location !== expectedLocation) return { state: 'stale', subject: file } + const snippet = excerpt(source, declaration) + if (snippet === null || excerpt(source, definition) === null) { + return { state: 'stale', subject: file } } return { state: 'ready', node: { ...common, evidence_kind: 'symbol_declaration', node_kind: nodeKind, - source_location: sourceLocation, line_number: startLine, end_line_number: endLine, - definition_range: definitionRange, declaration_range: declarationRange, snippet, + source_location: location, line_number: startLine, end_line_number: endLine, + definition_range: definition, declaration_range: declaration, snippet, }, } } -function relationshipFromEdge(edge: QueryPathEdge): EvidenceRelationship | null { - const sourceFile = edge.attributes.source_file - const sourceLocation = edge.attributes.source_location +function edgeResult(edge: QueryPathEdge): EvidenceRelationship | null { + const file = edge.attributes.source_file + const location = edge.attributes.source_location const provenance = edge.attributes.provenance if (!Array.isArray(provenance) || provenance.length === 0) return null return { @@ -173,25 +288,25 @@ function relationshipFromEdge(edge: QueryPathEdge): EvidenceRelationship | null from_id: edge.from, to_id: edge.to, relation: edge.relation, - ...(typeof sourceFile === 'string' && sourceFile.length > 0 ? { source_file: sourceFile } : {}), - ...(typeof sourceLocation === 'string' && sourceLocation.length > 0 ? { source_location: sourceLocation } : {}), + ...(typeof file === 'string' && file.length > 0 ? { source_file: file } : {}), + ...(typeof location === 'string' && location.length > 0 ? { source_location: location } : {}), provenance, } } -function outcomeFrom(nodes: readonly EvidenceNode[], boundaries: readonly EvidenceBoundary[]): RetrieveOutcome { +function outcome(nodes: readonly EvidenceNode[], boundaries: readonly EvidenceBoundary[]): RetrieveOutcome { if (nodes.length > 0) return 'evidence' for (const state of ['corrupt', 'unavailable', 'stale', 'unsupported', 'missing'] as const) { - if (boundaries.some((boundary) => boundary.kind === state)) return state + if (boundaries.some((limit) => limit.kind === state)) return state } return 'missing' } -function boundary(kind: EvidenceBoundary['kind'], subject: string): EvidenceBoundary { +function limit(kind: EvidenceBoundary['kind'], subject: string): EvidenceBoundary { return { kind, subject } } -function emptyResult( +function empty( request: NormalizedRetrieveRequest, outcome: RetrieveOutcome, boundaries: EvidenceBoundary[], ): RetrieveContextResult { return sliceEvidence({ @@ -205,48 +320,48 @@ function emptyResult( export function retrieveContext(index: QueryIndex, input: unknown): RetrieveContextResult { const request = normalizeRetrieveRequest(input) if (index.state !== 'ready') { - return emptyResult(request, index.state, [boundary(index.state, index.subject)]) + return empty(request, index.state, [limit(index.state, index.subject)]) } const ranking = rankQueryAnchors(index, request) if (ranking.anchors.length === 0) { const boundaries = ranking.boundaries.length > 0 ? ranking.boundaries - : [boundary('missing', request.question)] - return emptyResult(request, outcomeFrom([], boundaries), boundaries) + : [limit('missing', request.question)] + return empty(request, outcome([], boundaries), boundaries) } const traversal = traverseEvidencePaths(index, ranking) - const sourceCache = new Map() + const sources = new Map() let matchedNodes: EvidenceNode[] = [] const boundaries = [...ranking.boundaries, ...traversal.boundaries] for (const nodeId of traversal.nodeIds) { - const authenticated = authenticateNode(index, nodeId, sourceCache) - if (authenticated.state === 'ready') { - matchedNodes.push(authenticated.node) + const checked = authenticateNode(index, nodeId, sources) + if (checked.state === 'ready') { + matchedNodes.push(checked.node) } else { - boundaries.push(boundary(authenticated.state, authenticated.subject)) + boundaries.push(limit(checked.state, checked.subject)) } } - const selectedNodeIds = new Set(matchedNodes.map((node) => node.node_id)) + const selected = new Set(matchedNodes.map((node) => node.node_id)) const relationships: EvidenceRelationship[] = [] for (const edge of traversal.edges) { - if (!selectedNodeIds.has(edge.from) || !selectedNodeIds.has(edge.to)) continue - const relationship = relationshipFromEdge(edge) + if (!selected.has(edge.from) || !selected.has(edge.to)) continue + const relationship = edgeResult(edge) if (relationship) relationships.push(relationship) - else boundaries.push(boundary('corrupt', edge.id)) + else boundaries.push(limit('corrupt', edge.id)) } const related = new Set(relationships.flatMap((edge) => [edge.from_id, edge.to_id])) - const orphanFiles = matchedNodes.filter((node) => + const orphans = matchedNodes.filter((node) => node.evidence_kind === 'structural_file' && !related.has(node.node_id)) - for (const node of orphanFiles) boundaries.push(boundary('unavailable', node.source_file)) - const orphanIds = new Set(orphanFiles.map((node) => node.node_id)) + for (const node of orphans) boundaries.push(limit('unavailable', node.source_file)) + const orphanIds = new Set(orphans.map((node) => node.node_id)) matchedNodes = matchedNodes.filter((node) => !orphanIds.has(node.node_id)) return sliceEvidence({ request, - outcome: outcomeFrom(matchedNodes, boundaries), + outcome: outcome(matchedNodes, boundaries), matchedNodes, relationships, boundaries, diff --git a/src/domain/index/build-state.ts b/src/domain/index/build-state.ts index aa946c3c..7aebaab2 100644 --- a/src/domain/index/build-state.ts +++ b/src/domain/index/build-state.ts @@ -4,10 +4,10 @@ import { canonicalJsonString, canonicalJsonValue, compareCodeUnits } from '../gr import type { KnowledgeGraph } from '../graph/directed-multigraph.js' import { hasExactKeys, isRecord } from '../../shared/guards.js' -export const CANONICAL_INDEX_FORMAT_VERSION = 3 as const +export const CANONICAL_INDEX_FORMAT_VERSION = 4 as const export const GENERATION_POLICY_VERSION = 4 as const export const INDEX_BUILD_STATE_VERSION = 3 as const -export const INDEX_ENGINE_ID = 'madar-typescript-index-v3' as const +export const INDEX_ENGINE_ID = 'madar-typescript-index-v4' as const export const INDEXING_OUTCOME_STATUSES = [ 'indexed', 'indexed_with_warnings', 'skipped_by_policy', 'unsupported', 'failed', ] as const diff --git a/src/domain/index/model.ts b/src/domain/index/model.ts index 6a6dd388..e92ad3f4 100644 --- a/src/domain/index/model.ts +++ b/src/domain/index/model.ts @@ -1,12 +1,11 @@ +import { createHash } from 'node:crypto' // Compiler-independent facts used while the TypeScript adapter writes the // canonical graph. These records never leave the adapter as a second index. - export type IndexLanguage = | 'typescript' | 'javascript' | 'tsx' | 'jsx' - export type IndexFile = { id: string path: string @@ -14,7 +13,6 @@ export type IndexFile = { loc: number hash: string } - export type IndexSymbolKind = | 'function' | 'class' @@ -25,17 +23,643 @@ export type IndexSymbolKind = | 'constant' | 'variable' | 'namespace' - export type IndexPosition = { line: number column: number } - export type IndexRange = { start: IndexPosition end: IndexPosition } - +export type IndexSha256 = string +type Immutable = { readonly [K in keyof T]: T[K] } +export type IndexFactConfidence = 'high' | 'medium' | 'low' +export type IndexFactSource = + | 'typescript-semantic' | 'typescript-syntactic' | 'framework' | 'wrapper-summary' +export type IndexCallScheduling = 'sync' | 'awaited' | 'fire_and_forget' +export type IndexLiteralRole = + | 'argument' | 'initializer' | 'condition' | 'return' | 'channel' | 'configuration' +type IndexConditionKind = 'if' | 'switch' | 'ternary' | 'logical_and' | 'logical_or' | 'nullish' | 'guard'; +type IndexLoopKind = 'for' | 'for_in' | 'for_of' | 'while' | 'do_while' | 'array_iteration'; +export type IndexPromiseCombinator = 'all' | 'allSettled' | 'any' | 'race' +export type IndexParallelCompletion = + | 'all_or_first_rejection' | 'all_settled' | 'first_fulfilled' | 'first_settled' +export type IndexMutationOperation = + | 'assign' | 'increment' | 'decrement' | 'append' | 'remove' | 'delete' +export type IndexPersistenceOperation = + | 'read' | 'create' | 'update' | 'delete' | 'upsert' | 'transaction' + | 'file_read' | 'file_write' | 'object_read' | 'object_write' +type IndexUnknownReason = 'dynamic' | 'ambiguous' | 'unsupported' +const KINDS = [ + 'condition', 'loop', 'parallel', 'call', 'literal', + 'mutation', 'persistence', 'return', 'throw', +] as const +const LEVELS = ['high', 'medium', 'low'] as const +const SOURCES = [ + 'typescript-semantic', 'typescript-syntactic', 'framework', 'wrapper-summary', +] as const +const TIMING = ['sync', 'awaited', 'fire_and_forget'] as const +const ROLES = [ + 'argument', 'initializer', 'condition', 'return', 'channel', 'configuration', +] as const +const CONDITIONS = [ + 'if', 'switch', 'ternary', 'logical_and', 'logical_or', 'nullish', 'guard', +] as const +const LOOPS = [ + 'for', 'for_in', 'for_of', 'while', 'do_while', 'array_iteration', +] as const +const PROMISES = ['all', 'allSettled', 'any', 'race'] as const +const COMPLETION = [ + 'all_or_first_rejection', 'all_settled', 'first_fulfilled', 'first_settled', +] as const +const MUTATIONS = [ + 'assign', 'increment', 'decrement', 'append', 'remove', 'delete', +] as const +const STORAGE = [ + 'read', 'create', 'update', 'delete', 'upsert', 'transaction', + 'file_read', 'file_write', 'object_read', 'object_write', +] as const +const UNKNOWN = ['dynamic', 'ambiguous', 'unsupported'] as const +const SHA256 = /^[a-f0-9]{64}$/ +const MAX_ROWS = 8_192, MAX_ROW = 262_144, MAX_TABLE = 8_388_608 +const MAX_DEPTH = 5, MAX_ELEMENTS = 32, MAX_TEXT = 512 +export type IndexFactEvidence = Immutable<{ + file_id: string + /** Smallest expression or token range that proves the fact. */ + range: IndexRange + /** Bounded range whose exact UTF-8 bytes are authenticated by excerpt_sha256. */ + statement_range: IndexRange; excerpt_sha256: IndexSha256 +}> +export type IndexScalarValue = string | number | boolean | null +export type IndexObjectEntry = Immutable<{ key: string; value: IndexValue }> +/** + * A bounded, compiler-independent representation of statically known values. + * Collectors are responsible for depth, element-count, byte-length, and secret + * redaction limits before a value reaches this model. + */ +export type IndexValue = + | Immutable<{ kind: 'literal'; value: IndexScalarValue }> + | Immutable<{ kind: 'symbol'; symbol_id: string }> + | Immutable<{ kind: 'parameter'; position: number; scope?: 'iteration' }> + | Immutable<{ kind: 'array'; elements: readonly IndexValue[] }> + | Immutable<{ kind: 'object'; entries: readonly IndexObjectEntry[] }> + | Immutable<{ kind: 'template'; parts: readonly IndexValue[] }> + | Immutable<{ kind: 'redacted'; sha256: IndexSha256; byte_length: number }> + | Immutable<{ kind: 'unknown'; reason: IndexUnknownReason }> +export type IndexBranchArm = 'then' | 'else' | 'truthy' | 'falsy' | 'nullish' + | 'default' | `case:${string}` +export type IndexControlFrame = + | Immutable<{ kind: 'branch'; controller_fact_id: string; arm: IndexBranchArm }> + | Immutable<{ kind: 'loop'; controller_fact_id: string }> + | Immutable<{ kind: 'parallel'; controller_fact_id: string; lane: number | 'each' }> + | Immutable<{ kind: 'exception'; arm: 'try' | 'catch' | 'finally' }> +export type IndexBodyFactBase = Immutable<{ + id: string; owner_symbol_id: string + /** Numeric AST path; lexicographic comparison gives stable source order. */ + order: readonly number[]; evidence: IndexFactEvidence + control: readonly IndexControlFrame[] + confidence: IndexFactConfidence; source: IndexFactSource +}> +type Fact = + IndexBodyFactBase & Immutable<{ kind: K } & T> +export type IndexCallFact = Fact<'call', { + callee: string; target_symbol_id?: string + arguments: readonly IndexValue[]; scheduling: IndexCallScheduling +}> +export type IndexLiteralFact = Fact<'literal', { + value: IndexValue; role: IndexLiteralRole +}> +export type IndexConditionFact = Fact<'condition', { + condition_kind: IndexConditionKind; test?: IndexValue +}> +export type IndexLoopFact = Fact<'loop', { + loop_kind: IndexLoopKind; test?: IndexValue +}> +export type IndexParallelFact = Fact<'parallel', { + combinator: IndexPromiseCombinator + completion: IndexParallelCompletion; lane_count: number + /** The array/iterable passed to the combinator when statically representable. */ + input?: IndexValue; member_fact_ids: readonly string[] +}> +export type IndexReturnFact = Fact<'return', { value?: IndexValue }> +export type IndexThrowFact = Fact<'throw', { value?: IndexValue }> +export type IndexMutationFact = Fact<'mutation', { + operation: IndexMutationOperation + target: string; value?: IndexValue +}> +export type IndexPersistenceFact = Fact<'persistence', { + operation: IndexPersistenceOperation + call_fact_id: string; resource?: IndexValue + /** Receiver/type proof, not a method-name-only classification. */ + receiver_type: string +}> +export type IndexBodyFact = IndexCallFact | IndexLiteralFact | IndexConditionFact + | IndexLoopFact | IndexParallelFact | IndexReturnFact | IndexThrowFact + | IndexMutationFact | IndexPersistenceFact +/** + * Binds an operation identity to its owner, stable AST order, and authenticated + * statement bytes. Query-index validation recomputes this value so a + * well-shaped but replaced excerpt digest cannot silently become ready. + */ +export function indexBodyFactId( + ownerSymbolId: string, + kind: IndexBodyFact['kind'], + order: readonly number[], + excerptSha256: IndexSha256, +): string { + const identity = [ownerSymbolId, kind, order.join('.'), excerptSha256].join('\u0000') + return `operation:${createHash('sha256').update(identity, 'utf8') + .digest('hex').slice(0, 32)}` +} +/** Compact graph-artifact representation; rows are canonical JSON strings. */ +export type IndexBodyFactTable = readonly [version: 1, rows: readonly string[]] +export const INDEX_BODY_FACT_CONTROL_LIMIT = 64 +export class IndexBodyFactBoundsError extends Error {} +function enumPos(values: readonly string[], value: string): number { + const index = values.indexOf(value) + if (index < 0) throw new Error(`Unsupported execution value ${value}`) + return index +} +function orderCmp(left: readonly number[], right: readonly number[]): number { + for (let index = 0; index < Math.min(left.length, right.length); index += 1) { + const difference = left[index]! - right[index]! + if (difference !== 0) return difference + } + return left.length - right.length +} +function isDense(value: readonly unknown[]): boolean { + for (let index = 0; index < value.length; index += 1) + if (!Object.hasOwn(value, index)) return false + return true +} +function scalar(value: unknown): value is IndexScalarValue { + return (value === null || ['string', 'number', 'boolean'].includes(typeof value)) + && !(typeof value === 'number' && (!Number.isFinite(value) || Object.is(value, -0))) + && !(typeof value === 'string' && Buffer.byteLength(value, 'utf8') > MAX_TEXT) +} +function packVal(value: IndexValue, depth = 0): unknown { + const nestedCount = value.kind === 'array' ? value.elements.length + : value.kind === 'object' ? value.entries.length + : value.kind === 'template' ? value.parts.length : 0 + if (depth > MAX_DEPTH || (depth === MAX_DEPTH && nestedCount > 0)) + return [7, enumPos(UNKNOWN, 'unsupported')] + switch (value.kind) { + case 'literal': + if (!scalar(value.value)) throw new Error('Execution literal is not JSON-lossless') + return [0, value.value] + case 'symbol': + if (!validText(value.symbol_id, 1_024)) + throw new Error('Execution symbol reference is invalid') + return [1, value.symbol_id] + case 'parameter': + if (!safeInt(value.position) + || (value.scope !== undefined && value.scope !== 'iteration')) + throw new Error('Execution parameter position is invalid') + return value.scope === 'iteration' + ? [2, value.position, 1] + : [2, value.position] + case 'array': + if (value.elements.length > MAX_ELEMENTS || !isDense(value.elements)) + throw new Error('Execution array exceeds its element bound') + return [3, value.elements.map((entry) => packVal(entry, depth + 1))] + case 'object': { + const keys = new Set() + if (value.entries.length > MAX_ELEMENTS || !isDense(value.entries)) + throw new Error('Execution object exceeds its element bound') + for (const entry of value.entries) { + if (Buffer.byteLength(entry.key, 'utf8') > MAX_TEXT + || entry.key.includes('\0') || keys.has(entry.key)) + throw new Error('Execution object key is invalid') + keys.add(entry.key) + } + return [4, value.entries.map((entry) => [ + entry.key, packVal(entry.value, depth + 1), + ])] + } + case 'template': + if (value.parts.length > MAX_ELEMENTS || !isDense(value.parts)) + throw new Error('Execution template exceeds its element bound') + return [5, value.parts.map((entry) => packVal(entry, depth + 1))] + case 'redacted': + if (!SHA256.test(value.sha256) || !safeInt(value.byte_length)) + throw new Error('Execution redaction is invalid') + return [6, value.sha256, value.byte_length] + case 'unknown': return [7, enumPos(UNKNOWN, value.reason)] + } + throw new Error('Unsupported execution value') +} +function packEvidence(proof: IndexFactEvidence): unknown { + return [ + proof.range.start.line, proof.range.start.column, + proof.range.end.line, proof.range.end.column, + proof.statement_range.start.line, proof.statement_range.start.column, + proof.statement_range.end.line, proof.statement_range.end.column, proof.excerpt_sha256, + ] +} +export function encodeIndexBodyFactTable( + facts: readonly IndexBodyFact[], +): IndexBodyFactTable { + if (facts.length === 0 || facts.length > MAX_ROWS) { + throw new IndexBodyFactBoundsError( + 'Execution fact table is outside its row bound', + ) + } + if (!isDense(facts)) throw new Error('Execution fact table is sparse') + const ordered = [...facts].sort((left, right) => + orderCmp(left.order, right.order) + || (left.id < right.id ? -1 : left.id > right.id ? 1 : 0)) + const ordinals = new Map(ordered.map((fact, index) => [fact.id, index])) + if (ordinals.size !== ordered.length) + throw new Error('Execution fact IDs are not unique') + const ordinal = (id: string): number => { + const value = ordinals.get(id) + if (value === undefined) throw new Error(`Missing execution fact reference ${id}`) + return value + } + const control = (frame: IndexControlFrame): unknown => { + if (frame.kind === 'branch') { + if (!validText(frame.arm, 96) + || (!['then', 'else', 'truthy', 'falsy', 'nullish', 'default'].includes(frame.arm) + && !(frame.arm.startsWith('case:') && frame.arm.length > 5))) { + throw new Error('Execution branch arm is invalid') + } + return [0, ordinal(frame.controller_fact_id), frame.arm] + } + if (frame.kind === 'loop') return [1, ordinal(frame.controller_fact_id)] + if (frame.kind === 'parallel') { + if (frame.lane !== 'each' && !safeInt(frame.lane)) + throw new Error('Execution parallel lane is invalid') + return [2, ordinal(frame.controller_fact_id), frame.lane] + } + if (frame.kind === 'exception') + return [3, enumPos(['try', 'catch', 'finally'], frame.arm)] + throw new Error('Unsupported execution control frame') + } + let bytes = 0 + const orderKeys = new Set() + const rows = ordered.map((fact) => { + const orderKey = fact.order.join('.') + if (fact.order.length !== 4 + || !isDense(fact.order) || !fact.order.every((value) => safeInt(value)) + || !isDense(fact.control) + || fact.control.length > INDEX_BODY_FACT_CONTROL_LIMIT + || fact.order[1] !== enumPos(KINDS, fact.kind) + || orderKeys.has(orderKey) + || fact.id !== indexBodyFactId(fact.owner_symbol_id, fact.kind, fact.order, + fact.evidence.excerpt_sha256)) { + throw new Error(`Invalid execution fact identity ${fact.id}`) + } + orderKeys.add(orderKey) + let wire: unknown + switch (fact.kind) { + case 'call': + if (!isDense(fact.arguments)) throw new Error(`Sparse call arguments for ${fact.id}`) + wire = [ + fact.callee, fact.target_symbol_id ?? null, + fact.arguments.map(packVal), enumPos(TIMING, fact.scheduling), + ] + break + case 'literal': + wire = [packVal(fact.value), enumPos(ROLES, fact.role)] + break + case 'condition': + wire = [ + enumPos(CONDITIONS, fact.condition_kind), + fact.test ? packVal(fact.test) : null, + ] + break + case 'loop': + wire = [ + enumPos(LOOPS, fact.loop_kind), + fact.test ? packVal(fact.test) : null, + ] + break + case 'parallel': { + const combinator = enumPos(PROMISES, fact.combinator) + if (fact.completion !== COMPLETION[combinator] + || !safeInt(fact.lane_count) + || !isDense(fact.member_fact_ids) + || new Set(fact.member_fact_ids).size !== fact.member_fact_ids.length) + throw new Error(`Invalid parallel completion ${fact.id}`) + wire = [ + combinator, fact.input ? packVal(fact.input) : null, + fact.member_fact_ids.map(ordinal), fact.lane_count, + ] + break + } + case 'return': + case 'throw': + wire = [fact.value ? packVal(fact.value) : null] + break + case 'mutation': + wire = [ + enumPos(MUTATIONS, fact.operation), fact.target, + fact.value ? packVal(fact.value) : null, + ] + break + case 'persistence': + if (!validText(fact.receiver_type)) + throw new Error(`Persistence proof is missing for ${fact.id}`) + wire = [ + enumPos(STORAGE, fact.operation), ordinal(fact.call_fact_id), + fact.resource ? packVal(fact.resource) : null, + fact.receiver_type, + ] + break + } + const row = JSON.stringify([ + fact.id, enumPos(KINDS, fact.kind), + fact.order[0], fact.order[2], fact.order[3], packEvidence(fact.evidence), + fact.control.map(control), enumPos(LEVELS, fact.confidence), + enumPos(SOURCES, fact.source), wire, + ]) + const rowBytes = Buffer.byteLength(row, 'utf8') + bytes += rowBytes + if (rowBytes > MAX_ROW || bytes > MAX_TABLE) + throw new IndexBodyFactBoundsError( + `Execution fact table exceeds its byte bound at ${fact.id}`, + ) + return row + }) + return [1, rows] +} +function safeInt(value: unknown, minimum = 0): value is number { + return typeof value === 'number' + && Number.isSafeInteger(value) + && !Object.is(value, -0) + && value >= minimum +} +function validText(value: unknown, maxBytes = MAX_TEXT): value is string { + return typeof value === 'string' + && value.length > 0 + && !value.includes('\0') + && Buffer.byteLength(value, 'utf8') <= maxBytes +} +function tuple(value: unknown, length: number): unknown[] | null { + return Array.isArray(value) && value.length === length ? value : null +} +function enumValue(values: readonly T[], value: unknown): T | null { + return safeInt(value) && value < values.length ? values[value]! : null +} +function readVal(value: unknown, depth = 0): IndexValue | null { + if (!Array.isArray(value) + || !safeInt(value[0]) || value[0] > 7) return null + if (depth > MAX_DEPTH) return null + if (depth === MAX_DEPTH && [3, 4, 5].includes(value[0]) + && (!Array.isArray(value[1]) || value[1].length > 0)) return null + switch (value[0]) { + case 0: { + return value.length === 2 && scalar(value[1]) + ? { kind: 'literal', value: value[1] } : null + } + case 1: + return value.length === 2 && validText(value[1], 1_024) + ? { kind: 'symbol', symbol_id: value[1] } + : null + case 2: + return (value.length === 2 || (value.length === 3 && value[2] === 1)) + && safeInt(value[1]) + ? { + kind: 'parameter', + position: value[1], + ...(value[2] === 1 ? { scope: 'iteration' as const } : {}), + } + : null + case 3: + case 5: { + if (value.length !== 2 || !Array.isArray(value[1]) + || value[1].length > MAX_ELEMENTS) return null + const values = value[1].map((entry) => readVal(entry, depth + 1)) + if (!values.every((entry): entry is IndexValue => entry !== null)) return null + return value[0] === 3 + ? { kind: 'array', elements: values } + : { kind: 'template', parts: values } + } + case 4: { + if (value.length !== 2 || !Array.isArray(value[1]) + || value[1].length > MAX_ELEMENTS) return null + const keys = new Set() + const entries: IndexObjectEntry[] = [] + for (const raw of value[1]) { + const entry = tuple(raw, 2) + const decoded = entry ? readVal(entry[1], depth + 1) : null + if (!entry || typeof entry[0] !== 'string' || entry[0].includes('\0') + || Buffer.byteLength(entry[0], 'utf8') > MAX_TEXT + || keys.has(entry[0]) || !decoded) return null + keys.add(entry[0]) + entries.push({ key: entry[0], value: decoded }) + } + return { kind: 'object', entries } + } + case 6: + return value.length === 3 && typeof value[1] === 'string' + && SHA256.test(value[1]) && safeInt(value[2]) + ? { kind: 'redacted', sha256: value[1], byte_length: value[2] } + : null + case 7: { + const reason = enumValue(UNKNOWN, value[1]) + return value.length === 2 && reason ? { kind: 'unknown', reason } : null + } + } + return null +} +type DecodedRow = { + id: string; kind: IndexBodyFact['kind']; order: readonly number[] + evidence: IndexFactEvidence; control: readonly unknown[] + confidence: IndexFactConfidence; source: IndexFactSource; payload: unknown +} +function readEvidence(value: unknown, file: string): IndexFactEvidence | null { + const row = tuple(value, 9) + if (!row || !row.slice(0, 8).every((entry) => safeInt(entry, 1)) + || typeof row[8] !== 'string' || !SHA256.test(row[8])) return null + const range = { + start: { line: row[0] as number, column: row[1] as number }, + end: { line: row[2] as number, column: row[3] as number }, + } + const statement_range = { + start: { line: row[4] as number, column: row[5] as number }, + end: { line: row[6] as number, column: row[7] as number }, + } + const compare = (left: IndexPosition, right: IndexPosition): number => + left.line - right.line || left.column - right.column + return compare(range.start, range.end) <= 0 + && compare(statement_range.start, statement_range.end) <= 0 + && compare(statement_range.start, range.start) <= 0 + && compare(range.end, statement_range.end) <= 0 + ? { file_id: file, range, statement_range, excerpt_sha256: row[8] } + : null +} +function decodeRow(value: string, owner: string, file: string): DecodedRow | null { + if (Buffer.byteLength(value, 'utf8') > MAX_ROW) return null + let parsed: unknown + try { + parsed = JSON.parse(value) + } catch { + return null + } + if (JSON.stringify(parsed) !== value) return null + const row = tuple(parsed, 10) + if (!row || !validText(row[0], 64) + || !safeInt(row[1]) || row[1] >= KINDS.length + || !safeInt(row[2]) || !safeInt(row[3]) || !safeInt(row[4]) + || !Array.isArray(row[6]) + || row[6].length > INDEX_BODY_FACT_CONTROL_LIMIT) return null + const kind = KINDS[row[1]]! + const proof = readEvidence(row[5], file) + const confidence = enumValue(LEVELS, row[7]) + const source = enumValue(SOURCES, row[8]) + const order = [row[2], row[1], row[3], row[4]] as number[] + if (!proof || !confidence || !source + || row[0] !== indexBodyFactId(owner, kind, order, proof.excerpt_sha256)) { + return null + } + return { + id: row[0], kind, order, evidence: proof, control: row[6], confidence, source, + payload: row[9], + } +} +export function decodeIndexBodyFactTable( + value: unknown, + owner: string, + file: string, +): readonly IndexBodyFact[] | null { + const table = tuple(value, 2) + if (!validText(owner, 1_024) || !validText(file, 128) + || !table || table[0] !== 1 || !Array.isArray(table[1]) + || table[1].length === 0 || table[1].length > MAX_ROWS) return null + const decoded: DecodedRow[] = [] + let bytes = 0 + for (const value of table[1]) { + if (typeof value !== 'string') return null + bytes += Buffer.byteLength(value, 'utf8') + if (bytes > MAX_TABLE) return null + const row = decodeRow(value, owner, file) + if (!row) return null + decoded.push(row) + } + const ids = decoded.map((row) => row.id) + if (new Set(ids).size !== ids.length + || decoded.some((row, index) => index > 0 + && orderCmp(decoded[index - 1]!.order, row.order) >= 0)) { + return null + } + const idAt = (value: unknown): string | null => + safeInt(value) && value < ids.length ? ids[value]! : null + const control = (value: unknown): IndexControlFrame | null => { + if (!Array.isArray(value) || !safeInt(value[0])) return null + const controller_fact_id = idAt(value[1]) + if (value[0] === 0) { + return value.length === 3 && controller_fact_id + && validText(value[2], 96) + && (['then', 'else', 'truthy', 'falsy', 'nullish', 'default'].includes(value[2]) + || (value[2].startsWith('case:') && value[2].length > 5)) + ? { kind: 'branch', controller_fact_id, arm: value[2] as IndexBranchArm } + : null + } + if (value[0] === 1) return value.length === 2 && controller_fact_id + ? { kind: 'loop', controller_fact_id } : null + if (value[0] === 2) return value.length === 3 && controller_fact_id + && (value[2] === 'each' || safeInt(value[2])) + ? { kind: 'parallel', controller_fact_id, lane: value[2] } : null + const arm = enumValue(['try', 'catch', 'finally'] as const, value[1]) + return value[0] === 3 && value.length === 2 && arm + ? { kind: 'exception', arm } : null + } + const facts: IndexBodyFact[] = [] + for (const row of decoded) { + const frames = row.control.map(control) + if (!frames.every((frame): frame is IndexControlFrame => frame !== null)) return null + const base = { + id: row.id, owner_symbol_id: owner, order: row.order, + evidence: row.evidence, control: frames, + confidence: row.confidence, source: row.source, + } + const wire = Array.isArray(row.payload) ? row.payload : null + let fact: IndexBodyFact | null = null + if (row.kind === 'call' && wire?.length === 4) { + const scheduling = enumValue(TIMING, wire[3]) + const args = Array.isArray(wire[2]) + ? wire[2].map((entry) => readVal(entry)) + : [] + if (validText(wire[0]) && scheduling + && (wire[1] === null || validText(wire[1], 1_024)) + && Array.isArray(wire[2]) + && args.every((entry): entry is IndexValue => entry !== null)) { + fact = { + ...base, kind: 'call', callee: wire[0], + ...(typeof wire[1] === 'string' ? { target_symbol_id: wire[1] } : {}), + arguments: args, scheduling, + } + } + } else if (row.kind === 'literal' && wire?.length === 2) { + const decoded = readVal(wire[0]) + const role = enumValue(ROLES, wire[1]) + if (decoded && role) fact = { ...base, kind: 'literal', value: decoded, role } + } else if (row.kind === 'condition' && wire?.length === 2) { + const condition_kind = enumValue(CONDITIONS, wire[0]) + const test = wire[1] === null ? undefined : readVal(wire[1]) + if (condition_kind && (wire[1] === null || test)) { + fact = { ...base, kind: 'condition', condition_kind, ...(test ? { test } : {}) } + } + } else if (row.kind === 'loop' && wire?.length === 2) { + const loop_kind = enumValue(LOOPS, wire[0]) + const test = wire[1] === null ? undefined : readVal(wire[1]) + if (loop_kind && (wire[1] === null || test)) { + fact = { ...base, kind: 'loop', loop_kind, ...(test ? { test } : {}) } + } + } else if (row.kind === 'parallel' && wire?.length === 4) { + const combinator = enumValue(PROMISES, wire[0]) + const input = wire[1] === null ? undefined : readVal(wire[1]) + const members = Array.isArray(wire[2]) + ? wire[2].map(idAt) + : [] + if (combinator && (wire[1] === null || input) + && Array.isArray(wire[2]) + && members.every((id): id is string => id !== null) + && new Set(members).size === members.length + && safeInt(wire[3])) { + fact = { + ...base, kind: 'parallel', combinator, + completion: COMPLETION[PROMISES.indexOf(combinator)]!, + lane_count: wire[3], + ...(input ? { input } : {}), + member_fact_ids: members, + } + } + } else if ((row.kind === 'return' || row.kind === 'throw') + && wire?.length === 1) { + const decoded = wire[0] === null ? undefined : readVal(wire[0]) + if (wire[0] === null || decoded) { + fact = { ...base, kind: row.kind, ...(decoded ? { value: decoded } : {}) } + } + } else if (row.kind === 'mutation' && wire?.length === 3) { + const operation = enumValue(MUTATIONS, wire[0]) + const decoded = wire[2] === null ? undefined : readVal(wire[2]) + if (operation && validText(wire[1]) + && (wire[2] === null || decoded)) { + fact = { + ...base, kind: 'mutation', operation, target: wire[1], + ...(decoded ? { value: decoded } : {}), + } + } + } else if (row.kind === 'persistence' && wire?.length === 4) { + const operation = enumValue(STORAGE, wire[0]) + const call_fact_id = idAt(wire[1]) + const resource = wire[2] === null ? undefined : readVal(wire[2]) + if (operation && call_fact_id && (wire[2] === null || resource) + && validText(wire[3])) { + fact = { + ...base, kind: 'persistence', operation, call_fact_id, + ...(resource ? { resource } : {}), + receiver_type: wire[3], + } + } + } + if (!fact) return null + facts.push(fact) + } + return facts +} export type IndexFrameworkRole = | 'nest_module' | 'nest_controller' @@ -76,7 +700,6 @@ export type IndexFrameworkRole = | 'prisma_model_reader' | 'prisma_model_writer' | 'prisma_model_access' - export type IndexStorageOperation = | 'create' | 'createMany' @@ -94,15 +717,12 @@ export type IndexStorageOperation = | 'aggregate' | 'groupBy' | '$transaction' - export type IndexRuntimeBoundary = 'client' | 'server' - export type IndexFrameworkMetadata = { storage_operation?: IndexStorageOperation runtime_boundary?: IndexRuntimeBoundary [key: string]: unknown } - export type IndexSymbol = { id: string file_id: string @@ -113,14 +733,52 @@ export type IndexSymbol = { exported: boolean framework_role?: IndexFrameworkRole framework_metadata?: IndexFrameworkMetadata + body_facts?: readonly IndexBodyFact[] +} +export type IndexChannelKind = 'queue' | 'job' | 'event' +export type IndexChannelTransport = + | 'bull' + | 'bullmq' + | 'node-event-emitter' + | 'nestjs-event-emitter' +/** + * Shared channel identity. Only exact, statically resolved identities become + * nodes; dynamic or ambiguous references remain diagnostics/facts. + */ +export type IndexChannelNode = Immutable<{ + id: string + node_kind: 'channel' + channel_kind: IndexChannelKind + transport: IndexChannelTransport + key: string + /** Exact emitter-instance namespace; absent for globally named queues/jobs. */ + scope?: string + parent_channel_id?: string +}> +export function indexChannelId( + input: Omit, +): string { + const descriptor = { + channel_kind: input.channel_kind, + transport: input.transport, + key: input.key, + ...(input.parent_channel_id + ? { parent_channel_id: input.parent_channel_id } + : {}), + ...(input.scope ? { scope: input.scope } : {}), + } + return `channel:${createHash('sha256') + .update(JSON.stringify(descriptor), 'utf8').digest('hex').slice(0, 32)}` } - export type IndexEdgeKind = | 'imports' | 'reexports' | 'declares' | 'calls' | 'enqueues_job' + | 'publishes_to' + | 'routes_through' + | 'consumed_by' | 'extends' | 'implements' | 'param_type' @@ -135,20 +793,19 @@ export type IndexEdgeKind = | 'guards' | 'intercepts' | 'pipes' - export type IndexEdgeConfidence = 'high' | 'medium' | 'low' - export type IndexEdgeSource = | 'typescript-semantic' | 'typescript-syntactic' | 'framework-decorator' + | 'wrapper-summary' | 'heuristic' - export type IndexEdgeEvidence = { file_id: string range: IndexRange + statement_range?: IndexRange + excerpt_sha256?: IndexSha256 } - export type IndexEdge = { from: string to: string @@ -158,14 +815,11 @@ export type IndexEdge = { evidence?: IndexEdgeEvidence metadata?: Record } - export type IndexDiagnosticLevel = 'info' | 'warn' | 'error' - export type IndexDiagnosticEvidence = { file_id: string range?: IndexRange } - export type IndexDiagnostic = { id: string level: IndexDiagnosticLevel diff --git a/src/domain/query/index-status.ts b/src/domain/query/index-status.ts index 420011b4..1e98e6af 100644 --- a/src/domain/query/index-status.ts +++ b/src/domain/query/index-status.ts @@ -3,7 +3,23 @@ import { type GraphAttributes, type GraphEdge, } from '../graph/directed-multigraph.js' -import { CANONICAL_INDEX_FORMAT_VERSION, readBuildState, type SourceSnapshotEntry } from '../index/build-state.js' +import { compareCodeUnits } from '../graph/canonical-json.js' +import { + CANONICAL_INDEX_FORMAT_VERSION, + readBuildState, + type SourceSnapshotEntry, +} from '../index/build-state.js' +import type { + IndexBodyFact, + IndexChannelKind, + IndexChannelNode, + IndexChannelTransport, + IndexControlFrame, + IndexRange, + IndexValue, +} from '../index/model.js' +import { decodeIndexBodyFactTable, indexChannelId } from '../index/model.js' +import { isRecord } from '../../shared/guards.js' export interface QueryGraph { hasNode(id: string): boolean; hasEdge(source: string, target: string): boolean @@ -16,101 +32,597 @@ export interface QueryGraph { export interface ReadyQueryIndex { state: 'ready'; graph: QueryGraph; root_path: string file_hashes: ReadonlyMap; unsupported_sources: readonly SourceSnapshotEntry[] + operation_by_id: ReadonlyMap + operations_by_owner: ReadonlyMap + channels_by_id: ReadonlyMap + channels_by_key: ReadonlyMap } export interface FailedQueryIndex { state: 'unavailable' | 'corrupt'; subject: string } export type QueryIndex = ReadyQueryIndex | FailedQueryIndex -function immutableMap(entries: Iterable): ReadonlyMap { - const values = new Map(entries) - return Object.freeze({ - get size() { return values.size }, - get(key: string) { return values.get(key) }, - has(key: string) { return values.has(key) }, - entries() { return values.entries() }, - keys() { return values.keys() }, - values() { return values.values() }, - forEach( - callback: (value: string, key: string, map: ReadonlyMap) => void, - thisArg?: unknown, - ) { - values.forEach((value, key) => callback.call(thisArg, value, key, this)) - }, - [Symbol.iterator]() { return values[Symbol.iterator]() }, - }) as ReadonlyMap -} - -function graphSnapshot(source: KnowledgeGraph): KnowledgeGraph { - const snapshot = new KnowledgeGraph(source.graph) - for (const [id, attributes] of source.nodeEntries()) snapshot.addNode(id, attributes) - for (const [from, to, attributes, expectedId] of source.edgeEntries()) { - const id = snapshot.addEdge(from, to, attributes) - if (id !== expectedId) throw new Error('Canonical graph edge identity changed while sealing query index') +const SHA256 = /^[a-f0-9]{64}$/ +const MAX_TEXT = 512 +const KINDS = new Set(['queue', 'job', 'event']) +const TRANSPORTS = new Set([ + 'bull', + 'bullmq', + 'node-event-emitter', + 'nestjs-event-emitter', +]) +const RELATIONS = new Set([ + 'publishes_to', + 'routes_through', + 'consumed_by', +]) +const EDGE_SOURCES = new Set([ + 'typescript-semantic', + 'typescript-syntactic', + 'framework-decorator', + 'wrapper-summary', +]) + +class QueryIndexIntegrityError extends Error {} + +function fail(subject: string): never { + throw new QueryIndexIntegrityError(subject) +} + +function nonEmpty(value: unknown): value is string { + return typeof value === 'string' && value.length > 0 && !value.includes('\0') +} + +function bounded(value: unknown, maxBytes: number): value is string { + return nonEmpty(value) + && Buffer.byteLength(value, 'utf8') <= maxBytes +} + +function safeInt(value: unknown, minimum = 0): value is number { + return typeof value === 'number' + && Number.isSafeInteger(value) + && value >= minimum +} + +function exact(value: unknown, required: readonly string[], optional: readonly string[] = []): Record | null { + if (!isRecord(value)) + return null; + const allowed = new Set([...required, ...optional]); + return required.every((key) => Object.hasOwn(value, key)) + && Object.keys(value).every((key) => allowed.has(key)) + ? value + : null; +} + +function comparePos(left: IndexRange['start'], right: IndexRange['start']): number { + return left.line - right.line || left.column - right.column; +} + +function parseRange(value: unknown): IndexRange | null { + const range = exact(value, ['start', 'end']) + const start = exact(range?.start, ['line', 'column']) + const end = exact(range?.end, ['line', 'column']) + if (!range || !start || !end + || !safeInt(start.line, 1) || !safeInt(start.column, 1) + || !safeInt(end.line, 1) || !safeInt(end.column, 1)) return null + const parsed = { + start: { line: start.line, column: start.column }, + end: { line: end.line, column: end.column }, } - return snapshot + return comparePos(parsed.start, parsed.end) <= 0 ? parsed : null } -function immutableQueryGraph(snapshot: KnowledgeGraph): QueryGraph { - return Object.freeze({ - hasNode: (id: string) => snapshot.hasNode(id), - hasEdge: (source: string, target: string) => snapshot.hasEdge(source, target), - nodeEntries: () => snapshot.nodeEntries(), - edgeEntries: () => snapshot.edgeEntries(), - predecessors: (id: string) => snapshot.predecessors(id), - successors: (id: string) => snapshot.successors(id), - edgesBetween: (source: string, target: string) => snapshot.edgesBetween(source, target), - nodeAttributes: (id: string) => snapshot.nodeAttributes(id), - }) +function containsRange(container: IndexRange, value: IndexRange): boolean { + return comparePos(container.start, value.start) <= 0 + && comparePos(value.end, container.end) <= 0 +} + +function sameRange(left: IndexRange, right: IndexRange): boolean { + return comparePos(left.start, right.start) === 0 + && comparePos(left.end, right.end) === 0 +} + +function validBranchArm( + control: Extract, + arm: string, +): boolean { + if (control.condition_kind === 'if') return ['then', 'else'].includes(arm) + if (control.condition_kind === 'switch') { + return arm === 'default' || (arm.startsWith('case:') && arm.length > 5) + } + if (control.condition_kind === 'logical_and') return arm === 'truthy' + if (control.condition_kind === 'logical_or') return arm === 'falsy' + if (control.condition_kind === 'nullish') return arm === 'nullish' + return control.condition_kind === 'ternary' + ? ['truthy', 'falsy'].includes(arm) + : ['then', 'else'].includes(arm) +} + +function edgeProof(attrs: GraphAttributes, fileIds: ReadonlyMap, nodeById: ReadonlyMap): boolean { + const source = attrs.source_file; + const ownerId = attrs.execution_owner_id; + const owner = typeof ownerId === 'string' ? nodeById.get(ownerId) : undefined; + const span = parseRange(owner?.definition_range); + const record = exact(attrs.evidence, ['source', 'range', 'statement_range', 'excerpt_sha256']); + const range = parseRange(record?.range); + const statement = parseRange(record?.statement_range); + return typeof source === 'string' + && fileIds.has(source) + && typeof ownerId === 'string' + && owner?.source_file === source + && owner?.node_kind !== 'file' + && owner?.node_kind !== 'channel' + && span !== null + && record !== null + && EDGE_SOURCES.has(String(record.source)) + && range !== null + && statement !== null + && containsRange(span, statement) + && containsRange(statement, range) + && typeof record.excerpt_sha256 === 'string' + && SHA256.test(record.excerpt_sha256); } -export function failedQueryIndex(state: FailedQueryIndex['state'], subject: string): FailedQueryIndex { +function valueHas(value: IndexValue, test: (candidate: IndexValue) => boolean): boolean { + if (test(value)) + return true; + if (value.kind === 'array') { + return value.elements.some((entry) => valueHas(entry, test)); + } + if (value.kind === 'object') { + return value.entries.some((entry) => valueHas(entry.value, test)); + } + return value.kind === 'template' + && value.parts.some((entry) => valueHas(entry, test)); +} + +function factHas( + fact: IndexBodyFact, + test: (candidate: IndexValue) => boolean, +): boolean { + let values: readonly IndexValue[] + switch (fact.kind) { + case 'call': + values = fact.arguments; break + case 'literal': + values = [fact.value]; break + case 'condition': + case 'loop': + values = fact.test ? [fact.test] : []; break + case 'parallel': + values = fact.input ? [fact.input] : []; break + case 'return': + case 'throw': + case 'mutation': + values = fact.value ? [fact.value] : []; break + case 'persistence': + values = fact.resource ? [fact.resource] : [] + } + return values.some((value) => valueHas(value, test)) +} + +function channelFrom(id: string, attrs: GraphAttributes): IndexChannelNode | null { + if (!nonEmpty(id) + || !KINDS.has(attrs.channel_kind as IndexChannelKind) + || !TRANSPORTS.has(attrs.transport as IndexChannelTransport) + || !bounded(attrs.key, MAX_TEXT) + || (Object.hasOwn(attrs, 'parent_channel_id') + && !nonEmpty(attrs.parent_channel_id)) + || (Object.hasOwn(attrs, 'scope') + && !bounded(attrs.scope, 512))) + return null; + const channel: IndexChannelNode = { + id, + node_kind: 'channel', + channel_kind: attrs.channel_kind as IndexChannelKind, + transport: attrs.transport as IndexChannelTransport, + key: attrs.key, + ...(typeof attrs.parent_channel_id === 'string' + ? { parent_channel_id: attrs.parent_channel_id } + : {}), + ...(typeof attrs.scope === 'string' + ? { scope: attrs.scope } + : {}), + }; + return id === indexChannelId(channel) ? channel : null; +} + +function orderCmp(left: readonly number[], right: readonly number[]): number { + for (let index = 0; index < Math.min(left.length, right.length); index += 1) { + const difference = left[index]! - right[index]! + if (difference !== 0) return difference + } + return left.length - right.length +} + +function freeze(value: T): T { + if (value !== null && typeof value === 'object' && !Object.isFrozen(value)) { + for (const entry of Object.values(value)) freeze(entry) + Object.freeze(value) + } + return value +} + +function sealMap(entries: Iterable): ReadonlyMap { + const values = new Map(entries); + let facade: ReadonlyMap; + facade = { + get size() { return values.size; }, + get(key: K) { return values.get(key); }, + has(key: K) { return values.has(key); }, + entries() { return values.entries(); }, + keys() { return values.keys(); }, + values() { return values.values(); }, + forEach(callback: (value: V, key: K, map: ReadonlyMap) => void, thisArg?: unknown) { + values.forEach((value, key) => callback.call(thisArg, value, key, facade)); + }, + [Symbol.iterator]() { return values[Symbol.iterator](); }, + }; + return Object.freeze(facade); +} + +function sortEntries( + values: ReadonlyMap, +): Array { + return [...values.entries()] + .sort(([left], [right]) => compareCodeUnits(left, right)) +} + +type ExecutionIndexes = Pick; + +function buildMaps(view: KnowledgeGraph, fileIds: ReadonlyMap): ExecutionIndexes { + const nodes = view.nodeEntries(); + const nodeById = new Map(nodes); + const symbolIds = new Set(); + const factById = new Map(); + const ownedFacts = new Map(); + const channels = new Map(); + const byKey = new Map(); + const orderKeys = new Map>(); + for (const [id, attrs] of nodes) { + if (attrs.node_kind === 'channel') { + if (Object.hasOwn(attrs, 'body_facts')) { + fail('canonical channel body facts'); + } + const channel = channelFrom(id, attrs); + if (!channel) + fail('canonical channel node'); + channels.set(id, channel); + continue; + } + if (attrs.node_kind === 'file') { + if (Object.hasOwn(attrs, 'body_facts') + || Object.hasOwn(attrs, 'channel_kind') + || Object.hasOwn(attrs, 'parent_channel_id')) { + fail('canonical file-node execution metadata'); + } + continue; + } + if (Object.hasOwn(attrs, 'channel_kind') + || Object.hasOwn(attrs, 'parent_channel_id')) { + fail('canonical channel discriminator'); + } + symbolIds.add(id); + if (!Object.hasOwn(attrs, 'body_facts')) + continue; + const source = attrs.source_file; + const fileId = typeof source === 'string' + ? fileIds.get(source) + : undefined; + const span = parseRange(attrs.definition_range); + const ownerFile = fileId ? nodeById.get(fileId) : undefined; + if (!fileId || !span || !ownerFile + || ownerFile.node_kind !== 'file') { + fail('canonical operation owner'); + } + const facts = decodeIndexBodyFactTable(attrs.body_facts, id, fileId); + if (!facts) + fail('canonical symbol body facts'); + const orders = orderKeys.get(id) ?? new Set(); + orderKeys.set(id, orders); + for (const fact of facts) { + if (!containsRange(span, fact.evidence.statement_range) + || factById.has(fact.id)) { + fail('canonical operation fact'); + } + const orderKey = fact.order.join('.'); + if (orders.has(orderKey)) + fail('canonical operation order'); + orders.add(orderKey); + factById.set(fact.id, fact); + const owned = ownedFacts.get(id) ?? []; + owned.push(fact); + ownedFacts.set(id, owned); + } + } + for (const channel of channels.values()) { + if (channel.channel_kind === 'job') { + const parent = channel.parent_channel_id + ? channels.get(channel.parent_channel_id) + : undefined; + if (!parent || parent.channel_kind !== 'queue' + || parent.transport !== channel.transport) { + fail('canonical job parent channel'); + } + } + else if (channel.parent_channel_id !== undefined) { + fail('canonical non-job parent channel'); + } + if (channel.channel_kind === 'event') { + if (!bounded(channel.scope, 512)) { + fail('canonical event channel scope'); + } + } + else if (channel.scope !== undefined) { + fail('canonical non-event channel scope'); + } + const keyed = byKey.get(channel.key) ?? []; + keyed.push(channel); + byKey.set(channel.key, keyed); + } + for (const fact of factById.values()) { + if (factHas(fact, (value) => value.kind === 'symbol' && !symbolIds.has(value.symbol_id))) { + fail('canonical operation value reference'); + } + if (fact.kind === 'call' && fact.target_symbol_id + && !symbolIds.has(fact.target_symbol_id)) { + fail('canonical call target'); + } + const controlIds = new Set(); + for (const frame of fact.control) { + if (frame.kind === 'exception') + continue; + if (controlIds.has(frame.controller_fact_id)) { + fail('canonical duplicate control reference'); + } + controlIds.add(frame.controller_fact_id); + const control = factById.get(frame.controller_fact_id); + const expectedKind = frame.kind === 'branch' + ? 'condition' + : frame.kind; + const guardFallthrough = frame.kind === 'branch' + && control?.kind === 'condition' + && control.condition_kind === 'guard'; + if (!control || control.owner_symbol_id !== fact.owner_symbol_id + || control.kind !== expectedKind + || orderCmp(control.order, fact.order) >= 0 + || (!guardFallthrough && !containsRange(frame.kind === 'parallel' + ? control.evidence.range + : control.evidence.statement_range, fact.evidence.range)) + || (frame.kind === 'branch' && control.kind === 'condition' + && !validBranchArm(control, frame.arm)) + || (frame.kind === 'parallel' && control.kind === 'parallel' + && (frame.lane === 'each' + ? control.lane_count === 0 + : frame.lane >= control.lane_count)) + || (fact.kind === 'call' && frame.kind === 'parallel' + && control.kind === 'parallel' + && !control.member_fact_ids.includes(fact.id))) { + fail('canonical operation control reference'); + } + } + if (factHas(fact, (value) => value.kind === 'parameter' && value.scope === 'iteration') + && !fact.control.some((frame) => { + const control = frame.kind === 'loop' + ? factById.get(frame.controller_fact_id) + : undefined; + return control?.kind === 'loop' + && control.loop_kind === 'array_iteration'; + })) { + fail('canonical iteration parameter'); + } + if (fact.kind === 'parallel') { + const laneCount = fact.input?.kind === 'array' + ? fact.input.elements.length + : 0; + if (fact.member_fact_ids.some((id) => { + const member = factById.get(id); + const frame = member?.control.find((candidate): candidate is Extract => candidate.kind === 'parallel' + && candidate.controller_fact_id === fact.id); + const loop = frame?.lane === 'each' + ? member?.control.some((candidate) => { + const control = candidate.kind === 'loop' + ? factById.get(candidate.controller_fact_id) + : undefined; + return control?.kind === 'loop' + && control.loop_kind === 'array_iteration'; + }) + : true; + return !member || member.kind !== 'call' || !frame || !loop + || member.owner_symbol_id !== fact.owner_symbol_id; + }) || fact.lane_count !== laneCount) { + fail('canonical parallel member reference'); + } + } + if (fact.kind === 'persistence') { + const call = factById.get(fact.call_fact_id); + if (!call || call.kind !== 'call' + || call.owner_symbol_id !== fact.owner_symbol_id + || !sameRange(call.evidence.range, fact.evidence.range) + || !sameRange(call.evidence.statement_range, fact.evidence.statement_range) + || call.evidence.excerpt_sha256 !== fact.evidence.excerpt_sha256 + || call.order[0] !== fact.order[0] + || call.order[2] !== fact.order[2] + || call.order[3] !== fact.order[3] + || JSON.stringify(call.control) !== JSON.stringify(fact.control) + || !bounded(fact.receiver_type, MAX_TEXT)) { + fail('canonical persistence call reference'); + } + } + } + const routes = new Map(); + for (const [source, target, attrs] of view.edgeEntries()) { + const relation = attrs.relation; + const fromChannel = channels.get(source); + const toChannel = channels.get(target); + const usesChannel = fromChannel !== undefined || toChannel !== undefined; + if (!usesChannel && !RELATIONS.has(String(relation))) + continue; + if (!RELATIONS.has(String(relation))) { + fail('canonical channel relation'); + } + if (!edgeProof(attrs, fileIds, nodeById)) { + fail('canonical channel evidence'); + } + const edgeOwner = attrs.execution_owner_id; + if (relation === 'publishes_to') { + if (!symbolIds.has(source) || !toChannel + || source !== edgeOwner + || !['queue', 'job', 'event'].includes(toChannel.channel_kind)) { + fail('canonical publishes_to endpoints'); + } + } + else if (relation === 'routes_through') { + if (!fromChannel || fromChannel.channel_kind !== 'job' + || !toChannel || toChannel.channel_kind !== 'queue' + || fromChannel.parent_channel_id !== target + || fromChannel.transport !== toChannel.transport) { + fail('canonical routes_through endpoints'); + } + routes.set(source, (routes.get(source) ?? 0) + 1); + } + else if (relation === 'consumed_by') { + if (!fromChannel || !symbolIds.has(target) || toChannel) { + fail('canonical consumed_by endpoints'); + } + } + } + for (const channel of channels.values()) { + if (channel.channel_kind === 'job' + && routes.get(channel.id) !== 1) { + fail('canonical job routing'); + } + } + for (const values of ownedFacts.values()) { + values.sort((left, right) => orderCmp(left.order, right.order) || compareCodeUnits(left.id, right.id)); + values.forEach(freeze); + Object.freeze(values); + } + for (const values of byKey.values()) { + values.sort((left, right) => compareCodeUnits(left.id, right.id)); + values.forEach(freeze); + Object.freeze(values); + } + factById.forEach(freeze); + channels.forEach(freeze); + return { + operation_by_id: sealMap(sortEntries(factById)), + operations_by_owner: sealMap(sortEntries(ownedFacts)), + channels_by_id: sealMap(sortEntries(channels)), + channels_by_key: sealMap(sortEntries(byKey)), + }; +} + +function copyGraph(source: KnowledgeGraph): KnowledgeGraph { + const view = new KnowledgeGraph(source.graph) + for (const [id, attrs] of source.nodeEntries()) { + view.addNode(id, attrs) + } + for (const [from, to, attrs, expectedId] of source.edgeEntries()) { + const id = view.addEdge(from, to, attrs) + if (id !== expectedId) { + throw new Error('Canonical graph edge identity changed while sealing query index') + } + } + return view +} + +function sealGraph(view: KnowledgeGraph): QueryGraph { + return Object.freeze({ + hasNode: (id: string) => view.hasNode(id), + hasEdge: (source: string, target: string) => view.hasEdge(source, target), + nodeEntries: () => view.nodeEntries(), + edgeEntries: () => view.edgeEntries(), + predecessors: (id: string) => view.predecessors(id), + successors: (id: string) => view.successors(id), + edgesBetween: (source: string, target: string) => view.edgesBetween(source, target), + nodeAttributes: (id: string) => view.nodeAttributes(id), + }); +} + +export function failedQueryIndex( + state: FailedQueryIndex['state'], + subject: string, +): FailedQueryIndex { return { state, subject } } export function inspectQueryIndex(graph: KnowledgeGraph): QueryIndex { - let snapshot: KnowledgeGraph + let view: KnowledgeGraph try { - snapshot = graphSnapshot(graph) + view = copyGraph(graph) } catch { return failedQueryIndex('corrupt', 'canonical graph snapshot') } - const build = readBuildState(snapshot) - const root = snapshot.graph.root_path - if (!build || snapshot.graph.canonical_typescript_index !== true - || snapshot.graph.schema_version !== CANONICAL_INDEX_FORMAT_VERSION + const build = readBuildState(view) + const root = view.graph.root_path + if (!build || view.graph.canonical_typescript_index !== true + || view.graph.schema_version !== CANONICAL_INDEX_FORMAT_VERSION || typeof root !== 'string' || root.trim().length === 0 || build.source_root.root_path !== root) { return failedQueryIndex('corrupt', 'canonical TypeScript index metadata') } if (build.completeness.summary.state !== 'complete' || build.completeness.supported_failures.length > 0) { - return failedQueryIndex('unavailable', 'canonical TypeScript index incomplete') + return failedQueryIndex( + 'unavailable', + 'canonical TypeScript index incomplete', + ) } const hashes = new Map() - for (const [, attributes] of snapshot.nodeEntries()) { - if (attributes.node_kind !== 'file') continue - const sourceFile = attributes.source_file - const contentHash = attributes.content_hash - if (typeof sourceFile !== 'string' || typeof contentHash !== 'string' - || !/^[a-f0-9]{64}$/.test(contentHash)) { + const fileIds = new Map() + for (const [id, attrs] of view.nodeEntries()) { + if (attrs.node_kind !== 'file') continue + const source = attrs.source_file + const hash = attrs.content_hash + if (typeof source !== 'string' || typeof hash !== 'string' + || !SHA256.test(hash)) { return failedQueryIndex('corrupt', 'canonical file-node hash') } - if (hashes.has(sourceFile)) { - return failedQueryIndex('corrupt', sourceFile) + if (hashes.has(source) || fileIds.has(source)) { + return failedQueryIndex('corrupt', source) } - hashes.set(sourceFile, contentHash) + hashes.set(source, hash) + fileIds.set(source, id) } if (hashes.size !== build.sources.supported.length - || build.sources.supported.some((source) => hashes.get(source.path) !== source.hash)) { + || build.sources.supported.some((source) => + hashes.get(source.path) !== source.hash)) { return failedQueryIndex('corrupt', 'canonical file-node coverage') } + let execution: ExecutionIndexes + try { + execution = buildMaps(view, fileIds) + } catch (error) { + return failedQueryIndex( + 'corrupt', + error instanceof QueryIndexIntegrityError + ? error.message + : 'canonical execution index', + ) + } + for (const [id, attrs] of view.nodeEntries()) { + if (!Object.hasOwn(attrs, 'body_facts')) continue + const { body_facts: _decoded, ...retained } = attrs + view.replaceNodeAttributes(id, retained) + } + return Object.freeze({ - state: 'ready', graph: immutableQueryGraph(snapshot), root_path: root, - file_hashes: immutableMap(hashes), - unsupported_sources: Object.freeze(build.sources.unsupported.map((source) => Object.freeze({ ...source }))), + state: 'ready', + graph: sealGraph(view), + root_path: root, + file_hashes: sealMap(hashes), + unsupported_sources: Object.freeze( + build.sources.unsupported.map((source) => Object.freeze({ ...source })), + ), + ...execution, }) } diff --git a/src/domain/query/rank.ts b/src/domain/query/rank.ts index cef95cda..1a82096c 100644 --- a/src/domain/query/rank.ts +++ b/src/domain/query/rank.ts @@ -137,6 +137,10 @@ function buildCorpus(index: ReadyQueryIndex): Corpus { for (const [id, attributes] of index.graph.nodeEntries()) { const file = text(attributes, 'source_file') const kind = text(attributes, 'node_kind') + // Shared execution channels are traversal infrastructure for retrieval v2. + // Keeping them out of the v1 lexical corpus prevents their labels from + // changing document frequency and therefore existing symbol ranking. + if (kind === 'channel') continue if (!paths.has(file)) paths.set(file, field(file, 7)) const pathField = paths.get(file) ?? null const fields = [ diff --git a/tests/unit/canonical-index-execution-hardening.test.ts b/tests/unit/canonical-index-execution-hardening.test.ts new file mode 100644 index 00000000..49699ece --- /dev/null +++ b/tests/unit/canonical-index-execution-hardening.test.ts @@ -0,0 +1,771 @@ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' + +import { afterEach, describe, expect, it } from 'vitest' + +import { buildCanonicalTypeScriptIndex } from '../../src/adapters/typescript/index.js' +import { decodeIndexBodyFactTable } from '../../src/domain/index/model.js' + +type GraphNode = Record +type GraphEdge = readonly [string, string, Record, string] +type BodyFact = Record & { + kind: string +} + +const sandboxes: string[] = [] + +afterEach(() => { + for (const sandbox of sandboxes.splice(0)) { + rmSync(sandbox, { recursive: true, force: true }) + } +}) + +function build( + sources: Record, + options: { reverse?: boolean } = {}, +) { + const root = mkdtempSync(join(tmpdir(), 'madar-execution-hardening-')) + sandboxes.push(root) + const files = Object.entries(sources).map(([path, source]) => { + const absolute = join(root, path) + mkdirSync(dirname(absolute), { recursive: true }) + writeFileSync(absolute, source, 'utf8') + return absolute + }) + const result = buildCanonicalTypeScriptIndex({ + root, + files: options.reverse ? files.reverse() : files, + }) + return { + result, + nodes: new Map(result.graph.nodeEntries()), + edges: result.graph.edgeEntries() as GraphEdge[], + } +} + +function symbol( + nodes: ReadonlyMap, + qualifiedName: string, +): readonly [string, GraphNode] { + const entry = [...nodes].find(([, attributes]) => + attributes.qualified_name === qualifiedName) + if (!entry) throw new Error(`Missing fixture symbol ${qualifiedName}`) + return entry +} + +function channels( + nodes: ReadonlyMap, + predicate: (attributes: GraphNode) => boolean, +): Array { + return [...nodes].filter(([, attributes]) => + attributes.node_kind === 'channel' && predicate(attributes)) +} + +function facts( + nodes: ReadonlyMap, + [ownerId, attributes]: readonly [string, GraphNode], + kind?: string, +): BodyFact[] { + if (!Object.hasOwn(attributes, 'body_facts')) return [] + const sourceFile = attributes.source_file + const file = typeof sourceFile === 'string' + ? [...nodes].find(([, candidate]) => + candidate.node_kind === 'file' + && candidate.source_file === sourceFile) + : undefined + if (!file) throw new Error(`Missing file node for ${ownerId}`) + const decoded = decodeIndexBodyFactTable( + attributes.body_facts, + ownerId, + file[0], + ) + if (!decoded) throw new Error(`Invalid body-fact table for ${ownerId}`) + const all = [...decoded] as BodyFact[] + return kind ? all.filter((fact) => fact.kind === kind) : all +} + +function hasEdge( + edges: readonly GraphEdge[], + from: string, + to: string, + relation: string, +): boolean { + return edges.some(([source, target, attributes]) => + source === from + && target === to + && attributes.relation === relation) +} + +function outgoing( + edges: readonly GraphEdge[], + from: string, + relation: string, +): GraphEdge[] { + return edges.filter(([source, , attributes]) => + source === from && attributes.relation === relation) +} + +describe('canonical TypeScript execution hardening', () => { + it('resolves GoValidate-style Map-backed queue wrappers and inline worker delegates exactly', () => { + const source = `import { Queue, Worker, type Job } from 'bullmq' + +type AssemblyJobData = { ideaId: string } +const QUEUE_NAME = 'assembly-queue' +const JOB_NAME = 'assemble_report' + +class QueueRegistryService { + private readonly queues = new Map() + + addJob(queueName: string, jobName: string, data: AssemblyJobData) { + const queue = this.queues.get(queueName) + if (!queue) throw new Error('Queue not registered') + return queue.add(jobName, data) + } + + registerWorker( + queueName: string, + processor: (job: Job) => Promise, + ) { + return new Worker(queueName, processor) + } +} + +class AssemblyWorker { + constructor(private readonly registry: QueueRegistryService) {} + + start() { + return this.registry.registerWorker( + 'assembly-queue', + (job) => this.process(job), + ) + } + + async process(job: Job): Promise { + void job.data.ideaId + } +} + +export function dispatch( + registry: QueueRegistryService, + ideaId: string, +) { + return registry.addJob(QUEUE_NAME, JOB_NAME, { ideaId }) +} +` + const { nodes, edges } = build({ 'src/queue-registry.ts': source }) + const queue = channels(nodes, (node) => + node.channel_kind === 'queue' + && node.transport === 'bullmq' + && node.key === 'assembly-queue') + expect(queue).toHaveLength(1) + const job = channels(nodes, (node) => + node.channel_kind === 'job' + && node.transport === 'bullmq' + && node.key === 'assemble_report' + && node.parent_channel_id === queue[0]![0]) + expect(job).toHaveLength(1) + + const [dispatchId] = symbol(nodes, 'dispatch') + const [processId] = symbol(nodes, 'AssemblyWorker.process') + expect(hasEdge(edges, dispatchId, job[0]![0], 'publishes_to')).toBe(true) + expect(hasEdge(edges, job[0]![0], queue[0]![0], 'routes_through')).toBe(true) + expect(hasEdge(edges, queue[0]![0], processId, 'consumed_by')).toBe(true) + expect(outgoing(edges, queue[0]![0], 'consumed_by')).toHaveLength(1) + }) + + it('expands two wrapper hops but never joins cycles, dynamics, or unmatched channel halves', () => { + const source = `import { Queue, Worker, type Job } from 'bullmq' + +type Payload = { id: string } +const reports = new Queue('reports') + +function inner(queueName: string, jobName: string, data: Payload) { + if (queueName !== 'reports') throw new Error('wrong queue') + return reports.add(jobName, data) +} + +function outer(queueName: string, jobName: string, data: Payload) { + return inner(queueName, jobName, data) +} + +export function publishThroughTwoHops(id: string) { + return outer('reports', 'complete', { id }) +} + +function cycleA(queueName: string): unknown { + return cycleB(queueName) +} + +function cycleB(queueName: string): unknown { + return cycleA(queueName) +} + +export function publishCycle(queueName: string) { + return cycleA(queueName) +} + +export function publishDynamic( + queue: Queue, + jobName: string, + id: string, +) { + return queue.add(jobName, { id }) +} + +export function publishOnly(id: string) { + return new Queue('producer-only').add('orphan', { id }) +} + +export async function consumeReports(job: Job): Promise { + void job.data.id +} + +export const reportWorker = new Worker('reports', consumeReports) + +export async function consumeOnly(job: Job): Promise { + void job.data.id +} + +export const consumerOnlyWorker = + new Worker('consumer-only', consumeOnly) +` + const { nodes, edges } = build({ 'src/wrappers.ts': source }) + const reportQueue = channels(nodes, (node) => + node.channel_kind === 'queue' + && node.transport === 'bullmq' + && node.key === 'reports') + expect(reportQueue).toHaveLength(1) + const completeJob = channels(nodes, (node) => + node.channel_kind === 'job' + && node.transport === 'bullmq' + && node.key === 'complete' + && node.parent_channel_id === reportQueue[0]![0]) + expect(completeJob).toHaveLength(1) + + const [successId] = symbol(nodes, 'publishThroughTwoHops') + const [cycleId] = symbol(nodes, 'publishCycle') + const [dynamicId] = symbol(nodes, 'publishDynamic') + const [consumerId] = symbol(nodes, 'consumeReports') + const [consumerOnlyId] = symbol(nodes, 'consumeOnly') + + expect(hasEdge(edges, successId, completeJob[0]![0], 'publishes_to')).toBe(true) + expect(hasEdge(edges, reportQueue[0]![0], consumerId, 'consumed_by')).toBe(true) + expect(outgoing(edges, cycleId, 'publishes_to')).toEqual([]) + expect(outgoing(edges, dynamicId, 'publishes_to')).toEqual([]) + expect(hasEdge(edges, reportQueue[0]![0], consumerOnlyId, 'consumed_by')).toBe(false) + + const producerOnly = channels(nodes, (node) => + node.channel_kind === 'queue' && node.key === 'producer-only') + const consumerOnly = channels(nodes, (node) => + node.channel_kind === 'queue' && node.key === 'consumer-only') + expect(producerOnly).toHaveLength(1) + expect(consumerOnly).toHaveLength(1) + expect(outgoing(edges, producerOnly[0]![0], 'consumed_by')).toEqual([]) + expect(hasEdge(edges, consumerOnly[0]![0], consumerOnlyId, 'consumed_by')).toBe(true) + }) + + it('requires receiver proof for persistence and recognizes imported filesystem writes', () => { + const source = `import type { MongoRepository } from 'typeorm' +import { writeFile } from 'node:fs/promises' + +type Idea = { id: string; status: string } + +class CoincidentalCache { + update(id: string, value: unknown): void { + void id + void value + } +} + +export async function persistIdea( + repository: MongoRepository, + id: string, +) { + return repository.update(id, { status: 'complete' }) +} + +export function updateCache(cache: CoincidentalCache, id: string) { + return cache.update(id, { status: 'complete' }) +} + +export async function persistArtifact(path: string, value: string) { + await writeFile(path, value, 'utf8') +} + +export async function persistConditionally( + repository: MongoRepository, + id: string, + enabled: boolean, +) { + if (enabled) return repository.update(id, { status: 'complete' }) +} + +class TaskDelegate { + update(id: string): void { + void id + } +} + +export function coincidentalDelegate(delegate: TaskDelegate, id: string) { + return delegate.update(id) +} +` + const { nodes } = build({ 'src/persistence.ts': source }) + const persistIdea = facts(nodes, symbol(nodes, 'persistIdea'), 'persistence') + const updateCache = facts(nodes, symbol(nodes, 'updateCache'), 'persistence') + const persistArtifact = facts(nodes, symbol(nodes, 'persistArtifact'), 'persistence') + const conditional = facts( + nodes, + symbol(nodes, 'persistConditionally'), + 'persistence', + ) + const coincidentalDelegate = facts( + nodes, + symbol(nodes, 'coincidentalDelegate'), + 'persistence', + ) + + expect(persistIdea).toEqual([ + expect.objectContaining({ + kind: 'persistence', + operation: 'update', + call_fact_id: expect.any(String), + receiver_type: expect.stringMatching(/MongoRepository/), + }), + ]) + expect(updateCache).toEqual([]) + expect(persistArtifact).toEqual([ + expect.objectContaining({ + kind: 'persistence', + operation: 'file_write', + call_fact_id: expect.any(String), + }), + ]) + expect(coincidentalDelegate).toEqual([]) + expect(conditional[0]?.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'branch', arm: 'then' }), + ])) + + const callIds = new Set(facts(nodes, symbol(nodes, 'persistIdea'), 'call') + .map((fact) => fact.id)) + expect(callIds.has(String(persistIdea[0]!.call_fact_id))).toBe(true) + }) + + it('scopes identical event keys by emitter identity and prevents cross-handler edges', () => { + const source = `import { EventEmitter } from 'node:events' + +const domainEvents = new EventEmitter() +const auditEvents = new EventEmitter() + +export function emitDomain() { + domainEvents.emit('ready', { kind: 'domain' }) +} + +export function emitAudit() { + auditEvents.emit('ready', { kind: 'audit' }) +} + +export function onDomain(): void {} +export function onAudit(): void {} + +export function wireDomain() { + domainEvents.on('ready', onDomain) +} + +export function wireAudit() { + auditEvents.on('ready', onAudit) +} +` + const { nodes, edges } = build({ 'src/events.ts': source }) + const ready = channels(nodes, (node) => + node.channel_kind === 'event' + && node.transport === 'node-event-emitter' + && node.key === 'ready') + expect(ready).toHaveLength(2) + expect(new Set(ready.map(([, node]) => node.scope)).size).toBe(2) + + const [emitDomain] = symbol(nodes, 'emitDomain') + const [emitAudit] = symbol(nodes, 'emitAudit') + const [onDomain] = symbol(nodes, 'onDomain') + const [onAudit] = symbol(nodes, 'onAudit') + const domainEvent = ready.find(([id]) => + hasEdge(edges, emitDomain, id, 'publishes_to')) + const auditEvent = ready.find(([id]) => + hasEdge(edges, emitAudit, id, 'publishes_to')) + expect(domainEvent).toBeDefined() + expect(auditEvent).toBeDefined() + expect(domainEvent?.[0]).not.toBe(auditEvent?.[0]) + expect(hasEdge(edges, domainEvent![0], onDomain, 'consumed_by')).toBe(true) + expect(hasEdge(edges, domainEvent![0], onAudit, 'consumed_by')).toBe(false) + expect(hasEdge(edges, auditEvent![0], onAudit, 'consumed_by')).toBe(true) + expect(hasEdge(edges, auditEvent![0], onDomain, 'consumed_by')).toBe(false) + }) + + it('records Promise completion semantics and bounds secrets, long values, and nesting', () => { + const longValue = 'x'.repeat(600) + const unicodeKey = '😀'.repeat(200) + const templateParts = Array.from( + { length: 17 }, + (_, index) => '${value}part' + index, + ).join('') + const source = `export const SECRET_API_KEY = 'sk-test-never-expose' +export const LONG_TEXT = '${longValue}' +export const DEEP_VALUE = [[[[[[['too-deep']]]]]]] + +export async function coordinate(tasks: Array>) { + const all = await Promise.all(tasks) + const settled = await Promise.allSettled(tasks) + const any = await Promise.any(tasks) + const race = await Promise.race(tasks) + return { all, settled, any, race } +} + +declare function first(): Promise +declare function second(): Promise +declare function check(): boolean +declare function step(): void + +export async function exactLanes() { + return Promise.all([first(), second()]) +} + +export function repeatedCondition() { + while (check()) step() +} + +export function redactMutation(config: { password: string }) { + config.password = 'hunter2' +} + +export function jsonLosslessNumbers() { + return [1e400, -0] +} + +export function duplicateObject() { + return { value: 1, value: 2 } +} + +export function boundedLargeTemplate(value: string) { + return \`head${templateParts}\` +} + +export function boundedUnicodeMutation(record: Record) { + record['${unicodeKey}'] = 'safe' +} +` + const { nodes } = build({ 'src/bounds.ts': source }) + const parallel = facts(nodes, symbol(nodes, 'coordinate'), 'parallel') + expect(parallel.map((fact) => [fact.combinator, fact.completion])).toEqual([ + ['all', 'all_or_first_rejection'], + ['allSettled', 'all_settled'], + ['any', 'first_fulfilled'], + ['race', 'first_settled'], + ]) + + const secretJson = JSON.stringify(facts(nodes, symbol(nodes, 'SECRET_API_KEY'))) + const longJson = JSON.stringify(facts(nodes, symbol(nodes, 'LONG_TEXT'))) + const deepJson = JSON.stringify(facts(nodes, symbol(nodes, 'DEEP_VALUE'))) + expect(secretJson).not.toContain('sk-test-never-expose') + expect(secretJson).toContain('"kind":"redacted"') + expect(longJson).not.toContain(longValue) + expect(longJson).toContain('"kind":"redacted"') + expect(deepJson).not.toContain('too-deep') + expect(deepJson).toContain('"kind":"unknown"') + + const laneCalls = facts(nodes, symbol(nodes, 'exactLanes'), 'call') + .filter((fact) => ['first', 'second'].includes(String(fact.callee))) + expect(laneCalls.map((fact) => + (fact.control as Array>) + .find((frame) => frame.kind === 'parallel')?.lane)).toEqual([0, 1]) + + const repeated = facts(nodes, symbol(nodes, 'repeatedCondition'), 'call') + for (const fact of repeated) { + expect(fact.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'loop' }), + ])) + } + + const mutationJson = JSON.stringify( + facts(nodes, symbol(nodes, 'redactMutation'), 'mutation'), + ) + expect(mutationJson).not.toContain('hunter2') + expect(mutationJson).toContain('"kind":"redacted"') + + const numbers = JSON.stringify( + facts(nodes, symbol(nodes, 'jsonLosslessNumbers'), 'return'), + ) + expect(numbers.match(/"kind":"unknown"/g)).toHaveLength(2) + + const duplicate = facts( + nodes, + symbol(nodes, 'duplicateObject'), + 'return', + )[0]?.value as { entries?: Array<{ value: { value?: unknown } }> } + expect(duplicate.entries).toHaveLength(1) + expect(duplicate.entries?.[0]?.value.value).toBe(2) + + const template = JSON.stringify( + facts(nodes, symbol(nodes, 'boundedLargeTemplate'), 'return'), + ) + expect(template).toContain('"kind":"unknown"') + const unicodeMutation = facts( + nodes, + symbol(nodes, 'boundedUnicodeMutation'), + 'mutation', + )[0] + expect(Buffer.byteLength(String(unicodeMutation?.target), 'utf8')) + .toBeLessThanOrEqual(256) + }) + + it('deduplicates structural routes and follows exact cross-file channels', () => { + const sources = { + 'src/shared.ts': `import { EventEmitter } from 'node:events' +import { Queue } from 'bullmq' +export const reports = new Queue('reports') +export const events = new EventEmitter() +`, + 'src/publish.ts': `import { events, reports } from './shared.js' +export function publishOne() { + reports.add('complete', { id: 1 }) + events.emit('ready', { id: 1 }) +} +export function publishTwo() { + return reports.add('complete', { id: 2 }) +} +`, + } + const { nodes, edges } = build(sources) + const queue = channels(nodes, (node) => + node.channel_kind === 'queue' && node.key === 'reports') + const job = channels(nodes, (node) => + node.channel_kind === 'job' + && node.key === 'complete' + && node.parent_channel_id === queue[0]?.[0]) + const event = channels(nodes, (node) => + node.channel_kind === 'event' && node.key === 'ready') + expect(queue).toHaveLength(1) + expect(job).toHaveLength(1) + expect(event).toHaveLength(1) + expect(outgoing(edges, job[0]![0], 'routes_through')).toHaveLength(1) + expect(hasEdge( + edges, + symbol(nodes, 'publishOne')[0], + job[0]![0], + 'publishes_to', + )).toBe(true) + expect(hasEdge( + edges, + symbol(nodes, 'publishTwo')[0], + job[0]![0], + 'publishes_to', + )).toBe(true) + expect(hasEdge( + edges, + symbol(nodes, 'publishOne')[0], + event[0]![0], + 'publishes_to', + )).toBe(true) + }) + + it('keeps guards, nullish flow, deferred closures, and shadowed globals exact', () => { + const source = `import { Queue } from 'bullmq' +import type { Repository } from 'typeorm' + +const reports = new Queue('reports') +const Promise = { all: (values: unknown[]) => values } +class FakeEventEmitter { emit(_event: string): void {} } +namespace Local { + export class Repository { + update(_value: T): void {} + } +} + +declare function fallback(): string +declare function afterGuard(): void +declare function deadCode(): void + +export function flow(value: string | null, enabled: boolean) { + const selected = value ?? fallback() + if (!enabled) return selected + afterGuard() + return selected + deadCode() +} + +export function returnedClosure() { + return () => reports.add('deferred', {}) +} + +export function scheduled() { + setTimeout(() => reports.add('scheduled', {}), 0) +} + +export function dynamicJob(jobName: string) { + return reports.add(jobName, {}) +} + +export function shadowedPromise(tasks: unknown[]) { + return Promise.all(tasks.map((task) => task)) +} + +export class FakeService { + constructor( + private readonly events: FakeEventEmitter, + private readonly repository: Local.Repository, + _realTypeOnly: Repository, + ) {} + + run(): void { + this.events.emit('ready') + this.repository.update('value') + } +} +` + const { nodes, edges } = build({ 'src/exactness.ts': source }) + const flow = facts(nodes, symbol(nodes, 'flow')) + const fallbackCall = flow.find((fact) => + fact.kind === 'call' && fact.callee === 'fallback') + expect(fallbackCall?.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'branch', arm: 'nullish' }), + ])) + const nullish = flow.find((fact) => fact.kind === 'condition' + && fact.condition_kind === 'nullish') + expect(nullish).toBeDefined() + const guarded = flow.find((fact) => + fact.kind === 'call' && fact.callee === 'afterGuard') + expect(guarded?.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'branch', arm: 'else' }), + ])) + expect(flow.some((fact) => + fact.kind === 'call' && fact.callee === 'deadCode')).toBe(false) + for (const name of ['returnedClosure', 'scheduled']) { + expect(outgoing(edges, symbol(nodes, name)[0], 'publishes_to')).toEqual([]) + } + const reportQueue = channels(nodes, (node) => + node.channel_kind === 'queue' && node.key === 'reports') + expect(reportQueue).toHaveLength(1) + expect(hasEdge( + edges, + symbol(nodes, 'dynamicJob')[0], + reportQueue[0]![0], + 'publishes_to', + )).toBe(true) + expect(channels(nodes, (node) => + node.channel_kind === 'job' && node.parent_channel_id === reportQueue[0]![0])) + .toEqual([]) + expect(facts(nodes, symbol(nodes, 'shadowedPromise'), 'parallel')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'FakeService.run'), 'persistence')).toEqual([]) + expect(channels(nodes, (node) => + node.channel_kind === 'event' && node.key === 'ready')).toEqual([]) + }) + + it('redacts literal and comment trivia from every structural display field', () => { + const secrets = [ + 'sk-live-comment-secret', + 'sk-test-call-secret', + 'sk-test-target-secret', + 'sk-test-switch-secret', + ] + const source = `export function sensitive( + client: (value: string) => { send(): void }, + target: Record, + value: string, + code: string, +) { + client/* ${secrets[0]} */('${secrets[1]}').send() + target/* credential */['${secrets[2]}'] = value + switch (code) { + case '${secrets[3]}': + return + } +} +` + const { result } = build({ 'src/secrets.ts': source }) + const serialized = JSON.stringify(result.graph.nodeEntries()) + for (const secret of secrets) expect(serialized).not.toContain(secret) + expect(serialized).toContain('') + expect(serialized).not.toContain('credential') + }) + + it('separates same-named Nest classes and Bull transports deterministically', () => { + const sources = { + 'src/legacy.ts': `import { InjectQueue } from '@nestjs/bull' +import type { Queue } from 'bull' +export class WorkerService { + constructor(@InjectQueue('reports') private readonly reports: Queue) {} + publish() { return this.reports.add('legacy', {}) } +} +`, + 'src/modern.ts': `import { InjectQueue } from '@nestjs/bullmq' +import type { Queue } from 'bullmq' +export class WorkerService { + constructor(@InjectQueue('reports') private readonly reports: Queue) {} + publish() { return this.reports.add('modern', {}) } +} +`, + } + const forward = build(sources) + const reverse = build(sources, { reverse: true }) + expect(reverse.result.graph.nodeEntries()) + .toEqual(forward.result.graph.nodeEntries()) + expect(reverse.result.graph.edgeEntries()) + .toEqual(forward.result.graph.edgeEntries()) + const queues = channels(forward.nodes, (node) => + node.channel_kind === 'queue' && node.key === 'reports') + expect(queues.map(([, node]) => node.transport).sort()) + .toEqual(['bull', 'bullmq']) + const jobs = channels(forward.nodes, (node) => node.channel_kind === 'job') + expect(jobs.map(([, node]) => [node.key, node.transport]).sort()) + .toEqual([['legacy', 'bull'], ['modern', 'bullmq']]) + }) + + it('marks owner overflow incomplete and projects no partial channel topology', () => { + const calls = Array.from( + { length: 8_193 }, + (_, index) => ` reports.add('job-${index}', {})`, + ).join('\n') + const source = `import { Queue } from 'bullmq' +const reports = new Queue('reports') +export function overflow() { +${calls} +} +` + const { result, nodes, edges } = build({ 'src/overflow.ts': source }) + const [ownerId, owner] = symbol(nodes, 'overflow') + expect(Object.hasOwn(owner, 'body_facts')).toBe(false) + expect(result.diagnostics).toEqual(expect.arrayContaining([ + expect.objectContaining({ + level: 'error', + id: expect.stringContaining('execution.owner-bound'), + }), + ])) + expect(outgoing(edges, ownerId, 'publishes_to')).toEqual([]) + }) + + it('is deterministic when the scanner enumerates execution files in reverse', () => { + const sources = { + 'src/producer.ts': `import { Queue } from 'bullmq' +const queue = new Queue('reports') +export function publish() { + return queue.add('complete', { id: 'report-1' }) +} +`, + 'src/consumer.ts': `import { Worker, type Job } from 'bullmq' +type Payload = { id: string } +export async function consume(job: Job) { + void job.data.id +} +export const worker = new Worker('reports', consume) +`, + } + const forward = build(sources) + const reversed = build(sources, { reverse: true }) + + expect(reversed.result.graph.nodeEntries()) + .toEqual(forward.result.graph.nodeEntries()) + expect(reversed.result.graph.edgeEntries()) + .toEqual(forward.result.graph.edgeEntries()) + expect(reversed.result.diagnostics).toEqual(forward.result.diagnostics) + }) +}) diff --git a/tests/unit/canonical-index-execution.test.ts b/tests/unit/canonical-index-execution.test.ts new file mode 100644 index 00000000..913c2432 --- /dev/null +++ b/tests/unit/canonical-index-execution.test.ts @@ -0,0 +1,408 @@ +import { createHash } from 'node:crypto' +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' + +import { afterEach, describe, expect, it } from 'vitest' + +import { buildCanonicalTypeScriptIndex } from '../../src/adapters/typescript/index.js' +import { decodeIndexBodyFactTable } from '../../src/domain/index/model.js' + +type Position = { + line: number + column: number +} + +type Range = { + start: Position + end: Position +} + +type BodyFact = { + id: string + owner_symbol_id: string + order: readonly number[] + kind: string + evidence: { + file_id: string + range: Range + statement_range: Range + excerpt_sha256: string + } + control: ReadonlyArray<{ kind: string; [key: string]: unknown }> + confidence: string + source: string + [key: string]: unknown +} + +const roots: string[] = [] + +afterEach(() => { + for (const root of roots.splice(0)) { + rmSync(root, { recursive: true, force: true }) + } +}) + +function build(sources: Record) { + const root = mkdtempSync(join(tmpdir(), 'madar-canonical-execution-')) + roots.push(root) + const files = Object.entries(sources).map(([path, source]) => { + const absolute = join(root, path) + mkdirSync(dirname(absolute), { recursive: true }) + writeFileSync(absolute, source, 'utf8') + return absolute + }) + const result = buildCanonicalTypeScriptIndex({ root, files }) + return { + result, + nodes: new Map(result.graph.nodeEntries()), + edges: result.graph.edgeEntries(), + } +} + +function named( + nodes: ReadonlyMap>, + qualifiedName: string, +): [string, Record] { + const entry = [...nodes].find(([, attributes]) => + attributes.qualified_name === qualifiedName) + if (!entry) throw new Error(`Missing fixture symbol ${qualifiedName}`) + return entry +} + +function bodyFacts( + attributes: Record, + ownerId: string, + fileId: string, +): BodyFact[] { + if (!Object.hasOwn(attributes, 'body_facts')) return [] + const decoded = decodeIndexBodyFactTable( + attributes.body_facts, + ownerId, + fileId, + ) + if (!decoded) throw new Error(`Invalid body-fact table for ${ownerId}`) + return [...decoded] as BodyFact[] +} + +function comparePosition(left: Position, right: Position): number { + return left.line - right.line || left.column - right.column +} + +function compareOrder(left: readonly number[], right: readonly number[]): number { + for (let index = 0; index < Math.min(left.length, right.length); index += 1) { + const difference = left[index]! - right[index]! + if (difference !== 0) return difference + } + return left.length - right.length +} + +function excerpt(source: string, range: Range): string { + const lines = source.split('\n') + const offset = (position: Position): number => { + let value = 0 + for (let line = 1; line < position.line; line += 1) { + value += (lines[line - 1]?.length ?? 0) + 1 + } + return value + position.column - 1 + } + return source.slice(offset(range.start), offset(range.end)) +} + +function sha256(value: string): string { + return createHash('sha256').update(value, 'utf8').digest('hex') +} + +function plainIndexValue(value: unknown): unknown { + if (Array.isArray(value)) return value.map(plainIndexValue) + if (value === null || typeof value !== 'object') return value + const record = value as Record + if (record.kind === 'literal') return record.value + if (record.kind === 'array' && Array.isArray(record.elements)) { + return record.elements.map(plainIndexValue) + } + return value +} + +function hasRelation( + edges: ReturnType['edges'], + from: string, + to: string, + relation: string, +): boolean { + return edges.some(([source, target, attributes]) => + source === from + && target === to + && attributes.relation === relation) +} + +describe('canonical TypeScript semantic execution facts', () => { + it('stores selective authenticated facts with nested control and derivable Promise parallelism', () => { + const source = `const METRIC_BATCHES = [ + ['desirability', 'viability', 'feasibility'], + ['competition', 'impact', 'risk'], +] as const + +function record(metric: string): void { + void metric +} + +function scoreMetricBatch(metrics: readonly string[]): number { + return metrics.length +} + +export async function assemble(enabled: boolean) { + const unusedHumanNote = 'do-not-index-unused-local' + if (!enabled) return 'skipped' + + for (const metric of METRIC_BATCHES[0]) { + record(metric) + } + + const settled = await Promise.allSettled( + METRIC_BATCHES.map((metrics) => scoreMetricBatch(metrics)), + ) + return settled +} +` + const { nodes } = build({ 'src/execution.ts': source }) + const [assembleId, assembleNode] = named(nodes, 'assemble') + const [batchesId, batchesNode] = named(nodes, 'METRIC_BATCHES') + const file = [...nodes].find(([, attributes]) => + attributes.node_kind === 'file' + && attributes.source_file === 'src/execution.ts') + if (!file) throw new Error('Missing fixture file node src/execution.ts') + const facts = bodyFacts(assembleNode, assembleId, file[0]) + const batchFacts = bodyFacts(batchesNode, batchesId, file[0]) + + expect(facts).not.toEqual([]) + expect(batchFacts).not.toEqual([]) + expect(facts.map((fact) => fact.kind)).toEqual(expect.arrayContaining([ + 'call', + 'condition', + 'loop', + 'parallel', + 'return', + ])) + expect(batchFacts.map((fact) => fact.kind)).toContain('literal') + + const allFacts = [...facts, ...batchFacts] + for (const fact of allFacts) { + expect(fact.owner_symbol_id).toBe( + facts.includes(fact) ? assembleId : batchesId, + ) + expect(fact.order.length).toBeGreaterThan(0) + expect(fact.order.every((part) => + Number.isSafeInteger(part) && part >= 0)).toBe(true) + expect(fact.evidence).toMatchObject({ + file_id: file?.[0], + range: { + start: { line: expect.any(Number), column: expect.any(Number) }, + end: { line: expect.any(Number), column: expect.any(Number) }, + }, + statement_range: { + start: { line: expect.any(Number), column: expect.any(Number) }, + end: { line: expect.any(Number), column: expect.any(Number) }, + }, + excerpt_sha256: expect.stringMatching(/^[a-f0-9]{64}$/), + }) + expect(comparePosition( + fact.evidence.statement_range.start, + fact.evidence.range.start, + )).toBeLessThanOrEqual(0) + expect(comparePosition( + fact.evidence.range.end, + fact.evidence.statement_range.end, + )).toBeLessThanOrEqual(0) + expect(fact.evidence.excerpt_sha256).toBe( + sha256(excerpt(source, fact.evidence.statement_range)), + ) + } + + expect(facts.map((fact) => fact.order)).toEqual( + [...facts.map((fact) => fact.order)].sort(compareOrder), + ) + expect(new Set(facts.map((fact) => JSON.stringify(fact.order))).size) + .toBe(facts.length) + + const skippedReturn = facts.find((fact) => + fact.kind === 'return' && JSON.stringify(fact).includes('skipped')) + expect(skippedReturn?.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'branch' }), + ])) + + const loopCall = facts.find((fact) => + fact.kind === 'call' && JSON.stringify(fact).includes('record')) + expect(loopCall?.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'loop' }), + ])) + + const parallel = facts.find((fact) => fact.kind === 'parallel') + expect(parallel).toMatchObject({ + combinator: 'allSettled', + completion: 'all_settled', + lane_count: 2, + }) + expect(parallel?.kind === 'parallel' && parallel.input + ? plainIndexValue(parallel.input) + : null).toEqual([ + ['desirability', 'viability', 'feasibility'], + ['competition', 'impact', 'risk'], + ]) + + const groupedCall = facts.find((fact) => + fact.kind === 'call' + && fact.callee === 'scoreMetricBatch') + expect(groupedCall?.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'loop' }), + expect.objectContaining({ kind: 'parallel', lane: 'each' }), + ])) + const groupedArguments = groupedCall?.arguments as unknown[] | undefined + expect(groupedCall?.kind === 'call' && groupedArguments + ? plainIndexValue(groupedArguments[0]!) + : null).toEqual({ + kind: 'parameter', + position: 0, + scope: 'iteration', + }) + expect(parallel?.kind === 'parallel' + ? parallel.member_fact_ids + : []).toEqual([groupedCall?.id]) + + const structuredBatches = batchFacts + .filter((fact) => fact.kind === 'literal') + .map((fact) => plainIndexValue(fact.value)) + .find((value) => Array.isArray(value)) + expect(structuredBatches).toEqual([ + ['desirability', 'viability', 'feasibility'], + ['competition', 'impact', 'risk'], + ]) + expect(JSON.stringify(allFacts)).not.toContain('do-not-index-unused-local') + }) + + it('connects only exact queue/job/event producers and consumers through shared channels', () => { + const source = `import { EventEmitter } from 'node:events' +import { Queue, Worker } from 'bullmq' + +type ReportJob = { id: string } + +class QueueRegistry { + addJob(queueName: string, jobName: string, data: ReportJob) { + return new Queue(queueName).add(jobName, data) + } + + registerWorker( + queueName: string, + processor: (job: { data: ReportJob }) => Promise, + ) { + return new Worker(queueName, processor) + } +} + +const registry = new QueueRegistry() +const events = new EventEmitter() + +export function publishReport() { + return registry.addJob('reports', 'complete', { id: 'report-1' }) +} + +export async function consumeReport(job: { data: ReportJob }) { + void job.data.id +} + +export function wireReport() { + return registry.registerWorker('reports', consumeReport) +} + +export async function consumeOther(job: { data: ReportJob }) { + void job.data.id +} + +export function wireOther() { + return registry.registerWorker('other', consumeOther) +} + +export function publishDynamic(queueName: string) { + return registry.addJob(queueName, 'complete', { id: 'report-2' }) +} + +export function emitReady() { + events.emit('report.ready', { id: 'report-1' }) +} + +export function handleReady(): void {} + +export function wireReady() { + events.on('report.ready', handleReady) +} + +export function handleFailed(): void {} + +export function wireFailed() { + events.on('report.failed', handleFailed) +} + +export function emitDynamic(eventName: string) { + events.emit(eventName, { id: 'report-2' }) +} +` + const { nodes, edges } = build({ 'src/channels.ts': source }) + const channels = [...nodes].filter(([, attributes]) => + attributes.node_kind === 'channel') + const reportQueue = channels.find(([, attributes]) => + attributes.channel_kind === 'queue' + && attributes.transport === 'bullmq' + && attributes.key === 'reports') + const otherQueue = channels.find(([, attributes]) => + attributes.channel_kind === 'queue' + && attributes.transport === 'bullmq' + && attributes.key === 'other') + const reportJob = channels.find(([, attributes]) => + attributes.channel_kind === 'job' + && attributes.transport === 'bullmq' + && attributes.key === 'complete' + && attributes.parent_channel_id === reportQueue?.[0]) + const readyEvent = channels.find(([, attributes]) => + attributes.channel_kind === 'event' + && attributes.transport === 'node-event-emitter' + && attributes.key === 'report.ready') + const failedEvent = channels.find(([, attributes]) => + attributes.channel_kind === 'event' + && attributes.transport === 'node-event-emitter' + && attributes.key === 'report.failed') + + expect(reportQueue).toBeDefined() + expect(otherQueue).toBeDefined() + expect(reportJob).toBeDefined() + expect(readyEvent).toBeDefined() + expect(failedEvent).toBeDefined() + + const [publishReport] = named(nodes, 'publishReport') + const [consumeReport] = named(nodes, 'consumeReport') + const [consumeOther] = named(nodes, 'consumeOther') + const [publishDynamic] = named(nodes, 'publishDynamic') + const [emitReady] = named(nodes, 'emitReady') + const [handleReady] = named(nodes, 'handleReady') + const [handleFailed] = named(nodes, 'handleFailed') + const [emitDynamic] = named(nodes, 'emitDynamic') + + expect(hasRelation(edges, publishReport, reportJob![0], 'publishes_to')).toBe(true) + expect(hasRelation(edges, reportJob![0], reportQueue![0], 'routes_through')).toBe(true) + expect(hasRelation(edges, reportQueue![0], consumeReport, 'consumed_by')).toBe(true) + expect(hasRelation(edges, otherQueue![0], consumeOther, 'consumed_by')).toBe(true) + expect(hasRelation(edges, reportQueue![0], consumeOther, 'consumed_by')).toBe(false) + expect(hasRelation(edges, otherQueue![0], consumeReport, 'consumed_by')).toBe(false) + + expect(hasRelation(edges, emitReady, readyEvent![0], 'publishes_to')).toBe(true) + expect(hasRelation(edges, readyEvent![0], handleReady, 'consumed_by')).toBe(true) + expect(hasRelation(edges, readyEvent![0], handleFailed, 'consumed_by')).toBe(false) + expect(hasRelation(edges, failedEvent![0], handleFailed, 'consumed_by')).toBe(true) + + expect(edges.some(([source, , attributes]) => + source === publishDynamic + && attributes.relation === 'publishes_to')).toBe(false) + expect(edges.some(([source, , attributes]) => + source === emitDynamic + && attributes.relation === 'publishes_to')).toBe(false) + }) +}) diff --git a/tests/unit/core-reset-governance.test.ts b/tests/unit/core-reset-governance.test.ts index 947ec62b..1a964ead 100644 --- a/tests/unit/core-reset-governance.test.ts +++ b/tests/unit/core-reset-governance.test.ts @@ -412,6 +412,42 @@ const BETA_4_SHASUM = 'c5250a0d308b3d6df374851154ddb393a678a992' const BETA_4_INTEGRITY = 'sha512-772P+n4Cx55nqC+CAx8A1aTJ2rY4yk1hUH45lAlxNMMw4YRj8hhswgDiCwczS5hx1S3a+Z+KUv2jma/zWjQZ6w==' const BETA_4_SHA256 = '8bd8d501b8cd3546e16a5a1ddac1f7649434e685517e1171fbd5897515e76e6b' +const SEMANTIC_EXECUTION_INDEX_ID = 'semantic-execution-index-632' +const SEMANTIC_EXECUTION_INDEX_BASE = '9043320cfa08370e5cdd3911bfb9283005aa9912' +const SEMANTIC_EXECUTION_INDEX_BASE_TREE = 'f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc' +const SEMANTIC_EXECUTION_INDEX_FILES = [ + 'src/adapters/filesystem/graph-artifact.ts', + 'src/adapters/mcp/server.ts', + 'src/adapters/typescript/execution.ts', + 'src/adapters/typescript/index.ts', + 'src/application/retrieve-context.ts', + 'src/domain/index/build-state.ts', + 'src/domain/index/model.ts', + 'src/domain/query/index-status.ts', + 'src/domain/query/rank.ts', +] as const +const SEMANTIC_EXECUTION_PACKAGE_EXCLUSIONS = [ + 'examples/why-madar.md', + 'CHANGELOG.md', +] as const +const SEMANTIC_EXECUTION_SOURCE = { + production_typescript_files: 44, + production_typescript_loc: 15_873, + production_loc_added: 3_606, + production_loc_removed: 187, + production_loc_net: 3_419, +} as const +const SEMANTIC_EXECUTION_PACKAGE = { + npm_files: 102, + npm_packed_bytes: 143_170, + npm_unpacked_bytes: 638_803, + npm_shasum: '8ed94633cdf6f63dfabe12845e5e77644a89dbf1', + npm_integrity: + 'sha512-Nq3m/a7D/iZHYjjRd9b7ENth95qz4zssOPnBqmQEsy+aiebRp68UtMMWArYWFRTftdWwNfTcIvxns0JVdahrdw==', + npm_artifact_sha256: 'df9bb97b425db5063ab178920c7fb824c19c681378744c6cfc7094e99926f1eb', +} as const +const SEMANTIC_EXECUTION_DIFF_SHA256 = + '1916499d8e195e20c47cc05868a50f3ca408759e932596c16388da6e66e41ec6' const CAPABILITY_VALIDATION_V2_PROPOSAL_SHA256 = '4906405cbb806c850c0612305ef460e023e2060b5338734ae0af12303901cbd0' const CAPABILITY_VALIDATION_V2_ISSUE = 'https://github.com/mohanagy/madar/issues/612' @@ -819,7 +855,10 @@ describe('core reset governance', () => { expect(roadmap).toContain('## Passed — retrieval regression #618') expect(roadmap).toContain('## Published — `0.40.0-beta.3`') expect(roadmap).toContain('## Passed — retrieval regression #625') - expect(roadmap).toContain('## Ready — `0.40.0-beta.4`') + expect(roadmap).toContain('## Published — `0.40.0-beta.4`') + expect(roadmap).toContain('## In progress — semantic execution index #632') + expect(roadmap).toContain('## Pending — obligation-driven retrieval #630') + expect(roadmap).toContain('## Pending — installed no-fallback qualification #631') expect(roadmap).toContain(CAPABILITY_VALIDATION_PROPOSAL_SHA256) expect(roadmap).toContain(CAPABILITY_VALIDATION_OWNER_APPROVAL) expect(roadmap).toContain(CAPABILITY_VALIDATION_RFC_APPROVAL) @@ -1009,7 +1048,10 @@ describe('core reset governance', () => { expect(scorecard).toContain('| Capability validation v2 | **Stopped / not planned**') expect(scorecard).toContain('| Retrieval regression #618 | **Passed**') expect(scorecard).toContain('| Retrieval regression #625 | **Passed**') - expect(scorecard).toContain('| Beta release | **Ready**') + expect(scorecard).toContain('| Beta release | **Published**') + expect(scorecard).toContain('| Semantic execution index #632 | **In progress**') + expect(scorecard).toContain('| Obligation-driven retrieval #630 | **Pending**') + expect(scorecard).toContain('| No-fallback qualification #631 | **Pending**') expect(scorecard).toContain(CAPABILITY_VALIDATION_PROPOSAL_SHA256) expect(scorecard).toContain(CAPABILITY_VALIDATION_OWNER_APPROVAL) expect(scorecard).toContain(CAPABILITY_VALIDATION_RFC_APPROVAL) @@ -1055,6 +1097,7 @@ describe('core reset governance', () => { expect(scorecard).toContain('every warmup/measured result must remain correct; an empty positive result fails') expect(scorecard).toContain('| Retrieval regression #622 | **Passed**') expect(scorecard).toContain('Issues `#622` and `#625` are complete on `next`') + expect(scorecard).toContain('#632 active, then #630 pending, then #631 pending') expect(scorecard).toContain('first-candidate stop receipt') expect(scorecard).toContain('102 entries / 159,980 packed / 639,930 unpacked bytes') expect(scorecard).toContain('/compilerOptions/removeComments=true') @@ -1291,34 +1334,36 @@ describe('core reset governance', () => { expect(manifest.schema_version).toBe(1) expect(manifest.status).toBe('accepted') expect(manifest.current).toMatchObject({ - updated_at: '2026-07-30', + updated_at: '2026-07-31', completed_phase: EVIDENCE_SKELETON_RETRIEVAL_ID, - active_phase: null, - ready_phase: 'release-beta', - base_commit: EVIDENCE_SKELETON_RETRIEVAL_BASE, + active_phase: SEMANTIC_EXECUTION_INDEX_ID, + ready_phase: null, + base_commit: SEMANTIC_EXECUTION_INDEX_BASE, completed_phase_commit: EVIDENCE_SKELETON_RETRIEVAL_MERGE, - production_typescript_files: 43, - production_typescript_loc: 12_454, - production_loc_added: 1_409, - production_loc_removed: 1_102, - production_loc_net: 307, - npm_files: 102, - npm_packed_bytes: 159_937, - npm_unpacked_bytes: 639_875, - npm_shasum: BETA_4_SHASUM, - npm_integrity: BETA_4_INTEGRITY, - npm_artifact_sha256: BETA_4_SHA256, + ...SEMANTIC_EXECUTION_SOURCE, + ...SEMANTIC_EXECUTION_PACKAGE, measurement_state: 'source_and_package_exact', - snapshot_scope: 'release_candidate_source_and_package', + snapshot_scope: 'semantic_execution_index_632_candidate', }) expect(manifest.current.release_candidate).toMatchObject({ version: '0.40.0-beta.4', - protected_anchor_commit: EVIDENCE_SKELETON_RETRIEVAL_MERGE, - protected_anchor_tree: EVIDENCE_SKELETON_RETRIEVAL_FINAL_TREE, + protected_anchor_commit: SEMANTIC_EXECUTION_INDEX_BASE, + protected_anchor_tree: SEMANTIC_EXECUTION_INDEX_BASE_TREE, target_branch: 'next', npm_dist_tag: 'next', stable_release: false, - publication_state: 'preparation', + publication_state: 'published', + tag: 'v0.40.0-beta.4', + github_prerelease: 'https://github.com/mohanagy/madar/releases/tag/v0.40.0-beta.4', + published_at: '2026-07-30T13:07:45.103Z', + published_package: { + npm_files: 102, + npm_packed_bytes: 159_937, + npm_unpacked_bytes: 639_875, + npm_shasum: BETA_4_SHASUM, + npm_integrity: BETA_4_INTEGRITY, + npm_artifact_sha256: BETA_4_SHA256, + }, }) expect(manifest.rules.length).toBeGreaterThan(0) expect(manifest.items.length).toBeGreaterThan(10) @@ -1435,7 +1480,7 @@ describe('core reset governance', () => { expect(logicalLocAtCommit(legacyBase, deletionFiles)).toBe(20_951) const generation = manifest.items.find((item) => item.id === 'generation-and-incremental') expect(manifest.items.filter((item) => item.status === 'in_progress').map((item) => item.id)) - .toEqual([]) + .toEqual([SEMANTIC_EXECUTION_INDEX_ID]) const retrievalRegression = manifest.items.find((item) => item.id === RETRIEVAL_REGRESSION_ID) as any expect(retrievalRegression).toMatchObject({ disposition: 'keep', @@ -1813,10 +1858,109 @@ describe('core reset governance', () => { main_target: 'forbidden', }, }) - const changedEvidenceSkeletonProduction = [ + const semanticExecution = manifest.items.find( + (item) => item.id === SEMANTIC_EXECUTION_INDEX_ID, + ) as any + expect(semanticExecution).toMatchObject({ + disposition: 'keep', + status: 'in_progress', + destination: 'canonical JavaScript/TypeScript semantic execution index', + modified_sources: [...SEMANTIC_EXECUTION_INDEX_FILES], + activation: { + issue: 'https://github.com/mohanagy/madar/issues/632', + protected_base: SEMANTIC_EXECUTION_INDEX_BASE, + protected_base_tree: SEMANTIC_EXECUTION_INDEX_BASE_TREE, + target_branch: 'next', + }, + delivery_limits: { + new_production_files_max: 4, + net_production_loc_max: 3_500, + graph_artifact_size_ratio_max: 1.5, + indexing_median_ratio_max: 1.25, + warm_retrieval_p95_ms_less_than: 500, + }, + npm_package_budget: { + files_max: 102, + packed_bytes_max: 165_000, + unpacked_bytes_max: 640_000, + }, + package_metadata_contract: { + path: 'package.json', + removed_files_entries: [...SEMANTIC_EXECUTION_PACKAGE_EXCLUSIONS], + repository_files_deleted: false, + version_change: 'forbidden', + script_change: 'forbidden', + dependency_change: 'forbidden', + package_lock_change: 'forbidden', + }, + retrieval_budget: { + files_max: 12, + snippets_max: 25, + closure_passes_max: 1, + serialized_tokens_max: 4_000, + }, + constraints: { + repository_specific_rule: 'forbidden', + manual_channel_map: 'forbidden', + dependency_change: 'forbidden', + retrieval_result_v2_cutover: 'forbidden', + compatibility_reader_or_alias: 'forbidden', + provider_activity: 'forbidden', + npm_publication: 'forbidden', + github_release: 'forbidden', + registry_metadata_publication: 'forbidden', + tag: 'forbidden', + main_target: 'forbidden', + }, + }) + expect(semanticExecution.candidate).toMatchObject({ + source_measurement: { + production_typescript_files: SEMANTIC_EXECUTION_SOURCE.production_typescript_files, + production_typescript_loc: SEMANTIC_EXECUTION_SOURCE.production_typescript_loc, + added: SEMANTIC_EXECUTION_SOURCE.production_loc_added, + removed: SEMANTIC_EXECUTION_SOURCE.production_loc_removed, + net: SEMANTIC_EXECUTION_SOURCE.production_loc_net, + diff_sha256: SEMANTIC_EXECUTION_DIFF_SHA256, + }, + package_measurement: { + files: SEMANTIC_EXECUTION_PACKAGE.npm_files, + packed_bytes: SEMANTIC_EXECUTION_PACKAGE.npm_packed_bytes, + unpacked_bytes: SEMANTIC_EXECUTION_PACKAGE.npm_unpacked_bytes, + shasum: SEMANTIC_EXECUTION_PACKAGE.npm_shasum, + integrity: SEMANTIC_EXECUTION_PACKAGE.npm_integrity, + artifact_sha256: SEMANTIC_EXECUTION_PACKAGE.npm_artifact_sha256, + }, + local_verification: { + focused_tests_passed: 149, + last_pre_cache_full_tests_passed: 722, + candidate_full_suite_status: 'pending_clean_runner_after_local_host_memory_pressure', + candidate_full_suite_local_attempts: 3, + graph_nodes: 12_350, + graph_edges: 32_839, + exact_queue_channels: 6, + exact_channel_edges: 51, + graph_artifact_size_ratio: 1.2344886849638093, + beta4_indexing_trials_seconds: [20.32, 21.33, 22.16, 22.51, 22.74], + beta4_indexing_median_seconds: 22.16, + candidate_indexing_trials_seconds: [23.31, 24.16, 24.41, 24.87, 25.42], + candidate_indexing_median_seconds: 24.41, + indexing_median_ratio: 1.1015342960288808, + warm_retrieval_samples: 100, + warm_retrieval_median_ms: 165.343, + warm_retrieval_p95_ms: 216.599, + warm_retrieval_max_ms: 257.814, + deterministic_graph_sha256: + '478e1ed93fcef5a7592dfa0f884318713128e01857dc706a28f2e0d6142937d0', + beta4_retrieval_output_byte_identical: true, + exact_head_ci: 'pending', + independent_review: 'pending', + }, + }) + expect(semanticExecution).not.toHaveProperty('completion') + const changedSemanticExecutionProduction = [ ...execFileSync( git, - ['diff', '--name-only', EVIDENCE_SKELETON_RETRIEVAL_BASE, '--', 'src'], + ['diff', '--name-only', SEMANTIC_EXECUTION_INDEX_BASE, '--', 'src'], { encoding: 'utf8' }, ).trim().split('\n').filter(Boolean), ...execFileSync( @@ -1825,6 +1969,56 @@ describe('core reset governance', () => { { encoding: 'utf8' }, ).trim().split('\n').filter(Boolean), ].sort() + expect(changedSemanticExecutionProduction).toEqual([...SEMANTIC_EXECUTION_INDEX_FILES].sort()) + const semanticBaseFiles = new Set(productionTypeScriptFilesAtCommit(SEMANTIC_EXECUTION_INDEX_BASE)) + expect(SEMANTIC_EXECUTION_INDEX_FILES.filter((path) => !semanticBaseFiles.has(path))) + .toHaveLength(1) + expect(productionSourceDelta(SEMANTIC_EXECUTION_INDEX_BASE).net).toBeLessThanOrEqual(3_500) + expect(execFileSync( + git, + ['rev-parse', `${SEMANTIC_EXECUTION_INDEX_BASE}^{tree}`], + { encoding: 'utf8' }, + ).trim()).toBe(SEMANTIC_EXECUTION_INDEX_BASE_TREE) + const obligationRetrieval = manifest.items.find( + (item) => item.id === 'obligation-driven-retrieval-630', + ) as any + expect(obligationRetrieval).toMatchObject({ + disposition: 'keep', + status: 'planned', + depends_on: [SEMANTIC_EXECUTION_INDEX_ID], + activation: { + issue: 'https://github.com/mohanagy/madar/issues/630', + protected_base: SEMANTIC_EXECUTION_INDEX_BASE, + protected_base_tree: SEMANTIC_EXECUTION_INDEX_BASE_TREE, + target_branch: 'next', + }, + }) + const noFallbackQualification = manifest.items.find( + (item) => item.id === 'no-fallback-qualification-631', + ) as any + expect(noFallbackQualification).toMatchObject({ + disposition: 'keep', + status: 'planned', + depends_on: [SEMANTIC_EXECUTION_INDEX_ID, 'obligation-driven-retrieval-630'], + activation: { + issue: 'https://github.com/mohanagy/madar/issues/631', + protected_base: SEMANTIC_EXECUTION_INDEX_BASE, + protected_base_tree: SEMANTIC_EXECUTION_INDEX_BASE_TREE, + target_branch: 'next', + }, + }) + const changedEvidenceSkeletonProduction = execFileSync( + git, + [ + 'diff', + '--name-only', + EVIDENCE_SKELETON_RETRIEVAL_BASE, + EVIDENCE_SKELETON_RETRIEVAL_MERGE, + '--', + 'src', + ], + { encoding: 'utf8' }, + ).trim().split('\n').filter(Boolean).sort() expect(changedEvidenceSkeletonProduction).toEqual([...EVIDENCE_SKELETON_RETRIEVAL_FILES].sort()) expect(execFileSync( git, @@ -1894,7 +2088,14 @@ describe('core reset governance', () => { )).not.toThrow() expect(() => execFileSync( git, - ['diff', '--quiet', sourceTestSnapshot.commit, '--', ...sourceTestSnapshot.frozen_paths], + [ + 'diff', + '--quiet', + sourceTestSnapshot.commit, + EVIDENCE_SKELETON_RETRIEVAL_MERGE, + '--', + ...sourceTestSnapshot.frozen_paths, + ], )).not.toThrow() const sourceDiffSha256 = createHash('sha256').update(execFileSync( git, @@ -2224,22 +2425,18 @@ describe('core reset governance', () => { ['merge-base', '--is-ancestor', THIN_DELIVERY_IMPLEMENTATION_START, THIN_DELIVERY_MERGE], )).not.toThrow() expect(manifest.current).toMatchObject({ - updated_at: '2026-07-30', + updated_at: '2026-07-31', completed_phase: EVIDENCE_SKELETON_RETRIEVAL_ID, - active_phase: null, - ready_phase: 'release-beta', - base_commit: EVIDENCE_SKELETON_RETRIEVAL_BASE, + active_phase: SEMANTIC_EXECUTION_INDEX_ID, + ready_phase: null, + base_commit: SEMANTIC_EXECUTION_INDEX_BASE, completed_phase_commit: EVIDENCE_SKELETON_RETRIEVAL_MERGE, - production_typescript_files: 43, - production_typescript_loc: 12_454, - production_loc_added: 1_409, - production_loc_removed: 1_102, - production_loc_net: 307, + ...SEMANTIC_EXECUTION_SOURCE, measurement_state: 'source_and_package_exact', - snapshot_scope: 'release_candidate_source_and_package', + snapshot_scope: 'semantic_execution_index_632_candidate', }) expect(manifest.items.filter((item) => item.status === 'in_progress').map((item) => item.id)) - .toEqual([]) + .toEqual([SEMANTIC_EXECUTION_INDEX_ID]) expect(manifest.targets).toMatchObject({ production_typescript_files_max: 80, production_typescript_loc_max: 35_000, @@ -3092,21 +3289,17 @@ describe('core reset governance', () => { )).not.toThrow() expect(manifest.current).toMatchObject({ completed_phase: EVIDENCE_SKELETON_RETRIEVAL_ID, - active_phase: null, - ready_phase: 'release-beta', - base_commit: EVIDENCE_SKELETON_RETRIEVAL_BASE, + active_phase: SEMANTIC_EXECUTION_INDEX_ID, + ready_phase: null, + base_commit: SEMANTIC_EXECUTION_INDEX_BASE, completed_phase_commit: EVIDENCE_SKELETON_RETRIEVAL_MERGE, - production_typescript_files: 43, - production_typescript_loc: 12_454, - production_loc_added: 1_409, - production_loc_removed: 1_102, - production_loc_net: 307, - npm_files: 102, - npm_packed_bytes: 159_937, - npm_unpacked_bytes: 639_875, + ...SEMANTIC_EXECUTION_SOURCE, + npm_files: SEMANTIC_EXECUTION_PACKAGE.npm_files, + npm_packed_bytes: SEMANTIC_EXECUTION_PACKAGE.npm_packed_bytes, + npm_unpacked_bytes: SEMANTIC_EXECUTION_PACKAGE.npm_unpacked_bytes, }) expect(manifest.items.filter((item) => item.status === 'in_progress').map((item) => item.id)) - .toEqual([]) + .toEqual([SEMANTIC_EXECUTION_INDEX_ID]) const evaluation = manifest.items.find((item) => item.id === 'evaluation-tooling') expect(evaluation).toMatchObject({ @@ -3516,11 +3709,19 @@ describe('core reset governance', () => { expect(currentPackage).toEqual({ ...implementationPackage, version: '0.40.0-beta.4', + files: implementationPackage.files.filter( + (path) => !SEMANTIC_EXECUTION_PACKAGE_EXCLUSIONS.includes( + path as (typeof SEMANTIC_EXECUTION_PACKAGE_EXCLUSIONS)[number], + ), + ), scripts: { ...implementationPackage.scripts, 'publish:next': 'npm publish --tag next --access public --provenance', }, }) + expect(SEMANTIC_EXECUTION_PACKAGE_EXCLUSIONS.every( + (path) => !currentPackage.files.includes(path), + )).toBe(true) expect(currentPackage.files).not.toContain('dist-eval/') expect(currentPackage.scripts.prepack).not.toContain('build:eval') const implementationLock = JSON.parse(execFileSync( @@ -3606,17 +3807,13 @@ describe('core reset governance', () => { | undefined expect(manifest.current).toMatchObject({ completed_phase: EVIDENCE_SKELETON_RETRIEVAL_ID, - active_phase: null, - ready_phase: 'release-beta', - base_commit: EVIDENCE_SKELETON_RETRIEVAL_BASE, - production_typescript_files: 43, - production_typescript_loc: 12_454, - production_loc_added: 1_409, - production_loc_removed: 1_102, - production_loc_net: 307, + active_phase: SEMANTIC_EXECUTION_INDEX_ID, + ready_phase: null, + base_commit: SEMANTIC_EXECUTION_INDEX_BASE, + ...SEMANTIC_EXECUTION_SOURCE, }) expect(manifest.items.filter((item) => item.status === 'in_progress').map((item) => item.id)) - .toEqual([]) + .toEqual([SEMANTIC_EXECUTION_INDEX_ID]) expect(phase).toMatchObject({ disposition: 'keep', status: 'stopped', @@ -3864,17 +4061,13 @@ describe('core reset governance', () => { | undefined expect(manifest.current).toMatchObject({ completed_phase: EVIDENCE_SKELETON_RETRIEVAL_ID, - active_phase: null, - ready_phase: 'release-beta', - base_commit: EVIDENCE_SKELETON_RETRIEVAL_BASE, - production_typescript_files: 43, - production_typescript_loc: 12_454, - production_loc_added: 1_409, - production_loc_removed: 1_102, - production_loc_net: 307, + active_phase: SEMANTIC_EXECUTION_INDEX_ID, + ready_phase: null, + base_commit: SEMANTIC_EXECUTION_INDEX_BASE, + ...SEMANTIC_EXECUTION_SOURCE, }) expect(manifest.items.filter((item) => item.status === 'in_progress').map((item) => item.id)) - .toEqual([]) + .toEqual([SEMANTIC_EXECUTION_INDEX_ID]) expect(phase).toMatchObject({ disposition: 'keep', status: 'stopped', @@ -4498,25 +4691,21 @@ describe('core reset governance', () => { expect(execFileSync(git, ['rev-parse', `${EVIDENCE_BASE}^{tree}`], { encoding: 'utf8' }).trim()) .toBe(EVIDENCE_BASE_TREE) expect(manifest.current).toMatchObject({ - updated_at: '2026-07-30', + updated_at: '2026-07-31', completed_phase: EVIDENCE_SKELETON_RETRIEVAL_ID, - active_phase: null, - ready_phase: 'release-beta', - base_commit: EVIDENCE_SKELETON_RETRIEVAL_BASE, + active_phase: SEMANTIC_EXECUTION_INDEX_ID, + ready_phase: null, + base_commit: SEMANTIC_EXECUTION_INDEX_BASE, completed_phase_commit: EVIDENCE_SKELETON_RETRIEVAL_MERGE, - production_typescript_files: 43, - production_typescript_loc: 12_454, - production_loc_added: 1_409, - production_loc_removed: 1_102, - production_loc_net: 307, - npm_files: 102, - npm_packed_bytes: 159_937, - npm_unpacked_bytes: 639_875, + ...SEMANTIC_EXECUTION_SOURCE, + npm_files: SEMANTIC_EXECUTION_PACKAGE.npm_files, + npm_packed_bytes: SEMANTIC_EXECUTION_PACKAGE.npm_packed_bytes, + npm_unpacked_bytes: SEMANTIC_EXECUTION_PACKAGE.npm_unpacked_bytes, measurement_state: 'source_and_package_exact', - snapshot_scope: 'release_candidate_source_and_package', + snapshot_scope: 'semantic_execution_index_632_candidate', }) expect(manifest.items.filter((item) => item.status === 'in_progress').map((item) => item.id)) - .toEqual([]) + .toEqual([SEMANTIC_EXECUTION_INDEX_ID]) const evidence = manifest.items.find((item) => item.id === 'evidence-path-query') expect(evidence).toMatchObject({ @@ -5647,6 +5836,7 @@ describe('core reset governance', () => { status: string modified_sources?: string[] production_loc_budget?: { added_max: number; removed_min: number; net_max: number } + delivery_limits?: { net_production_loc_max: number } activation?: { protected_base: string } candidate?: { source_measurement?: { @@ -5668,8 +5858,8 @@ describe('core reset governance', () => { }> } const { current } = manifest - const candidatePhase = manifest.items.find((item) => item.id === EVIDENCE_SKELETON_RETRIEVAL_ID) - ?? manifest.items.find((item) => item.id === current.active_phase) + const candidatePhase = manifest.items.find((item) => item.id === current.active_phase) + ?? manifest.items.find((item) => item.id === EVIDENCE_SKELETON_RETRIEVAL_ID) ?? manifest.items.find((item) => item.id === FULL_FLOW_RETRIEVAL_ID) const baseline = candidatePhase?.activation?.protected_base ?? current.base_commit expect(execFileSync(git, ['cat-file', '-t', `${baseline}^{commit}`], { encoding: 'utf8' }).trim()).toBe('commit') @@ -5680,15 +5870,16 @@ describe('core reset governance', () => { const phase = candidatePhase ?? manifest.items.find((item) => item.id === (current.active_phase ?? current.completed_phase)) const budget = phase?.production_loc_budget - expect(budget).toBeDefined() + const netMax = phase?.delivery_limits?.net_production_loc_max ?? budget?.net_max + expect(netMax).toBeDefined() expect(inventory.filesystemViolations).toEqual([]) - expect(delta.added).toBeLessThanOrEqual(budget!.added_max) + if (budget) expect(delta.added).toBeLessThanOrEqual(budget.added_max) const isNonCompleteCandidate = phase?.id === EVIDENCE_SKELETON_RETRIEVAL_ID && ['in_progress', 'stopped'].includes(phase.status) - && delta.added <= budget!.added_max - && delta.net <= budget!.net_max - const meetsExitBudget = delta.removed >= budget!.removed_min - && delta.net <= budget!.net_max + && delta.added <= (budget?.added_max ?? Number.POSITIVE_INFINITY) + && delta.net <= netMax! + const meetsExitBudget = (!budget || delta.removed >= budget.removed_min) + && delta.net <= netMax! expect(isNonCompleteCandidate || meetsExitBudget).toBe(true) const measuredInventory = { production_typescript_files: inventory.files, @@ -6081,7 +6272,10 @@ describe('core reset governance', () => { expect(governance).toContain('## Passed — retrieval regression #618') expect(governance).toContain('## Published — `0.40.0-beta.3`') expect(governance).toContain('## Passed — retrieval regression #625') - expect(governance).toContain('## Ready — `0.40.0-beta.4`') + expect(governance).toContain('## Published — `0.40.0-beta.4`') + expect(governance).toContain('## In progress — semantic execution index #632') + expect(governance).toContain('## Pending — obligation-driven retrieval #630') + expect(governance).toContain('## Pending — installed no-fallback qualification #631') expect(governance).toContain('## Stopped amendment — capability validation v1') expect(governance).toContain('## Historical accepted amendment — capability validation v2') expect(governance).toContain('## Cancelled amendment — capability validation') diff --git a/tests/unit/graph-artifact.test.ts b/tests/unit/graph-artifact.test.ts index bfd17d9f..851e2fe7 100644 --- a/tests/unit/graph-artifact.test.ts +++ b/tests/unit/graph-artifact.test.ts @@ -4,7 +4,13 @@ import { join } from 'node:path' import { afterEach, describe, expect, it } from 'vitest' -import { loadGraphArtifact, parseGraphArtifact, readBoundedUtf8, readGraphArtifactReceipt } from '../../src/adapters/filesystem/graph-artifact.js' +import { + graphArtifactIdentity, + loadGraphArtifact, + parseGraphArtifact, + readBoundedUtf8, + readGraphArtifactReceipt, +} from '../../src/adapters/filesystem/graph-artifact.js' import { GRAPH_ARTIFACT_REGENERATE_MESSAGE, serializeGraphArtifact } from '../../src/domain/graph/artifact.js' import { KnowledgeGraph } from '../../src/domain/graph/directed-multigraph.js' @@ -49,6 +55,7 @@ describe('stored graph artifact guard', () => { writeFileSync(graphPath, firstArtifact) const first = readGraphArtifactReceipt(graphPath) + expect(graphArtifactIdentity(graphPath)).toBe(first.identity) expect(readGraphArtifactReceipt(graphPath, first)).toBe(first) expect(first.graphSha256).toBe(createHash('sha256').update(firstArtifact).digest('hex')) @@ -58,6 +65,7 @@ describe('stored graph artifact guard', () => { expect(next).not.toBe(first) expect(next.identity).not.toBe(first.identity) + expect(graphArtifactIdentity(graphPath)).toBe(next.identity) expect(next.graphSha256).toBe(createHash('sha256').update(nextArtifact).digest('hex')) expect(next.graph.graph.fixture_tags).toEqual(['api']) }) diff --git a/tests/unit/query-index-execution-validation.test.ts b/tests/unit/query-index-execution-validation.test.ts new file mode 100644 index 00000000..fd7490e5 --- /dev/null +++ b/tests/unit/query-index-execution-validation.test.ts @@ -0,0 +1,637 @@ +import { + mkdirSync, + mkdtempSync, + rmSync, + writeFileSync, +} from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' + +import { afterEach, describe, expect, it } from 'vitest' + +import { loadGraphArtifact } from '../../src/adapters/filesystem/graph-artifact.js' +import { generateIndex } from '../../src/application/generate-index.js' +import { retrieveContext } from '../../src/application/retrieve-context.js' +import { computeBuildId } from '../../src/domain/index/build-state.js' +import { + indexBodyFactId, + indexChannelId, + type IndexBodyFact, +} from '../../src/domain/index/model.js' +import { + inspectQueryIndex, + type ReadyQueryIndex, +} from '../../src/domain/query/index-status.js' + +const roots: string[] = [] +const wireKinds = [ + 'condition', 'loop', 'parallel', 'call', 'literal', + 'mutation', 'persistence', 'return', 'throw', +] as const satisfies readonly IndexBodyFact['kind'][] + +afterEach(() => { + for (const root of roots.splice(0)) { + rmSync(root, { recursive: true, force: true }) + } +}) + +function resign( + graph: ReturnType, +): void { + const current = graph.graph.index_build as Record + graph.graph.index_build = { ...current, build_id: '' } + graph.graph.index_build = { + ...current, + build_id: computeBuildId(graph), + } +} + +type Fixture = { + graph: ReturnType + root: string + source: string + sourcePath: string + runId: string + queueId: string + dynamicQueueId: string + jobId: string + eventId: string + operationIds: readonly string[] +} + +function ready(value: ReturnType): ReadyQueryIndex { + if (value.state !== 'ready') { + throw new Error(`Expected ready index, received ${value.state}: ${value.subject}`) + } + return value +} + +function fixture(bom = false): Fixture { + const root = mkdtempSync(join(tmpdir(), 'madar-query-execution-')) + roots.push(root) + const source = `${bom ? '\uFEFF' : ''}import type { MongoRepository } from 'typeorm' + +type RecordRow = { id: string } + +export async function run( + repository: MongoRepository, + enabled: boolean, +): Promise { + if (enabled) { + await repository.update('record-1', { id: 'record-1' }) + } + const values = await Promise.all([Promise.resolve(1), Promise.resolve(2)]) + return values +} +` + const sourcePath = join(root, 'src/run.ts') + mkdirSync(dirname(sourcePath), { recursive: true }) + writeFileSync(sourcePath, source, 'utf8') + const generated = generateIndex(root) + const graph = loadGraphArtifact(generated.graphPath) + const run = graph.nodeEntries().find(([, attributes]) => + attributes.qualified_name === 'run') + if (!run) throw new Error('Execution validation fixture is incomplete') + const [runId] = run + const initial = ready(inspectQueryIndex(graph)) + const operations = initial.operations_by_owner.get(runId) + if (!operations?.length) { + throw new Error('Execution validation fixture has no generated operations') + } + const proof = operations[0]!.evidence + + const queueId = indexChannelId({ + channel_kind: 'queue', + transport: 'bullmq', + key: 'reports', + }) + const dynamicQueueId = indexChannelId({ + channel_kind: 'queue', + transport: 'bullmq', + key: 'dynamic-jobs', + }) + const jobId = indexChannelId({ + channel_kind: 'job', + transport: 'bullmq', + key: 'complete', + parent_channel_id: queueId, + }) + const eventId = indexChannelId({ + channel_kind: 'event', + transport: 'node-event-emitter', + key: 'report.ready', + scope: 'instance:events', + }) + graph.addNode(queueId, { + node_kind: 'channel', + channel_kind: 'queue', + transport: 'bullmq', + key: 'reports', + }) + graph.addNode(jobId, { + node_kind: 'channel', + channel_kind: 'job', + transport: 'bullmq', + key: 'complete', + parent_channel_id: queueId, + }) + graph.addNode(dynamicQueueId, { + node_kind: 'channel', + channel_kind: 'queue', + transport: 'bullmq', + key: 'dynamic-jobs', + }) + graph.addNode(eventId, { + node_kind: 'channel', + channel_kind: 'event', + transport: 'node-event-emitter', + key: 'report.ready', + scope: 'instance:events', + }) + const channelEvidence = { + source_file: 'src/run.ts', + execution_owner_id: runId, + evidence: { + source: 'typescript-syntactic', + range: proof.range, + statement_range: proof.statement_range, + excerpt_sha256: proof.excerpt_sha256, + }, + } + graph.addEdge(runId, jobId, { + relation: 'publishes_to', + ...channelEvidence, + }) + graph.addEdge(runId, dynamicQueueId, { + relation: 'publishes_to', + ...channelEvidence, + }) + graph.addEdge(jobId, queueId, { + relation: 'routes_through', + ...channelEvidence, + }) + graph.addEdge(queueId, runId, { + relation: 'consumed_by', + ...channelEvidence, + }) + graph.addEdge(runId, eventId, { + relation: 'publishes_to', + ...channelEvidence, + }) + graph.addEdge(eventId, runId, { + relation: 'consumed_by', + ...channelEvidence, + }) + resign(graph) + return { + graph, + root, + source, + sourcePath, + runId, + queueId, + dynamicQueueId, + jobId, + eventId, + operationIds: operations.map((operation) => operation.id), + } +} + +function bodyFactWireRows( + graph: Fixture['graph'], + runId: string, +): string[] { + const table = graph.nodeAttributes(runId).body_facts + if (!Array.isArray(table) || table[0] !== 1 || !Array.isArray(table[1]) + || table[1].some((row) => typeof row !== 'string')) { + throw new Error('Expected compact execution fact table fixture') + } + return [...table[1]] as string[] +} + +function parsedBodyFactRows( + graph: Fixture['graph'], + runId: string, +): unknown[][] { + return bodyFactWireRows(graph, runId).map((value) => { + const row = JSON.parse(value) as unknown + if (!Array.isArray(row)) throw new Error('Expected execution fact wire row') + return row + }) +} + +function replaceBodyFactWireRows( + graph: Fixture['graph'], + runId: string, + rows: readonly unknown[], +): void { + const attributes = graph.nodeAttributes(runId) + graph.replaceNodeAttributes(runId, { + ...attributes, + body_facts: [1, rows], + }) +} + +function mutateBodyFactWireRows( + graph: Fixture['graph'], + runId: string, + mutate: (rows: unknown[][]) => void, +): void { + const rows = parsedBodyFactRows(graph, runId) + mutate(rows) + replaceBodyFactWireRows( + graph, + runId, + rows.map((row) => JSON.stringify(row)), + ) +} + +function rowOfKind(rows: unknown[][], kind: number): unknown[] { + const row = rows.find((candidate) => candidate[1] === kind) + if (!row) throw new Error(`Expected wire fact kind ${kind}`) + return row +} + +function controlRows(row: unknown[]): unknown[][] { + const controls = row[6] + if (!Array.isArray(controls) + || controls.some((control) => !Array.isArray(control))) { + throw new Error('Expected wire control rows') + } + return controls as unknown[][] +} + +function payload(row: unknown[]): unknown[] { + if (!Array.isArray(row[9])) throw new Error('Expected wire payload') + return row[9] +} + +describe('query execution index validation', () => { + it('builds deeply immutable generated operation and channel indexes', () => { + const current = fixture() + const value = ready(inspectQueryIndex(current.graph)) + const operations = value.operations_by_owner.get(current.runId) + const operation = value.operation_by_id.get(current.operationIds[0]!) + const reports = value.channels_by_key.get('reports') + + expect(operations?.map((fact) => fact.kind)).toEqual(expect.arrayContaining([ + 'condition', + 'parallel', + 'persistence', + 'return', + ])) + expect(operation).toBeDefined() + expect(value.operations_by_owner.get(operation!.owner_symbol_id)) + .toEqual(operations) + expect(reports).toEqual([ + expect.objectContaining({ + channel_kind: 'queue', + transport: 'bullmq', + key: 'reports', + }), + ]) + expect(value.channels_by_key.get('complete')).toEqual([ + expect.objectContaining({ + channel_kind: 'job', + parent_channel_id: reports![0]!.id, + }), + ]) + expect(value.channels_by_key.get('dynamic-jobs')).toEqual([ + expect.objectContaining({ + channel_kind: 'queue', + key: 'dynamic-jobs', + }), + ]) + expect(value.channels_by_key.get('report.ready')).toEqual([ + expect.objectContaining({ + channel_kind: 'event', + scope: 'instance:events', + }), + ]) + expect(Object.isFrozen(operation)).toBe(true) + expect(Object.isFrozen(operation!.evidence)).toBe(true) + expect(Object.isFrozen(operations)).toBe(true) + expect(Object.isFrozen(reports)).toBe(true) + expect((value.operation_by_id as Map).set).toBeUndefined() + expect(Object.hasOwn( + value.graph.nodeAttributes(current.runId), + 'body_facts', + )).toBe(false) + }) + + it.each([ + { + name: 'a sparse compact row slot', + mutate: ({ graph, runId }: Fixture) => { + const rows: unknown[] = bodyFactWireRows(graph, runId) + rows[1] = null + replaceBodyFactWireRows(graph, runId, rows) + }, + }, + { + name: 'a noncanonical compact row', + mutate: ({ graph, runId }: Fixture) => { + const rows = bodyFactWireRows(graph, runId) + rows[0] = rows[0]!.replace(',', ', ') + replaceBodyFactWireRows(graph, runId, rows) + }, + }, + { + name: 'reordered compact rows', + mutate: ({ graph, runId }: Fixture) => { + const rows = bodyFactWireRows(graph, runId) + ;[rows[0], rows[1]] = [rows[1]!, rows[0]!] + replaceBodyFactWireRows(graph, runId, rows) + }, + }, + { + name: 'an invalid excerpt hash', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + const evidence = rows[0]![5] + if (!Array.isArray(evidence)) throw new Error('Missing wire evidence') + evidence[8] = 'not-a-hash' + }) + }, + }, + { + name: 'a well-shaped replaced excerpt hash', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + const evidence = rows[0]![5] + if (!Array.isArray(evidence)) throw new Error('Missing wire evidence') + evidence[8] = 'a'.repeat(64) + }) + }, + }, + { + name: 'a statement outside its owner range', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + const evidence = rows[0]![5] + if (!Array.isArray(evidence)) throw new Error('Missing wire evidence') + evidence[6] = 99 + evidence[7] = 1 + }) + }, + }, + { + name: 'a wrong authenticated operation id', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + rows[0]![0] = `operation:${'f'.repeat(32)}` + }) + }, + }, + { + name: 'duplicate branch controls', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + const row = rows.find((candidate) => + candidate[1] === 3 + && controlRows(candidate).some((control) => control[0] === 0)) + if (!row) throw new Error('Expected branch-controlled call') + const branch = controlRows(row).find((control) => control[0] === 0)! + controlRows(row).push([...branch]) + }) + }, + }, + { + name: 'an incompatible branch arm', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + const row = rows.find((candidate) => + candidate[1] === 3 + && controlRows(candidate).some((control) => control[0] === 0)) + if (!row) throw new Error('Expected branch-controlled call') + const branch = controlRows(row).find((control) => control[0] === 0)! + branch[2] = 'truthy' + }) + }, + }, + { + name: 'a null persistence receiver', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + payload(rowOfKind(rows, 6))[3] = null + }) + }, + }, + { + name: 'an unrelated persistence call target', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + const persistence = payload(rowOfKind(rows, 6)) + const unrelated = rows.findIndex((row, ordinal) => + row[1] === 3 && ordinal !== persistence[1]) + if (unrelated < 0) throw new Error('Expected unrelated call') + persistence[1] = unrelated + }) + }, + }, + { + name: 'dropped persistence control context', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + rowOfKind(rows, 6)[6] = [] + }) + }, + }, + { + name: 'an unrelated parallel member', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + const parallelOrdinal = rows.findIndex((row) => row[1] === 2) + if (parallelOrdinal < 0) throw new Error('Expected parallel fact') + const members = payload(rows[parallelOrdinal]!)[2] + if (!Array.isArray(members)) throw new Error('Expected parallel members') + const unrelated = rows.findIndex((row, ordinal) => + row[1] === 3 + && !members.includes(ordinal) + && !controlRows(row).some((control) => + control[0] === 2 && control[1] === parallelOrdinal)) + if (unrelated < 0) throw new Error('Expected unrelated parallel call') + members[0] = unrelated + }) + }, + }, + { + name: 'a parallel lane at lane_count', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + const parallelOrdinal = rows.findIndex((row) => row[1] === 2) + if (parallelOrdinal < 0) throw new Error('Expected parallel fact') + const laneCount = payload(rows[parallelOrdinal]!)[3] + const member = rows.find((row) => + controlRows(row).some((control) => + control[0] === 2 && control[1] === parallelOrdinal)) + if (!member || typeof laneCount !== 'number') { + throw new Error('Expected parallel member and lane count') + } + const frame = controlRows(member).find((control) => + control[0] === 2 && control[1] === parallelOrdinal)! + frame[2] = laneCount + }) + }, + }, + { + name: 'a channel descriptor changed without its canonical ID', + mutate: ({ graph, queueId }: Fixture) => { + graph.replaceNodeAttributes(queueId, { + ...graph.nodeAttributes(queueId), + key: 'forged-reports', + }) + }, + }, + { + name: 'a missing job parent', + mutate: ({ graph, jobId }: Fixture) => { + graph.replaceNodeAttributes(jobId, { + ...graph.nodeAttributes(jobId), + parent_channel_id: 'channel:bullmq:queue:missing', + }) + }, + }, + { + name: 'reversed publish endpoints', + mutate: ({ graph, runId, jobId }: Fixture) => { + graph.addEdge(jobId, runId, { relation: 'publishes_to' }) + }, + }, + { + name: 'a channel relation without authenticated evidence', + mutate: ({ graph, runId, queueId }: Fixture) => { + graph.addEdge(runId, queueId, { relation: 'publishes_to' }) + }, + }, + { + name: 'an unscoped event', + mutate: ({ graph, eventId }: Fixture) => { + const attributes = graph.nodeAttributes(eventId) + delete attributes.scope + graph.replaceNodeAttributes(eventId, attributes) + }, + }, + ])('rejects $name as corrupt after re-signing', ({ mutate }) => { + const value = fixture() + mutate(value) + resign(value.graph) + + expect(inspectQueryIndex(value.graph)).toMatchObject({ + state: 'corrupt', + }) + }) + + it('rejects a structurally self-consistent forged fact at retrieval', () => { + const current = fixture() + mutateBodyFactWireRows(current.graph, current.runId, (rows) => { + const parallel = rowOfKind(rows, 2) + const members = payload(parallel)[2] + if (!Array.isArray(members) || typeof members[0] !== 'number') { + throw new Error('Expected parallel member ordinal') + } + const member = rows[members[0]] + if (!member) throw new Error('Expected parallel member row') + const evidence = member[5] + if (!Array.isArray(evidence)) throw new Error('Expected member evidence') + const forgedHash = 'b'.repeat(64) + evidence[8] = forgedHash + const kindOrdinal = member[1] + if (typeof kindOrdinal !== 'number' || !wireKinds[kindOrdinal]) { + throw new Error('Expected wire fact kind') + } + member[0] = indexBodyFactId( + current.runId, + wireKinds[kindOrdinal]!, + [member[2], kindOrdinal, member[3], member[4]] as number[], + forgedHash, + ) + }) + resign(current.graph) + const structurallyReady = ready(inspectQueryIndex(current.graph)) + + const result = retrieveContext(structurallyReady, { + question: 'Explain the `run` function.', + budget: 4_000, + }) + expect(result.outcome).toBe('corrupt') + expect(result.matched_nodes).toEqual([]) + expect(result.boundaries).toContainEqual({ + kind: 'corrupt', + subject: current.runId, + }) + }) + + it('authenticates selected-owner channel edge bytes before returning evidence', () => { + const current = fixture() + const edge = current.graph.edgeEntries().find(([from, , attributes]) => + from === current.runId && attributes.relation === 'publishes_to') + if (!edge) throw new Error('Expected generated channel edge') + const attributes = structuredClone(edge[2]) + const evidence = attributes.evidence as Record + evidence.excerpt_sha256 = 'c'.repeat(64) + const forgedId = current.graph.addEdge(edge[0], edge[1], attributes) + resign(current.graph) + const index = ready(inspectQueryIndex(current.graph)) + + const result = retrieveContext(index, { + question: 'Explain the `run` function.', + budget: 4_000, + }) + expect(result.outcome).toBe('corrupt') + expect(result.boundaries).toContainEqual({ + kind: 'corrupt', + subject: forgedId, + }) + }) + + it('reports stale when selected-owner source bytes change', () => { + const current = fixture() + const index = ready(inspectQueryIndex(current.graph)) + writeFileSync( + current.sourcePath, + current.source.replace('return values', 'return values.slice()'), + 'utf8', + ) + + const result = retrieveContext(index, { + question: 'Where is the `run` function defined?', + budget: 4_000, + }) + + expect(result.outcome).toBe('stale') + expect(result.matched_nodes).toEqual([]) + expect(result.boundaries).toContainEqual({ + kind: 'stale', + subject: 'src/run.ts', + }) + }) + + it('authenticates operation ranges generated from a BOM-prefixed source', () => { + const current = fixture(true) + const result = retrieveContext(ready(inspectQueryIndex(current.graph)), { + question: 'Where is the `run` function defined?', + budget: 4_000, + }) + + expect(result.outcome).toBe('evidence') + expect(result.matched_nodes.some((node) => node.node_id === current.runId)).toBe(true) + }) + + it('reports stale before decoding mutated invalid UTF-8 bytes', () => { + const current = fixture() + const index = ready(inspectQueryIndex(current.graph)) + writeFileSync(current.sourcePath, Buffer.from([0xff])) + + const result = retrieveContext(index, { + question: 'Where is the `run` function defined?', + budget: 4_000, + }) + + expect(result.outcome).toBe('stale') + expect(result.boundaries).toContainEqual({ + kind: 'stale', + subject: 'src/run.ts', + }) + }) +}) diff --git a/tests/unit/retrieve-context.test.ts b/tests/unit/retrieve-context.test.ts index 16e2edbe..0af2e25f 100644 --- a/tests/unit/retrieve-context.test.ts +++ b/tests/unit/retrieve-context.test.ts @@ -1841,6 +1841,10 @@ describe('retrieve context', () => { root_path: '/workspace', file_hashes: new Map(), unsupported_sources: [], + operation_by_id: new Map(), + operations_by_owner: new Map(), + channels_by_id: new Map(), + channels_by_key: new Map(), } const slice = traverseEvidencePaths(index, { diff --git a/tests/unit/retrieve-evidence-skeleton-adversarial.test.ts b/tests/unit/retrieve-evidence-skeleton-adversarial.test.ts index 096f8c7f..d1b06dc0 100644 --- a/tests/unit/retrieve-evidence-skeleton-adversarial.test.ts +++ b/tests/unit/retrieve-evidence-skeleton-adversarial.test.ts @@ -113,6 +113,10 @@ function syntheticIndex(graph: KnowledgeGraph): ReadyQueryIndex { root_path: '/workspace', file_hashes: new Map(), unsupported_sources: [], + operation_by_id: new Map(), + operations_by_owner: new Map(), + channels_by_id: new Map(), + channels_by_key: new Map(), } } diff --git a/tests/unit/retrieve-evidence-skeleton-regression.test.ts b/tests/unit/retrieve-evidence-skeleton-regression.test.ts index ad018e6c..17178db3 100644 --- a/tests/unit/retrieve-evidence-skeleton-regression.test.ts +++ b/tests/unit/retrieve-evidence-skeleton-regression.test.ts @@ -255,10 +255,14 @@ beforeAll(() => { if (!moduleEntry || !methodEntry) { throw new Error('Expected deterministic module distractor nodes') } + const { + body_facts: _authenticatedBodyFacts, + ...distractorMethodAttributes + } = methodEntry[1] for (let ordinal = 0; ordinal < 10_001; ordinal += 1) { const nodeId = `issue-625-overlay-${ordinal.toString().padStart(5, '0')}` overlayGraph.addNode(nodeId, { - ...methodEntry[1], + ...distractorMethodAttributes, label: `ideaReportPipelineAssemblyPersistenceOverlay${ordinal}()`, qualified_name: `PipelineAssemblyCatalogModule.ideaReportPipelineAssemblyPersistenceOverlay${ordinal}`, From c977de03ecba7958d03966df728abed9f1b36ff7 Mon Sep 17 00:00:00 2001 From: mohammed naji Date: Fri, 31 Jul 2026 10:38:36 +0400 Subject: [PATCH 2/5] fix(index): enforce execution proof exactness --- docs/core-reset/removal-manifest.yml | 67 +- docs/core-reset/scorecard.md | 5 +- src/adapters/typescript/execution.ts | 1274 +++++++++-------- src/domain/index/model.ts | 235 +-- src/domain/query/index-status.ts | 516 ++++--- ...anonical-index-execution-hardening.test.ts | 12 +- ...index-execution-review-regressions.test.ts | 498 +++++++ tests/unit/core-reset-governance.test.ts | 43 +- .../query-index-execution-validation.test.ts | 9 + 9 files changed, 1618 insertions(+), 1041 deletions(-) create mode 100644 tests/unit/canonical-index-execution-review-regressions.test.ts diff --git a/docs/core-reset/removal-manifest.yml b/docs/core-reset/removal-manifest.yml index 0aa82846..babc776b 100644 --- a/docs/core-reset/removal-manifest.yml +++ b/docs/core-reset/removal-manifest.yml @@ -33,16 +33,16 @@ current: base_commit: 9043320cfa08370e5cdd3911bfb9283005aa9912 completed_phase_commit: b6562b715133304bd46e537b6f39008bc1e02095 production_typescript_files: 44 - production_typescript_loc: 15873 - production_loc_added: 3606 + production_typescript_loc: 15942 + production_loc_added: 3675 production_loc_removed: 187 - production_loc_net: 3419 + production_loc_net: 3488 npm_files: 102 - npm_packed_bytes: 143170 - npm_unpacked_bytes: 638803 - npm_shasum: 8ed94633cdf6f63dfabe12845e5e77644a89dbf1 - npm_integrity: sha512-Nq3m/a7D/iZHYjjRd9b7ENth95qz4zssOPnBqmQEsy+aiebRp68UtMMWArYWFRTftdWwNfTcIvxns0JVdahrdw== - npm_artifact_sha256: df9bb97b425db5063ab178920c7fb824c19c681378744c6cfc7094e99926f1eb + npm_packed_bytes: 143549 + npm_unpacked_bytes: 639164 + npm_shasum: 590e27d375f95e5212374b7478e480f58199d702 + npm_integrity: sha512-hN7chlybxfu1ZGZptit9rEP03eaYmBBGoFMornzenbbp9taNjjdl+Lj3DSjZQDhV9pJc5YdRPOlRVrccS1s6tA== + npm_artifact_sha256: cf5a6c25c4fec77e2a2da03bfd7564ec394b238456eb4f38e738f6386d392e76 measurement_state: source_and_package_exact snapshot_scope: semantic_execution_index_632_candidate release_candidate: @@ -2233,6 +2233,7 @@ items: verification: - tests/unit/canonical-index-execution.test.ts - tests/unit/canonical-index-execution-hardening.test.ts + - tests/unit/canonical-index-execution-review-regressions.test.ts - tests/unit/query-index-execution-validation.test.ts - tests/unit/retrieve-context.test.ts - tests/unit/graph-artifact.test.ts @@ -2265,28 +2266,28 @@ items: candidate: source_measurement: production_typescript_files: 44 - production_typescript_loc: 15873 - added: 3606 + production_typescript_loc: 15942 + added: 3675 removed: 187 - net: 3419 - diff_sha256: 1916499d8e195e20c47cc05868a50f3ca408759e932596c16388da6e66e41ec6 + net: 3488 + diff_sha256: 03023cece3f0cb641aa05971e903ca4b3904d613c614a994f6427c472d67f4f4 package_measurement: files: 102 - packed_bytes: 143170 - unpacked_bytes: 638803 - shasum: 8ed94633cdf6f63dfabe12845e5e77644a89dbf1 - integrity: sha512-Nq3m/a7D/iZHYjjRd9b7ENth95qz4zssOPnBqmQEsy+aiebRp68UtMMWArYWFRTftdWwNfTcIvxns0JVdahrdw== - artifact_sha256: df9bb97b425db5063ab178920c7fb824c19c681378744c6cfc7094e99926f1eb + packed_bytes: 143549 + unpacked_bytes: 639164 + shasum: 590e27d375f95e5212374b7478e480f58199d702 + integrity: sha512-hN7chlybxfu1ZGZptit9rEP03eaYmBBGoFMornzenbbp9taNjjdl+Lj3DSjZQDhV9pJc5YdRPOlRVrccS1s6tA== + artifact_sha256: cf5a6c25c4fec77e2a2da03bfd7564ec394b238456eb4f38e738f6386d392e76 local_verification: - focused_tests_passed: 149 + focused_tests_passed: 162 last_pre_cache_full_tests_passed: 722 candidate_full_suite_status: pending_clean_runner_after_local_host_memory_pressure candidate_full_suite_local_attempts: 3 - graph_nodes: 12350 - graph_edges: 32839 + graph_nodes: 12313 + graph_edges: 32726 exact_queue_channels: 6 exact_channel_edges: 51 - graph_artifact_size_ratio: 1.2344886849638093 + graph_artifact_size_ratio: 1.2293214469979519 beta4_indexing_trials_seconds: - 20.32 - 21.33 @@ -2295,18 +2296,18 @@ items: - 22.74 beta4_indexing_median_seconds: 22.16 candidate_indexing_trials_seconds: - - 23.31 - - 24.16 - - 24.41 - - 24.87 - - 25.42 - candidate_indexing_median_seconds: 24.41 - indexing_median_ratio: 1.1015342960288808 + - 17.08 + - 15.08 + - 14.96 + - 13.82 + - 13.24 + candidate_indexing_median_seconds: 14.96 + indexing_median_ratio: 0.6750902527075813 warm_retrieval_samples: 100 - warm_retrieval_median_ms: 165.343 - warm_retrieval_p95_ms: 216.599 - warm_retrieval_max_ms: 257.814 - deterministic_graph_sha256: 478e1ed93fcef5a7592dfa0f884318713128e01857dc706a28f2e0d6142937d0 + warm_retrieval_median_ms: 225.692208 + warm_retrieval_p95_ms: 253.599958 + warm_retrieval_max_ms: 300.229625 + deterministic_graph_sha256: 82469164dbdf1229195d6fb71d8220736c15c5a13093dd2f2fc6f2a633f73a36 beta4_retrieval_output_byte_identical: true exact_head_ci: pending independent_review: pending @@ -2327,7 +2328,7 @@ items: registry_metadata_publication: forbidden tag: forbidden main_target: forbidden - notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. Exact local source, package, focused-test, real-corpus graph-size, channel-topology, beta.4 retrieval-compatibility, alternating indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The pre-cache candidate passed all 722 tests; the exact final candidate full suite remains pending a clean hosted runner because three local attempts were stopped by fork-start or per-test timeouts while an unrelated user process held approximately 19 GB of resident memory. Exact-head CI, independent review, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' + notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. First PR head 9fe3c2448958c6b8cead2452758077fef093cf4e passed all six hosted jobs but independent review blocked twelve semantic-proof classes, so it was not merged. The corrective candidate binds fact IDs to semantic payloads; rejects unproven Promise lanes, mutable channels, uninvoked nested functions, false persistence, custom array lookalikes, unreachable control flow, and synchronous fire-and-forget claims; scopes event emitters; bounds wrapper expansion; and redacts secret-bearing receiver text. Exact corrected local source, package, focused-test, real-corpus graph-size, channel-topology, beta.4 retrieval-compatibility, indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The pre-cache candidate passed all 722 tests; the exact final candidate full suite remains pending a clean hosted runner because three local attempts were stopped by fork-start or per-test timeouts while an unrelated user process held approximately 19 GB of resident memory. Exact corrected-head CI, independent review, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' exit_gate: Every retained fact and exact channel edge is deterministic, source-authenticated and mutation-sensitive; false persistence/channel matches remain absent; source, graph-size, indexing, warm-retrieval, package, full-test, exact-head CI, independent-review, and zero-thread gates pass without a v2 result cutover, new dependency, publication, tag, release, Registry metadata, or main target. - id: obligation-driven-retrieval-630 diff --git a/docs/core-reset/scorecard.md b/docs/core-reset/scorecard.md index bbb28c85..199c0161 100644 --- a/docs/core-reset/scorecard.md +++ b/docs/core-reset/scorecard.md @@ -43,7 +43,7 @@ The schema-validated, share-safe receipt was recorded at tooling checkout `250a6 | Retrieval regression #618 | **Passed** | Restore grounded natural-flow retrieval in one call or at most one bounded recovery without repository-specific rules, graph/index changes, dependencies, or fallback engines | [#618](https://github.com/mohanagy/madar/issues/618) completed through [PR #620](https://github.com/mohanagy/madar/pull/620), merged at `eaa1a8781eda28dad5395d6da378a2cc40bf81fe`; all six exact-head CI jobs, two independent no-blocker reviews, and zero review threads passed | | Retrieval regression #622 | **Passed** | Stabilize equivalent end-to-end report-flow prompts and expose honest asynchronous handoff targets within the unchanged retrieval and package ceilings | [#622](https://github.com/mohanagy/madar/issues/622) completed through [PR #623](https://github.com/mohanagy/madar/pull/623), merged at `6416dbc02cefb3bd79157cf440e420b30dda8cf0`; [six-job CI](https://github.com/mohanagy/madar/actions/runs/30452883659), two exact-head no-blocker reviews, CodeRabbit PASS, and zero unresolved threads | | Retrieval regression #625 | **Passed** | Replace phrase-gated recovery with a generic bounded, graph-coherent evidence skeleton/forest without exceeding the inherited package ceilings | [#625](https://github.com/mohanagy/madar/issues/625) completed through [PR #626](https://github.com/mohanagy/madar/pull/626), merged at `b6562b715133304bd46e537b6f39008bc1e02095`; [six-job CI](https://github.com/mohanagy/madar/actions/runs/30533140531), independent exact-head review, CodeRabbit PASS, and zero unresolved threads | -| Semantic execution index #632 | **In progress** | Authenticated ordered body facts, exact async channels and receiver/type-proven persistence pass every source, graph, indexing, latency, package, CI, review and zero-thread gate | [#632](https://github.com/mohanagy/madar/issues/632); protected base `9043320cfa08370e5cdd3911bfb9283005aa9912`; no candidate, CI, review or merge receipt yet | +| Semantic execution index #632 | **In progress** | Authenticated ordered body facts, exact async channels and receiver/type-proven persistence pass every source, graph, indexing, latency, package, CI, review and zero-thread gate | [#632](https://github.com/mohanagy/madar/issues/632); protected base `9043320cfa08370e5cdd3911bfb9283005aa9912`; first PR head passed CI but was independently blocked and superseded; corrected-head CI/review/merge remain pending | | Obligation-driven retrieval #630 | **Pending** | Return a complete authenticated workflow dossier or exact missing obligations within unchanged budgets | [#630](https://github.com/mohanagy/madar/issues/630); blocked on #632 | | No-fallback qualification #631 | **Pending** | Installed exact-head package matches or beats the strongest frozen baseline and requires zero repository-tool fallback | [#631](https://github.com/mohanagy/madar/issues/631); blocked on #632 and #630 | | External validation | **Deferred** | Activation, retention, and paid-intent evidence remains required for later stable claims, not this beta | No external-validation claim in `0.40.0-beta.4` | @@ -203,7 +203,8 @@ The following contract facts are historical. Issues #610 and #612, together with - Delivery is blocked above four new production files, 3,500 net new production lines, 1.5x the beta.4 GoValidate graph size, 1.25x the beta.4 same-machine indexing median, or warm retrieval p95 greater than or equal to 500 ms. Package ceilings remain 102 files / 165,000 packed / 640,000 unpacked bytes. - The package whitelist may remove only `examples/why-madar.md` and `CHANGELOG.md`; the repository files remain present, and version, scripts, dependencies, package lock, publication and public surface cannot change. - #632 owns authenticated ordered body facts, exact queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. Retrieval-result v2, obligation planning, response dossier generation, comparator claims, provider activity, npm publication, GitHub Release, Registry metadata, tags, and `main` are outside this phase. -- No candidate final receipt, CI result, review result, merge commit, package result, performance result, or publication is claimed here. Those fields remain open until the exact head passes its gates. +- First PR head `9fe3c2448958c6b8cead2452758077fef093cf4e` passed all six hosted jobs but was independently blocked on twelve semantic-proof classes and was not merged. The corrected local candidate addresses all twelve with dedicated regression coverage. It measures 44 production files / 15,942 LOC at `+3,675/-187/net +3,488`; 102 package files / 143,549 packed / 639,164 unpacked bytes; and a 60,274,421-byte real GoValidate graph, ratio `1.2293214469979519`, containing 12,313 nodes / 32,726 edges / six exact queue channels / 51 typed channel edges. Five corrected indexing trials have 14.96-second median, ratio `0.6750902527075813`; 100 warm retrieval samples have 225.692208 ms median / 253.599958 ms p95 / 300.229625 ms maximum. Repeated graph SHA-256 is `82469164dbdf1229195d6fb71d8220736c15c5a13093dd2f2fc6f2a633f73a36`, and beta.4 v1 retrieval output remains byte-identical. +- These are local corrective-candidate measurements, not a final receipt. Corrected-head commit/tree, all-six CI, independent no-blocker review, CodeRabbit completion, zero unresolved threads, merge commit, and publication remain open until those exact gates pass. ### Successors #630 and #631 (pending) diff --git a/src/adapters/typescript/execution.ts b/src/adapters/typescript/execution.ts index cfbf2eca..abf12770 100644 --- a/src/adapters/typescript/execution.ts +++ b/src/adapters/typescript/execution.ts @@ -10,24 +10,17 @@ const { } = ts const K = ts.SyntaxKind import { - decodeIndexBodyFactTable, - encodeIndexBodyFactTable, - INDEX_BODY_FACT_CONTROL_LIMIT, - IndexBodyFactBoundsError, - indexBodyFactId, - indexChannelId, + decodeIndexBodyFactTable, encodeIndexBodyFactTable, INDEX_BODY_FACT_CONTROL_LIMIT, + IndexBodyFactBoundsError, indexBodyFactId, indexChannelId, } from '../../domain/index/model.js' import type { IndexBodyFact, IndexCallFact, IndexChannelNode, IndexControlFrame, IndexDiagnostic, IndexEdge, IndexFactEvidence, IndexFactSource, - IndexChannelTransport, IndexPersistenceOperation, IndexRange, IndexSymbol, - IndexValue, + IndexChannelTransport, IndexPersistenceOperation, IndexRange, IndexSymbol, IndexValue, } from '../../domain/index/model.js' export type CollectExecutionInput = { - program: ts.Program - sourceFiles: readonly ts.SourceFile[] - checker: ts.TypeChecker - pathToFileId: ReadonlyMap + program: ts.Program; sourceFiles: readonly ts.SourceFile[] + checker: ts.TypeChecker; pathToFileId: ReadonlyMap symbols: IndexSymbol[] symbolsByFile: ReadonlyMap } @@ -38,92 +31,87 @@ export type CollectExecutionResult = { type Confidence = 'high' | 'medium' | 'low' type OwnerSpan = { symbol: IndexSymbol; start: number; end: number } type ImportBinding = { imported: string; module: string; namespace: boolean } -type CallSite = readonly [ - targetId: string, arguments: readonly IndexValue[], node: EffectWitness, -] +type CallSite = readonly [targetId: string, arguments: readonly IndexValue[], node: EffectWitness] type QueueTransport = Extract type QueueOrigin = readonly [key: IndexValue, transport: QueueTransport] -type EmitterScope = readonly [ - scope: string, - transport: 'node-event-emitter' | 'nestjs-event-emitter', -] +type MapQueueEntry = readonly [key: ts.Expression, file: FileContext, queueKey: ts.Expression, transport: QueueTransport] +type EmitterScope = readonly [scope: string, transport: 'node-event-emitter' | 'nestjs-event-emitter'] type EffectWitness = ts.CallExpression | ts.NewExpression type BullEffect = readonly [ kind: 'bull-publish' | 'bull-consume', queue: IndexValue, endpoint: IndexValue, transport: QueueTransport, scope: undefined, - witness: EffectWitness, confidence: Confidence, source: IndexFactSource, -] + witness: EffectWitness, confidence: Confidence, source: IndexFactSource] type EventEffect = readonly [ kind: 'event-publish' | 'event-consume', event: IndexValue, handler: IndexValue | undefined, transport: 'node-event-emitter' | 'nestjs-event-emitter', scope: string, - witness: EffectWitness, confidence: Confidence, source: IndexFactSource, -] + witness: EffectWitness, confidence: Confidence, source: IndexFactSource] type PersistenceEffect = readonly [ kind: 'persistence', operation: IndexPersistenceOperation, resource: IndexValue | undefined, receiverType: string, scope: undefined, - witness: EffectWitness, confidence: Confidence, source: IndexFactSource, -] + witness: EffectWitness, confidence: Confidence, source: IndexFactSource] type ExecutionEffect = BullEffect | EventEffect | PersistenceEffect type FileContext = { - sf: ts.SourceFile; fileId: string - imports: ReadonlyMap; owners: readonly OwnerSpan[] + sf: ts.SourceFile; fileId: string; imports: ReadonlyMap + owners: readonly OwnerSpan[] } type CollectionState = { - input: CollectExecutionInput - symbolsById: ReadonlyMap - facts: Map - overflow: Set - effects: Map - calls: Map - callIds: Map - channels: Map - edges: IndexEdge[] - diagnostics: IndexDiagnostic[] - seenDiagnostics: Set - queues: Map> - emitters: Map - nestQueues: Map> + input: CollectExecutionInput; symbolsById: ReadonlyMap + facts: Map; overflow: Set + effects: Map; calls: Map + callIds: Map; channels: Map + edges: IndexEdge[]; diagnostics: IndexDiagnostic[]; seenDiagnostics: Set + unstable: Set + mapQueues: Map> + emitters: Map; nestQueues: Map> files: Map } -const MAX_VALUE_DEPTH = 5, MAX_VALUE_ELEMENTS = 32 -const MAX_STRING_BYTES = 512, MAX_TEXT_BYTES = 256 -const MAX_WRAPPER_HOPS = 2 -const MAX_FACTS_PER_OWNER = 8_192, MAX_EFFECTS_PER_OWNER = 8_192 -const FS_MODULES = new Set(['node:fs', 'node:fs/promises', 'fs', 'fs/promises']) -const FS_OPERATIONS = { - readFile: 'file_read', readFileSync: 'file_read', open: 'file_read', +// Internal helpers are abbreviated because their emitted names count against +// the protected npm ceiling; exported names and serialized fields stay explicit. +const VDEP = 5, VELE = 32 +const SBYT = 512, TBYT = 256 +const WHOP = 2 +const FMAX = 8_192, EMAX = 8_192 +const FSM = new Set(['node:fs', 'node:fs/promises', 'fs', 'fs/promises']) +const FSO = { + readFile: 'file_read', readFileSync: 'file_read', opendir: 'file_read', readdir: 'file_read', appendFile: 'file_write', appendFileSync: 'file_write', copyFile: 'file_write', copyFileSync: 'file_write', rename: 'file_write', writeFile: 'file_write', writeFileSync: 'file_write', rm: 'delete', rmSync: 'delete', unlink: 'delete', unlinkSync: 'delete', } as const satisfies Record -const REPOSITORY_OPERATIONS = { +const TOO = { find: 'read', findOne: 'read', findOneBy: 'read', findMany: 'read', - findUnique: 'read', count: 'read', aggregate: 'read', create: 'create', - insert: 'create', save: 'create', createMany: 'create', update: 'update', + findUnique: 'read', count: 'read', aggregate: 'read', insert: 'create', + save: 'upsert', update: 'update', updateOne: 'update', updateMany: 'update', delete: 'delete', deleteOne: 'delete', deleteMany: 'delete', remove: 'delete', upsert: 'upsert', - transaction: 'transaction', $transaction: 'transaction', + transaction: 'transaction', } as const satisfies Record -const PROMISE_COMPLETIONS = { +const PRO = { + findUnique: 'read', findFirst: 'read', findMany: 'read', count: 'read', + aggregate: 'read', groupBy: 'read', create: 'create', createMany: 'create', + update: 'update', updateMany: 'update', delete: 'delete', + deleteMany: 'delete', upsert: 'upsert', $transaction: 'transaction', +} as const satisfies Record +const PMC = { all: 'all_or_first_rejection', allSettled: 'all_settled', any: 'first_fulfilled', race: 'first_settled', } as const -const LOGICAL_FLOW = new Map([ +const LFL = new Map([ [K.AmpersandAmpersandToken, ['logical_and', 'truthy']], [K.BarBarToken, ['logical_or', 'falsy']], [K.QuestionQuestionToken, ['nullish', 'nullish']], ]) -const ARRAY_MUTATIONS = new Map([ +const AMU = new Map([ ['push', 'append'], ['unshift', 'append'], ['pop', 'remove'], ['shift', 'remove'], ['splice', 'remove'], ]) -const FACT_ORDER: Readonly> = { +const FORD: Readonly> = { condition: 0, loop: 1, parallel: 2, call: 3, literal: 4, mutation: 5, persistence: 6, return: 7, throw: 8, } -const ASSIGNMENT_OPERATORS = new Set([ +const AOP = new Set([ K.EqualsToken, K.PlusEqualsToken, K.MinusEqualsToken, K.AsteriskEqualsToken, K.AsteriskAsteriskEqualsToken, K.SlashEqualsToken, @@ -133,22 +121,22 @@ const ASSIGNMENT_OPERATORS = new Set([ K.BarBarEqualsToken, K.AmpersandAmpersandEqualsToken, K.QuestionQuestionEqualsToken, ]) -const ARRAY_ITERATION_METHODS = new Set([ +const AIM = new Set([ 'every', 'filter', 'find', 'findIndex', 'flatMap', 'forEach', 'map', 'reduce', 'reduceRight', 'some', ]) -const STRUCTURAL_LITERAL_TOKENS = new Set([ +const SLT = new Set([ K.StringLiteral, K.NumericLiteral, K.BigIntLiteral, K.RegularExpressionLiteral, K.NoSubstitutionTemplateLiteral, K.TemplateHead, K.TemplateMiddle, K.TemplateTail, ]) -const SECRET_NAME = /(?:api[_-]?key|authorization|cookie|credential|passwd|password|private[_-]?key|secret|token)/i -const SECRET_VALUE = /^(?:bearer\s+|gh[pousr]_|github_pat_|sk-(?:live|test|proj)-|xox[baprs]-)/i -function sha256(value: string): string { +const SNM = /(?:api[_-]?key|authorization|cookie|credential|database[_-]?url|dsn|jwt|passwd|password|private[_-]?key|secret|token)/i +const SVAL = /^(?:bearer\s+|gh[pousr]_|github_pat_|sk-(?:live|test|proj)-|xox[baprs]-|[a-z][a-z\d+.-]*:\/\/[^/\s:@]+:[^@\s/]+@|eyJ[\w-]+\.[\w-]+\.[\w-]+$)/i +function hash(value: string): string { return createHash('sha256').update(value, 'utf8').digest('hex') } -function boundedText(value: string, maxBytes = MAX_TEXT_BYTES): string { +function bd(value: string, maxBytes = TBYT): string { if (Buffer.byteLength(value, 'utf8') <= maxBytes) return value let result = '' for (const character of value) { @@ -157,18 +145,18 @@ function boundedText(value: string, maxBytes = MAX_TEXT_BYTES): string { } return result } -function structuralText(node: ts.Node, sf: ts.SourceFile): string { +function st(node: ts.Node, sf: ts.SourceFile): string { const scanner = ts.createScanner(ts.ScriptTarget.Latest, true, sf.languageVariant, node.getText(sf)); const tokens: string[] = []; for (let token = scanner.scan(); token !== K.EndOfFileToken; token = scanner.scan()) { - tokens.push(STRUCTURAL_LITERAL_TOKENS.has(token) ? '' : scanner.getTokenText()); + tokens.push(SLT.has(token) ? '' : scanner.getTokenText()); } - return boundedText(tokens.join(' ')); + return bd(tokens.join(' ')); } -function compareText(left: string, right: string): number { +function ct(left: string, right: string): number { return left < right ? -1 : left > right ? 1 : 0 } -function compareOrder(left: readonly number[], right: readonly number[]): number { +function co(left: readonly number[], right: readonly number[]): number { const length = Math.min(left.length, right.length) for (let index = 0; index < length; index += 1) { const difference = (left[index] ?? 0) - (right[index] ?? 0) @@ -176,10 +164,10 @@ function compareOrder(left: readonly number[], right: readonly number[]): number } return left.length - right.length } -function rangeOf(node: ts.Node, sf: ts.SourceFile): IndexRange { - return rangeFromOffsets(sf, node.getStart(sf, false), node.getEnd()) +function ro(node: ts.Node, sf: ts.SourceFile): IndexRange { + return rf(sf, node.getStart(sf, false), node.getEnd()) } -function rangeFromOffsets(sf: ts.SourceFile, start: number, end: number): IndexRange { +function rf(sf: ts.SourceFile, start: number, end: number): IndexRange { const startPosition = sf.getLineAndCharacterOfPosition(start) const endPosition = sf.getLineAndCharacterOfPosition(end) return { @@ -187,7 +175,7 @@ function rangeFromOffsets(sf: ts.SourceFile, start: number, end: number): IndexR end: { line: endPosition.line + 1, column: endPosition.character + 1 }, } } -function statementFor(node: ts.Node): ts.Node { +function stmt(node: ts.Node): ts.Node { let current: ts.Node = node; while (current.parent) { if (ts.isStatement(current) @@ -202,7 +190,7 @@ function statementFor(node: ts.Node): ts.Node { } return current; } -function evidenceFor(node: ts.Node, sf: ts.SourceFile, fileId: string, stmtNode: ts.Node = statementFor(node), bounds?: { +function ev(node: ts.Node, sf: ts.SourceFile, fileId: string, stmtNode: ts.Node = stmt(node), bounds?: { start: number; end: number; }): IndexFactEvidence { @@ -216,29 +204,29 @@ function evidenceFor(node: ts.Node, sf: ts.SourceFile, fileId: string, stmtNode: : rawStatementEnd; return { file_id: fileId, - range: rangeOf(node, sf), - statement_range: rangeFromOffsets(sf, statementStart, statementEnd), - excerpt_sha256: sha256(sf.text.slice(statementStart, statementEnd)), + range: ro(node, sf), + statement_range: rf(sf, statementStart, statementEnd), + excerpt_sha256: hash(sf.text.slice(statementStart, statementEnd)), }; } -function factOrder(kind: IndexBodyFact['kind'], node: ts.Node, suffix = 0): readonly number[] { +function fo(kind: IndexBodyFact['kind'], node: ts.Node, suffix = 0): readonly number[] { const sf = node.getSourceFile() return [ node.getStart(sf, false), - FACT_ORDER[kind], + FORD[kind], node.getEnd(), suffix, ] } -function factBase(ownerId: string, kind: IndexBodyFact['kind'], node: ts.Node, file: FileContext, control: readonly IndexControlFrame[], opts: { +function fb(ownerId: string, kind: IndexBodyFact['kind'], node: ts.Node, file: FileContext, control: readonly IndexControlFrame[], opts: { confidence?: Confidence; source?: IndexFactSource; statementNode?: ts.Node; orderSuffix?: number; } = {}): Pick { const ownerBounds = file.owners.find((span) => span.symbol.id === ownerId); - const order = factOrder(kind, node, opts.orderSuffix); - const evidence = evidenceFor(node, file.sf, file.fileId, opts.statementNode, ownerBounds); + const order = fo(kind, node, opts.orderSuffix); + const evidence = ev(node, file.sf, file.fileId, opts.statementNode, ownerBounds); return { id: indexBodyFactId(ownerId, kind, order, evidence.excerpt_sha256), owner_symbol_id: ownerId, @@ -252,79 +240,75 @@ function factBase(ownerId: string, kind: IndexBodyFact['kind'], node: ts.Node, f type ConditionKind = Extract['condition_kind'] type BranchArm = Extract['arm'] type MutationOperation = Extract['operation'] -function addCondition( - ownerId: string, - conditionKind: ConditionKind, - expr: ts.Expression, - file: FileContext, - ctx: CollectionState, - control: readonly IndexControlFrame[], +function ac( + ownerId: string, conditionKind: ConditionKind, expr: ts.Expression, + file: FileContext, ctx: CollectionState, control: readonly IndexControlFrame[], stmtNode: ts.Node, -): ReturnType { - const base = factBase(ownerId, 'condition', expr, file, control, { statementNode: stmtNode }) - addFact(ctx, { +): ReturnType { + const base = fb(ownerId, 'condition', expr, file, control, { statementNode: stmtNode }) + af(ctx, { ...base, kind: 'condition', condition_kind: conditionKind, - test: readValue(expr, file, ctx, { constants: true }), + test: rd(expr, file, ctx, { constants: true }), }) return base } -function branchFrame(control: readonly IndexControlFrame[], controllerFactId: string, arm: BranchArm): IndexControlFrame[] { +function br(control: readonly IndexControlFrame[], controllerFactId: string, arm: BranchArm): IndexControlFrame[] { return [...control, { kind: 'branch', controller_fact_id: controllerFactId, arm }]; } -function addMutation( +function am( ownerId: string, operationNode: ts.Node, operation: MutationOperation, targetNode: ts.Node, file: FileContext, ctx: CollectionState, control: readonly IndexControlFrame[], value?: ts.Expression, orderSuffix = 0, ): void { - const target = structuralText(targetNode, file.sf) - addFact(ctx, { - ...factBase(ownerId, 'mutation', operationNode, file, control, { orderSuffix }), + const target = st(targetNode, file.sf) + af(ctx, { + ...fb(ownerId, 'mutation', operationNode, file, control, { orderSuffix }), kind: 'mutation', operation, - target: boundedText(target), + target: bd(target), ...(value ? { - value: readValue(value, file, ctx, { + value: rd(value, file, ctx, { constants: true, - secret: SECRET_NAME.test(target), + secret: SNM.test(target), }), } : {}), }) } -function addIteration(ownerId: string, node: ts.Node, file: FileContext, ctx: CollectionState, control: readonly IndexControlFrame[]): string { - const base = factBase(ownerId, 'loop', node, file, control, { orderSuffix: 1 }); - addFact(ctx, { ...base, kind: 'loop', loop_kind: 'array_iteration' }); +function ai(ownerId: string, node: ts.Node, file: FileContext, ctx: CollectionState, control: readonly IndexControlFrame[]): string { + const base = fb(ownerId, 'loop', node, file, control, { orderSuffix: 1 }); + af(ctx, { ...base, kind: 'loop', loop_kind: 'array_iteration' }); return base.id; } -function addFact(ctx: CollectionState, fact: IndexBodyFact): void { +function af(ctx: CollectionState, fact: IndexBodyFact): void { if (ctx.overflow.has(fact.owner_symbol_id)) return const facts = ctx.facts.get(fact.owner_symbol_id) if (!facts) { ctx.facts.set(fact.owner_symbol_id, [fact]); return } - if (facts.length >= MAX_FACTS_PER_OWNER) { + if (facts.length >= FMAX) { ctx.overflow.add(fact.owner_symbol_id); return } facts.push(fact) } -function addBounded( +function ab( ctx: CollectionState, map: Map, key: string, value: T, ): void { const values = map.get(key) if (!values) { map.set(key, [value]); return } - if (values.length >= MAX_EFFECTS_PER_OWNER) { + if (values.length >= EMAX) { ctx.overflow.add(key) } else { values.push(value) } } -function addEffect(ctx: CollectionState, ownerId: string, fx: ExecutionEffect): void { - addBounded(ctx, ctx.effects, ownerId, fx) +function ae(ctx: CollectionState, ownerId: string, fx: ExecutionEffect): void { + ab(ctx, ctx.effects, ownerId, fx) } -function addCall(ctx: CollectionState, ownerId: string, callSite: CallSite): void { - addBounded(ctx, ctx.calls, ownerId, callSite); +function al(ctx: CollectionState, ownerId: string, callSite: CallSite): void { + ab(ctx, ctx.calls, ownerId, callSite); } -function isOwnerSymbol(symbol: IndexSymbol): boolean { +function io(symbol: IndexSymbol): boolean { if (!['function', 'method', 'constant', 'variable'].includes(symbol.kind)) return false // Execution facts require an authenticated owner span. Framework-only // synthetic nodes without declaration/definition ranges remain topology @@ -334,27 +318,27 @@ function isOwnerSymbol(symbol: IndexSymbol): boolean { if (typeof symbol.framework_metadata?.storage_operation === 'string') return false return true } -function offsetOf(sf: ts.SourceFile, position: IndexRange['start']): number { +function oo(sf: ts.SourceFile, position: IndexRange['start']): number { return sf.getPositionOfLineAndCharacter(position.line - 1, position.column - 1) } -function owners(sf: ts.SourceFile, symbols: readonly IndexSymbol[]): OwnerSpan[] { +function os(sf: ts.SourceFile, symbols: readonly IndexSymbol[]): OwnerSpan[] { return symbols - .filter(isOwnerSymbol) + .filter(io) .map((symbol) => ({ symbol, - start: offsetOf(sf, symbol.range.start), - end: offsetOf(sf, symbol.range.end), + start: oo(sf, symbol.range.start), + end: oo(sf, symbol.range.end), })) .sort((left, right) => (left.end - left.start) - (right.end - right.start) || left.start - right.start - || compareText(left.symbol.id, right.symbol.id)); + || ct(left.symbol.id, right.symbol.id)); } -function ownerFor(node: ts.Node, file: FileContext): IndexSymbol | null { +function ow(node: ts.Node, file: FileContext): IndexSymbol | null { const start = node.getStart(file.sf, false) const end = node.getEnd() return file.owners.find((span) => span.start <= start && span.end >= end)?.symbol ?? null } -function importsFor(sf: ts.SourceFile): ReadonlyMap { +function im(sf: ts.SourceFile): ReadonlyMap { const bindings = new Map() for (const stmt of sf.statements) { if (!ts.isImportDeclaration(stmt) || !ts.isStringLiteral(stmt.moduleSpecifier)) continue @@ -380,7 +364,7 @@ function importsFor(sf: ts.SourceFile): ReadonlyMap { } return bindings } -function importedBinding(expr: ts.Expression, file: FileContext): ImportBinding | null { +function ib(expr: ts.Expression, file: FileContext): ImportBinding | null { if (isIdentifier(expr)) return file.imports.get(expr.text) ?? null; if (isAccess(expr) @@ -396,13 +380,13 @@ function importedBinding(expr: ts.Expression, file: FileContext): ImportBinding } return null; } -function isImport(expr: ts.Expression, file: FileContext, modules: readonly string[], names: readonly string[]): boolean { - const binding = importedBinding(expr, file); +function ii(expr: ts.Expression, file: FileContext, modules: readonly string[], names: readonly string[]): boolean { + const binding = ib(expr, file); return binding !== null && modules.includes(binding.module) && names.includes(binding.imported); } -function followAlias(symbol: ts.Symbol | undefined, checker: ts.TypeChecker): ts.Symbol | undefined { +function fa(symbol: ts.Symbol | undefined, checker: ts.TypeChecker): ts.Symbol | undefined { if (!symbol || (symbol.flags & ts.SymbolFlags.Alias) === 0) return symbol try { return checker.getAliasedSymbol(symbol) @@ -410,54 +394,70 @@ function followAlias(symbol: ts.Symbol | undefined, checker: ts.TypeChecker): ts return symbol } } -function symbolAt(node: ts.Node, ctx: CollectionState): ts.Symbol | undefined { +function sy(node: ts.Node, ctx: CollectionState): ts.Symbol | undefined { const checker = ctx.input.checker - return followAlias(checker.getSymbolAtLocation(node), checker) + return fa(checker.getSymbolAtLocation(node), checker) } -function declSymbol(node: ts.Node, file: FileContext, ctx: CollectionState): IndexSymbol | null { +function ds(node: ts.Node, file: FileContext, ctx: CollectionState): IndexSymbol | null { const sf = node.getSourceFile(); const fileId = ctx.input.pathToFileId.get(sf.fileName); if (!fileId) return null; const spans = sf === file.sf ? file.owners - : owners(sf, ctx.input.symbolsByFile.get(fileId) ?? []); + : os(sf, ctx.input.symbolsByFile.get(fileId) ?? []); const start = node.getStart(sf, false); const end = node.getEnd(); return spans.find((span) => span.start <= start && span.end >= end)?.symbol ?? null; } -function symbolFor(expr: ts.Expression, file: FileContext, ctx: CollectionState): IndexSymbol | null { - const symbol = symbolAt(isAccess(expr) ? expr.name : expr, ctx); +function ed(node: ts.Node, file: FileContext, ctx: CollectionState): IndexSymbol | null { + const symbol = ds(node, file, ctx) + if (!symbol) return null + const sf = node.getSourceFile(), start = node.getStart(sf, false), end = node.getEnd() + const spans = sf === file.sf ? file.owners + : os(sf, ctx.input.symbolsByFile.get(ctx.input.pathToFileId.get(sf.fileName) ?? '') ?? []) + return spans.some((span) => + span.symbol.id === symbol.id && span.start === start && span.end === end) + ? symbol : null +} +function sb(decl: ts.Declaration, ctx: CollectionState): boolean { + const name = (isVariable(decl) || isPropertyDecl(decl)) && isIdentifier(decl.name) + ? decl.name : null + const symbol = name ? sy(name, ctx) : undefined + return !!symbol && !ctx.unstable.has(symbol) +} +function sfor(expr: ts.Expression, file: FileContext, ctx: CollectionState): IndexSymbol | null { + const symbol = sy(isAccess(expr) ? expr.name : expr, ctx); const declarations = symbol?.declarations ?? []; for (const decl of declarations) { - const indexed = declSymbol(decl, file, ctx); + const indexed = ds(decl, file, ctx); if (indexed) return indexed; } return null; } -function callSymbol(call: ts.CallExpression | ts.NewExpression, file: FileContext, ctx: CollectionState): IndexSymbol | null { +function cs(call: ts.CallExpression | ts.NewExpression, file: FileContext, ctx: CollectionState): IndexSymbol | null { const signature = ctx.input.checker.getResolvedSignature(call); const decl = signature?.getDeclaration(); if (decl && !decl.getSourceFile().isDeclarationFile) { - const indexed = declSymbol(decl, file, ctx); + const indexed = ds(decl, file, ctx); if (indexed) return indexed; } - return symbolFor(call.expression, file, ctx); + return sfor(call.expression, file, ctx); } -function callableOwner(node: ts.SignatureDeclaration, file: FileContext, ctx: CollectionState): IndexSymbol | null { +function ca(node: ts.SignatureDeclaration, file: FileContext, ctx: CollectionState): IndexSymbol | null { if (isArrow(node) || isFunction(node)) { const parent = node.parent; if (isVariable(parent) && parent.initializer === node) { const stmt = parent.parent.parent; return ts.isVariableStatement(stmt) && ts.isSourceFile(stmt.parent) - ? declSymbol(parent, file, ctx) + ? ds(parent, file, ctx) : null; } return isBinary(parent) - ? declSymbol(node, file, ctx) + ? ds(node, file, ctx) : null; } return ts.isFunctionDeclaration(node) @@ -465,11 +465,11 @@ function callableOwner(node: ts.SignatureDeclaration, file: FileContext, ctx: Co || ts.isConstructorDeclaration(node) || ts.isGetAccessorDeclaration(node) || ts.isSetAccessorDeclaration(node) - ? declSymbol(node, file, ctx) + ? ed(node, file, ctx) : null; } -function parameterValue(identifier: ts.Identifier, file: FileContext, ctx: CollectionState): IndexValue | null { - const symbol = symbolAt(identifier, ctx); +function pv(identifier: ts.Identifier, file: FileContext, ctx: CollectionState): IndexValue | null { + const symbol = sy(identifier, ctx); for (const decl of symbol?.declarations ?? []) { if (!isParameter(decl)) continue; @@ -478,31 +478,31 @@ function parameterValue(identifier: ts.Identifier, file: FileContext, ctx: Colle continue; const position = parent.parameters.indexOf(decl); if (position >= 0) { - return callableOwner(parent, file, ctx) + return ca(parent, file, ctx) ? { kind: 'parameter', position } : { kind: 'parameter', position, scope: 'iteration' }; } } return null; } -function redactedValue(value: string): IndexValue { +function red(value: string): IndexValue { return { kind: 'redacted', - sha256: sha256(value), + sha256: hash(value), byte_length: Buffer.byteLength(value, 'utf8'), } } -function literalString(value: string, secret = false): IndexValue { +function ls(value: string, secret = false): IndexValue { const byteLength = Buffer.byteLength(value, 'utf8') - if (secret || SECRET_VALUE.test(value) || byteLength > MAX_STRING_BYTES) { - return redactedValue(value) + if (secret || SVAL.test(value) || byteLength > SBYT) { + return red(value) } return { kind: 'literal', value } } -function unknown(reason: 'dynamic' | 'ambiguous' | 'unsupported' = 'dynamic'): IndexValue { +function uk(reason: 'dynamic' | 'ambiguous' | 'unsupported' = 'dynamic'): IndexValue { return { kind: 'unknown', reason } } -function unwrap(node: ts.Expression): ts.Expression { +function uw(node: ts.Expression): ts.Expression { let current = node; while (ts.isAsExpression(current) || ts.isTypeAssertionExpression(current) @@ -513,37 +513,32 @@ function unwrap(node: ts.Expression): ts.Expression { } return current; } -type ValueOptions = { - constants?: boolean - secret?: boolean - depth?: number - seen?: ReadonlySet -} -function readValue( +type ValueOptions = { constants?: boolean; secret?: boolean; depth?: number; seen?: ReadonlySet } +function rd( expr: ts.Expression, file: FileContext, ctx: CollectionState, opts: ValueOptions = {}, ): IndexValue { const depth = opts.depth ?? 0 - if (depth >= MAX_VALUE_DEPTH) return unknown('unsupported') + if (depth >= VDEP) return uk('unsupported') const seen = new Set(opts.seen ?? []) - const node = unwrap(expr) - if (seen.has(node)) return unknown('ambiguous') + const node = uw(expr) + if (seen.has(node)) return uk('ambiguous') seen.add(node) const nested = (value: ts.Expression, extra: Partial = {}): IndexValue => - readValue(value, file, ctx, { + rd(value, file, ctx, { ...opts, ...extra, depth: depth + 1, seen, }) - if (ts.isStringLiteralLike(node)) return literalString(node.text, opts.secret) + if (ts.isStringLiteralLike(node)) return ls(node.text, opts.secret) if (isNumeric(node)) { const value = Number(node.text) return Number.isFinite(value) && !Object.is(value, -0) ? { kind: 'literal', value } - : unknown('unsupported') + : uk('unsupported') } if (node.kind === K.TrueKeyword) return { kind: 'literal', value: true } if (node.kind === K.FalseKeyword) return { kind: 'literal', value: false } @@ -558,66 +553,67 @@ function readValue( } } if (isIdentifier(node)) { - const parameter = parameterValue(node, file, ctx) + const parameter = pv(node, file, ctx) if (parameter) return parameter - const symbol = symbolAt(node, ctx) + const symbol = sy(node, ctx) const decl = symbol?.valueDeclaration ?? symbol?.declarations?.find((candidate) => isVariable(candidate)) if ( opts.constants && decl && isVariable(decl) + && sb(decl, ctx) && decl.initializer ) { return nested(decl.initializer, { - secret: opts.secret || SECRET_NAME.test(node.text), + secret: opts.secret || SNM.test(node.text), }) } const indexed = decl - ? declSymbol(decl, file, ctx) - : symbolFor(node, file, ctx) - return indexed ? { kind: 'symbol', symbol_id: indexed.id } : unknown() + ? ds(decl, file, ctx) + : sfor(node, file, ctx) + return indexed ? { kind: 'symbol', symbol_id: indexed.id } : uk() } if (ts.isArrayLiteralExpression(node)) { - if (node.elements.length > MAX_VALUE_ELEMENTS) return unknown('unsupported') + if (node.elements.length > VELE) return uk('unsupported') const elements: IndexValue[] = [] for (const element of node.elements) { - if (ts.isSpreadElement(element) || ts.isOmittedExpression(element)) return unknown('unsupported') + if (ts.isSpreadElement(element) || ts.isOmittedExpression(element)) return uk('unsupported') elements.push(nested(element, { constants: true })) } return { kind: 'array', elements } } if (ts.isObjectLiteralExpression(node)) { - if (node.properties.length > MAX_VALUE_ELEMENTS) return unknown('unsupported') + if (node.properties.length > VELE) return uk('unsupported') const entries = new Map() for (const property of node.properties) { if (ts.isPropertyAssignment(property)) { - const key = propertyName(property.name) + const key = pn(property.name) if (key === null || key.includes('\0') - || Buffer.byteLength(key, 'utf8') > MAX_STRING_BYTES) { - return unknown('unsupported') + || Buffer.byteLength(key, 'utf8') > SBYT) { + return uk('unsupported') } entries.set( key, nested(property.initializer, { constants: true, - secret: SECRET_NAME.test(key), + secret: SNM.test(key), }), ) } else if (ts.isShorthandPropertyAssignment(property)) { const key = property.name.text - if (Buffer.byteLength(key, 'utf8') > MAX_STRING_BYTES) { - return unknown('unsupported') + if (Buffer.byteLength(key, 'utf8') > SBYT) { + return uk('unsupported') } entries.set( key, nested(property.name, { constants: true, - secret: SECRET_NAME.test(key), + secret: SNM.test(key), }), ) } else { - return unknown('unsupported') + return uk('unsupported') } } return { @@ -625,15 +621,15 @@ function readValue( entries: [...entries].map(([key, value]) => ({ key, value })), } } - if (ts.isNoSubstitutionTemplateLiteral(node)) return literalString(node.text, opts.secret) + if (ts.isNoSubstitutionTemplateLiteral(node)) return ls(node.text, opts.secret) if (ts.isTemplateExpression(node)) { - if (1 + (2 * node.templateSpans.length) > MAX_VALUE_ELEMENTS) { - return unknown('unsupported') + if (1 + (2 * node.templateSpans.length) > VELE) { + return uk('unsupported') } - const parts: IndexValue[] = [literalString(node.head.text, opts.secret)] + const parts: IndexValue[] = [ls(node.head.text, opts.secret)] for (const span of node.templateSpans) { parts.push(nested(span.expression, { constants: true })) - parts.push(literalString(span.literal.text, opts.secret)) + parts.push(ls(span.literal.text, opts.secret)) } return { kind: 'template', parts } } @@ -642,10 +638,10 @@ function readValue( const receiver = node.expression.expression if (method === 'slice') { const value = nested(receiver, { constants: true }) - if (value.kind !== 'array') return unknown() - const start = numericIndex(node.arguments[0], file, ctx) - const end = numericIndex(node.arguments[1], file, ctx) - if (start === null || (node.arguments[1] && end === null)) return unknown() + if (value.kind !== 'array') return uk() + const start = ni(node.arguments[0], file, ctx) + const end = ni(node.arguments[1], file, ctx) + if (start === null || (node.arguments[1] && end === null)) return uk() return { kind: 'array', elements: value.elements.slice(start, end ?? undefined) } } if (method === 'map') { @@ -655,34 +651,34 @@ function readValue( if (ts.isElementAccessExpression(node)) { const collection = nested(node.expression, { constants: true }) const index = node.argumentExpression - ? numericIndex(node.argumentExpression, file, ctx) + ? ni(node.argumentExpression, file, ctx) : null if (collection.kind === 'array' && index !== null) { - return collection.elements[index] ?? unknown() + return collection.elements[index] ?? uk() } } - const target = symbolFor(node, file, ctx) - return target ? { kind: 'symbol', symbol_id: target.id } : unknown() + const target = sfor(node, file, ctx) + return target ? { kind: 'symbol', symbol_id: target.id } : uk() } -function propertyName(name: ts.PropertyName): string | null { +function pn(name: ts.PropertyName): string | null { if (isIdentifier(name) || ts.isStringLiteralLike(name) || isNumeric(name)) { return name.text } return null } -function numericIndex(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): number | null { +function ni(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): number | null { if (!expr) return 0; - const value = readValue(expr, file, ctx, { constants: true }); + const value = rd(expr, file, ctx, { constants: true }); return value.kind === 'literal' && typeof value.value === 'number' && Number.isSafeInteger(value.value) ? value.value : null; } -function staticString(value: IndexValue): string | null { +function ss(value: IndexValue): string | null { if (value.kind === 'literal' && typeof value.value === 'string') { - return value.value.length > 0 && Buffer.byteLength(value.value, 'utf8') <= MAX_TEXT_BYTES + return value.value.length > 0 && Buffer.byteLength(value.value, 'utf8') <= TBYT ? value.value : null } @@ -693,80 +689,89 @@ function staticString(value: IndexValue): string | null { || !['string', 'number', 'boolean'].includes(typeof part.value)) return null result += String(part.value) } - return result.length > 0 && Buffer.byteLength(result, 'utf8') <= MAX_TEXT_BYTES + return result.length > 0 && Buffer.byteLength(result, 'utf8') <= TBYT ? result : null } -function symbolId(value: IndexValue): string | null { +function si(value: IndexValue): string | null { return value.kind === 'symbol' ? value.symbol_id : null } -function mapValue(value: IndexValue, depth: number, resolve?: (position: number) => IndexValue): IndexValue { +function mv(value: IndexValue, depth: number, resolve?: (position: number) => IndexValue): IndexValue { if (value.kind === 'parameter' && value.scope !== 'iteration' && resolve) { - return mapValue(resolve(value.position), depth); + return mv(resolve(value.position), depth); } if (value.kind === 'array') { - if (depth >= MAX_VALUE_DEPTH && value.elements.length > 0) - return unknown('unsupported'); + if (depth >= VDEP && value.elements.length > 0) + return uk('unsupported'); return { kind: 'array', - elements: value.elements.map((entry) => mapValue(entry, depth + 1, resolve)), + elements: value.elements.map((entry) => mv(entry, depth + 1, resolve)), }; } if (value.kind === 'object') { - if (depth >= MAX_VALUE_DEPTH && value.entries.length > 0) - return unknown('unsupported'); + if (depth >= VDEP && value.entries.length > 0) + return uk('unsupported'); return { kind: 'object', entries: value.entries.map((entry) => ({ key: entry.key, - value: mapValue(entry.value, depth + 1, resolve), + value: mv(entry.value, depth + 1, resolve), })), }; } if (value.kind === 'template') { - if (depth >= MAX_VALUE_DEPTH && value.parts.length > 0) - return unknown('unsupported'); + if (depth >= VDEP && value.parts.length > 0) + return uk('unsupported'); return { kind: 'template', - parts: value.parts.map((entry) => mapValue(entry, depth + 1, resolve)), + parts: value.parts.map((entry) => mv(entry, depth + 1, resolve)), }; } return value; } -function substitute(value: IndexValue, args: readonly IndexValue[]): IndexValue { - return mapValue(value, 0, (position) => args[position] ?? unknown()) +function sub(value: IndexValue, args: readonly IndexValue[]): IndexValue { + return mv(value, 0, (position) => args[position] ?? uk()) } -function instantiateEffect(fx: ExecutionEffect, args: readonly IndexValue[], witness: EffectWitness): ExecutionEffect { +function ie(fx: ExecutionEffect, args: readonly IndexValue[], witness: EffectWitness): ExecutionEffect { switch (fx[0]) { case 'bull-publish': case 'bull-consume': return [ - fx[0], substitute(fx[1], args), substitute(fx[2], args), + fx[0], sub(fx[1], args), sub(fx[2], args), fx[3], undefined, witness, fx[6], 'wrapper-summary', ]; case 'event-publish': return [ - fx[0], substitute(fx[1], args), undefined, fx[3], fx[4], + fx[0], sub(fx[1], args), undefined, fx[3], fx[4], witness, fx[6], 'wrapper-summary', ]; case 'event-consume': return [ - fx[0], substitute(fx[1], args), substitute(fx[2]!, args), + fx[0], sub(fx[1], args), sub(fx[2]!, args), fx[3], fx[4], witness, fx[6], 'wrapper-summary', ]; case 'persistence': return [ - fx[0], fx[1], fx[2] ? substitute(fx[2], args) : undefined, + fx[0], fx[1], fx[2] ? sub(fx[2], args) : undefined, fx[3], undefined, witness, fx[6], 'wrapper-summary', ]; } } -function callName(call: ts.CallExpression | ts.NewExpression): string { +function cn(call: ts.CallExpression | ts.NewExpression): string { const sf = call.getSourceFile() - const text = structuralText(call.expression, sf) - return boundedText(isNew(call) ? `new ${text}` : text) + const text = st(call.expression, sf) + return bd(isNew(call) ? `new ${text}` : text) } -function callScheduling(call: ts.CallExpression | ts.NewExpression): IndexCallFact['scheduling'] { +function th(call: ts.CallExpression | ts.NewExpression, ctx: CollectionState): boolean { + try { + const signature = ctx.input.checker.getResolvedSignature(call) + const type = signature && ctx.input.checker.getReturnTypeOfSignature(signature) + const then = type?.getProperty('then') + return !!then + && ctx.input.checker.getTypeOfSymbolAtLocation(then, call).getCallSignatures().length > 0 + } catch { return false } +} +function sch(call: ts.CallExpression | ts.NewExpression, ctx: CollectionState): IndexCallFact['scheduling'] { let current: ts.Node = call; while (ts.isParenthesizedExpression(current.parent) || ts.isAsExpression(current.parent) @@ -775,53 +780,55 @@ function callScheduling(call: ts.CallExpression | ts.NewExpression): IndexCallFa } if (ts.isAwaitExpression(current.parent)) return 'awaited'; - if (ts.isVoidExpression(current.parent) - || ts.isExpressionStatement(current.parent)) { + if ((ts.isVoidExpression(current.parent) + || ts.isExpressionStatement(current.parent)) && th(call, ctx)) { return 'fire_and_forget'; } return 'sync'; } -function callFact( - call: ts.CallExpression | ts.NewExpression, - sym: IndexSymbol, - file: FileContext, - ctx: CollectionState, +function iar(expr: ts.Expression, ctx: CollectionState): boolean { + try { + const type = ctx.input.checker.getTypeAtLocation(expr) + return ctx.input.checker.isArrayType(type) + || ctx.input.checker.isTupleType(type) + } catch { return false } +} +function cf( + call: ts.CallExpression | ts.NewExpression, sym: IndexSymbol, + file: FileContext, ctx: CollectionState, control: readonly IndexControlFrame[], ): IndexCallFact { - const target = callSymbol(call, file, ctx) + const target = cs(call, file, ctx) const args = (call.arguments ?? []).map((argument) => { - const value = unwrap(argument) + const value = uw(argument) return isArrow(value) || isFunction(value) - ? handlerValue(value, file, ctx) - : readValue(value, file, ctx, { + ? hv(value, file, ctx) + : rd(value, file, ctx, { constants: true, - secret: SECRET_NAME.test(value.getText(file.sf)), + secret: SNM.test(value.getText(file.sf)), }) }) const fact: IndexCallFact = { - ...factBase(sym.id, 'call', call, file, control, { + ...fb(sym.id, 'call', call, file, control, { confidence: target ? 'high' : 'medium', source: target ? 'typescript-semantic' : 'typescript-syntactic', }), kind: 'call', - callee: callName(call), + callee: cn(call), ...(target ? { target_symbol_id: target.id } : {}), arguments: args, - scheduling: callScheduling(call), + scheduling: sch(call, ctx), } - addFact(ctx, fact) + af(ctx, fact) ctx.callIds.set(call, fact.id) if (target) { - addCall(ctx, sym.id, [target.id, fact.arguments, call]) + al(ctx, sym.id, [target.id, fact.arguments, call]) } return fact } -function receiverType( - expr: ts.Expression, - ctx: CollectionState, -): string { +function rty(expr: ts.Expression, ctx: CollectionState): string { try { - return boundedText( + return bd( ctx.input.checker.typeToString( ctx.input.checker.getTypeAtLocation(expr), undefined, @@ -832,8 +839,8 @@ function receiverType( return '' } } -function typeImport(expr: ts.Expression, file: FileContext, ctx: CollectionState): ImportBinding | null { - const symbol = symbolAt(expr, ctx); +function ti(expr: ts.Expression, file: FileContext, ctx: CollectionState): ImportBinding | null { + const symbol = sy(expr, ctx); for (const decl of symbol?.declarations ?? []) { const typeNode = isParameter(decl) || isPropertyDecl(decl) @@ -853,79 +860,145 @@ function typeImport(expr: ts.Expression, file: FileContext, ctx: CollectionState } return null; } -function bullTransport(binding: ImportBinding | null): QueueTransport | null { +function bt(binding: ImportBinding | null): QueueTransport | null { if (!binding || !['bull', 'bullmq'].includes(binding.module) || !['Queue', 'default'].includes(binding.imported)) return null return binding.module as QueueTransport } -function queueTypeTransport( - node: ts.TypeNode | undefined, - file: FileContext, -): QueueTransport | null { - if (!node || !isTypeReference(node)) return null - const name = node.typeName - if (isIdentifier(name)) return bullTransport(file.imports.get(name.text) ?? null) - if (!isIdentifier(name.left) || name.right.text !== 'Queue') return null - const binding = file.imports.get(name.left.text) - return binding?.namespace ? bullTransport({ ...binding, imported: 'Queue' }) : null -} -function mapQueueTransport(expr: ts.Expression, file: FileContext, ctx: CollectionState): QueueTransport | null { - const target = isAccess(expr) - ? expr.name - : expr; - const symbol = symbolAt(target, ctx); - const decl = symbol?.valueDeclaration; - const declFile = decl - ? contextFor(decl.getSourceFile(), ctx) - : null; - if (!decl || !declFile) - return null; - const type = (isVariable(decl) - || isPropertyDecl(decl) - || isParameter(decl)) - ? decl.type - : undefined; - const init = (isVariable(decl) - || isPropertyDecl(decl)) - ? decl.initializer - : undefined; - const mapType = type && isTypeReference(type) - ? type.typeArguments?.[1] - : init && isNew(unwrap(init)) - ? (unwrap(init) as ts.NewExpression).typeArguments?.[1] - : undefined; - return queueTypeTransport(mapType, declFile); -} -function contextFor(sf: ts.SourceFile, ctx: CollectionState): FileContext | null { +function cx(sf: ts.SourceFile, ctx: CollectionState): FileContext | null { return ctx.files.get(sf) ?? null; } -function queueOrigin(expr: ts.Expression, sym: IndexSymbol, file: FileContext, ctx: CollectionState, seen: ReadonlySet = new Set()): QueueOrigin | null { - const node = unwrap(expr); +function xs(expr: ts.Expression, ctx: CollectionState): ts.Symbol | undefined { + const node = uw(expr) + return sy(isAccess(node) ? node.name : node, ctx) +} +function eq(left: ts.Expression, right: ts.Expression, ctx: CollectionState): boolean { + const a = uw(left), b = uw(right) + if (ts.isStringLiteralLike(a) && ts.isStringLiteralLike(b)) { + return a.text === b.text + } + if (isIdentifier(a) && isIdentifier(b)) return sy(a, ctx) === sy(b, ctx) + if (isAccess(a) && isAccess(b)) { + return a.name.text === b.name.text + && xs(a.name, ctx) === xs(b.name, ctx) + && (a.expression.kind === K.ThisKeyword && b.expression.kind === K.ThisKeyword + || eq(a.expression, b.expression, ctx)) + } + return false +} +function qc(expr: ts.Expression, ctx: CollectionState, seen: ReadonlySet = new Set()): readonly [ts.Expression, FileContext, QueueTransport] | null { + const node = uw(expr) + if (seen.has(node)) return null + const next = new Set(seen).add(node) + const file = cx(node.getSourceFile(), ctx) + if (!file) return null + if (isNew(node) && node.arguments?.[0]) { + const transport = bt(ib(node.expression, file)) + return transport ? [node.arguments[0], file, transport] : null + } + if (!isIdentifier(node)) return null + const decl = sy(node, ctx)?.valueDeclaration + return decl && isVariable(decl) && sb(decl, ctx) && decl.initializer + ? qc(decl.initializer, ctx, next) : null +} +function prep(ctx: CollectionState): void { + const sets: Array = [] + const mark = (node: ts.Node): void => { + if (isIdentifier(node)) { + const symbol = sy(node, ctx) + if (symbol) ctx.unstable.add(symbol) + } + ts.forEachChild(node, mark) + } + for (const sf of ctx.input.sourceFiles) { + const visit = (node: ts.Node): void => { + if (isBinary(node) && AOP.has(node.operatorToken.kind)) { + mark(node.left) + } else if ((ts.isPrefixUnaryExpression(node) || ts.isPostfixUnaryExpression(node)) + && [K.PlusPlusToken, K.MinusMinusToken].includes(node.operator)) { + mark(node.operand) + } else if (ts.isDeleteExpression(node)) { + mark(node.expression) + } + if (isCall(node) && isAccess(node.expression) + && node.expression.name.text === 'set' + && node.arguments[0] && node.arguments[1]) { + const symbol = xs(node.expression.expression, ctx) + if (symbol) sets.push([symbol, node.arguments[0], node.arguments[1]]) + } + ts.forEachChild(node, visit) + } + visit(sf) + } + for (const [symbol, key, value] of sets) { + const entries = ctx.mapQueues.get(symbol) ?? [] + const queue = qc(value, ctx) + const file = cx(key.getSourceFile(), ctx) + entries.push(queue && file ? [key, file, queue[0], queue[2]] : null) + ctx.mapQueues.set(symbol, entries) + } +} +function sm(decl: ts.Declaration, ctx: CollectionState): boolean { + try { + const symbol = ctx.input.checker.getTypeAtLocation(decl).getSymbol() + return symbol?.name === 'Map' + && !!symbol.declarations?.some((item) => + /\/typescript\/lib\/lib\..+\.d\.ts$/u.test( + item.getSourceFile().fileName.replaceAll('\\', '/'), + )) + } catch { return false } +} +function mq(map: ts.Expression, key: ts.Expression, file: FileContext, ctx: CollectionState): QueueOrigin | null { + const mapSymbol = xs(map, ctx) + const decl = mapSymbol?.valueDeclaration + if (!mapSymbol || !decl + || !(isVariable(decl) || isPropertyDecl(decl)) + || !sb(decl, ctx) || !sm(decl, ctx)) return null + const entries = ctx.mapQueues.get(mapSymbol) ?? [] + if (entries.length === 0 || entries.some((entry) => !entry)) return null + const proven = entries as MapQueueEntry[] + const lookup = ss(rd(key, file, ctx, { constants: true })) + if (lookup) { + const matches = proven.filter(([entryKey, entryFile]) => + ss(rd(entryKey, entryFile, ctx, { constants: true })) === lookup) + if (matches.length) { + const origins = matches.map(([, entryFile, queueKey, transport]) => + [rd(queueKey, entryFile, ctx, { constants: true }), transport] as const) + const first = JSON.stringify(origins[0]) + return origins.every((origin) => JSON.stringify(origin) === first) + ? origins[0]! : null + } + } + const transport = proven[0]![3] + return proven.every(([entryKey, , queueKey, entryTransport]) => + entryTransport === transport && eq(entryKey, queueKey, ctx)) + ? [rd(key, file, ctx, { constants: true }), transport] : null +} +function qo(expr: ts.Expression, sym: IndexSymbol, file: FileContext, ctx: CollectionState, seen: ReadonlySet = new Set()): QueueOrigin | null { + const node = uw(expr); if (seen.has(node)) return null; const nextSeen = new Set(seen); nextSeen.add(node); if (isNew(node) && node.arguments?.[0]) { - const transport = bullTransport(importedBinding(node.expression, file)); + const transport = bt(ib(node.expression, file)); if (transport) { return [ - readValue(node.arguments[0], file, ctx, { constants: true }), + rd(node.arguments[0], file, ctx, { constants: true }), transport, ]; } } if (isIdentifier(node)) { - const local = ctx.queues.get(sym.id)?.get(node.text); - if (local) - return local; - const symbol = symbolAt(node, ctx); + const symbol = sy(node, ctx); const decl = symbol?.valueDeclaration; if (decl && isVariable(decl) + && sb(decl, ctx) && decl.initializer) { - const declFile = contextFor(decl.getSourceFile(), ctx); + const declFile = cx(decl.getSourceFile(), ctx); return declFile - ? queueOrigin(decl.initializer, sym, declFile, ctx, nextSeen) + ? qo(decl.initializer, sym, declFile, ctx, nextSeen) : null; } } @@ -938,13 +1011,14 @@ function queueOrigin(expr: ts.Expression, sym: IndexSymbol, file: FileContext, c .get(`${sym.file_id}\0${className}`)?.get(node.name.text); if (injected) return injected; - const symbol = symbolAt(node.name, ctx); + const symbol = sy(node.name, ctx); const decl = symbol?.valueDeclaration; if (decl && isPropertyDecl(decl) + && sb(decl, ctx) && decl.initializer) { - const declFile = contextFor(decl.getSourceFile(), ctx); + const declFile = cx(decl.getSourceFile(), ctx); if (declFile) { - return queueOrigin(decl.initializer, sym, declFile, ctx, nextSeen); + return qo(decl.initializer, sym, declFile, ctx, nextSeen); } } } @@ -952,46 +1026,41 @@ function queueOrigin(expr: ts.Expression, sym: IndexSymbol, file: FileContext, c && isAccess(node.expression) && node.expression.name.text === 'get' && node.arguments[0]) { - const transport = mapQueueTransport(node.expression.expression, file, ctx); - if (transport) { - return [ - readValue(node.arguments[0], file, ctx, { constants: true }), - transport, - ]; - } + return mq( + node.expression.expression, node.arguments[0], file, ctx, + ) } return null; } -function emitterScope(expr: ts.Expression, sym: IndexSymbol, file: FileContext, ctx: CollectionState, seen: ReadonlySet = new Set()): EmitterScope | null { - const node = unwrap(expr); +function es(expr: ts.Expression, sym: IndexSymbol, file: FileContext, ctx: CollectionState, seen: ReadonlySet = new Set()): EmitterScope | null { + const node = uw(expr); if (seen.has(node)) return null; const nextSeen = new Set(seen); nextSeen.add(node); if (isIdentifier(node)) { - const symbol = symbolAt(node, ctx); + const symbol = sy(node, ctx); const decl = symbol?.valueDeclaration; if (decl) { - const indexed = declSymbol(decl, file, ctx); - if (indexed) { - const cached = ctx.emitters.get(indexed.id); - if (cached) - return cached; - } + const declFile = cx(decl.getSourceFile(), ctx); + const identity = declFile + ? `${declFile.fileId}:${decl.getStart(declFile.sf, false)}` + : null; + const cached = identity ? ctx.emitters.get(identity) : undefined; + if (cached) + return cached; if (isVariable(decl) + && sb(decl, ctx) && decl.initializer - && isNew(unwrap(decl.initializer))) { - const init = unwrap(decl.initializer) as ts.NewExpression; - const declFile = contextFor(decl.getSourceFile(), ctx); + && isNew(uw(decl.initializer))) { + const init = uw(decl.initializer) as ts.NewExpression; if (!declFile) return null; - const transport = emitterTransport(init.expression, declFile); + const transport = et(init.expression, declFile); if (transport) { - const scope = indexed?.id - ?? `${sym.id}:local:${node.text}:${decl.getStart(declFile.sf, false)}`; + const scope = identity!; const value: EmitterScope = [scope, transport]; - if (indexed) - ctx.emitters.set(indexed.id, value); + ctx.emitters.set(scope, value); return value; } } @@ -999,18 +1068,19 @@ function emitterScope(expr: ts.Expression, sym: IndexSymbol, file: FileContext, } if (isAccess(node) && node.expression.kind === K.ThisKeyword) { - const binding = typeImport(node, file, ctx); + const binding = ti(node, file, ctx); const transport = binding?.module === '@nestjs/event-emitter' ? 'nestjs-event-emitter' : binding?.module === 'node:events' || binding?.module === 'events' ? 'node-event-emitter' : null; - const decl = symbolAt(node.name, ctx)?.valueDeclaration; + const decl = sy(node.name, ctx)?.valueDeclaration; const inferred = decl && isPropertyDecl(decl) + && sb(decl, ctx) && decl.initializer - && isNew(unwrap(decl.initializer)) - ? emitterTransport((unwrap(decl.initializer) as ts.NewExpression).expression, contextFor(decl.getSourceFile(), ctx) ?? file) + && isNew(uw(decl.initializer)) + ? et((uw(decl.initializer) as ts.NewExpression).expression, cx(decl.getSourceFile(), ctx) ?? file) : null; if (transport || inferred) { const className = sym.kind === 'method' @@ -1024,24 +1094,24 @@ function emitterScope(expr: ts.Expression, sym: IndexSymbol, file: FileContext, } return null; } -function emitterTransport(expr: ts.Expression, file: FileContext): 'node-event-emitter' | 'nestjs-event-emitter' | null { - if (isImport(expr, file, ['node:events', 'events'], ['EventEmitter'])) { +function et(expr: ts.Expression, file: FileContext): 'node-event-emitter' | 'nestjs-event-emitter' | null { + if (ii(expr, file, ['node:events', 'events'], ['EventEmitter'])) { return 'node-event-emitter'; } - if (isImport(expr, file, ['@nestjs/event-emitter'], ['EventEmitter2'])) { + if (ii(expr, file, ['@nestjs/event-emitter'], ['EventEmitter2'])) { return 'nestjs-event-emitter'; } return null; } -function handlerValue(expr: ts.Expression, file: FileContext, ctx: CollectionState): IndexValue { - const node = unwrap(expr); +function hv(expr: ts.Expression, file: FileContext, ctx: CollectionState): IndexValue { + const node = uw(expr); if (isArrow(node) || isFunction(node)) { const targets = new Set(); const visit = (child: ts.Node): void => { if (child !== node && (isArrow(child) || isFunction(child))) return; if (isCall(child)) { - const target = callSymbol(child, file, ctx); + const target = cs(child, file, ctx); if (target) targets.add(target.id); } @@ -1050,85 +1120,77 @@ function handlerValue(expr: ts.Expression, file: FileContext, ctx: CollectionSta visit(node.body); return targets.size === 1 ? { kind: 'symbol', symbol_id: [...targets][0]! } - : unknown(targets.size > 1 ? 'ambiguous' : 'dynamic'); - } - return readValue(node, file, ctx, { constants: false }); -} -type PersistenceSummary = readonly [ - operation: IndexPersistenceOperation, - resource: IndexValue | undefined, - receiverType: string, -] -function storageEffect( - operation: IndexPersistenceOperation | null, - receiverType: string, - resource: ts.Expression | undefined, - file: FileContext, - ctx: CollectionState, + : uk(targets.size > 1 ? 'ambiguous' : 'dynamic'); + } + return rd(node, file, ctx, { constants: false }); +} +type PersistenceSummary = readonly [operation: IndexPersistenceOperation, resource: IndexValue | undefined, receiverType: string] +function se( + operation: IndexPersistenceOperation | null, receiverType: string, + resource: ts.Expression | undefined, file: FileContext, ctx: CollectionState, ): PersistenceSummary | null { return operation ? [ operation, - resource ? readValue(resource, file, ctx, { constants: true }) : undefined, + resource ? rd(resource, file, ctx, { constants: true }) : undefined, receiverType, ] : null } -function persistenceOperation(call: ts.CallExpression, file: FileContext, ctx: CollectionState): PersistenceSummary | null { +function po(call: ts.CallExpression, file: FileContext, ctx: CollectionState): PersistenceSummary | null { const expr = call.expression; if (isIdentifier(expr)) { const binding = file.imports.get(expr.text); - if (binding && FS_MODULES.has(binding.module)) { - return storageEffect(fsOperation(binding.imported), `${binding.module}:${binding.imported}`, call.arguments[0], file, ctx); + if (binding && FSM.has(binding.module)) { + return se(fsop(binding.imported, call.arguments[1], file, ctx), `${binding.module}:${binding.imported}`, call.arguments[0], file, ctx); } } if (!isAccess(expr)) return null; const method = expr.name.text; const receiver = expr.expression; - const root = leftmostIdentifier(receiver); + const root = li(receiver); const namespace = root ? file.imports.get(root.text) : undefined; - if (namespace?.namespace && FS_MODULES.has(namespace.module)) { - const summary = storageEffect(fsOperation(method), `${namespace.module}:namespace`, call.arguments[0], file, ctx); + if (namespace?.namespace && FSM.has(namespace.module)) { + const summary = se(fsop(method, call.arguments[1], file, ctx), `${namespace.module}:namespace`, call.arguments[0], file, ctx); if (summary) return summary; } - const type = receiverType(receiver, ctx); - const typeBinding = typeImport(receiver, file, ctx); + const type = rty(receiver, ctx); + const typeBinding = ti(receiver, file, ctx); if (typeBinding?.module === 'typeorm' - && ['Repository', 'MongoRepository'].includes(typeBinding.imported) - || packageDecl(receiver, ctx, '/node_modules/typeorm/')) { - const summary = storageEffect(repoOperation(method), type || `${typeBinding?.module}:${typeBinding?.imported}`, call.arguments[0], file, ctx); + && ['Repository', 'MongoRepository'].includes(typeBinding.imported)) { + const summary = se(typeormOperation(method), type || `${typeBinding.module}:${typeBinding.imported}`, call.arguments[0], file, ctx); if (summary) return summary; } - if (packageDecl(receiver, ctx, '/node_modules/@prisma/client/', '/node_modules/.prisma/client/')) { - const summary = storageEffect(repoOperation(method), type || 'PrismaClient', call.arguments[0], file, ctx); + if (pd(receiver, ctx, '/node_modules/@prisma/client/', '/node_modules/.prisma/client/')) { + const summary = se(prismaOperation(method), type || 'PrismaClient', call.arguments[0], file, ctx); if (summary) return summary; } if (method !== 'send' || !(typeBinding?.module === '@aws-sdk/client-s3' && typeBinding.imported === 'S3Client' - || packageDecl(receiver, ctx, '/node_modules/@aws-sdk/client-s3/'))) + || pd(receiver, ctx, '/node_modules/@aws-sdk/client-s3/'))) return null; const command = call.arguments[0]; - if (command && isNew(unwrap(command))) { - const constructor = unwrap(command) as ts.NewExpression; - const binding = importedBinding(constructor.expression, file); + if (command && isNew(uw(command))) { + const constructor = uw(command) as ts.NewExpression; + const binding = ib(constructor.expression, file); if (binding?.module === '@aws-sdk/client-s3') { const writes = ['PutObjectCommand', 'UploadPartCommand', 'CompleteMultipartUploadCommand']; const reads = ['GetObjectCommand', 'HeadObjectCommand']; - return storageEffect(writes.includes(binding.imported) ? 'object_write' + return se(writes.includes(binding.imported) ? 'object_write' : reads.includes(binding.imported) ? 'object_read' : null, type, constructor.arguments?.[0], file, ctx); } } return null; } -function leftmostIdentifier(expr: ts.Expression): ts.Identifier | null { +function li(expr: ts.Expression): ts.Identifier | null { let current = expr while (isAccess(current)) current = current.expression return isIdentifier(current) ? current : null } -function packageDecl(expr: ts.Expression, ctx: CollectionState, ...packagePaths: readonly string[]): boolean { +function pd(expr: ts.Expression, ctx: CollectionState, ...packagePaths: readonly string[]): boolean { const checker = ctx.input.checker; let current: ts.Expression = expr; while (true) { @@ -1150,21 +1212,33 @@ function packageDecl(expr: ts.Expression, ctx: CollectionState, ...packagePaths: current = current.expression; } } -function namedOperation(method: string, operations: Readonly>): IndexPersistenceOperation | null { +function no(method: string, operations: Readonly>): IndexPersistenceOperation | null { return Object.hasOwn(operations, method) ? operations[method]! : null; } -const fsOperation = (method: string): IndexPersistenceOperation | null => namedOperation(method, FS_OPERATIONS); -const repoOperation = (method: string): IndexPersistenceOperation | null => namedOperation(method, REPOSITORY_OPERATIONS); -function recordEffects(call: ts.CallExpression | ts.NewExpression, sym: IndexSymbol, file: FileContext, ctx: CollectionState): void { +function fsop( + method: string, flags: ts.Expression | undefined, file: FileContext, ctx: CollectionState, +): IndexPersistenceOperation | null { + if (!['open', 'openSync'].includes(method)) return no(method, FSO) + if (!flags) return null + const value = ss(rd(flags, file, ctx, { constants: true })) + if (!value) return null + if (value.includes('+') || /^[aw]/u.test(value)) return 'file_write' + return /^r(?:s|sr)?$/u.test(value) ? 'file_read' : null +} +const typeormOperation = (method: string): IndexPersistenceOperation | null => + no(method, TOO) +const prismaOperation = (method: string): IndexPersistenceOperation | null => + no(method, PRO) +function re(call: ts.CallExpression | ts.NewExpression, sym: IndexSymbol, file: FileContext, ctx: CollectionState): void { const proof = [call, 'high', 'framework'] as const; if (isNew(call)) { - if (isImport(call.expression, file, ['bullmq'], ['Worker']) + if (ii(call.expression, file, ['bullmq'], ['Worker']) && call.arguments?.[0] && call.arguments[1]) { - addEffect(ctx, sym.id, [ + ae(ctx, sym.id, [ 'bull-consume', - readValue(call.arguments[0], file, ctx, { constants: true }), - handlerValue(call.arguments[1], file, ctx), + rd(call.arguments[0], file, ctx, { constants: true }), + hv(call.arguments[1], file, ctx), 'bullmq', undefined, ...proof, ]); } @@ -1174,20 +1248,20 @@ function recordEffects(call: ts.CallExpression | ts.NewExpression, sym: IndexSym const method = call.expression.name.text; const receiver = call.expression.expression; if (method === 'add' && call.arguments[0]) { - const queue = queueOrigin(receiver, sym, file, ctx); + const queue = qo(receiver, sym, file, ctx); if (queue) { - addEffect(ctx, sym.id, [ + ae(ctx, sym.id, [ 'bull-publish', queue[0], - readValue(call.arguments[0], file, ctx, { constants: true }), + rd(call.arguments[0], file, ctx, { constants: true }), queue[1], undefined, ...proof, ]); } } - const emitter = emitterScope(receiver, sym, file, ctx); + const emitter = es(receiver, sym, file, ctx); if (emitter && method === 'emit' && call.arguments[0]) { - addEffect(ctx, sym.id, [ + ae(ctx, sym.id, [ 'event-publish', - readValue(call.arguments[0], file, ctx, { constants: true }), + rd(call.arguments[0], file, ctx, { constants: true }), undefined, emitter[1], emitter[0], ...proof, ]); } @@ -1195,106 +1269,91 @@ function recordEffects(call: ts.CallExpression | ts.NewExpression, sym: IndexSym && ['addListener', 'on', 'once', 'prependListener'].includes(method) && call.arguments[0] && call.arguments[1]) { - addEffect(ctx, sym.id, [ + ae(ctx, sym.id, [ 'event-consume', - readValue(call.arguments[0], file, ctx, { constants: true }), - handlerValue(call.arguments[1], file, ctx), + rd(call.arguments[0], file, ctx, { constants: true }), + hv(call.arguments[1], file, ctx), emitter[1], emitter[0], ...proof, ]); } } - const persistence = persistenceOperation(call, file, ctx); + const persistence = po(call, file, ctx); if (persistence) { - addEffect(ctx, sym.id, ['persistence', ...persistence, undefined, ...proof]); + ae(ctx, sym.id, ['persistence', ...persistence, undefined, ...proof]); } } -function promiseCombinator(call: ts.CallExpression, ctx: CollectionState): { - combinator: 'all' | 'allSettled' | 'any' | 'race'; - completion: 'all_or_first_rejection' | 'all_settled' | 'first_fulfilled' | 'first_settled'; +function pc(call: ts.CallExpression, ctx: CollectionState): { + combinator: 'all' | 'allSettled' | 'any' | 'race' + completion: 'all_or_first_rejection' | 'all_settled' | 'first_fulfilled' | 'first_settled' } | null { if (!isAccess(call.expression) || !isIdentifier(call.expression.expression) || call.expression.expression.text !== 'Promise') { return null; } - const symbol = symbolAt(call.expression.expression, ctx); + const symbol = sy(call.expression.expression, ctx); if (!symbol?.declarations?.some((decl) => decl.getSourceFile().isDeclarationFile && /\/typescript\/lib\/lib\..+\.d\.ts$/u.test(decl.getSourceFile().fileName.replaceAll('\\', '/')))) return null; const combinator = call.expression.name.text; - return Object.hasOwn(PROMISE_COMPLETIONS, combinator) + return Object.hasOwn(PMC, combinator) ? { - combinator: combinator as keyof typeof PROMISE_COMPLETIONS, - completion: PROMISE_COMPLETIONS[combinator as keyof typeof PROMISE_COMPLETIONS], + combinator: combinator as keyof typeof PMC, + completion: PMC[combinator as keyof typeof PMC], } : null; } -function parallelSource(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): IndexValue | undefined { - if (!expr) - return undefined; - const node = unwrap(expr); - if (isCall(node) - && isAccess(node.expression) - && node.expression.name.text === 'map') { - return readValue(node.expression.expression, file, ctx, { constants: true }); - } - return readValue(node, file, ctx, { constants: true }); -} -function directParallelLanes(expr: ts.Expression | undefined): readonly ts.Expression[] { - if (!expr) - return []; - const node = unwrap(expr); - if (!ts.isArrayLiteralExpression(node)) - return []; - return node.elements.flatMap((element) => ts.isOmittedExpression(element) || ts.isSpreadElement(element) - ? [] - : [element]); -} -function mappedInput(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): { - call: ts.CallExpression; - input: IndexValue; - receiver: ts.Expression; +function pl(expr: ts.Expression | undefined): readonly ts.Expression[] | null { + if (!expr) return null + const node = uw(expr); + if (!ts.isArrayLiteralExpression(node) + || node.elements.length > VELE + || node.elements.some((element) => + ts.isOmittedExpression(element) || ts.isSpreadElement(element))) return null + return [...node.elements] as ts.Expression[] +} +function mi(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): { + call: ts.CallExpression; input: IndexValue; receiver: ts.Expression } | null { if (!expr) return null; - const node = unwrap(expr); + const node = uw(expr); if (!isCall(node) || !isAccess(node.expression) || node.expression.name.text !== 'map') return null; - const input = readValue(node.expression.expression, file, ctx, { constants: true }); + const input = rd(node.expression.expression, file, ctx, { constants: true }); return input.kind === 'array' ? { call: node, input, receiver: node.expression.expression } : null; } -function exits(stmt: ts.Statement): boolean { - if (ts.isReturnStatement(stmt) || ts.isThrowStatement(stmt)) return true +function ex(stmt: ts.Statement): boolean { + if (ts.isReturnStatement(stmt) || ts.isThrowStatement(stmt) + || ts.isBreakStatement(stmt) || ts.isContinueStatement(stmt)) return true if (ts.isBlock(stmt)) { const last = stmt.statements.at(-1) - return last ? exits(last) : false + return last ? ex(last) : false } if (isIf(stmt) && stmt.elseStatement) { - return exits(stmt.thenStatement) && exits(stmt.elseStatement) + return ex(stmt.thenStatement) && ex(stmt.elseStatement) } return false } -function guardContinuation( - stmt: ts.IfStatement, -): BranchArm | 'unreachable' | null { - const thenExits = exits(stmt.thenStatement) +function gc(stmt: ts.IfStatement): BranchArm | 'unreachable' | null { + const thenExits = ex(stmt.thenStatement) const elseExits = stmt.elseStatement - ? exits(stmt.elseStatement) + ? ex(stmt.elseStatement) : false if (thenExits && elseExits) return 'unreachable' if (thenExits) return 'else' return elseExits ? 'then' : null } -function topLevelValue(decl: ts.VariableDeclaration, sym: IndexSymbol, file: FileContext, ctx: CollectionState): IndexValue | null { +function tv(decl: ts.VariableDeclaration, sym: IndexSymbol, file: FileContext, ctx: CollectionState): IndexValue | null { if (!decl.initializer || decl.parent.parent.parent !== file.sf) return null; - const value = readValue(decl.initializer, file, ctx, { + const value = rd(decl.initializer, file, ctx, { constants: true, - secret: isIdentifier(decl.name) && SECRET_NAME.test(decl.name.text), + secret: isIdentifier(decl.name) && SNM.test(decl.name.text), }); if (value.kind === 'unknown' || value.kind === 'symbol' || value.kind === 'parameter') return null; @@ -1303,18 +1362,18 @@ function topLevelValue(decl: ts.VariableDeclaration, sym: IndexSymbol, file: Fil && value.value.length === 0) { return null; } - return sym.id === declSymbol(decl, file, ctx)?.id ? value : null; + return sym.id === ds(decl, file, ctx)?.id ? value : null; } -function collectFacts(file: FileContext, ctx: CollectionState): void { +function collect(file: FileContext, ctx: CollectionState): void { const visit = ( node: ts.Node, control: readonly IndexControlFrame[], executeCallable = false, ): void => { if (ts.isDecorator(node)) return - const sym = ownerFor(node, file) + const sym = ow(node, file) if (ts.isFunctionLike(node) - && callableOwner(node, file, ctx)?.id !== sym?.id + && ca(node, file, ctx)?.id !== sym?.id && !executeCallable) return if (sym && control.length > INDEX_BODY_FACT_CONTROL_LIMIT) { ctx.overflow.add(sym.id) @@ -1325,13 +1384,13 @@ function collectFacts(file: FileContext, ctx: CollectionState): void { for (const stmt of node.statements) { visit(stmt, nextControl) if (!isIf(stmt)) { - if (exits(stmt)) break + if (ex(stmt)) break continue } - const continuation = guardContinuation(stmt) + const continuation = gc(stmt) if (continuation === 'unreachable') break if (continuation) { - const base = factBase( + const base = fb( sym.id, 'condition', stmt.expression, @@ -1339,95 +1398,97 @@ function collectFacts(file: FileContext, ctx: CollectionState): void { nextControl, { statementNode: stmt }, ) - nextControl = branchFrame(nextControl, base.id, continuation) + nextControl = br(nextControl, base.id, continuation) } } return } if (sym && isVariable(node) && isIdentifier(node.name)) { - const value = topLevelValue(node, sym, file, ctx) + const value = tv(node, sym, file, ctx) if (value) { - addFact(ctx, { - ...factBase(sym.id, 'literal', node.initializer!, file, control, { - statementNode: statementFor(node), + af(ctx, { + ...fb(sym.id, 'literal', node.initializer!, file, control, { + statementNode: stmt(node), }), kind: 'literal', value, role: 'initializer', }) } - if (node.initializer) { - const queue = queueOrigin(node.initializer, sym, file, ctx) - if (queue) { - const origins = ctx.queues.get(sym.id) ?? new Map() - origins.set(node.name.text, queue) - ctx.queues.set(sym.id, origins) - } - } } if (sym && isIf(node)) { - const continuation = guardContinuation(node) - const base = addCondition( + const continuation = gc(node) + const base = ac( sym.id, continuation ? 'guard' : 'if', node.expression, file, ctx, control, node, ) visit(node.expression, control) - visit(node.thenStatement, branchFrame(control, base.id, 'then')) + visit(node.thenStatement, br(control, base.id, 'then')) if (node.elseStatement) { - visit(node.elseStatement, branchFrame(control, base.id, 'else')) + visit(node.elseStatement, br(control, base.id, 'else')) } return } if (sym && ts.isSwitchStatement(node)) { - const base = addCondition( + const base = ac( sym.id, 'switch', node.expression, file, ctx, control, node, ) visit(node.expression, control) for (const clause of node.caseBlock.clauses) { const arm = ts.isDefaultClause(clause) ? 'default' as const - : boundedText( - `case:${structuralText(clause.expression, file.sf)}`, - 96, - ).replaceAll('\0', '') as `case:${string}` + : `case:${hash(`${clause.expression.getText(file.sf)}:${clause.pos}`).slice(0, 16)}` as const if (ts.isCaseClause(clause)) visit(clause.expression, control) + let armControl = br(control, base.id, arm) for (const stmt of clause.statements) { - visit(stmt, branchFrame(control, base.id, arm)) + visit(stmt, armControl) + if (!isIf(stmt)) { + if (ex(stmt)) break + continue + } + const continuation = gc(stmt) + if (continuation === 'unreachable') break + if (continuation) armControl = br( + armControl, fb( + sym.id, 'condition', stmt.expression, file, armControl, + { statementNode: stmt }, + ).id, continuation, + ) } } return } if (sym && ts.isConditionalExpression(node)) { - const base = addCondition( - sym.id, 'ternary', node.condition, file, ctx, control, statementFor(node), + const base = ac( + sym.id, 'ternary', node.condition, file, ctx, control, stmt(node), ) visit(node.condition, control) - visit(node.whenTrue, branchFrame(control, base.id, 'truthy')) - visit(node.whenFalse, branchFrame(control, base.id, 'falsy')) + visit(node.whenTrue, br(control, base.id, 'truthy')) + visit(node.whenFalse, br(control, base.id, 'falsy')) return } - const logical = isBinary(node) ? LOGICAL_FLOW.get(node.operatorToken.kind) : undefined + const logical = isBinary(node) ? LFL.get(node.operatorToken.kind) : undefined if (sym && isBinary(node) && logical) { - const base = addCondition( - sym.id, logical[0], node.left, file, ctx, control, statementFor(node), + const base = ac( + sym.id, logical[0], node.left, file, ctx, control, stmt(node), ) visit(node.left, control) - visit(node.right, branchFrame(control, base.id, logical[1])) + visit(node.right, br(control, base.id, logical[1])) return } - const loop = loopDescriptor(node) + const loop = ld(node) if (sym && loop) { - const base = factBase(sym.id, 'loop', node, file, control) + const base = fb(sym.id, 'loop', node, file, control) const repeatedControl = [...control, { kind: 'loop' as const, controller_fact_id: base.id, }] - addFact(ctx, { + af(ctx, { ...base, kind: 'loop', loop_kind: loop.kind, ...(loop.test - ? { test: readValue(loop.test, file, ctx, { constants: true }) } + ? { test: rd(loop.test, file, ctx, { constants: true }) } : {}), }) for (const setup of loop.once) visit(setup, control) @@ -1446,57 +1507,65 @@ function collectFacts(file: FileContext, ctx: CollectionState): void { return } if (sym && ts.isReturnStatement(node)) { - addFact(ctx, { - ...factBase(sym.id, 'return', node, file, control), + af(ctx, { + ...fb(sym.id, 'return', node, file, control), kind: 'return', ...(node.expression - ? { value: readValue(node.expression, file, ctx, { constants: true }) } + ? { value: rd(node.expression, file, ctx, { constants: true }) } : {}), }) if (node.expression) visit(node.expression, control) return } if (sym && ts.isThrowStatement(node)) { - addFact(ctx, { - ...factBase(sym.id, 'throw', node, file, control), + af(ctx, { + ...fb(sym.id, 'throw', node, file, control), kind: 'throw', - value: readValue(node.expression, file, ctx, { constants: true }), + value: rd(node.expression, file, ctx, { constants: true }), }) visit(node.expression, control) return } - if (sym && isBinary(node) && ASSIGNMENT_OPERATORS.has(node.operatorToken.kind)) { - addMutation(sym.id, node, 'assign', node.left, file, ctx, control, node.right) + if (sym && isBinary(node) && AOP.has(node.operatorToken.kind)) { + am(sym.id, node, 'assign', node.left, file, ctx, control, node.right) } else if ( sym && (ts.isPrefixUnaryExpression(node) || ts.isPostfixUnaryExpression(node)) && [K.PlusPlusToken, K.MinusMinusToken].includes(node.operator) ) { - addMutation( + am( sym.id, node, node.operator === K.PlusPlusToken ? 'increment' : 'decrement', node.operand, file, ctx, control, ) } else if (sym && ts.isDeleteExpression(node)) { - addMutation(sym.id, node, 'delete', node.expression, file, ctx, control) + am(sym.id, node, 'delete', node.expression, file, ctx, control) } if (sym && (isCall(node) || isNew(node))) { - const call = callFact(node, sym, file, ctx, control) - recordEffects(node, sym, file, ctx) + cf(node, sym, file, ctx, control) + re(node, sym, file, ctx) if (isCall(node)) { - const promise = promiseCombinator(node, ctx) + const promise = pc(node, ctx) if (promise) { - const base = factBase(sym.id, 'parallel', node, file, control) + const mapped = mi(node.arguments[0], file, ctx) + const lanes = mapped ? null : pl(node.arguments[0]) + const input = mapped?.input ?? (lanes + ? rd(node.arguments[0]!, file, ctx, { constants: true }) + : null) + if (!input || input.kind !== 'array' + || (lanes && lanes.length !== input.elements.length)) { + for (const argument of node.arguments) visit(argument, control) + return + } + const base = fb(sym.id, 'parallel', node, file, control) const before = ctx.facts.get(sym.id)?.length ?? 0 - const input = parallelSource(node.arguments[0], file, ctx) - const mapped = mappedInput(node.arguments[0], file, ctx) if (mapped) { - callFact(mapped.call, sym, file, ctx, control) - recordEffects(mapped.call, sym, file, ctx) - const loopId = addIteration(sym.id, mapped.call, file, ctx, control) + cf(mapped.call, sym, file, ctx, control) + re(mapped.call, sym, file, ctx) + const loopId = ai(sym.id, mapped.call, file, ctx, control) visit(mapped.receiver, control) for (const argument of mapped.call.arguments) { - const callback = unwrap(argument) + const callback = uw(argument) const executes = isArrow(callback) || isFunction(callback) visit( @@ -1516,9 +1585,7 @@ function collectFacts(file: FileContext, ctx: CollectionState): void { ) } } else { - for (const [lane, expr] of directParallelLanes( - node.arguments[0], - ).entries()) { + for (const [lane, expr] of lanes!.entries()) { visit(expr, [...control, { kind: 'parallel', controller_fact_id: base.id, @@ -1533,21 +1600,21 @@ function collectFacts(file: FileContext, ctx: CollectionState): void { frame.kind === 'parallel' && frame.controller_fact_id === base.id)) .map((fact) => fact.id) - addFact(ctx, { + af(ctx, { ...base, kind: 'parallel', ...promise, - lane_count: input?.kind === 'array' ? input.elements.length : 0, - ...(input ? { input } : {}), + lane_count: input.elements.length, + input, member_fact_ids: memberFactIds, }) return } if ( isAccess(node.expression) - && ARRAY_ITERATION_METHODS.has(node.expression.name.text) + && AIM.has(node.expression.name.text) ) { - const input = readValue( + const input = rd( node.expression.expression, file, ctx, @@ -1558,9 +1625,9 @@ function collectFacts(file: FileContext, ctx: CollectionState): void { for (const argument of node.arguments) visit(argument, control) return } - const loopId = addIteration(sym.id, node, file, ctx, control) + const loopId = ai(sym.id, node, file, ctx, control) for (const argument of node.arguments) { - const callback = unwrap(argument) + const callback = uw(argument) visit(argument, [...control, { kind: 'loop', controller_fact_id: loopId, @@ -1569,10 +1636,11 @@ function collectFacts(file: FileContext, ctx: CollectionState): void { return } const arrayMutation = isAccess(node.expression) - ? ARRAY_MUTATIONS.get(node.expression.name.text) + ? AMU.get(node.expression.name.text) : undefined - if (arrayMutation && isAccess(node.expression)) { - addMutation( + if (arrayMutation && isAccess(node.expression) + && iar(node.expression.expression, ctx)) { + am( sym.id, node, arrayMutation, node.expression.expression, file, ctx, control, arrayMutation === 'append' ? node.arguments[0] : undefined, 1, ) @@ -1583,12 +1651,9 @@ function collectFacts(file: FileContext, ctx: CollectionState): void { } visit(file.sf, []) } -function loopDescriptor(node: ts.Node): { - kind: 'for' | 'for_in' | 'for_of' | 'while' | 'do_while' - test?: ts.Expression - once: readonly ts.Node[] - repeated: readonly ts.Node[] - body: ts.Statement +function ld(node: ts.Node): { + kind: 'for' | 'for_in' | 'for_of' | 'while' | 'do_while'; test?: ts.Expression + once: readonly ts.Node[]; repeated: readonly ts.Node[]; body: ts.Statement } | null { if (ts.isForStatement(node)) { const once: ts.Node[] = [] @@ -1624,32 +1689,30 @@ function loopDescriptor(node: ts.Node): { } return null } -function decorators(node: ts.Node): readonly ts.Decorator[] { +function dc(node: ts.Node): readonly ts.Decorator[] { return ts.canHaveDecorators(node) ? ts.getDecorators(node) ?? [] : [] } -function bullDecoratorValue( - node: ts.Node, - name: 'InjectQueue' | 'Processor' | 'Process', - file: FileContext, - ctx: CollectionState, +function bv( + node: ts.Node, name: 'InjectQueue' | 'Processor' | 'Process', + file: FileContext, ctx: CollectionState, ): { value: IndexValue; transport: QueueTransport } | null { let result: { value: IndexValue; transport: QueueTransport } | null = null - for (const decorator of decorators(node)) { + for (const decorator of dc(node)) { if (!isCall(decorator.expression)) continue const call = decorator.expression if (!call.arguments[0]) continue - const binding = importedBinding(call.expression, file) + const binding = ib(call.expression, file) if (binding?.imported === name && ['@nestjs/bull', '@nestjs/bullmq'].includes(binding.module)) { result = { - value: readValue(call.arguments[0], file, ctx, { constants: true }), + value: rd(call.arguments[0], file, ctx, { constants: true }), transport: binding.module === '@nestjs/bull' ? 'bull' : 'bullmq', } } } return result } -function collectNest(file: FileContext, ctx: CollectionState): void { +function cnest(file: FileContext, ctx: CollectionState): void { for (const stmt of file.sf.statements) { if (!ts.isClassDeclaration(stmt) || !stmt.name) continue const classKey = `${file.fileId}\0${stmt.name.text}` @@ -1659,7 +1722,7 @@ function collectNest(file: FileContext, ctx: CollectionState): void { if (!ts.isConstructorDeclaration(member)) continue for (const parameter of member.parameters) { if (!isIdentifier(parameter.name)) continue - const queue = bullDecoratorValue(parameter, 'InjectQueue', file, ctx) + const queue = bv(parameter, 'InjectQueue', file, ctx) if (queue) { properties.set(parameter.name.text, [queue.value, queue.transport]) } @@ -1668,65 +1731,77 @@ function collectNest(file: FileContext, ctx: CollectionState): void { if (properties.size > 0) ctx.nestQueues.set(classKey, properties) } } -function collectNestConsumers(file: FileContext, ctx: CollectionState): void { +function nc(file: FileContext, ctx: CollectionState): void { for (const stmt of file.sf.statements) { if (!ts.isClassDeclaration(stmt) || !stmt.name) continue - const queue = bullDecoratorValue(stmt, 'Processor', file, ctx) + const queue = bv(stmt, 'Processor', file, ctx) if (!queue) continue for (const member of stmt.members) { if (!ts.isMethodDeclaration(member) || !member.name || !isIdentifier(member.name)) continue - const symbol = declSymbol(member, file, ctx) + const symbol = ds(member, file, ctx) if (!symbol) continue - const job = bullDecoratorValue(member, 'Process', file, ctx) + const job = bv(member, 'Process', file, ctx) if (job?.transport === queue.transport) { - const queueKey = staticString(queue.value) - const jobKey = staticString(job.value) + const queueKey = ss(queue.value) + const jobKey = ss(job.value) if (queueKey && jobKey) { - const queueNode = channel(ctx, { + const queueNode = ch(ctx, { channel_kind: 'queue', transport: queue.transport, key: queueKey, }) - const jobNode = channel(ctx, { + const jobNode = ch(ctx, { channel_kind: 'job', transport: queue.transport, key: jobKey, parent_channel_id: queueNode.id, }) - appendChannelEdge(ctx, symbol.id, jobNode.id, symbol.id, 'consumed_by', member, file, 'framework-decorator') - appendChannelEdge(ctx, symbol.id, jobNode.id, queueNode.id, 'routes_through', member, file, 'framework-decorator') + ce(ctx, symbol.id, jobNode.id, symbol.id, 'consumed_by', member, file, 'framework-decorator') + ce(ctx, symbol.id, jobNode.id, queueNode.id, 'routes_through', member, file, 'framework-decorator') } } else if (!job && member.name.text === 'process') { - const queueKey = staticString(queue.value) + const queueKey = ss(queue.value) if (queueKey) { - const queueNode = channel(ctx, { + const queueNode = ch(ctx, { channel_kind: 'queue', transport: queue.transport, key: queueKey, }) - appendChannelEdge(ctx, symbol.id, queueNode.id, symbol.id, 'consumed_by', member, file, 'framework-decorator') + ce(ctx, symbol.id, queueNode.id, symbol.id, 'consumed_by', member, file, 'framework-decorator') } } } } } -function expandedEffects(ownerId: string, ctx: CollectionState, depth: number, stack: ReadonlySet): ExecutionEffect[] { +function ee(ownerId: string, ctx: CollectionState, depth: number, stack: ReadonlySet): ExecutionEffect[] { const direct = [...(ctx.effects.get(ownerId) ?? [])]; - if (depth >= MAX_WRAPPER_HOPS || stack.has(ownerId)) + if (direct.length > EMAX) { + ctx.overflow.add(ownerId); + return []; + } + if (depth >= WHOP || stack.has(ownerId)) return direct; const nextStack = new Set(stack); nextStack.add(ownerId); for (const call of ctx.calls.get(ownerId) ?? []) { if (nextStack.has(call[0])) continue; - const nested = expandedEffects(call[0], ctx, depth + 1, nextStack); + const nested = ee(call[0], ctx, depth + 1, nextStack); + if (ctx.overflow.has(call[0])) { + ctx.overflow.add(ownerId); + return []; + } for (const fx of nested) { - direct.push(instantiateEffect(fx, call[1], call[2])); + if (direct.length >= EMAX) { + ctx.overflow.add(ownerId); + return []; + } + direct.push(ie(fx, call[1], call[2])); } } - return dedupeEffects(direct); + return de(direct); } -function dedupeEffects(effects: readonly ExecutionEffect[]): ExecutionEffect[] { +function de(effects: readonly ExecutionEffect[]): ExecutionEffect[] { const values = new Map() for (const fx of effects) { const witness = fx[5] @@ -1741,7 +1816,7 @@ function dedupeEffects(effects: readonly ExecutionEffect[]): ExecutionEffect[] { return [...values.values()] } type ChannelDescriptor = Omit -function channel(ctx: CollectionState, descriptor: ChannelDescriptor): IndexChannelNode { +function ch(ctx: CollectionState, descriptor: ChannelDescriptor): IndexChannelNode { const id = indexChannelId(descriptor) const node: IndexChannelNode = { id, @@ -1755,18 +1830,13 @@ function channel(ctx: CollectionState, descriptor: ChannelDescriptor): IndexChan ctx.channels.set(id, node) return node } -function appendChannelEdge( - ctx: CollectionState, - ownerId: string, - from: string, - to: string, +function ce( + ctx: CollectionState, ownerId: string, from: string, to: string, kind: Extract, - witness: ts.Node, - file: FileContext, - source: IndexEdge['source'], + witness: ts.Node, file: FileContext, source: IndexEdge['source'], confidence: Confidence = 'high', ): void { - const evidence = evidenceFor(witness, file.sf, file.fileId) + const evidence = ev(witness, file.sf, file.fileId) ctx.edges.push({ from, to, @@ -1777,17 +1847,17 @@ function appendChannelEdge( metadata: { execution_owner_id: ownerId }, }) } -function edgeSource(source: IndexFactSource): IndexEdge['source'] { +function edgeS(source: IndexFactSource): IndexEdge['source'] { if (source === 'framework') return 'framework-decorator' return source } -function fileContextForNode(node: ts.Node, ctx: CollectionState, filesById: ReadonlyMap): FileContext | null { +function fn(node: ts.Node, ctx: CollectionState, filesById: ReadonlyMap): FileContext | null { const fileId = ctx.input.pathToFileId.get(node.getSourceFile().fileName); return fileId ? filesById.get(fileId) ?? null : null; } -function unresolved(ctx: CollectionState, ownerId: string, fx: ExecutionEffect, file: FileContext): void { +function ur(ctx: CollectionState, ownerId: string, fx: ExecutionEffect, file: FileContext): void { const witness = fx[5]; - const id = `canonical-index.execution.unresolved.${sha256([ + const id = `canonical-index.execution.unresolved.${hash([ ownerId, fx[0], file.fileId, @@ -1803,44 +1873,44 @@ function unresolved(ctx: CollectionState, ownerId: string, fx: ExecutionEffect, message: `Dynamic or ambiguous ${fx[0]} identity; unresolved channel parts were omitted`, evidence: { file_id: file.fileId, - range: rangeOf(witness, file.sf), + range: ro(witness, file.sf), }, }); } -function projectEffects(ctx: CollectionState, filesById: ReadonlyMap): void { - for (const sym of ctx.input.symbols.filter(isOwnerSymbol)) { +function pe(ctx: CollectionState, filesById: ReadonlyMap): void { + for (const sym of ctx.input.symbols.filter(io)) { if (ctx.overflow.has(sym.id)) continue - for (const fx of expandedEffects(sym.id, ctx, 0, new Set())) { + for (const fx of ee(sym.id, ctx, 0, new Set())) { if (ctx.overflow.has(sym.id)) break const [kind, primary, endpoint, qualifier, scope, witness, confidence, source] = fx - const file = fileContextForNode(witness, ctx, filesById) + const file = fn(witness, ctx, filesById) if (!file) continue const emit = ( fromId: string, toId: string, relation: 'publishes_to' | 'consumed_by' | 'routes_through', - ): void => appendChannelEdge( + ): void => ce( ctx, sym.id, fromId, toId, relation, witness, file, - edgeSource(source), confidence, + edgeS(source), confidence, ) if (kind === 'bull-publish') { - const queueKey = staticString(primary) - const jobKey = staticString(endpoint) + const queueKey = ss(primary) + const jobKey = ss(endpoint) if (!queueKey) { - unresolved(ctx, sym.id, fx, file) + ur(ctx, sym.id, fx, file) continue } - const queueNode = channel(ctx, { + const queueNode = ch(ctx, { channel_kind: 'queue', transport: qualifier, key: queueKey, }) if (!jobKey) { emit(sym.id, queueNode.id, 'publishes_to') - unresolved(ctx, sym.id, fx, file) + ur(ctx, sym.id, fx, file) continue } - const jobNode = channel(ctx, { + const jobNode = ch(ctx, { channel_kind: 'job', transport: qualifier, key: jobKey, @@ -1849,27 +1919,27 @@ function projectEffects(ctx: CollectionState, filesById: ReadonlyMap @@ -1896,8 +1966,8 @@ function projectEffects(ctx: CollectionState, filesById: ReadonlyMap fact.id === callFactId)?.control ?? [] - addFact(ctx, { - ...factBase( + af(ctx, { + ...fb( sym.id, 'persistence', witness, @@ -1918,20 +1988,20 @@ function projectEffects(ctx: CollectionState, filesById: ReadonlyMap fact.kind === 'call' && fact.evidence.range.start.line === range.start.line && fact.evidence.range.start.column === range.start.column && fact.evidence.range.end.line === range.end.line && fact.evidence.range.end.column === range.end.column)?.id ?? null; } -function attachFacts(ctx: CollectionState): void { +function at(ctx: CollectionState): void { for (const symbol of ctx.input.symbols) { if (ctx.overflow.has(symbol.id)) { ctx.diagnostics.push({ - id: `canonical-index.execution.owner-bound.${sha256(symbol.id).slice(0, 16)}`, + id: `canonical-index.execution.owner-bound.${hash(symbol.id).slice(0, 16)}`, level: 'error', message: `Execution facts exceeded a per-owner safety bound for ${symbol.name}; body facts were omitted`, evidence: { file_id: symbol.file_id, range: symbol.range }, @@ -1944,7 +2014,7 @@ function attachFacts(ctx: CollectionState): void { const byId = new Map(); for (const fact of facts) byId.set(fact.id, fact); - const sorted = [...byId.values()].sort((left, right) => compareOrder(left.order, right.order) || compareText(left.id, right.id)); + const sorted = [...byId.values()].sort((left, right) => co(left.order, right.order) || ct(left.id, right.id)); try { const encoded = encodeIndexBodyFactTable(sorted); const normalized = decodeIndexBodyFactTable(encoded, symbol.id, symbol.file_id); @@ -1957,7 +2027,7 @@ function attachFacts(ctx: CollectionState): void { throw new Error(`Invalid execution facts for ${symbol.name}: ${error instanceof Error ? error.message : String(error)}`, { cause: error }); } ctx.diagnostics.push({ - id: `canonical-index.execution.owner-bound.${sha256(symbol.id).slice(0, 16)}`, + id: `canonical-index.execution.owner-bound.${hash(symbol.id).slice(0, 16)}`, level: 'error', message: `Execution facts exceeded a per-owner safety bound for ${symbol.name}; body facts were omitted`, evidence: { file_id: symbol.file_id, range: symbol.range }, @@ -1965,7 +2035,7 @@ function attachFacts(ctx: CollectionState): void { } } } -function sortEdges(edges: readonly IndexEdge[]): IndexEdge[] { +function sort(edges: readonly IndexEdge[]): IndexEdge[] { const structuralRoutes = new Map(); const retained: IndexEdge[] = []; for (const edge of edges) { @@ -1976,11 +2046,11 @@ function sortEdges(edges: readonly IndexEdge[]): IndexEdge[] { const key = `${edge.from}\u0000${edge.to}\u0000${edge.kind}`; const existing = structuralRoutes.get(key); if (!existing - || compareText(JSON.stringify(edge), JSON.stringify(existing)) < 0) { + || ct(JSON.stringify(edge), JSON.stringify(existing)) < 0) { structuralRoutes.set(key, edge); } } - return [...retained, ...structuralRoutes.values()].sort((left, right) => compareText(JSON.stringify(left), JSON.stringify(right))); + return [...retained, ...structuralRoutes.values()].sort((left, right) => ct(JSON.stringify(left), JSON.stringify(right))); } export function collectExecutionSemantics(input: CollectExecutionInput): CollectExecutionResult { const symbolsById = new Map(input.symbols.map((symbol) => [symbol.id, symbol])) @@ -1996,7 +2066,8 @@ export function collectExecutionSemantics(input: CollectExecutionInput): Collect edges: [], diagnostics: [], seenDiagnostics: new Set(), - queues: new Map(), + unstable: new Set(), + mapQueues: new Map(), emitters: new Map(), nestQueues: new Map(), files: new Map(), @@ -2008,26 +2079,27 @@ export function collectExecutionSemantics(input: CollectExecutionInput): Collect const file: FileContext = { sf, fileId, - imports: importsFor(sf), - owners: owners(sf, input.symbolsByFile.get(fileId) ?? []), + imports: im(sf), + owners: os(sf, input.symbolsByFile.get(fileId) ?? []), } filesById.set(fileId, file) ctx.files.set(sf, file) } + prep(ctx) for (const file of filesById.values()) { - collectNest(file, ctx) + cnest(file, ctx) } for (const file of filesById.values()) { - collectFacts(file, ctx) - collectNestConsumers(file, ctx) + collect(file, ctx) + nc(file, ctx) } - projectEffects(ctx, filesById) - attachFacts(ctx) + pe(ctx, filesById) + at(ctx) return { channels: [...ctx.channels.values()].sort((left, right) => - compareText(left.id, right.id)), - edges: sortEdges(ctx.edges), + ct(left.id, right.id)), + edges: sort(ctx.edges), diagnostics: [...ctx.diagnostics].sort((left, right) => - compareText(left.id, right.id)), + ct(left.id, right.id)), } } diff --git a/src/domain/index/model.ts b/src/domain/index/model.ts index e92ad3f4..1902b764 100644 --- a/src/domain/index/model.ts +++ b/src/domain/index/model.ts @@ -156,18 +156,17 @@ export type IndexPersistenceFact = Fact<'persistence', { export type IndexBodyFact = IndexCallFact | IndexLiteralFact | IndexConditionFact | IndexLoopFact | IndexParallelFact | IndexReturnFact | IndexThrowFact | IndexMutationFact | IndexPersistenceFact -/** - * Binds an operation identity to its owner, stable AST order, and authenticated - * statement bytes. Query-index validation recomputes this value so a - * well-shaped but replaced excerpt digest cannot silently become ready. - */ +/** Creates a draft collector ID, or a sealed ID when given canonical wire semantics. */ export function indexBodyFactId( ownerSymbolId: string, kind: IndexBodyFact['kind'], order: readonly number[], excerptSha256: IndexSha256, + semantics?: readonly unknown[], ): string { - const identity = [ownerSymbolId, kind, order.join('.'), excerptSha256].join('\u0000') + const identity = semantics + ? JSON.stringify([ownerSymbolId, ...semantics]) + : [ownerSymbolId, kind, order.join('.'), excerptSha256].join('\u0000') return `operation:${createHash('sha256').update(identity, 'utf8') .digest('hex').slice(0, 32)}` } @@ -175,56 +174,56 @@ export function indexBodyFactId( export type IndexBodyFactTable = readonly [version: 1, rows: readonly string[]] export const INDEX_BODY_FACT_CONTROL_LIMIT = 64 export class IndexBodyFactBoundsError extends Error {} -function enumPos(values: readonly string[], value: string): number { +function ep(values: readonly string[], value: string): number { const index = values.indexOf(value) if (index < 0) throw new Error(`Unsupported execution value ${value}`) return index } -function orderCmp(left: readonly number[], right: readonly number[]): number { +function oc(left: readonly number[], right: readonly number[]): number { for (let index = 0; index < Math.min(left.length, right.length); index += 1) { const difference = left[index]! - right[index]! if (difference !== 0) return difference } return left.length - right.length } -function isDense(value: readonly unknown[]): boolean { +function dn(value: readonly unknown[]): boolean { for (let index = 0; index < value.length; index += 1) if (!Object.hasOwn(value, index)) return false return true } -function scalar(value: unknown): value is IndexScalarValue { +function sc(value: unknown): value is IndexScalarValue { return (value === null || ['string', 'number', 'boolean'].includes(typeof value)) && !(typeof value === 'number' && (!Number.isFinite(value) || Object.is(value, -0))) && !(typeof value === 'string' && Buffer.byteLength(value, 'utf8') > MAX_TEXT) } -function packVal(value: IndexValue, depth = 0): unknown { +function pv(value: IndexValue, depth = 0): unknown { const nestedCount = value.kind === 'array' ? value.elements.length : value.kind === 'object' ? value.entries.length : value.kind === 'template' ? value.parts.length : 0 if (depth > MAX_DEPTH || (depth === MAX_DEPTH && nestedCount > 0)) - return [7, enumPos(UNKNOWN, 'unsupported')] + return [7, ep(UNKNOWN, 'unsupported')] switch (value.kind) { case 'literal': - if (!scalar(value.value)) throw new Error('Execution literal is not JSON-lossless') + if (!sc(value.value)) throw new Error('Execution literal is not JSON-lossless') return [0, value.value] case 'symbol': - if (!validText(value.symbol_id, 1_024)) + if (!vt(value.symbol_id, 1_024)) throw new Error('Execution symbol reference is invalid') return [1, value.symbol_id] case 'parameter': - if (!safeInt(value.position) + if (!si(value.position) || (value.scope !== undefined && value.scope !== 'iteration')) throw new Error('Execution parameter position is invalid') return value.scope === 'iteration' ? [2, value.position, 1] : [2, value.position] case 'array': - if (value.elements.length > MAX_ELEMENTS || !isDense(value.elements)) + if (value.elements.length > MAX_ELEMENTS || !dn(value.elements)) throw new Error('Execution array exceeds its element bound') - return [3, value.elements.map((entry) => packVal(entry, depth + 1))] + return [3, value.elements.map((entry) => pv(entry, depth + 1))] case 'object': { const keys = new Set() - if (value.entries.length > MAX_ELEMENTS || !isDense(value.entries)) + if (value.entries.length > MAX_ELEMENTS || !dn(value.entries)) throw new Error('Execution object exceeds its element bound') for (const entry of value.entries) { if (Buffer.byteLength(entry.key, 'utf8') > MAX_TEXT @@ -233,22 +232,22 @@ function packVal(value: IndexValue, depth = 0): unknown { keys.add(entry.key) } return [4, value.entries.map((entry) => [ - entry.key, packVal(entry.value, depth + 1), + entry.key, pv(entry.value, depth + 1), ])] } case 'template': - if (value.parts.length > MAX_ELEMENTS || !isDense(value.parts)) + if (value.parts.length > MAX_ELEMENTS || !dn(value.parts)) throw new Error('Execution template exceeds its element bound') - return [5, value.parts.map((entry) => packVal(entry, depth + 1))] + return [5, value.parts.map((entry) => pv(entry, depth + 1))] case 'redacted': - if (!SHA256.test(value.sha256) || !safeInt(value.byte_length)) + if (!SHA256.test(value.sha256) || !si(value.byte_length)) throw new Error('Execution redaction is invalid') return [6, value.sha256, value.byte_length] - case 'unknown': return [7, enumPos(UNKNOWN, value.reason)] + case 'unknown': return [7, ep(UNKNOWN, value.reason)] } throw new Error('Unsupported execution value') } -function packEvidence(proof: IndexFactEvidence): unknown { +function pe(proof: IndexFactEvidence): unknown { return [ proof.range.start.line, proof.range.start.column, proof.range.end.line, proof.range.end.column, @@ -264,9 +263,9 @@ export function encodeIndexBodyFactTable( 'Execution fact table is outside its row bound', ) } - if (!isDense(facts)) throw new Error('Execution fact table is sparse') + if (!dn(facts)) throw new Error('Execution fact table is sparse') const ordered = [...facts].sort((left, right) => - orderCmp(left.order, right.order) + oc(left.order, right.order) || (left.id < right.id ? -1 : left.id > right.id ? 1 : 0)) const ordinals = new Map(ordered.map((fact, index) => [fact.id, index])) if (ordinals.size !== ordered.length) @@ -278,7 +277,7 @@ export function encodeIndexBodyFactTable( } const control = (frame: IndexControlFrame): unknown => { if (frame.kind === 'branch') { - if (!validText(frame.arm, 96) + if (!vt(frame.arm, 96) || (!['then', 'else', 'truthy', 'falsy', 'nullish', 'default'].includes(frame.arm) && !(frame.arm.startsWith('case:') && frame.arm.length > 5))) { throw new Error('Execution branch arm is invalid') @@ -287,12 +286,12 @@ export function encodeIndexBodyFactTable( } if (frame.kind === 'loop') return [1, ordinal(frame.controller_fact_id)] if (frame.kind === 'parallel') { - if (frame.lane !== 'each' && !safeInt(frame.lane)) + if (frame.lane !== 'each' && !si(frame.lane)) throw new Error('Execution parallel lane is invalid') return [2, ordinal(frame.controller_fact_id), frame.lane] } if (frame.kind === 'exception') - return [3, enumPos(['try', 'catch', 'finally'], frame.arm)] + return [3, ep(['try', 'catch', 'finally'], frame.arm)] throw new Error('Unsupported execution control frame') } let bytes = 0 @@ -300,79 +299,85 @@ export function encodeIndexBodyFactTable( const rows = ordered.map((fact) => { const orderKey = fact.order.join('.') if (fact.order.length !== 4 - || !isDense(fact.order) || !fact.order.every((value) => safeInt(value)) - || !isDense(fact.control) + || !dn(fact.order) || !fact.order.every((value) => si(value)) + || !dn(fact.control) || fact.control.length > INDEX_BODY_FACT_CONTROL_LIMIT - || fact.order[1] !== enumPos(KINDS, fact.kind) - || orderKeys.has(orderKey) - || fact.id !== indexBodyFactId(fact.owner_symbol_id, fact.kind, fact.order, - fact.evidence.excerpt_sha256)) { + || fact.order[1] !== ep(KINDS, fact.kind) + || orderKeys.has(orderKey)) { throw new Error(`Invalid execution fact identity ${fact.id}`) } orderKeys.add(orderKey) let wire: unknown switch (fact.kind) { case 'call': - if (!isDense(fact.arguments)) throw new Error(`Sparse call arguments for ${fact.id}`) + if (!dn(fact.arguments)) throw new Error(`Sparse call arguments for ${fact.id}`) wire = [ fact.callee, fact.target_symbol_id ?? null, - fact.arguments.map(packVal), enumPos(TIMING, fact.scheduling), + fact.arguments.map(pv), ep(TIMING, fact.scheduling), ] break case 'literal': - wire = [packVal(fact.value), enumPos(ROLES, fact.role)] + wire = [pv(fact.value), ep(ROLES, fact.role)] break case 'condition': wire = [ - enumPos(CONDITIONS, fact.condition_kind), - fact.test ? packVal(fact.test) : null, + ep(CONDITIONS, fact.condition_kind), + fact.test ? pv(fact.test) : null, ] break case 'loop': wire = [ - enumPos(LOOPS, fact.loop_kind), - fact.test ? packVal(fact.test) : null, + ep(LOOPS, fact.loop_kind), + fact.test ? pv(fact.test) : null, ] break case 'parallel': { - const combinator = enumPos(PROMISES, fact.combinator) + const combinator = ep(PROMISES, fact.combinator) if (fact.completion !== COMPLETION[combinator] - || !safeInt(fact.lane_count) - || !isDense(fact.member_fact_ids) + || !si(fact.lane_count) + || !dn(fact.member_fact_ids) || new Set(fact.member_fact_ids).size !== fact.member_fact_ids.length) throw new Error(`Invalid parallel completion ${fact.id}`) wire = [ - combinator, fact.input ? packVal(fact.input) : null, + combinator, fact.input ? pv(fact.input) : null, fact.member_fact_ids.map(ordinal), fact.lane_count, ] break } case 'return': case 'throw': - wire = [fact.value ? packVal(fact.value) : null] + wire = [fact.value ? pv(fact.value) : null] break case 'mutation': wire = [ - enumPos(MUTATIONS, fact.operation), fact.target, - fact.value ? packVal(fact.value) : null, + ep(MUTATIONS, fact.operation), fact.target, + fact.value ? pv(fact.value) : null, ] break case 'persistence': - if (!validText(fact.receiver_type)) + if (!vt(fact.receiver_type)) throw new Error(`Persistence proof is missing for ${fact.id}`) wire = [ - enumPos(STORAGE, fact.operation), ordinal(fact.call_fact_id), - fact.resource ? packVal(fact.resource) : null, + ep(STORAGE, fact.operation), ordinal(fact.call_fact_id), + fact.resource ? pv(fact.resource) : null, fact.receiver_type, ] break } - const row = JSON.stringify([ - fact.id, enumPos(KINDS, fact.kind), - fact.order[0], fact.order[2], fact.order[3], packEvidence(fact.evidence), - fact.control.map(control), enumPos(LEVELS, fact.confidence), - enumPos(SOURCES, fact.source), wire, - ]) + const semantics = [ + ep(KINDS, fact.kind), + fact.order[0], fact.order[2], fact.order[3], pe(fact.evidence), + fact.control.map(control), ep(LEVELS, fact.confidence), + ep(SOURCES, fact.source), wire, + ] + const sealedId = indexBodyFactId( + fact.owner_symbol_id, fact.kind, fact.order, + fact.evidence.excerpt_sha256, semantics, + ) + if (fact.id !== sealedId && fact.id !== indexBodyFactId( + fact.owner_symbol_id, fact.kind, fact.order, fact.evidence.excerpt_sha256, + )) throw new Error(`Invalid execution fact identity ${fact.id}`) + const row = JSON.stringify([sealedId, ...semantics]) const rowBytes = Buffer.byteLength(row, 'utf8') bytes += rowBytes if (rowBytes > MAX_ROW || bytes > MAX_TABLE) @@ -383,42 +388,42 @@ export function encodeIndexBodyFactTable( }) return [1, rows] } -function safeInt(value: unknown, minimum = 0): value is number { +function si(value: unknown, minimum = 0): value is number { return typeof value === 'number' && Number.isSafeInteger(value) && !Object.is(value, -0) && value >= minimum } -function validText(value: unknown, maxBytes = MAX_TEXT): value is string { +function vt(value: unknown, maxBytes = MAX_TEXT): value is string { return typeof value === 'string' && value.length > 0 && !value.includes('\0') && Buffer.byteLength(value, 'utf8') <= maxBytes } -function tuple(value: unknown, length: number): unknown[] | null { +function tu(value: unknown, length: number): unknown[] | null { return Array.isArray(value) && value.length === length ? value : null } -function enumValue(values: readonly T[], value: unknown): T | null { - return safeInt(value) && value < values.length ? values[value]! : null +function ev(values: readonly T[], value: unknown): T | null { + return si(value) && value < values.length ? values[value]! : null } -function readVal(value: unknown, depth = 0): IndexValue | null { +function rv(value: unknown, depth = 0): IndexValue | null { if (!Array.isArray(value) - || !safeInt(value[0]) || value[0] > 7) return null + || !si(value[0]) || value[0] > 7) return null if (depth > MAX_DEPTH) return null if (depth === MAX_DEPTH && [3, 4, 5].includes(value[0]) && (!Array.isArray(value[1]) || value[1].length > 0)) return null switch (value[0]) { case 0: { - return value.length === 2 && scalar(value[1]) + return value.length === 2 && sc(value[1]) ? { kind: 'literal', value: value[1] } : null } case 1: - return value.length === 2 && validText(value[1], 1_024) + return value.length === 2 && vt(value[1], 1_024) ? { kind: 'symbol', symbol_id: value[1] } : null case 2: return (value.length === 2 || (value.length === 3 && value[2] === 1)) - && safeInt(value[1]) + && si(value[1]) ? { kind: 'parameter', position: value[1], @@ -429,7 +434,7 @@ function readVal(value: unknown, depth = 0): IndexValue | null { case 5: { if (value.length !== 2 || !Array.isArray(value[1]) || value[1].length > MAX_ELEMENTS) return null - const values = value[1].map((entry) => readVal(entry, depth + 1)) + const values = value[1].map((entry) => rv(entry, depth + 1)) if (!values.every((entry): entry is IndexValue => entry !== null)) return null return value[0] === 3 ? { kind: 'array', elements: values } @@ -441,8 +446,8 @@ function readVal(value: unknown, depth = 0): IndexValue | null { const keys = new Set() const entries: IndexObjectEntry[] = [] for (const raw of value[1]) { - const entry = tuple(raw, 2) - const decoded = entry ? readVal(entry[1], depth + 1) : null + const entry = tu(raw, 2) + const decoded = entry ? rv(entry[1], depth + 1) : null if (!entry || typeof entry[0] !== 'string' || entry[0].includes('\0') || Buffer.byteLength(entry[0], 'utf8') > MAX_TEXT || keys.has(entry[0]) || !decoded) return null @@ -453,11 +458,11 @@ function readVal(value: unknown, depth = 0): IndexValue | null { } case 6: return value.length === 3 && typeof value[1] === 'string' - && SHA256.test(value[1]) && safeInt(value[2]) + && SHA256.test(value[1]) && si(value[2]) ? { kind: 'redacted', sha256: value[1], byte_length: value[2] } : null case 7: { - const reason = enumValue(UNKNOWN, value[1]) + const reason = ev(UNKNOWN, value[1]) return value.length === 2 && reason ? { kind: 'unknown', reason } : null } } @@ -468,9 +473,9 @@ type DecodedRow = { evidence: IndexFactEvidence; control: readonly unknown[] confidence: IndexFactConfidence; source: IndexFactSource; payload: unknown } -function readEvidence(value: unknown, file: string): IndexFactEvidence | null { - const row = tuple(value, 9) - if (!row || !row.slice(0, 8).every((entry) => safeInt(entry, 1)) +function re(value: unknown, file: string): IndexFactEvidence | null { + const row = tu(value, 9) + if (!row || !row.slice(0, 8).every((entry) => si(entry, 1)) || typeof row[8] !== 'string' || !SHA256.test(row[8])) return null const range = { start: { line: row[0] as number, column: row[1] as number }, @@ -489,7 +494,7 @@ function readEvidence(value: unknown, file: string): IndexFactEvidence | null { ? { file_id: file, range, statement_range, excerpt_sha256: row[8] } : null } -function decodeRow(value: string, owner: string, file: string): DecodedRow | null { +function dr(value: string, owner: string, file: string): DecodedRow | null { if (Buffer.byteLength(value, 'utf8') > MAX_ROW) return null let parsed: unknown try { @@ -498,19 +503,21 @@ function decodeRow(value: string, owner: string, file: string): DecodedRow | nul return null } if (JSON.stringify(parsed) !== value) return null - const row = tuple(parsed, 10) - if (!row || !validText(row[0], 64) - || !safeInt(row[1]) || row[1] >= KINDS.length - || !safeInt(row[2]) || !safeInt(row[3]) || !safeInt(row[4]) + const row = tu(parsed, 10) + if (!row || !vt(row[0], 64) + || !si(row[1]) || row[1] >= KINDS.length + || !si(row[2]) || !si(row[3]) || !si(row[4]) || !Array.isArray(row[6]) || row[6].length > INDEX_BODY_FACT_CONTROL_LIMIT) return null const kind = KINDS[row[1]]! - const proof = readEvidence(row[5], file) - const confidence = enumValue(LEVELS, row[7]) - const source = enumValue(SOURCES, row[8]) + const proof = re(row[5], file) + const confidence = ev(LEVELS, row[7]) + const source = ev(SOURCES, row[8]) const order = [row[2], row[1], row[3], row[4]] as number[] if (!proof || !confidence || !source - || row[0] !== indexBodyFactId(owner, kind, order, proof.excerpt_sha256)) { + || row[0] !== indexBodyFactId( + owner, kind, order, proof.excerpt_sha256, row.slice(1), + )) { return null } return { @@ -523,8 +530,8 @@ export function decodeIndexBodyFactTable( owner: string, file: string, ): readonly IndexBodyFact[] | null { - const table = tuple(value, 2) - if (!validText(owner, 1_024) || !validText(file, 128) + const table = tu(value, 2) + if (!vt(owner, 1_024) || !vt(file, 128) || !table || table[0] !== 1 || !Array.isArray(table[1]) || table[1].length === 0 || table[1].length > MAX_ROWS) return null const decoded: DecodedRow[] = [] @@ -533,24 +540,24 @@ export function decodeIndexBodyFactTable( if (typeof value !== 'string') return null bytes += Buffer.byteLength(value, 'utf8') if (bytes > MAX_TABLE) return null - const row = decodeRow(value, owner, file) + const row = dr(value, owner, file) if (!row) return null decoded.push(row) } const ids = decoded.map((row) => row.id) if (new Set(ids).size !== ids.length || decoded.some((row, index) => index > 0 - && orderCmp(decoded[index - 1]!.order, row.order) >= 0)) { + && oc(decoded[index - 1]!.order, row.order) >= 0)) { return null } const idAt = (value: unknown): string | null => - safeInt(value) && value < ids.length ? ids[value]! : null + si(value) && value < ids.length ? ids[value]! : null const control = (value: unknown): IndexControlFrame | null => { - if (!Array.isArray(value) || !safeInt(value[0])) return null + if (!Array.isArray(value) || !si(value[0])) return null const controller_fact_id = idAt(value[1]) if (value[0] === 0) { return value.length === 3 && controller_fact_id - && validText(value[2], 96) + && vt(value[2], 96) && (['then', 'else', 'truthy', 'falsy', 'nullish', 'default'].includes(value[2]) || (value[2].startsWith('case:') && value[2].length > 5)) ? { kind: 'branch', controller_fact_id, arm: value[2] as IndexBranchArm } @@ -559,9 +566,9 @@ export function decodeIndexBodyFactTable( if (value[0] === 1) return value.length === 2 && controller_fact_id ? { kind: 'loop', controller_fact_id } : null if (value[0] === 2) return value.length === 3 && controller_fact_id - && (value[2] === 'each' || safeInt(value[2])) + && (value[2] === 'each' || si(value[2])) ? { kind: 'parallel', controller_fact_id, lane: value[2] } : null - const arm = enumValue(['try', 'catch', 'finally'] as const, value[1]) + const arm = ev(['try', 'catch', 'finally'] as const, value[1]) return value[0] === 3 && value.length === 2 && arm ? { kind: 'exception', arm } : null } @@ -577,12 +584,12 @@ export function decodeIndexBodyFactTable( const wire = Array.isArray(row.payload) ? row.payload : null let fact: IndexBodyFact | null = null if (row.kind === 'call' && wire?.length === 4) { - const scheduling = enumValue(TIMING, wire[3]) + const scheduling = ev(TIMING, wire[3]) const args = Array.isArray(wire[2]) - ? wire[2].map((entry) => readVal(entry)) + ? wire[2].map((entry) => rv(entry)) : [] - if (validText(wire[0]) && scheduling - && (wire[1] === null || validText(wire[1], 1_024)) + if (vt(wire[0]) && scheduling + && (wire[1] === null || vt(wire[1], 1_024)) && Array.isArray(wire[2]) && args.every((entry): entry is IndexValue => entry !== null)) { fact = { @@ -592,24 +599,24 @@ export function decodeIndexBodyFactTable( } } } else if (row.kind === 'literal' && wire?.length === 2) { - const decoded = readVal(wire[0]) - const role = enumValue(ROLES, wire[1]) + const decoded = rv(wire[0]) + const role = ev(ROLES, wire[1]) if (decoded && role) fact = { ...base, kind: 'literal', value: decoded, role } } else if (row.kind === 'condition' && wire?.length === 2) { - const condition_kind = enumValue(CONDITIONS, wire[0]) - const test = wire[1] === null ? undefined : readVal(wire[1]) + const condition_kind = ev(CONDITIONS, wire[0]) + const test = wire[1] === null ? undefined : rv(wire[1]) if (condition_kind && (wire[1] === null || test)) { fact = { ...base, kind: 'condition', condition_kind, ...(test ? { test } : {}) } } } else if (row.kind === 'loop' && wire?.length === 2) { - const loop_kind = enumValue(LOOPS, wire[0]) - const test = wire[1] === null ? undefined : readVal(wire[1]) + const loop_kind = ev(LOOPS, wire[0]) + const test = wire[1] === null ? undefined : rv(wire[1]) if (loop_kind && (wire[1] === null || test)) { fact = { ...base, kind: 'loop', loop_kind, ...(test ? { test } : {}) } } } else if (row.kind === 'parallel' && wire?.length === 4) { - const combinator = enumValue(PROMISES, wire[0]) - const input = wire[1] === null ? undefined : readVal(wire[1]) + const combinator = ev(PROMISES, wire[0]) + const input = wire[1] === null ? undefined : rv(wire[1]) const members = Array.isArray(wire[2]) ? wire[2].map(idAt) : [] @@ -617,7 +624,7 @@ export function decodeIndexBodyFactTable( && Array.isArray(wire[2]) && members.every((id): id is string => id !== null) && new Set(members).size === members.length - && safeInt(wire[3])) { + && si(wire[3])) { fact = { ...base, kind: 'parallel', combinator, completion: COMPLETION[PROMISES.indexOf(combinator)]!, @@ -628,14 +635,14 @@ export function decodeIndexBodyFactTable( } } else if ((row.kind === 'return' || row.kind === 'throw') && wire?.length === 1) { - const decoded = wire[0] === null ? undefined : readVal(wire[0]) + const decoded = wire[0] === null ? undefined : rv(wire[0]) if (wire[0] === null || decoded) { fact = { ...base, kind: row.kind, ...(decoded ? { value: decoded } : {}) } } } else if (row.kind === 'mutation' && wire?.length === 3) { - const operation = enumValue(MUTATIONS, wire[0]) - const decoded = wire[2] === null ? undefined : readVal(wire[2]) - if (operation && validText(wire[1]) + const operation = ev(MUTATIONS, wire[0]) + const decoded = wire[2] === null ? undefined : rv(wire[2]) + if (operation && vt(wire[1]) && (wire[2] === null || decoded)) { fact = { ...base, kind: 'mutation', operation, target: wire[1], @@ -643,11 +650,11 @@ export function decodeIndexBodyFactTable( } } } else if (row.kind === 'persistence' && wire?.length === 4) { - const operation = enumValue(STORAGE, wire[0]) + const operation = ev(STORAGE, wire[0]) const call_fact_id = idAt(wire[1]) - const resource = wire[2] === null ? undefined : readVal(wire[2]) + const resource = wire[2] === null ? undefined : rv(wire[2]) if (operation && call_fact_id && (wire[2] === null || resource) - && validText(wire[3])) { + && vt(wire[3])) { fact = { ...base, kind: 'persistence', operation, call_fact_id, ...(resource ? { resource } : {}), diff --git a/src/domain/query/index-status.ts b/src/domain/query/index-status.ts index 1e98e6af..0998bc37 100644 --- a/src/domain/query/index-status.ts +++ b/src/domain/query/index-status.ts @@ -63,43 +63,39 @@ const EDGE_SOURCES = new Set([ 'wrapper-summary', ]) -class QueryIndexIntegrityError extends Error {} +class IntegrityError extends Error {} function fail(subject: string): never { - throw new QueryIndexIntegrityError(subject) + throw new IntegrityError(`canonical ${subject}`) } -function nonEmpty(value: unknown): value is string { - return typeof value === 'string' && value.length > 0 && !value.includes('\0') +function nonEmpty(v: unknown): v is string { + return typeof v === 'string' && v.length > 0 && !v.includes('\0') } -function bounded(value: unknown, maxBytes: number): value is string { - return nonEmpty(value) - && Buffer.byteLength(value, 'utf8') <= maxBytes +function bounded(v: unknown, maxBytes: number): v is string { + return nonEmpty(v) && Buffer.byteLength(v, 'utf8') <= maxBytes } -function safeInt(value: unknown, minimum = 0): value is number { - return typeof value === 'number' - && Number.isSafeInteger(value) - && value >= minimum +function safeInt(v: unknown, minimum = 0): v is number { + return typeof v === 'number' + && Number.isSafeInteger(v) + && v >= minimum } -function exact(value: unknown, required: readonly string[], optional: readonly string[] = []): Record | null { - if (!isRecord(value)) - return null; - const allowed = new Set([...required, ...optional]); - return required.every((key) => Object.hasOwn(value, key)) - && Object.keys(value).every((key) => allowed.has(key)) - ? value - : null; +function exact(v: unknown, keys: readonly string[]): Record | null { + return isRecord(v) + && Object.keys(v).length === keys.length + && keys.every((key) => Object.hasOwn(v, key)) + ? v : null } -function comparePos(left: IndexRange['start'], right: IndexRange['start']): number { - return left.line - right.line || left.column - right.column; +function posCmp(a: IndexRange['start'], b: IndexRange['start']): number { + return a.line - b.line || a.column - b.column } -function parseRange(value: unknown): IndexRange | null { - const range = exact(value, ['start', 'end']) +function rangeOf(v: unknown): IndexRange | null { + const range = exact(v, ['start', 'end']) const start = exact(range?.start, ['line', 'column']) const end = exact(range?.end, ['line', 'column']) if (!range || !start || !end @@ -109,45 +105,45 @@ function parseRange(value: unknown): IndexRange | null { start: { line: start.line, column: start.column }, end: { line: end.line, column: end.column }, } - return comparePos(parsed.start, parsed.end) <= 0 ? parsed : null + return posCmp(parsed.start, parsed.end) <= 0 ? parsed : null } -function containsRange(container: IndexRange, value: IndexRange): boolean { - return comparePos(container.start, value.start) <= 0 - && comparePos(value.end, container.end) <= 0 +function contains(outer: IndexRange, inner: IndexRange): boolean { + return posCmp(outer.start, inner.start) <= 0 + && posCmp(inner.end, outer.end) <= 0 } -function sameRange(left: IndexRange, right: IndexRange): boolean { - return comparePos(left.start, right.start) === 0 - && comparePos(left.end, right.end) === 0 +function sameSpan(a: IndexRange, b: IndexRange): boolean { + return posCmp(a.start, b.start) === 0 + && posCmp(a.end, b.end) === 0 } -function validBranchArm( - control: Extract, +function validArm( + ctl: Extract, arm: string, ): boolean { - if (control.condition_kind === 'if') return ['then', 'else'].includes(arm) - if (control.condition_kind === 'switch') { + if (ctl.condition_kind === 'if') return ['then', 'else'].includes(arm) + if (ctl.condition_kind === 'switch') { return arm === 'default' || (arm.startsWith('case:') && arm.length > 5) } - if (control.condition_kind === 'logical_and') return arm === 'truthy' - if (control.condition_kind === 'logical_or') return arm === 'falsy' - if (control.condition_kind === 'nullish') return arm === 'nullish' - return control.condition_kind === 'ternary' + if (ctl.condition_kind === 'logical_and') return arm === 'truthy' + if (ctl.condition_kind === 'logical_or') return arm === 'falsy' + if (ctl.condition_kind === 'nullish') return arm === 'nullish' + return ctl.condition_kind === 'ternary' ? ['truthy', 'falsy'].includes(arm) : ['then', 'else'].includes(arm) } -function edgeProof(attrs: GraphAttributes, fileIds: ReadonlyMap, nodeById: ReadonlyMap): boolean { - const source = attrs.source_file; - const ownerId = attrs.execution_owner_id; - const owner = typeof ownerId === 'string' ? nodeById.get(ownerId) : undefined; - const span = parseRange(owner?.definition_range); - const record = exact(attrs.evidence, ['source', 'range', 'statement_range', 'excerpt_sha256']); - const range = parseRange(record?.range); - const statement = parseRange(record?.statement_range); +function edgeProof(a: GraphAttributes, files: ReadonlyMap, nodes: ReadonlyMap): boolean { + const source = a.source_file; + const ownerId = a.execution_owner_id; + const owner = typeof ownerId === 'string' ? nodes.get(ownerId) : undefined; + const span = rangeOf(owner?.definition_range); + const record = exact(a.evidence, ['source', 'range', 'statement_range', 'excerpt_sha256']); + const range = rangeOf(record?.range); + const statement = rangeOf(record?.statement_range); return typeof source === 'string' - && fileIds.has(source) + && files.has(source) && typeof ownerId === 'string' && owner?.source_file === source && owner?.node_kind !== 'file' @@ -157,374 +153,368 @@ function edgeProof(attrs: GraphAttributes, fileIds: ReadonlyMap, && EDGE_SOURCES.has(String(record.source)) && range !== null && statement !== null - && containsRange(span, statement) - && containsRange(statement, range) + && contains(span, statement) + && contains(statement, range) && typeof record.excerpt_sha256 === 'string' && SHA256.test(record.excerpt_sha256); } -function valueHas(value: IndexValue, test: (candidate: IndexValue) => boolean): boolean { - if (test(value)) - return true; - if (value.kind === 'array') { - return value.elements.some((entry) => valueHas(entry, test)); - } - if (value.kind === 'object') { - return value.entries.some((entry) => valueHas(entry.value, test)); - } - return value.kind === 'template' - && value.parts.some((entry) => valueHas(entry, test)); +function valueHas(v: IndexValue, test: (candidate: IndexValue) => boolean): boolean { + return test(v) + || v.kind === 'array' && v.elements.some((entry) => valueHas(entry, test)) + || v.kind === 'object' && v.entries.some((entry) => valueHas(entry.value, test)) + || v.kind === 'template' && v.parts.some((entry) => valueHas(entry, test)) } function factHas( fact: IndexBodyFact, test: (candidate: IndexValue) => boolean, ): boolean { - let values: readonly IndexValue[] + let xs: readonly IndexValue[] switch (fact.kind) { case 'call': - values = fact.arguments; break + xs = fact.arguments; break case 'literal': - values = [fact.value]; break + xs = [fact.value]; break case 'condition': case 'loop': - values = fact.test ? [fact.test] : []; break + xs = fact.test ? [fact.test] : []; break case 'parallel': - values = fact.input ? [fact.input] : []; break + xs = fact.input ? [fact.input] : []; break case 'return': case 'throw': case 'mutation': - values = fact.value ? [fact.value] : []; break + xs = fact.value ? [fact.value] : []; break case 'persistence': - values = fact.resource ? [fact.resource] : [] + xs = fact.resource ? [fact.resource] : [] } - return values.some((value) => valueHas(value, test)) + return xs.some((value) => valueHas(value, test)) } -function channelFrom(id: string, attrs: GraphAttributes): IndexChannelNode | null { +function readChannel(id: string, a: GraphAttributes): IndexChannelNode | null { if (!nonEmpty(id) - || !KINDS.has(attrs.channel_kind as IndexChannelKind) - || !TRANSPORTS.has(attrs.transport as IndexChannelTransport) - || !bounded(attrs.key, MAX_TEXT) - || (Object.hasOwn(attrs, 'parent_channel_id') - && !nonEmpty(attrs.parent_channel_id)) - || (Object.hasOwn(attrs, 'scope') - && !bounded(attrs.scope, 512))) + || !KINDS.has(a.channel_kind as IndexChannelKind) + || !TRANSPORTS.has(a.transport as IndexChannelTransport) + || !bounded(a.key, MAX_TEXT) + || (Object.hasOwn(a, 'parent_channel_id') + && !nonEmpty(a.parent_channel_id)) + || (Object.hasOwn(a, 'scope') + && !bounded(a.scope, 512))) return null; const channel: IndexChannelNode = { id, node_kind: 'channel', - channel_kind: attrs.channel_kind as IndexChannelKind, - transport: attrs.transport as IndexChannelTransport, - key: attrs.key, - ...(typeof attrs.parent_channel_id === 'string' - ? { parent_channel_id: attrs.parent_channel_id } + channel_kind: a.channel_kind as IndexChannelKind, + transport: a.transport as IndexChannelTransport, + key: a.key, + ...(typeof a.parent_channel_id === 'string' + ? { parent_channel_id: a.parent_channel_id } : {}), - ...(typeof attrs.scope === 'string' - ? { scope: attrs.scope } + ...(typeof a.scope === 'string' + ? { scope: a.scope } : {}), }; return id === indexChannelId(channel) ? channel : null; } -function orderCmp(left: readonly number[], right: readonly number[]): number { - for (let index = 0; index < Math.min(left.length, right.length); index += 1) { - const difference = left[index]! - right[index]! +function orderCmp(a: readonly number[], b: readonly number[]): number { + for (let index = 0; index < Math.min(a.length, b.length); index += 1) { + const difference = a[index]! - b[index]! if (difference !== 0) return difference } - return left.length - right.length + return a.length - b.length } -function freeze(value: T): T { - if (value !== null && typeof value === 'object' && !Object.isFrozen(value)) { - for (const entry of Object.values(value)) freeze(entry) - Object.freeze(value) +function freeze(v: T): T { + if (v !== null && typeof v === 'object' && !Object.isFrozen(v)) { + for (const entry of Object.values(v)) freeze(entry) + Object.freeze(v) } - return value + return v } function sealMap(entries: Iterable): ReadonlyMap { - const values = new Map(entries); - let facade: ReadonlyMap; - facade = { - get size() { return values.size; }, - get(key: K) { return values.get(key); }, - has(key: K) { return values.has(key); }, - entries() { return values.entries(); }, - keys() { return values.keys(); }, - values() { return values.values(); }, + const xs = new Map(entries); + let view: ReadonlyMap; + view = { + get size() { return xs.size; }, + get(key: K) { return xs.get(key); }, + has(key: K) { return xs.has(key); }, + entries() { return xs.entries(); }, + keys() { return xs.keys(); }, + values() { return xs.values(); }, forEach(callback: (value: V, key: K, map: ReadonlyMap) => void, thisArg?: unknown) { - values.forEach((value, key) => callback.call(thisArg, value, key, facade)); + xs.forEach((value, key) => callback.call(thisArg, value, key, view)); }, - [Symbol.iterator]() { return values[Symbol.iterator](); }, + [Symbol.iterator]() { return xs[Symbol.iterator](); }, }; - return Object.freeze(facade); + return Object.freeze(view); } function sortEntries( - values: ReadonlyMap, + xs: ReadonlyMap, ): Array { - return [...values.entries()] + return [...xs.entries()] .sort(([left], [right]) => compareCodeUnits(left, right)) } type ExecutionIndexes = Pick; -function buildMaps(view: KnowledgeGraph, fileIds: ReadonlyMap): ExecutionIndexes { +function buildMaps(view: KnowledgeGraph, files: ReadonlyMap): ExecutionIndexes { const nodes = view.nodeEntries(); - const nodeById = new Map(nodes); - const symbolIds = new Set(); - const factById = new Map(); - const ownedFacts = new Map(); - const channels = new Map(); + const byId = new Map(nodes); + const symbols = new Set(); + const facts = new Map(); + const owned = new Map(); + const chs = new Map(); const byKey = new Map(); const orderKeys = new Map>(); - for (const [id, attrs] of nodes) { - if (attrs.node_kind === 'channel') { - if (Object.hasOwn(attrs, 'body_facts')) { - fail('canonical channel body facts'); + for (const [id, a] of nodes) { + if (a.node_kind === 'channel') { + if (Object.hasOwn(a, 'body_facts')) { + fail('channel body facts'); } - const channel = channelFrom(id, attrs); - if (!channel) - fail('canonical channel node'); - channels.set(id, channel); + const ch = readChannel(id, a); + if (!ch) + fail('channel node'); + chs.set(id, ch); continue; } - if (attrs.node_kind === 'file') { - if (Object.hasOwn(attrs, 'body_facts') - || Object.hasOwn(attrs, 'channel_kind') - || Object.hasOwn(attrs, 'parent_channel_id')) { - fail('canonical file-node execution metadata'); + if (a.node_kind === 'file') { + if (Object.hasOwn(a, 'body_facts') + || Object.hasOwn(a, 'channel_kind') + || Object.hasOwn(a, 'parent_channel_id')) { + fail('file-node execution metadata'); } continue; } - if (Object.hasOwn(attrs, 'channel_kind') - || Object.hasOwn(attrs, 'parent_channel_id')) { - fail('canonical channel discriminator'); + if (Object.hasOwn(a, 'channel_kind') + || Object.hasOwn(a, 'parent_channel_id')) { + fail('channel discriminator'); } - symbolIds.add(id); - if (!Object.hasOwn(attrs, 'body_facts')) + symbols.add(id); + if (!Object.hasOwn(a, 'body_facts')) continue; - const source = attrs.source_file; + const source = a.source_file; const fileId = typeof source === 'string' - ? fileIds.get(source) + ? files.get(source) : undefined; - const span = parseRange(attrs.definition_range); - const ownerFile = fileId ? nodeById.get(fileId) : undefined; + const span = rangeOf(a.definition_range); + const ownerFile = fileId ? byId.get(fileId) : undefined; if (!fileId || !span || !ownerFile || ownerFile.node_kind !== 'file') { - fail('canonical operation owner'); + fail('operation owner'); } - const facts = decodeIndexBodyFactTable(attrs.body_facts, id, fileId); - if (!facts) - fail('canonical symbol body facts'); + const bodyFacts = decodeIndexBodyFactTable(a.body_facts, id, fileId); + if (!bodyFacts) + fail('symbol body facts'); const orders = orderKeys.get(id) ?? new Set(); orderKeys.set(id, orders); - for (const fact of facts) { - if (!containsRange(span, fact.evidence.statement_range) - || factById.has(fact.id)) { - fail('canonical operation fact'); + for (const fact of bodyFacts) { + if (!contains(span, fact.evidence.statement_range) + || facts.has(fact.id)) { + fail('operation fact'); } const orderKey = fact.order.join('.'); if (orders.has(orderKey)) - fail('canonical operation order'); + fail('operation order'); orders.add(orderKey); - factById.set(fact.id, fact); - const owned = ownedFacts.get(id) ?? []; - owned.push(fact); - ownedFacts.set(id, owned); + facts.set(fact.id, fact); + const ownerFacts = owned.get(id) ?? []; + ownerFacts.push(fact); + owned.set(id, ownerFacts); } } - for (const channel of channels.values()) { - if (channel.channel_kind === 'job') { - const parent = channel.parent_channel_id - ? channels.get(channel.parent_channel_id) + for (const ch of chs.values()) { + if (ch.channel_kind === 'job') { + const parent = ch.parent_channel_id + ? chs.get(ch.parent_channel_id) : undefined; if (!parent || parent.channel_kind !== 'queue' - || parent.transport !== channel.transport) { - fail('canonical job parent channel'); + || parent.transport !== ch.transport) { + fail('job parent channel'); } } - else if (channel.parent_channel_id !== undefined) { - fail('canonical non-job parent channel'); + else if (ch.parent_channel_id !== undefined) { + fail('non-job parent channel'); } - if (channel.channel_kind === 'event') { - if (!bounded(channel.scope, 512)) { - fail('canonical event channel scope'); + if (ch.channel_kind === 'event') { + if (!bounded(ch.scope, 512)) { + fail('event channel scope'); } } - else if (channel.scope !== undefined) { - fail('canonical non-event channel scope'); + else if (ch.scope !== undefined) { + fail('non-event channel scope'); } - const keyed = byKey.get(channel.key) ?? []; - keyed.push(channel); - byKey.set(channel.key, keyed); + const keyed = byKey.get(ch.key) ?? []; + keyed.push(ch); + byKey.set(ch.key, keyed); } - for (const fact of factById.values()) { - if (factHas(fact, (value) => value.kind === 'symbol' && !symbolIds.has(value.symbol_id))) { - fail('canonical operation value reference'); + for (const fact of facts.values()) { + if (factHas(fact, (value) => value.kind === 'symbol' && !symbols.has(value.symbol_id))) { + fail('operation value reference'); } if (fact.kind === 'call' && fact.target_symbol_id - && !symbolIds.has(fact.target_symbol_id)) { - fail('canonical call target'); + && !symbols.has(fact.target_symbol_id)) { + fail('call target'); } const controlIds = new Set(); - for (const frame of fact.control) { - if (frame.kind === 'exception') + for (const f of fact.control) { + if (f.kind === 'exception') continue; - if (controlIds.has(frame.controller_fact_id)) { - fail('canonical duplicate control reference'); + if (controlIds.has(f.controller_fact_id)) { + fail('duplicate control reference'); } - controlIds.add(frame.controller_fact_id); - const control = factById.get(frame.controller_fact_id); - const expectedKind = frame.kind === 'branch' + controlIds.add(f.controller_fact_id); + const ctl = facts.get(f.controller_fact_id); + const expectedKind = f.kind === 'branch' ? 'condition' - : frame.kind; - const guardFallthrough = frame.kind === 'branch' - && control?.kind === 'condition' - && control.condition_kind === 'guard'; - if (!control || control.owner_symbol_id !== fact.owner_symbol_id - || control.kind !== expectedKind - || orderCmp(control.order, fact.order) >= 0 - || (!guardFallthrough && !containsRange(frame.kind === 'parallel' - ? control.evidence.range - : control.evidence.statement_range, fact.evidence.range)) - || (frame.kind === 'branch' && control.kind === 'condition' - && !validBranchArm(control, frame.arm)) - || (frame.kind === 'parallel' && control.kind === 'parallel' - && (frame.lane === 'each' - ? control.lane_count === 0 - : frame.lane >= control.lane_count)) - || (fact.kind === 'call' && frame.kind === 'parallel' - && control.kind === 'parallel' - && !control.member_fact_ids.includes(fact.id))) { - fail('canonical operation control reference'); + : f.kind; + const guardFallthrough = f.kind === 'branch' + && ctl?.kind === 'condition' + && ctl.condition_kind === 'guard'; + if (!ctl || ctl.owner_symbol_id !== fact.owner_symbol_id + || ctl.kind !== expectedKind + || orderCmp(ctl.order, fact.order) >= 0 + || (!guardFallthrough && !contains(f.kind === 'parallel' + ? ctl.evidence.range + : ctl.evidence.statement_range, fact.evidence.range)) + || (f.kind === 'branch' && ctl.kind === 'condition' + && !validArm(ctl, f.arm)) + || (f.kind === 'parallel' && ctl.kind === 'parallel' + && (f.lane === 'each' + ? ctl.lane_count === 0 + : f.lane >= ctl.lane_count)) + || (fact.kind === 'call' && f.kind === 'parallel' + && ctl.kind === 'parallel' + && !ctl.member_fact_ids.includes(fact.id))) { + fail('operation control reference'); } } if (factHas(fact, (value) => value.kind === 'parameter' && value.scope === 'iteration') - && !fact.control.some((frame) => { - const control = frame.kind === 'loop' - ? factById.get(frame.controller_fact_id) + && !fact.control.some((f) => { + const ctl = f.kind === 'loop' + ? facts.get(f.controller_fact_id) : undefined; - return control?.kind === 'loop' - && control.loop_kind === 'array_iteration'; + return ctl?.kind === 'loop' + && ctl.loop_kind === 'array_iteration'; })) { - fail('canonical iteration parameter'); + fail('iteration parameter'); } if (fact.kind === 'parallel') { const laneCount = fact.input?.kind === 'array' ? fact.input.elements.length : 0; if (fact.member_fact_ids.some((id) => { - const member = factById.get(id); - const frame = member?.control.find((candidate): candidate is Extract => candidate.kind === 'parallel' - && candidate.controller_fact_id === fact.id); + }> => f.kind === 'parallel' + && f.controller_fact_id === fact.id); const loop = frame?.lane === 'each' - ? member?.control.some((candidate) => { - const control = candidate.kind === 'loop' - ? factById.get(candidate.controller_fact_id) + ? member?.control.some((f) => { + const ctl = f.kind === 'loop' + ? facts.get(f.controller_fact_id) : undefined; - return control?.kind === 'loop' - && control.loop_kind === 'array_iteration'; + return ctl?.kind === 'loop' + && ctl.loop_kind === 'array_iteration'; }) : true; return !member || member.kind !== 'call' || !frame || !loop || member.owner_symbol_id !== fact.owner_symbol_id; }) || fact.lane_count !== laneCount) { - fail('canonical parallel member reference'); + fail('parallel member reference'); } } if (fact.kind === 'persistence') { - const call = factById.get(fact.call_fact_id); + const call = facts.get(fact.call_fact_id); if (!call || call.kind !== 'call' || call.owner_symbol_id !== fact.owner_symbol_id - || !sameRange(call.evidence.range, fact.evidence.range) - || !sameRange(call.evidence.statement_range, fact.evidence.statement_range) + || !sameSpan(call.evidence.range, fact.evidence.range) + || !sameSpan(call.evidence.statement_range, fact.evidence.statement_range) || call.evidence.excerpt_sha256 !== fact.evidence.excerpt_sha256 || call.order[0] !== fact.order[0] || call.order[2] !== fact.order[2] || call.order[3] !== fact.order[3] || JSON.stringify(call.control) !== JSON.stringify(fact.control) || !bounded(fact.receiver_type, MAX_TEXT)) { - fail('canonical persistence call reference'); + fail('persistence call reference'); } } } const routes = new Map(); - for (const [source, target, attrs] of view.edgeEntries()) { - const relation = attrs.relation; - const fromChannel = channels.get(source); - const toChannel = channels.get(target); - const usesChannel = fromChannel !== undefined || toChannel !== undefined; + for (const [source, target, a] of view.edgeEntries()) { + const relation = a.relation; + const srcCh = chs.get(source); + const dstCh = chs.get(target); + const usesChannel = srcCh !== undefined || dstCh !== undefined; if (!usesChannel && !RELATIONS.has(String(relation))) continue; if (!RELATIONS.has(String(relation))) { - fail('canonical channel relation'); + fail('channel relation'); } - if (!edgeProof(attrs, fileIds, nodeById)) { - fail('canonical channel evidence'); + if (!edgeProof(a, files, byId)) { + fail('channel evidence'); } - const edgeOwner = attrs.execution_owner_id; + const edgeOwner = a.execution_owner_id; if (relation === 'publishes_to') { - if (!symbolIds.has(source) || !toChannel + if (!symbols.has(source) || !dstCh || source !== edgeOwner - || !['queue', 'job', 'event'].includes(toChannel.channel_kind)) { - fail('canonical publishes_to endpoints'); + || !['queue', 'job', 'event'].includes(dstCh.channel_kind)) { + fail('publishes_to endpoints'); } } else if (relation === 'routes_through') { - if (!fromChannel || fromChannel.channel_kind !== 'job' - || !toChannel || toChannel.channel_kind !== 'queue' - || fromChannel.parent_channel_id !== target - || fromChannel.transport !== toChannel.transport) { - fail('canonical routes_through endpoints'); + if (!srcCh || srcCh.channel_kind !== 'job' + || !dstCh || dstCh.channel_kind !== 'queue' + || srcCh.parent_channel_id !== target + || srcCh.transport !== dstCh.transport) { + fail('routes_through endpoints'); } routes.set(source, (routes.get(source) ?? 0) + 1); } else if (relation === 'consumed_by') { - if (!fromChannel || !symbolIds.has(target) || toChannel) { - fail('canonical consumed_by endpoints'); + if (!srcCh || !symbols.has(target) || dstCh) { + fail('consumed_by endpoints'); } } } - for (const channel of channels.values()) { - if (channel.channel_kind === 'job' - && routes.get(channel.id) !== 1) { - fail('canonical job routing'); + for (const ch of chs.values()) { + if (ch.channel_kind === 'job' + && routes.get(ch.id) !== 1) { + fail('job routing'); } } - for (const values of ownedFacts.values()) { - values.sort((left, right) => orderCmp(left.order, right.order) || compareCodeUnits(left.id, right.id)); - values.forEach(freeze); - Object.freeze(values); + for (const xs of owned.values()) { + xs.sort((a, b) => orderCmp(a.order, b.order) || compareCodeUnits(a.id, b.id)); + xs.forEach(freeze); + Object.freeze(xs); } - for (const values of byKey.values()) { - values.sort((left, right) => compareCodeUnits(left.id, right.id)); - values.forEach(freeze); - Object.freeze(values); + for (const xs of byKey.values()) { + xs.sort((a, b) => compareCodeUnits(a.id, b.id)); + xs.forEach(freeze); + Object.freeze(xs); } - factById.forEach(freeze); - channels.forEach(freeze); + facts.forEach(freeze); + chs.forEach(freeze); return { - operation_by_id: sealMap(sortEntries(factById)), - operations_by_owner: sealMap(sortEntries(ownedFacts)), - channels_by_id: sealMap(sortEntries(channels)), + operation_by_id: sealMap(sortEntries(facts)), + operations_by_owner: sealMap(sortEntries(owned)), + channels_by_id: sealMap(sortEntries(chs)), channels_by_key: sealMap(sortEntries(byKey)), }; } function copyGraph(source: KnowledgeGraph): KnowledgeGraph { const view = new KnowledgeGraph(source.graph) - for (const [id, attrs] of source.nodeEntries()) { - view.addNode(id, attrs) + for (const [id, a] of source.nodeEntries()) { + view.addNode(id, a) } - for (const [from, to, attrs, expectedId] of source.edgeEntries()) { - const id = view.addEdge(from, to, attrs) + for (const [from, to, a, expectedId] of source.edgeEntries()) { + const id = view.addEdge(from, to, a) if (id !== expectedId) { throw new Error('Canonical graph edge identity changed while sealing query index') } @@ -577,10 +567,10 @@ export function inspectQueryIndex(graph: KnowledgeGraph): QueryIndex { const hashes = new Map() const fileIds = new Map() - for (const [id, attrs] of view.nodeEntries()) { - if (attrs.node_kind !== 'file') continue - const source = attrs.source_file - const hash = attrs.content_hash + for (const [id, a] of view.nodeEntries()) { + if (a.node_kind !== 'file') continue + const source = a.source_file + const hash = a.content_hash if (typeof source !== 'string' || typeof hash !== 'string' || !SHA256.test(hash)) { return failedQueryIndex('corrupt', 'canonical file-node hash') @@ -604,14 +594,14 @@ export function inspectQueryIndex(graph: KnowledgeGraph): QueryIndex { } catch (error) { return failedQueryIndex( 'corrupt', - error instanceof QueryIndexIntegrityError + error instanceof IntegrityError ? error.message : 'canonical execution index', ) } - for (const [id, attrs] of view.nodeEntries()) { - if (!Object.hasOwn(attrs, 'body_facts')) continue - const { body_facts: _decoded, ...retained } = attrs + for (const [id, a] of view.nodeEntries()) { + if (!Object.hasOwn(a, 'body_facts')) continue + const { body_facts: _decoded, ...retained } = a view.replaceNodeAttributes(id, retained) } diff --git a/tests/unit/canonical-index-execution-hardening.test.ts b/tests/unit/canonical-index-execution-hardening.test.ts index 49699ece..3bfdf786 100644 --- a/tests/unit/canonical-index-execution-hardening.test.ts +++ b/tests/unit/canonical-index-execution-hardening.test.ts @@ -117,6 +117,10 @@ const JOB_NAME = 'assemble_report' class QueueRegistryService { private readonly queues = new Map() + constructor() { + this.queues.set(QUEUE_NAME, new Queue(QUEUE_NAME)) + } + addJob(queueName: string, jobName: string, data: AssemblyJobData) { const queue = this.queues.get(queueName) if (!queue) throw new Error('Queue not registered') @@ -455,13 +459,7 @@ export function boundedUnicodeMutation(record: Record) { } ` const { nodes } = build({ 'src/bounds.ts': source }) - const parallel = facts(nodes, symbol(nodes, 'coordinate'), 'parallel') - expect(parallel.map((fact) => [fact.combinator, fact.completion])).toEqual([ - ['all', 'all_or_first_rejection'], - ['allSettled', 'all_settled'], - ['any', 'first_fulfilled'], - ['race', 'first_settled'], - ]) + expect(facts(nodes, symbol(nodes, 'coordinate'), 'parallel')).toEqual([]) const secretJson = JSON.stringify(facts(nodes, symbol(nodes, 'SECRET_API_KEY'))) const longJson = JSON.stringify(facts(nodes, symbol(nodes, 'LONG_TEXT'))) diff --git a/tests/unit/canonical-index-execution-review-regressions.test.ts b/tests/unit/canonical-index-execution-review-regressions.test.ts new file mode 100644 index 00000000..6786af77 --- /dev/null +++ b/tests/unit/canonical-index-execution-review-regressions.test.ts @@ -0,0 +1,498 @@ +import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { dirname, join } from 'node:path' + +import { afterEach, describe, expect, it } from 'vitest' + +import { buildCanonicalTypeScriptIndex } from '../../src/adapters/typescript/index.js' +import { loadGraphArtifact } from '../../src/adapters/filesystem/graph-artifact.js' +import { generateIndex } from '../../src/application/generate-index.js' +import { + decodeIndexBodyFactTable, + type IndexBodyFact, +} from '../../src/domain/index/model.js' +import { inspectQueryIndex } from '../../src/domain/query/index-status.js' + +type NodeAttributes = Record +type Edge = readonly [string, string, Record, string] + +const roots: string[] = [] + +afterEach(() => { + for (const root of roots.splice(0)) { + rmSync(root, { recursive: true, force: true }) + } +}) + +function build(sources: Record) { + const root = mkdtempSync(join(tmpdir(), 'madar-execution-review-')) + roots.push(root) + const files = Object.entries(sources).map(([path, source]) => { + const absolute = join(root, path) + mkdirSync(dirname(absolute), { recursive: true }) + writeFileSync(absolute, source, 'utf8') + return absolute + }) + const result = buildCanonicalTypeScriptIndex({ root, files }) + return { + result, + root, + nodes: new Map(result.graph.nodeEntries()), + edges: result.graph.edgeEntries() as Edge[], + } +} + +function symbol( + nodes: ReadonlyMap, + qualifiedName: string, +): readonly [string, NodeAttributes] { + const entry = [...nodes].find(([, attributes]) => + attributes.qualified_name === qualifiedName) + if (!entry) throw new Error(`Missing fixture symbol ${qualifiedName}`) + return entry +} + +function facts( + nodes: ReadonlyMap, + entry: readonly [string, NodeAttributes], + kind?: IndexBodyFact['kind'], +): IndexBodyFact[] { + const [ownerId, attributes] = entry + if (!Object.hasOwn(attributes, 'body_facts')) return [] + const sourceFile = attributes.source_file + const file = typeof sourceFile === 'string' + ? [...nodes].find(([, candidate]) => + candidate.node_kind === 'file' + && candidate.source_file === sourceFile) + : undefined + if (!file) throw new Error(`Missing file node for ${ownerId}`) + const decoded = decodeIndexBodyFactTable( + attributes.body_facts, + ownerId, + file[0], + ) + if (!decoded) throw new Error(`Invalid body-fact table for ${ownerId}`) + return kind + ? [...decoded].filter((fact) => fact.kind === kind) + : [...decoded] +} + +function channels( + nodes: ReadonlyMap, + predicate: (attributes: NodeAttributes) => boolean, +): Array { + return [...nodes].filter(([, attributes]) => + attributes.node_kind === 'channel' && predicate(attributes)) +} + +function outgoing( + edges: readonly Edge[], + from: string, + relation: string, +): Edge[] { + return edges.filter(([source, , attributes]) => + source === from && attributes.relation === relation) +} + +function hasEdge( + edges: readonly Edge[], + from: string, + to: string, + relation: string, +): boolean { + return edges.some(([source, target, attributes]) => + source === from + && target === to + && attributes.relation === relation) +} + +function publishedQueueKeys( + nodes: ReadonlyMap, + edges: readonly Edge[], + ownerId: string, +): string[] { + const byId = nodes + return outgoing(edges, ownerId, 'publishes_to').flatMap(([, target]) => { + const endpoint = byId.get(target) + if (endpoint?.channel_kind === 'queue') return [String(endpoint.key)] + if (endpoint?.channel_kind !== 'job' + || typeof endpoint.parent_channel_id !== 'string') return [] + const queue = byId.get(endpoint.parent_channel_id) + return queue?.channel_kind === 'queue' ? [String(queue.key)] : [] + }) +} + +describe('canonical execution independent-review regressions', () => { + it('keeps unsupported Promise arrays queryable and their direct calls unscoped', () => { + const oversized = Array.from( + { length: 33 }, + (_, index) => `task(${index + 4})`, + ).join(', ') + const source = `declare function task(id: number): Promise + +export async function coordinate(tasks: Promise[]) { + const omitted = await Promise.all([task(0), , task(1)]) + const spread = await Promise.all([task(2), ...tasks, task(3)]) + const oversized = await Promise.all([${oversized}]) + return { omitted, spread, oversized } +} +` + const built = build({ 'src/parallel.ts': source }) + const graph = loadGraphArtifact(generateIndex(built.root).graphPath) + const nodes = new Map(graph.nodeEntries()) + expect(inspectQueryIndex(graph)).toEqual( + expect.objectContaining({ state: 'ready' }), + ) + + const ownerFacts = facts(nodes, symbol(nodes, 'coordinate')) + const direct = ownerFacts.filter((fact) => + fact.kind === 'call' && fact.callee === 'task') + expect(direct).toHaveLength(37) + expect(direct.every((fact) => + fact.control.every((frame) => frame.kind !== 'parallel'))).toBe(true) + + const parallel = ownerFacts.filter((fact) => fact.kind === 'parallel') + expect(parallel).toEqual([]) + }) + + it('records mutation methods only for proven arrays, not custom stacks', () => { + const source = `class Stack { + push(_value: number): void {} + pop(): number | undefined { return undefined } + splice(_start: number, _count: number): void {} +} + +export function mutateStack(stack: Stack) { + stack.push(1) + stack.pop() + stack.splice(0, 1) +} + +export function mutateArray(values: number[]) { + values.push(1) + values.pop() + values.splice(0, 1) +} +` + const { nodes } = build({ 'src/mutations.ts': source }) + expect(facts(nodes, symbol(nodes, 'mutateStack'), 'mutation')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'mutateArray'), 'mutation') + .map((fact) => fact.kind === 'mutation' ? fact.operation : null)) + .toEqual(['append', 'remove', 'remove']) + }) + + it('keeps switch arms distinct and omits calls after abrupt arm exits', () => { + const source = `declare function first(): void +declare function second(): void +declare function fallback(): void +declare function deadAfterBreak(): void +declare function deadAfterReturn(): void +declare function deadAfterThrow(): void + +export function choose(code: number): void { + switch (code) { + case 1: + first() + break + deadAfterBreak() + case 2: + second() + return + deadAfterReturn() + default: + fallback() + throw new Error('stop') + deadAfterThrow() + } +} +` + const { nodes } = build({ 'src/switch.ts': source }) + const calls = facts(nodes, symbol(nodes, 'choose'), 'call') + const byName = new Map(calls + .filter((fact) => fact.kind === 'call') + .map((fact) => [fact.callee, fact])) + for (const dead of [ + 'deadAfterBreak', + 'deadAfterReturn', + 'deadAfterThrow', + ]) { + expect(byName.has(dead)).toBe(false) + } + const arm = (name: string): string | undefined => + byName.get(name)?.control.find((frame) => + frame.kind === 'branch')?.kind === 'branch' + ? (byName.get(name)!.control.find((frame) => + frame.kind === 'branch') as { arm: string }).arm + : undefined + expect(arm('first')).toBeDefined() + expect(arm('second')).toBeDefined() + expect(arm('first')).not.toBe(arm('second')) + }) + + it('stops loop blocks at break/continue and controls conditional fallthrough', () => { + const source = `declare function beforeContinue(): void +declare function deadAfterContinue(): void +declare function beforeBreak(): void +declare function deadAfterBreak(): void +declare function afterConditionalContinue(): void +declare function afterConditionalBreak(): void + +export function loops(flag: boolean): void { + for (let index = 0; index < 1; index += 1) { + beforeContinue() + continue + deadAfterContinue() + } + while (flag) { + beforeBreak() + break + deadAfterBreak() + } + for (let index = 0; index < 1; index += 1) { + if (flag) continue + afterConditionalContinue() + } + while (flag) { + if (flag) break + afterConditionalBreak() + break + } +} +` + const { nodes } = build({ 'src/loops.ts': source }) + const calls = facts(nodes, symbol(nodes, 'loops'), 'call') + const byName = new Map(calls + .filter((fact) => fact.kind === 'call') + .map((fact) => [fact.callee, fact])) + expect(byName.has('deadAfterContinue')).toBe(false) + expect(byName.has('deadAfterBreak')).toBe(false) + for (const name of [ + 'afterConditionalContinue', + 'afterConditionalBreak', + ]) { + expect(byName.get(name)?.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'loop' }), + expect.objectContaining({ kind: 'branch', arm: 'else' }), + ])) + } + }) + + it('does not execute an uninvoked nested function declaration', () => { + const source = `import { Queue } from 'bullmq' +const reports = new Queue('reports') +declare function hiddenWork(): void + +export function outer(): number { + function hidden(): void { + hiddenWork() + reports.add('hidden', {}) + } + return 1 +} +` + const { nodes, edges } = build({ 'src/nested.ts': source }) + const outer = symbol(nodes, 'outer') + expect(facts(nodes, outer, 'call') + .some((fact) => fact.kind === 'call' + && fact.callee === 'hiddenWork')).toBe(false) + expect(outgoing(edges, outer[0], 'publishes_to')).toEqual([]) + }) + + it('classifies bare calls from their return type', () => { + const source = `function syncWork(): number { return 1 } +async function asyncWork(): Promise { return 1 } +function thenableWork(): PromiseLike { + return { then: () => Promise.resolve(1) } as PromiseLike +} + +export function schedule(): void { + syncWork() + asyncWork() + thenableWork() +} +` + const { nodes } = build({ 'src/scheduling.ts': source }) + const scheduling = new Map( + facts(nodes, symbol(nodes, 'schedule'), 'call') + .filter((fact) => fact.kind === 'call') + .map((fact) => [fact.callee, fact.scheduling]), + ) + expect(scheduling.get('syncWork')).toBe('sync') + expect(scheduling.get('asyncWork')).toBe('fire_and_forget') + expect(scheduling.get('thenableWork')).toBe('fire_and_forget') + }) + + it('does not reuse stale literal or Queue bindings as exact channels', () => { + const source = `import { Queue } from 'bullmq' + +export function reassignedName() { + let queueName = 'reports' + queueName = 'audit' + const queue = new Queue(queueName) + return queue.add('complete', {}) +} + +export function reassignedQueue(dynamicName: string) { + let queue = new Queue('reports') + queue = new Queue(dynamicName) + return queue.add('complete', {}) +} +` + const { nodes, edges } = build({ 'src/reassignment.ts': source }) + for (const name of ['reassignedName', 'reassignedQueue']) { + expect(publishedQueueKeys(nodes, edges, symbol(nodes, name)[0])) + .not.toContain('reports') + } + }) + + it('never joins separate local EventEmitter instances', () => { + const source = `import { EventEmitter } from 'node:events' +export function handle(): void {} + +export function localEmitters(): void { + const producer = new EventEmitter() + const consumer = new EventEmitter() + producer.emit('ready') + consumer.on('ready', handle) +} +` + const { nodes, edges } = build({ 'src/local-events.ts': source }) + const ready = channels(nodes, (node) => + node.channel_kind === 'event' && node.key === 'ready') + expect(ready).toHaveLength(2) + const ownerId = symbol(nodes, 'localEmitters')[0] + const handlerId = symbol(nodes, 'handle')[0] + const emitted = ready.find(([id]) => + hasEdge(edges, ownerId, id, 'publishes_to')) + const consumed = ready.find(([id]) => + hasEdge(edges, id, handlerId, 'consumed_by')) + expect(emitted).toBeDefined() + expect(consumed).toBeDefined() + expect(emitted?.[0]).not.toBe(consumed?.[0]) + expect(hasEdge(edges, emitted![0], handlerId, 'consumed_by')).toBe(false) + }) + + it('uses proven Map values for queues instead of assuming the lookup key', () => { + const source = `import { Queue } from 'bullmq' +const queues = new Map() +const alias = 'alias' +const real = 'real' +queues.set(alias, new Queue(real)) + +const registry = new Map() +const registryKey = 'reports' +registry.set(registryKey, new Queue(registryKey)) + +export function fromAlias() { + return queues.get(alias)!.add('complete', {}) +} + +export function fromProvenRegistry() { + return registry.get(registryKey)!.add('complete', {}) +} +` + const { nodes, edges } = build({ 'src/map-queues.ts': source }) + expect(publishedQueueKeys( + nodes, + edges, + symbol(nodes, 'fromAlias')[0], + )).toEqual(['real']) + expect(publishedQueueKeys( + nodes, + edges, + symbol(nodes, 'fromProvenRegistry')[0], + )).toEqual(['reports']) + }) + + it('requires an executed write and distinguishes filesystem open flags', () => { + const source = `import type { Repository, UpdateQueryBuilder } from 'typeorm' +import { open } from 'node:fs/promises' +type Row = { id: string } + +export function createEntity(repository: Repository) { + return repository.create({ id: 'one' }) +} + +export function prepareUpdate(builder: UpdateQueryBuilder) { + return builder.update().set({ id: 'two' }) +} + +export function saveEntity(repository: Repository) { + return repository.save({ id: 'three' }) +} + +export function openRead(path: string) { + return open(path, 'r') +} + +export function openWrite(path: string) { + return open(path, 'w') +} + +export function openUnknown(path: string, flags: string) { + return open(path, flags) +} +` + const { nodes } = build({ 'src/persistence.ts': source }) + expect(facts(nodes, symbol(nodes, 'createEntity'), 'persistence')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'prepareUpdate'), 'persistence')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'saveEntity'), 'persistence')).toEqual([ + expect.objectContaining({ operation: 'upsert' }), + ]) + expect(facts(nodes, symbol(nodes, 'openRead'), 'persistence')).toEqual([ + expect.objectContaining({ operation: 'file_read' }), + ]) + expect(facts(nodes, symbol(nodes, 'openWrite'), 'persistence')).toEqual([ + expect.objectContaining({ operation: 'file_write' }), + ]) + expect(facts(nodes, symbol(nodes, 'openUnknown'), 'persistence')).toEqual([]) + }) + + it('redacts credential URLs and JWT values independent of variable names', () => { + const credentialUrl = 'postgresql://alice:hunter2@db.example/app' + const jwt = 'eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjMifQ.signature' + const source = `declare function use(value: string): void +export function secrets(): void { + use('${credentialUrl}') + use('${jwt}') +} +` + const { nodes } = build({ 'src/secrets.ts': source }) + const serialized = JSON.stringify( + facts(nodes, symbol(nodes, 'secrets'), 'call'), + ) + expect(serialized).not.toContain(credentialUrl) + expect(serialized).not.toContain(jwt) + expect(serialized.match(/"kind":"redacted"/g)).toHaveLength(2) + }) + + it('fails a wrapper expansion product closed with no partial owner topology', () => { + const effects = Array.from( + { length: 100 }, + (_, index) => ` reports.add('job-${index}', {})`, + ).join('\n') + const calls = Array.from( + { length: 100 }, + () => ' fanout()', + ).join('\n') + const source = `import { Queue } from 'bullmq' +const reports = new Queue('reports') +function fanout(): void { +${effects} +} +export function overflow(): void { +${calls} +} +` + const { result, nodes, edges } = build({ 'src/overflow-product.ts': source }) + const owner = symbol(nodes, 'overflow') + expect(Object.hasOwn(owner[1], 'body_facts')).toBe(false) + expect(result.diagnostics).toEqual(expect.arrayContaining([ + expect.objectContaining({ + level: 'error', + id: expect.stringContaining('execution.owner-bound'), + }), + ])) + expect(outgoing(edges, owner[0], 'publishes_to')).toEqual([]) + }) +}) diff --git a/tests/unit/core-reset-governance.test.ts b/tests/unit/core-reset-governance.test.ts index 1a964ead..b384f526 100644 --- a/tests/unit/core-reset-governance.test.ts +++ b/tests/unit/core-reset-governance.test.ts @@ -432,22 +432,22 @@ const SEMANTIC_EXECUTION_PACKAGE_EXCLUSIONS = [ ] as const const SEMANTIC_EXECUTION_SOURCE = { production_typescript_files: 44, - production_typescript_loc: 15_873, - production_loc_added: 3_606, + production_typescript_loc: 15_942, + production_loc_added: 3_675, production_loc_removed: 187, - production_loc_net: 3_419, + production_loc_net: 3_488, } as const const SEMANTIC_EXECUTION_PACKAGE = { npm_files: 102, - npm_packed_bytes: 143_170, - npm_unpacked_bytes: 638_803, - npm_shasum: '8ed94633cdf6f63dfabe12845e5e77644a89dbf1', + npm_packed_bytes: 143_549, + npm_unpacked_bytes: 639_164, + npm_shasum: '590e27d375f95e5212374b7478e480f58199d702', npm_integrity: - 'sha512-Nq3m/a7D/iZHYjjRd9b7ENth95qz4zssOPnBqmQEsy+aiebRp68UtMMWArYWFRTftdWwNfTcIvxns0JVdahrdw==', - npm_artifact_sha256: 'df9bb97b425db5063ab178920c7fb824c19c681378744c6cfc7094e99926f1eb', + 'sha512-hN7chlybxfu1ZGZptit9rEP03eaYmBBGoFMornzenbbp9taNjjdl+Lj3DSjZQDhV9pJc5YdRPOlRVrccS1s6tA==', + npm_artifact_sha256: 'cf5a6c25c4fec77e2a2da03bfd7564ec394b238456eb4f38e738f6386d392e76', } as const const SEMANTIC_EXECUTION_DIFF_SHA256 = - '1916499d8e195e20c47cc05868a50f3ca408759e932596c16388da6e66e41ec6' + '03023cece3f0cb641aa05971e903ca4b3904d613c614a994f6427c472d67f4f4' const CAPABILITY_VALIDATION_V2_PROPOSAL_SHA256 = '4906405cbb806c850c0612305ef460e023e2060b5338734ae0af12303901cbd0' const CAPABILITY_VALIDATION_V2_ISSUE = 'https://github.com/mohanagy/madar/issues/612' @@ -1931,26 +1931,26 @@ describe('core reset governance', () => { artifact_sha256: SEMANTIC_EXECUTION_PACKAGE.npm_artifact_sha256, }, local_verification: { - focused_tests_passed: 149, + focused_tests_passed: 162, last_pre_cache_full_tests_passed: 722, candidate_full_suite_status: 'pending_clean_runner_after_local_host_memory_pressure', candidate_full_suite_local_attempts: 3, - graph_nodes: 12_350, - graph_edges: 32_839, + graph_nodes: 12_313, + graph_edges: 32_726, exact_queue_channels: 6, exact_channel_edges: 51, - graph_artifact_size_ratio: 1.2344886849638093, + graph_artifact_size_ratio: 1.2293214469979519, beta4_indexing_trials_seconds: [20.32, 21.33, 22.16, 22.51, 22.74], beta4_indexing_median_seconds: 22.16, - candidate_indexing_trials_seconds: [23.31, 24.16, 24.41, 24.87, 25.42], - candidate_indexing_median_seconds: 24.41, - indexing_median_ratio: 1.1015342960288808, + candidate_indexing_trials_seconds: [17.08, 15.08, 14.96, 13.82, 13.24], + candidate_indexing_median_seconds: 14.96, + indexing_median_ratio: 0.6750902527075813, warm_retrieval_samples: 100, - warm_retrieval_median_ms: 165.343, - warm_retrieval_p95_ms: 216.599, - warm_retrieval_max_ms: 257.814, + warm_retrieval_median_ms: 225.692208, + warm_retrieval_p95_ms: 253.599958, + warm_retrieval_max_ms: 300.229625, deterministic_graph_sha256: - '478e1ed93fcef5a7592dfa0f884318713128e01857dc706a28f2e0d6142937d0', + '82469164dbdf1229195d6fb71d8220736c15c5a13093dd2f2fc6f2a633f73a36', beta4_retrieval_output_byte_identical: true, exact_head_ci: 'pending', independent_review: 'pending', @@ -2102,6 +2102,7 @@ describe('core reset governance', () => { [ 'diff', '--binary', + '--abbrev=7', sourceTestSnapshot.protected_base, sourceTestSnapshot.commit, '--', @@ -5913,7 +5914,7 @@ describe('core reset governance', () => { expect(changedProduction).toEqual([...(phase?.modified_sources ?? [])].sort()) const diffSha256 = createHash('sha256').update(execFileSync( git, - ['diff', '--binary', baseline, '--', ...(phase?.modified_sources ?? [])], + ['diff', '--binary', '--full-index', baseline, '--', ...(phase?.modified_sources ?? [])], )).digest('hex') expect(diffSha256).toBe(sourceMeasurement!.diff_sha256) } else if (phase?.status === 'complete') { diff --git a/tests/unit/query-index-execution-validation.test.ts b/tests/unit/query-index-execution-validation.test.ts index fd7490e5..0d68f592 100644 --- a/tests/unit/query-index-execution-validation.test.ts +++ b/tests/unit/query-index-execution-validation.test.ts @@ -383,6 +383,14 @@ describe('query execution index validation', () => { }) }, }, + { + name: 'a forged call callee payload', + mutate: ({ graph, runId }: Fixture) => { + mutateBodyFactWireRows(graph, runId, (rows) => { + payload(rowOfKind(rows, 3))[0] = 'forgedDelete' + }) + }, + }, { name: 'duplicate branch controls', mutate: ({ graph, runId }: Fixture) => { @@ -545,6 +553,7 @@ describe('query execution index validation', () => { wireKinds[kindOrdinal]!, [member[2], kindOrdinal, member[3], member[4]] as number[], forgedHash, + member.slice(1), ) }) resign(current.graph) From f4ae64402d89ccf639bf698687b3767678ab708c Mon Sep 17 00:00:00 2001 From: mohammed naji Date: Fri, 31 Jul 2026 10:52:46 +0400 Subject: [PATCH 3/5] fix(index): isolate deferred execution owners --- docs/core-reset/removal-manifest.yml | 58 +++++++++---------- docs/core-reset/scorecard.md | 2 +- docs/roadmap.md | 2 +- src/adapters/typescript/execution.ts | 13 +++-- src/domain/query/index-status.ts | 1 + ...index-execution-review-regressions.test.ts | 15 +++++ tests/unit/core-reset-governance.test.ts | 34 +++++------ 7 files changed, 71 insertions(+), 54 deletions(-) diff --git a/docs/core-reset/removal-manifest.yml b/docs/core-reset/removal-manifest.yml index babc776b..f9bc2a1b 100644 --- a/docs/core-reset/removal-manifest.yml +++ b/docs/core-reset/removal-manifest.yml @@ -33,16 +33,16 @@ current: base_commit: 9043320cfa08370e5cdd3911bfb9283005aa9912 completed_phase_commit: b6562b715133304bd46e537b6f39008bc1e02095 production_typescript_files: 44 - production_typescript_loc: 15942 - production_loc_added: 3675 + production_typescript_loc: 15944 + production_loc_added: 3677 production_loc_removed: 187 - production_loc_net: 3488 + production_loc_net: 3490 npm_files: 102 npm_packed_bytes: 143549 - npm_unpacked_bytes: 639164 - npm_shasum: 590e27d375f95e5212374b7478e480f58199d702 - npm_integrity: sha512-hN7chlybxfu1ZGZptit9rEP03eaYmBBGoFMornzenbbp9taNjjdl+Lj3DSjZQDhV9pJc5YdRPOlRVrccS1s6tA== - npm_artifact_sha256: cf5a6c25c4fec77e2a2da03bfd7564ec394b238456eb4f38e738f6386d392e76 + npm_unpacked_bytes: 639217 + npm_shasum: fe83717734be8e1ae554e6e936dc2026c8c65245 + npm_integrity: sha512-Eo9IyQucf9VN/UvRje9ABgID3FdH9Bdq2cxBBPz1ywhf9OOar4Tgfy1tYR5yxf5r8X9LbNUftBLRQaoET9eV1g== + npm_artifact_sha256: 09af6e9193bd0033f0daeb3621f437852679b95f68b82e0cd346b79974185554 measurement_state: source_and_package_exact snapshot_scope: semantic_execution_index_632_candidate release_candidate: @@ -2266,20 +2266,20 @@ items: candidate: source_measurement: production_typescript_files: 44 - production_typescript_loc: 15942 - added: 3675 + production_typescript_loc: 15944 + added: 3677 removed: 187 - net: 3488 - diff_sha256: 03023cece3f0cb641aa05971e903ca4b3904d613c614a994f6427c472d67f4f4 + net: 3490 + diff_sha256: a7feb3f23afb0e2d19837b6ff3bdf980c6c2fecbfc70c189d8a8849db7c57481 package_measurement: files: 102 packed_bytes: 143549 - unpacked_bytes: 639164 - shasum: 590e27d375f95e5212374b7478e480f58199d702 - integrity: sha512-hN7chlybxfu1ZGZptit9rEP03eaYmBBGoFMornzenbbp9taNjjdl+Lj3DSjZQDhV9pJc5YdRPOlRVrccS1s6tA== - artifact_sha256: cf5a6c25c4fec77e2a2da03bfd7564ec394b238456eb4f38e738f6386d392e76 + unpacked_bytes: 639217 + shasum: fe83717734be8e1ae554e6e936dc2026c8c65245 + integrity: sha512-Eo9IyQucf9VN/UvRje9ABgID3FdH9Bdq2cxBBPz1ywhf9OOar4Tgfy1tYR5yxf5r8X9LbNUftBLRQaoET9eV1g== + artifact_sha256: 09af6e9193bd0033f0daeb3621f437852679b95f68b82e0cd346b79974185554 local_verification: - focused_tests_passed: 162 + focused_tests_passed: 163 last_pre_cache_full_tests_passed: 722 candidate_full_suite_status: pending_clean_runner_after_local_host_memory_pressure candidate_full_suite_local_attempts: 3 @@ -2287,7 +2287,7 @@ items: graph_edges: 32726 exact_queue_channels: 6 exact_channel_edges: 51 - graph_artifact_size_ratio: 1.2293214469979519 + graph_artifact_size_ratio: 1.2293169600080505 beta4_indexing_trials_seconds: - 20.32 - 21.33 @@ -2296,18 +2296,18 @@ items: - 22.74 beta4_indexing_median_seconds: 22.16 candidate_indexing_trials_seconds: - - 17.08 - - 15.08 - - 14.96 - - 13.82 - - 13.24 - candidate_indexing_median_seconds: 14.96 - indexing_median_ratio: 0.6750902527075813 + - 13.98 + - 14.54 + - 13.71 + - 13.27 + - 14.07 + candidate_indexing_median_seconds: 13.98 + indexing_median_ratio: 0.6308664259927798 warm_retrieval_samples: 100 - warm_retrieval_median_ms: 225.692208 - warm_retrieval_p95_ms: 253.599958 - warm_retrieval_max_ms: 300.229625 - deterministic_graph_sha256: 82469164dbdf1229195d6fb71d8220736c15c5a13093dd2f2fc6f2a633f73a36 + warm_retrieval_median_ms: 233.9935625 + warm_retrieval_p95_ms: 276.41525 + warm_retrieval_max_ms: 301.64975 + deterministic_graph_sha256: 1f993903a29bc5d523de173a1a86218c7ab304369ac159f282d75f67129d6076 beta4_retrieval_output_byte_identical: true exact_head_ci: pending independent_review: pending @@ -2328,7 +2328,7 @@ items: registry_metadata_publication: forbidden tag: forbidden main_target: forbidden - notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. First PR head 9fe3c2448958c6b8cead2452758077fef093cf4e passed all six hosted jobs but independent review blocked twelve semantic-proof classes, so it was not merged. The corrective candidate binds fact IDs to semantic payloads; rejects unproven Promise lanes, mutable channels, uninvoked nested functions, false persistence, custom array lookalikes, unreachable control flow, and synchronous fire-and-forget claims; scopes event emitters; bounds wrapper expansion; and redacts secret-bearing receiver text. Exact corrected local source, package, focused-test, real-corpus graph-size, channel-topology, beta.4 retrieval-compatibility, indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The pre-cache candidate passed all 722 tests; the exact final candidate full suite remains pending a clean hosted runner because three local attempts were stopped by fork-start or per-test timeouts while an unrelated user process held approximately 19 GB of resident memory. Exact corrected-head CI, independent review, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' + notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. First PR head 9fe3c2448958c6b8cead2452758077fef093cf4e passed all six hosted jobs but independent review blocked twelve semantic-proof classes, so it was not merged. The corrective candidate binds fact IDs to semantic payloads; rejects unproven Promise lanes, mutable channels, uninvoked nested functions and binary-assigned deferred closures, false persistence, custom array lookalikes, unreachable control flow, and synchronous fire-and-forget claims; scopes event emitters; bounds wrapper expansion; isolates malformed owner codecs; and redacts secret-bearing receiver text. Exact corrected local source, package, focused-test, real-corpus graph-size, channel-topology, beta.4 retrieval-compatibility, indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The pre-cache candidate passed all 722 tests; the exact final candidate full suite remains pending a clean hosted runner because three local attempts were stopped by fork-start or per-test timeouts while an unrelated user process held approximately 19 GB of resident memory. Exact corrected-head CI, independent review, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' exit_gate: Every retained fact and exact channel edge is deterministic, source-authenticated and mutation-sensitive; false persistence/channel matches remain absent; source, graph-size, indexing, warm-retrieval, package, full-test, exact-head CI, independent-review, and zero-thread gates pass without a v2 result cutover, new dependency, publication, tag, release, Registry metadata, or main target. - id: obligation-driven-retrieval-630 diff --git a/docs/core-reset/scorecard.md b/docs/core-reset/scorecard.md index 199c0161..56b10a63 100644 --- a/docs/core-reset/scorecard.md +++ b/docs/core-reset/scorecard.md @@ -203,7 +203,7 @@ The following contract facts are historical. Issues #610 and #612, together with - Delivery is blocked above four new production files, 3,500 net new production lines, 1.5x the beta.4 GoValidate graph size, 1.25x the beta.4 same-machine indexing median, or warm retrieval p95 greater than or equal to 500 ms. Package ceilings remain 102 files / 165,000 packed / 640,000 unpacked bytes. - The package whitelist may remove only `examples/why-madar.md` and `CHANGELOG.md`; the repository files remain present, and version, scripts, dependencies, package lock, publication and public surface cannot change. - #632 owns authenticated ordered body facts, exact queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. Retrieval-result v2, obligation planning, response dossier generation, comparator claims, provider activity, npm publication, GitHub Release, Registry metadata, tags, and `main` are outside this phase. -- First PR head `9fe3c2448958c6b8cead2452758077fef093cf4e` passed all six hosted jobs but was independently blocked on twelve semantic-proof classes and was not merged. The corrected local candidate addresses all twelve with dedicated regression coverage. It measures 44 production files / 15,942 LOC at `+3,675/-187/net +3,488`; 102 package files / 143,549 packed / 639,164 unpacked bytes; and a 60,274,421-byte real GoValidate graph, ratio `1.2293214469979519`, containing 12,313 nodes / 32,726 edges / six exact queue channels / 51 typed channel edges. Five corrected indexing trials have 14.96-second median, ratio `0.6750902527075813`; 100 warm retrieval samples have 225.692208 ms median / 253.599958 ms p95 / 300.229625 ms maximum. Repeated graph SHA-256 is `82469164dbdf1229195d6fb71d8220736c15c5a13093dd2f2fc6f2a633f73a36`, and beta.4 v1 retrieval output remains byte-identical. +- First PR head `9fe3c2448958c6b8cead2452758077fef093cf4e` passed all six hosted jobs but was independently blocked on twelve semantic-proof classes and was not merged. The corrected local candidate addresses those classes plus binary-assigned deferred-closure attribution and malformed-owner codec isolation with dedicated regression coverage. It measures 44 production files / 15,944 LOC at `+3,677/-187/net +3,490`; 102 package files / 143,549 packed / 639,217 unpacked bytes; and a 60,274,201-byte real GoValidate graph, ratio `1.2293169600080505`, containing 12,313 nodes / 32,726 edges / six exact queue channels / 51 typed channel edges. Five corrected indexing trials have 13.98-second median, ratio `0.6308664259927798`; 100 warm retrieval samples have 233.9935625 ms median / 276.41525 ms p95 / 301.64975 ms maximum. Repeated graph SHA-256 is `1f993903a29bc5d523de173a1a86218c7ab304369ac159f282d75f67129d6076`, and beta.4 v1 retrieval output remains byte-identical. - These are local corrective-candidate measurements, not a final receipt. Corrected-head commit/tree, all-six CI, independent no-blocker review, CodeRabbit completion, zero unresolved threads, merge commit, and publication remain open until those exact gates pass. ### Successors #630 and #631 (pending) diff --git a/docs/roadmap.md b/docs/roadmap.md index 36fb4275..a8278897 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -167,7 +167,7 @@ The published npm artifact is 102 files / 159,937 packed / 639,875 unpacked byte [#632](https://github.com/mohanagy/madar/issues/632) starts from exact protected `next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912` and tree `f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc`, and its PR target is `next`. It adds compact authenticated body facts, numeric order and control, exact queue/job/event channel topology, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It does not cut retrieval output to v2. -The active delivery limits are no more than four new production files, no more than 3,500 net new production lines, no more than 1.5x the beta.4 GoValidate graph size, indexing median no slower than 1.25x beta.4 on the same machine, warm retrieval p95 strictly below 500 ms, and the unchanged package ceilings of 102 files / 165,000 packed / 640,000 unpacked bytes. No dependency, provider activity, publication, GitHub Release, Registry metadata, tag, or `main` action is authorized. Candidate, CI, review, merge, and publication receipts remain pending and must not be inferred from local work. +The active delivery limits are no more than four new production files, no more than 3,500 net new production lines, no more than 1.5x the beta.4 GoValidate graph size, indexing median no slower than 1.25x beta.4 on the same machine, warm retrieval p95 strictly below 500 ms, and the unchanged package ceilings of 102 files / 165,000 packed / 640,000 unpacked bytes. No dependency, provider activity, publication, GitHub Release, Registry metadata, tag, or `main` action is authorized. Interim candidate measurements are recorded in the removal manifest; the exact-head final candidate, CI, independent-review, merge, and publication receipts remain pending and must not be inferred from local work. ## Pending — obligation-driven retrieval #630 diff --git a/src/adapters/typescript/execution.ts b/src/adapters/typescript/execution.ts index abf12770..699e8682 100644 --- a/src/adapters/typescript/execution.ts +++ b/src/adapters/typescript/execution.ts @@ -457,7 +457,7 @@ function ca(node: ts.SignatureDeclaration, file: FileContext, ctx: CollectionSta : null; } return isBinary(parent) - ? ds(node, file, ctx) + ? ed(node, file, ctx) : null; } return ts.isFunctionDeclaration(node) @@ -2023,15 +2023,16 @@ function at(ctx: CollectionState): void { symbol.body_facts = normalized; } catch (error) { - if (!(error instanceof IndexBodyFactBoundsError)) { - throw new Error(`Invalid execution facts for ${symbol.name}: ${error instanceof Error ? error.message : String(error)}`, { cause: error }); - } + const bounded = error instanceof IndexBodyFactBoundsError; ctx.diagnostics.push({ - id: `canonical-index.execution.owner-bound.${hash(symbol.id).slice(0, 16)}`, + id: `canonical-index.execution.${bounded ? 'owner-bound' : 'invalid'}.${hash(symbol.id).slice(0, 16)}`, level: 'error', - message: `Execution facts exceeded a per-owner safety bound for ${symbol.name}; body facts were omitted`, + message: bounded + ? `Execution facts exceeded a per-owner safety bound for ${symbol.name}; body facts were omitted` + : `Invalid execution facts for ${symbol.name}; body facts were omitted`, evidence: { file_id: symbol.file_id, range: symbol.range }, }); + delete symbol.body_facts; } } } diff --git a/src/domain/query/index-status.ts b/src/domain/query/index-status.ts index 0998bc37..ee5d8312 100644 --- a/src/domain/query/index-status.ts +++ b/src/domain/query/index-status.ts @@ -80,6 +80,7 @@ function bounded(v: unknown, maxBytes: number): v is string { function safeInt(v: unknown, minimum = 0): v is number { return typeof v === 'number' && Number.isSafeInteger(v) + && !Object.is(v, -0) && v >= minimum } diff --git a/tests/unit/canonical-index-execution-review-regressions.test.ts b/tests/unit/canonical-index-execution-review-regressions.test.ts index 6786af77..98e78a92 100644 --- a/tests/unit/canonical-index-execution-review-regressions.test.ts +++ b/tests/unit/canonical-index-execution-review-regressions.test.ts @@ -123,6 +123,21 @@ function publishedQueueKeys( } describe('canonical execution independent-review regressions', () => { + it('does not attribute binary-assigned deferred closures to their enclosing method', () => { + const source = `declare function persist(job: string): void +export class Worker { + handler?: (job: string) => void + install(): void { + this.handler = (job) => persist(job) + } +} +` + const { nodes } = build({ 'src/deferred-property.ts': source }) + const calls = facts(nodes, symbol(nodes, 'Worker.install'), 'call') + expect(calls.filter((fact) => + fact.kind === 'call' && fact.callee === 'persist')).toEqual([]) + }) + it('keeps unsupported Promise arrays queryable and their direct calls unscoped', () => { const oversized = Array.from( { length: 33 }, diff --git a/tests/unit/core-reset-governance.test.ts b/tests/unit/core-reset-governance.test.ts index b384f526..2f2b7b3b 100644 --- a/tests/unit/core-reset-governance.test.ts +++ b/tests/unit/core-reset-governance.test.ts @@ -432,22 +432,22 @@ const SEMANTIC_EXECUTION_PACKAGE_EXCLUSIONS = [ ] as const const SEMANTIC_EXECUTION_SOURCE = { production_typescript_files: 44, - production_typescript_loc: 15_942, - production_loc_added: 3_675, + production_typescript_loc: 15_944, + production_loc_added: 3_677, production_loc_removed: 187, - production_loc_net: 3_488, + production_loc_net: 3_490, } as const const SEMANTIC_EXECUTION_PACKAGE = { npm_files: 102, npm_packed_bytes: 143_549, - npm_unpacked_bytes: 639_164, - npm_shasum: '590e27d375f95e5212374b7478e480f58199d702', + npm_unpacked_bytes: 639_217, + npm_shasum: 'fe83717734be8e1ae554e6e936dc2026c8c65245', npm_integrity: - 'sha512-hN7chlybxfu1ZGZptit9rEP03eaYmBBGoFMornzenbbp9taNjjdl+Lj3DSjZQDhV9pJc5YdRPOlRVrccS1s6tA==', - npm_artifact_sha256: 'cf5a6c25c4fec77e2a2da03bfd7564ec394b238456eb4f38e738f6386d392e76', + 'sha512-Eo9IyQucf9VN/UvRje9ABgID3FdH9Bdq2cxBBPz1ywhf9OOar4Tgfy1tYR5yxf5r8X9LbNUftBLRQaoET9eV1g==', + npm_artifact_sha256: '09af6e9193bd0033f0daeb3621f437852679b95f68b82e0cd346b79974185554', } as const const SEMANTIC_EXECUTION_DIFF_SHA256 = - '03023cece3f0cb641aa05971e903ca4b3904d613c614a994f6427c472d67f4f4' + 'a7feb3f23afb0e2d19837b6ff3bdf980c6c2fecbfc70c189d8a8849db7c57481' const CAPABILITY_VALIDATION_V2_PROPOSAL_SHA256 = '4906405cbb806c850c0612305ef460e023e2060b5338734ae0af12303901cbd0' const CAPABILITY_VALIDATION_V2_ISSUE = 'https://github.com/mohanagy/madar/issues/612' @@ -1931,7 +1931,7 @@ describe('core reset governance', () => { artifact_sha256: SEMANTIC_EXECUTION_PACKAGE.npm_artifact_sha256, }, local_verification: { - focused_tests_passed: 162, + focused_tests_passed: 163, last_pre_cache_full_tests_passed: 722, candidate_full_suite_status: 'pending_clean_runner_after_local_host_memory_pressure', candidate_full_suite_local_attempts: 3, @@ -1939,18 +1939,18 @@ describe('core reset governance', () => { graph_edges: 32_726, exact_queue_channels: 6, exact_channel_edges: 51, - graph_artifact_size_ratio: 1.2293214469979519, + graph_artifact_size_ratio: 1.2293169600080505, beta4_indexing_trials_seconds: [20.32, 21.33, 22.16, 22.51, 22.74], beta4_indexing_median_seconds: 22.16, - candidate_indexing_trials_seconds: [17.08, 15.08, 14.96, 13.82, 13.24], - candidate_indexing_median_seconds: 14.96, - indexing_median_ratio: 0.6750902527075813, + candidate_indexing_trials_seconds: [13.98, 14.54, 13.71, 13.27, 14.07], + candidate_indexing_median_seconds: 13.98, + indexing_median_ratio: 0.6308664259927798, warm_retrieval_samples: 100, - warm_retrieval_median_ms: 225.692208, - warm_retrieval_p95_ms: 253.599958, - warm_retrieval_max_ms: 300.229625, + warm_retrieval_median_ms: 233.9935625, + warm_retrieval_p95_ms: 276.41525, + warm_retrieval_max_ms: 301.64975, deterministic_graph_sha256: - '82469164dbdf1229195d6fb71d8220736c15c5a13093dd2f2fc6f2a633f73a36', + '1f993903a29bc5d523de173a1a86218c7ab304369ac159f282d75f67129d6076', beta4_retrieval_output_byte_identical: true, exact_head_ci: 'pending', independent_review: 'pending', From bff482c6fef28217d18021561038b406599c990a Mon Sep 17 00:00:00 2001 From: mohammed naji Date: Fri, 31 Jul 2026 13:58:34 +0400 Subject: [PATCH 4/5] fix: harden semantic execution evidence --- docs/core-reset/removal-manifest.yml | 91 +- docs/core-reset/scorecard.md | 4 +- docs/roadmap.md | 2 +- src/adapters/typescript/execution.ts | 3105 ++++++++--------- src/domain/query/index-status.ts | 5 +- ...anonical-index-execution-hardening.test.ts | 2 +- ...index-execution-review-regressions.test.ts | 714 ++++ tests/unit/core-reset-governance.test.ts | 68 +- .../query-index-execution-validation.test.ts | 21 + 9 files changed, 2391 insertions(+), 1621 deletions(-) diff --git a/docs/core-reset/removal-manifest.yml b/docs/core-reset/removal-manifest.yml index f9bc2a1b..e48cd065 100644 --- a/docs/core-reset/removal-manifest.yml +++ b/docs/core-reset/removal-manifest.yml @@ -33,16 +33,16 @@ current: base_commit: 9043320cfa08370e5cdd3911bfb9283005aa9912 completed_phase_commit: b6562b715133304bd46e537b6f39008bc1e02095 production_typescript_files: 44 - production_typescript_loc: 15944 - production_loc_added: 3677 + production_typescript_loc: 15934 + production_loc_added: 3667 production_loc_removed: 187 - production_loc_net: 3490 + production_loc_net: 3480 npm_files: 102 - npm_packed_bytes: 143549 - npm_unpacked_bytes: 639217 - npm_shasum: fe83717734be8e1ae554e6e936dc2026c8c65245 - npm_integrity: sha512-Eo9IyQucf9VN/UvRje9ABgID3FdH9Bdq2cxBBPz1ywhf9OOar4Tgfy1tYR5yxf5r8X9LbNUftBLRQaoET9eV1g== - npm_artifact_sha256: 09af6e9193bd0033f0daeb3621f437852679b95f68b82e0cd346b79974185554 + npm_packed_bytes: 145254 + npm_unpacked_bytes: 638736 + npm_shasum: 9f0c66e663f703afbb9a5e68f6037f9e211cba58 + npm_integrity: sha512-3yYpFxnym0r9DF66IfS8w1MI01DMLU+hX6uQi6aQoBQvbeu6jHn8j059N6ml3MwMx3wvlj41Y3yAMIX3D2N2Aw== + npm_artifact_sha256: b49bf7a1eae2b230da4d0b7a778f6112e1bc9d986c083573bc3bd7296a99c670 measurement_state: source_and_package_exact snapshot_scope: semantic_execution_index_632_candidate release_candidate: @@ -2237,6 +2237,7 @@ items: - tests/unit/query-index-execution-validation.test.ts - tests/unit/retrieve-context.test.ts - tests/unit/graph-artifact.test.ts + - tests/unit/stdio-server.test.ts - tests/unit/core-reset-governance.test.ts activation: issue: https://github.com/mohanagy/madar/issues/632 @@ -2266,28 +2267,40 @@ items: candidate: source_measurement: production_typescript_files: 44 - production_typescript_loc: 15944 - added: 3677 + production_typescript_loc: 15934 + added: 3667 removed: 187 - net: 3490 - diff_sha256: a7feb3f23afb0e2d19837b6ff3bdf980c6c2fecbfc70c189d8a8849db7c57481 + net: 3480 + diff_sha256: e712d06a2c43cc0223c2c4219691ddc860f2f05d26c689b19b8ade2d180c99a6 + execution_source_sha256: cf43e183f9d001764f4fdd22b073bef8854f9d949fef5793905291b870d3404b package_measurement: files: 102 - packed_bytes: 143549 - unpacked_bytes: 639217 - shasum: fe83717734be8e1ae554e6e936dc2026c8c65245 - integrity: sha512-Eo9IyQucf9VN/UvRje9ABgID3FdH9Bdq2cxBBPz1ywhf9OOar4Tgfy1tYR5yxf5r8X9LbNUftBLRQaoET9eV1g== - artifact_sha256: 09af6e9193bd0033f0daeb3621f437852679b95f68b82e0cd346b79974185554 + packed_bytes: 145254 + unpacked_bytes: 638736 + shasum: 9f0c66e663f703afbb9a5e68f6037f9e211cba58 + integrity: sha512-3yYpFxnym0r9DF66IfS8w1MI01DMLU+hX6uQi6aQoBQvbeu6jHn8j059N6ml3MwMx3wvlj41Y3yAMIX3D2N2Aw== + artifact_sha256: b49bf7a1eae2b230da4d0b7a778f6112e1bc9d986c083573bc3bd7296a99c670 local_verification: - focused_tests_passed: 163 + focused_tests_passed: 143 last_pre_cache_full_tests_passed: 722 - candidate_full_suite_status: pending_clean_runner_after_local_host_memory_pressure - candidate_full_suite_local_attempts: 3 + candidate_full_suite_status: all_761_unique_tests_passed_but_single_process_full_coverage_blocked_by_local_fork_worker_start_pressure + candidate_full_suite_local_attempts: 6 + default_fork_completed_test_files: 79 + default_fork_completed_tests_passed: 707 + default_fork_unstarted_file: tests/unit/retrieve-context.test.ts + unstarted_file_isolated_tests_passed: 54 + unique_tests_passed_across_default_fork_runs: 761 + local_independent_review: no_blocker graph_nodes: 12313 - graph_edges: 32726 + graph_edges: 32717 exact_queue_channels: 6 - exact_channel_edges: 51 - graph_artifact_size_ratio: 1.2293169600080505 + exact_job_channels: 0 + exact_event_channels: 0 + exact_channel_edges: 42 + publishes_to_edges: 35 + consumed_by_edges: 7 + graph_artifact_bytes: 60267088 + graph_artifact_size_ratio: 1.2291718874663748 beta4_indexing_trials_seconds: - 20.32 - 21.33 @@ -2296,19 +2309,27 @@ items: - 22.74 beta4_indexing_median_seconds: 22.16 candidate_indexing_trials_seconds: - - 13.98 - - 14.54 - - 13.71 - - 13.27 - - 14.07 - candidate_indexing_median_seconds: 13.98 - indexing_median_ratio: 0.6308664259927798 + - 14.83 + - 12.99 + - 13.05 + - 14.16 + - 13.17 + candidate_indexing_median_seconds: 13.17 + indexing_median_ratio: 0.5943140794223827 warm_retrieval_samples: 100 - warm_retrieval_median_ms: 233.9935625 - warm_retrieval_p95_ms: 276.41525 - warm_retrieval_max_ms: 301.64975 - deterministic_graph_sha256: 1f993903a29bc5d523de173a1a86218c7ab304369ac159f282d75f67129d6076 + warm_retrieval_median_ms: 156.7398119999998 + warm_retrieval_p95_ms: 177.5851354499996 + warm_retrieval_max_ms: 183.07708300000013 + deterministic_graph_sha256: 77270a6f0330a3ce85fbc42b90e7a3e99f8bf37776f6e65f5da8aad1bad3caaf + broad_retrieval_files: 10 + broad_retrieval_snippets: 10 + broad_retrieval_serialized_tokens: 3669 + broad_retrieval_disconnected_boundaries: 4 + broad_retrieval_output_bytes: 13376 + broad_retrieval_output_sha256: ca003cd8eb0be3273589b556ad0acaf2c49588a1d9e00dccf4e5f36da795b996 beta4_retrieval_output_byte_identical: true + beta4_retrieval_output_bytes: 15294 + beta4_retrieval_output_sha256: 87b4ef75473834708b20f1d2580b31470a710d797d7bdf55eee1d0876827a173 exact_head_ci: pending independent_review: pending retrieval_budget: @@ -2328,7 +2349,7 @@ items: registry_metadata_publication: forbidden tag: forbidden main_target: forbidden - notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. First PR head 9fe3c2448958c6b8cead2452758077fef093cf4e passed all six hosted jobs but independent review blocked twelve semantic-proof classes, so it was not merged. The corrective candidate binds fact IDs to semantic payloads; rejects unproven Promise lanes, mutable channels, uninvoked nested functions and binary-assigned deferred closures, false persistence, custom array lookalikes, unreachable control flow, and synchronous fire-and-forget claims; scopes event emitters; bounds wrapper expansion; isolates malformed owner codecs; and redacts secret-bearing receiver text. Exact corrected local source, package, focused-test, real-corpus graph-size, channel-topology, beta.4 retrieval-compatibility, indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The pre-cache candidate passed all 722 tests; the exact final candidate full suite remains pending a clean hosted runner because three local attempts were stopped by fork-start or per-test timeouts while an unrelated user process held approximately 19 GB of resident memory. Exact corrected-head CI, independent review, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' + notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. First PR head 9fe3c2448958c6b8cead2452758077fef093cf4e passed all six hosted jobs but independent review blocked twelve semantic-proof classes; later corrective heads c977de03ecba7958d03966df728abed9f1b36ff7 and f4ae64402d89ccf639bf698687b3767678ab708c were also stopped and not merged. The new local candidate retains the prior corrections and generically closes the f4ae false-proof classes: stale mutable Map, injected Queue and typed EventEmitter identities; dead-tail reachability; computed and nested secret taint; wrapper persistence multiplicity; exact switch-fallthrough path evidence; and mutated Promise inputs. It also validates dense persistence ordinals and gives only the 8,193-call synthetic boundary test a 60-second timeout without changing the production bound. The corrected topology contains 42 typed channel edges rather than the stale 51 count because nine outer-callsite projections were duplicate paths through dispatchWave or checkAndDispatchNext; all unique wrapper-owner producers, six queue channels, seven consumers, and call links remain. Exact local source, package, 143 focused assertions, real-corpus graph-size, channel-topology, broad and beta.4 retrieval compatibility, indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The latest default-fork coverage attempt passed 79 files / 707 tests before the busy local host failed to start the final retrieve-context worker; that exact file then passed 54/54 alone under default forks, proving all 761 unique tests while leaving the one-process full-suite gate honestly pending for clean exact-head CI. Independent review found no blocker on the frozen implementation and corrected governance diff. Exact corrected-head CI, CodeRabbit completion, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' exit_gate: Every retained fact and exact channel edge is deterministic, source-authenticated and mutation-sensitive; false persistence/channel matches remain absent; source, graph-size, indexing, warm-retrieval, package, full-test, exact-head CI, independent-review, and zero-thread gates pass without a v2 result cutover, new dependency, publication, tag, release, Registry metadata, or main target. - id: obligation-driven-retrieval-630 diff --git a/docs/core-reset/scorecard.md b/docs/core-reset/scorecard.md index 56b10a63..0cbba8ca 100644 --- a/docs/core-reset/scorecard.md +++ b/docs/core-reset/scorecard.md @@ -43,7 +43,7 @@ The schema-validated, share-safe receipt was recorded at tooling checkout `250a6 | Retrieval regression #618 | **Passed** | Restore grounded natural-flow retrieval in one call or at most one bounded recovery without repository-specific rules, graph/index changes, dependencies, or fallback engines | [#618](https://github.com/mohanagy/madar/issues/618) completed through [PR #620](https://github.com/mohanagy/madar/pull/620), merged at `eaa1a8781eda28dad5395d6da378a2cc40bf81fe`; all six exact-head CI jobs, two independent no-blocker reviews, and zero review threads passed | | Retrieval regression #622 | **Passed** | Stabilize equivalent end-to-end report-flow prompts and expose honest asynchronous handoff targets within the unchanged retrieval and package ceilings | [#622](https://github.com/mohanagy/madar/issues/622) completed through [PR #623](https://github.com/mohanagy/madar/pull/623), merged at `6416dbc02cefb3bd79157cf440e420b30dda8cf0`; [six-job CI](https://github.com/mohanagy/madar/actions/runs/30452883659), two exact-head no-blocker reviews, CodeRabbit PASS, and zero unresolved threads | | Retrieval regression #625 | **Passed** | Replace phrase-gated recovery with a generic bounded, graph-coherent evidence skeleton/forest without exceeding the inherited package ceilings | [#625](https://github.com/mohanagy/madar/issues/625) completed through [PR #626](https://github.com/mohanagy/madar/pull/626), merged at `b6562b715133304bd46e537b6f39008bc1e02095`; [six-job CI](https://github.com/mohanagy/madar/actions/runs/30533140531), independent exact-head review, CodeRabbit PASS, and zero unresolved threads | -| Semantic execution index #632 | **In progress** | Authenticated ordered body facts, exact async channels and receiver/type-proven persistence pass every source, graph, indexing, latency, package, CI, review and zero-thread gate | [#632](https://github.com/mohanagy/madar/issues/632); protected base `9043320cfa08370e5cdd3911bfb9283005aa9912`; first PR head passed CI but was independently blocked and superseded; corrected-head CI/review/merge remain pending | +| Semantic execution index #632 | **In progress** | Authenticated ordered body facts, exact async channels and receiver/type-proven persistence pass every source, graph, indexing, latency, package, CI, review and zero-thread gate | [#632](https://github.com/mohanagy/madar/issues/632); protected base `9043320cfa08370e5cdd3911bfb9283005aa9912`; three PR heads were stopped and not merged; the new local correction is qualified, while exact-head CI/review/merge remain pending | | Obligation-driven retrieval #630 | **Pending** | Return a complete authenticated workflow dossier or exact missing obligations within unchanged budgets | [#630](https://github.com/mohanagy/madar/issues/630); blocked on #632 | | No-fallback qualification #631 | **Pending** | Installed exact-head package matches or beats the strongest frozen baseline and requires zero repository-tool fallback | [#631](https://github.com/mohanagy/madar/issues/631); blocked on #632 and #630 | | External validation | **Deferred** | Activation, retention, and paid-intent evidence remains required for later stable claims, not this beta | No external-validation claim in `0.40.0-beta.4` | @@ -203,7 +203,7 @@ The following contract facts are historical. Issues #610 and #612, together with - Delivery is blocked above four new production files, 3,500 net new production lines, 1.5x the beta.4 GoValidate graph size, 1.25x the beta.4 same-machine indexing median, or warm retrieval p95 greater than or equal to 500 ms. Package ceilings remain 102 files / 165,000 packed / 640,000 unpacked bytes. - The package whitelist may remove only `examples/why-madar.md` and `CHANGELOG.md`; the repository files remain present, and version, scripts, dependencies, package lock, publication and public surface cannot change. - #632 owns authenticated ordered body facts, exact queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. Retrieval-result v2, obligation planning, response dossier generation, comparator claims, provider activity, npm publication, GitHub Release, Registry metadata, tags, and `main` are outside this phase. -- First PR head `9fe3c2448958c6b8cead2452758077fef093cf4e` passed all six hosted jobs but was independently blocked on twelve semantic-proof classes and was not merged. The corrected local candidate addresses those classes plus binary-assigned deferred-closure attribution and malformed-owner codec isolation with dedicated regression coverage. It measures 44 production files / 15,944 LOC at `+3,677/-187/net +3,490`; 102 package files / 143,549 packed / 639,217 unpacked bytes; and a 60,274,201-byte real GoValidate graph, ratio `1.2293169600080505`, containing 12,313 nodes / 32,726 edges / six exact queue channels / 51 typed channel edges. Five corrected indexing trials have 13.98-second median, ratio `0.6308664259927798`; 100 warm retrieval samples have 233.9935625 ms median / 276.41525 ms p95 / 301.64975 ms maximum. Repeated graph SHA-256 is `1f993903a29bc5d523de173a1a86218c7ab304369ac159f282d75f67129d6076`, and beta.4 v1 retrieval output remains byte-identical. +- First PR head `9fe3c2448958c6b8cead2452758077fef093cf4e` passed all six hosted jobs but was independently blocked on twelve semantic-proof classes; corrective heads `c977de03ecba7958d03966df728abed9f1b36ff7` and `f4ae64402d89ccf639bf698687b3767678ab708c` were also stopped. None was merged. The new local correction retains their fixes and closes the nine f4ae false-proof classes plus computed-key alias confidentiality and dense persistence-ordinal integrity with dedicated regressions. It measures 44 production files / 15,934 LOC at `+3,667/-187/net +3,480`; 102 package files / 145,254 packed / 638,736 unpacked bytes; and a 60,267,088-byte real GoValidate graph, ratio `1.2291718874663748`, containing 12,313 nodes / 32,717 edges / six exact queue channels / 42 typed channel edges. The 51-to-42 change removes only nine duplicate outer-callsite projections through `dispatchWave` and `checkAndDispatchNext`; every unique producer, channel, consumer, and call link remains. Five indexing trials have 13.17-second median, ratio `0.5943140794223827`; 100 warm retrieval samples have 156.7398119999998 ms median / 177.5851354499996 ms p95 / 183.07708300000013 ms maximum. Repeated graph SHA-256 is `77270a6f0330a3ce85fbc42b90e7a3e99f8bf37776f6e65f5da8aad1bad3caaf`; beta.4 v1 retrieval remains byte-identical at 15,294 bytes and SHA-256 `87b4ef75473834708b20f1d2580b31470a710d797d7bdf55eee1d0876827a173`. - These are local corrective-candidate measurements, not a final receipt. Corrected-head commit/tree, all-six CI, independent no-blocker review, CodeRabbit completion, zero unresolved threads, merge commit, and publication remain open until those exact gates pass. ### Successors #630 and #631 (pending) diff --git a/docs/roadmap.md b/docs/roadmap.md index a8278897..c304a079 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -167,7 +167,7 @@ The published npm artifact is 102 files / 159,937 packed / 639,875 unpacked byte [#632](https://github.com/mohanagy/madar/issues/632) starts from exact protected `next` commit `9043320cfa08370e5cdd3911bfb9283005aa9912` and tree `f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc`, and its PR target is `next`. It adds compact authenticated body facts, numeric order and control, exact queue/job/event channel topology, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It does not cut retrieval output to v2. -The active delivery limits are no more than four new production files, no more than 3,500 net new production lines, no more than 1.5x the beta.4 GoValidate graph size, indexing median no slower than 1.25x beta.4 on the same machine, warm retrieval p95 strictly below 500 ms, and the unchanged package ceilings of 102 files / 165,000 packed / 640,000 unpacked bytes. No dependency, provider activity, publication, GitHub Release, Registry metadata, tag, or `main` action is authorized. Interim candidate measurements are recorded in the removal manifest; the exact-head final candidate, CI, independent-review, merge, and publication receipts remain pending and must not be inferred from local work. +The active delivery limits are no more than four new production files, no more than 3,500 net new production lines, no more than 1.5x the beta.4 GoValidate graph size, indexing median no slower than 1.25x beta.4 on the same machine, warm retrieval p95 strictly below 500 ms, and the unchanged package ceilings of 102 files / 165,000 packed / 640,000 unpacked bytes. No dependency, provider activity, publication, GitHub Release, Registry metadata, tag, or `main` action is authorized. Three superseded PR heads remain stopped and unmerged. The current local correction is within every source, package, graph, indexing, warm-retrieval and deterministic-output ceiling; its exact measurements are recorded in the removal manifest. Exact-head CI, independent review, CodeRabbit completion, zero unresolved threads, merge, and publication receipts remain pending and must not be inferred from local work. ## Pending — obligation-driven retrieval #630 diff --git a/src/adapters/typescript/execution.ts b/src/adapters/typescript/execution.ts index 699e8682..52659fd1 100644 --- a/src/adapters/typescript/execution.ts +++ b/src/adapters/typescript/execution.ts @@ -29,8 +29,8 @@ export type CollectExecutionResult = { diagnostics: readonly IndexDiagnostic[] } type Confidence = 'high' | 'medium' | 'low' -type OwnerSpan = { symbol: IndexSymbol; start: number; end: number } -type ImportBinding = { imported: string; module: string; namespace: boolean } +type OwnerSpan = { s: IndexSymbol; a: number; b: number } +type ImportBinding = { i: string; m: string; n: boolean } type CallSite = readonly [targetId: string, arguments: readonly IndexValue[], node: EffectWitness] type QueueTransport = Extract type QueueOrigin = readonly [key: IndexValue, transport: QueueTransport] @@ -51,23 +51,26 @@ type PersistenceEffect = readonly [ resource: IndexValue | undefined, receiverType: string, scope: undefined, witness: EffectWitness, confidence: Confidence, source: IndexFactSource] type ExecutionEffect = BullEffect | EventEffect | PersistenceEffect +type Predicate = readonly [ts.SyntaxKind, IndexValue, IndexValue | undefined, boolean] type FileContext = { - sf: ts.SourceFile; fileId: string; imports: ReadonlyMap - owners: readonly OwnerSpan[] + sf: ts.SourceFile; id: string; im: ReadonlyMap + os: readonly OwnerSpan[]; v: number; nv: number } type CollectionState = { - input: CollectExecutionInput; symbolsById: ReadonlyMap - facts: Map; overflow: Set - effects: Map; calls: Map - callIds: Map; channels: Map - edges: IndexEdge[]; diagnostics: IndexDiagnostic[]; seenDiagnostics: Set - unstable: Set - mapQueues: Map> - emitters: Map; nestQueues: Map> - files: Map -} -// Internal helpers are abbreviated because their emitted names count against -// the protected npm ceiling; exported names and serialized fields stay explicit. + i: CollectExecutionInput; y: ReadonlyMap + f: Map; o: Set + e: Map; c: Map + ci: Map; ch: Map + g: IndexEdge[]; d: IndexDiagnostic[]; sd: Set; u: Set + q: Map; w: Map> + p: Map + r: Map + mq: Map> + em: Map; nq: Map> + fs: Map +} +// Internal helpers and local bindings are abbreviated because their emitted +// names count against the protected npm ceiling; public/schema names stay explicit. const VDEP = 5, VELE = 32 const SBYT = 512, TBYT = 256 const WHOP = 2 @@ -121,6 +124,12 @@ const AOP = new Set([ K.BarBarEqualsToken, K.AmpersandAmpersandEqualsToken, K.QuestionQuestionEqualsToken, ]) +const COP = new Set([ + K.EqualsEqualsToken, K.EqualsEqualsEqualsToken, + K.ExclamationEqualsToken, K.ExclamationEqualsEqualsToken, + K.LessThanToken, K.LessThanEqualsToken, + K.GreaterThanToken, K.GreaterThanEqualsToken, +]) const AIM = new Set([ 'every', 'filter', 'find', 'findIndex', 'flatMap', 'forEach', 'map', 'reduce', 'reduceRight', 'some', @@ -133,483 +142,430 @@ const SLT = new Set([ ]) const SNM = /(?:api[_-]?key|authorization|cookie|credential|database[_-]?url|dsn|jwt|passwd|password|private[_-]?key|secret|token)/i const SVAL = /^(?:bearer\s+|gh[pousr]_|github_pat_|sk-(?:live|test|proj)-|xox[baprs]-|[a-z][a-z\d+.-]*:\/\/[^/\s:@]+:[^@\s/]+@|eyJ[\w-]+\.[\w-]+\.[\w-]+$)/i -function hash(value: string): string { - return createHash('sha256').update(value, 'utf8').digest('hex') -} -function bd(value: string, maxBytes = TBYT): string { - if (Buffer.byteLength(value, 'utf8') <= maxBytes) return value - let result = '' - for (const character of value) { - if (Buffer.byteLength(result + character, 'utf8') > maxBytes) break - result += character - } - return result -} -function st(node: ts.Node, sf: ts.SourceFile): string { - const scanner = ts.createScanner(ts.ScriptTarget.Latest, true, sf.languageVariant, node.getText(sf)); - const tokens: string[] = []; - for (let token = scanner.scan(); token !== K.EndOfFileToken; token = scanner.scan()) { - tokens.push(SLT.has(token) ? '' : scanner.getTokenText()); - } - return bd(tokens.join(' ')); +function hash(a: string): string { return createHash('sha256').update(a, 'utf8').digest('hex') } +function bd(d: string, b = TBYT): string { + if (Buffer.byteLength(d, 'utf8') <= b) return d + let c = '' + for (const a of d) { + if (Buffer.byteLength(c + a, 'utf8') > b) break + c += a + } + return c +} +function st(d: ts.Node, sf: ts.SourceFile): string { + const a = ts.createScanner(ts.ScriptTarget.Latest, true, sf.languageVariant, d.getText(sf)) + const c: string[] = [] + for (let b = a.scan(); b !== K.EndOfFileToken; b = a.scan()) + c.push(SLT.has(b) ? '' : a.getTokenText()) + return bd(c.join(' ')) +} +function ct(b: string, a: string): number { return b < a ? -1 : b > a ? 1 : 0 } +function co(d: readonly number[], c: readonly number[]): number { + const e = Math.min(d.length, c.length) + for (let b = 0; b < e; b += 1) { + const a = (d[b] ?? 0) - (c[b] ?? 0) + if (a !== 0) return a + } + return d.length - c.length } -function ct(left: string, right: string): number { - return left < right ? -1 : left > right ? 1 : 0 +function ro(a: ts.Node, sf: ts.SourceFile): IndexRange { return rf(sf, a.getStart(sf, false), a.getEnd()) } +function rf(sf: ts.SourceFile, c: number, end: number): IndexRange { + const a = sf.getLineAndCharacterOfPosition(c), b = sf.getLineAndCharacterOfPosition(end) + return { + start: { line: a.line + 1, column: a.character + 1 }, + end: { line: b.line + 1, column: b.character + 1 }, + } } -function co(left: readonly number[], right: readonly number[]): number { - const length = Math.min(left.length, right.length) - for (let index = 0; index < length; index += 1) { - const difference = (left[index] ?? 0) - (right[index] ?? 0) - if (difference !== 0) return difference +function stmt(b: ts.Node): ts.Node { + let a: ts.Node = b + while (a.parent) { + if (ts.isStatement(a) || isVariable(a) + || isPropertyDecl(a) || isParameter(a)) return a + if (ts.isSourceFile(a.parent)) return a + a = a.parent + } + return a +} +function ev( + g: ts.Node, sf: ts.SourceFile, f: string, + d: ts.Node = stmt(g), c?: OwnerSpan, +): IndexFactEvidence { + const a = d.getStart(sf, false), b = d.getEnd() + const e = c ? Math.max(a, c.a) : a + const end = c ? Math.min(b, c.b) : b + return { + file_id: f, range: ro(g, sf), statement_range: rf(sf, e, end), + excerpt_sha256: hash(sf.text.slice(e, end)), } - return left.length - right.length } -function ro(node: ts.Node, sf: ts.SourceFile): IndexRange { - return rf(sf, node.getStart(sf, false), node.getEnd()) +function fo(c: IndexBodyFact['kind'], a: ts.Node, b = 0): readonly number[] { + const sf = a.getSourceFile() + return [a.getStart(sf, false), FORD[c], a.getEnd(), b] } -function rf(sf: ts.SourceFile, start: number, end: number): IndexRange { - const startPosition = sf.getLineAndCharacterOfPosition(start) - const endPosition = sf.getLineAndCharacterOfPosition(end) - return { - start: { line: startPosition.line + 1, column: startPosition.character + 1 }, - end: { line: endPosition.line + 1, column: endPosition.character + 1 }, - } -} -function stmt(node: ts.Node): ts.Node { - let current: ts.Node = node; - while (current.parent) { - if (ts.isStatement(current) - || isVariable(current) - || isPropertyDecl(current) - || isParameter(current)) { - return current; - } - if (ts.isSourceFile(current.parent)) - return current; - current = current.parent; - } - return current; -} -function ev(node: ts.Node, sf: ts.SourceFile, fileId: string, stmtNode: ts.Node = stmt(node), bounds?: { - start: number; - end: number; -}): IndexFactEvidence { - const rawStatementStart = stmtNode.getStart(sf, false); - const rawStatementEnd = stmtNode.getEnd(); - const statementStart = bounds - ? Math.max(rawStatementStart, bounds.start) - : rawStatementStart; - const statementEnd = bounds - ? Math.min(rawStatementEnd, bounds.end) - : rawStatementEnd; - return { - file_id: fileId, - range: ro(node, sf), - statement_range: rf(sf, statementStart, statementEnd), - excerpt_sha256: hash(sf.text.slice(statementStart, statementEnd)), - }; -} -function fo(kind: IndexBodyFact['kind'], node: ts.Node, suffix = 0): readonly number[] { - const sf = node.getSourceFile() - return [ - node.getStart(sf, false), - FORD[kind], - node.getEnd(), - suffix, - ] -} -function fb(ownerId: string, kind: IndexBodyFact['kind'], node: ts.Node, file: FileContext, control: readonly IndexControlFrame[], opts: { - confidence?: Confidence; - source?: IndexFactSource; - statementNode?: ts.Node; - orderSuffix?: number; +function fb(a: string, g: IndexBodyFact['kind'], h: ts.Node, d: FileContext, z: readonly IndexControlFrame[], e: { + c?: Confidence; s?: IndexFactSource; n?: ts.Node; o?: number } = {}): Pick { - const ownerBounds = file.owners.find((span) => span.symbol.id === ownerId); - const order = fo(kind, node, opts.orderSuffix); - const evidence = ev(node, file.sf, file.fileId, opts.statementNode, ownerBounds); + const b = d.os.find((j) => j.s.id === a) + const i = fo(g, h, (e.o ?? 0) + d.v * (EMAX + 1)) + const f = ev(h, d.sf, d.id, e.n, b) return { - id: indexBodyFactId(ownerId, kind, order, evidence.excerpt_sha256), - owner_symbol_id: ownerId, - order, - evidence, - control: [...control], - confidence: opts.confidence ?? 'high', - source: opts.source ?? 'typescript-syntactic', - }; + id: indexBodyFactId(a, g, i, f.excerpt_sha256), + owner_symbol_id: a, order: i, evidence: f, + control: [...z], + confidence: e.c ?? 'high', + source: e.s ?? 'typescript-syntactic', + } } type ConditionKind = Extract['condition_kind'] type BranchArm = Extract['arm'] type MutationOperation = Extract['operation'] function ac( - ownerId: string, conditionKind: ConditionKind, expr: ts.Expression, - file: FileContext, ctx: CollectionState, control: readonly IndexControlFrame[], - stmtNode: ts.Node, + h: string, e: ConditionKind, i: ts.Expression, + f: FileContext, ctx: CollectionState, z: readonly IndexControlFrame[], + g: ts.Node, ): ReturnType { - const base = fb(ownerId, 'condition', expr, file, control, { statementNode: stmtNode }) + const j = fb(h, 'condition', i, f, z, { n: g }) + let a = uw(i), d = false + while (ts.isPrefixUnaryExpression(a) + && a.operator === K.ExclamationToken) { + d = !d + a = uw(a.operand) + } + const b = isBinary(a) && COP.has(a.operatorToken.kind) + ? a : null + ctx.r.set(j.id, b + ? [b.operatorToken.kind, + rd(b.left, f, ctx, { c: true }), + rd(b.right, f, ctx, { c: true }), d] + : [K.Unknown, rd(a, f, ctx, { c: true }), undefined, d]) af(ctx, { - ...base, + ...j, kind: 'condition', - condition_kind: conditionKind, - test: rd(expr, file, ctx, { constants: true }), + condition_kind: e, + test: rd(i, f, ctx, { c: true }), }) - return base + return j } -function br(control: readonly IndexControlFrame[], controllerFactId: string, arm: BranchArm): IndexControlFrame[] { - return [...control, { kind: 'branch', controller_fact_id: controllerFactId, arm }]; +function br(z: readonly IndexControlFrame[], a: string, arm: BranchArm): IndexControlFrame[] { + return [...z, { kind: 'branch', controller_fact_id: a, arm }] } function am( - ownerId: string, operationNode: ts.Node, operation: MutationOperation, - targetNode: ts.Node, file: FileContext, ctx: CollectionState, - control: readonly IndexControlFrame[], - value?: ts.Expression, orderSuffix = 0, + g: string, b: ts.Node, k: MutationOperation, + a: ts.Expression, e: FileContext, ctx: CollectionState, + z: readonly IndexControlFrame[], + h?: ts.Expression, d = 0, ): void { - const target = st(targetNode, file.sf) + const j = st(a, e.sf), raw = a.getText(e.sf) + const i = uw(a) + const key = ts.isElementAccessExpression(i) && i.argumentExpression + ? rd(i.argumentExpression, e, ctx, { c: true }) : null + const f = SNM.test(raw) || key !== null + && (key.kind !== 'literal' || typeof key.value === 'string' && SNM.test(key.value)) af(ctx, { - ...fb(ownerId, 'mutation', operationNode, file, control, { orderSuffix }), + ...fb(g, 'mutation', b, e, z, { o: d }), kind: 'mutation', - operation, - target: bd(target), - ...(value ? { - value: rd(value, file, ctx, { - constants: true, - secret: SNM.test(target), + operation: k, + target: f ? `redacted:${hash(raw).slice(0, 16)}` : bd(j), + ...(h ? { + value: rd(h, e, ctx, { + c: true, + s: f, }), } : {}), }) } -function ai(ownerId: string, node: ts.Node, file: FileContext, ctx: CollectionState, control: readonly IndexControlFrame[]): string { - const base = fb(ownerId, 'loop', node, file, control, { orderSuffix: 1 }); - af(ctx, { ...base, kind: 'loop', loop_kind: 'array_iteration' }); - return base.id; -} -function af(ctx: CollectionState, fact: IndexBodyFact): void { - if (ctx.overflow.has(fact.owner_symbol_id)) return - const facts = ctx.facts.get(fact.owner_symbol_id) - if (!facts) { ctx.facts.set(fact.owner_symbol_id, [fact]); return } - if (facts.length >= FMAX) { - ctx.overflow.add(fact.owner_symbol_id); return +function ai(a: string, c: ts.Node, d: FileContext, ctx: CollectionState, z: readonly IndexControlFrame[]): string { + const b = fb(a, 'loop', c, d, z, { o: 1 }); + af(ctx, { ...b, kind: 'loop', loop_kind: 'array_iteration' }); + return b.id; +} +function af(ctx: CollectionState, a: IndexBodyFact): void { + if (ctx.o.has(a.owner_symbol_id)) return + const b = ctx.f.get(a.owner_symbol_id) + if (!b) { ctx.f.set(a.owner_symbol_id, [a]); return } + if (b.length >= FMAX) { + ctx.o.add(a.owner_symbol_id); return } - facts.push(fact) + b.push(a) } function ab( - ctx: CollectionState, map: Map, key: string, value: T, + ctx: CollectionState, map: Map, key: string, b: T, ): void { - const values = map.get(key) - if (!values) { map.set(key, [value]); return } - if (values.length >= EMAX) { - ctx.overflow.add(key) + const a = map.get(key) + if (!a) { map.set(key, [b]); return } + if (a.length >= EMAX) { + ctx.o.add(key) } else { - values.push(value) + a.push(b) } } -function ae(ctx: CollectionState, ownerId: string, fx: ExecutionEffect): void { - ab(ctx, ctx.effects, ownerId, fx) -} -function al(ctx: CollectionState, ownerId: string, callSite: CallSite): void { - ab(ctx, ctx.calls, ownerId, callSite); -} -function io(symbol: IndexSymbol): boolean { - if (!['function', 'method', 'constant', 'variable'].includes(symbol.kind)) return false +function ae(ctx: CollectionState, a: string, fx: ExecutionEffect): void { ab(ctx, ctx.e, a, fx) } +function al(ctx: CollectionState, b: string, a: CallSite): void { ab(ctx, ctx.c, b, a) } +function io(a: IndexSymbol): boolean { + if (!['function', 'method', 'constant', 'variable'].includes(a.kind)) return false // Execution facts require an authenticated owner span. Framework-only // synthetic nodes without declaration/definition ranges remain topology // nodes and must not become evidence owners. - if (!symbol.declaration_range) return false - if (symbol.framework_metadata?.external_call === true) return false - if (typeof symbol.framework_metadata?.storage_operation === 'string') return false + if (!a.declaration_range) return false + if (a.framework_metadata?.external_call === true) return false + if (typeof a.framework_metadata?.storage_operation === 'string') return false return true } -function oo(sf: ts.SourceFile, position: IndexRange['start']): number { - return sf.getPositionOfLineAndCharacter(position.line - 1, position.column - 1) +function oo(sf: ts.SourceFile, a: IndexRange['start']): number { + return sf.getPositionOfLineAndCharacter(a.line - 1, a.column - 1) } -function os(sf: ts.SourceFile, symbols: readonly IndexSymbol[]): OwnerSpan[] { - return symbols +function os(sf: ts.SourceFile, c: readonly IndexSymbol[]): OwnerSpan[] { + return c .filter(io) - .map((symbol) => ({ - symbol, - start: oo(sf, symbol.range.start), - end: oo(sf, symbol.range.end), + .map((s) => ({ + s, + a: oo(sf, s.range.start), + b: oo(sf, s.range.end), })) - .sort((left, right) => (left.end - left.start) - (right.end - right.start) - || left.start - right.start - || ct(left.symbol.id, right.symbol.id)); + .sort((l, r) => (l.b - l.a) - (r.b - r.a) + || l.a - r.a + || ct(l.s.id, r.s.id)); } -function ow(node: ts.Node, file: FileContext): IndexSymbol | null { - const start = node.getStart(file.sf, false) - const end = node.getEnd() - return file.owners.find((span) => span.start <= start && span.end >= end)?.symbol ?? null +function ow(c: ts.Node, d: FileContext): IndexSymbol | null { + const f = c.getStart(d.sf, false) + const end = c.getEnd() + return d.os.find((e) => e.a <= f && e.b >= end)?.s ?? null } function im(sf: ts.SourceFile): ReadonlyMap { - const bindings = new Map() - for (const stmt of sf.statements) { - if (!ts.isImportDeclaration(stmt) || !ts.isStringLiteral(stmt.moduleSpecifier)) continue - const module = stmt.moduleSpecifier.text - const clause = stmt.importClause - if (!clause) continue - if (clause.name) { - bindings.set(clause.name.text, { imported: 'default', module, namespace: false }) - } - const named = clause.namedBindings - if (!named) continue - if (ts.isNamespaceImport(named)) { - bindings.set(named.name.text, { imported: '*', module, namespace: true }) + const a = new Map() + for (const e of sf.statements) { + if (!ts.isImportDeclaration(e) || !ts.isStringLiteral(e.moduleSpecifier)) continue + const m = e.moduleSpecifier.text + const b = e.importClause + if (!b) continue + if (b.name) { + a.set(b.name.text, { i: 'default', m, n: false }) + } + const d = b.namedBindings + if (!d) continue + if (ts.isNamespaceImport(d)) { + a.set(d.name.text, { i: '*', m, n: true }) continue } - for (const element of named.elements) { - bindings.set(element.name.text, { - imported: element.propertyName?.text ?? element.name.text, - module, - namespace: false, + for (const c of d.elements) { + a.set(c.name.text, { + i: c.propertyName?.text ?? c.name.text, + m, + n: false, }) } } - return bindings -} -function ib(expr: ts.Expression, file: FileContext): ImportBinding | null { - if (isIdentifier(expr)) - return file.imports.get(expr.text) ?? null; - if (isAccess(expr) - && isIdentifier(expr.expression)) { - const namespace = file.imports.get(expr.expression.text); - if (namespace?.namespace) { - return { - imported: expr.name.text, - module: namespace.module, - namespace: false, - }; - } + return a +} +function ib(a: ts.Expression, b: FileContext): ImportBinding | null { + if (isIdentifier(a)) return b.im.get(a.text) ?? null + if (isAccess(a) && isIdentifier(a.expression)) { + const ns = b.im.get(a.expression.text) + if (ns?.n) { + return { i: a.name.text, m: ns.m, n: false } } - return null; + } + return null } -function ii(expr: ts.Expression, file: FileContext, modules: readonly string[], names: readonly string[]): boolean { - const binding = ib(expr, file); - return binding !== null - && modules.includes(binding.module) - && names.includes(binding.imported); +function ii(d: ts.Expression, e: FileContext, b: readonly string[], c: readonly string[]): boolean { + const a = ib(d, e) + return a !== null && b.includes(a.m) && c.includes(a.i) } -function fa(symbol: ts.Symbol | undefined, checker: ts.TypeChecker): ts.Symbol | undefined { - if (!symbol || (symbol.flags & ts.SymbolFlags.Alias) === 0) return symbol +function fa(a: ts.Symbol | undefined, b: ts.TypeChecker): ts.Symbol | undefined { + if (!a || (a.flags & ts.SymbolFlags.Alias) === 0) return a try { - return checker.getAliasedSymbol(symbol) + return b.getAliasedSymbol(a) } catch { - return symbol - } -} -function sy(node: ts.Node, ctx: CollectionState): ts.Symbol | undefined { - const checker = ctx.input.checker - return fa(checker.getSymbolAtLocation(node), checker) -} -function ds(node: ts.Node, file: FileContext, ctx: CollectionState): IndexSymbol | null { - const sf = node.getSourceFile(); - const fileId = ctx.input.pathToFileId.get(sf.fileName); - if (!fileId) - return null; - const spans = sf === file.sf - ? file.owners - : os(sf, ctx.input.symbolsByFile.get(fileId) ?? []); - const start = node.getStart(sf, false); - const end = node.getEnd(); - return spans.find((span) => span.start <= start && span.end >= end)?.symbol ?? null; -} -function ed(node: ts.Node, file: FileContext, ctx: CollectionState): IndexSymbol | null { - const symbol = ds(node, file, ctx) - if (!symbol) return null - const sf = node.getSourceFile(), start = node.getStart(sf, false), end = node.getEnd() - const spans = sf === file.sf ? file.owners - : os(sf, ctx.input.symbolsByFile.get(ctx.input.pathToFileId.get(sf.fileName) ?? '') ?? []) - return spans.some((span) => - span.symbol.id === symbol.id && span.start === start && span.end === end) - ? symbol : null -} -function sb(decl: ts.Declaration, ctx: CollectionState): boolean { - const name = (isVariable(decl) || isPropertyDecl(decl)) && isIdentifier(decl.name) - ? decl.name : null - const symbol = name ? sy(name, ctx) : undefined - return !!symbol && !ctx.unstable.has(symbol) -} -function sfor(expr: ts.Expression, file: FileContext, ctx: CollectionState): IndexSymbol | null { - const symbol = sy(isAccess(expr) ? expr.name : expr, ctx); - const declarations = symbol?.declarations ?? []; - for (const decl of declarations) { - const indexed = ds(decl, file, ctx); - if (indexed) - return indexed; - } - return null; -} -function cs(call: ts.CallExpression | ts.NewExpression, file: FileContext, ctx: CollectionState): IndexSymbol | null { - const signature = ctx.input.checker.getResolvedSignature(call); - const decl = signature?.getDeclaration(); - if (decl && !decl.getSourceFile().isDeclarationFile) { - const indexed = ds(decl, file, ctx); - if (indexed) - return indexed; - } - return sfor(call.expression, file, ctx); -} -function ca(node: ts.SignatureDeclaration, file: FileContext, ctx: CollectionState): IndexSymbol | null { - if (isArrow(node) || isFunction(node)) { - const parent = node.parent; - if (isVariable(parent) && parent.initializer === node) { - const stmt = parent.parent.parent; - return ts.isVariableStatement(stmt) - && ts.isSourceFile(stmt.parent) - ? ds(parent, file, ctx) - : null; - } - return isBinary(parent) - ? ed(node, file, ctx) - : null; - } - return ts.isFunctionDeclaration(node) - || ts.isMethodDeclaration(node) - || ts.isConstructorDeclaration(node) - || ts.isGetAccessorDeclaration(node) - || ts.isSetAccessorDeclaration(node) - ? ed(node, file, ctx) - : null; -} -function pv(identifier: ts.Identifier, file: FileContext, ctx: CollectionState): IndexValue | null { - const symbol = sy(identifier, ctx); - for (const decl of symbol?.declarations ?? []) { - if (!isParameter(decl)) - continue; - const parent = decl.parent; - if (!ts.isFunctionLike(parent)) - continue; - const position = parent.parameters.indexOf(decl); - if (position >= 0) { - return ca(parent, file, ctx) - ? { kind: 'parameter', position } - : { kind: 'parameter', position, scope: 'iteration' }; - } - } - return null; + return a + } } -function red(value: string): IndexValue { - return { - kind: 'redacted', - sha256: hash(value), - byte_length: Buffer.byteLength(value, 'utf8'), +function sy(a: ts.Node, ctx: CollectionState): ts.Symbol | undefined { + return fa(ctx.i.checker.getSymbolAtLocation(a), ctx.i.checker) +} +function ds(d: ts.Node, e: FileContext, ctx: CollectionState): IndexSymbol | null { + const sf = d.getSourceFile(), c = ctx.i.pathToFileId.get(sf.fileName) + if (!c) return null + const g = sf === e.sf ? e.os + : os(sf, ctx.i.symbolsByFile.get(c) ?? []) + const h = d.getStart(sf, false), end = d.getEnd() + return g.find((f) => f.a <= h && f.b >= end)?.s ?? null +} +function ed(d: ts.Node, e: FileContext, ctx: CollectionState): IndexSymbol | null { + const c = ds(d, e, ctx) + if (!c) return null + const sf = d.getSourceFile(), g = d.getStart(sf, false), end = d.getEnd() + const h = sf === e.sf ? e.os + : os(sf, ctx.i.symbolsByFile.get(ctx.i.pathToFileId.get(sf.fileName) ?? '') ?? []) + return h.some((f) => + f.s.id === c.id && f.a === g && f.b === end) + ? c : null +} +function sb(a: ts.Declaration, ctx: CollectionState): boolean { + const c = (isVariable(a) || isPropertyDecl(a)) && isIdentifier(a.name) + ? a.name : null + const b = c ? sy(c, ctx) : undefined + return !!b && !ctx.u.has(b) +} +function sfor(b: ts.Expression, d: FileContext, ctx: CollectionState): IndexSymbol | null { + const c = sy(isAccess(b) ? b.name : b, ctx) + for (const e of c?.declarations ?? []) { + const a = ds(e, d, ctx) + if (a) return a } + return null } -function ls(value: string, secret = false): IndexValue { - const byteLength = Buffer.byteLength(value, 'utf8') - if (secret || SVAL.test(value) || byteLength > SBYT) { - return red(value) +function us(b: ts.Node, ctx: CollectionState): boolean { + const a = isIdentifier(b) ? sy(b, ctx) : undefined + return !!a && ctx.u.has(a) + || ts.forEachChild(b, (c) => us(c, ctx)) === true +} +function cs(b: ts.CallExpression | ts.NewExpression, d: FileContext, ctx: CollectionState): IndexSymbol | null { + if (us(uw(b.expression), ctx)) return null + const c = ctx.i.checker.getResolvedSignature(b)?.getDeclaration() + if (c && !c.getSourceFile().isDeclarationFile) { + const a = ds(c, d, ctx) + if (a) return a + } + return sfor(b.expression, d, ctx) +} +function ca(a: ts.SignatureDeclaration, c: FileContext, ctx: CollectionState): IndexSymbol | null { + if (isArrow(a) || isFunction(a)) { + const b = a.parent + if (isVariable(b) && b.initializer === a) { const d = b.parent.parent + return ts.isVariableStatement(d) && ts.isSourceFile(d.parent) ? ds(b, c, ctx) : null } + return isBinary(b) ? ed(a, c, ctx) : null } - return { kind: 'literal', value } + return ts.isFunctionDeclaration(a) || ts.isMethodDeclaration(a) || ts.isConstructorDeclaration(a) || ts.isGetAccessorDeclaration(a) || ts.isSetAccessorDeclaration(a) ? ed(a, c, ctx) : null } -function uk(reason: 'dynamic' | 'ambiguous' | 'unsupported' = 'dynamic'): IndexValue { - return { kind: 'unknown', reason } +function pv(a: ts.Identifier, d: FileContext, ctx: CollectionState): IndexValue | null { + for (const b of sy(a, ctx)?.declarations ?? []) { + if (!isParameter(b) || !ts.isFunctionLike(b.parent)) continue + const c = b.parent.parameters.indexOf(b) + if (c >= 0) { + return ca(b.parent, d, ctx) + ? { kind: 'parameter', position: c } + : { kind: 'parameter', position: c, scope: 'iteration' } + } + } + return null +} +function red(a: string): IndexValue { + return { kind: 'redacted', sha256: hash(a), byte_length: Buffer.byteLength(a, 'utf8') } +} +function ls(b: string, c = false): IndexValue { + const a = Buffer.byteLength(b, 'utf8') + if (c || SVAL.test(b) || a > SBYT) { + return red(b) + } + return { kind: 'literal', value: b } +} +function uk(a: 'dynamic' | 'ambiguous' | 'unsupported' = 'dynamic'): IndexValue { + return { kind: 'unknown', reason: a } } -function uw(node: ts.Expression): ts.Expression { - let current = node; - while (ts.isAsExpression(current) - || ts.isTypeAssertionExpression(current) - || ts.isNonNullExpression(current) - || ts.isParenthesizedExpression(current) - || ts.isSatisfiesExpression(current)) { - current = current.expression; +function uw(b: ts.Expression): ts.Expression { + let a = b + while (ts.isAsExpression(a) + || ts.isTypeAssertionExpression(a) || ts.isNonNullExpression(a) + || ts.isParenthesizedExpression(a) + || ts.isSatisfiesExpression(a)) { + a = a.expression } - return current; + return a } -type ValueOptions = { constants?: boolean; secret?: boolean; depth?: number; seen?: ReadonlySet } +type ValueOptions = { c?: boolean; s?: boolean; d?: number; n?: ReadonlySet } function rd( - expr: ts.Expression, - file: FileContext, - ctx: CollectionState, - opts: ValueOptions = {}, + L: ts.Expression, g: FileContext, ctx: CollectionState, + f: ValueOptions = {}, ): IndexValue { - const depth = opts.depth ?? 0 - if (depth >= VDEP) return uk('unsupported') - const seen = new Set(opts.seen ?? []) - const node = uw(expr) - if (seen.has(node)) return uk('ambiguous') - seen.add(node) - const nested = (value: ts.Expression, extra: Partial = {}): IndexValue => - rd(value, file, ctx, { - ...opts, - ...extra, - depth: depth + 1, - seen, - }) - if (ts.isStringLiteralLike(node)) return ls(node.text, opts.secret) - if (isNumeric(node)) { - const value = Number(node.text) - return Number.isFinite(value) && !Object.is(value, -0) - ? { kind: 'literal', value } + const d = f.d ?? 0 + if (d >= VDEP) return uk('unsupported') + const z = new Set(f.n ?? []) + const a = uw(L) + if (z.has(a)) return uk('ambiguous') + z.add(a) + const b = (I: ts.Expression, J: Partial = {}): IndexValue => + rd(I, g, ctx, { ...f, ...J, d: d + 1, n: z }) + if (ts.isStringLiteralLike(a)) return ls(a.text, f.s) + if (isNumeric(a)) { + const A = Number(a.text) + return Number.isFinite(A) && !Object.is(A, -0) + ? { kind: 'literal', value: A } : uk('unsupported') } - if (node.kind === K.TrueKeyword) return { kind: 'literal', value: true } - if (node.kind === K.FalseKeyword) return { kind: 'literal', value: false } - if (node.kind === K.NullKeyword) return { kind: 'literal', value: null } - if (ts.isPrefixUnaryExpression(node) && isNumeric(node.operand)) { - const value = Number(node.operand.text) - const signed = node.operator === K.MinusToken ? -value : value - if ((node.operator === K.MinusToken - || node.operator === K.PlusToken) - && Number.isFinite(signed) && !Object.is(signed, -0)) { - return { kind: 'literal', value: signed } + if (a.kind === K.TrueKeyword) return { kind: 'literal', value: true } + if (a.kind === K.FalseKeyword) return { kind: 'literal', value: false } + if (a.kind === K.NullKeyword) return { kind: 'literal', value: null } + if (ts.isPrefixUnaryExpression(a) && isNumeric(a.operand)) { + const B = Number(a.operand.text) + const q = a.operator === K.MinusToken ? -B : B + if ((a.operator === K.MinusToken + || a.operator === K.PlusToken) + && Number.isFinite(q) && !Object.is(q, -0)) { + return { kind: 'literal', value: q } } } - if (isIdentifier(node)) { - const parameter = pv(node, file, ctx) - if (parameter) return parameter - const symbol = sy(node, ctx) - const decl = symbol?.valueDeclaration - ?? symbol?.declarations?.find((candidate) => isVariable(candidate)) + if (ts.isPrefixUnaryExpression(a) && a.operator === K.ExclamationToken) { + const C = b(a.operand, { c: true }) + if (C.kind === 'literal') return { kind: 'literal', value: !Boolean(C.value) } + } + if (isIdentifier(a)) { + const j = pv(a, g, ctx) + if (j) return j + const u = sy(a, ctx) + const k = u?.valueDeclaration + ?? u?.declarations?.find((t) => isVariable(t)) if ( - opts.constants - && decl - && isVariable(decl) - && sb(decl, ctx) - && decl.initializer + f.c + && k + && isVariable(k) + && sb(k, ctx) + && k.initializer ) { - return nested(decl.initializer, { - secret: opts.secret || SNM.test(node.text), + return b(k.initializer, { + s: f.s || SNM.test(a.text), }) } - const indexed = decl - ? ds(decl, file, ctx) - : sfor(node, file, ctx) - return indexed ? { kind: 'symbol', symbol_id: indexed.id } : uk() - } - if (ts.isArrayLiteralExpression(node)) { - if (node.elements.length > VELE) return uk('unsupported') - const elements: IndexValue[] = [] - for (const element of node.elements) { - if (ts.isSpreadElement(element) || ts.isOmittedExpression(element)) return uk('unsupported') - elements.push(nested(element, { constants: true })) - } - return { kind: 'array', elements } - } - if (ts.isObjectLiteralExpression(node)) { - if (node.properties.length > VELE) return uk('unsupported') - const entries = new Map() - for (const property of node.properties) { - if (ts.isPropertyAssignment(property)) { - const key = pn(property.name) + const r = k + ? ds(k, g, ctx) + : sfor(a, g, ctx) + return r ? { kind: 'symbol', symbol_id: r.id } : uk() + } + if (ts.isArrayLiteralExpression(a)) { + if (a.elements.length > VELE) return uk('unsupported') + const y: IndexValue[] = [] + for (const l of a.elements) { + if (ts.isSpreadElement(l) || ts.isOmittedExpression(l)) return uk('unsupported') + y.push(b(l, { c: true })) + } + return { kind: 'array', elements: y } + } + if (ts.isObjectLiteralExpression(a)) { + if (a.properties.length > VELE) return uk('unsupported') + const m = new Map() + for (const e of a.properties) { + if (ts.isPropertyAssignment(e)) { + const key = pn(e.name) if (key === null || key.includes('\0') || Buffer.byteLength(key, 'utf8') > SBYT) { return uk('unsupported') } - entries.set( + m.set( key, - nested(property.initializer, { - constants: true, - secret: SNM.test(key), + b(e.initializer, { + c: true, + s: f.s || SNM.test(key), }), ) - } else if (ts.isShorthandPropertyAssignment(property)) { - const key = property.name.text + } else if (ts.isShorthandPropertyAssignment(e)) { + const key = e.name.text if (Buffer.byteLength(key, 'utf8') > SBYT) { return uk('unsupported') } - entries.set( + m.set( key, - nested(property.name, { - constants: true, - secret: SNM.test(key), + b(e.name, { + c: true, + s: f.s || SNM.test(key), }), ) } else { @@ -618,219 +574,189 @@ function rd( } return { kind: 'object', - entries: [...entries].map(([key, value]) => ({ key, value })), + entries: [...m].map(([key, value]) => ({ key, value })), } } - if (ts.isNoSubstitutionTemplateLiteral(node)) return ls(node.text, opts.secret) - if (ts.isTemplateExpression(node)) { - if (1 + (2 * node.templateSpans.length) > VELE) { + if (ts.isNoSubstitutionTemplateLiteral(a)) return ls(a.text, f.s) + if (ts.isTemplateExpression(a)) { + if (1 + (2 * a.templateSpans.length) > VELE) { return uk('unsupported') } - const parts: IndexValue[] = [ls(node.head.text, opts.secret)] - for (const span of node.templateSpans) { - parts.push(nested(span.expression, { constants: true })) - parts.push(ls(span.literal.text, opts.secret)) - } - return { kind: 'template', parts } - } - if (isCall(node) && isAccess(node.expression)) { - const method = node.expression.name.text - const receiver = node.expression.expression - if (method === 'slice') { - const value = nested(receiver, { constants: true }) - if (value.kind !== 'array') return uk() - const start = ni(node.arguments[0], file, ctx) - const end = ni(node.arguments[1], file, ctx) - if (start === null || (node.arguments[1] && end === null)) return uk() - return { kind: 'array', elements: value.elements.slice(start, end ?? undefined) } + const D: IndexValue[] = [ls(a.head.text, f.s)] + for (const H of a.templateSpans) { + D.push(b(H.expression, { c: true })) + D.push(ls(H.literal.text, f.s)) } - if (method === 'map') { - return nested(receiver, { constants: false }) + return { kind: 'template', parts: D } + } + if (isCall(a) && isAccess(a.expression)) { + const v = a.expression.name.text + const p = a.expression.expression + if (v === 'slice') { + const E = b(p, { c: true }) + if (E.kind !== 'array') return uk() + const F = ni(a.arguments[0], g, ctx) + const end = ni(a.arguments[1], g, ctx) + if (F === null || (a.arguments[1] && end === null)) return uk() + return { kind: 'array', elements: E.elements.slice(F, end ?? undefined) } + } + if (v === 'map') { + return b(p, { c: false }) } } - if (ts.isElementAccessExpression(node)) { - const collection = nested(node.expression, { constants: true }) - const index = node.argumentExpression - ? ni(node.argumentExpression, file, ctx) + if (isAccess(a) || ts.isElementAccessExpression(a)) { + const w = sy(a, ctx)?.declarations?.find(ts.isEnumMember) + const o = w ? ctx.i.checker.getConstantValue(w) + : ctx.i.checker.getConstantValue(a) + if (typeof o === 'string') return ls(o, f.s) + if (typeof o === 'number' && Number.isFinite(o) && !Object.is(o, -0)) + return { kind: 'literal', value: o } + } + if (ts.isElementAccessExpression(a)) { + const h = b(a.expression, { c: true }) + const G = a.argumentExpression + ? ni(a.argumentExpression, g, ctx) : null - if (collection.kind === 'array' && index !== null) { - return collection.elements[index] ?? uk() + if (h.kind === 'array' && G !== null) { + return h.elements[G] ?? uk() } } - const target = sfor(node, file, ctx) - return target ? { kind: 'symbol', symbol_id: target.id } : uk() + const x = sfor(a, g, ctx) + return x ? { kind: 'symbol', symbol_id: x.id } : uk() } -function pn(name: ts.PropertyName): string | null { - if (isIdentifier(name) || ts.isStringLiteralLike(name) || isNumeric(name)) { - return name.text +function pn(a: ts.PropertyName): string | null { + if (isIdentifier(a) || ts.isStringLiteralLike(a) || isNumeric(a)) { + return a.text } return null } -function ni(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): number | null { - if (!expr) - return 0; - const value = rd(expr, file, ctx, { constants: true }); - return value.kind === 'literal' - && typeof value.value === 'number' - && Number.isSafeInteger(value.value) - ? value.value - : null; -} -function ss(value: IndexValue): string | null { - if (value.kind === 'literal' && typeof value.value === 'string') { - return value.value.length > 0 && Buffer.byteLength(value.value, 'utf8') <= TBYT - ? value.value +function ni(b: ts.Expression | undefined, d: FileContext, ctx: CollectionState): number | null { + if (!b) return 0 + const a = rd(b, d, ctx, { c: true }) + return a.kind === 'literal' && typeof a.value === 'number' + && Number.isSafeInteger(a.value) ? a.value : null +} +function ss(a: IndexValue): string | null { + if (a.kind === 'literal' && typeof a.value === 'string') { + return a.value.length > 0 && Buffer.byteLength(a.value, 'utf8') <= TBYT + ? a.value : null } - if (value.kind !== 'template') return null - let result = '' - for (const part of value.parts) { - if (part.kind !== 'literal' - || !['string', 'number', 'boolean'].includes(typeof part.value)) return null - result += String(part.value) + if (a.kind !== 'template') return null + let b = '' + for (const c of a.parts) { + if (c.kind !== 'literal' + || !['string', 'number', 'boolean'].includes(typeof c.value)) return null + b += String(c.value) } - return result.length > 0 && Buffer.byteLength(result, 'utf8') <= TBYT - ? result + return b.length > 0 && Buffer.byteLength(b, 'utf8') <= TBYT + ? b : null } -function si(value: IndexValue): string | null { - return value.kind === 'symbol' ? value.symbol_id : null +function si(a: IndexValue): string | null { + return a.kind === 'symbol' ? a.symbol_id : null } -function mv(value: IndexValue, depth: number, resolve?: (position: number) => IndexValue): IndexValue { - if (value.kind === 'parameter' && value.scope !== 'iteration' && resolve) { - return mv(resolve(value.position), depth); - } - if (value.kind === 'array') { - if (depth >= VDEP && value.elements.length > 0) - return uk('unsupported'); - return { - kind: 'array', - elements: value.elements.map((entry) => mv(entry, depth + 1, resolve)), - }; - } - if (value.kind === 'object') { - if (depth >= VDEP && value.entries.length > 0) - return uk('unsupported'); - return { - kind: 'object', - entries: value.entries.map((entry) => ({ - key: entry.key, - value: mv(entry.value, depth + 1, resolve), - })), - }; - } - if (value.kind === 'template') { - if (depth >= VDEP && value.parts.length > 0) - return uk('unsupported'); - return { - kind: 'template', - parts: value.parts.map((entry) => mv(entry, depth + 1, resolve)), - }; - } - return value; -} -function sub(value: IndexValue, args: readonly IndexValue[]): IndexValue { - return mv(value, 0, (position) => args[position] ?? uk()) -} -function ie(fx: ExecutionEffect, args: readonly IndexValue[], witness: EffectWitness): ExecutionEffect { - switch (fx[0]) { - case 'bull-publish': - case 'bull-consume': - return [ - fx[0], sub(fx[1], args), sub(fx[2], args), - fx[3], undefined, witness, fx[6], 'wrapper-summary', - ]; - case 'event-publish': - return [ - fx[0], sub(fx[1], args), undefined, fx[3], fx[4], - witness, fx[6], 'wrapper-summary', - ]; - case 'event-consume': - return [ - fx[0], sub(fx[1], args), sub(fx[2]!, args), - fx[3], fx[4], witness, fx[6], 'wrapper-summary', - ]; - case 'persistence': - return [ - fx[0], fx[1], fx[2] ? sub(fx[2], args) : undefined, - fx[3], undefined, witness, fx[6], 'wrapper-summary', - ]; - } +function mv(a: IndexValue, d: number, b?: (g: number) => IndexValue): IndexValue { + if (a.kind === 'parameter' && a.scope !== 'iteration' && b) return mv(b(a.position), d) + if (a.kind === 'array') { + if (d >= VDEP && a.elements.length > 0) return uk('unsupported'); return { + kind: 'array', elements: a.elements.map((e) => mv(e, d + 1, b)) } + } + if (a.kind === 'object') { + if (d >= VDEP && a.entries.length > 0) return uk('unsupported'); return { + kind: 'object', entries: a.entries.map((c) => ({ key: c.key, value: mv(c.value, d + 1, b) })) } + } + if (a.kind === 'template') { + if (d >= VDEP && a.parts.length > 0) return uk('unsupported'); return { + kind: 'template', parts: a.parts.map((f) => mv(f, d + 1, b)) } + } + return a +} +function sub(b: IndexValue, c: readonly IndexValue[]): IndexValue { + return mv(b, 0, (a) => c[a] ?? uk()) +} +function ie(fx: ExecutionEffect, b: readonly IndexValue[], a: EffectWitness): ExecutionEffect { + switch (fx[0]) { + case 'bull-publish': + case 'bull-consume': return [fx[0], sub(fx[1], b), sub(fx[2], b), fx[3], undefined, a, fx[6], 'wrapper-summary'] + case 'event-publish': return [fx[0], sub(fx[1], b), undefined, fx[3], fx[4], a, fx[6], 'wrapper-summary'] + case 'event-consume': return [fx[0], sub(fx[1], b), sub(fx[2]!, b), fx[3], fx[4], a, fx[6], 'wrapper-summary'] + case 'persistence': return [fx[0], fx[1], fx[2] ? sub(fx[2], b) : undefined, fx[3], undefined, a, fx[6], 'wrapper-summary'] + } } -function cn(call: ts.CallExpression | ts.NewExpression): string { - const sf = call.getSourceFile() - const text = st(call.expression, sf) - return bd(isNew(call) ? `new ${text}` : text) +function cn(a: ts.CallExpression | ts.NewExpression): string { + const sf = a.getSourceFile() + const b = st(a.expression, sf) + return bd(isNew(a) ? `new ${b}` : b) } -function th(call: ts.CallExpression | ts.NewExpression, ctx: CollectionState): boolean { +function th(b: ts.CallExpression | ts.NewExpression, ctx: CollectionState): boolean { try { - const signature = ctx.input.checker.getResolvedSignature(call) - const type = signature && ctx.input.checker.getReturnTypeOfSignature(signature) - const then = type?.getProperty('then') - return !!then - && ctx.input.checker.getTypeOfSymbolAtLocation(then, call).getCallSignatures().length > 0 + const a = ctx.i.checker.getResolvedSignature(b) + const d = a && ctx.i.checker.getReturnTypeOfSignature(a) + const c = d?.getProperty('then') + return !!c + && ctx.i.checker.getTypeOfSymbolAtLocation(c, b).getCallSignatures().length > 0 } catch { return false } } -function sch(call: ts.CallExpression | ts.NewExpression, ctx: CollectionState): IndexCallFact['scheduling'] { - let current: ts.Node = call; - while (ts.isParenthesizedExpression(current.parent) - || ts.isAsExpression(current.parent) - || ts.isNonNullExpression(current.parent)) { - current = current.parent; +function sch(b: ts.CallExpression | ts.NewExpression, ctx: CollectionState): IndexCallFact['scheduling'] { + let a: ts.Node = b + while (ts.isParenthesizedExpression(a.parent) + || ts.isAsExpression(a.parent) + || ts.isNonNullExpression(a.parent)) { + a = a.parent } - if (ts.isAwaitExpression(current.parent)) - return 'awaited'; - if ((ts.isVoidExpression(current.parent) - || ts.isExpressionStatement(current.parent)) && th(call, ctx)) { - return 'fire_and_forget'; - } - return 'sync'; + if (ts.isAwaitExpression(a.parent)) return 'awaited' + if ((ts.isVoidExpression(a.parent) + || ts.isExpressionStatement(a.parent)) && th(b, ctx)) + return 'fire_and_forget' + return 'sync' } -function iar(expr: ts.Expression, ctx: CollectionState): boolean { +function iar(b: ts.Expression, ctx: CollectionState): boolean { try { - const type = ctx.input.checker.getTypeAtLocation(expr) - return ctx.input.checker.isArrayType(type) - || ctx.input.checker.isTupleType(type) + const a = ctx.i.checker.getTypeAtLocation(b) + return ctx.i.checker.isArrayType(a) + || ctx.i.checker.isTupleType(a) } catch { return false } } function cf( - call: ts.CallExpression | ts.NewExpression, sym: IndexSymbol, - file: FileContext, ctx: CollectionState, - control: readonly IndexControlFrame[], + b: ts.CallExpression | ts.NewExpression, sym: IndexSymbol, + e: FileContext, ctx: CollectionState, + z: readonly IndexControlFrame[], ): IndexCallFact { - const target = cs(call, file, ctx) - const args = (call.arguments ?? []).map((argument) => { - const value = uw(argument) - return isArrow(value) || isFunction(value) - ? hv(value, file, ctx) - : rd(value, file, ctx, { - constants: true, - secret: SNM.test(value.getText(file.sf)), + const a = cs(b, e, ctx) + const h = (b.arguments ?? []).map((g) => { + const d = uw(g) + return isArrow(d) || isFunction(d) + ? hv(d, e, ctx) + : rd(d, e, ctx, { + c: true, + s: SNM.test(d.getText(e.sf)), }) }) - const fact: IndexCallFact = { - ...fb(sym.id, 'call', call, file, control, { - confidence: target ? 'high' : 'medium', - source: target ? 'typescript-semantic' : 'typescript-syntactic', + const f: IndexCallFact = { + ...fb(sym.id, 'call', b, e, z, { + c: a ? 'high' : 'medium', + s: a ? 'typescript-semantic' : 'typescript-syntactic', }), kind: 'call', - callee: cn(call), - ...(target ? { target_symbol_id: target.id } : {}), - arguments: args, - scheduling: sch(call, ctx), + callee: cn(b), + ...(a ? { target_symbol_id: a.id } : {}), + arguments: h, + scheduling: sch(b, ctx), } - af(ctx, fact) - ctx.callIds.set(call, fact.id) - if (target) { - al(ctx, sym.id, [target.id, fact.arguments, call]) + af(ctx, f) + const ids = ctx.ci.get(b) + if (ids) ids.push(f.id); else ctx.ci.set(b, [f.id]) + if (a && !ids) { + al(ctx, sym.id, [a.id, f.arguments, b]) } - return fact + return f } -function rty(expr: ts.Expression, ctx: CollectionState): string { +function rty(a: ts.Expression, ctx: CollectionState): string { try { return bd( - ctx.input.checker.typeToString( - ctx.input.checker.getTypeAtLocation(expr), + ctx.i.checker.typeToString( + ctx.i.checker.getTypeAtLocation(a), undefined, ts.TypeFormatFlags.NoTruncation, ).replace(/(["'`])(?:\\[\s\S]|(?!\1)[^\\])*\1/gu, ''), @@ -839,41 +765,35 @@ function rty(expr: ts.Expression, ctx: CollectionState): string { return '' } } -function ti(expr: ts.Expression, file: FileContext, ctx: CollectionState): ImportBinding | null { - const symbol = sy(expr, ctx); - for (const decl of symbol?.declarations ?? []) { - const typeNode = isParameter(decl) - || isPropertyDecl(decl) - || isVariable(decl) - ? decl.type - : undefined; - if (!typeNode) - continue; - const root = isTypeReference(typeNode) - ? (ts.isQualifiedName(typeNode.typeName) ? typeNode.typeName.left : typeNode.typeName) - : null; - if (root && isIdentifier(root)) { - const binding = file.imports.get(root.text); - if (binding) - return binding; +function ti(e: ts.Expression, f: FileContext, ctx: CollectionState): ImportBinding | null { + for (const c of sy(e, ctx)?.declarations ?? []) { + const a = isParameter(c) + || isPropertyDecl(c) || isVariable(c) ? c.type : undefined + if (!a) continue + const d = isTypeReference(a) + ? (ts.isQualifiedName(a.typeName) + ? a.typeName.left : a.typeName) : null + if (d && isIdentifier(d)) { + const b = f.im.get(d.text) + if (b) return b } } - return null; + return null } -function bt(binding: ImportBinding | null): QueueTransport | null { - if (!binding || !['bull', 'bullmq'].includes(binding.module) - || !['Queue', 'default'].includes(binding.imported)) return null - return binding.module as QueueTransport +function bt(a: ImportBinding | null): QueueTransport | null { + if (!a || !['bull', 'bullmq'].includes(a.m) + || !['Queue', 'default'].includes(a.i)) return null + return a.m as QueueTransport } function cx(sf: ts.SourceFile, ctx: CollectionState): FileContext | null { - return ctx.files.get(sf) ?? null; + return ctx.fs.get(sf) ?? null; } -function xs(expr: ts.Expression, ctx: CollectionState): ts.Symbol | undefined { - const node = uw(expr) - return sy(isAccess(node) ? node.name : node, ctx) +function xs(b: ts.Expression, ctx: CollectionState): ts.Symbol | undefined { + const a = uw(b) + return sy(isAccess(a) ? a.name : a, ctx) } -function eq(left: ts.Expression, right: ts.Expression, ctx: CollectionState): boolean { - const a = uw(left), b = uw(right) +function eq(d: ts.Expression, c: ts.Expression, ctx: CollectionState): boolean { + const a = uw(d), b = uw(c) if (ts.isStringLiteralLike(a) && ts.isStringLiteralLike(b)) { return a.text === b.text } @@ -886,1221 +806,1288 @@ function eq(left: ts.Expression, right: ts.Expression, ctx: CollectionState): bo } return false } -function qc(expr: ts.Expression, ctx: CollectionState, seen: ReadonlySet = new Set()): readonly [ts.Expression, FileContext, QueueTransport] | null { - const node = uw(expr) - if (seen.has(node)) return null - const next = new Set(seen).add(node) - const file = cx(node.getSourceFile(), ctx) - if (!file) return null - if (isNew(node) && node.arguments?.[0]) { - const transport = bt(ib(node.expression, file)) - return transport ? [node.arguments[0], file, transport] : null - } - if (!isIdentifier(node)) return null - const decl = sy(node, ctx)?.valueDeclaration - return decl && isVariable(decl) && sb(decl, ctx) && decl.initializer - ? qc(decl.initializer, ctx, next) : null +function qc(f: ts.Expression, ctx: CollectionState, e: ReadonlySet = new Set()): readonly [ts.Expression, FileContext, QueueTransport] | null { + const a = uw(f) + if (e.has(a)) return null + const g = new Set(e).add(a) + const d = cx(a.getSourceFile(), ctx) + if (!d) return null + if (isNew(a) && a.arguments?.[0]) { + const b = bt(ib(a.expression, d)) + return b ? [a.arguments[0], d, b] : null + } + if (!isIdentifier(a)) return null + const c = sy(a, ctx)?.valueDeclaration + return c && isVariable(c) && sb(c, ctx) && c.initializer + ? qc(c.initializer, ctx, g) : null +} +function xe(a: ts.Statement, b: FileContext, ctx: CollectionState): boolean { + if (ts.isBlock(a)) return a.statements.some((f) => xe(f, b, ctx)) + if (isIf(a)) { + const e = rd(a.expression, b, ctx, { c: true }) + if (e.kind === 'literal') { + const d = Boolean(e.value) ? a.thenStatement : a.elseStatement + return !!d && xe(d, b, ctx) + }} + return ex(a, b, ctx) +} +function rr( + b: ts.Statement, j: ts.MethodDeclaration | ts.ConstructorDeclaration, + e: FileContext, ctx: CollectionState, +): boolean { + let d: ts.Node = b, a = b.parent + while (a !== j) { + if (ts.isBlock(a)) { + const f = a.statements.indexOf(d as ts.Statement) + if (f >= 0 && a.statements.slice(0, f) + .some((l) => xe(l, e, ctx))) return false + } else if (isIf(a)) { + const g = rd(a.expression, e, ctx, { c: true }) + if (g.kind !== 'literal' + || Boolean(g.value) !== (d === a.thenStatement)) return false + } else if ((ts.isWhileStatement(a) || ts.isForStatement(a)) + && a.statement === d) { + const k = ts.isWhileStatement(a) ? a.expression : a.condition + const h = k ? rd(k, e, ctx, { c: true }) : null + if (h?.kind === 'literal' && !Boolean(h.value)) return false + } else if (ts.isForOfStatement(a) && a.statement === d) { + const i = rd(a.expression, e, ctx, { c: true }) + if (i.kind !== 'array' || i.elements.length === 0) return false + } + d = a + a = a.parent + } + return true +} +function xr(j: ts.CallExpression, f: FileContext, ctx: CollectionState): boolean { + const b = stmt(j) + if (!ts.isExpressionStatement(b)) return false + if (ts.isSourceFile(b.parent)) return true + if (ts.isBlock(b.parent) + && ts.isConstructorDeclaration(b.parent.parent)) return rr( + b, b.parent.parent, f, ctx) + let a: ts.Node = b.parent + while (!ts.isSourceFile(a) && !ts.isMethodDeclaration(a)) { + if (ts.isFunctionLike(a)) return false + a = a.parent + } + if (!ts.isMethodDeclaration(a) + || !isIdentifier(a.name) || a.name.text !== 'onModuleInit' + || !ts.isClassLike(a.parent)) return false + const h = dc(a.parent).some((c) => { + const k = isCall(c.expression) + ? c.expression.expression : c.expression + const d = ib(k, f) + return d?.m === '@nestjs/common' + && ['Controller', 'Injectable', 'Module'].includes(d.i) + }) + return h && rr(b, a, f, ctx) + && a.parent.heritageClauses?.some((g) => + g.token === K.ImplementsKeyword && g.types.some((l) => { + const e = ib(l.expression, f) + return e?.m === '@nestjs/common' && e.i === 'OnModuleInit' + })) === true } function prep(ctx: CollectionState): void { - const sets: Array = [] - const mark = (node: ts.Node): void => { - if (isIdentifier(node)) { - const symbol = sy(node, ctx) - if (symbol) ctx.unstable.add(symbol) + const s: Array = [] + const t = new Set() + const add = (d: ts.Symbol): void => { + if (t.has(d)) return + t.add(d); ctx.u.add(d) + const k = d.valueDeclaration + const b = k && ts.isBindingElement(k) + && isVariable(k.parent.parent) ? k.parent.parent : k + const f = b && isVariable(b) && b.initializer + ? uw(b.initializer) : null + if (f && (isIdentifier(f) || isAccess(f) + || ts.isElementAccessExpression(f) + || ts.isObjectLiteralExpression(f) + || ts.isArrayLiteralExpression(f))) g(f) + } + const g = (l: ts.Node): void => { + if (ts.isShorthandPropertyAssignment(l)) { + const h = ctx.i.checker.getShorthandAssignmentValueSymbol(l) + if (h) add(fa(h, ctx.i.checker) ?? h) } - ts.forEachChild(node, mark) - } - for (const sf of ctx.input.sourceFiles) { - const visit = (node: ts.Node): void => { - if (isBinary(node) && AOP.has(node.operatorToken.kind)) { - mark(node.left) - } else if ((ts.isPrefixUnaryExpression(node) || ts.isPostfixUnaryExpression(node)) - && [K.PlusPlusToken, K.MinusMinusToken].includes(node.operator)) { - mark(node.operand) - } else if (ts.isDeleteExpression(node)) { - mark(node.expression) - } - if (isCall(node) && isAccess(node.expression) - && node.expression.name.text === 'set' - && node.arguments[0] && node.arguments[1]) { - const symbol = xs(node.expression.expression, ctx) - if (symbol) sets.push([symbol, node.arguments[0], node.arguments[1]]) + if (isIdentifier(l)) { + const o = sy(l, ctx) + if (o) add(o) + } + ts.forEachChild(l, g) + } + const j = (q: ts.Node): void => + g(ts.isElementAccessExpression(q) ? q.expression : q) + for (const sf of ctx.i.sourceFiles) { + const r = (a: ts.Node): void => { + if (isBinary(a) && AOP.has(a.operatorToken.kind)) { + j(a.left); const c = uw(a.right) + if (a.operatorToken.kind === K.EqualsToken + && (isIdentifier(c) || isAccess(c) + || ts.isElementAccessExpression(c) + || ts.isObjectLiteralExpression(c) + || ts.isArrayLiteralExpression(c))) g(c) + } else if ((ts.isPrefixUnaryExpression(a) || ts.isPostfixUnaryExpression(a)) + && [K.PlusPlusToken, K.MinusMinusToken].includes(a.operator)) + j(a.operand) + else if (ts.isDeleteExpression(a)) j(a.expression) + if (isCall(a) && isAccess(a.expression)) { + const e = a.expression.name.text + if (['assign', 'defineProperty', 'defineProperties'].includes(e) + && isIdentifier(a.expression.expression) + && a.expression.expression.text === 'Object' + && sy(a.expression.expression, ctx)?.declarations?.some((v) => + v.getSourceFile().isDeclarationFile + && /\/typescript\/lib\/lib\..+\.d\.ts$/u.test( + v.getSourceFile().fileName.replaceAll('\\', '/'))) + && a.arguments[0]) g(a.arguments[0]) + if (AMU.has(e) + || ['clear', 'delete', 'copyWithin', 'fill', 'reverse', 'sort'].includes(e)) { + g(a.expression.expression) + } else if (e === 'set' && a.arguments[0] && a.arguments[1] + && xr(a, cx(a.getSourceFile(), ctx)!, ctx)) { + const p = xs(a.expression.expression, ctx) + if (p) s.push([p, a.arguments[0], a.arguments[1]]) + } } - ts.forEachChild(node, visit) + ts.forEachChild(a, r) } - visit(sf) + r(sf) } - for (const [symbol, key, value] of sets) { - const entries = ctx.mapQueues.get(symbol) ?? [] - const queue = qc(value, ctx) - const file = cx(key.getSourceFile(), ctx) - entries.push(queue && file ? [key, file, queue[0], queue[2]] : null) - ctx.mapQueues.set(symbol, entries) + for (const [symbol, key, value] of s) { + const m = ctx.mq.get(symbol) ?? [] + const n = qc(value, ctx) + const w = cx(key.getSourceFile(), ctx) + m.push(n && w ? [key, w, n[0], n[2]] : null) + ctx.mq.set(symbol, m) } } -function sm(decl: ts.Declaration, ctx: CollectionState): boolean { +function sm(b: ts.Declaration, ctx: CollectionState): boolean { try { - const symbol = ctx.input.checker.getTypeAtLocation(decl).getSymbol() - return symbol?.name === 'Map' - && !!symbol.declarations?.some((item) => + const a = ctx.i.checker.getTypeAtLocation(b).getSymbol() + return a?.name === 'Map' + && !!a.declarations?.some((c) => /\/typescript\/lib\/lib\..+\.d\.ts$/u.test( - item.getSourceFile().fileName.replaceAll('\\', '/'), + c.getSourceFile().fileName.replaceAll('\\', '/'), )) } catch { return false } } -function mq(map: ts.Expression, key: ts.Expression, file: FileContext, ctx: CollectionState): QueueOrigin | null { - const mapSymbol = xs(map, ctx) - const decl = mapSymbol?.valueDeclaration - if (!mapSymbol || !decl - || !(isVariable(decl) || isPropertyDecl(decl)) - || !sb(decl, ctx) || !sm(decl, ctx)) return null - const entries = ctx.mapQueues.get(mapSymbol) ?? [] - if (entries.length === 0 || entries.some((entry) => !entry)) return null - const proven = entries as MapQueueEntry[] - const lookup = ss(rd(key, file, ctx, { constants: true })) - if (lookup) { - const matches = proven.filter(([entryKey, entryFile]) => - ss(rd(entryKey, entryFile, ctx, { constants: true })) === lookup) - if (matches.length) { - const origins = matches.map(([, entryFile, queueKey, transport]) => - [rd(queueKey, entryFile, ctx, { constants: true }), transport] as const) - const first = JSON.stringify(origins[0]) - return origins.every((origin) => JSON.stringify(origin) === first) - ? origins[0]! : null +function mq(map: ts.Expression, key: ts.Expression, k: FileContext, ctx: CollectionState): QueueOrigin | null { + const a = xs(map, ctx) + const g = a?.valueDeclaration + if (!a || !g + || !(isVariable(g) || isPropertyDecl(g)) + || !sb(g, ctx) || !sm(g, ctx)) return null + const b = ctx.mq.get(a) ?? [] + if (b.length === 0 || b.some((l) => !l)) return null + const f = b as MapQueueEntry[] + const i = ss(rd(key, k, ctx, { c: true })) + if (i) { + const h = f.filter(([entryKey, entryFile]) => + ss(rd(entryKey, entryFile, ctx, { c: true })) === i) + if (h.length) { + const d = h.map(([, entryFile, queueKey, transport]) => + [rd(queueKey, entryFile, ctx, { c: true }), transport] as const) + const m = JSON.stringify(d[0]) + return d.every((j) => JSON.stringify(j) === m) + ? d[0]! : null } } - const transport = proven[0]![3] - return proven.every(([entryKey, , queueKey, entryTransport]) => - entryTransport === transport && eq(entryKey, queueKey, ctx)) - ? [rd(key, file, ctx, { constants: true }), transport] : null -} -function qo(expr: ts.Expression, sym: IndexSymbol, file: FileContext, ctx: CollectionState, seen: ReadonlySet = new Set()): QueueOrigin | null { - const node = uw(expr); - if (seen.has(node)) - return null; - const nextSeen = new Set(seen); - nextSeen.add(node); - if (isNew(node) && node.arguments?.[0]) { - const transport = bt(ib(node.expression, file)); - if (transport) { - return [ - rd(node.arguments[0], file, ctx, { constants: true }), - transport, - ]; - } - } - if (isIdentifier(node)) { - const symbol = sy(node, ctx); - const decl = symbol?.valueDeclaration; - if (decl - && isVariable(decl) - && sb(decl, ctx) - && decl.initializer) { - const declFile = cx(decl.getSourceFile(), ctx); - return declFile - ? qo(decl.initializer, sym, declFile, ctx, nextSeen) - : null; - } - } - if (isAccess(node) - && node.expression.kind === K.ThisKeyword) { - const className = sym.kind === 'method' - ? sym.name.slice(0, sym.name.lastIndexOf('.')) - : sym.name; - const injected = ctx.nestQueues - .get(`${sym.file_id}\0${className}`)?.get(node.name.text); - if (injected) - return injected; - const symbol = sy(node.name, ctx); - const decl = symbol?.valueDeclaration; - if (decl && isPropertyDecl(decl) - && sb(decl, ctx) - && decl.initializer) { - const declFile = cx(decl.getSourceFile(), ctx); - if (declFile) { - return qo(decl.initializer, sym, declFile, ctx, nextSeen); - } - } - } - if (isCall(node) - && isAccess(node.expression) - && node.expression.name.text === 'get' - && node.arguments[0]) { - return mq( - node.expression.expression, node.arguments[0], file, ctx, - ) + const e = f[0]![3] + return f.every(([entryKey, , queueKey, entryTransport]) => + entryTransport === e && eq(entryKey, queueKey, ctx)) + ? [rd(key, k, ctx, { c: true }), e] : null +} +function qo(n: ts.Expression, sym: IndexSymbol, l: FileContext, ctx: CollectionState, m: ReadonlySet = new Set()): QueueOrigin | null { + const a = uw(n); if (m.has(a)) return null + const d = new Set(m).add(a) + if (isNew(a) && a.arguments?.[0]) { + const b = bt(ib(a.expression, l)); if (b) return [rd(a.arguments[0], l, ctx, { c: true }), b] + } + if (isIdentifier(a)) { + const e = sy(a, ctx)?.valueDeclaration + if (e && isVariable(e) && sb(e, ctx) && e.initializer) { + const f = cx(e.getSourceFile(), ctx); return f ? qo(e.initializer, sym, f, ctx, d) : null } - return null; -} -function es(expr: ts.Expression, sym: IndexSymbol, file: FileContext, ctx: CollectionState, seen: ReadonlySet = new Set()): EmitterScope | null { - const node = uw(expr); - if (seen.has(node)) - return null; - const nextSeen = new Set(seen); - nextSeen.add(node); - if (isIdentifier(node)) { - const symbol = sy(node, ctx); - const decl = symbol?.valueDeclaration; - if (decl) { - const declFile = cx(decl.getSourceFile(), ctx); - const identity = declFile - ? `${declFile.fileId}:${decl.getStart(declFile.sf, false)}` - : null; - const cached = identity ? ctx.emitters.get(identity) : undefined; - if (cached) - return cached; - if (isVariable(decl) - && sb(decl, ctx) - && decl.initializer - && isNew(uw(decl.initializer))) { - const init = uw(decl.initializer) as ts.NewExpression; - if (!declFile) - return null; - const transport = et(init.expression, declFile); - if (transport) { - const scope = identity!; - const value: EmitterScope = [scope, transport]; - ctx.emitters.set(scope, value); - return value; - } - } - } + } + if (isAccess(a) && a.expression.kind === K.ThisKeyword) { + const j = sy(a.name, ctx), k = sym.kind === 'method' ? sym.name.slice(0, sym.name.lastIndexOf('.')) : sym.name + const g = ctx.nq.get(`${sym.file_id}\0${k}`)?.get(a.name.text) + if (g && j && !ctx.u.has(j)) return g + const h = j?.valueDeclaration + if (h && isPropertyDecl(h) && sb(h, ctx) && h.initializer) { + const i = cx(h.getSourceFile(), ctx); if (i) return qo(h.initializer, sym, i, ctx, d) } - if (isAccess(node) - && node.expression.kind === K.ThisKeyword) { - const binding = ti(node, file, ctx); - const transport = binding?.module === '@nestjs/event-emitter' - ? 'nestjs-event-emitter' - : binding?.module === 'node:events' - || binding?.module === 'events' - ? 'node-event-emitter' - : null; - const decl = sy(node.name, ctx)?.valueDeclaration; - const inferred = decl && isPropertyDecl(decl) - && sb(decl, ctx) - && decl.initializer - && isNew(uw(decl.initializer)) - ? et((uw(decl.initializer) as ts.NewExpression).expression, cx(decl.getSourceFile(), ctx) ?? file) - : null; - if (transport || inferred) { - const className = sym.kind === 'method' - ? sym.name.slice(0, sym.name.lastIndexOf('.')) - : sym.name; - return [ - `${file.fileId}:${className}.${node.name.text}`, - transport ?? inferred!, - ]; + } + if (isCall(a) && isAccess(a.expression) && a.expression.name.text === 'get' && a.arguments[0]) + return mq(a.expression.expression, a.arguments[0], l, ctx) + return null +} +function es(j: ts.Expression, sym: IndexSymbol, l: FileContext, ctx: CollectionState, g: ReadonlySet = new Set()): EmitterScope | null { + const e = uw(j); if (g.has(e)) return null + g = new Set(g).add(e) + if (isIdentifier(e)) { + const c = sy(e, ctx)?.valueDeclaration + if (c) { + const a = cx(c.getSourceFile(), ctx), b = a ? `${a.id}:${c.getStart(a.sf, false)}` : null + const f = b ? ctx.em.get(b) : undefined + if (f) return f + if (isVariable(c) && sb(c, ctx) && c.initializer && isNew(uw(c.initializer))) { + const k = uw(c.initializer) as ts.NewExpression; if (!a) return null + const d = et(k.expression, a) + if (d) { + const h = b!, i: EmitterScope = [h, d]; ctx.em.set(h, i); return i } + } } - return null; + } + return null } -function et(expr: ts.Expression, file: FileContext): 'node-event-emitter' | 'nestjs-event-emitter' | null { - if (ii(expr, file, ['node:events', 'events'], ['EventEmitter'])) { - return 'node-event-emitter'; - } - if (ii(expr, file, ['@nestjs/event-emitter'], ['EventEmitter2'])) { - return 'nestjs-event-emitter'; - } - return null; -} -function hv(expr: ts.Expression, file: FileContext, ctx: CollectionState): IndexValue { - const node = uw(expr); - if (isArrow(node) || isFunction(node)) { - const targets = new Set(); - const visit = (child: ts.Node): void => { - if (child !== node && (isArrow(child) || isFunction(child))) - return; - if (isCall(child)) { - const target = cs(child, file, ctx); - if (target) - targets.add(target.id); - } - ts.forEachChild(child, visit); - }; - visit(node.body); - return targets.size === 1 - ? { kind: 'symbol', symbol_id: [...targets][0]! } - : uk(targets.size > 1 ? 'ambiguous' : 'dynamic'); - } - return rd(node, file, ctx, { constants: false }); +function et(a: ts.Expression, b: FileContext): 'node-event-emitter' | 'nestjs-event-emitter' | null { + if (ii(a, b, ['node:events', 'events'], ['EventEmitter'])) return 'node-event-emitter' + return ii(a, b, ['@nestjs/event-emitter'], ['EventEmitter2']) + ? 'nestjs-event-emitter' : null +} +function hv(h: ts.Expression, g: FileContext, ctx: CollectionState): IndexValue { + const d = uw(h) + if (isArrow(d) || isFunction(d)) { + const a = new Set() + const f = (b: ts.Node): void => { + if (b !== d && (isArrow(b) || isFunction(b))) return + if (isCall(b)) { const e = cs(b, g, ctx); if (e) a.add(e.id) } + ts.forEachChild(b, f) + } + f(d.body) + return a.size === 1 ? { kind: 'symbol', symbol_id: [...a][0]! } + : uk(a.size > 1 ? 'ambiguous' : 'dynamic') + } + return rd(d, g, ctx, { c: false }) } type PersistenceSummary = readonly [operation: IndexPersistenceOperation, resource: IndexValue | undefined, receiverType: string] function se( - operation: IndexPersistenceOperation | null, receiverType: string, - resource: ts.Expression | undefined, file: FileContext, ctx: CollectionState, + a: IndexPersistenceOperation | null, b: string, + d: ts.Expression | undefined, e: FileContext, ctx: CollectionState, ): PersistenceSummary | null { - return operation ? [ - operation, - resource ? rd(resource, file, ctx, { constants: true }) : undefined, - receiverType, + return a ? [ + a, + d ? rd(d, e, ctx, { c: true }) : undefined, + b, ] : null } -function po(call: ts.CallExpression, file: FileContext, ctx: CollectionState): PersistenceSummary | null { - const expr = call.expression; - if (isIdentifier(expr)) { - const binding = file.imports.get(expr.text); - if (binding && FSM.has(binding.module)) { - return se(fsop(binding.imported, call.arguments[1], file, ctx), `${binding.module}:${binding.imported}`, call.arguments[0], file, ctx); - } - } - if (!isAccess(expr)) - return null; - const method = expr.name.text; - const receiver = expr.expression; - const root = li(receiver); - const namespace = root ? file.imports.get(root.text) : undefined; - if (namespace?.namespace && FSM.has(namespace.module)) { - const summary = se(fsop(method, call.arguments[1], file, ctx), `${namespace.module}:namespace`, call.arguments[0], file, ctx); - if (summary) - return summary; - } - const type = rty(receiver, ctx); - const typeBinding = ti(receiver, file, ctx); - if (typeBinding?.module === 'typeorm' - && ['Repository', 'MongoRepository'].includes(typeBinding.imported)) { - const summary = se(typeormOperation(method), type || `${typeBinding.module}:${typeBinding.imported}`, call.arguments[0], file, ctx); - if (summary) - return summary; - } - if (pd(receiver, ctx, '/node_modules/@prisma/client/', '/node_modules/.prisma/client/')) { - const summary = se(prismaOperation(method), type || 'PrismaClient', call.arguments[0], file, ctx); - if (summary) - return summary; - } - if (method !== 'send' - || !(typeBinding?.module === '@aws-sdk/client-s3' - && typeBinding.imported === 'S3Client' - || pd(receiver, ctx, '/node_modules/@aws-sdk/client-s3/'))) - return null; - const command = call.arguments[0]; - if (command && isNew(uw(command))) { - const constructor = uw(command) as ts.NewExpression; - const binding = ib(constructor.expression, file); - if (binding?.module === '@aws-sdk/client-s3') { - const writes = ['PutObjectCommand', 'UploadPartCommand', 'CompleteMultipartUploadCommand']; - const reads = ['GetObjectCommand', 'HeadObjectCommand']; - return se(writes.includes(binding.imported) ? 'object_write' - : reads.includes(binding.imported) ? 'object_read' : null, type, constructor.arguments?.[0], file, ctx); - } +function po(f: ts.CallExpression, c: FileContext, ctx: CollectionState): PersistenceSummary | null { + const k = f.expression + if (isIdentifier(k)) { + const d = c.im.get(k.text); if (d && FSM.has(d.m)) + return se(fsop(d.i, f.arguments[1], c, ctx), `${d.m}:${d.i}`, f.arguments[0], c, ctx) + } + if (!isAccess(k)) return null + const g = k.name.text, b = k.expression, s = li(b) + const ns = s ? c.im.get(s.text) : undefined + if (ns?.n && FSM.has(ns.m)) { + const l = se(fsop(g, f.arguments[1], c, ctx), `${ns.m}:namespace`, f.arguments[0], c, ctx); if (l) return l + } + const q = rty(b, ctx), a = ti(b, c, ctx) + if (a?.m === 'typeorm' && ['Repository', 'MongoRepository'].includes(a.i)) { + const o = se(typeormOperation(g), q || `${a.m}:${a.i}`, f.arguments[0], c, ctx); if (o) return o + } + if (pd(b, ctx, '/node_modules/@prisma/client/', '/node_modules/.prisma/client/')) { + const p = se(prismaOperation(g), q || 'PrismaClient', f.arguments[0], c, ctx); if (p) return p + } + if (g !== 'send' || !(a?.m === '@aws-sdk/client-s3' && a.i === 'S3Client' || pd(b, ctx, '/node_modules/@aws-sdk/client-s3/'))) return null + const h = f.arguments[0] + if (h && isNew(uw(h))) { + const e = uw(h) as ts.NewExpression, j = ib(e.expression, c) + if (j?.m === '@aws-sdk/client-s3') { + const r = ['PutObjectCommand', 'UploadPartCommand', 'CompleteMultipartUploadCommand'], t = ['GetObjectCommand', 'HeadObjectCommand'] + return se(r.includes(j.i) ? 'object_write' : t.includes(j.i) ? 'object_read' : null, q, e.arguments?.[0], c, ctx) } - return null; + } + return null } -function li(expr: ts.Expression): ts.Identifier | null { - let current = expr - while (isAccess(current)) current = current.expression - return isIdentifier(current) ? current : null +function li(b: ts.Expression): ts.Identifier | null { + let a = b + while (isAccess(a)) a = a.expression + return isIdentifier(a) ? a : null } -function pd(expr: ts.Expression, ctx: CollectionState, ...packagePaths: readonly string[]): boolean { - const checker = ctx.input.checker; - let current: ts.Expression = expr; +function pd(h: ts.Expression, ctx: CollectionState, ...b: readonly string[]): boolean { + const d = ctx.i.checker + let a: ts.Expression = h while (true) { try { - const type = checker.getTypeAtLocation(current); - const symbols = [type.aliasSymbol, type.getSymbol()]; - if (symbols.some((symbol) => symbol?.declarations?.some((decl) => { - const path = decl.getSourceFile().fileName.replaceAll('\\', '/'); - return packagePaths.some((packagePath) => path.includes(packagePath)); - }))) { - return true; - } - } - catch { - return false; - } - if (!isAccess(current)) - return false; - current = current.expression; + const g = d.getTypeAtLocation(a) + const e = [g.aliasSymbol, g.getSymbol()] + if (e.some((f) => f?.declarations?.some((j) => { + const k = j.getSourceFile().fileName.replaceAll('\\', '/') + return b.some((c) => k.includes(c)) + }))) return true + } catch { return false } + if (!isAccess(a)) return false + a = a.expression } } -function no(method: string, operations: Readonly>): IndexPersistenceOperation | null { - return Object.hasOwn(operations, method) ? operations[method]! : null; +function no(b: string, a: Readonly>): IndexPersistenceOperation | null { + return Object.hasOwn(a, b) ? a[b]! : null; } function fsop( - method: string, flags: ts.Expression | undefined, file: FileContext, ctx: CollectionState, + b: string, d: ts.Expression | undefined, e: FileContext, ctx: CollectionState, ): IndexPersistenceOperation | null { - if (!['open', 'openSync'].includes(method)) return no(method, FSO) - if (!flags) return null - const value = ss(rd(flags, file, ctx, { constants: true })) - if (!value) return null - if (value.includes('+') || /^[aw]/u.test(value)) return 'file_write' - return /^r(?:s|sr)?$/u.test(value) ? 'file_read' : null -} -const typeormOperation = (method: string): IndexPersistenceOperation | null => - no(method, TOO) -const prismaOperation = (method: string): IndexPersistenceOperation | null => - no(method, PRO) -function re(call: ts.CallExpression | ts.NewExpression, sym: IndexSymbol, file: FileContext, ctx: CollectionState): void { - const proof = [call, 'high', 'framework'] as const; - if (isNew(call)) { - if (ii(call.expression, file, ['bullmq'], ['Worker']) - && call.arguments?.[0] - && call.arguments[1]) { - ae(ctx, sym.id, [ - 'bull-consume', - rd(call.arguments[0], file, ctx, { constants: true }), - hv(call.arguments[1], file, ctx), - 'bullmq', undefined, ...proof, - ]); - } - return; - } - if (isAccess(call.expression)) { - const method = call.expression.name.text; - const receiver = call.expression.expression; - if (method === 'add' && call.arguments[0]) { - const queue = qo(receiver, sym, file, ctx); - if (queue) { - ae(ctx, sym.id, [ - 'bull-publish', queue[0], - rd(call.arguments[0], file, ctx, { constants: true }), - queue[1], undefined, ...proof, - ]); - } - } - const emitter = es(receiver, sym, file, ctx); - if (emitter && method === 'emit' && call.arguments[0]) { - ae(ctx, sym.id, [ - 'event-publish', - rd(call.arguments[0], file, ctx, { constants: true }), - undefined, emitter[1], emitter[0], ...proof, - ]); - } - else if (emitter - && ['addListener', 'on', 'once', 'prependListener'].includes(method) - && call.arguments[0] - && call.arguments[1]) { - ae(ctx, sym.id, [ - 'event-consume', - rd(call.arguments[0], file, ctx, { constants: true }), - hv(call.arguments[1], file, ctx), - emitter[1], emitter[0], ...proof, - ]); - } - } - const persistence = po(call, file, ctx); - if (persistence) { - ae(ctx, sym.id, ['persistence', ...persistence, undefined, ...proof]); - } + if (!['open', 'openSync'].includes(b)) return no(b, FSO) + if (!d) return null + const a = ss(rd(d, e, ctx, { c: true })) + if (!a) return null + if (a.includes('+') || /^[aw]/u.test(a)) return 'file_write' + return /^r(?:s|sr)?$/u.test(a) ? 'file_read' : null +} +const typeormOperation = (a: string): IndexPersistenceOperation | null => + no(a, TOO) +const prismaOperation = (a: string): IndexPersistenceOperation | null => + no(a, PRO) +function re(a: ts.CallExpression | ts.NewExpression, sym: IndexSymbol, d: FileContext, ctx: CollectionState): void { + if ((ctx.ci.get(a)?.length ?? 0) > 1) return + const f = [a, 'high', 'framework'] as const + if (isNew(a)) { + if (ii(a.expression, d, ['bullmq'], ['Worker']) && a.arguments?.[0] && a.arguments[1]) + ae(ctx, sym.id, ['bull-consume', rd(a.arguments[0], d, ctx, { c: true }), hv(a.arguments[1], d, ctx), 'bullmq', undefined, ...f]) + return + } + if (isAccess(a.expression)) { + const h = a.expression.name.text, g = a.expression.expression + if (h === 'add' && a.arguments[0]) { + const i = qo(g, sym, d, ctx); if (i) ae(ctx, sym.id, ['bull-publish', i[0], + rd(a.arguments[0], d, ctx, { c: true }), i[1], undefined, ...f]) + } + const b = es(g, sym, d, ctx) + if (b && h === 'emit' && a.arguments[0]) + ae(ctx, sym.id, ['event-publish', rd(a.arguments[0], d, ctx, { c: true }), undefined, b[1], b[0], ...f]) + else if (b && ['addListener', 'on', 'once', 'prependListener'].includes(h) + && a.arguments[0] && a.arguments[1]) + ae(ctx, sym.id, ['event-consume', rd(a.arguments[0], d, ctx, { c: true }), hv(a.arguments[1], d, ctx), b[1], b[0], ...f]) + } + const e = po(a, d, ctx); if (e) ae(ctx, sym.id, ['persistence', ...e, undefined, ...f]) } -function pc(call: ts.CallExpression, ctx: CollectionState): { +function pc(b: ts.CallExpression, ctx: CollectionState): { combinator: 'all' | 'allSettled' | 'any' | 'race' completion: 'all_or_first_rejection' | 'all_settled' | 'first_fulfilled' | 'first_settled' } | null { - if (!isAccess(call.expression) - || !isIdentifier(call.expression.expression) - || call.expression.expression.text !== 'Promise') { - return null; - } - const symbol = sy(call.expression.expression, ctx); - if (!symbol?.declarations?.some((decl) => decl.getSourceFile().isDeclarationFile - && /\/typescript\/lib\/lib\..+\.d\.ts$/u.test(decl.getSourceFile().fileName.replaceAll('\\', '/')))) - return null; - const combinator = call.expression.name.text; - return Object.hasOwn(PMC, combinator) - ? { - combinator: combinator as keyof typeof PMC, - completion: PMC[combinator as keyof typeof PMC], - } - : null; -} -function pl(expr: ts.Expression | undefined): readonly ts.Expression[] | null { - if (!expr) return null - const node = uw(expr); - if (!ts.isArrayLiteralExpression(node) - || node.elements.length > VELE - || node.elements.some((element) => - ts.isOmittedExpression(element) || ts.isSpreadElement(element))) return null - return [...node.elements] as ts.Expression[] -} -function mi(expr: ts.Expression | undefined, file: FileContext, ctx: CollectionState): { + if (!isAccess(b.expression) + || !isIdentifier(b.expression.expression) + || b.expression.expression.text !== 'Promise') return null + const c = sy(b.expression.expression, ctx) + if (!c?.declarations?.some((d) => d.getSourceFile().isDeclarationFile + && /\/typescript\/lib\/lib\..+\.d\.ts$/u.test(d.getSourceFile().fileName.replaceAll('\\', '/')))) + return null + const a = b.expression.name.text + return Object.hasOwn(PMC, a) + ? { combinator: a as keyof typeof PMC, + completion: PMC[a as keyof typeof PMC] } : null +} +function pl(c: ts.Expression | undefined): readonly ts.Expression[] | null { + if (!c) return null + const b = uw(c); + if (!ts.isArrayLiteralExpression(b) + || b.elements.length > VELE + || b.elements.some((a) => + ts.isOmittedExpression(a) || ts.isSpreadElement(a))) return null + return [...b.elements] as ts.Expression[] +} +function mi(b: ts.Expression | undefined, e: FileContext, ctx: CollectionState): { call: ts.CallExpression; input: IndexValue; receiver: ts.Expression } | null { - if (!expr) - return null; - const node = uw(expr); - if (!isCall(node) - || !isAccess(node.expression) - || node.expression.name.text !== 'map') - return null; - const input = rd(node.expression.expression, file, ctx, { constants: true }); - return input.kind === 'array' - ? { call: node, input, receiver: node.expression.expression } - : null; -} -function ex(stmt: ts.Statement): boolean { - if (ts.isReturnStatement(stmt) || ts.isThrowStatement(stmt) - || ts.isBreakStatement(stmt) || ts.isContinueStatement(stmt)) return true - if (ts.isBlock(stmt)) { - const last = stmt.statements.at(-1) - return last ? ex(last) : false - } - if (isIf(stmt) && stmt.elseStatement) { - return ex(stmt.thenStatement) && ex(stmt.elseStatement) + if (!b) return null + const a = uw(b) + if (!isCall(a) + || !isAccess(a.expression) + || a.expression.name.text !== 'map') return null + const d = rd(a.expression.expression, e, ctx, { c: true }) + return d.kind === 'array' + ? { call: a, input: d, receiver: a.expression.expression } : null +} +function sk( + f: ts.Expression, g?: FileContext, ctx?: CollectionState, +): string | null { + const d = g && ctx ? rd(f, g, ctx, { c: true }) : null + if (d?.kind === 'literal') + return JSON.stringify([typeof d.value, d.value]) + const b = uw(f) + if (ts.isPrefixUnaryExpression(b) && isNumeric(b.operand) + && [K.PlusToken, K.MinusToken].includes(b.operator) + && Number(b.operand.text) === 0) return JSON.stringify(['number', 0]) + if (ctx && (isAccess(b) || ts.isElementAccessExpression(b))) { + const e = sy(b, ctx)?.declarations?.find(ts.isEnumMember) + const a = e ? ctx.i.checker.getConstantValue(e) + : ctx.i.checker.getConstantValue(b) + if (typeof a === 'number' && Number.isFinite(a)) + return JSON.stringify(['number', Object.is(a, -0) ? 0 : a]) } - return false + return null } -function gc(stmt: ts.IfStatement): BranchArm | 'unreachable' | null { - const thenExits = ex(stmt.thenStatement) - const elseExits = stmt.elseStatement - ? ex(stmt.elseStatement) - : false - if (thenExits && elseExits) return 'unreachable' - if (thenExits) return 'else' - return elseExits ? 'then' : null -} -function tv(decl: ts.VariableDeclaration, sym: IndexSymbol, file: FileContext, ctx: CollectionState): IndexValue | null { - if (!decl.initializer || decl.parent.parent.parent !== file.sf) - return null; - const value = rd(decl.initializer, file, ctx, { - constants: true, - secret: isIdentifier(decl.name) && SNM.test(decl.name.text), - }); - if (value.kind === 'unknown' || value.kind === 'symbol' || value.kind === 'parameter') - return null; - if (value.kind === 'literal' - && typeof value.value === 'string' - && value.value.length === 0) { - return null; +const XN = 1, XT = 2, XO = 4, XB = 8 +function xq( + b: readonly ts.Statement[], d?: FileContext, + ctx?: CollectionState, a = XN, +): number { + for (const c of b) { + if (!(a & XN)) break + a = a & ~XN | xp(c, d, ctx) + } + return a +} +function xp( + a: ts.Statement, d?: FileContext, ctx?: CollectionState, +): number { + if (ts.isReturnStatement(a) || ts.isContinueStatement(a)) return XO + if (ts.isBreakStatement(a)) return a.label ? XO : XB + if (ts.isThrowStatement(a)) return XT + if (ts.isBlock(a)) return xq(a.statements, d, ctx) + if (isIf(a)) return xp(a.thenStatement, d, ctx) + | (a.elseStatement ? xp(a.elseStatement, d, ctx) : XN) + if (ts.isSwitchStatement(a)) { + let f = a.caseBlock.clauses.some(ts.isDefaultClause) ? 0 : XN + const j = new Set() + for (let e = 0; e < a.caseBlock.clauses.length; e += 1) { + const g = a.caseBlock.clauses[e]! + if (ts.isCaseClause(g)) { + const key = sk(g.expression, d, ctx) + if (key && j.has(key)) continue + if (key) j.add(key) + } + let b = XN + for (let h = e; + h < a.caseBlock.clauses.length && b & XN; + h += 1) { + b = xq(a.caseBlock.clauses[h]!.statements, d, ctx, b) + } + if (b & XB) b = b & ~XB | XN + f |= b } - return sym.id === ds(decl, file, ctx)?.id ? value : null; -} -function collect(file: FileContext, ctx: CollectionState): void { - const visit = ( - node: ts.Node, - control: readonly IndexControlFrame[], - executeCallable = false, + return f + } + if (ts.isTryStatement(a)) { + let c = xp(a.tryBlock, d, ctx) + if (a.catchClause && c & XT) + c = c & ~XT | xp(a.catchClause.block, d, ctx) + if (a.finallyBlock) { + const i = xp(a.finallyBlock, d, ctx) + c = (i & XN ? c : 0) | i & ~XN + } + return c + } + return XN +} +function ex( + a: ts.Statement, b?: FileContext, ctx?: CollectionState, +): boolean { return !(xp(a, b, ctx) & XN) } +function sx(a: ts.Node): boolean { + return isCall(a) || isNew(a) + || isBinary(a) && AOP.has(a.operatorToken.kind) + || (ts.isPrefixUnaryExpression(a) || ts.isPostfixUnaryExpression(a)) + && [K.PlusPlusToken, K.MinusMinusToken].includes(a.operator) + || ts.isDeleteExpression(a) || ts.isTaggedTemplateExpression(a) + || ts.isAwaitExpression(a) || ts.isYieldExpression(a) + || ts.forEachChild(a, sx) === true +} +function gc( + c: ts.IfStatement, d?: FileContext, ctx?: CollectionState, +): BranchArm | 'unreachable' | null { + const a = ex(c.thenStatement, d, ctx) + const b = c.elseStatement + ? ex(c.elseStatement, d, ctx) : false + if (a && b) return 'unreachable' + if (a) return 'else' + return b ? 'then' : null +} +function tv(b: ts.VariableDeclaration, sym: IndexSymbol, d: FileContext, ctx: CollectionState): IndexValue | null { + if (!b.initializer || b.parent.parent.parent !== d.sf) return null + const a = rd(b.initializer, d, ctx, { + c: true, + s: isIdentifier(b.name) && SNM.test(b.name.text), + }); + if (a.kind === 'unknown' || a.kind === 'symbol' + || a.kind === 'parameter') return null + if (a.kind === 'literal' + && typeof a.value === 'string' + && a.value.length === 0) return null + return sym.id === ds(b, d, ctx)?.id ? a : null; +} +function collect(b: FileContext, ctx: CollectionState): void { + const d = ( + a: ts.Node, + z: readonly IndexControlFrame[], + i = false, ): void => { - if (ts.isDecorator(node)) return - const sym = ow(node, file) - if (ts.isFunctionLike(node) - && ca(node, file, ctx)?.id !== sym?.id - && !executeCallable) return - if (sym && control.length > INDEX_BODY_FACT_CONTROL_LIMIT) { - ctx.overflow.add(sym.id) + if (ts.isDecorator(a)) return + const sym = ow(a, b) + if (ts.isFunctionLike(a)) { + const t = ca(a, b, ctx) + if (t?.id !== sym?.id && !i) return + if (t?.id === sym?.id && sym && !ctx.p.has(sym.id)) { + ctx.p.set(sym.id, a.parameters.map((j) => + j.initializer + ? rd(j.initializer, b, ctx, { c: true }) : uk())) + } + } + if (sym && z.length > INDEX_BODY_FACT_CONTROL_LIMIT) { + ctx.o.add(sym.id) return } - if (sym && ts.isBlock(node)) { - let nextControl = control - for (const stmt of node.statements) { - visit(stmt, nextControl) - if (!isIf(stmt)) { - if (ex(stmt)) break + if (sym && ts.isBlock(a)) { + let e = z + for (const w of a.statements) { + d(w, e) + if (!isIf(w)) { + if (ex(w, b, ctx)) break continue } - const continuation = gc(stmt) - if (continuation === 'unreachable') break - if (continuation) { - const base = fb( + const u = gc(w, b, ctx) + if (u === 'unreachable') break + if (u) { + const T = fb( sym.id, 'condition', - stmt.expression, - file, - nextControl, - { statementNode: stmt }, + w.expression, + b, + e, + { n: w }, ) - nextControl = br(nextControl, base.id, continuation) + ctx.q.set(T.id, u) + e = br(e, T.id, u) } } return } - if (sym && isVariable(node) && isIdentifier(node.name)) { - const value = tv(node, sym, file, ctx) - if (value) { + if (sym && isVariable(a) && isIdentifier(a.name)) { + const Z = tv(a, sym, b, ctx) + if (Z) { af(ctx, { - ...fb(sym.id, 'literal', node.initializer!, file, control, { - statementNode: stmt(node), + ...fb(sym.id, 'literal', a.initializer!, b, z, { + n: stmt(a), }), kind: 'literal', - value, + value: Z, role: 'initializer', }) } } - if (sym && isIf(node)) { - const continuation = gc(node) - const base = ac( - sym.id, continuation ? 'guard' : 'if', - node.expression, file, ctx, control, node, + if (sym && isIf(a)) { + const u = gc(a, b, ctx) + const U = ac( + sym.id, u ? 'guard' : 'if', + a.expression, b, ctx, z, a, ) - visit(node.expression, control) - visit(node.thenStatement, br(control, base.id, 'then')) - if (node.elseStatement) { - visit(node.elseStatement, br(control, base.id, 'else')) + d(a.expression, z) + d(a.thenStatement, br(z, U.id, 'then')) + if (a.elseStatement) { + d(a.elseStatement, br(z, U.id, 'else')) } return } - if (sym && ts.isSwitchStatement(node)) { - const base = ac( - sym.id, 'switch', node.expression, file, ctx, control, node, + if (sym && ts.isSwitchStatement(a)) { + if (a.caseBlock.clauses.length > VELE + || a.caseBlock.clauses.some((H) => + ts.isCaseClause(H) && sx(H.expression))) { + ctx.o.add(sym.id) + return + } + const $c = ac( + sym.id, 'switch', a.expression, b, ctx, z, a, ) - visit(node.expression, control) - for (const clause of node.caseBlock.clauses) { - const arm = ts.isDefaultClause(clause) + d(a.expression, z) + let ft: IndexControlFrame[][] = [] + const iv = b.v + const k = new Set() + for (const g of a.caseBlock.clauses) { + const arm = ts.isDefaultClause(g) ? 'default' as const - : `case:${hash(`${clause.expression.getText(file.sf)}:${clause.pos}`).slice(0, 16)}` as const - if (ts.isCaseClause(clause)) visit(clause.expression, control) - let armControl = br(control, base.id, arm) - for (const stmt of clause.statements) { - visit(stmt, armControl) - if (!isIf(stmt)) { - if (ex(stmt)) break - continue + : `case:${hash(`${g.expression.getText(b.sf)}:${g.pos}`).slice(0, 16)}` as const + if (ts.isCaseClause(g)) d(g.expression, z) + let I = true + if (ts.isCaseClause(g)) { + const key = sk(g.expression, b, ctx) + if (key) { + I = !k.has(key) + k.add(key) } - const continuation = gc(stmt) - if (continuation === 'unreachable') break - if (continuation) armControl = br( - armControl, fb( - sym.id, 'condition', stmt.expression, file, armControl, - { statementNode: stmt }, - ).id, continuation, - ) } + const $ = [...(I ? [br(z, $c.id, arm)] : []), ...ft] + const V: IndexControlFrame[][] = [] + for (const [path, entry] of $.entries()) { + b.v = path === 0 + ? iv + : b.nv++ + let q = entry, O = true + for (const x of g.statements) { + d(x, q) + if (!isIf(x)) { + if (ex(x, b, ctx)) { O = false; break } + continue + } + const u = gc(x, b, ctx) + if (u === 'unreachable') { O = false; break } + if (u) q = br( + q, fb( + sym.id, 'condition', x.expression, b, q, + { n: x }, + ).id, u, + ) + } + if (O) V.push(q) + if (ctx.o.has(sym.id)) break + } + b.v = iv + ft = V + if (ctx.o.has(sym.id)) break } return } - if (sym && ts.isConditionalExpression(node)) { - const base = ac( - sym.id, 'ternary', node.condition, file, ctx, control, stmt(node), + if (sym && ts.isConditionalExpression(a)) { + const W = ac( + sym.id, 'ternary', a.condition, b, ctx, z, stmt(a), ) - visit(node.condition, control) - visit(node.whenTrue, br(control, base.id, 'truthy')) - visit(node.whenFalse, br(control, base.id, 'falsy')) + d(a.condition, z) + d(a.whenTrue, br(z, W.id, 'truthy')) + d(a.whenFalse, br(z, W.id, 'falsy')) return } - const logical = isBinary(node) ? LFL.get(node.operatorToken.kind) : undefined - if (sym && isBinary(node) && logical) { - const base = ac( - sym.id, logical[0], node.left, file, ctx, control, stmt(node), + const l = isBinary(a) ? LFL.get(a.operatorToken.kind) : undefined + if (sym && isBinary(a) && l) { + const $d = ac( + sym.id, l[0], a.left, b, ctx, z, stmt(a), ) - visit(node.left, control) - visit(node.right, br(control, base.id, logical[1])) + d(a.left, z) + d(a.right, br(z, $d.id, l[1])) return } - const loop = ld(node) - if (sym && loop) { - const base = fb(sym.id, 'loop', node, file, control) - const repeatedControl = [...control, { + const s = ld(a) + if (sym && s) { + const X = fb(sym.id, 'loop', a, b, z) + const r = [...z, { kind: 'loop' as const, - controller_fact_id: base.id, + controller_fact_id: X.id, }] af(ctx, { - ...base, + ...X, kind: 'loop', - loop_kind: loop.kind, - ...(loop.test - ? { test: rd(loop.test, file, ctx, { constants: true }) } + loop_kind: s.kind, + ...(s.test + ? { test: rd(s.test, b, ctx, { c: true }) } : {}), }) - for (const setup of loop.once) visit(setup, control) - for (const repeated of loop.repeated) visit(repeated, repeatedControl) - visit(loop.body, repeatedControl) + for (const _ of s.once) d(_, z) + for (const L of s.repeated) d(L, r) + d(s.body, r) return } - if (sym && ts.isTryStatement(node)) { - visit(node.tryBlock, [...control, { kind: 'exception', arm: 'try' }]) - if (node.catchClause) { - visit(node.catchClause, [...control, { kind: 'exception', arm: 'catch' }]) + if (sym && ts.isTryStatement(a)) { + d(a.tryBlock, [...z, { kind: 'exception', arm: 'try' }]) + if (a.catchClause) { + d(a.catchClause, [...z, { kind: 'exception', arm: 'catch' }]) } - if (node.finallyBlock) { - visit(node.finallyBlock, [...control, { kind: 'exception', arm: 'finally' }]) + if (a.finallyBlock) { + d(a.finallyBlock, [...z, { kind: 'exception', arm: 'finally' }]) } return } - if (sym && ts.isReturnStatement(node)) { + if (sym && ts.isReturnStatement(a)) { af(ctx, { - ...fb(sym.id, 'return', node, file, control), + ...fb(sym.id, 'return', a, b, z), kind: 'return', - ...(node.expression - ? { value: rd(node.expression, file, ctx, { constants: true }) } + ...(a.expression + ? { value: rd(a.expression, b, ctx, { c: true }) } : {}), }) - if (node.expression) visit(node.expression, control) + if (a.expression) d(a.expression, z) return } - if (sym && ts.isThrowStatement(node)) { + if (sym && ts.isThrowStatement(a)) { af(ctx, { - ...fb(sym.id, 'throw', node, file, control), + ...fb(sym.id, 'throw', a, b, z), kind: 'throw', - value: rd(node.expression, file, ctx, { constants: true }), + value: rd(a.expression, b, ctx, { c: true }), }) - visit(node.expression, control) + d(a.expression, z) return } - if (sym && isBinary(node) && AOP.has(node.operatorToken.kind)) { - am(sym.id, node, 'assign', node.left, file, ctx, control, node.right) + if (sym && isBinary(a) && AOP.has(a.operatorToken.kind)) { + am(sym.id, a, 'assign', a.left, b, ctx, z, a.right) } else if ( sym - && (ts.isPrefixUnaryExpression(node) || ts.isPostfixUnaryExpression(node)) - && [K.PlusPlusToken, K.MinusMinusToken].includes(node.operator) + && (ts.isPrefixUnaryExpression(a) || ts.isPostfixUnaryExpression(a)) + && [K.PlusPlusToken, K.MinusMinusToken].includes(a.operator) ) { am( - sym.id, node, - node.operator === K.PlusPlusToken ? 'increment' : 'decrement', - node.operand, file, ctx, control, + sym.id, a, + a.operator === K.PlusPlusToken ? 'increment' : 'decrement', + a.operand, b, ctx, z, ) - } else if (sym && ts.isDeleteExpression(node)) { - am(sym.id, node, 'delete', node.expression, file, ctx, control) - } - if (sym && (isCall(node) || isNew(node))) { - cf(node, sym, file, ctx, control) - re(node, sym, file, ctx) - if (isCall(node)) { - const promise = pc(node, ctx) - if (promise) { - const mapped = mi(node.arguments[0], file, ctx) - const lanes = mapped ? null : pl(node.arguments[0]) - const input = mapped?.input ?? (lanes - ? rd(node.arguments[0]!, file, ctx, { constants: true }) + } else if (sym && ts.isDeleteExpression(a)) { + am(sym.id, a, 'delete', a.expression, b, ctx, z) + } + if (sym && (isCall(a) || isNew(a))) { + cf(a, sym, b, ctx, z) + re(a, sym, b, ctx) + if (isCall(a)) { + const G = pc(a, ctx) + if (G) { + const h = mi(a.arguments[0], b, ctx) + const E = h ? null : pl(a.arguments[0]) + const F = h?.input ?? (E + ? rd(a.arguments[0]!, b, ctx, { c: true }) : null) - if (!input || input.kind !== 'array' - || (lanes && lanes.length !== input.elements.length)) { - for (const argument of node.arguments) visit(argument, control) + if (!F || F.kind !== 'array' + || (E && E.length !== F.elements.length)) { + for (const M of a.arguments) d(M, z) return } - const base = fb(sym.id, 'parallel', node, file, control) - const before = ctx.facts.get(sym.id)?.length ?? 0 - if (mapped) { - cf(mapped.call, sym, file, ctx, control) - re(mapped.call, sym, file, ctx) - const loopId = ai(sym.id, mapped.call, file, ctx, control) - visit(mapped.receiver, control) - for (const argument of mapped.call.arguments) { - const callback = uw(argument) - const executes = isArrow(callback) - || isFunction(callback) - visit( - argument, - executes + const J = fb(sym.id, 'parallel', a, b, z) + const Q = ctx.f.get(sym.id)?.length ?? 0 + if (h) { + cf(h.call, sym, b, ctx, z) + re(h.call, sym, b, ctx) + const R = ai(sym.id, h.call, b, ctx, z) + d(h.receiver, z) + for (const y of h.call.arguments) { + const A = uw(y) + const B = isArrow(A) + || isFunction(A) + d( + y, + B ? [ - ...control, - { kind: 'loop', controller_fact_id: loopId }, + ...z, + { kind: 'loop', controller_fact_id: R }, { kind: 'parallel', - controller_fact_id: base.id, + controller_fact_id: J.id, lane: 'each', }, ] - : control, - executes, + : z, + B, ) } } else { - for (const [lane, expr] of lanes!.entries()) { - visit(expr, [...control, { + for (const [lane, expr] of E!.entries()) { + d(expr, [...z, { kind: 'parallel', - controller_fact_id: base.id, + controller_fact_id: J.id, lane, }]) } } - const memberFactIds = (ctx.facts.get(sym.id) ?? []) - .slice(before) - .filter((fact) => fact.kind === 'call' - && fact.control.some((frame) => - frame.kind === 'parallel' - && frame.controller_fact_id === base.id)) - .map((fact) => fact.id) + const ids = (ctx.f.get(sym.id) ?? []) + .slice(Q) + .filter((Y) => Y.kind === 'call' + && Y.control.some((P) => + P.kind === 'parallel' + && P.controller_fact_id === J.id)) + .map(($e) => $e.id) af(ctx, { - ...base, + ...J, kind: 'parallel', - ...promise, - lane_count: input.elements.length, - input, - member_fact_ids: memberFactIds, + ...G, + lane_count: F.elements.length, + input: F, + member_fact_ids: ids, }) return } if ( - isAccess(node.expression) - && AIM.has(node.expression.name.text) + isAccess(a.expression) + && AIM.has(a.expression.name.text) ) { - const input = rd( - node.expression.expression, - file, + const $a = rd( + a.expression.expression, + b, ctx, - { constants: true }, + { c: true }, ) - visit(node.expression.expression, control) - if (input.kind !== 'array') { - for (const argument of node.arguments) visit(argument, control) + d(a.expression.expression, z) + if ($a.kind !== 'array') { + for (const N of a.arguments) d(N, z) return } - const loopId = ai(sym.id, node, file, ctx, control) - for (const argument of node.arguments) { - const callback = uw(argument) - visit(argument, [...control, { + const S = ai(sym.id, a, b, ctx, z) + for (const C of a.arguments) { + const D = uw(C) + d(C, [...z, { kind: 'loop', - controller_fact_id: loopId, - }], isArrow(callback) || isFunction(callback)) + controller_fact_id: S, + }], isArrow(D) || isFunction(D)) } return } - const arrayMutation = isAccess(node.expression) - ? AMU.get(node.expression.name.text) + const m = isAccess(a.expression) + ? AMU.get(a.expression.name.text) : undefined - if (arrayMutation && isAccess(node.expression) - && iar(node.expression.expression, ctx)) { + if (m && isAccess(a.expression) + && iar(a.expression.expression, ctx)) { am( - sym.id, node, arrayMutation, node.expression.expression, file, ctx, - control, arrayMutation === 'append' ? node.arguments[0] : undefined, 1, + sym.id, a, m, a.expression.expression, b, ctx, + z, m === 'append' ? a.arguments[0] : undefined, 1, ) } } } - ts.forEachChild(node, (child) => visit(child, control)) + ts.forEachChild(a, ($b) => d($b, z)) } - visit(file.sf, []) + d(b.sf, []) } -function ld(node: ts.Node): { +function ld(a: ts.Node): { kind: 'for' | 'for_in' | 'for_of' | 'while' | 'do_while'; test?: ts.Expression once: readonly ts.Node[]; repeated: readonly ts.Node[]; body: ts.Statement } | null { - if (ts.isForStatement(node)) { - const once: ts.Node[] = [] - const repeated: ts.Node[] = [] - if (node.initializer) once.push(node.initializer) - if (node.condition) repeated.push(node.condition) - if (node.incrementor) repeated.push(node.incrementor) + if (ts.isForStatement(a)) { + const c: ts.Node[] = [] + const b: ts.Node[] = [] + if (a.initializer) c.push(a.initializer) + if (a.condition) b.push(a.condition) + if (a.incrementor) b.push(a.incrementor) return { kind: 'for', - ...(node.condition ? { test: node.condition } : {}), - once, - repeated, - body: node.statement, + ...(a.condition ? { test: a.condition } : {}), + once: c, + repeated: b, + body: a.statement, } } - if (ts.isForInStatement(node) || ts.isForOfStatement(node)) { + if (ts.isForInStatement(a) || ts.isForOfStatement(a)) { return { - kind: ts.isForInStatement(node) ? 'for_in' : 'for_of', - test: node.expression, - once: [node.expression], - repeated: [node.initializer], - body: node.statement, + kind: ts.isForInStatement(a) ? 'for_in' : 'for_of', + test: a.expression, + once: [a.expression], + repeated: [a.initializer], + body: a.statement, } } - if (ts.isWhileStatement(node) || ts.isDoStatement(node)) { + if (ts.isWhileStatement(a) || ts.isDoStatement(a)) { return { - kind: ts.isWhileStatement(node) ? 'while' : 'do_while', - test: node.expression, + kind: ts.isWhileStatement(a) ? 'while' : 'do_while', + test: a.expression, once: [], - repeated: [node.expression], - body: node.statement, + repeated: [a.expression], + body: a.statement, } } return null } -function dc(node: ts.Node): readonly ts.Decorator[] { - return ts.canHaveDecorators(node) ? ts.getDecorators(node) ?? [] : [] +function dc(a: ts.Node): readonly ts.Decorator[] { + return ts.canHaveDecorators(a) ? ts.getDecorators(a) ?? [] : [] } function bv( - node: ts.Node, name: 'InjectQueue' | 'Processor' | 'Process', - file: FileContext, ctx: CollectionState, + g: ts.Node, h: 'InjectQueue' | 'Processor' | 'Process', + f: FileContext, ctx: CollectionState, ): { value: IndexValue; transport: QueueTransport } | null { - let result: { value: IndexValue; transport: QueueTransport } | null = null - for (const decorator of dc(node)) { - if (!isCall(decorator.expression)) continue - const call = decorator.expression - if (!call.arguments[0]) continue - const binding = ib(call.expression, file) - if (binding?.imported === name - && ['@nestjs/bull', '@nestjs/bullmq'].includes(binding.module)) { - result = { - value: rd(call.arguments[0], file, ctx, { constants: true }), - transport: binding.module === '@nestjs/bull' ? 'bull' : 'bullmq', + let d: { value: IndexValue; transport: QueueTransport } | null = null + for (const a of dc(g)) { + if (!isCall(a.expression)) continue + const e = a.expression + if (!e.arguments[0]) continue + const b = ib(e.expression, f) + if (b?.i === h + && ['@nestjs/bull', '@nestjs/bullmq'].includes(b.m)) { + d = { + value: rd(e.arguments[0], f, ctx, { c: true }), + transport: b.m === '@nestjs/bull' ? 'bull' : 'bullmq', } } } - return result + return d } -function cnest(file: FileContext, ctx: CollectionState): void { - for (const stmt of file.sf.statements) { - if (!ts.isClassDeclaration(stmt) || !stmt.name) continue - const classKey = `${file.fileId}\0${stmt.name.text}` - const properties = ctx.nestQueues.get(classKey) +function cnest(g: FileContext, ctx: CollectionState): void { + for (const e of g.sf.statements) { + if (!ts.isClassDeclaration(e) || !e.name) continue + const c = `${g.id}\0${e.name.text}` + const a = ctx.nq.get(c) ?? new Map() - for (const member of stmt.members) { - if (!ts.isConstructorDeclaration(member)) continue - for (const parameter of member.parameters) { - if (!isIdentifier(parameter.name)) continue - const queue = bv(parameter, 'InjectQueue', file, ctx) - if (queue) { - properties.set(parameter.name.text, [queue.value, queue.transport]) + for (const f of e.members) { + if (!ts.isConstructorDeclaration(f)) continue + for (const b of f.parameters) { + if (!isIdentifier(b.name)) continue + const d = bv(b, 'InjectQueue', g, ctx) + if (d) { + a.set(b.name.text, [d.value, d.transport]) } } } - if (properties.size > 0) ctx.nestQueues.set(classKey, properties) - } -} -function nc(file: FileContext, ctx: CollectionState): void { - for (const stmt of file.sf.statements) { - if (!ts.isClassDeclaration(stmt) || !stmt.name) continue - const queue = bv(stmt, 'Processor', file, ctx) - if (!queue) continue - for (const member of stmt.members) { - if (!ts.isMethodDeclaration(member) || !member.name || !isIdentifier(member.name)) continue - const symbol = ds(member, file, ctx) - if (!symbol) continue - const job = bv(member, 'Process', file, ctx) - if (job?.transport === queue.transport) { - const queueKey = ss(queue.value) - const jobKey = ss(job.value) - if (queueKey && jobKey) { - const queueNode = ch(ctx, { + if (a.size > 0) ctx.nq.set(c, a) + } +} +function nc(d: FileContext, ctx: CollectionState): void { + for (const j of d.sf.statements) { + if (!ts.isClassDeclaration(j) || !j.name) continue + const c = bv(j, 'Processor', d, ctx) + if (!c) continue + for (const a of j.members) { + if (!ts.isMethodDeclaration(a) || !a.name || !isIdentifier(a.name)) continue + const b = ds(a, d, ctx) + if (!b) continue + const job = bv(a, 'Process', d, ctx) + if (job?.transport === c.transport) { + const f = ss(c.value) + const k = ss(job.value) + if (f && k) { + const e = ch(ctx, { channel_kind: 'queue', - transport: queue.transport, - key: queueKey, + transport: c.transport, + key: f, }) - const jobNode = ch(ctx, { + const h = ch(ctx, { channel_kind: 'job', - transport: queue.transport, - key: jobKey, - parent_channel_id: queueNode.id, + transport: c.transport, + key: k, + parent_channel_id: e.id, }) - ce(ctx, symbol.id, jobNode.id, symbol.id, 'consumed_by', member, file, 'framework-decorator') - ce(ctx, symbol.id, jobNode.id, queueNode.id, 'routes_through', member, file, 'framework-decorator') + ce(ctx, b.id, h.id, b.id, 'consumed_by', a, d, 'framework-decorator') + ce(ctx, b.id, h.id, e.id, 'routes_through', a, d, 'framework-decorator') } - } else if (!job && member.name.text === 'process') { - const queueKey = ss(queue.value) - if (queueKey) { - const queueNode = ch(ctx, { + } else if (!job && a.name.text === 'process') { + const g = ss(c.value) + if (g) { + const i = ch(ctx, { channel_kind: 'queue', - transport: queue.transport, - key: queueKey, + transport: c.transport, + key: g, }) - ce(ctx, symbol.id, queueNode.id, symbol.id, 'consumed_by', member, file, 'framework-decorator') + ce(ctx, b.id, i.id, b.id, 'consumed_by', a, d, 'framework-decorator') } } } } } -function ee(ownerId: string, ctx: CollectionState, depth: number, stack: ReadonlySet): ExecutionEffect[] { - const direct = [...(ctx.effects.get(ownerId) ?? [])]; - if (direct.length > EMAX) { - ctx.overflow.add(ownerId); - return []; +function ep(e: Predicate, h: readonly IndexValue[]): boolean | null { + const [op, left, right, negated] = e + const a = sub(left, h) + if (a.kind !== 'literal') return null + let c: boolean + if (op === K.Unknown) c = Boolean(a.value) + else { + const b = sub(right!, h) + if (b.kind !== 'literal') return null + const g = typeof a.value === typeof b.value + || a.value === null && b.value === null + if ([K.EqualsEqualsToken, K.EqualsEqualsEqualsToken].includes(op)) + c = g && a.value === b.value + else if ([K.ExclamationEqualsToken, K.ExclamationEqualsEqualsToken].includes(op)) + c = !g || a.value !== b.value + else { + if (!g || !['number', 'string'].includes(typeof a.value)) return null + const f = a.value as number | string + const d = b.value as number | string + if (op === K.LessThanToken) c = f < d + else if (op === K.LessThanEqualsToken) c = f <= d + else if (op === K.GreaterThanToken) c = f > d + else c = f >= d } - if (depth >= WHOP || stack.has(ownerId)) - return direct; - const nextStack = new Set(stack); - nextStack.add(ownerId); - for (const call of ctx.calls.get(ownerId) ?? []) { - if (nextStack.has(call[0])) - continue; - const nested = ee(call[0], ctx, depth + 1, nextStack); - if (ctx.overflow.has(call[0])) { - ctx.overflow.add(ownerId); - return []; - } - for (const fx of nested) { - if (direct.length >= EMAX) { - ctx.overflow.add(ownerId); - return []; - } - direct.push(ie(fx, call[1], call[2])); + } + return negated ? !c : c +} +function wp( + d: string, fx: ExecutionEffect, h: readonly IndexValue[], + ctx: CollectionState, +): boolean { + let e = ctx.w.get(d) + if (!e) { + e = new Map((ctx.f.get(d) ?? []).map((i) => [i.id, i])) + ctx.w.set(d, e) + } + const ids = ctx.ci.get(fx[5]) ?? [] + return ids.some((id) => { + const j = e.get(id) + return j?.kind === 'call' + && j.control.every((a) => { + if (a.kind === 'loop' || a.kind === 'parallel') return false + if (a.kind === 'exception') return a.arm !== 'catch' + const b = e.get(a.controller_fact_id) + if (!b || b.kind !== 'condition') return false + if (!b.test || b.condition_kind === 'switch') return false + const c = ctx.r.get(a.controller_fact_id) + const g = c ? ep(c, h) : null + if (g !== null) { + if (a.arm === 'nullish') { + const raw = c![0] === K.Unknown + ? sub(c![1], h) : null + return !c![3] && raw?.kind === 'literal' && raw.value === null } + if (a.arm === 'then' || a.arm === 'truthy') return g + return (a.arm === 'else' || a.arm === 'falsy') + && !g + } + return b.condition_kind === 'guard' + && ctx.q.get(a.controller_fact_id) === a.arm + }) + }) +} +function da(e: string, d: readonly IndexValue[], ctx: CollectionState): readonly IndexValue[] { + const a = ctx.p.get(e) + if (!a || d.length >= a.length) return d + const b = [...d] + for (let c = d.length; c < a.length; c += 1) + b.push(sub(a[c]!, b)) + return b +} +function ee(a: string, ctx: CollectionState, d: number, h: ReadonlySet): ExecutionEffect[] { + const b = [...(ctx.e.get(a) ?? [])] + if (b.length > EMAX) { ctx.o.add(a); return [] } + if (d >= WHOP || h.has(a)) return b + const f = new Set(h).add(a) + for (const g of ctx.c.get(a) ?? []) { + if (f.has(g[0])) continue + const i = ee(g[0], ctx, d + 1, f), j = da(g[0], g[1], ctx) + if (ctx.o.has(g[0])) { ctx.o.add(a); return [] } + for (const fx of i) { + if (b.length >= EMAX) { ctx.o.add(a); return [] } + if (!wp(g[0], fx, j, ctx)) continue; b.push(ie(fx, j, g[2])) } - return de(direct); + } + return de(b) } -function de(effects: readonly ExecutionEffect[]): ExecutionEffect[] { - const values = new Map() - for (const fx of effects) { - const witness = fx[5] +function de(c: readonly ExecutionEffect[]): ExecutionEffect[] { + const b: ExecutionEffect[] = [], d = new Set() + for (const fx of c) { + const a = fx[5] const key = JSON.stringify([ ...fx.slice(0, 5), - witness.getSourceFile().fileName, - witness.getStart(witness.getSourceFile(), false), - witness.getEnd(), + a.getSourceFile().fileName, + a.getStart(a.getSourceFile(), false), + a.getEnd(), ]) - if (!values.has(key)) values.set(key, fx) + if (fx[0] === 'persistence' || !d.has(key)) b.push(fx) + d.add(key) } - return [...values.values()] + return b } type ChannelDescriptor = Omit -function ch(ctx: CollectionState, descriptor: ChannelDescriptor): IndexChannelNode { - const id = indexChannelId(descriptor) - const node: IndexChannelNode = { - id, - node_kind: 'channel', - ...descriptor, - } - const prior = ctx.channels.get(id) - if (prior && JSON.stringify(prior) !== JSON.stringify(node)) { +function ch(ctx: CollectionState, a: ChannelDescriptor): IndexChannelNode { + const id = indexChannelId(a) + const b: IndexChannelNode = { id, node_kind: 'channel', ...a } + const c = ctx.ch.get(id) + if (c && JSON.stringify(c) !== JSON.stringify(b)) throw new Error(`Conflicting execution channel identity ${id}`) - } - ctx.channels.set(id, node) - return node + ctx.ch.set(id, b) + return b } function ce( - ctx: CollectionState, ownerId: string, from: string, to: string, - kind: Extract, - witness: ts.Node, file: FileContext, source: IndexEdge['source'], - confidence: Confidence = 'high', + ctx: CollectionState, a: string, h: string, to: string, + i: Extract, + b: ts.Node, c: FileContext, f: IndexEdge['source'], + d: Confidence = 'high', ): void { - const evidence = ev(witness, file.sf, file.fileId) - ctx.edges.push({ - from, - to, - kind, - confidence, - source, - evidence, - metadata: { execution_owner_id: ownerId }, + const e = ev(b, c.sf, c.id) + ctx.g.push({ + from: h, to, kind: i, confidence: d, source: f, evidence: e, + metadata: { execution_owner_id: a }, }) } -function edgeS(source: IndexFactSource): IndexEdge['source'] { - if (source === 'framework') return 'framework-decorator' - return source +function edgeS(a: IndexFactSource): IndexEdge['source'] { + return a === 'framework' ? 'framework-decorator' : a } -function fn(node: ts.Node, ctx: CollectionState, filesById: ReadonlyMap): FileContext | null { - const fileId = ctx.input.pathToFileId.get(node.getSourceFile().fileName); - return fileId ? filesById.get(fileId) ?? null : null; +function fn(c: ts.Node, ctx: CollectionState, a: ReadonlyMap): FileContext | null { + const b = ctx.i.pathToFileId.get(c.getSourceFile().fileName) + return b ? a.get(b) ?? null : null } -function ur(ctx: CollectionState, ownerId: string, fx: ExecutionEffect, file: FileContext): void { - const witness = fx[5]; +function ur(ctx: CollectionState, c: string, fx: ExecutionEffect, b: FileContext): void { + const a = fx[5]; const id = `canonical-index.execution.unresolved.${hash([ - ownerId, + c, fx[0], - file.fileId, - witness.getStart(file.sf, false), - witness.getEnd(), + b.id, + a.getStart(b.sf, false), + a.getEnd(), ].join(':')).slice(0, 16)}`; - if (ctx.seenDiagnostics.has(id)) + if (ctx.sd.has(id)) return; - ctx.seenDiagnostics.add(id); - ctx.diagnostics.push({ + ctx.sd.add(id); + ctx.d.push({ id, level: 'info', message: `Dynamic or ambiguous ${fx[0]} identity; unresolved channel parts were omitted`, evidence: { - file_id: file.fileId, - range: ro(witness, file.sf), + file_id: b.id, + range: ro(a, b.sf), }, }); } -function pe(ctx: CollectionState, filesById: ReadonlyMap): void { - for (const sym of ctx.input.symbols.filter(io)) { - if (ctx.overflow.has(sym.id)) continue +function pe(ctx: CollectionState, w: ReadonlyMap): void { + for (const sym of ctx.i.symbols.filter(io)) { + if (ctx.o.has(sym.id)) continue + let b = 0 for (const fx of ee(sym.id, ctx, 0, new Set())) { - if (ctx.overflow.has(sym.id)) break + if (ctx.o.has(sym.id)) break const [kind, primary, endpoint, qualifier, scope, witness, confidence, source] = fx - const file = fn(witness, ctx, filesById) - if (!file) continue - const emit = ( - fromId: string, - toId: string, - relation: 'publishes_to' | 'consumed_by' | 'routes_through', + const k = fn(witness, ctx, w) + if (!k) continue + const m = ( + C: string, + F: string, + A: 'publishes_to' | 'consumed_by' | 'routes_through', ): void => ce( - ctx, sym.id, fromId, toId, relation, witness, file, + ctx, sym.id, C, F, A, witness, k, edgeS(source), confidence, ) if (kind === 'bull-publish') { - const queueKey = ss(primary) - const jobKey = ss(endpoint) - if (!queueKey) { - ur(ctx, sym.id, fx, file) + const n = ss(primary) + const z = ss(endpoint) + if (!n) { + ur(ctx, sym.id, fx, k) continue } - const queueNode = ch(ctx, { + const e = ch(ctx, { channel_kind: 'queue', transport: qualifier, - key: queueKey, + key: n, }) - if (!jobKey) { - emit(sym.id, queueNode.id, 'publishes_to') - ur(ctx, sym.id, fx, file) + if (!z) { + m(sym.id, e.id, 'publishes_to') + ur(ctx, sym.id, fx, k) continue } - const jobNode = ch(ctx, { + const u = ch(ctx, { channel_kind: 'job', transport: qualifier, - key: jobKey, - parent_channel_id: queueNode.id, + key: z, + parent_channel_id: e.id, }) - emit(sym.id, jobNode.id, 'publishes_to') - emit(jobNode.id, queueNode.id, 'routes_through') + m(sym.id, u.id, 'publishes_to') + m(u.id, e.id, 'routes_through') } else if (kind === 'bull-consume') { - const queueKey = ss(primary) - const handlerId = si(endpoint) - if (!queueKey || !handlerId || !ctx.symbolsById.has(handlerId)) { - ur(ctx, sym.id, fx, file) + const p = ss(primary) + const g = si(endpoint) + if (!p || !g || !ctx.y.has(g)) { + ur(ctx, sym.id, fx, k) continue } - const queueNode = ch(ctx, { + const x = ch(ctx, { channel_kind: 'queue', transport: qualifier, - key: queueKey, + key: p, }) - emit(queueNode.id, handlerId, 'consumed_by') + m(x.id, g, 'consumed_by') } else if (kind === 'event-publish' || kind === 'event-consume') { - const eventKey = ss(primary) - const handlerId = kind === 'event-consume' ? si(endpoint!) : null - if (!eventKey || (kind === 'event-consume' - && (!handlerId || !ctx.symbolsById.has(handlerId)))) { - ur(ctx, sym.id, fx, file) + const q = ss(primary) + const h = kind === 'event-consume' ? si(endpoint!) : null + if (!q || (kind === 'event-consume' + && (!h || !ctx.y.has(h)))) { + ur(ctx, sym.id, fx, k) continue } - const eventNode = ch(ctx, { + const l = ch(ctx, { channel_kind: 'event', transport: qualifier, - key: eventKey, + key: q, scope, }) if (kind === 'event-publish') { - emit(sym.id, eventNode.id, 'publishes_to') + m(sym.id, l.id, 'publishes_to') } else { - emit(eventNode.id, handlerId!, 'consumed_by') + m(l.id, h!, 'consumed_by') } } else { - const persistence = fx as PersistenceEffect - const operation = persistence[1] - const resource = persistence[2] - const receiverType = persistence[3] - const callFactId = ctx.callIds.get(witness) - ?? fi(sym.id, witness, ctx) - if (!callFactId || !receiverType) continue - const ownerFacts = ctx.facts.get(sym.id) ?? [] - const existing = ownerFacts.some((fact) => - fact.kind === 'persistence' - && fact.call_fact_id === callFactId - && fact.operation === operation) - if (existing) continue - const callControl = ownerFacts.find((fact) => - fact.id === callFactId)?.control ?? [] - af(ctx, { - ...fb( - sym.id, - 'persistence', - witness, - file, - callControl, - { - confidence, - source, - }, - ), - kind: 'persistence', - operation, - call_fact_id: callFactId, - ...(resource ? { resource } : {}), - receiver_type: receiverType, - }) + const a = fx as PersistenceEffect + const E = a[1] + const B = a[2] + const d = a[3] + const v = ctx.f.get(sym.id) ?? [] + const r = fi(sym.id, witness, ctx) + const t = [...new Set(ctx.ci.get(witness) + ?? (r ? [r] : []))] + for (const j of t) { + const D = v.find((G) => G.id === j) + if (!d || D?.kind !== 'call') continue + af(ctx, { + ...fb( + sym.id, 'persistence', witness, k, D.control, + { c: confidence, s: source, o: ++b }, + ), + kind: 'persistence', + operation: E, + call_fact_id: j, + ...(B ? { resource: B } : {}), + receiver_type: d, + }) + } } } } } -function fi(ownerId: string, witness: ts.Node, ctx: CollectionState): string | null { - const sf = witness.getSourceFile(); - const range = ro(witness, sf); - return ctx.facts.get(ownerId)?.find((fact) => fact.kind === 'call' - && fact.evidence.range.start.line === range.start.line - && fact.evidence.range.start.column === range.start.column - && fact.evidence.range.end.line === range.end.line - && fact.evidence.range.end.column === range.end.column)?.id ?? null; +function fi(d: string, b: ts.Node, ctx: CollectionState): string | null { + const sf = b.getSourceFile(), a = ro(b, sf) + return ctx.f.get(d)?.find((c) => c.kind === 'call' + && c.evidence.range.start.line === a.start.line + && c.evidence.range.start.column === a.start.column + && c.evidence.range.end.line === a.end.line + && c.evidence.range.end.column === a.end.column)?.id ?? null } function at(ctx: CollectionState): void { - for (const symbol of ctx.input.symbols) { - if (ctx.overflow.has(symbol.id)) { - ctx.diagnostics.push({ - id: `canonical-index.execution.owner-bound.${hash(symbol.id).slice(0, 16)}`, - level: 'error', - message: `Execution facts exceeded a per-owner safety bound for ${symbol.name}; body facts were omitted`, - evidence: { file_id: symbol.file_id, range: symbol.range }, - }); - continue; - } - const facts = ctx.facts.get(symbol.id); - if (!facts || facts.length === 0) - continue; - const byId = new Map(); - for (const fact of facts) - byId.set(fact.id, fact); - const sorted = [...byId.values()].sort((left, right) => co(left.order, right.order) || ct(left.id, right.id)); - try { - const encoded = encodeIndexBodyFactTable(sorted); - const normalized = decodeIndexBodyFactTable(encoded, symbol.id, symbol.file_id); - if (!normalized) - throw new Error('execution fact codec rejected its output'); - symbol.body_facts = normalized; - } - catch (error) { - const bounded = error instanceof IndexBodyFactBoundsError; - ctx.diagnostics.push({ - id: `canonical-index.execution.${bounded ? 'owner-bound' : 'invalid'}.${hash(symbol.id).slice(0, 16)}`, - level: 'error', - message: bounded - ? `Execution facts exceeded a per-owner safety bound for ${symbol.name}; body facts were omitted` - : `Invalid execution facts for ${symbol.name}; body facts were omitted`, - evidence: { file_id: symbol.file_id, range: symbol.range }, - }); - delete symbol.body_facts; - } + for (const a of ctx.i.symbols) { + if (ctx.o.has(a.id)) { + ctx.d.push({ + id: `canonical-index.execution.owner-bound.${hash(a.id).slice(0, 16)}`, + level: 'error', evidence: { file_id: a.file_id, range: a.range }, message: `Execution facts exceeded a per-owner safety bound for ${a.name}; body facts were omitted` }) + continue + } + const e = ctx.f.get(a.id); if (!e || e.length === 0) continue + const k = new Map(); for (const l of e) k.set(l.id, l) + const j = [...k.values()].sort((m, g) => co(m.order, g.order) || ct(m.id, g.id)) + try { + const h = encodeIndexBodyFactTable(j), b = decodeIndexBodyFactTable(h, a.id, a.file_id) + if (!b) throw new Error('execution fact codec rejected its output'); a.body_facts = b + } catch (n) { + const c = n instanceof IndexBodyFactBoundsError + ctx.d.push({ + id: `canonical-index.execution.${c ? 'owner-bound' : 'invalid'}.${hash(a.id).slice(0, 16)}`, + level: 'error', evidence: { file_id: a.file_id, range: a.range }, message: c + ? `Execution facts exceeded a per-owner safety bound for ${a.name}; body facts were omitted` : `Invalid execution facts for ${a.name}; body facts were omitted` }) + delete a.body_facts } + } } -function sort(edges: readonly IndexEdge[]): IndexEdge[] { - const structuralRoutes = new Map(); - const retained: IndexEdge[] = []; - for (const edge of edges) { - if (edge.kind !== 'routes_through') { - retained.push(edge); - continue; - } - const key = `${edge.from}\u0000${edge.to}\u0000${edge.kind}`; - const existing = structuralRoutes.get(key); - if (!existing - || ct(JSON.stringify(edge), JSON.stringify(existing)) < 0) { - structuralRoutes.set(key, edge); +function sort(e: readonly IndexEdge[]): IndexEdge[] { + const a = new Map(), c: IndexEdge[] = [] + for (const b of e) { + if (b.kind !== 'routes_through') { + c.push(b); continue } + const key = `${b.from}\u0000${b.to}\u0000${b.kind}` + const d = a.get(key) + if (!d + || ct(JSON.stringify(b), JSON.stringify(d)) < 0) + a.set(key, b) } - return [...retained, ...structuralRoutes.values()].sort((left, right) => ct(JSON.stringify(left), JSON.stringify(right))); + return [...c, ...a.values()].sort((g, f) => + ct(JSON.stringify(g), JSON.stringify(f))) } -export function collectExecutionSemantics(input: CollectExecutionInput): CollectExecutionResult { - const symbolsById = new Map(input.symbols.map((symbol) => [symbol.id, symbol])) +export function collectExecutionSemantics(h: CollectExecutionInput): CollectExecutionResult { + const j = new Map(h.symbols.map((k) => [k.id, k])) const ctx: CollectionState = { - input, - symbolsById, - facts: new Map(), - overflow: new Set(), - effects: new Map(), - calls: new Map(), - callIds: new Map(), - channels: new Map(), - edges: [], - diagnostics: [], - seenDiagnostics: new Set(), - unstable: new Set(), - mapQueues: new Map(), - emitters: new Map(), - nestQueues: new Map(), - files: new Map(), - } - const filesById = new Map() - for (const sf of input.sourceFiles) { - const fileId = input.pathToFileId.get(sf.fileName) - if (!fileId) continue - const file: FileContext = { - sf, - fileId, - imports: im(sf), - owners: os(sf, input.symbolsByFile.get(fileId) ?? []), - } - filesById.set(fileId, file) - ctx.files.set(sf, file) + i: h, y: j, f: new Map(), o: new Set(), + e: new Map(), c: new Map(), ci: new Map(), ch: new Map(), + g: [], d: [], sd: new Set(), u: new Set(), q: new Map(), w: new Map(), + p: new Map(), r: new Map(), mq: new Map(), em: new Map(), + nq: new Map(), fs: new Map(), } - prep(ctx) - for (const file of filesById.values()) { - cnest(file, ctx) + const a = new Map() + for (const sf of h.sourceFiles) { + const b = h.pathToFileId.get(sf.fileName) + if (!b) continue + const l: FileContext = { + sf, id: b, im: im(sf), + os: os(sf, h.symbolsByFile.get(b) ?? []), v: 0, nv: 1, + } + a.set(b, l) + ctx.fs.set(sf, l) } - for (const file of filesById.values()) { - collect(file, ctx) - nc(file, ctx) + prep(ctx) + for (const t of a.values()) cnest(t, ctx) + for (const m of a.values()) { + collect(m, ctx) + nc(m, ctx) } - pe(ctx, filesById) + pe(ctx, a) at(ctx) return { - channels: [...ctx.channels.values()].sort((left, right) => - ct(left.id, right.id)), - edges: sort(ctx.edges), - diagnostics: [...ctx.diagnostics].sort((left, right) => - ct(left.id, right.id)), + channels: [...ctx.ch.values()].sort((x, n) => + ct(x.id, n.id)), + edges: sort(ctx.g), + diagnostics: [...ctx.d].sort((z, s) => + ct(z.id, s.id)), } } diff --git a/src/domain/query/index-status.ts b/src/domain/query/index-status.ts index ee5d8312..3eac4239 100644 --- a/src/domain/query/index-status.ts +++ b/src/domain/query/index-status.ts @@ -312,6 +312,10 @@ function buildMaps(view: KnowledgeGraph, files: ReadonlyMap): Ex const bodyFacts = decodeIndexBodyFactTable(a.body_facts, id, fileId); if (!bodyFacts) fail('symbol body facts'); + const ps = bodyFacts.filter((fact) => fact.kind === 'persistence'); + const po = new Set(ps.map((fact) => fact.order[3])); + if (po.size !== ps.length || ps.some((_, index) => !po.has(index + 1))) + fail('persistence order'); const orders = orderKeys.get(id) ?? new Set(); orderKeys.set(id, orders); for (const fact of bodyFacts) { @@ -439,7 +443,6 @@ function buildMaps(view: KnowledgeGraph, files: ReadonlyMap): Ex || call.evidence.excerpt_sha256 !== fact.evidence.excerpt_sha256 || call.order[0] !== fact.order[0] || call.order[2] !== fact.order[2] - || call.order[3] !== fact.order[3] || JSON.stringify(call.control) !== JSON.stringify(fact.control) || !bounded(fact.receiver_type, MAX_TEXT)) { fail('persistence call reference'); diff --git a/tests/unit/canonical-index-execution-hardening.test.ts b/tests/unit/canonical-index-execution-hardening.test.ts index 3bfdf786..0f6e806b 100644 --- a/tests/unit/canonical-index-execution-hardening.test.ts +++ b/tests/unit/canonical-index-execution-hardening.test.ts @@ -739,7 +739,7 @@ ${calls} }), ])) expect(outgoing(edges, ownerId, 'publishes_to')).toEqual([]) - }) + }, 60_000) it('is deterministic when the scanner enumerates execution files in reverse', () => { const sources = { diff --git a/tests/unit/canonical-index-execution-review-regressions.test.ts b/tests/unit/canonical-index-execution-review-regressions.test.ts index 98e78a92..6d607600 100644 --- a/tests/unit/canonical-index-execution-review-regressions.test.ts +++ b/tests/unit/canonical-index-execution-review-regressions.test.ts @@ -510,4 +510,718 @@ ${calls} ])) expect(outgoing(edges, owner[0], 'publishes_to')).toEqual([]) }) + + it('does not prove a queue removed from a Map', () => { + const source = `import { Queue } from 'bullmq' +const queues = new Map() +queues.set('alias', new Queue('real')) +queues.delete('alias') +export function publish() { + return queues.get('alias')!.add('complete', {}) +} +` + const { nodes, edges } = build({ 'src/map-delete.ts': source }) + expect(publishedQueueKeys(nodes, edges, symbol(nodes, 'publish')[0])) + .not.toContain('real') + }) + + it('preserves conditional reachability after dead tail text', () => { + const source = `declare function dead(): void +declare function after(): void +export function run(flag: boolean): void { + if (flag) { + return + dead() + } + after() +} +` + const { nodes } = build({ 'src/reachability-tail.ts': source }) + const calls = facts(nodes, symbol(nodes, 'run'), 'call') + expect(calls.some((fact) => + fact.kind === 'call' && fact.callee === 'dead')).toBe(false) + const after = calls.find((fact) => + fact.kind === 'call' && fact.callee === 'after') + expect(after?.control).toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'branch', arm: 'else' }), + ])) + }) + + it('resolves computed mutation keys and fails closed when they are dynamic', () => { + const source = `export function sensitive(target: Record): void { + const key = 'password' + target[key] = 'hunter2' +} +export function nested(target: Record): void { + const first = 'credentials' + const key = first + target[key] = { nested: 'hunter2' } +} +export function dynamic(target: Record, key: string): void { + target[key] = 'hunter2' +} +export function safe(target: Record): void { + const key = 'displayName' + target[key] = 'Ada' +} +export function numeric(items: string[]): void { + items[0] = 'visible' +} +` + const { nodes } = build({ 'src/computed-secret.ts': source }) + for (const name of ['sensitive', 'nested', 'dynamic']) { + const mutation = facts(nodes, symbol(nodes, name), 'mutation') + expect(mutation).toHaveLength(1) + expect(mutation[0]).toMatchObject({ target: expect.stringMatching(/^redacted:/) }) + expect(JSON.stringify(mutation)).not.toContain('hunter2') + } + expect(facts(nodes, symbol(nodes, 'safe'), 'mutation')[0]) + .toMatchObject({ + target: expect.not.stringMatching(/^redacted:/), + value: { kind: 'literal', value: 'Ada' }, + }) + expect(facts(nodes, symbol(nodes, 'numeric'), 'mutation')[0]) + .toMatchObject({ value: { kind: 'literal', value: 'visible' } }) + }) + + it('preserves every persistence effect behind a wrapper call', () => { + const source = `import { writeFile } from 'node:fs/promises' +async function persistBoth(): Promise { + await writeFile('first.json', 'one') + await writeFile('second.json', 'two') +} +export async function run(): Promise { + await persistBoth() + await writeFile('third.json', 'three') +} +` + const built = build({ 'src/wrapped-persistence.ts': source }) + const graph = loadGraphArtifact(generateIndex(built.root).graphPath) + expect(inspectQueryIndex(graph)).toMatchObject({ state: 'ready' }) + const nodes = new Map(graph.nodeEntries()) + const persisted = facts(nodes, symbol(nodes, 'run'), 'persistence') + expect(persisted).toHaveLength(3) + expect(persisted.map((fact) => fact.order[3])).toEqual([2, 3, 1]) + expect(JSON.stringify(persisted)).toContain('first.json') + expect(JSON.stringify(persisted)).toContain('second.json') + expect(JSON.stringify(persisted)).toContain('third.json') + }) + + it('represents switch fallthrough in the executed arm set', () => { + const source = `declare function first(): void +declare function second(): void +export function choose(code: number): void { + switch (code) { + case 1: + first() + case 2: + second() + break + } +} +` + const { nodes } = build({ 'src/switch-fallthrough.ts': source }) + const calls = facts(nodes, symbol(nodes, 'choose'), 'call') + const first = calls.find((fact) => + fact.kind === 'call' && fact.callee === 'first') + const second = calls.filter((fact) => + fact.kind === 'call' && fact.callee === 'second') + const firstArm = first?.control.find((frame) => frame.kind === 'branch') + const secondArms = second.flatMap((fact) => + fact.control.filter((frame) => frame.kind === 'branch')) + expect(secondArms).toEqual(expect.arrayContaining([ + firstArm, + expect.objectContaining({ kind: 'branch' }), + ])) + expect(new Set(secondArms.map((frame) => + frame.kind === 'branch' ? frame.arm : '')).size).toBe(2) + }) + + it('keeps only the surviving guarded switch-fallthrough path', () => { + const source = `declare function first(): void +declare function second(): void +export function choose(code: number, stop: boolean): void { + switch (code) { + case 1: + first() + if (stop) break + case 2: + second() + break + } +} +` + const { nodes } = build({ 'src/switch-guarded-fallthrough.ts': source }) + const calls = facts(nodes, symbol(nodes, 'choose'), 'call') + const first = calls.find((fact) => + fact.kind === 'call' && fact.callee === 'first') + const second = calls.filter((fact) => + fact.kind === 'call' && fact.callee === 'second') + const firstArm = first?.control.find((frame) => + frame.kind === 'branch')?.arm + expect(second).toHaveLength(2) + expect(second.some((fact) => fact.control.some((frame) => + frame.kind === 'branch' && frame.arm === firstArm))).toBe(true) + expect(second.find((fact) => fact.control.some((frame) => + frame.kind === 'branch' && frame.arm === firstArm))?.control) + .toEqual(expect.arrayContaining([ + expect.objectContaining({ kind: 'branch', arm: 'else' }), + ])) + }) + + it('does not fall through when every conditional arm exits a switch clause', () => { + const source = `declare function later(): void +export function choose(code: number, stop: boolean): void { + switch (code) { + case 1: + if (stop) break + else return + case 2: + later() + break + } +} +` + const { nodes } = build({ 'src/switch-all-exit.ts': source }) + expect(facts(nodes, symbol(nodes, 'choose'), 'call') + .filter((fact) => fact.kind === 'call' && fact.callee === 'later')) + .toHaveLength(1) + }) + + it('preserves default and three-clause switch entry paths deterministically', () => { + const source = `declare function shared(): void +export function choose(code: number): void { + switch (code) { + case 1: + default: + shared() + case 2: + shared() + break + } +} +` + const first = build({ + 'src/switch-chain.ts': source, + 'src/companion.ts': 'export const companion = true\n', + }) + const second = build({ + 'src/companion.ts': 'export const companion = true\n', + 'src/switch-chain.ts': source, + }) + const firstOwner = symbol(first.nodes, 'choose') + const secondOwner = symbol(second.nodes, 'choose') + const firstCalls = facts(first.nodes, firstOwner, 'call') + .filter((fact) => fact.kind === 'call' && fact.callee === 'shared') + expect(firstCalls).toHaveLength(5) + expect(new Set(firstCalls.map((fact) => + fact.control.flatMap((frame) => frame.kind === 'branch' + && (frame.arm === 'default' || frame.arm.startsWith('case:')) + ? [frame.arm] + : [])[0])).size).toBe(3) + expect(firstOwner[1].body_facts).toEqual(secondOwner[1].body_facts) + }) + + it('binds fallthrough persistence facts to their matching call paths', () => { + const source = `import { writeFile } from 'node:fs/promises' +export async function persist(code: number): Promise { + switch (code) { + case 1: + case 2: + await writeFile('report.json', 'ready') + break + } +} +` + const { nodes } = build({ 'src/switch-persistence.ts': source }) + const owner = symbol(nodes, 'persist') + const calls = facts(nodes, owner, 'call').filter((fact) => + fact.kind === 'call' && fact.callee === 'writeFile') + const persisted = facts(nodes, owner, 'persistence') + expect(calls).toHaveLength(2) + expect(persisted).toHaveLength(2) + expect(new Set(persisted.map((fact) => + fact.kind === 'persistence' ? fact.call_fact_id : ''))) + .toEqual(new Set(calls.map((fact) => fact.id))) + }) + + it('fails an oversized or effectful switch owner closed', () => { + const clauses = Array.from( + { length: 33 }, + (_, index) => `case ${index}: break`, + ).join('\n') + const oversized = build({ + 'src/switch-overflow.ts': `export function choose(code: number): void { + switch (code) { + ${clauses} + } +} +`, + }) + const effectful = build({ + 'src/switch-effect.ts': `declare function selector(): number +declare function work(): void +export function choose(code: number): void { + switch (code) { + case selector(): work() + } +} +`, + }) + for (const candidate of [oversized, effectful]) { + expect(Object.hasOwn( + symbol(candidate.nodes, 'choose')[1], + 'body_facts', + )).toBe(false) + expect(candidate.result.diagnostics).toEqual(expect.arrayContaining([ + expect.objectContaining({ + level: 'error', + id: expect.stringContaining('execution.owner-bound'), + }), + ])) + } + }) + + it('preserves secret taint through nested object keys', () => { + const source = `export const SETTINGS = { + credentials: { value: 'hunter2' }, +} +` + const { nodes } = build({ 'src/nested-secret.ts': source }) + expect(JSON.stringify( + facts(nodes, symbol(nodes, 'SETTINGS'), 'literal'), + )).not.toContain('hunter2') + }) + + it('does not reuse a reassigned injected Queue identity', () => { + const source = `import { InjectQueue } from '@nestjs/bullmq' +import type { Queue } from 'bullmq' +export class Service { + constructor(@InjectQueue('reports') private queue: Queue) {} + replace(queue: Queue): void { this.queue = queue } + publish() { return this.queue.add('complete', {}) } +} +` + const { nodes, edges } = build({ 'src/injected-reassign.ts': source }) + expect(publishedQueueKeys( + nodes, + edges, + symbol(nodes, 'Service.publish')[0], + )).not.toContain('reports') + }) + + it('does not prove stale Promise lanes after array mutation', () => { + const source = `declare function task(value: string): Promise +const BATCHES = [['a'], ['b']] +BATCHES.push(['c']) +export async function run(): Promise { + await Promise.allSettled(BATCHES.map((batch) => task(batch[0]!))) +} +` + const { nodes } = build({ 'src/mutable-lanes.ts': source }) + expect(facts(nodes, symbol(nodes, 'run'), 'parallel')).toEqual([]) + }) + + it('does not reuse a reassigned typed emitter scope', () => { + const source = `import { EventEmitter } from 'node:events' +export class Service { + constructor(private events: EventEmitter) {} + replace(events: EventEmitter): void { this.events = events } + publish(): void { this.events.emit('ready') } +} +` + const { nodes, edges } = build({ 'src/emitter-reassign.ts': source }) + expect(outgoing( + edges, + symbol(nodes, 'Service.publish')[0], + 'publishes_to', + )).toEqual([]) + }) + + it('does not prove Map topology from an uninvoked function', () => { + const source = `import { Queue } from 'bullmq' +const queues = new Map() +function neverCalled(): void { + queues.set('alias', new Queue('real')) +} +export function publish() { + return queues.get('alias')!.add('complete', {}) +} +` + const { nodes, edges } = build({ 'src/uninvoked-map.ts': source }) + expect(publishedQueueKeys( + nodes, + edges, + symbol(nodes, 'publish')[0], + )).toEqual([]) + }) + + it('accepts Map topology only from an authenticated Nest lifecycle root', () => { + const source = `import { Injectable, OnModuleInit } from '@nestjs/common' +import { Queue } from 'bullmq' +const CONFIGS = [{ name: 'reports' }] +@Injectable() +class Registry implements OnModuleInit { + private queues = new Map() + onModuleInit(): void { + for (const config of CONFIGS) { + this.queues.set(config.name, new Queue(config.name)) + } + } + add(name: string) { return this.queues.get(name)!.add('complete', {}) } +} +class Plain implements OnModuleInit { + private queues = new Map() + onModuleInit(): void { + for (const config of CONFIGS) { + this.queues.set(config.name, new Queue(config.name)) + } + } + add(name: string) { return this.queues.get(name)!.add('complete', {}) } +} +@Injectable() +class Dead implements OnModuleInit { + private queues = new Map() + onModuleInit(): void { + return + this.queues.set('alias', new Queue('dead')) + } + add(name: string) { return this.queues.get(name)!.add('complete', {}) } +} +@Injectable() +class Disabled implements OnModuleInit { + private queues = new Map() + onModuleInit(): void { + if (false) this.queues.set('alias', new Queue('disabled')) + } + add(name: string) { return this.queues.get(name)!.add('complete', {}) } +} +@Injectable() +class Conditional implements OnModuleInit { + private queues = new Map() + constructor(private enabled: boolean) {} + onModuleInit(): void { + if (this.enabled) this.queues.set('alias', new Queue('conditional')) + } + add(name: string) { return this.queues.get(name)!.add('complete', {}) } +} +@Injectable() +class PossiblyEmpty implements OnModuleInit { + private queues = new Map() + constructor(private configs: Array<{ name: string }>) {} + onModuleInit(): void { + for (const config of this.configs) + this.queues.set(config.name, new Queue(config.name)) + } + add(name: string) { return this.queues.get(name)!.add('complete', {}) } +} +export function managed(registry: Registry) { return registry.add('reports') } +export function unregistered(registry: Plain) { return registry.add('reports') } +export function dead(registry: Dead) { return registry.add('alias') } +export function disabled(registry: Disabled) { return registry.add('alias') } +export function conditional(registry: Conditional) { return registry.add('alias') } +export function possiblyEmpty(registry: PossiblyEmpty) { return registry.add('reports') } +` + const { nodes, edges } = build({ 'src/nest-lifecycle.ts': source }) + expect(publishedQueueKeys( + nodes, + edges, + symbol(nodes, 'managed')[0], + )).toEqual(['reports']) + expect(publishedQueueKeys( + nodes, + edges, + symbol(nodes, 'unregistered')[0], + )).toEqual([]) + for (const name of ['dead', 'disabled', 'conditional', 'possiblyEmpty']) { + expect(publishedQueueKeys(nodes, edges, symbol(nodes, name)[0])).toEqual([]) + } + }) + + it('invalidates Map topology after mutation through a stable alias', () => { + const source = `import { Queue } from 'bullmq' +const queues = new Map() +queues.set('alias', new Queue('real')) +const alias = queues +alias.clear() +export function publish() { + return queues.get('alias')!.add('complete', {}) +} +` + const { nodes, edges } = build({ 'src/aliased-map.ts': source }) + expect(publishedQueueKeys( + nodes, + edges, + symbol(nodes, 'publish')[0], + )).toEqual([]) + }) + + it('invalidates topology through container and later-assigned aliases', () => { + const source = `import { Queue } from 'bullmq' +const contained = new Map() +contained.set('alias', new Queue('contained')) +const holder = { contained } +holder.contained.clear() +const assigned = new Map() +assigned.set('alias', new Queue('assigned')) +let alias: Map +alias = assigned +alias.clear() +const destructured = new Map() +destructured.set('alias', new Queue('destructured')) +const wrapper = { destructured } +const { destructured: destructuredAlias } = wrapper +destructuredAlias.clear() +export function fromContainer() { + return contained.get('alias')!.add('complete', {}) +} +export function fromAssignment() { + return assigned.get('alias')!.add('complete', {}) +} +export function fromDestructure() { + return destructured.get('alias')!.add('complete', {}) +} +` + const { nodes, edges } = build({ 'src/indirect-aliases.ts': source }) + for (const name of ['fromContainer', 'fromAssignment', 'fromDestructure']) { + expect(publishedQueueKeys(nodes, edges, symbol(nodes, name)[0])).toEqual([]) + } + }) + + it('invalidates Promise lanes after mutation through a stable alias', () => { + const source = `declare function task(value: string): Promise +const BATCHES = [['a'], ['b']] +const alias = BATCHES +alias.push(['c']) +export async function run(): Promise { + await Promise.allSettled(BATCHES.map((batch) => task(batch[0]!))) +} +` + const { nodes } = build({ 'src/aliased-lanes.ts': source }) + expect(facts(nodes, symbol(nodes, 'run'), 'parallel')).toEqual([]) + }) + + it('does not collapse EventEmitter properties across runtime instances', () => { + const source = `import { EventEmitter } from 'node:events' +export function handle(): void {} +export class Service { + private readonly events = new EventEmitter() + publish(): void { this.events.emit('ready') } + consume(): void { this.events.on('ready', handle) } +} +export function wire(): void { + const producer = new Service() + const consumer = new Service() + producer.publish() + consumer.consume() +} +` + const { nodes } = build({ 'src/instance-events.ts': source }) + expect(channels(nodes, (node) => + node.channel_kind === 'event' && node.key === 'ready')).toEqual([]) + }) + + it('models try-catch-finally completion without indexing dead tails', () => { + const source = `declare function cleanup(): void +declare function dead(): void +declare function reachable(): void +export function finalReturn(): void { + try { return } finally { cleanup() } + dead() +} +export function caughtThrow(): void { + try { throw new Error('stop') } catch {} + reachable() +} +export function caughtReturn(): void { + try { throw new Error('stop') } catch { return } + dead() +} +export function overridden(): void { + try { + try { return } finally { throw new Error('override') } + } catch {} + reachable() +} +export function exhaustiveSwitch(code: 1 | 2): void { + try { + switch (code) { + case 1: return + case 2: return + default: return + } + } finally { cleanup() } + dead() +} +` + const { nodes } = build({ 'src/try-completion.ts': source }) + expect(facts(nodes, symbol(nodes, 'finalReturn'), 'call') + .some((fact) => fact.kind === 'call' && fact.callee === 'dead')).toBe(false) + expect(facts(nodes, symbol(nodes, 'caughtThrow'), 'call') + .some((fact) => fact.kind === 'call' && fact.callee === 'reachable')).toBe(true) + expect(facts(nodes, symbol(nodes, 'caughtReturn'), 'call') + .some((fact) => fact.kind === 'call' && fact.callee === 'dead')).toBe(false) + expect(facts(nodes, symbol(nodes, 'overridden'), 'call') + .some((fact) => fact.kind === 'call' && fact.callee === 'reachable')).toBe(true) + expect(facts(nodes, symbol(nodes, 'exhaustiveSwitch'), 'call') + .some((fact) => fact.kind === 'call' && fact.callee === 'dead')).toBe(false) + }) + + it('omits the impossible direct path for a duplicate switch case', () => { + const source = `declare function first(): void +declare function second(): void +export function run(code: number): void { + switch (code) { + case 1: + first() + case 1: + second() + break + } +} +enum Code { First = 1, Alias = 1 } +export function enumRun(code: Code): void { + switch (code) { + case Code.First: + first() + case Code.Alias: + second() + break + } +} +export function zeroRun(code: number): void { + switch (code) { + case 0: + first() + case -0: + second() + break + } +} +` + const { nodes } = build({ 'src/duplicate-case.ts': source }) + expect(facts(nodes, symbol(nodes, 'run'), 'call') + .filter((fact) => fact.kind === 'call' && fact.callee === 'second')) + .toHaveLength(1) + expect(facts(nodes, symbol(nodes, 'enumRun'), 'call') + .filter((fact) => fact.kind === 'call' && fact.callee === 'second')) + .toHaveLength(1) + expect(facts(nodes, symbol(nodes, 'zeroRun'), 'call') + .filter((fact) => fact.kind === 'call' && fact.callee === 'second')) + .toHaveLength(1) + }) + + it('substitutes wrapper conditions before propagating persistence', () => { + const source = `import { writeFile } from 'node:fs/promises' +async function maybePersist(enabled: boolean, path: string): Promise { + if (enabled) await writeFile(path, 'ready') +} +export async function disabled(): Promise { + await maybePersist(false, 'disabled.json') +} +export async function enabled(): Promise { + await maybePersist(true, 'enabled.json') +} +async function afterGuard(skip: boolean, path: string): Promise { + if (skip) return + await writeFile(path, 'ready') +} +export async function skipped(): Promise { + await afterGuard(true, 'skipped.json') +} +export async function continued(): Promise { + await afterGuard(false, 'continued.json') +} +async function equalityGuard(enabled: boolean, path: string): Promise { + if (enabled === false) return + await writeFile(path, 'ready') +} +export async function equalityDisabled(): Promise { + await equalityGuard(false, 'equality-disabled.json') +} +export async function equalityEnabled(): Promise { + await equalityGuard(true, 'equality-enabled.json') +} +async function negated(enabled: boolean, path: string): Promise { + if (!enabled) return + await writeFile(path, 'ready') +} +export async function negatedDisabled(): Promise { + await negated(false, 'negated-disabled.json') +} +export async function negatedEnabled(): Promise { + await negated(true, 'negated-enabled.json') +} +async function defaults(enabled = false, path = 'default.json'): Promise { + if (!enabled) return + await writeFile(path, 'ready') +} +export async function omitted(): Promise { + await defaults() +} +async function insideTry(path: string): Promise { + try { await writeFile(path, 'ready') } finally {} +} +export async function tried(): Promise { + await insideTry('tried.json') +} +` + const { nodes } = build({ 'src/conditional-wrapper.ts': source }) + expect(facts(nodes, symbol(nodes, 'disabled'), 'persistence')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'enabled'), 'persistence')).toEqual([ + expect.objectContaining({ + operation: 'file_write', + resource: { kind: 'literal', value: 'enabled.json' }, + }), + ]) + expect(facts(nodes, symbol(nodes, 'skipped'), 'persistence')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'continued'), 'persistence')).toEqual([ + expect.objectContaining({ + operation: 'file_write', + resource: { kind: 'literal', value: 'continued.json' }, + }), + ]) + expect(facts(nodes, symbol(nodes, 'equalityDisabled'), 'persistence')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'equalityEnabled'), 'persistence')).toEqual([ + expect.objectContaining({ + resource: { kind: 'literal', value: 'equality-enabled.json' }, + }), + ]) + expect(facts(nodes, symbol(nodes, 'negatedDisabled'), 'persistence')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'negatedEnabled'), 'persistence')).toEqual([ + expect.objectContaining({ resource: { kind: 'literal', value: 'negated-enabled.json' } }), + ]) + expect(facts(nodes, symbol(nodes, 'omitted'), 'persistence')).toEqual([]) + expect(facts(nodes, symbol(nodes, 'tried'), 'persistence')).toEqual([ + expect.objectContaining({ resource: { kind: 'literal', value: 'tried.json' } }), + ]) + }) + + it('does not propagate effects through reassigned callable identities', () => { + const source = `import { writeFile } from 'node:fs/promises' +async function persist(path: string): Promise { + await writeFile(path, 'ready') +} +async function ignore(_path: string): Promise {} +async function assignedPersist(path: string): Promise { + await writeFile(path, 'ready') +} +let selected = persist +selected = ignore +const holder = { selected: persist } +holder.selected = ignore +const assigned = { selected: assignedPersist } +Object.assign(assigned, { selected: ignore }) +const defined = { selected: persist } +Object.defineProperty(defined, 'selected', { value: ignore }) +export async function run(): Promise { + await selected('first.json') + await holder.selected('second.json') + await assigned.selected('third.json') + await defined.selected('fourth.json') +} +` + const { nodes } = build({ 'src/reassigned-callable.ts': source }) + expect(facts(nodes, symbol(nodes, 'run'), 'persistence')).toEqual([]) + }) }) diff --git a/tests/unit/core-reset-governance.test.ts b/tests/unit/core-reset-governance.test.ts index 2f2b7b3b..d9320518 100644 --- a/tests/unit/core-reset-governance.test.ts +++ b/tests/unit/core-reset-governance.test.ts @@ -432,22 +432,22 @@ const SEMANTIC_EXECUTION_PACKAGE_EXCLUSIONS = [ ] as const const SEMANTIC_EXECUTION_SOURCE = { production_typescript_files: 44, - production_typescript_loc: 15_944, - production_loc_added: 3_677, + production_typescript_loc: 15_934, + production_loc_added: 3_667, production_loc_removed: 187, - production_loc_net: 3_490, + production_loc_net: 3_480, } as const const SEMANTIC_EXECUTION_PACKAGE = { npm_files: 102, - npm_packed_bytes: 143_549, - npm_unpacked_bytes: 639_217, - npm_shasum: 'fe83717734be8e1ae554e6e936dc2026c8c65245', + npm_packed_bytes: 145_254, + npm_unpacked_bytes: 638_736, + npm_shasum: '9f0c66e663f703afbb9a5e68f6037f9e211cba58', npm_integrity: - 'sha512-Eo9IyQucf9VN/UvRje9ABgID3FdH9Bdq2cxBBPz1ywhf9OOar4Tgfy1tYR5yxf5r8X9LbNUftBLRQaoET9eV1g==', - npm_artifact_sha256: '09af6e9193bd0033f0daeb3621f437852679b95f68b82e0cd346b79974185554', + 'sha512-3yYpFxnym0r9DF66IfS8w1MI01DMLU+hX6uQi6aQoBQvbeu6jHn8j059N6ml3MwMx3wvlj41Y3yAMIX3D2N2Aw==', + npm_artifact_sha256: 'b49bf7a1eae2b230da4d0b7a778f6112e1bc9d986c083573bc3bd7296a99c670', } as const const SEMANTIC_EXECUTION_DIFF_SHA256 = - 'a7feb3f23afb0e2d19837b6ff3bdf980c6c2fecbfc70c189d8a8849db7c57481' + 'e712d06a2c43cc0223c2c4219691ddc860f2f05d26c689b19b8ade2d180c99a6' const CAPABILITY_VALIDATION_V2_PROPOSAL_SHA256 = '4906405cbb806c850c0612305ef460e023e2060b5338734ae0af12303901cbd0' const CAPABILITY_VALIDATION_V2_ISSUE = 'https://github.com/mohanagy/madar/issues/612' @@ -1921,6 +1921,8 @@ describe('core reset governance', () => { removed: SEMANTIC_EXECUTION_SOURCE.production_loc_removed, net: SEMANTIC_EXECUTION_SOURCE.production_loc_net, diff_sha256: SEMANTIC_EXECUTION_DIFF_SHA256, + execution_source_sha256: + 'cf43e183f9d001764f4fdd22b073bef8854f9d949fef5793905291b870d3404b', }, package_measurement: { files: SEMANTIC_EXECUTION_PACKAGE.npm_files, @@ -1931,27 +1933,49 @@ describe('core reset governance', () => { artifact_sha256: SEMANTIC_EXECUTION_PACKAGE.npm_artifact_sha256, }, local_verification: { - focused_tests_passed: 163, + focused_tests_passed: 143, last_pre_cache_full_tests_passed: 722, - candidate_full_suite_status: 'pending_clean_runner_after_local_host_memory_pressure', - candidate_full_suite_local_attempts: 3, + candidate_full_suite_status: + 'all_761_unique_tests_passed_but_single_process_full_coverage_blocked_by_local_fork_worker_start_pressure', + candidate_full_suite_local_attempts: 6, + default_fork_completed_test_files: 79, + default_fork_completed_tests_passed: 707, + default_fork_unstarted_file: 'tests/unit/retrieve-context.test.ts', + unstarted_file_isolated_tests_passed: 54, + unique_tests_passed_across_default_fork_runs: 761, + local_independent_review: 'no_blocker', graph_nodes: 12_313, - graph_edges: 32_726, + graph_edges: 32_717, exact_queue_channels: 6, - exact_channel_edges: 51, - graph_artifact_size_ratio: 1.2293169600080505, + exact_job_channels: 0, + exact_event_channels: 0, + exact_channel_edges: 42, + publishes_to_edges: 35, + consumed_by_edges: 7, + graph_artifact_bytes: 60_267_088, + graph_artifact_size_ratio: 1.2291718874663748, beta4_indexing_trials_seconds: [20.32, 21.33, 22.16, 22.51, 22.74], beta4_indexing_median_seconds: 22.16, - candidate_indexing_trials_seconds: [13.98, 14.54, 13.71, 13.27, 14.07], - candidate_indexing_median_seconds: 13.98, - indexing_median_ratio: 0.6308664259927798, + candidate_indexing_trials_seconds: [14.83, 12.99, 13.05, 14.16, 13.17], + candidate_indexing_median_seconds: 13.17, + indexing_median_ratio: 0.5943140794223827, warm_retrieval_samples: 100, - warm_retrieval_median_ms: 233.9935625, - warm_retrieval_p95_ms: 276.41525, - warm_retrieval_max_ms: 301.64975, + warm_retrieval_median_ms: 156.7398119999998, + warm_retrieval_p95_ms: 177.5851354499996, + warm_retrieval_max_ms: 183.07708300000013, deterministic_graph_sha256: - '1f993903a29bc5d523de173a1a86218c7ab304369ac159f282d75f67129d6076', + '77270a6f0330a3ce85fbc42b90e7a3e99f8bf37776f6e65f5da8aad1bad3caaf', + broad_retrieval_files: 10, + broad_retrieval_snippets: 10, + broad_retrieval_serialized_tokens: 3_669, + broad_retrieval_disconnected_boundaries: 4, + broad_retrieval_output_bytes: 13_376, + broad_retrieval_output_sha256: + 'ca003cd8eb0be3273589b556ad0acaf2c49588a1d9e00dccf4e5f36da795b996', beta4_retrieval_output_byte_identical: true, + beta4_retrieval_output_bytes: 15_294, + beta4_retrieval_output_sha256: + '87b4ef75473834708b20f1d2580b31470a710d797d7bdf55eee1d0876827a173', exact_head_ci: 'pending', independent_review: 'pending', }, diff --git a/tests/unit/query-index-execution-validation.test.ts b/tests/unit/query-index-execution-validation.test.ts index 0d68f592..ffd143db 100644 --- a/tests/unit/query-index-execution-validation.test.ts +++ b/tests/unit/query-index-execution-validation.test.ts @@ -571,6 +571,27 @@ describe('query execution index validation', () => { }) }) + it('rejects a re-sealed sparse persistence ordinal', () => { + const current = fixture() + mutateBodyFactWireRows(current.graph, current.runId, (rows) => { + const persistence = rowOfKind(rows, 6) + const evidence = persistence[5] + if (!Array.isArray(evidence) || typeof evidence[8] !== 'string') { + throw new Error('Expected persistence evidence') + } + persistence[4] = 9 + persistence[0] = indexBodyFactId( + current.runId, + 'persistence', + [persistence[2], 6, persistence[3], persistence[4]] as number[], + evidence[8], + persistence.slice(1), + ) + }) + resign(current.graph) + expect(inspectQueryIndex(current.graph)).toMatchObject({ state: 'corrupt' }) + }) + it('authenticates selected-owner channel edge bytes before returning evidence', () => { const current = fixture() const edge = current.graph.edgeEntries().find(([from, , attributes]) => From c9d2c6f730c5e4e426d6f2f4b1aa4c0623bb4067 Mon Sep 17 00:00:00 2001 From: mohammed naji Date: Fri, 31 Jul 2026 14:13:02 +0400 Subject: [PATCH 5/5] docs: complete semantic verification cohort --- docs/core-reset/removal-manifest.yml | 6 ++++-- tests/unit/core-reset-governance.test.ts | 2 +- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/core-reset/removal-manifest.yml b/docs/core-reset/removal-manifest.yml index e48cd065..a9e6c8d0 100644 --- a/docs/core-reset/removal-manifest.yml +++ b/docs/core-reset/removal-manifest.yml @@ -2236,6 +2236,8 @@ items: - tests/unit/canonical-index-execution-review-regressions.test.ts - tests/unit/query-index-execution-validation.test.ts - tests/unit/retrieve-context.test.ts + - tests/unit/retrieve-evidence-skeleton-adversarial.test.ts + - tests/unit/retrieve-evidence-skeleton-regression.test.ts - tests/unit/graph-artifact.test.ts - tests/unit/stdio-server.test.ts - tests/unit/core-reset-governance.test.ts @@ -2281,7 +2283,7 @@ items: integrity: sha512-3yYpFxnym0r9DF66IfS8w1MI01DMLU+hX6uQi6aQoBQvbeu6jHn8j059N6ml3MwMx3wvlj41Y3yAMIX3D2N2Aw== artifact_sha256: b49bf7a1eae2b230da4d0b7a778f6112e1bc9d986c083573bc3bd7296a99c670 local_verification: - focused_tests_passed: 143 + focused_tests_passed: 196 last_pre_cache_full_tests_passed: 722 candidate_full_suite_status: all_761_unique_tests_passed_but_single_process_full_coverage_blocked_by_local_fork_worker_start_pressure candidate_full_suite_local_attempts: 6 @@ -2349,7 +2351,7 @@ items: registry_metadata_publication: forbidden tag: forbidden main_target: forbidden - notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. First PR head 9fe3c2448958c6b8cead2452758077fef093cf4e passed all six hosted jobs but independent review blocked twelve semantic-proof classes; later corrective heads c977de03ecba7958d03966df728abed9f1b36ff7 and f4ae64402d89ccf639bf698687b3767678ab708c were also stopped and not merged. The new local candidate retains the prior corrections and generically closes the f4ae false-proof classes: stale mutable Map, injected Queue and typed EventEmitter identities; dead-tail reachability; computed and nested secret taint; wrapper persistence multiplicity; exact switch-fallthrough path evidence; and mutated Promise inputs. It also validates dense persistence ordinals and gives only the 8,193-call synthetic boundary test a 60-second timeout without changing the production bound. The corrected topology contains 42 typed channel edges rather than the stale 51 count because nine outer-callsite projections were duplicate paths through dispatchWave or checkAndDispatchNext; all unique wrapper-owner producers, six queue channels, seven consumers, and call links remain. Exact local source, package, 143 focused assertions, real-corpus graph-size, channel-topology, broad and beta.4 retrieval compatibility, indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The latest default-fork coverage attempt passed 79 files / 707 tests before the busy local host failed to start the final retrieve-context worker; that exact file then passed 54/54 alone under default forks, proving all 761 unique tests while leaving the one-process full-suite gate honestly pending for clean exact-head CI. Independent review found no blocker on the frozen implementation and corrected governance diff. Exact corrected-head CI, CodeRabbit completion, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' + notes: 'Issue #632 extends the canonical index with authenticated ordered body facts, exact shared queue/job/event channels, bounded two-hop wrapper substitution, concurrency groups, and receiver/type-proven persistence. It is active from exact protected next commit 9043320cfa08370e5cdd3911bfb9283005aa9912 and tree f51d6e75e3b806dec6caf9ff0be43fc2ab5713fc. First PR head 9fe3c2448958c6b8cead2452758077fef093cf4e passed all six hosted jobs but independent review blocked twelve semantic-proof classes; later corrective heads c977de03ecba7958d03966df728abed9f1b36ff7 and f4ae64402d89ccf639bf698687b3767678ab708c were also stopped and not merged. The new local candidate retains the prior corrections and generically closes the f4ae false-proof classes: stale mutable Map, injected Queue and typed EventEmitter identities; dead-tail reachability; computed and nested secret taint; wrapper persistence multiplicity; exact switch-fallthrough path evidence; and mutated Promise inputs. It also validates dense persistence ordinals and gives only the 8,193-call synthetic boundary test a 60-second timeout without changing the production bound. The corrected topology contains 42 typed channel edges rather than the stale 51 count because nine outer-callsite projections were duplicate paths through dispatchWave or checkAndDispatchNext; all unique wrapper-owner producers, six queue channels, seven consumers, and call links remain. Exact local source, package, 196 focused assertions, real-corpus graph-size, channel-topology, broad and beta.4 retrieval compatibility, indexing-median, warm-retrieval-p95, and deterministic-graph receipts are recorded above. The latest default-fork coverage attempt passed 79 files / 707 tests before the busy local host failed to start the final retrieve-context worker; that exact file then passed 54/54 alone under default forks, proving all 761 unique tests while leaving the one-process full-suite gate honestly pending for clean exact-head CI. Independent review found no blocker on the frozen implementation and corrected governance diff. Exact corrected-head CI, CodeRabbit completion, zero-thread, merge, and publication receipts remain absent until those gates actually pass.' exit_gate: Every retained fact and exact channel edge is deterministic, source-authenticated and mutation-sensitive; false persistence/channel matches remain absent; source, graph-size, indexing, warm-retrieval, package, full-test, exact-head CI, independent-review, and zero-thread gates pass without a v2 result cutover, new dependency, publication, tag, release, Registry metadata, or main target. - id: obligation-driven-retrieval-630 diff --git a/tests/unit/core-reset-governance.test.ts b/tests/unit/core-reset-governance.test.ts index d9320518..70617df1 100644 --- a/tests/unit/core-reset-governance.test.ts +++ b/tests/unit/core-reset-governance.test.ts @@ -1933,7 +1933,7 @@ describe('core reset governance', () => { artifact_sha256: SEMANTIC_EXECUTION_PACKAGE.npm_artifact_sha256, }, local_verification: { - focused_tests_passed: 143, + focused_tests_passed: 196, last_pre_cache_full_tests_passed: 722, candidate_full_suite_status: 'all_761_unique_tests_passed_but_single_process_full_coverage_blocked_by_local_fork_worker_start_pressure',