Skip to content

feat: match repeated findings across scan history - #575

Open
mldangelo-oai wants to merge 17 commits into
mdangelo/codex/scan-comparison-storagefrom
mdangelo/codex/match-repeated-findings
Open

feat: match repeated findings across scan history#575
mldangelo-oai wants to merge 17 commits into
mdangelo/codex/scan-comparison-storagefrom
mdangelo/codex/match-repeated-findings

Conversation

@mldangelo-oai

@mldangelo-oai mldangelo-oai commented Aug 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Match repeated findings across completed scan history with compact, paged catalogues and deterministic confirmed identities. Keep this pull request stacked on #574, reuse existing Codex authentication, and add the intended public SDK matcher without changing the public CLI surface.

Changes

  • Group stable finding identities and previously confirmed relationships into compact catalogue records, then expand matching decisions to every original occurrence.
  • Page catalogue records and requested evidence within the existing upstream message limit; require complete evidence before accepting a result.
  • Batch omitted finding-evidence selections using the exact existing Unicode message limit, including serialized identifiers and continuation-envelope overhead.
  • Confirm known and stable identities deterministically, reconcile intermediate model ambiguity, and validate every final comparison before saving it.
  • Preserve confirmed, uncertain, and related relationships separately across chronological matching, scan history, existing interactive output, and component findings.
  • Reject empty or globally repeated occurrence identifiers before grouping or model execution, and ignore empty or whitespace-only stable finding identities without merging unrelated findings.
  • Disable repository project instructions, tools, inherited MCP servers, approvals, and network access for read-only matching turns.
  • Preserve cancellation, already saved comparisons, existing cost-budget behavior, surviving forced-pair relationships, and line-wrap-tolerant cross-platform workbench capability detection.
  • Export matchScanFindings with self-contained public input, result, progress, cancellation, and reasoning-effort types, including the existing supported maximum reasoning effort.
  • Inherit the reviewed parent fixes, append-only history migration, bundled-plugin version 0.1.39, and branch package version 0.1.19; the conflict-free current-base merge ref retains the newer package release.
  • Extend installed-package consumer checks for the public matcher while retaining the current Cloud and publication API checks.

Testing

Validated on exact proposed head 02abe26a52c32e6a6caf323f9088a26da2f57bc9:

  • Matching, catalogue, property, and workbench CLI suites: 129 passed, 3 platform-specific skips, 0 failed.
  • Component, public API, scan-history rendering and storage, publication-store, Windows-fixture, and scan-contract suites: 246 passed, 3 platform-specific skips, 0 failed.
  • Sealed scan-matching end-to-end, general CLI, and completed post-scan artifact-preservation suites: 149 passed, 0 failed.
  • Complete runtime suite, including previous installed-plugin upgrades and legacy, previous, independently versioned, development, current, and narrow-terminal custom-plugin comparison support: 134 passed, 11 platform-specific skips, 0 failed.
  • Cloud, Linear, publication-preflight, publication, and publication CLI suites: 173 passed, 0 failed.
  • pnpm run types, pnpm run format, package build and packing, and git diff --check: passed.
  • Installed npm-package smoke: @openai/codex-security@0.1.19, public import and matcher, NodeNext consumer types using maximum reasoning effort, CLI, 113 bundled-plugin files, bundled Codex version, nested worker, current publication exports, and all 273 archive entries passed.
  • Regression coverage includes duplicate and empty occurrence identifiers, blank stable identities and malformed historical bridges, repository-instruction isolation, transitive finding identities, omitted or split stable matches, legitimate resolved and newly discovered findings, model ambiguity, Unicode evidence batching at the upstream limit, wrapped capability discovery, cancellation, and unchanged sealed artifacts.

Risk and rollout

This pull request depends on #574 and must be landed after its parent. Existing saved comparisons are not rewritten automatically; the existing forced all-scan matching command can recompute prior comparisons when explicitly requested.

Confirmed matches can affect later history comparisons. Deterministic identity reconciliation, complete evidence requirements, globally unique occurrence identifiers, and final comparison validation keep existing relationships internally consistent without collapsing distinct related issues.

The intended matchScanFindings SDK export is public API. No CLI commands, flags, accepted values, public environment variables, or defaults are added or changed.

Public disclosure review

  • No customer, partner, prospect, or user identities, data, or identifying details are included.
  • No credentials, personal data, private source, scan findings, or nonpublic links or tickets are included.
  • I reviewed the branch name, title, description, commits, changes, comments, logs, screenshots, attachments, and links for public disclosure.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, 5875cb27239e4a310be1bc3cd4c58b67275b5319, against #574. This is the matcher and existing CLI/SDK integration split from #567. The storage contract is reviewed separately in #574.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 5875cb2723

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

The description now includes the final QA results and merge order. Please review the current head, 5875cb27239e4a310be1bc3cd4c58b67275b5319. The code has not changed since the earlier review.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: 5875cb2723

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 20, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-25T06:57:21.218057Z 02abe26 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c3d4a5661b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/scan-comparison.ts

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, c3d4a5661b31870dd2960432f3390f491af5d9ee. The PR description includes the latest cleanup and QA results.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: c3d4a5661b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, c8b75fc305061c6fa0a39e842f111a0d9c9707e4. This also fixes the omitted-evidence case raised in the previous review and adds tests for all four final outcomes.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: c8b75fc305

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review the current head, 1b1e546213d1a041e3daf03985f24d24b8ade8a8, against #574. This additive update merges current main, preserves failed-history output semantics, recomputes forced single-pair matches without old semantic groups, and compares complete selected scans before caching automatic results. The PR description includes the regressions, package checks, synthetic live QA, and the remaining full-suite checks.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 1b1e546213

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

The description now includes the completed full-suite runs, installed-package and synthetic live QA, and the green exact-head CI result. Please review the current head, 1b1e546213d1a041e3daf03985f24d24b8ade8a8, against #574. The code has not changed since the previous review.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 1b1e546213

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/scan-comparison.ts Outdated
@mldangelo-oai mldangelo-oai changed the title feat: match repeated findings across scans feat: match repeated findings across scan history Aug 22, 2026
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 043fa2a80cbfe13cc9ae640339ce92dcb60e9d63 against #574. This update resolves the stacked merge, rejects confirmed and uncertain conflicts before caching, preserves cross-scan uncertainty, uses standard input for saved comparisons, and keeps the published matcher types self-contained.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex security review

Please run a security review on exact head 043fa2a80cbfe13cc9ae640339ce92dcb60e9d63 against #574. Focus on scan-history persistence, model-result validation, cancellation, standard-input transport, and the public SDK boundary.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 043fa2a80c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/scan-comparison.ts
Comment thread sdk/typescript/src/index.ts
…/pr-575

# Conflicts:
#	sdk/typescript/scripts/fixtures/package-consumer.ts
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review

Please review exact head 4110ecec1822c5c09f516e7fa4b61ca6eb8ae185 against #574.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 4110ecec18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kmbroai kmbroai left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Critical review

Reviewed head 4110ecec1822c5c09f516e7fa4b61ca6eb8ae185 against its declared base #574.

Recommendation: the paged matcher is justified; coordinate the base's bundle upgrade before shipping. No new blocking defect found in this child diff. Repeated full-history occurrences should not consume the input budget as if they were independent issues, and large valid evidence should not be silently omitted to fit a request.

Correctness and current feedback

The current matchScanFindingsInternal fixes the earlier review findings: occurrence IDs are checked for nonempty global uniqueness before deterministic reconciliation; native comparison configuration sets project_doc_max_bytes: 0; omitted cards require full evidence even for a final no-match decision. The repository-instruction and occurrence-collision comments referred to an earlier head and should not be reported as still present.

Catalogue/evidence progress is enforced by the host, unknown IDs and invalid offsets are rejected, and Unicode-character pagination matches the documented upstream message boundary rather than imposing a new arbitrary finding cap. Uncertainty crossing a confirmed group remains invalid at the pair projection boundary instead of being silently discarded. The CLI now forwards saved finding groups, and newly confirmed groups are reused by later batches.

Simplification and feature scope

The essential work is a compact catalogue, a bounded-message evidence protocol, and deterministic expansion back to occurrences. Keep those separate from presentation and authentication. The new public SDK export is optional to fixing existing scans match/compare; it commits the project to an imported-finding contract, so retain it only if that API is intended for supported consumers. The added input validation is warranted if it is exported.

For cost-limited automatic matching, the single-turn guard prevents this feature from silently adding pagination turns; it is not itself a dollar-accurate spending guarantee. Keep that wording distinct from the scan cost-accounting contract.

I reproduced #574's unchanged-version cache problem separately. Neither this child's source tests nor its new SDK export fixes the stale installed plugin payload; advance the final stack's bundle version once.

Verification

Ran catalogue, comparison, matching end-to-end, CLI-workbench, and history-renderer suites: 114 passed, 3 skipped, 0 failed. Linux, Bun 1.3.14 / Node 22.13.1 with cached dependencies. Tests use synthetic model replies; I did not assess live model matching quality, run a production history migration, or perform a native Windows run.

@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review
@codex security review

Please review exact head aae0256, including deterministic finding identities, paged untrusted evidence, read-only matcher isolation, globally unique occurrence IDs, final relationship reconciliation, component boundaries, and installed public SDK types.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aae0256d4e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/scan-comparison.ts Outdated
Comment thread sdk/typescript/src/finding-catalogue.ts
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review
@codex security review

Please review exact head a59c713, including canonical nonblank finding identities, upstream-limit Unicode evidence batching, read-only matcher isolation, globally unique occurrence IDs, final relationship reconciliation, component boundaries, explicit wrapped plugin capabilities, and installed public SDK types.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a59c713ebc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread sdk/typescript/src/scan-comparison.ts Outdated
@mldangelo-oai

Copy link
Copy Markdown
Collaborator Author

@codex review
@codex security review

Please review exact head 02abe26, including canonical nonblank finding identities, upstream-limit Unicode evidence batching, read-only matcher isolation, globally unique occurrence IDs, final relationship reconciliation, component boundaries, explicit wrapped plugin capabilities, and installed public SDK types with supported maximum reasoning effort.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 02abe26a52

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants