Skip to content

fix(cli): count only the transitive findings the report carries - #141

Merged
rodrigorjsf merged 4 commits into
mainfrom
fix/finding-selection-contract
Aug 22, 2026
Merged

fix(cli): count only the transitive findings the report carries#141
rodrigorjsf merged 4 commits into
mainfrom
fix/finding-selection-contract

Conversation

@rodrigorjsf

Copy link
Copy Markdown
Owner

Closes #130.

Read this first: the issue's premise does not reproduce on this tree

#130 frames the over-count as a live, user-visible defect — "a --transitive run with a
--baseline that accepts a transitive finding publishes 2 and reports 1", reachable from an ordinary
invocation. That measurement was true when the issue was written and is not true now. Upstream
73dd1f1 ("Security fixes", NVIDIA#393) narrowed what the report node writes into filtered_findings to
the kept side alone, so the state the transitive count reads no longer carries the suppressed
findings it was over-counting.

The fix still lands, and the contract test still earns its place — it induces the pre-73dd1f1
shape deliberately, through a spy, because that shape is exactly what the transitive merge path
hands over today even though the report node no longer produces it. What changes is the framing: this
is a latent contract defect held shut by an upstream change, not a bug a user hits this afternoon. A
reviewer should know that before reading the diff, and #130 gets a comment saying so.

What lands

The over-count, first, as the brief sequenced it. 5e149c4 makes the transitive finding count
count only the findings the report carries. A failing test came first, as the brief required.

The redundancy, second and separately. 1b89e97 leaves exactly one function in the package
deciding which findings a finished Scan reports. #130 measured the fork's reported_findings and
upstream's effective_findings diverging on the raw-findings fallback — the fork's subtracted
suppressed_findings, upstream's deliberately did not. Upstream's semantics won, per this fork's
standing rule that upstream owns the truth about the core, so the fork's function became a
delegation and the reasoning it carried moved to the function that answers.

Review, and what the critics changed

Three independent critics graded this branch — acceptance-criteria conformance, scope and contract,
adversarial correctness. Zero blocking findings. Three of their non-blocking findings were promoted
by the main thread and closed here, because each is a claim that is not true:

The AC-4 assertion was true of itself. test_the_budget_and_the_summary_count_one_result_identically
compared _multi_skill_public_record_count(result) - 1 against len(reported_findings(result))
and that right-hand side is character-for-character the production expression at the call site the
acceptance criterion names. A critic proved it: inlining a raw filtered_findings read at
_combined_skill_entry's finding_countprecisely the regression this issue exists to prevent
left every case green. e9f2b25 routes the comparison through _combined_skill_entry instead and
derives the suppressed record count from the state rather than hard-coding 1. Re-proved by mutating
that one site: now red, restored by hand, git diff --exit-code clean.

Three docstrings still asserted the falsified premise. 1b89e97's body claims the stale claim
lived in three places and "all three are corrected here". Four survived. 199f555 closes them.

A contract document stated a rule the tree does not keep. The new filtered_findings row in
docs/DEVELOPMENT.md ended "Read it through suppression.effective_findings, never directly".
Three legitimate mid-graph reads contradict the absolute, and the row described only what the report
node writes while the transitive merge writes the pre-partition merged population into the same
key beforehand. Now qualified at both ends: what the key holds at each point, and a rule that binds
the consumer asking which findings does a finished Scan report rather than the merge path, which
reads it before anything has been partitioned.

Deliberate deferral, filed rather than buried

reported_findings is now a one-line delegation, so one selection has two names and six call sites.
Removing it is a mechanical change with its own review surface and was deliberately sequenced after
the defect fix. Filed as #140 rather than left in a commit body, per CLAUDE.md's standing rule.
#140 also records the argument for keeping the name, which is not obviously wrong.

Verification

Run in the main thread on this branch, at e9f2b25:

make test-unit      4424 passed, 14 skipped, 38 deselected, 4 xfailed
make lint           All checks passed!
make format-check   258 files already formatted
make update-snapshots + git status --short   -> clean, no churn

Zero snapshots moved. This change is entirely in the CLI's and the report node's reader
functions; the Behavior Snapshot projection scans through the graph and its recorded state is
untouched.

`transitive_finding_count` was a third expression of "which findings does a
finished Scan report", beside `suppression.effective_findings` and
`report.reported_findings`, and the only one that subtracted nothing: it counted
every `filtered_findings` entry carrying a `source_url` straight out of the
report node's return. The merge path now selects through `effective_findings`
first and filters on `source_url` second, so a Baseline that accepts a
transitive Finding lowers the count and the report's list together.

Measured, and it moves the issue's premise: upstream `73dd1f1` ("Security fixes
(NVIDIA#393)", which arrived in the 2.9.6 sync) narrowed what the report node writes
into `filtered_findings` from the pre-partition population to the kept side
alone. The over-count #130 describes -- 2 published against 1 reported -- no
longer reproduces end to end on this tree, and a `--transitive` scan whose
`--baseline` fingerprints a dependency Finding already answers 1 beside a report
listing one. `test_scan_transitive_count_agrees_with_the_report_it_publishes`
pins that as the control.

`test_scan_transitive_count_subtracts_the_baseline_suppressed_partition` pins
the contract instead of the current key's contents: it asserts the count over a
`filtered_findings` holding one kept and one Baseline-suppressed transitive
Finding -- the shape the report node wrote before `73dd1f1` -- where the old
expression answered 2. Restore that expression and it fails `assert 2 == 1`.

Refs #130

Claude-Session: https://claude.ai/code/session_01NsjXzHBDr49KjMJPnLyLCJ
Signed-off-by: Rodrigo França <rodrigo_rjsf@hotmail.com>
`report.reported_findings` and `suppression.effective_findings` were written for
one defect on the two sides of a fork that had not synced, and the 2.9.6 sync
kept both. They agree on selecting `filtered_findings` by presence rather than
truthiness and on subtracting the `suppressed_findings` partition, and diverge
on one input: with `filtered_findings` absent or malformed, the fork's
subtracted that partition from the raw `findings` list and upstream's
deliberately did not, because raw findings are not the population that produced
it.

Upstream's semantics win and the fork's function becomes a one-line delegation,
so the CLI's two summary tables, its advisory count, `skillspector baseline` and
the MCP verdict answer the same question as the recursive public-record budget
and the transitive count. The name stays: five CLI call sites and the MCP server
read findings through it, and the report node is where a reader looks for what a
report contains.

The five CLI stream tests that go through this reader were re-read against the
adopted semantics before the change and none of them moves. Three assert shapes
the two functions already agreed on -- `filtered_findings` empty, absent with
nothing suppressed, and present beside a suppressed partition -- and two run a
real Scan, where the report node always writes the key. No expectation changed.

Both docstrings asserted that `filtered_findings` is the pre-partition
population. Upstream `73dd1f1` falsified that: the report node writes the kept
side alone. The survivor now says what the key holds, why the subtraction is
retained anyway -- the two keys are one partition wherever they are written
together, and mid-graph and hand-assembled results still carry the wider shape
-- and why the raw-`findings` fallback does not subtract. That stale premise was
also a CLAUDE.md Applied Learning bullet and a test docstring, and
`docs/DEVELOPMENT.md` described the key as a projection of
`effective_finding_ids`, a name the report node never reads; all three are
corrected here.

`TestOneRecursiveScanCountsItsFindingsOnce` pins the agreement across every
shape either reader can be handed, including the four a real Scan cannot
produce, and asserts that the recursive summary's `finding_count` and the
recursive public-record budget select identically on the shape they used to
disagree about.

`test_report_writes_only_the_kept_side_into_filtered_findings` pins the fact
all of that prose rests on. Nothing asserted it before: the Behavior Snapshot
corpus cannot, because `tests/behavior/COVERAGE_LIMITS.md` records that
`filtered_findings` is byte-identical to `findings` in every fixture, no fixture
exercising a Baseline at all. Widening the key back now fails a test instead of
making two docstrings and an Applied Learning bullet quietly wrong.

Refs #130

Claude-Session: https://claude.ai/code/session_01NsjXzHBDr49KjMJPnLyLCJ
Signed-off-by: Rodrigo França <rodrigo_rjsf@hotmail.com>
Three places still asserted that `filtered_findings` is the kept and suppressed
findings together. Upstream `73dd1f1` narrowed what the report node writes to
the kept side alone, and the branch corrected the claim everywhere except here.

The state-key row also stated a rule the tree does not keep. "Read it through
`suppression.effective_findings`, never directly" is contradicted by three
legitimate mid-graph reads, and the row described only what the report node
writes while the transitive merge writes the pre-partition merged population
into the same key beforehand. Qualify both: the row says what the key holds at
each point, and the rule binds the consumer asking which findings a finished
Scan reports rather than the merge path, which reads it before anything has
been partitioned.

Signed-off-by: Rodrigo França <rodrigo_rjsf@hotmail.com>
…second reader

The assertion re-expressed the production selection verbatim, so it was true of
itself: replacing the combined entry's `finding_count` with an inlined read of
`filtered_findings` -- exactly the regression this issue exists to prevent --
left it green. Route the comparison through `_combined_skill_entry` so the call
site is what is asserted, and derive the suppressed record count from the state
instead of hard-coding it.

The neighbouring docstring in the same class asserted the partition premise
upstream `73dd1f1` falsified, two screens below the class docstring that
already corrected it.

Signed-off-by: Rodrigo França <rodrigo_rjsf@hotmail.com>
@rodrigorjsf
rodrigorjsf merged commit e47ec3f into main Aug 22, 2026
5 checks passed
@rodrigorjsf
rodrigorjsf deleted the fix/finding-selection-contract branch August 22, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Three readers of one contract: the transitive finding count over-reports, and two named readers disagree

1 participant