fix(cli): count only the transitive findings the report carries - #141
Merged
Conversation
`transitive_finding_count` was a third expression of "which findings does a
finished Scan report", beside `suppression.effective_findings` and
`report.reported_findings`, and the only one that subtracted nothing: it counted
every `filtered_findings` entry carrying a `source_url` straight out of the
report node's return. The merge path now selects through `effective_findings`
first and filters on `source_url` second, so a Baseline that accepts a
transitive Finding lowers the count and the report's list together.
Measured, and it moves the issue's premise: upstream `73dd1f1` ("Security fixes
(NVIDIA#393)", which arrived in the 2.9.6 sync) narrowed what the report node writes
into `filtered_findings` from the pre-partition population to the kept side
alone. The over-count #130 describes -- 2 published against 1 reported -- no
longer reproduces end to end on this tree, and a `--transitive` scan whose
`--baseline` fingerprints a dependency Finding already answers 1 beside a report
listing one. `test_scan_transitive_count_agrees_with_the_report_it_publishes`
pins that as the control.
`test_scan_transitive_count_subtracts_the_baseline_suppressed_partition` pins
the contract instead of the current key's contents: it asserts the count over a
`filtered_findings` holding one kept and one Baseline-suppressed transitive
Finding -- the shape the report node wrote before `73dd1f1` -- where the old
expression answered 2. Restore that expression and it fails `assert 2 == 1`.
Refs #130
Claude-Session: https://claude.ai/code/session_01NsjXzHBDr49KjMJPnLyLCJ
Signed-off-by: Rodrigo França <rodrigo_rjsf@hotmail.com>
`report.reported_findings` and `suppression.effective_findings` were written for one defect on the two sides of a fork that had not synced, and the 2.9.6 sync kept both. They agree on selecting `filtered_findings` by presence rather than truthiness and on subtracting the `suppressed_findings` partition, and diverge on one input: with `filtered_findings` absent or malformed, the fork's subtracted that partition from the raw `findings` list and upstream's deliberately did not, because raw findings are not the population that produced it. Upstream's semantics win and the fork's function becomes a one-line delegation, so the CLI's two summary tables, its advisory count, `skillspector baseline` and the MCP verdict answer the same question as the recursive public-record budget and the transitive count. The name stays: five CLI call sites and the MCP server read findings through it, and the report node is where a reader looks for what a report contains. The five CLI stream tests that go through this reader were re-read against the adopted semantics before the change and none of them moves. Three assert shapes the two functions already agreed on -- `filtered_findings` empty, absent with nothing suppressed, and present beside a suppressed partition -- and two run a real Scan, where the report node always writes the key. No expectation changed. Both docstrings asserted that `filtered_findings` is the pre-partition population. Upstream `73dd1f1` falsified that: the report node writes the kept side alone. The survivor now says what the key holds, why the subtraction is retained anyway -- the two keys are one partition wherever they are written together, and mid-graph and hand-assembled results still carry the wider shape -- and why the raw-`findings` fallback does not subtract. That stale premise was also a CLAUDE.md Applied Learning bullet and a test docstring, and `docs/DEVELOPMENT.md` described the key as a projection of `effective_finding_ids`, a name the report node never reads; all three are corrected here. `TestOneRecursiveScanCountsItsFindingsOnce` pins the agreement across every shape either reader can be handed, including the four a real Scan cannot produce, and asserts that the recursive summary's `finding_count` and the recursive public-record budget select identically on the shape they used to disagree about. `test_report_writes_only_the_kept_side_into_filtered_findings` pins the fact all of that prose rests on. Nothing asserted it before: the Behavior Snapshot corpus cannot, because `tests/behavior/COVERAGE_LIMITS.md` records that `filtered_findings` is byte-identical to `findings` in every fixture, no fixture exercising a Baseline at all. Widening the key back now fails a test instead of making two docstrings and an Applied Learning bullet quietly wrong. Refs #130 Claude-Session: https://claude.ai/code/session_01NsjXzHBDr49KjMJPnLyLCJ Signed-off-by: Rodrigo França <rodrigo_rjsf@hotmail.com>
Three places still asserted that `filtered_findings` is the kept and suppressed findings together. Upstream `73dd1f1` narrowed what the report node writes to the kept side alone, and the branch corrected the claim everywhere except here. The state-key row also stated a rule the tree does not keep. "Read it through `suppression.effective_findings`, never directly" is contradicted by three legitimate mid-graph reads, and the row described only what the report node writes while the transitive merge writes the pre-partition merged population into the same key beforehand. Qualify both: the row says what the key holds at each point, and the rule binds the consumer asking which findings a finished Scan reports rather than the merge path, which reads it before anything has been partitioned. Signed-off-by: Rodrigo França <rodrigo_rjsf@hotmail.com>
…second reader The assertion re-expressed the production selection verbatim, so it was true of itself: replacing the combined entry's `finding_count` with an inlined read of `filtered_findings` -- exactly the regression this issue exists to prevent -- left it green. Route the comparison through `_combined_skill_entry` so the call site is what is asserted, and derive the suppressed record count from the state instead of hard-coding it. The neighbouring docstring in the same class asserted the partition premise upstream `73dd1f1` falsified, two screens below the class docstring that already corrected it. Signed-off-by: Rodrigo França <rodrigo_rjsf@hotmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #130.
Read this first: the issue's premise does not reproduce on this tree
#130 frames the over-count as a live, user-visible defect — "a
--transitiverun with a--baselinethat accepts a transitive finding publishes 2 and reports 1", reachable from an ordinaryinvocation. That measurement was true when the issue was written and is not true now. Upstream
73dd1f1("Security fixes", NVIDIA#393) narrowed what the report node writes intofiltered_findingstothe kept side alone, so the state the transitive count reads no longer carries the suppressed
findings it was over-counting.
The fix still lands, and the contract test still earns its place — it induces the pre-
73dd1f1shape deliberately, through a spy, because that shape is exactly what the transitive merge path
hands over today even though the report node no longer produces it. What changes is the framing: this
is a latent contract defect held shut by an upstream change, not a bug a user hits this afternoon. A
reviewer should know that before reading the diff, and #130 gets a comment saying so.
What lands
The over-count, first, as the brief sequenced it.
5e149c4makes the transitive finding countcount only the findings the report carries. A failing test came first, as the brief required.
The redundancy, second and separately.
1b89e97leaves exactly one function in the packagedeciding which findings a finished Scan reports. #130 measured the fork's
reported_findingsandupstream's
effective_findingsdiverging on the raw-findingsfallback — the fork's subtractedsuppressed_findings, upstream's deliberately did not. Upstream's semantics won, per this fork'sstanding rule that upstream owns the truth about the core, so the fork's function became a
delegation and the reasoning it carried moved to the function that answers.
Review, and what the critics changed
Three independent critics graded this branch — acceptance-criteria conformance, scope and contract,
adversarial correctness. Zero blocking findings. Three of their non-blocking findings were promoted
by the main thread and closed here, because each is a claim that is not true:
The AC-4 assertion was true of itself.
test_the_budget_and_the_summary_count_one_result_identicallycompared
_multi_skill_public_record_count(result) - 1againstlen(reported_findings(result))—and that right-hand side is character-for-character the production expression at the call site the
acceptance criterion names. A critic proved it: inlining a raw
filtered_findingsread at_combined_skill_entry'sfinding_count— precisely the regression this issue exists to prevent —left every case green.
e9f2b25routes the comparison through_combined_skill_entryinstead andderives the suppressed record count from the state rather than hard-coding
1. Re-proved by mutatingthat one site: now red, restored by hand,
git diff --exit-codeclean.Three docstrings still asserted the falsified premise.
1b89e97's body claims the stale claimlived in three places and "all three are corrected here". Four survived.
199f555closes them.A contract document stated a rule the tree does not keep. The new
filtered_findingsrow indocs/DEVELOPMENT.mdended "Read it throughsuppression.effective_findings, never directly".Three legitimate mid-graph reads contradict the absolute, and the row described only what the report
node writes while the transitive merge writes the pre-partition merged population into the same
key beforehand. Now qualified at both ends: what the key holds at each point, and a rule that binds
the consumer asking which findings does a finished Scan report rather than the merge path, which
reads it before anything has been partitioned.
Deliberate deferral, filed rather than buried
reported_findingsis now a one-line delegation, so one selection has two names and six call sites.Removing it is a mechanical change with its own review surface and was deliberately sequenced after
the defect fix. Filed as #140 rather than left in a commit body, per
CLAUDE.md's standing rule.#140 also records the argument for keeping the name, which is not obviously wrong.
Verification
Run in the main thread on this branch, at
e9f2b25:Zero snapshots moved. This change is entirely in the CLI's and the report node's reader
functions; the Behavior Snapshot projection scans through the graph and its recorded state is
untouched.