Skip to content

fix(output): improve violation comprehension in CI/CD mode - #21

Open
B1naryVision wants to merge 1 commit into
certinia:mainfrom
B1naryVision:improve-violation-comprehension-for-ci-cd
Open

fix(output): improve violation comprehension in CI/CD mode#21
B1naryVision wants to merge 1 commit into
certinia:mainfrom
B1naryVision:improve-violation-comprehension-for-ci-cd

Conversation

@B1naryVision

Copy link
Copy Markdown
Contributor

Problem

When running with the CI/CD flag (-j), a single rule breaching its
cicdmaxissues threshold caused ASIST to print findings for all rules.
With e.g. 4 rules at 10 issues each and a cap of 10, introducing 2 new
violations printed ~40 findings, making the 2 new ones very hard to spot.

Additionally, findings JSON was written to stdout while the threshold
summary went to stderr, so output ordering was unpredictable across
platforms and CI log viewers.

Changes

  • CI/CD filtering — when thresholds are breached, the findings JSON now
    contains only the rules that exceeded their cicdmaxissues limit
    (Count reflects the filtered total). Exit code behavior is unchanged.
  • Single stream, fixed layout — findings and the threshold summary are
    both written sequentially to stdout: issue details on top, summary at the
    very bottom.
  • Explicit pass message — CI/CD runs with no breaches now end with
    No rules exceeded their cicdmaxissues threshold. instead of silence.
  • Threshold logic split into countFindingsPerRule, getViolatedRuleIds
    and filterFindingsByRules helpers; CheckThresholdViolations keeps its
    existing signature and output format.

No regression without the flag

Runs without -j are byte-for-byte unchanged: all findings printed, no
threshold text, exit 0.

Testing

  • New unit tests for breached-rule selection/sorting, filtering + count
    update, and the empty-rule-list edge case; full go test ./... passes.
  • Verified end-to-end against integrationtests/src:
    • -j + breach → only breached rule's findings, summary at bottom,
      stderr empty, exit 1
    • -j + no breach → all findings + pass message, exit 0
    • no flag → unchanged output, exit 0

…utput streams

When the CI/CD flag was active and any rule exceeded its cicdmaxissues
threshold, the tool printed findings for every rule, burying the new
violations. The findings JSON also went to stdout while the threshold
summary went to stderr, so the two could interleave unpredictably
across platforms.

- In CI/CD mode, print findings only for rules that breached their
  cicdmaxissues threshold, with Count updated to match
- Write findings and threshold summary sequentially to stdout: issue
  details first, summary at the bottom
- Print an explicit "No rules exceeded their cicdmaxissues threshold."
  summary when CI/CD mode passes
- Extract countFindingsPerRule, getViolatedRuleIds and
  filterFindingsByRules helpers with unit tests

Behavior without the CI/CD flag is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant