Skip to content

[Product Gap] Make Clean Scan an evidence-qualified assurance state #927

Description

@seonghobae

Buyer-visible problem

The dashboard currently distinguishes No findings loaded from Clean scan, but the clean state still communicates only a zero finding count. A buyer, operator, or auditor cannot tell whether:

  • all configured built-in detectors completed;
  • optional external engines were present, unavailable, failed, or skipped;
  • the findings artifact matches the expected repository and exact commit;
  • the scan covered all intended paths/languages;
  • the result is current or stale;
  • exclusions or non-blocking contexts affected the gate;
  • the artifact was schema-valid and provenance-verified.

A celebratory icon does not close this gap and can overstate assurance. 0 findings is meaningful only within a completed, explicitly scoped detector/evidence contract.

Product requirement

Create an evidence-qualified scan outcome model and dashboard state with at least:

  • scan_outcome_code: clean, findings_present, incomplete, failed, untrusted;
  • exact repository, commit SHA, generated time, scanner/AppGuardrail version, schema version;
  • configured and completed detector families;
  • external-engine states: completed, unavailable, failed, not requested;
  • scanned file/path/language profile and explicit exclusions;
  • blocker/non-blocker counts and gate threshold;
  • provenance/digest verification status;
  • freshness/staleness indication;
  • accessible exact-value table and machine-readable JSON/SARIF/report parity.

The dashboard may use restrained positive styling only when the artifact proves clean. Incomplete, failed, unknown, or untrusted evidence must never render as clean.

Test-first acceptance

  1. Add failing fixtures for clean, incomplete, external-tool-unavailable, failed, stale, wrong-commit, malformed, and untrusted artifacts.
  2. Verify only the fully completed/trusted fixture renders the clean assurance state.
  3. Verify screen-reader text includes scope and completion, not only 0 findings.
  4. Verify no emoji or visual style is the sole status distinction.
  5. Verify normalized JSON, dashboard, reports, and deploy-gate semantics agree.
  6. Preserve production statement/branch coverage and public docstrings at 100%.

Architecture/documentation

Update PRD, TRD, Architecture, UML/state model, evidence/data model, threat model, test strategy, operability, traceability, and CHANGELOG. Distinguish detector completion from finding count and document that tool failure/unavailability cannot become a clean result.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: apiAPI, protocol, event, or external contractarea: ci-cdCI, GitHub Actions, checks, release, or supply chainarea: dataDatabase, schema, migration, ETL, or lineagearea: ui-uxFrontend, interaction, design, or user experiencepriority: mediumNormal-priority or P2 workscope: product-gapCustomer-visible product gapstatus: blockedBlocked by conflict, dependency, or required prerequisitetype: featureNew or expanded product capability

    Type

    No type

    Projects

    • Status
      In progress

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions