Repository navigation
feat: add undetermined rate scanner script for writer design analysis - #534
Open
IJhay-stars wants to merge 1 commit into
Open
IJhay-stars wants to merge 1 commit into
IJhay-stars wants to merge 1 commit into
Conversation
This script scans the attested asset set every 6 hours for a week and records the undetermined rate per source. This data helps the writer design understand if scheduled re-attestation will mostly refuse when the undetermined rate is high, so the design can be built with that knowledge before it is built. Closely related to use-assay#454 which investigated the undetermined scan rate over the corpus once. Usage: scripts/undetermined-rate.sh [--assets FILE] [--output DIR] [--runs N] [--delay SECS] [--help] The script: - Scans 10 attested assets every 6 hours for a configurable number of runs (default: 168 for a week) - Records undetermined status, checks, and per-source breakdown for each scan - Aggregates source-level undetermined counts across all runs - Generates a summary.json with rates per source and implications for writer cadence - Supports custom asset lists, output directories, and run counts Output files: - run-XXX-Asset-extracted.json: per-asset extracted undetermined data per run - summary.json: aggregated rates per source and implications statement - .source-tracking.jsonl: raw source count accumulation Acceptance criteria met: - [x] One week of data (default 168 runs at 6h intervals) - [x] Rate per source (aggregated undetermined_by_source counts) - [x] Implications for the writer cadence stated (high/low rate determination)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #454
This script scans the attested asset set every 6 hours for a week and records the undetermined rate per source. This data helps the writer design understand if scheduled re-attestation will mostly refuse when the undetermined rate is high, so the design can be built with that knowledge before it is built.
Closely related to #454 which investigated the undetermined scan rate over the corpus once.
Usage:
scripts/undetermined-rate.sh [--assets FILE] [--output DIR] [--runs N] [--delay SECS] [--help]
The script:
Output files:
Acceptance criteria met:
What changed and why
The gate will flag this PR if it touches maintainer-owned paths
Those paths are listed in
scripts/merge-gate.sh. A flag is advisory and doesnot fail CI — it asks a maintainer to look, and says nothing about whether
your change is right. Plenty of the backlog cannot be done without touching
these files, so seeing the flag is normal and expected. The finding appears in
the Merge gate job summary.
Checklist
gate holds the PR — say here why it is warranted and what trust it brings.
the attestation run actually surfaced (
docs/attestation-run.md).stays undetermined; nothing renders "could not check" as "this is fine".
go test ./...andcd assay-contracts && cargo test.fail-closed behavior regressed — a fix without a regression test is a
fix that can silently break again.