Skip to content

Add aviator evidence submit and an evidence-source flag for externally verified sessions - #54

Draft
simsinght wants to merge 1 commit into
masterfrom
external-evidence-submit
Draft

simsinght wants to merge 1 commit into
masterfrom
external-evidence-submit

Conversation

@simsinght

@simsinght simsinght commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Review: https://app.aviator.co/r/1837

For sessions where the customer runs the change themselves and Aviator only judges (mergeit: external evidence sessions).

  • aviator verify --evidence-source aviator|external at submission and aviator edit --evidence-source on an existing session; the value is echoed in the output and --json.
  • aviator evidence submit r/N <manifest>: reads a YAML or JSON manifest of the scenarios that were run (summary, ordered steps, evidence files per step with a type and a label, optional pinned criteria), resolves each file's size and content type from its extension, posts the manifest, PUTs every file to the signed URL the server returns with the declared content type and no bearer token, then confirms the uploads. The commit is required: the full 40-character SHA from --commit or the manifest's commit, taken as given (the CLI never runs git). Exits non-zero and names the files when an upload fails or the server reports one missing.
  • aviator scenarios shows "submitted by for " instead of a tool call count for a submitted scenario run; --json carries source, submitted_by and commit_sha.

Manifest validation (missing file, unsupported extension, missing label, malformed criterion handle, empty manifest) happens before any API call.

@aviator-app

aviator-app Bot commented Oct 29, 2026

Copy link
Copy Markdown

Current Aviator status

Aviator will automatically update this comment as the status of the PR changes.
Comment /aviator refresh to force Aviator to re-examine your PR (or learn about other /aviator commands).

This pull request is currently open (not queued).

How to merge

To merge this PR, comment /aviator merge or add the mergequeue label.


See the real-time status of this PR on the Aviator webapp.
Use the Aviator Chrome Extension to see the status of your PR within GitHub.

@aviator-app

aviator-app Bot commented Oct 29, 2026

Copy link
Copy Markdown

Connected this PR to r/1837.

@simsinght
simsinght force-pushed the external-evidence-submit branch 4 times, most recently from 9b42a51 to c063621 Compare October 29, 2026 17:03
@simsinght
simsinght force-pushed the external-evidence-submit branch from c063621 to 678e4ad Compare October 29, 2026 20:51

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant