Normalized Output Schema #9

Objective

Standardize evaluation outputs so they are reusable, comparable, and CI-friendly.

Scope

Create a stable JSON results schema and Markdown summary format.

Tasks

• Define results schema including:

  • schema_version
  • run_id
  • timestamp
  • bundle_name
  • git_sha if available
  • per-item metrics
  • aggregated metrics
  • pass_fail indicators

• Implement JSON result writer

• Implement Markdown summary generator

• Add threshold comparison logic

• Ensure deterministic output formatting

Acceptance Criteria

• Running evaluation produces:

  • results.json
  • summary.md

• Schema includes version field

• Exit code is non-zero if thresholds fail

• Aggregated metrics computed correctly

Out of Scope

• Azure Monitor export
• KQL integration
• Dashboard publishing