Skip to content

feat(cli): add JSON task validation output - #253

Open
steven-ji wants to merge 1 commit into
Human-Agent-Society:devfrom
steven-ji:feat/validate-json-output
Open

feat(cli): add JSON task validation output#253
steven-ji wants to merge 1 commit into
Human-Agent-Society:devfrom
steven-ji:feat/validate-json-output

Conversation

@steven-ji

Copy link
Copy Markdown
Contributor

Motivation

Issue #242 proposes frontend-independent task validation that CLI, TUI, and
future web clients can share. PRs #249 and #250 added structured diagnostics
and a reusable validation runner, but scripts still have to parse the
human-oriented coral validate output.

This PR exposes the existing structured result through an additive --json
flag so tools and future frontends can consume validation results directly.

Refs #242.

Changes

  • Add coral validate <path> --json and document it in CLI help.
  • Emit one serialized ValidationRunResult containing diagnostics, progress
    events, the baseline score, and any structured failure.
  • Suppress human-oriented progress rendering in JSON mode while preserving
    the existing text output when the flag is absent.
  • Preserve exit status 0 for successful validation and 1 for validation
    failures in both output modes.
  • Add regression coverage for CLI help, successful JSON output, failed JSON
    output, and legacy text-output compatibility.
  • Update the CLI reference and developer command examples.

Migration: this is an additive optional flag. Existing coral validate
commands and their text output are unchanged, so no migration is required.

This PR does not add a TUI framework, change the validation result schema, or
modify the validation runner.

Authored with assistance from Codex. The submitting human reviewed the changed
files and confirmed the behavior before publication.

Test plan

  • uv run pytest tests/test_validation.py -v — 26 passed.
  • uv run pytest tests/ -v — 715 passed, 1 skipped.
  • uv run ruff check . — passed.
  • uv run ruff format --check . — 139 files already formatted.
  • uv run mypy --follow-imports=skip coral/cli/author.py — passed.
  • git diff --check upstream/dev...HEAD — passed.

Affected areas

  • coral/agent/ (runtime, manager, heartbeat, warmstart)
  • coral/grader/ (daemon, TaskGrader, subprocess grader, loader)
  • coral/hub/ (attempts, notes, skills, checkpoint)
  • coral/workspace/ (project setup, worktrees, grader env)
  • coral/cli/ (commands, helpers)
  • coral/hooks/ (post_commit / submit_eval)
  • coral/template/ (CORAL.md, bundled skills/agents)
  • coral/gateway/ (LiteLLM gateway)
  • coral/web/ (dashboard)
  • examples/ (new or modified task)
  • docs/ (docs site)
  • CI / tooling / packaging
  • Other:

Checklist

  • PR targets the dev branch (not main).
  • Title follows Conventional Commits (feat:, fix:, docs:, refactor:, ...).
  • uv run pytest tests/ -v passes locally.
  • uv run ruff check . and uv run ruff format --check . pass.
  • Added or updated tests under tests/ for any behavior change.
  • Updated docs (docs/content/, README, or relevant skill under .claude/skills/) for any user-visible or contract change.
  • If this changes a config field, CLI flag, hook, or runtime contract — noted the migration path in the PR description.
  • New examples/<task>/: coral validate <task> succeeds and a smoke run produces at least one finalized score. No hidden answer keys committed under seed/. (Not applicable; no new example.)
  • AI-assisted PR: a human author has read every changed line and can defend the design. See AGENTS.md.

@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

@steven-ji is attempting to deploy a commit to the Compounding Intelligence Team on Vercel.

A member of the Team first needs to authorize it.

@steven-ji
steven-ji marked this pull request as ready for review August 25, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant