Skip to content

feat(task): add structured validation reports - #249

Merged
BobbyZhouZijian merged 1 commit into
devfrom
feat/structured-task-validation
Aug 21, 2026
Merged

feat(task): add structured validation reports#249
BobbyZhouZijian merged 1 commit into
devfrom
feat/structured-task-validation

Conversation

@BobbyZhouZijian

Copy link
Copy Markdown
Collaborator

Motivation

Issue #242 proposes frontend-independent task inspection and validation that can be shared by the CLI, TUI, and future web clients. This PR implements the smallest backward-compatible part of that roadmap: structured diagnostics for the existing static task checks.

Refs #242.

Changes

  • Add coral.task.validation with immutable ValidationDiagnostic and ValidationReport types, stable diagnostic codes, severity, path context, and dictionary serialization.
  • Move the existing static task checks into the shared task layer without changing their messages or ordering.
  • Keep coral.cli.validation.validate_task() as a compatibility adapter returning the existing list[str] shape, preserving coral validate and coral start behavior.
  • Add regression coverage for serialization, warnings, private-path diagnostics, and the legacy adapter.
  • Document the new coral/task/ package in CLAUDE.md. AGENTS.md is unchanged because no contribution rule or agent-specific guidance changed.

Authored with assistance from Codex. The submitting human should read every changed line and be able to defend the design before marking this PR ready for review.

Test plan

  • .venv/bin/python -m pytest tests/ -v — 697 passed, 1 skipped.
  • .venv/bin/ruff check . — passed.
  • .venv/bin/ruff format --check . — 139 files already formatted.
  • .venv/bin/mypy --follow-imports=skip coral/task/validation.py coral/cli/validation.py — passed.

Affected areas

  • coral/agent/ (runtime, manager, heartbeat, warmstart)
  • coral/grader/ (daemon, TaskGrader, subprocess grader, loader)
  • coral/hub/ (attempts, notes, skills, checkpoint)
  • coral/workspace/ (project setup, worktrees, grader env)
  • coral/cli/ (commands, helpers)
  • coral/hooks/ (post_commit / submit_eval)
  • coral/template/ (CORAL.md, bundled skills/agents)
  • coral/gateway/ (LiteLLM gateway)
  • coral/web/ (dashboard)
  • examples/ (new or modified task)
  • docs/ (docs site)
  • CI / tooling / packaging
  • Other: shared task validation API and developer architecture guidance

Checklist

  • PR targets the dev branch (not main).
  • Title follows Conventional Commits (feat:, fix:, docs:, refactor:, ...).
  • uv run pytest tests/ -v passes locally.
  • uv run ruff check . and uv run ruff format --check . pass.
  • Added or updated tests under tests/ for any behavior change.
  • Updated docs (docs/content/, README, or relevant skill under .claude/skills/) for any user-visible or contract change.
  • If this changes a config field, CLI flag, hook, or runtime contract — noted the migration path in the PR description.
  • New examples/<task>/: coral validate <task> succeeds and a smoke run produces at least one finalized score. No hidden answer keys committed under seed/.
  • AI-assisted PR: a human author has read every changed line and can defend the design. See AGENTS.md.

The migration-path and new-example checklist items are not applicable. The AI-assisted human-review item is intentionally left unchecked while this PR remains a draft.

@vercel

vercel Bot commented Aug 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
coral-docs Ready Ready Preview Aug 21, 2026 12:47pm
coral-web Ready Ready Preview Aug 21, 2026 12:47pm

Request Review

@BobbyZhouZijian
BobbyZhouZijian marked this pull request as ready for review August 21, 2026 12:50
@BobbyZhouZijian
BobbyZhouZijian merged commit ddf55b9 into dev Aug 21, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant