Test: Validate skills-assisted pipeline creation #83

Objective

Validate that AgentOps skills help users create correct evaluation configs and CI/CD workflows with minimal manual guesswork.

Scope

Cover both agentops skills install and agentops workflow generate. The docs describe eight AgentOps skills, including config, dataset, eval, report, regression, and workflow support, and the GitHub Actions guide says workflow generation always creates a PR pipeline and may also create CI/CD pipelines when multiple run configs or bundles exist.

Tasks

• Run agentops skills install in a real test repo
• Use Copilot with the installed skills to create or refine run.yaml
• Verify the skills steer users toward implemented commands only
• Run agentops workflow generate
• Inspect generated workflows under .github/workflows/
• Validate required vars/secrets and branch triggers
• Open a sample PR to verify the generated workflow behavior
• Log usability and docs issues

Acceptance Criteria

• Skills install cleanly and are discoverable by the coding agent
• Skill-guided setup produces usable config/workflow output
• Generated workflow files are coherent and align with repo branching strategy
• The team can identify where skill instructions need refinement
• Clear follow-up backlog items are created

Out of Scope

• Full deployment automation
• Supporting every coding agent platform in this sprint
• Reworking the skills architecture