Sprint 4 - End-to-End Scenario Validation #15

Goal

Expand reusable evaluation bundles and validate all evaluation scenarios end-to-end.

Scope

• Conversational agent bundle
• Tool-heavy workflow bundle
• Multi-turn evaluation pattern
• Custom evaluator extension example
• Dataset expansion
• End-to-end validation of all evaluation scenarios

Deliverables

• 3-4 production-ready bundle templates
• Extended example datasets
• Bundle selection documentation
• Validated evaluation scenarios (Foundry agent, multi-agent, model, RAG, HTTP, skills)

Sub-issues

Evaluation Scenario Testing

• Test: Validate single Foundry agent evaluation #78 — Test: Validate single Foundry agent evaluation
• Test: Validate Agent Framework multi-agent implementation #79 — Test: Validate Agent Framework multi-agent implementation
• Test: Validate model evaluation and comparison #80 — Test: Validate model evaluation and comparison
• Test: Validate RAG agent evaluation #81 — Test: Validate RAG agent evaluation
• Test: Validate HTTP runtime targets #82 — Test: Validate HTTP runtime targets
• Test: Validate skills-assisted pipeline creation #83 — Test: Validate skills-assisted pipeline creation

Bundles & Datasets

• Continuous Automated Evaluation Using Golden Datasets #19 — Continuous Automated Evaluation Using Golden Datasets

Definition of Done

• Bundles reusable across projects
• Extension mechanism documented
• Evaluation configuration modularized
• All 6 evaluation scenarios validated end-to-end