Repository navigation
docs: AgentOps lifecycle diagram and quick-reference card #98
Copy link
Copy link
Closed
Description
Summary
Create a visual lifecycle diagram (Mermaid) and a quick-reference card that developers can use as a cheat sheet for AgentOps Toolkit.
Motivation
AgentOps has many moving parts: commands, config files, output files, environment variables, exit codes, and evaluation scenarios. A single visual that shows the full lifecycle helps developers orient themselves. A reference card gives them a 1-page printable summary they can keep next to their keyboard.
Proposed Content
1. Lifecycle Diagram (Mermaid)
Full evaluation lifecycle as a flowchart:
Create Agent/Model
|
v
agentops init --> .agentops/ workspace
|
v
Configure (run.yaml + bundle + dataset)
|
v
agentops eval run --> results.json + report.md
| |
v v
agentops eval compare agentops report generate
|
v
agentops workflow generate --> CI/CD pipelines
|
v
PR Gate (exit code 0/2) --> Merge or Block
|
v
Tracing (AGENTOPS_OTLP_ENDPOINT) --> AI Toolkit / Jaeger / Azure Monitor
|
v
Monitor + Iterate
Should be rendered as a proper Mermaid diagram with decision nodes, color coding by phase (setup, eval, CI/CD, observability).
2. Quick Reference Card
Single-page summary:
Commands:
| Command | Purpose |
|---|---|
agentops init |
Create workspace |
agentops eval run |
Run evaluation |
agentops eval compare |
Compare two runs |
agentops report generate |
Regenerate report |
agentops workflow generate |
Generate CI/CD workflows |
agentops skills install |
Install Copilot skills |
Exit Codes:
| Code | Meaning |
|---|---|
0 |
Passed — all thresholds met |
1 |
Error — config or runtime failure |
2 |
Failed — thresholds not met |
Key Files:
| File | Purpose |
|---|---|
.agentops/run.yaml |
Run configuration |
.agentops/bundles/*.yaml |
Evaluator bundles |
.agentops/datasets/*.yaml |
Dataset configs |
.agentops/data/*.jsonl |
Dataset rows |
results.json |
Machine-readable output |
report.md |
Human-readable output |
Environment Variables:
| Variable | Purpose |
|---|---|
AZURE_AI_FOUNDRY_PROJECT_ENDPOINT |
Foundry project URL |
AGENTOPS_OTLP_ENDPOINT |
OTLP trace export (e.g. http://localhost:4318) |
AGENTOPS_FOUNDRY_MODE |
cloud or local |
Acceptance Criteria
- Mermaid lifecycle diagram covers setup through monitoring
- Quick reference card fits on one printed page
- Both artifacts can be embedded in README or docs
- Diagram uses color coding or grouping by phase
Context
- The lifecycle diagram can be the opening visual in the E2E adoption guide (docs: end-to-end adoption guide (workshop-ready, from zero to production) #95)
- The reference card can be included in the README or as a standalone
docs/reference-card.md - Related: docs: end-to-end adoption guide (workshop-ready, from zero to production) #95 (E2E guide), docs: use case playbooks (persona-driven evaluation guides) #96 (playbooks), docs: operational runbook (day-2 quality operations) #97 (runbook)
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity
placerda commentedon May 8, 2026
Closing this for now because adoption readiness issues/items will be simplified and tracked differently.