Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"name": "agentops",
"metadata": {
"description": "AgentOps Toolkit plugin marketplace — evaluation skills for Microsoft Foundry agents",
"version": "1.0.0"
},
"owner": {
"name": "AgentOps Toolkit",
"email": "agentops@microsoft.com"
},
"plugins": [
{
"name": "agentops-toolkit",
"source": "../../plugins/agentops",
"description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Toolkit and Microsoft Foundry agents.",
"version": "0.1.7",
"keywords": [
"agentops",
"evaluation",
"foundry",
"copilot",
"agent-skills",
"ai-evaluation"
],
"license": "MIT",
"repository": "https://github.com/Azure/agentops"
}
]
}
29 changes: 29 additions & 0 deletions .github/plugin/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"name": "agentops",
"metadata": {
"description": "AgentOps Toolkit plugin marketplace — evaluation skills for Microsoft Foundry agents",
"version": "1.0.0"
},
"owner": {
"name": "AgentOps Toolkit",
"email": "agentops@microsoft.com"
},
"plugins": [
{
"name": "agentops-toolkit",
"source": "../../plugins/agentops",
"description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Toolkit and Microsoft Foundry agents.",
"version": "0.1.7",
"keywords": [
"agentops",
"evaluation",
"foundry",
"copilot",
"agent-skills",
"ai-evaluation"
],
"license": "MIT",
"repository": "https://github.com/Azure/agentops"
}
]
}
28 changes: 22 additions & 6 deletions .github/workflows/cut-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -76,12 +76,28 @@ jobs:
echo "CHANGELOG updated with [${{ env.version }}] - $DATE"
fi

- name: Sync VS Code extension version
- name: Sync plugin versions
run: |
jq --arg v "${{ env.version }}" '.version = $v' \
VERSION="${{ env.version }}"

# package.json (VSIX)
jq --arg v "$VERSION" '.version = $v' \
plugins/agentops/package.json > plugins/agentops/package.json.tmp
mv plugins/agentops/package.json.tmp plugins/agentops/package.json
echo "VSIX version set to ${{ env.version }}"
echo "package.json → $VERSION"

# plugin.json (Agent Plugins)
jq --arg v "$VERSION" '.version = $v' \
plugins/agentops/plugin.json > plugins/agentops/plugin.json.tmp
mv plugins/agentops/plugin.json.tmp plugins/agentops/plugin.json
echo "plugin.json → $VERSION"

# marketplace.json (GitHub + Claude Code)
for mp in .github/plugin/marketplace.json .claude-plugin/marketplace.json; do
jq --arg v "$VERSION" '.plugins[0].version = $v' "$mp" > "$mp.tmp"
mv "$mp.tmp" "$mp"
echo "$mp → $VERSION"
done

- name: Configure git
run: |
Expand All @@ -90,7 +106,7 @@ jobs:

- name: Commit and push
run: |
git add CHANGELOG.md plugins/agentops/package.json
git add CHANGELOG.md plugins/agentops/package.json plugins/agentops/plugin.json .github/plugin/marketplace.json .claude-plugin/marketplace.json
git commit -m "chore: prepare release ${{ env.version }}"
git push origin "release/v${{ env.version }}"

Expand All @@ -109,7 +125,7 @@ jobs:
### What happened
- Branch \`release/v${{ env.version }}\` created from \`develop\`
- \`CHANGELOG.md\` updated: versioned section \`[${{ env.version }}]\` added
- \`plugins/agentops/package.json\` version synced to \`${{ env.version }}\`
- Plugin versions synced to \`${{ env.version }}\` (package.json, plugin.json, marketplace.json)
- Staging pipeline triggered automatically (build → TestPyPI + VSIX pre-release → verify)

### Next steps
Expand All @@ -135,7 +151,7 @@ jobs:
echo "" >> "$GITHUB_STEP_SUMMARY"
echo "- Branch: \`release/v${{ env.version }}\`" >> "$GITHUB_STEP_SUMMARY"
echo "- CHANGELOG updated with version **${{ env.version }}**" >> "$GITHUB_STEP_SUMMARY"
echo "- VS Code extension version synced to **${{ env.version }}**" >> "$GITHUB_STEP_SUMMARY"
echo "- Plugin versions synced to **${{ env.version }}** (package.json, plugin.json, marketplace.json)" >> "$GITHUB_STEP_SUMMARY"
echo "- PR opened: \`release/v${{ env.version }}\` → \`main\`" >> "$GITHUB_STEP_SUMMARY"
echo "- Staging pipeline triggered automatically" >> "$GITHUB_STEP_SUMMARY"
echo "" >> "$GITHUB_STEP_SUMMARY"
Expand Down
8 changes: 7 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,5 +4,11 @@
"approve": true,
"matchCommandLine": true
}
}
},
"chat.plugins.extraKnownMarketplaces": [
"Azure/agentops"
],
"chat.plugins.enabledPlugins": [
"agentops-toolkit"
]
}
25 changes: 25 additions & 0 deletions docs/tutorial-copilot-skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,31 @@ cp -r <agentops-repo>/plugins/agentops/skills/* plugins/agentops/skills/

This way the skills travel with the repo and every contributor gets them automatically.

### Option 5: Agent Plugin Marketplace (cross-tool)

The AgentOps plugin is published to the **Agent Plugin Marketplace**, which works
across VS Code Copilot, Copilot CLI, and Claude Code.

**VS Code** — add the marketplace to your workspace or user settings:

```json
{
"chat.plugins.extraKnownMarketplaces": ["Azure/agentops"],
"chat.plugins.enabledPlugins": ["agentops-toolkit"]
}
```

**Claude Code** — register the marketplace from the CLI:

```bash
claude plugin marketplace add Azure/agentops
```

The marketplace is defined in `.github/plugin/marketplace.json` (the canonical
location for VS Code and Copilot CLI) and `.claude-plugin/marketplace.json`
(the Claude Code discovery location). Both point to the same plugin at
`plugins/agentops/`.

## Verifying the installation

Check that the skill directories exist:
Expand Down
1 change: 1 addition & 0 deletions plugins/agentops/.vscodeignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,6 @@ node_modules
!CHANGELOG.md
!skills/**/SKILL.md
!package.json
!plugin.json
!LICENSE
!icon.png
22 changes: 22 additions & 0 deletions plugins/agentops/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,32 @@ Copilot agent skills for running standardized evaluation workflows with

## Installation

### VS Code Extension Marketplace

Install from the
[VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=AgentOpsToolkit.agentops-toolkit)
or search **"AgentOps Skills"** in the VS Code Extensions view.

### Agent Plugin Marketplace

The AgentOps plugin is also available through the cross-tool **Agent Plugin
Marketplace**, which works with VS Code Copilot, Copilot CLI, and Claude Code.

**VS Code** — add this to your `.vscode/settings.json`:

```json
{
"chat.plugins.extraKnownMarketplaces": ["Azure/agentops"],
"chat.plugins.enabledPlugins": ["agentops-toolkit"]
}
```

**Claude Code** — register the marketplace:

```bash
claude plugin marketplace add Azure/agentops
```

## Usage

Open **Copilot Chat** in VS Code and describe what you want to do.
Expand Down
10 changes: 10 additions & 0 deletions plugins/agentops/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"name": "agentops-toolkit",
"description": "Copilot agent skills for running standardized evaluation workflows with AgentOps Toolkit and Microsoft Foundry agents.",
"version": "0.1.7",
"author": {
"name": "AgentOps Toolkit",
"url": "https://github.com/Azure/agentops"
},
"skills": "skills/"
}
27 changes: 22 additions & 5 deletions scripts/cut-release.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -59,17 +59,34 @@ if ($changelog -match [regex]::Escape("## [$version]")) {
Write-Host "CHANGELOG updated with [$version] - $date" -ForegroundColor Green
}

# ── Step 6: Sync VSIX version ──────────────────────────────────────
Write-Host "`n>>> Syncing VSIX version in package.json..." -ForegroundColor Yellow
# ── Step 6: Sync plugin versions ────────────────────────────────────
Write-Host "`n>>> Syncing plugin versions..." -ForegroundColor Yellow

# 6a. package.json (VSIX)
$pkgPath = "plugins/agentops/package.json"
$pkg = Get-Content $pkgPath -Raw | ConvertFrom-Json
$pkg.version = $version
$pkg | ConvertTo-Json -Depth 10 | Set-Content $pkgPath -NoNewline
Write-Host "VSIX version set to $version" -ForegroundColor Green
Write-Host " package.json → $version" -ForegroundColor Green

# 6b. plugin.json (Agent Plugins)
$pluginPath = "plugins/agentops/plugin.json"
$plugin = Get-Content $pluginPath -Raw | ConvertFrom-Json
$plugin.version = $version
$plugin | ConvertTo-Json -Depth 10 | Set-Content $pluginPath -NoNewline
Write-Host " plugin.json → $version" -ForegroundColor Green

# 6c. marketplace.json (GitHub + Claude Code)
foreach ($mpPath in @(".github/plugin/marketplace.json", ".claude-plugin/marketplace.json")) {
$mp = Get-Content $mpPath -Raw | ConvertFrom-Json
$mp.plugins[0].version = $version
$mp | ConvertTo-Json -Depth 10 | Set-Content $mpPath -NoNewline
Write-Host " $mpPath → $version" -ForegroundColor Green
}

# ── Step 7: Configure git, commit and push ──────────────────────────
Write-Host "`n>>> Committing and pushing..." -ForegroundColor Yellow
git add CHANGELOG.md plugins/agentops/package.json
git add CHANGELOG.md plugins/agentops/package.json plugins/agentops/plugin.json .github/plugin/marketplace.json .claude-plugin/marketplace.json
git commit -m "chore: prepare release $version"
git push origin "release/v$version"

Expand All @@ -83,7 +100,7 @@ Automated release branch created from ``develop``.
### What happened
- Branch ``release/v$version`` created from ``develop``
- ``CHANGELOG.md`` updated: versioned section ``[$version]`` added
- ``plugins/agentops/package.json`` version synced to ``$version``
- Plugin versions synced to ``$version`` (package.json, plugin.json, marketplace.json)

### Next steps
1. Run staging locally: ``.\scripts\staging.ps1``
Expand Down
27 changes: 21 additions & 6 deletions scripts/cut-release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
# 3. Verifies release branch doesn't exist
# 4. Creates release/v<version> branch
# 5. Updates CHANGELOG.md with versioned section
# 6. Syncs VSIX version in plugins/agentops/package.json
# 6. Syncs plugin versions (package.json, plugin.json, marketplace.json)
# 7. Commits and pushes the branch
# 8. Creates a PR to main via gh CLI
#
Expand Down Expand Up @@ -55,16 +55,31 @@ else
echo "CHANGELOG updated with [$version] - $date_today"
fi

# ── Step 6: Sync VSIX version ──────────────────────────────────────
echo -e "\n>>> Syncing VSIX version in package.json..."
# ── Step 6: Sync plugin versions ────────────────────────────────────
echo -e "\n>>> Syncing plugin versions..."

# 6a. package.json (VSIX)
pkg_path="plugins/agentops/package.json"
jq --arg v "$version" '.version = $v' "$pkg_path" > "${pkg_path}.tmp"
mv "${pkg_path}.tmp" "$pkg_path"
echo "VSIX version set to $version"
echo " package.json → $version"

# 6b. plugin.json (Agent Plugins)
plugin_path="plugins/agentops/plugin.json"
jq --arg v "$version" '.version = $v' "$plugin_path" > "${plugin_path}.tmp"
mv "${plugin_path}.tmp" "$plugin_path"
echo " plugin.json → $version"

# 6c. marketplace.json (GitHub + Claude Code)
for mp in .github/plugin/marketplace.json .claude-plugin/marketplace.json; do
jq --arg v "$version" '.plugins[0].version = $v' "$mp" > "$mp.tmp"
mv "$mp.tmp" "$mp"
echo " $mp → $version"
done

# ── Step 7: Commit and push ────────────────────────────────────────
echo -e "\n>>> Committing and pushing..."
git add CHANGELOG.md plugins/agentops/package.json
git add CHANGELOG.md plugins/agentops/package.json plugins/agentops/plugin.json .github/plugin/marketplace.json .claude-plugin/marketplace.json
git commit -m "chore: prepare release $version"
git push origin "release/v$version"

Expand All @@ -81,7 +96,7 @@ Automated release branch created from \`develop\`.
### What happened
- Branch \`release/v$version\` created from \`develop\`
- \`CHANGELOG.md\` updated: versioned section \`[$version]\` added
- \`plugins/agentops/package.json\` version synced to \`$version\`
- Plugin versions synced to \`$version\` (package.json, plugin.json, marketplace.json)

### Next steps
1. Run staging locally: \`./scripts/staging.sh\`
Expand Down