Agent skill benchmarking platform with CLI and public leaderboards.
Website: skillmark.sh
┌─────────────────────────────────────────────────────────────────────┐
│ SKILLMARK │
├─────────────────────────────────────────────────────────────────────┤
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
│ │ CLI (npm) │────▶│ Claude │────▶│ Results JSON │ │
│ │ skillmark │ │ Engine │ │ + Markdown │ │
│ └──────────────┘ └──────────────┘ └────────┬─────────┘ │
│ │ │ │
│ │ skill source │ upload │
│ ▼ ▼ │
│ ┌──────────────┐ ┌──────────────────┐ │
│ │ Skill │ │ Cloudflare │ │
│ │ Sources │ │ Workers + D1 │ │
│ │ - local │ │ (API + Web) │ │
│ │ - git │ └────────┬─────────┘ │
│ │ - skill.sh │ │ │
│ └──────────────┘ ▼ │
│ ┌──────────────────┐ │
│ │ Public │ │
│ │ Leaderboard │ │
│ └──────────────────┘ │
└─────────────────────────────────────────────────────────────────────┘
npm install -g skillmark
# or
npx skillmark# Local skill
skillmark run ~/.claude/skills/my-skill
# Git repository
skillmark run https://github.com/user/skill-repo
# skill.sh reference
skillmark run skill.sh/user/skill-name
# With options
skillmark run ./my-skill \
--tests ./tests \
--model opus \
--runs 5 \
--output ./resultsskillmark publish ./skillmark-results/result.json --api-key <your-key># Show all skills
skillmark leaderboard
# Show specific skill
skillmark leaderboard my-skill-nameCreate markdown files with YAML frontmatter in a tests/ directory:
---
name: multi-agent-reasoning
type: knowledge
concepts:
- orchestrator
- consensus
- context isolation
timeout: 120
---
# Prompt
How do you design multi-agent systems with context isolation?
# Expected
The response should cover:
- [ ] Orchestrator pattern for coordination
- [ ] Consensus mechanisms for decisions
- [ ] Cost implications (~15x token multiplier)
- [ ] Context isolation strategies| Type | Description |
|---|---|
knowledge |
Q&A style tests checking concept coverage |
task |
Execution tests verifying tool usage and outcomes |
| Metric | Description |
|---|---|
| accuracy | Percentage of expected concepts matched |
| tokens_total | Total tokens consumed |
| duration_ms | Wall-clock time |
| tool_count | Number of tool calls |
| cost_usd | Estimated API cost |
Results are saved to the output directory:
skillmark-results/
├── result.json # Machine-readable metrics
└── report.md # Human-readable report
Submit benchmark results to the leaderboard.
curl -X POST https://skillmark.workers.dev/api/results \
-H "Authorization: Bearer <api-key>" \
-H "Content-Type: application/json" \
-d @result.jsonGet skill rankings.
curl https://skillmark.workers.dev/api/leaderboardGet specific skill details and history.
curl https://skillmark.workers.dev/api/skill/my-skill# Install dependencies
pnpm install
# Build CLI
pnpm --filter skillmark build
# Run webapp locally
pnpm --filter @skillmark/webapp dev
# Deploy webapp
pnpm --filter @skillmark/webapp deployskillmark/
├── packages/
│ ├── cli/ # Node.js CLI
│ │ └── src/
│ │ ├── cli-entry-point.ts
│ │ ├── commands/
│ │ ├── engine/
│ │ ├── sources/
│ │ └── types/
│ └── webapp/ # Cloudflare Worker
│ └── src/
│ ├── worker-entry-point.ts
│ ├── routes/
│ └── db/
├── examples/
│ └── tests/ # Example test suites
├── package.json
├── pnpm-workspace.yaml
└── README.md
Complete documentation is available in the docs/ directory:
- Project Overview & PDR - Goals, requirements, and success criteria
- Codebase Summary - Directory structure and key modules
- Code Standards - TypeScript conventions and best practices
- System Architecture - Architecture diagrams and data flow
- Project Roadmap - Phases, milestones, and timeline
- Deployment Guide - CLI npm publishing and Cloudflare deployment
Contributions welcome! Please:
- Read Code Standards
- Follow the Development Rules
- Ensure tests pass:
pnpm test - Run linting:
pnpm lint
MIT