Skip to content

Speed scores passing tasks only, CI, MCP version - #5

Merged
drewmattie-code merged 1 commit into
mainfrom
fix/speed-ci-mcp-version
Sep 24, 2026
Merged

drewmattie-code merged 1 commit into
mainfrom
fix/speed-ci-mcp-version

Conversation

@drewmattie-code

Copy link
Copy Markdown
Owner
  • Speed: only passing tasks count toward throughput, so a model that answers instantly but wrong no longer gets Speed 100 (reproduced with a stub server that failed all 34 tasks). New tests fail on the old code and pass now.
  • CI: new workflow checks cli (typecheck + tests), mcp (typecheck + build), backend (typecheck), web (lint + tsc) on every push and PR.
  • MCP: reports its package version (was hardcoded 0.1.0).
  • web lint: ignore .open-next / .wrangler build output (429 local-only errors).
  • Versions: cli 0.4.2, mcp 0.1.2 (npm publish follows the merge). CHANGELOG updated.

- cli 0.4.2: throughput speed is the median tokens/sec across passing tasks, so fast
  wrong answers no longer earn Speed 100. Tests cover the all-fail and mixed cases.
- mcp 0.1.2: serverInfo.version reads package.json instead of a hardcoded 0.1.0.
- CI: typecheck + tests for cli, typecheck/build for mcp and backend, lint + tsc for web.
- web: eslint ignores the OpenNext/Wrangler build output.
@drewmattie-code
drewmattie-code merged commit 95e44e1 into main Sep 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant