Skip to content

docs(skills): stop hardcoding MCP tool schemas in the optimize skill#451

Merged
fargito merged 1 commit into
mainfrom
chore/trim-optimize-skill-mcp-schema
Jul 10, 2026
Merged

docs(skills): stop hardcoding MCP tool schemas in the optimize skill#451
fargito merged 1 commit into
mainfrom
chore/trim-optimize-skill-mcp-schema

Conversation

@fargito

@fargito fargito commented Jul 10, 2026

Copy link
Copy Markdown
Member

Trim the codspeed-optimize skill so it no longer hand-lists the CodSpeed MCP tools and their parameters, leaving strategy and guidance and letting the live tool schemas carry each tool's name, parameters, and behavior.

The skill duplicated the MCP schema in two places (the "Analyze with flamegraphs" step and a "MCP tools reference" section), and it had silently rotted: it advertised benchmark_name (renamed to benchmark_uri), a depth_limit parameter that no longer exists on query_flamegraph, and a top-level root_function_name (now nested under filters). A skill that hardcodes stale arguments teaches the agent to call tools with parameters that get validated away.

Follow-up to the MCP param rename where @adriencaccia flagged this skill as duplicating the MCP schema by hand.

Refs COD-2567

@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown

Greptile Summary

This PR trims skills/codspeed-optimize/SKILL.md to stop duplicating the CodSpeed MCP tool schemas inline, removing parameter lists that had silently rotted (stale benchmark_name, non-existent depth_limit, and top-level root_function_name that is now nested under filters). The skill now directs agents to use the live tool schemas as the source of truth and keeps only strategy-level guidance.

  • Removes the hardcoded parameter lists from the "Analyze with flamegraphs" step and the "MCP tools reference" section, replacing them with prose guidance that does not name individual parameters.
  • Retains the full optimization workflow, language-specific notes, and guiding principles unchanged.

Confidence Score: 5/5

Documentation-only change that removes stale parameter references; safe to merge.

The change touches only SKILL.md — no code paths, no logic, no schemas. It replaces hardcoded parameter lists that had drifted from the actual MCP tool signatures with prose that defers to the live schemas. The optimization workflow steps and guiding principles are unchanged. There is no risk of regression.

No files require special attention.

Important Files Changed

Filename Overview
skills/codspeed-optimize/SKILL.md Removes stale hardcoded MCP parameter names (benchmark_name, depth_limit, root_function_name) and replaces them with strategy-level guidance that defers to live tool schemas for parameter details.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Start: User wants to optimize] --> B[Establish baseline run\ncodspeed run -m simulation]
    B --> C[Analyze with flamegraphs\nlist_runs → query_flamegraph]
    C --> D[Identify top 2-3 bottlenecks\nby self time]
    D --> E[Make ONE targeted change]
    E --> F[Re-measure\ncodspeed run -m simulation]
    F --> G[compare_runs: baseline vs new]
    G --> H{Improvement?}
    H -->|Yes, >5%| I[Report to user\nask to continue]
    H -->|No / regression| J[Revert change\ntry different approach]
    I --> K{Continue?}
    K -->|Yes| C
    K -->|No| L[Validate with walltime\ncodspeed run -m walltime]
    J --> D
    L --> M[Done]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Start: User wants to optimize] --> B[Establish baseline run\ncodspeed run -m simulation]
    B --> C[Analyze with flamegraphs\nlist_runs → query_flamegraph]
    C --> D[Identify top 2-3 bottlenecks\nby self time]
    D --> E[Make ONE targeted change]
    E --> F[Re-measure\ncodspeed run -m simulation]
    F --> G[compare_runs: baseline vs new]
    G --> H{Improvement?}
    H -->|Yes, >5%| I[Report to user\nask to continue]
    H -->|No / regression| J[Revert change\ntry different approach]
    I --> K{Continue?}
    K -->|Yes| C
    K -->|No| L[Validate with walltime\ncodspeed run -m walltime]
    J --> D
    L --> M[Done]
Loading

Reviews (2): Last reviewed commit: "docs(skills): stop hardcoding MCP tool s..." | Re-trigger Greptile

@codspeed-hq

codspeed-hq Bot commented Jul 10, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 7 untouched benchmarks


Comparing chore/trim-optimize-skill-mcp-schema (f149eec) with main (7791c07)

Open in CodSpeed

The codspeed-optimize skill hand-listed the CodSpeed MCP tools and their
parameters, duplicating the live tool schemas the agent already has. That block
had silently rotted: it advertised `benchmark_name` (renamed to `benchmark_uri`),
a `depth_limit` parameter that no longer exists on `query_flamegraph`, and a
top-level `root_function_name` (now under `filters`).

Trim it to strategy and guidance and let the live tool schemas carry each tool's
name, parameters, and behavior, so the skill can't tell the agent to call tools
with stale arguments the next time the MCP surface changes.

Refs COD-2567
@fargito fargito force-pushed the chore/trim-optimize-skill-mcp-schema branch from bd9fa9b to f149eec Compare July 10, 2026 10:13
@fargito fargito merged commit f149eec into main Jul 10, 2026
22 checks passed
@fargito fargito deleted the chore/trim-optimize-skill-mcp-schema branch July 10, 2026 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants