Skip to content

test: end-to-end test with a Go-parity fixture (#9) - #26

Merged
erikist merged 2 commits into
mainfrom
factory/9-test-end-to-end-test-with
Oct 10, 2026
Merged

erikist merged 2 commits into
mainfrom
factory/9-test-end-to-end-test-with

Conversation

@erikist

@erikist erikist commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

Closes #9

Summary

Adds examples/tests/e2e.rs, the Rust counterpart of Go's
examples/gen/example/v1/e2e_test.go at
stablekernel/protoc-gen-go-mcp commit 9e072f9: a fake VibeService
backend runs behind a real in-process tonic server, the generated
VibeServiceMcpServer sits in front of it, and the test talks MCP over
an in-memory tokio::io::duplex transport to drive tools/list and
tools/call the same way a real MCP client would. Every rmcp type is
confined to the harness (new_test_harness/list_tools/call_tool);
the test functions work only with plain ToolInfo/ToolResult values
decoded from the wire response, as Go's do.

Ports all nine named Go tests:

  • TestToolsList → tools_list_reports_all_five_tools_with_descriptions_and_schemas
  • TestCallSetVibe → call_set_vibe_sends_request_and_returns_backend_result
  • TestCallGetVibe → call_get_vibe_sends_request_and_returns_backend_result
  • TestCallSetVibeDetails → call_set_vibe_details_sends_request_and_returns_backend_result
  • TestCallSetVibeArray → call_set_vibe_array_sends_request_and_returns_backend_result
  • TestCallSetVibeObjects → call_set_vibe_objects_sends_request_and_returns_backend_result
  • TestCallSetVibe_GRPCError → call_set_vibe_grpc_error_becomes_tool_error
  • TestCallSetVibe_InvalidArgumentType → call_set_vibe_invalid_argument_type_never_reaches_backend
  • TestCallSetVibe_UnknownField → call_set_vibe_unknown_field_never_reaches_backend

Same inputs as Go, and the same assertions: what the fake backend
received (via a small RecordedRequest enum keyed by RPC name, the
counterpart of Go's fakeVibeService.lastRequest/callCount) and what
the client got back.

Parity fixture: adds examples/tests/parity/go-v0.3.0-tools.json,
Go 0.3.0's tools/list result for example.proto's VibeService
(name/description/inputSchema per tool). How it was captured: no Go
toolchain was available in this environment to go run ./cmd/mcp-vibe, so it was extracted from Go's committed
examples/gen/example/v1/example_mcp.pb.go at 9e072f9 (each tool's
Name, Description and InputSchema literal, fetched from
raw.githubusercontent.com and copied verbatim) — see
examples/tests/parity/README.md for the full explanation. This is the
same source examples/tests/mcp_tool_parity.rs (issue #6) already
draws from; the README cross-references both so a future change to
either stays in sync.

  • tools_list_matches_go_v0_3_0_parity_fixture asserts the Rust
    server's real tools/list response is JSON-equal to the fixture
    (sorted by tool name on both sides, since this generator's
    BTreeMap-backed registration order differs from Go's
    insertion-ordered slice — not part of MCP's tools/list contract).
  • set_vibe_details_result_matches_go_tools_call_result asserts
    SetVibeDetails's tools/call result text, for the same input as
    TestCallSetVibeDetails, is JSON-equal to
    {"previousVibe":"mellow","vibe":"groovy"} — the value Go's
    protojson.Marshal would produce for the backend's
    SetVibeResponse{PreviousVibe: "mellow", Vibe: "groovy"} (both
    fields non-zero, so protojson/pbjson both include them). Derived by
    reading protojson's field-presence rules rather than running Go,
    for the same toolchain-availability reason as above.

Decisions made

  • Test and harness naming/shape follows the conventions already
    established in examples/tests/handler.rs and
    protoc-gen-rust-mcp/tests/server_integration.rs (snake_case
    descriptive test names instead of Go's TestXxx, exact-match
    assertions on gRPC error text, a lazy_client-style never-connects
    client isn't needed here since every test in this file exercises a
    real backend).
  • Used a small RecordedRequest enum instead of Box<dyn Any> to keep
    the "what did the backend receive" assertions type-safe, since Rust
    doesn't have Go's convenient any + type-assert idiom.

Test evidence

$ cargo fmt --all --check
(no output — clean)

$ cargo clippy --workspace --all-targets -- -D warnings
Finished `dev` profile [unoptimized + debuginfo] target(s) in ...
(no warnings)

$ cargo build --workspace
Finished `dev` profile [unoptimized + debuginfo] target(s) in ...

$ cargo test --workspace
... (all existing suites, 51+12+8+1+1+1 = unaffected and green) ...
Running tests/e2e.rs
running 11 tests
test call_set_vibe_invalid_argument_type_never_reaches_backend ... ok
test call_set_vibe_grpc_error_becomes_tool_error ... ok
test call_get_vibe_sends_request_and_returns_backend_result ... ok
test call_set_vibe_details_sends_request_and_returns_backend_result ... ok
test call_set_vibe_array_sends_request_and_returns_backend_result ... ok
test call_set_vibe_unknown_field_never_reaches_backend ... ok
test call_set_vibe_objects_sends_request_and_returns_backend_result ... ok
test call_set_vibe_sends_request_and_returns_backend_result ... ok
test set_vibe_details_result_matches_go_tools_call_result ... ok
test tools_list_reports_all_five_tools_with_descriptions_and_schemas ... ok
test tools_list_matches_go_v0_3_0_parity_fixture ... ok

test result: ok. 11 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

No changes to the generator or example.proto, so make generate was
only run to confirm no drift (git status clean after regenerating).

Mutation check (acceptance criterion)

Broke one schema rule in the generator — schema.rs's message schema
builder, changed additionalProperties: false to true (so unknown
fields would silently validate) — and reran cargo test --workspace:

  • protoc-gen-rust-mcp's existing unit tests caught it immediately:
    schema::tests::parity_with_go_example_input_schemas,
    schema::tests::all_scalars, schema::tests::schema_test_message
    all failed (schema no longer matches the committed/expected JSON).
  • After regenerating examples/ with the mutated generator
    (make generate), this issue's new end-to-end tests also caught it:
    tools_list_reports_all_five_tools_with_descriptions_and_schemas
    and tools_list_matches_go_v0_3_0_parity_fixture both failed (the
    live server's schema no longer matches the Go parity fixture).

Reverted the mutation and regenerated again before committing; git status was clean (no generated-file drift from the real generator).

Notes for reviewers

  • No Go toolchain was available in this sandbox to run mcp-vibe
    directly over stdio, so the fixture was captured by extraction
    instead (per the issue's explicit fallback option); see
    examples/tests/parity/README.md.

  • This PR only adds test files (examples/tests/e2e.rs,
    examples/tests/parity/go-v0.3.0-tools.json,
    examples/tests/parity/README.md); no generator or generated-code
    changes.

  • Self-reviewed via the reviewer subagent; addressed its two
    blocking findings (a doc comment pointing at a nonexistent JSON
    "header comment" — JSON has no comment syntax, now replaced with a
    pointer to the new parity/README.md — and a looser-than-convention
    contains()-based assertion on the gRPC error text, now an exact
    match) plus a stale doc comment it flagged.

  • Factory-authored

sf-factory-bot added 2 commits October 10, 2026 01:48
Port Go's examples/gen/example/v1/e2e_test.go (stablekernel/protoc-gen-go-mcp
@ 9e072f9) to examples/tests/e2e.rs: a fake VibeService backend runs behind
a real in-process tonic server, the generated VibeServiceMcpServer sits in
front of it, and the test talks MCP over an in-memory transport to drive
tools/list and tools/call the same way a real MCP client would. All rmcp
types stay inside the harness; the test functions work with plain ToolInfo/
ToolResult values decoded from the wire response.

Ports TestToolsList, TestCallSetVibe, TestCallGetVibe, TestCallSetVibeDetails,
TestCallSetVibeArray, TestCallSetVibeObjects, TestCallSetVibe_GRPCError,
TestCallSetVibe_InvalidArgumentType and TestCallSetVibe_UnknownField.

Adds the cross-implementation parity fixture
examples/tests/parity/go-v0.3.0-tools.json: Go 0.3.0's tools/list result for
example.proto, extracted from its committed example_mcp.pb.go (no Go
toolchain was available to run mcp-vibe directly). Asserts the Rust server's
tools/list is JSON-equal to it, and that SetVibeDetails's tools/call result
text matches Go's expected output for the same input.
- Add parity/README.md documenting how the Go tools/list fixture was
  captured (extracted from Go's committed example_mcp.pb.go, no Go
  toolchain available), replacing doc comments that incorrectly
  pointed at a 'header comment' in the JSON fixture file (JSON has no
  comment syntax).
- Assert the gRPC-error tool result text exactly
  ('rpc error: code = NotFound desc = vibe not found'), matching the
  exact-match convention already used by handler.rs and
  server_integration.rs, instead of two separate .contains() checks.
- Fix a stale doc comment on new_test_harness describing a return
  shape it no longer has.
@erikist

erikist commented Oct 10, 2026

Copy link
Copy Markdown
Contributor Author

🔍 Review: LGTM. I checked the fixture against a live Go server. I built Go's cmd/mcp-vibe at 9e072f9 (go 1.26.2) and drove it over stdio with initialize, tools/list and tools/call. Its live tools/list is JSON-equal to go-v0.3.0-tools.json (5 tools; each has exactly name, description and inputSchema), so extracting it from source was faithful.

Two observations, neither blocking:

  • Ordering: Go's live tools/list order is alphabetical too (GetVibe, SetVibe, SetVibeArray, SetVibeDetails, SetVibeObjects), the same as this generator's BTreeMap. Sorting in the test is therefore harmless but not needed; the order matches.
  • Result envelope: Go's success result is {"content":[…]}, with isError omitted. rmcp emits "isError":false plus "resultType":"complete". That's semantically equal, and it's why comparing only the result text (as this PR does) is the right parity check. Live Go output for SetVibe{vibe:\"\"} is rpc error: code = Unknown desc = vibe cannot be empty with isError:true, matching our format.

@erikist
erikist merged commit c5b10b1 into main Oct 10, 2026
6 checks passed
@erikist
erikist deleted the factory/9-test-end-to-end-test-with branch October 10, 2026 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: end-to-end test with a Go-parity fixture

1 participant