test: expand E2E integration test suite for cross-SDK parity - #83
Merged
stenalpjolly merged 6 commits intoSep 9, 2026
Merged
stenalpjolly merged 6 commits into
stenalpjolly merged 6 commits into
Conversation
- Add missing toolset and tool loading negative error tests - Add argument validation tests for missing and wrong parameter types - Add bound parameter schema pruning assertion in live integration tests - Add auth failure and missing token negative test cases - Add complex data types suite covering optional search-rows and process-data - Add protocol version selection and client headers E2E tests - Add TOOLBOX_SERVER_URL environment support to ToolboxE2ESetup
anubhav756
requested changes
Aug 18, 2026
anubhav756
left a comment
anubhav756
left a comment
Contributor
There was a problem hiding this comment.
Tests are incorrect, they seem to be passing wrong param values and on failure they seem to be swallowing the error and passing anyway.
anubhav756
requested changes
Aug 18, 2026
stenalpjolly
force-pushed
the
stenalpjolly_integration-test-parity
branch
from
September 3, 2026 09:39
16bb008 to
ec12882
Compare
anubhav756
requested changes
Sep 3, 2026
anubhav756
left a comment
anubhav756
left a comment
Contributor
There was a problem hiding this comment.
The PR summary states:
Adds test cases for missing tokens on authenticated tools, invalid tokens, unauthenticated tools receiving tokens, and failing token supplier futures.
Is that really implemented in this PR?
anubhav756
requested changes
Sep 3, 2026
- Correct process-data parameter payload to use execution_context, user_scores, and feature_flags instead of fictional input_data - Remove exception-swallowing try-catch blocks in testProcessDataWithMapParams and testRunToolAuthWithoutProvidingAuth - Implement missing testProcessDataOmittingOptionalMap test method - Add comprehensive parameter schema assertions in testProcessDataDefinitionSchema - Parse JSON structurally in process-data tests to eliminate non-deterministic map ordering flakes - Narrow exception expectations from Exception.class to CompletionException with specific cause message checks - Consolidate duplicated getTextContent helper into ToolboxE2ESetup for DRY compliance - Align testSearchRowsNonMatchingData with expected empty or 'null' return on argument mismatch - Support TOOLBOX_AUTH_TOKEN_1 and TOOLBOX_AUTH_TOKEN_2 fallback environment variables in ToolboxE2ESetup TAG=agy CONV=f9ec8a85-66e3-479e-a944-3b621f8bcd18
stenalpjolly
force-pushed
the
stenalpjolly_integration-test-parity
branch
from
September 3, 2026 12:07
ec12882 to
e6826b9
Compare
…er precedence
- Align testLoadNonExistentToolset cause check with server error message
- Assert parameter immutability on original tool definition after bindParam
- Check parameter types ('string', 'integer') in search-rows definition schema
- Use LinkedHashMap for map parameters to ensure deterministic key ordering
- Ensure TOOLBOX_SERVER_URL takes precedence when configured in test setup
TAG=agy
CONV=f9ec8a85-66e3-479e-a944-3b621f8bcd18
anubhav756
requested changes
Sep 9, 2026
anubhav756
requested changes
Sep 9, 2026
anubhav756
requested changes
Sep 9, 2026
anubhav756
requested changes
Sep 9, 2026
- Consolidate all E2E integration tests into McpToolboxClientE2ETest - Add text() convenience method to ToolResult - Add tests for null parameters, type mismatches, and invalid map types - Use Jackson structural JSON node parsing in process-data assertions - Use destroyForcibly() for server process teardown - Parameterize protocol version test across supported versions TAG=agy CONV=f9ec8a85-66e3-479e-a944-3b621f8bcd18
…atching output - Unwrap PostgreSQL jsonb_build_object column if present in Jackson parsing - Allow both empty string and 'null' for non-matching search-rows output TAG=agy CONV=f9ec8a85-66e3-479e-a944-3b621f8bcd18
anubhav756
approved these changes
Sep 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Expands the Java SDK E2E integration test suite to achieve full functional parity with the Go, JavaScript/TypeScript, and Python SDKs. Adds comprehensive error handling, argument type and presence validations, schema pruning assertions, optional/default parameter tests, structured Map data handling, protocol version negotiations, and custom client header verifications. Adds a convenience
text()method toToolResult. Consolidates all E2E integration tests into a single test file (McpToolboxClientE2ETest) to optimize CI execution time by booting the live server container only once.Expectation & Implementation
public String text()toToolResultfor easy access to concatenated text content from result blocks.McpToolboxClientE2ETestmanaged byToolboxE2ESetup, eliminating redundant container boot cycles and saving 20-30s in CI.ToolboxE2ESetupto usedestroyForcibly()if the subprocess fails to terminate gracefully within the timeout.TOOLBOX_SERVER_URLis set, reminding contributors of manifest revision requirements.tool.bindParam(...)are pruned from the exposed tool definition schema.search-rows(optional string and integer parameters with omission and explicit values, strict"null"assertion for non-matching data) andprocess-data(nestedMap<String, Object>payloads with structural Jackson JSON parsing handling both array and object payloads).VERSION_2024_11_05,VERSION_2025_03_26,VERSION_2025_06_18,VERSION_2025_11_25) against the live MCP Toolbox server.Test cases
McpToolboxClientE2ETest:testLoadToolsetSpecifictestLoadToolsetDefaulttestLoadNonExistentToolsettestLoadNonExistentTooltestRunTooltestRunToolMissingRequiredParamstestRunToolWrongParamTypetestRunToolWithNullOptionalParamstestRunToolWithNullRequiredParamstestBindParamstestBindParamsCallabletestBoundParamPruningSchematestRunToolAuthtestRunToolWrongAuthtestRunToolAuthWithoutProvidingAuthtestRunToolParamAuthtestRunToolParamAuthNoFieldtestRunToolWithFailingTokenSuppliertestSearchRowsDefinitionSchematestSearchRowsOmittingOptionalstestSearchRowsWithAllParamsProvidedtestSearchRowsMissingRequiredParamtestSearchRowsNonMatchingDatatestProcessDataDefinitionSchematestProcessDataWithMapParamstestProcessDataOmittingOptionalMaptestProcessDataWithWrongMapValueTypetestClientWithCustomHeaderstestClientWithExplicitProtocolVersions(Parameterized across 4 MCP protocol versions)Acceptance criteria
core-java-sdk-prin Cloud Build and GitHub Actions workflows) pass cleanly.Breaking changes
None.