Repository navigation
Conversation
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* feat: add MCP Apps extension substrate Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: document MCP Apps extension substrate Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: linearize MCP Apps migration Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: cover MCP Apps extension branches Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * feat: fixing mcp apps rendering Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: linting cleanop Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: formatting and linting Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: clarify MCP Apps UI behavior Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * feat: code improvements Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: breaking test Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh branch after rebase Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: linearize MCP Apps migration after rebase Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: stabilize MCP Apps CI failures Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: harden mcp apps security substrate Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: refresh mcp apps branch after rebase Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: cover mcp apps ci gaps Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: support mcp app helper tools in clients Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: stabilize router and wheel checks Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: satisfy pylint for app tool proxy Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh validation metadata Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: harden MCP Apps tool visibility Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: align MCP Apps metadata contract Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: fix MCP Apps UI resource metadata assertion Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: keep pyproject aligned with main Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * feat: addressed comments Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: breaking test Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: address MCP Apps review feedback Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: keep app tools behind AppBridge Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: trim unused MCP Apps registry helpers Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh secrets baseline Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: keep app-only tools out of model listings Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: handle AppBridge resource lookup failures Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: finalize MCP Apps branch validation Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
…otifications now relayed (IBM#5603) The POST-correlated notification relay fix closes GAP-001 (server-initiated log notifications) and GAP-002 (progress notifications) on gateway_proxy and gateway_virtual targets. - Remove xfail_on from test_log_message_reaches_client and test_progress_notifications_delivered - Move GAP-001 and GAP-002 to Closed gaps in COMPLIANCE_GAPS.md - Update GAP-011 cross-reference to exclude closed gaps Signed-off-by: Jonathan Springer <jps@s390x.com>
* test: retry live RBAC per-server access Signed-off-by: lucarlig <luca.carlig@ibm.com> * feat: make dataplane publisher snapshot interval configurable The dataplane publisher interval was a hardcoded 60s module constant, so runtime-created users and servers stayed invisible to the dataplane for up to a full minute. Expose it as DATAPLANE_PUBLISHER_INTERVAL_SECONDS (default 60, unchanged) and derive the UserConfig key TTL as interval + 10, preserving the current 60/70 relationship. Functional test stacks can now run a short interval while load benchmarks keep the default. Signed-off-by: lucarlig <luca.carlig@ibm.com> * test: cover a full publisher interval in RBAC allow-path retries A 5 x 100ms retry window cannot cover the dataplane publisher snapshot cycle (default 60s), so the allow-path tests still failed on the split stack. Switch the helper to a deadline-based wait that covers one full publish interval plus slack by default (75s), tunable via MCP_E2E_PUBLISHER_SYNC_DEADLINE for stacks that run a short DATAPLANE_PUBLISHER_INTERVAL_SECONDS. Signed-off-by: lucarlig <luca.carlig@ibm.com> * test: drop publisher sync deadline env var The deadline only bounds how long a genuine failure takes; passing runs return as soon as the publisher converges, so a fixed 75s covering one default publish interval plus slack works on every stack and the knob adds nothing. Signed-off-by: lucarlig <luca.carlig@ibm.com> * test: make stdin reader test deterministic Signed-off-by: lucarlig <luca.carlig@ibm.com> * test: trim RBAC retry comments Signed-off-by: lucarlig <luca.carlig@ibm.com> * test: assert allow-path tools lists are non-empty The per-server allow-path checks only asserted that tools/list succeeded, so a deployment that silently drops the server's backend passed them with an empty list. Assert the visibility servers actually advertise their associated tools. Signed-off-by: lucarlig <luca.carlig@ibm.com> (cherry picked from commit 3eea93d) * fix: apply dataplane publisher interval dynamically Signed-off-by: lucarlig <luca.carlig@ibm.com> * test: make publisher sync deadline configurable Signed-off-by: lucarlig <luca.carlig@ibm.com> * chore: refresh secrets baseline Signed-off-by: lucarlig <luca.carlig@ibm.com> --------- Signed-off-by: lucarlig <luca.carlig@ibm.com> Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com>
* fix(output-length-guard): enforce limits on resource.text in content array Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * test(output-length-guard): fix always-skipped double-truncation test Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * test(output-length-guard): trim docstring for double-truncation test Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore: merge main and update .secrets.baseline line numbers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
…BM#5622) * Update secrets and reset some of content from .secrets.baseline file Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Updates with linting formatting Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Update to pass grype issues with container build Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> --------- Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
IBM#5515) The compliance virtual_server fixture connected to /servers/{id}/mcp immediately after creation. On split deployments that route is served from asynchronously published dataplane config, so early gateway_virtual rows failed with 404/400 before the first snapshot landed. Probe the route until it accepts (75s deadline, one default publish interval plus slack); control-plane-only deployments return on the first probe. Signed-off-by: lucarlig <luca.carlig@ibm.com>
…ts that run on Go (IBM#5596) * fix: remove premature exit in Containerfile and restore Go linting targets - Containerfile: Remove stray "exit 0" inside the ENABLE_RUST=false fallback block that caused a buildkit warning and would short-circuit the build when ENABLE_RUST=true with a trailing backslash. - Makefile: Add explicit GOPATH, GOMODCACHE, GOCACHE, and GOBIN environment variables to all Go-based linting targets (actionlint, reviewdog, chart-testing, kube-linter) so they function correctly regardless of the host Go toolchain configuration. Signed-off-by: Jonathan Springer <jps@s390x.com> * Remove LINT_GO_TOOLCHAIN Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com>
… connections (IBM#4724) * fix(redis): raise maxclients from 10K to 30K to support 10 replicas. (IBM#4630) - Update maxclients to 30,000 in all docker-compose files - Add redis.maxclients configuration to Helm chart (values.yaml, schema, configmap) - Update capacity formula comments: replicas × workers × pool < maxclients - Add documentation about server-side vs client-side connection limits - Fixes IBM#4630: Redis maxclients exceeded at 10 replicas Formula: 10 replicas × 24 workers × 100 pool = 24,000 connections With 20% headroom: 28,800 → 30,000 maxclients This addresses the root cause (server-side capacity) rather than degrading client-side performance by reducing connection pools. Signed-off-by: Claudia Gray <claudiag082004@gmail.com> Change maxclients to 15000 and connection pool size to 50 Signed-off-by: Claudia Gray <claudiag082004@gmail.com> Replace accidentally deleted config: line Signed-off-by: Claudia Gray <claudiag082004@gmail.com> * Pass maxclients as fallback and fix math calculation error in comment Signed-off-by: Claudia Gray <claudiag082004@gmail.com> update baseline secrets Signed-off-by: Claudia Gray <claudiag082004@gmail.com> * add updates for docs Signed-off-by: Claudia Gray <claudiag082004@gmail.com> Update secrets baseline and resolve conflict Signed-off-by: Claudia Gray <claudiag082004@gmail.com> fixed pre-commit Signed-off-by: Claudia Gray <claudiag082004@gmail.com> * fix pre-commit Signed-off-by: Claudia Gray <claudiag082004@gmail.com> * fix dead anchor in configuration.md, add helm-unittest for deployment-redis Signed-off-by: Claudia Gray <claudiag082004@gmail.com> --------- Signed-off-by: Claudia Gray <claudiag082004@gmail.com>
* Fix playwright tests Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Adress comments Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Fix some more failing tests Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Address comments Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Address comments Signed-off-by: Gabriel Costa <gabrielcg@proton.me> --------- Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
…s/{tool_id} (IBM#5314)
* fix(schemas): extend ToolCreate/ToolUpdate assemble_auth to handle auth_headers array for authheaders auth type
Signed-off-by: Shoumi <shoumimukherjee@gmail.com>
* fix tests
Signed-off-by: Shoumi <shoumimukherjee@gmail.com>
* chore: update .secrets.baseline line numbers after rebase
Signed-off-by: Shoumi <shoumimukherjee@gmail.com>
* fix(schemas): unify authheaders array validation across tool/gateway/a2a schemas
Signed-off-by: Shoumi <shoumimukherjee@gmail.com>
* fix: reject invalid auth_headers with 422 and share header validation
Signed-off-by: Marek Dano <Marek.Dano@ibm.com>
* fix(schemas): cap auth_headers length before iterating and correct unset semantics
Signed-off-by: Marek Dano <Marek.Dano@ibm.com>
* chore: update .secrets.baseline line numbers after schemas.py changes
Signed-off-by: Marek Dano <Marek.Dano@ibm.com>
* fix(admin): treat non-list auth_headers form value as no headers instead of 500
Signed-off-by: Marek Dano <Marek.Dano@ibm.com>
* chore: update .secrets.baseline line numbers after test_admin.py changes
Signed-off-by: Marek Dano <Marek.Dano@ibm.com>
* fixed pre-commit
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
* fixed pre-commit
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
---------
Signed-off-by: Shoumi <shoumimukherjee@gmail.com>
Signed-off-by: Marek Dano <Marek.Dano@ibm.com>
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
Co-authored-by: Marek Dano <Marek.Dano@ibm.com>
Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
IBM#5611) * feat(plugins): extend S4 metrics allowlist for 5 new cpex-* plugins Extends _SAFE_STRING_FIELD_NAMES / _SAFE_NUMERIC_FIELD_NAMES so secrets_detection, encoded_exfil_detection, url_reputation, rate_limiter, and retry_with_backoff metrics fields survive sanitization instead of being silently dropped as not-yet-allowlisted field names. Part of IBM#5554. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat(plugins): bump cpex-secrets-detection floor to 0.3.7, document metrics contract cpex-secrets-detection 0.3.7 is the first release that emits result.metadata["secrets_detection"] for the S4 metrics pipeline; bump the pyproject.toml floor to match, following the cpex-pii-filter>=0.3.6 comment convention from IBM#5470. Deliberately do NOT flip plugins/config.yaml's SecretsDetection mode from "disabled" to "sequential". Per cpex.framework.PluginMode / manager.py load_plugins(), a "disabled" plugin is never instantiated or registered -- its hooks never run at all. This block's config already has block_on_detection: true and min_findings_to_block: 1, so enabling it would newly start blocking tool calls/prompts/resources on secret-shaped content for every existing deployment using the default config -- a materially bigger change than "add metrics", and bigger than the pii_filter precedent (which stays disabled by design, with block_on_detection: false even in its documented enabled state). Add version: "0.3.7" plus a comment documenting this decision instead. Extend the internal-observability.md plugin-metrics doc with a per-plugin field reference table (pii_filter baseline + secrets_detection's total_detections/total_masked/total_blocked/secret_types) and note that metrics require both the version floor and an enabled mode. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat(plugins): bump cpex-encoded-exfil-detection floor to 0.3.6, document metrics contract cpex-encoded-exfil-detection 0.3.6 is the first release that emits result.metadata["encoded_exfil_detection"] for the S4 metrics pipeline; bump the pyproject.toml floor to match, following the cpex-pii-filter>=0.3.6 and cpex-secrets-detection>=0.3.7 comment convention from IBM#5470 and the prior secrets_detection pin bump. Deliberately do NOT flip plugins/config.yaml's EncodedExfilDetector mode from "disabled" to "sequential". Per cpex.framework.PluginMode / manager.py load_plugins(), a "disabled" plugin is never instantiated or registered -- its hooks never run at all. This block's config already has block_on_detection: true and min_findings_to_block: 1, so enabling it would newly start blocking tool calls/prompts/resources on encoded-payload-shaped content for every existing deployment using the default config -- the same materially bigger change documented for secrets_detection. Add version: "0.3.6" plus a comment documenting this decision instead, and clean up the stale commented-out `# mode: "sequential"` line that was left over from before this decision was settled. Extend the internal-observability.md plugin-metrics doc with encoded_exfil_detection's total_detections/encoding_types/redacted fields in the per-plugin field reference table, and note its 0.3.6 metrics floor alongside pii_filter/secrets_detection in the dependency section. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat(plugins): bump cpex-url-reputation floor to 0.3.5, document metrics contract cpex-url-reputation 0.3.5 is the first release that emits result.metadata["url_reputation"] for the S4 metrics pipeline; bump the pyproject.toml floor to match, following the cpex-pii-filter>=0.3.6, cpex-secrets-detection>=0.3.7, and cpex-encoded-exfil-detection>=0.3.6 comment convention from IBM#5470 and the prior secrets_detection/encoded_exfil_detection pin bumps. Deliberately do NOT flip plugins/config.yaml's URLReputationPlugin mode from "disabled" to "sequential". Per cpex.framework.PluginMode / manager.py load_plugins(), a "disabled" plugin is never instantiated or registered -- its hooks never run at all. This block's config already has blocked_domains and blocked_patterns entries, so enabling it would newly start blocking resource fetches for every existing deployment using the default config -- the same materially bigger change documented for secrets_detection and encoded_exfil_detection. Add version: "0.3.5" plus a comment documenting this decision instead. Extend the internal-observability.md plugin-metrics doc with url_reputation's total_checked/total_blocked/reputation_categories fields in the per-plugin field reference table, using the actual violation-reason slugs the plugin emits (blocked_domain, insecure_scheme, high_entropy_domain, illegal_tld, unicode_spoofing, blocked_pattern, malformed_url, malformed_domain, internal_error, other) rather than the malware/phishing placeholders from an earlier draft of the plan, and note its 0.3.5 metrics floor alongside the other three plugins. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat(plugins): bump cpex-rate-limiter floor to 0.1.7, document metrics contract Bumps the cpex-rate-limiter dependency pin to >=0.1.7, the first release that emits result.metadata["rate_limiter"] (per-call allowed/throttled 0-or-1 values plus backend) when a trace is active, and documents the metrics contract in plugins/config.yaml and the internal observability docs, following the same pattern established for SecretsDetection, EncodedExfilDetector, and URLReputationPlugin. mode remains "disabled". Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat(plugins): bump cpex-retry-with-backoff floor to 0.3.6, document metrics contract Bumps the cpex-retry-with-backoff dependency pin to >=0.3.6, the first release that emits result.metadata["retry_with_backoff"] (retry_count, retry_delay_ms per-attempt) when a trace is active, and documents the metrics contract in plugins/config.yaml and the internal observability docs, following the same pattern established for SecretsDetection, EncodedExfilDetector, URLReputationPlugin, and RateLimiterPlugin. mode remains "disabled". Also drops the pre-existing <0.3.2 upper bound. git log -S confirms it was introduced in IBM#5332 ("fix(ci): pin retry-with-backoff to 0.3.1") as a deliberate CI-compat pin, not a documented runtime incompatibility -- no CI config, workflow comment, or test file references an actual incompatibility with versions 0.3.2-0.3.5. Since the new floor (0.3.6) is well past the old ceiling, the ceiling is superseded and removed. Note: this could not be verified against the real published 0.3.6 package in this sandbox; a real CI run is still needed before merge. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(plugins): add integration + e2e metrics coverage for 5 new plugins Adds real-DB integration tests for rate_limiter, retry_with_backoff, encoded_exfil_detection, and url_reputation in test_plugin_metrics_consumer_integration.py (10/10 passing locally with --with-integration), following the existing pii_filter test class's fixtures and style exactly. Adds a secrets_detection e2e test (TestOtelSecretsDetectionMetadataE2E) in test_otel_plugin_metadata_e2e.py as the second Rust-core parity check alongside pii_filter, via a new traced_app_secrets_detection fixture that force-enables SecretsDetection with block_on_detection=false/redact=true (mirroring PIIFilter's redact-not-block default) so record_plugin_metrics actually gets a chance to run. Verified this test genuinely passes after dev-installing cpex-secrets-detection into the local .venv (git-ignored, not part of this commit); without that package installed it fails at fixture setup with the same ModuleNotFoundError-derived AssertionError the existing pii_filter tests already hit in this environment -- that gap is pre-existing and out of scope here. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(deps): extend exclude-newer-package cutoffs for 5 bumped cpex floors Tasks 2-6 each bumped one cpex-* plugin's floor version in [project.optional-dependencies].plugins to the first release emitting result.metadata for the S4 metrics pipeline, but none updated the separate [tool.uv.exclude-newer-package] table, which still capped resolution at 2026-06-29T23:59:59Z for all five. That made the new floors (cpex-secrets-detection>=0.3.7, cpex-encoded-exfil-detection>=0.3.6, cpex-url-reputation>=0.3.5, cpex-rate-limiter>=0.1.7, cpex-retry-with-backoff>=0.3.6) unreachable to the resolver, so `uv lock`/`uv run` failed before any test could run -- caught by Task 8's verification pass. Extend each package's cutoff to 2026-07-13T23:59:59Z, following the cpex-pii-filter precedent (extended cutoff + explanatory comment naming the version, its result.metadata field, and the dependent test files). Verified against the sibling cpex-plugins repo's git history: the version-bump commits for all five packages landed today (2026-07-13), matching the new cutoff date. Note: uv lock still cannot fully resolve in this sandbox -- confirmed via direct PyPI queries that 4 of the 5 target versions (all but cpex-secrets-detection, which is separately blocked upstream) have not actually been published yet, independent of this cutoff fix. Signed-off-by: Suresh Kumar Moharajan <suresh12.dmm@gmail.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(plugins): correct bool-bypass doc claim and cover redacted field The "Per-plugin metadata fields" preamble in internal-observability.md claimed every field must appear in _SAFE_STRING_FIELD_NAMES or _SAFE_NUMERIC_FIELD_NAMES or it's silently dropped. That's inaccurate for bool-typed fields: _sanitize_plugin_metrics() accepts any bool value unconditionally, before either allowlist check runs, which is exactly why encoded_exfil_detection's redacted field survives despite not being in either allowlist. Correct the doc to call out the bool exception explicitly. Also add unit coverage proving the redacted bool field survives sanitization (previously untested), and fix a cosmetic type mismatch in the url_reputation integration test where reputation_categories was passed as a plain string instead of the documented list[str]. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs(plans): add implementation plan for issue IBM#5554 gateway rollout Records the plan this branch executed. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(deps): bump exclude-newer-package cutoffs to 2026-07-14 cpex-plugins#131 merged and published all 5 packages today (2026-07-14, ~13:46-14:27 UTC) rather than on 2026-07-13 as originally guessed when the cutoff dates were set — uv's own resolver error named the exact mismatch: "cpex-encoded-exfil-detection v0.3.6 was published at 2026-07-14T13:46:18Z" vs a 2026-07-13T23:59:59Z cutoff. Regenerated uv.lock against the corrected cutoffs; all 5 floors now resolve. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(tests): resolve detect-secrets false positive on AWS key placeholder detect-secrets dedupes identical secret values within a file to a single finding keyed by the last occurrence's line, which was masking the actual issue: RAW_SECRET_VALUE (AWS's own published documentation placeholder key, deliberately AWS-key-shaped to exercise the real secrets_detection plugin) was never actually audited. Mark it inline per detect-secrets' own suggested fix; drop the redundant restatement of the literal value in a nearby comment. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(tests): update pre-existing rate_limiter/encoded_exfil unit tests for the metrics migration These gateway-side tests import the real cpex_rate_limiter/ cpex_encoded_exfil_detection packages directly and never actually ran until now (dependency resolution against the real published versions was broken until the previous commit) -- so they'd never caught up with the OTel metrics migration: - rate_limiter: asserted the old unconditional flat metadata write (result.metadata["remaining"]/["reset_in"]), removed when the gateway-side review (cpex-plugins#131) trimmed the metrics dict to only allowed/throttled/backend. Fixed the packaged-core call-signature assertions to include the extensions arg the real 0.1.7 wrapper always passes. - encoded_exfil_detection: asserted the old flat "encoded_exfil_redacted"/"encoded_exfil_count" keys, removed in the 0.3.6 migration in favor of the namespaced, trace-gated result.metadata["encoded_exfil_detection"] dict. Updated the redaction tests (no trace context passed) to expect empty metadata, and modernized the two "returns metadata" tests to pass real trace context and assert the new schema, preserving their original intent. Verified: both files pass in full (94 passed, 2 xfailed). Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(plugins): bump cpex-retry-with-backoff to 0.3.7 (retry-detection fix) 0.3.6 added the result.metadata["retry_with_backoff"] emission this PR's metrics wiring depends on, but shipped a retry-detection regression: the plugin's is_failure read the tool_post_invoke result via PyDict::get_item (C-level dict storage), which silently misses every key on the gateway's CopyOnWriteDict payload-isolation wrapper (its data lives in a Python-level overlay) -- every retryable failure looked like a success and the plugin never retried a single call. This is why the retry-with-backoff static and binding E2E legs were red on this PR. 0.3.7 (IBM/cpex-plugins#137, closing IBM#124) fixes retry detection by reading the result through the Python mapping protocol instead. Bumps the pyproject floor, the exclude-newer-package cutoff, the plugins/config.yaml version field, and refreshes uv.lock (uv lock resolved cleanly against the new floor; only cpex-retry-with-backoff's pin changed). Verified: installed 0.3.7 from PyPI (not a local build) via `uv sync --extra plugins --extra redis` and re-ran tests/live_gateway/plugins/test_retry_with_backoff_e2e.py end-to-end against the real fast-time-server image -- both static and binding legs pass. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: untrack docs/superpowers planning doc Removes the implementation-plan doc from git tracking without deleting it locally -- it's a planning artifact, not something that belongs in the repo history. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(e2e): extend real observability e2e coverage to the 4 remaining plugins Reviewer feedback on this PR: only secrets_detection (and pre-existing pii_filter) had real traced-HTTP-pipeline coverage in tests/e2e/test_otel_plugin_metadata_e2e.py; the other 4 plugins this branch also wires metrics for (rate_limiter, retry_with_backoff, encoded_exfil_detection, url_reputation) only had the fake-metadata-dict integration test in tests/integration/plugins/test_plugin_metrics_consumer_integration.py, which proves the DB-write/sanitization logic but not that the real plugin actually emits that shape over a real traced request. Adds one real-e2e test per plugin, each driving the real cpex-* Rust-backed plugin (installed from PyPI per this branch's pyproject.toml floors) through a genuinely traced HTTP request against the real gateway request pipeline, asserting the resulting plugin.metrics.<name> span reaches GET /observability/traces/{trace_id}: - encoded_exfil_detection: tool_post_invoke, mirrors the secrets_detection pattern exactly (block_on_detection=false, redact=true), asserts total_detections/encoding_types and that the raw encoded payload never leaks into the response. - retry_with_backoff: tool_post_invoke against a persistently-failing mocked upstream, so the gateway genuinely retries (real backoff delays, real recursive re-invocation) up to max_retries -- asserts multiple spans with strictly increasing retry_count. This is the exact regression IBM/cpex-plugins#124 introduced and IBM#137 (0.3.7) fixed: against the buggy 0.3.6, this test would see exactly one span with retry_count=0 (no retry ever detected). - url_reputation: resource_pre_fetch, not tool_post_invoke -- registers a real resource and reads it back (POST/GET /resources) rather than tools/call. Along the way, discovered /resources/* isn't in the shipped settings.observability_include_paths allowlist, so this fixture patches it in for itself (and clears path_filter.py's lru_caches on setup/teardown) -- otherwise resource reads are never traced under the shipped defaults. - rate_limiter: tool_pre_invoke, requires a real reachable Redis (the plugin fails open with empty metadata when Redis is unreachable, so there's no way to prove real metrics without one). tests/e2e/ has no Redis service in CI (unlike plugin-integration.yml's dedicated legs), so this test skips cleanly via a reachable_redis_url fixture (mirrors the existing otel_memory_exporter graceful-skip-when-extra-missing pattern) rather than failing CI. Factored the ~90-line real-temp-DB/dedicated-FastAPI-app/dependency-override-auth boilerplate shared by 3 of the 4 new fixtures into _build_traced_app() (asynccontextmanager) -- the existing traced_app/traced_app_secrets_detection fixtures are deliberately left untouched per their own docstrings' stated reasoning for staying self-contained; url_reputation's resource-based flow doesn't fit the tool-registration shape at all and gets its own dedicated fixture. Verified: all 4 new tests pass individually and together with the existing 3 (tests/e2e/test_otel_plugin_metadata_e2e.py -- 6 passed, 1 pre-existing unrelated skip for the optional opentelemetry-sdk extra), the rate_limiter test both passes against a real Redis and skips cleanly with none reachable, and the full tests/e2e/ directory (74 files) runs clean with only 3 pre-existing/unrelated skips and zero failures. pre-commit passes on the changed file. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh12.dmm@gmail.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…de dependabot (IBM#5623) * Remove renovate json file and update security.md to include dependabot Signed-off-by: Claudia Gray <claudiag082004@gmail.com> * Update security.md wording to be accurate Signed-off-by: Claudia Gray <claudiag082004@gmail.com> --------- Signed-off-by: Claudia Gray <claudiag082004@gmail.com>
…rker across instances (IBM#5430) * feat(plugins): add redis backend for cross-instance primary-worker election Adds an opt-in Redis backend so a single primary can be elected across multiple gateway instances/replicas, not just per host. The default stays filelock (per-host), so existing single-host deployments are unchanged. - mcpgateway/services/leader_election.py: PrimaryWorkerElector with two backends. filelock is passive (OS releases on exit, no task); redis uses a lease (SET NX EX) renewed by an atomic compare-and-renew Lua heartbeat, released by an if-owner Lua, with a follower loop that re-acquires on expiry. - is_primary_worker() reads the elector's cached flag for the redis backend and fails closed if it isn't started (a redis deployment never silently degrades to per-host scope). - main.py lifespan starts the elector before plugin init (awaits the initial election) and stops it on shutdown; only the redis backend needs an elector. - Config (conservative defaults): PRIMARY_WORKER_ELECTION_BACKEND=filelock, PRIMARY_WORKER_REDIS_KEY, PRIMARY_WORKER_LEASE_TTL, PRIMARY_WORKER_HEARTBEAT_INTERVAL, PRIMARY_WORKER_REDIS_UNAVAILABLE_POLICY= fail_closed (filelock_fallback optional). - Tests with fakeredis cover single-primary, renew, release, follower takeover, fail-closed and filelock-fallback; e2e script is backend-aware. Election under the redis backend is best-effort under network partitions; keep side effects idempotent. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(plugins): prove hooks run on all workers and default path skips the elector - e2e: add an ungated hook plugin and assert it loads on every worker (one marker line per worker), alongside the gated non-hook plugin asserting one. The e2e runner is backend-aware, so this holds under filelock and redis — election only gates the opt-in side effect, never the hook path. - unit: assert the default election backend is filelock and that the filelock path never consults the elector (locks the non-hook default behavior). Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(plugins): add multi-instance e2e showcase for redis primary-worker election Scales the standard docker-compose gateway to 2 replicas with the redis election backend and asserts exactly one primary is elected across the containers — the cross-instance guarantee a per-host file lock cannot provide. - docker-compose.yml: expose PRIMARY_WORKER_ELECTION_BACKEND and PRIMARY_WORKER_REDIS_UNAVAILABLE_POLICY as env-configurable settings, defaulting to filelock / fail_closed (no behavior change by default). - plugins/primary_worker_multiinstance: a non-hook marker plugin that RPUSHes <host>:<pid> to a shared Redis list when primary, so the count is observable across containers (a per-container file marker cannot be). - run_primary_worker_multiinstance.sh + make test-primary-worker-multiinstance: redis up -> clear key -> up --scale gateway=2 -> wait healthy -> assert the list has exactly one entry. Not part of `make test` (needs Docker + a branch-built image). Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * docs(plugins): tighten election comments and drop spike wording Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(plugins): add KEEP_UP flag to the multi-instance e2e for redis inspection Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(plugins): count distinct primaries via a redis set (robust to re-init) Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(plugins): cover redis elector lifecycle, heartbeat, and singleton paths Raise diff coverage on the redis backend above the CI gate: exercise the maintenance loop (lost-lease demotion via compare-and-renew, mid-heartbeat error -> fail-closed), the own-client build/close path, the started/instance_id properties, and the module-level start/get/stop singleton helpers. Brings leader_election.py to 100% line coverage. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix(plugins): demote on mid-heartbeat redis error under filelock_fallback The redis elector's maintenance loop only demoted a stale primary for the fail_closed policy. Under filelock_fallback, a Redis error mid-heartbeat left _is_primary=True with no election performed: the lease then expired unrenewed, a follower acquired it via SET NX EX, and two processes reported primary -- the split-brain this backend prevents. Route the maintenance-loop error through _handle_redis_unavailable() for both policies, matching initial-connect behavior: fail_closed demotes; filelock_fallback re-elects per host (and demotes if the lock is held). Make _acquire_filelock() idempotent (reuse one FileLock, short-circuit when already held) so the loop can call it each heartbeat during an outage without churning the lock. Add regression tests for filelock_fallback + mid-heartbeat error: demotes when the file lock is held elsewhere, re-elects when it is free. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * refactor(plugins): address non-blocking review notes on the redis elector - Dedup the port-scoped lock-path helper: reuse _lock_path() from mcpgateway.utils.primary_worker instead of a hand-mirrored copy (aliased on import, so call sites are unchanged). No circular import — primary_worker only imports leader_election lazily. - Honor explicit falsy options: use 'x if x is not None else settings.y' rather than 'x or settings.y' for backend/redis_url/redis_key/lease_ttl/heartbeat/policy, so an explicit 0 or "" isn't silently replaced by the settings default. - Make stop() unconditionally best-effort: also swallow non-CancelledError errors from the cancelled maintenance task (matching the redis-release block) so a task error can't interrupt the rest of the shutdown sequence. Adds a shutdown-robustness test; leader_election.py stays at 100% line coverage. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix(plugins): make elector start()/stop() idempotent on the redis backend start() had no guard for already-started: a second call on the redis backend re-issued SET NX (which fails since we already hold the key) and demoted the active primary to False, and overwrote self._task with a new maintenance loop, orphaning the first (unreachable by stop()). This contradicts the documented "second start reuses the same singleton" contract; the existing singleton test only covered the idempotent filelock path, which masked it. Guard start() with an early return when already started. Also drop the owned Redis client in stop() (self._redis = None in a finally) so a repeat stop() doesn't aclose() an already-closed client; injected clients are left as-is. Add a redis-backend start()-idempotency regression test (is_primary stays True, maintenance task not replaced) and extend the own-client test to assert a second stop() is a no-op. leader_election.py stays at 100% coverage. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * chore(plugins): warn on bad heartbeat/lease-ttl ratio and document redis election caveats Add a startup model-validator (redis backend only) that warns when PRIMARY_WORKER_HEARTBEAT_INTERVAL >= PRIMARY_WORKER_LEASE_TTL/2, since the lease would expire before renewal and cause continuous re-election. Warns rather than raises, matching the existing config-validator style (non-breaking). Document three redis-backend caveats in configuration.md: namespace the lease key when sharing one Redis across deployments, keep heartbeat < lease_ttl/2, and note that a boot-time Redis outage doesn't auto-recover (restart the worker). Related to IBM#5139 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix(plugins): log exception type, not str(exc), in elector redis warnings (CWE-532) A redis.asyncio connection error can stringify the connection target, so logging str(exc) could leak the password from REDIS_URL/PRIMARY_WORKER_REDIS_URL into the application log. Log type(exc).__name__ instead across all elector warnings (redis-unavailable, maintenance error, stop cleanup, task-cancel). Add a regression test that raises a ConnectionError with an embedded DSN mid- heartbeat and asserts the secret never reaches the log while the exception type still does. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(plugins): add concurrent-start, stop-release-error, and heartbeat-boundary tests Harden coverage per review: - test_redis_concurrent_start_elects_one_primary: many electors start() via asyncio.gather, exactly one wins the lease (SET NX is atomic). - test_stop_swallows_redis_release_error: the lease-release eval raising during stop() is swallowed (best-effort shutdown), no propagation. - test_primary_worker_heartbeat_warns_at_exact_boundary: heartbeat == lease_ttl/2 warns (the requirement is strictly less than). leader_election.py stays at 100% coverage. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * chore: reindex .secrets.baseline after rebase onto main Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
* docs: add generic MCP extension framework ADR Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: align extension framework ADR with security governance Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* fix: bind join requests to team context Signed-off-by: lucarlig <luca.carlig@ibm.com> * fix: align join request not found handling Signed-off-by: lucarlig <luca.carlig@ibm.com> * test: cover join request error branches Signed-off-by: lucarlig <luca.carlig@ibm.com> * fix: strengthen join request review handling Signed-off-by: lucarlig <luca.carlig@ibm.com> --------- Signed-off-by: lucarlig <luca.carlig@ibm.com>
…check errors (IBM#5654) * fix(services): rebuild fork-poisoned FileLock and back off on health-check errors GatewayService's FileLock is constructed once, before gunicorn --preload forks worker processes. Newer filelock releases detect reuse of a pre-fork lock from a different PID and raise instead of silently working. The health-check loop's generic exception handler logged that error and immediately re-entered the loop with no delay, producing an unbounded busy loop that starves the worker of the CPU it needs to serve requests (observed as CI jobs hanging until the workflow timeout, with the gateway log growing to millions of lines). Track the PID the lock was built for and rebuild it when a worker detects it's running under a different PID than its creator. Also add the same backoff used by the Timeout branch to the generic exception handler, so any other unexpected lock error can no longer busy-loop. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore(deps): cap filelock below the fork-unsafe 3.30.0 release filelock 3.30.0 (published 2026-07-16T03:53:56Z) added a fork-safety guard that raises RuntimeError when a lock object is reused from a different PID than its creator - exactly the pattern GatewayService._file_lock hits under gunicorn --preload, which was the trigger for the busy-loop fixed in the previous commit. The unbounded floor (filelock>=3.29.4) combined with `make install` resolving live against PyPI (uv pip install ., not uv sync) meant every fresh install silently picked up 3.30.0 the moment it was published, regardless of what's pinned in uv.lock. Capping the floor keeps installs on a known-good release; the previous commit's fork-safe rebuild is the real fix and keeps this code correct even past this ceiling once verified against future filelock releases. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore(deps): raise filelock ceiling to allow the verified-safe 3.30.0 Reproduced the 3.30.0 fork-reuse RuntimeError directly against the real package (os.fork() + FileLock.acquire() on the inherited object) and confirmed the PID-rebuild fix from the first commit neutralizes it - the rebuilt lock never touches the poisoned pre-fork instance, so the new guard never fires. Full gateway_service test suite passes under 3.30.0 as well. <3.30.0 was overly conservative once the actual fork-safety bug is fixed; <3.31.0 keeps a ceiling against an unverified future release without blocking a version we've now tested against directly. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…rom change-password-required.html (IBM#5646) * fix(templates): remove CSP-violating airgapped Tailwind JS branch from change-password-required.html Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * added .secrets.baseline Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…admin (IBM#5647) The pre-seeded Keycloak test user was configured with email admin@example.com, identical to the default PLATFORM_ADMIN_EMAIL value in .env.example. On SSO login, resolve_session_teams() looks up the local user record by email and would find the built-in admin entry, silently granting is_admin:true to the Keycloak test account. Rename username and email: admin@example.com -> admincf@example.com in infra/keycloak/realm-export.json and update the matching reference in the Keycloak SSO tutorial doc. Closes IBM#5542 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
… test endpoint (IBM#5023) * fix: respect global ssrf_protection_enabled flag in gateway test endpoint The /admin/gateways/test endpoint was unconditionally blocking private IPs regardless of the global ssrf_protection_enabled configuration flag. This created inconsistency with other endpoints and prevented testing internal services even when SSRF protection was intentionally disabled. Changes: - Modified validate_gateway_test_url() to check ssrf_protection_enabled before applying private IP blocking (both direct and DNS-resolved IPs) - Allowlist enforcement remains active regardless of SSRF flag (defense in depth) - Added comprehensive test suite with 18 tests covering all scenarios - Created bug report documenting the issue and fix This ensures consistent behavior across all endpoints while maintaining security through layered protections (SSRF + allowlist enforcement). Fixes: Gateway test endpoint SSRF flag bypass Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: address code review findings for gateway test SSRF protection Addresses all code review findings from PR feedback: 1. Updated docstring in validate_gateway_test_url() to accurately describe conditional SSRF behavior based on ssrf_protection_enabled flag. Previous docstring claimed "unconditional blocking" which was no longer true. 2. Updated doctest examples to reflect conditional blocking behavior, adding examples showing private IPs are blocked when ssrf_protection_enabled=true and allowed when false. 3. Renamed three existing tests in test_validators_advanced.py from test_*_blocked_unconditionally to test_*_blocked_when_ssrf_enabled to accurately reflect their behavior. Added explicit ssrf_protection_enabled=True flag pinning so tests verify what they claim. 4. Added WARNING-level audit logging when SSRF protection is bypassed for forensic visibility. Logs are emitted when private/internal IPs are allowed through due to ssrf_protection_enabled=false, both for direct IPs and DNS-resolved IPs. Includes hostname, resolved IP, and IP type classification. 5. Mocked DNS resolution in three integration tests (test_wildcard_subdomain_matching, test_dns_rebinding_protection, test_fqdn_normalization_prevents_bypass) to eliminate network dependencies and improve CI stability. Removed @pytest.mark.integration markers as tests no longer hit real DNS. All tests pass. No functionality changes - only documentation, test accuracy, and observability improvements. Related to: IBM#5022 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: resolve pylint and ruff formatting issues - Remove redundant logger redefinition (use module-level logger) - Fix string concatenation formatting in log messages - Resolves pylint W0621 (redefined-outer-name) - Resolves ruff formatting violations Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(security): address code review findings for SSRF protection PR Resolved three code quality issues from review: 1. DNS Mock Type Mismatch: - Fixed test_dns_timeout_configuration to mock socket.getaddrinfo as synchronous function (with time.sleep) instead of async - Added clarifying comment explaining getaddrinfo is sync but run in executor for async safety 2. Duplicate Exception Handling: - Consolidated exception handling comments at lines 1680-1682 and 1761-1764 for better readability - Clarified when each ValueError case applies 3. Duplicate SSRF Check Logic: - Extracted complex boolean condition into is_dangerous variable at line 1734-1747 for improved readability - No functional changes All tests pass. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * refactor(validators): apply DRY principle to SSRF protection checks Move ssrf_protection_enabled condition into nested if blocks within the main IP validation logic, eliminating duplicate code paths for IP parsing and network checks. This reduces redundancy while maintaining the same security behavior and logging. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * style: apply ruff formatting to is_dangerous check Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
IBM#5637) * Update fix_file_headers to have generic default AUTHOR, defined COPYRIGHT line, and new line fix, and add pre-commit-config.yaml to execute the fix headers check as part of the pre-commit checks so that the author can see the changes Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Update all headers so that checks pass Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Fixed all files with new fix-header rules Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Update .secrets.baseline after rebase Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Correcting straggling files after rebase Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Update CONTRIBUTING doc to be consistent Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Rebased off main and adjusted findings Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Correct Makefile to not refer to authors for fix-headers Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> --------- Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
(IBM#5664) * fix(resources): revert resource name uniqueness constraint from IBM#5158 Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore: update .secrets.baseline line numbers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore: update .secrets.baseline line numbers after code removals Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * test: update resource tests for name-uniqueness revert Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix cwe-117 improper output neutralization logs. Signed-off-by: AHERDERE <aherdere@ie.ibm.com> * cwe_117_improper_output_neutralization_logs Signed-off-by: AHERDERE <aherdere@ie.ibm.com> * fix cwe-117 improper output neutralization logs Signed-off-by: AHERDERE <aherdere@ie.ibm.com> * fix cwe-117 improper output neutralization logs Signed-off-by: AHERDERE <aherdere@ie.ibm.com> * fix cwe-117 improper output neutralization logs Signed-off-by: AHERDERE <aherdere@ie.ibm.com> --------- Signed-off-by: AHERDERE <aherdere@ie.ibm.com> Co-authored-by: AHERDERE <aherdere@ie.ibm.com>
…5669) Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
…rget (IBM#5634) * ci(pyright): add PR-scoped type check workflow and pyright-pr make target Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * ci(pyright): move pyright check into linting-full workflow Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * ci(pyright): move pyrightconfig.json into pyproject.toml [tool.pyright] Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…touched file to then become pyright compliant (IBM#5703) Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
…ion errors (IBM#5663) * fix(transport,middleware): remove manual Content-Length headers causing compression errors Fixes IBM#5457 When compression is enabled (COMPRESSION_ENABLED=true), manually setting Content-Length headers before the compression middleware runs causes "Too much data for declared Content-Length" errors because the declared size is uncompressed but the actual body is compressed. Changes: - Remove manual Content-Length from streamablehttp transport (4 locations) - Skip sanitization of compressed responses in ValidationMiddleware - Add 16 regression tests to prevent future occurrences The compression middleware now correctly sets Content-Length after compressing the response body, ensuring the header matches the actual transmitted byte count. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * addressed the comments Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed headers Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
…BM#6004) * Fixed issue IBM#5980 Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * Fixed non-dict payload handling Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * ci: rerun failed lint workflow Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * fixed edge cases Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * Address review feedback Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * Enhanced docstring Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * Address review feedback Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * Fixed malformed cached JWT teams Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * run pre-commit Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * Fix broken symlinks Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Sagar-Mondal <sagarandshiva@gmail.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* fix: persist Rust control denial telemetry Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: retain plugin denial details in OpenTelemetry spans Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
Signed-off-by: nightcityblade <jackchen@haloailabs.com>
* test: assert the denial reason in test_outsider_denied_tools_execute The assert sat inside a try whose except caught Exception. AssertionError is an Exception, so the test passed under every gateway behaviour. Move the assert outside the try. Narrow the except to the transport errors the MCP SDK raises. Assert the denial reason on both the returned-result path and the raised path, so an unrelated error cannot read as a confirmed denial. Refs IBM#6839 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: assert the error reason in test_outsider_calls_nonexistent_tool_error Both branches passed unconditionally. The assert sat inside a try whose except caught Exception, and AssertionError is an Exception. Move the assert outside the try. Narrow the except to the transport errors the MCP SDK raises. Assert the denial reason, because isError alone stays true for a plain name-resolution failure and cannot detect an RBAC regression. Closes IBM#6839 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: narrow swallowing except in translate dynamic-env E2E tests Three tests put a real assert inside a try whose except caught Exception broadly. AssertionError is an Exception, so a real assertion failure became a skip instead of a failure. Narrow the except to httpx.HTTPError, the base of httpx's transport-error hierarchy. A real connection or stream failure still produces a skip. An assertion failure now propagates as a test failure. Closes IBM#6873 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: narrow RBAC denial status check to 403 only _DENIED_STATUSES accepted 401 alongside 403 as evidence of a confirmed RBAC denial. The outsider fixture holds a valid token, so a 401 means authentication failed before RBAC ran, not that RBAC denied it. Every RBAC denial in mcpgateway/middleware/rbac.py raises 403 with _ACCESS_DENIED_MSG; none raise 401. Accepting 401 let a broken-auth regression masquerade as a confirmed RBAC denial. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: extract the RBAC denial assertion into a shared helper Both outsider tests repeated the same transport and result assertions. Move them into `_assert_denied_for_rbac()`, which takes the call and a label. Replace the `if result is not None` guard with `else`. `_mcp_tool_call()` either returns a result or raises, so the guard was misleading. The `else` body is not covered by the `try`, so it still cannot swallow an `AssertionError`. Hoist `_TRANSPORT_ERRORS` and `_DENIED_STATUSES` to module level. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix(security): require explicit allowlist for CORS origin reflection SecurityHeadersMiddleware reflected any request Origin when ALLOWED_ORIGINS was empty and ENVIRONMENT was not production. It also sent Access-Control-Allow-Credentials: true. A malicious site could read credentialed responses from an exposed development or staging gateway. Reflect an Origin only when it appears in ALLOWED_ORIGINS, in every environment. An empty allowlist now blocks all origins. Replace the development-mode test that asserted the old behavior. Add deny-path tests for an empty allowlist in each environment and for an unlisted origin. Add allow-path tests for listed origins in development and staging. Signed-off-by: Marek Dano <mk.dano@gmail.com> * chore: update secrets baseline for CHANGELOG CORS entry Audit the 'Access-Control-Allow-Credentials: true' string in the new CHANGELOG.md entry as a false positive (Secret Keyword) and refresh line numbers shifted by the entry. Signed-off-by: Marek Dano <mk.dano@gmail.com> * test: cover CORS with no Origin header Add test_cors_no_origin_header_no_cors_headers, parametrized across development, staging, and production. Confirm a request without an Origin header gets no Access-Control-Allow-Origin, Access-Control-Allow-Credentials, or Access-Control-Expose-Headers, even when ALLOWED_ORIGINS is populated. Addresses review feedback on PR IBM#6941 Signed-off-by: Marek Dano <mk.dano@gmail.com> --------- Signed-off-by: Marek Dano <mk.dano@gmail.com>
…P server/virtual server) (IBM#6770) * docs: document API-to-product vocabulary mapping (gateway/server → MCP server/virtual server) Signed-off-by: Marek Dano <mk.dano@gmail.com> * docs: address PR IBM#6770 review comments on terminology.md Signed-off-by: Marek Dano <mk.dano@gmail.com> --------- Signed-off-by: Marek Dano <mk.dano@gmail.com>
* Dual-era gateway: MCP SDK 2.0 migration + 1.x/2026-07-28 interop (IBM#6485) * chore(deps): drop fastmcp dev dependency and protocol-compliance make targets Signed-off-by: Jonathan Springer <jps@s390x.com> * test(live-gateway): rewrite MCP protocol E2E on the official mcp SDK Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(tests): remove the compliance reference server and matrix tooling The fastmcp-based protocol-compliance harness was the only consumer of the compliance reference server and the compliance matrix generator, so both go with it. - delete mcp-servers/python/compliance_reference_server/ and scripts/compliance_matrix.py - drop the compliance-reference-server dev dependency and its [tool.uv.sources] editable entry; regenerate uv.lock (also sheds fastmcp, fastmcp-slim, authlib and their transitives) - drop the compliance-only pytest markers from pyproject.toml - refresh the pip-audit skip-list note and the PYTEST_IGNORE comment Signed-off-by: Jonathan Springer <jps@s390x.com> * docs: drop protocol-compliance references and name the mcp SDK client Follow-up to the harness removal: the PR-review validation gate, the live-gateway README/target list, and the test docs still pointed at `make test-protocol-compliance` and described the protocol E2E suites as FastMCP-driven. - AGENTS.md, tests/AGENTS.md, llms/NOTES.template.md, docs/docs/development/release-management.md, tests/live_gateway/README.md: retire the removed targets, name the official mcp SDK client - tests/live_gateway/helpers/mcp_test_helpers.py: correct the module docstring's consumer list and point new tests at the SDK ClientSession pattern Signed-off-by: Jonathan Springer <jps@s390x.com> * test(sso): move Entra ID integration test to tests/integration Ports the relocation from mcp-2-0-0-prerelease (d7c1f9a): the Entra ID suite is fully in-process (httpx.ASGITransport) and self-skips when the AZURE_* credentials are absent, so it belongs with the integration tests that run under the default 'make test' rather than the live-gateway tree. - tests/live_gateway/sso/test_entra_id_integration.py -> tests/integration/test_entra_id_integration.py (header updated) - tests/live_gateway/sso/__init__.py: docstring now covers only the Keycloak suite and points at the new Entra ID location - docs/docs/testing/entra-id-e2e.md: all pytest invocation paths updated - Makefile: test-e2e-sso targets Keycloak only, notes the Entra ID move (help comment aligned as well) Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(deps): remove fastmcp and rebuild live-gateway E2E on the official mcp SDK Drops the fastmcp dev dependency and everything that depended on it, replacing the live-gateway protocol and RBAC E2E suites with a client built on the official mcp Python SDK. - drop fastmcp dev dependency and protocol-compliance make targets - remove the fastmcp-based protocol compliance harness and the fast-server go/rust parity test - rewrite the MCP protocol and RBAC transport E2E suites on the official mcp SDK ClientSession - remove the compliance reference server and compliance matrix tooling (the harness was their only consumer); regenerate uv.lock, which also sheds fastmcp-slim, authlib and their transitives - update AGENTS.md, test docs, and the live-gateway README to retire the removed targets and name the mcp SDK client Closes IBM#5910 Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(deps): drop fastmcp dev dependency and protocol-compliance make targets Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(tests): remove the compliance reference server and matrix tooling The fastmcp-based protocol-compliance harness was the only consumer of the compliance reference server and the compliance matrix generator, so both go with it. - delete mcp-servers/python/compliance_reference_server/ and scripts/compliance_matrix.py - drop the compliance-reference-server dev dependency and its [tool.uv.sources] editable entry; regenerate uv.lock (also sheds fastmcp, fastmcp-slim, authlib and their transitives) - drop the compliance-only pytest markers from pyproject.toml - refresh the pip-audit skip-list note and the PYTEST_IGNORE comment Signed-off-by: Jonathan Springer <jps@s390x.com> * docs: drop protocol-compliance references and name the mcp SDK client Follow-up to the harness removal: the PR-review validation gate, the live-gateway README/target list, and the test docs still pointed at `make test-protocol-compliance` and described the protocol E2E suites as FastMCP-driven. - AGENTS.md, tests/AGENTS.md, llms/NOTES.template.md, docs/docs/development/release-management.md, tests/live_gateway/README.md: retire the removed targets, name the official mcp SDK client - tests/live_gateway/helpers/mcp_test_helpers.py: correct the module docstring's consumer list and point new tests at the SDK ClientSession pattern Signed-off-by: Jonathan Springer <jps@s390x.com> * feat: migrate to MCP SDK 2.0.0b2 with modern 2026-07-28 upstream negotiation Squashed arc: - MCP SDK migration 2.0.0a2 -> 2.0.0b1 -> 2.0.0b2 (Python 3.14 support added; floor stays >=3.11 — mcp 2.0.0b2 needs only >=3.10, cpex >=3.11, CI is py3.12, MCPError rename, transport/Client compatibility fixes) - Upstream federation path migrated from raw mcp.ClientSession to mcp.client.Client: mode=auto probes server/discover and adopts modern 2026-07-28, with transparent legacy fallback; MCP_CLIENT_CONNECT_MODE=legacy rollback flag - Pooled upstream session registry on the high-level Client with typed SessionLifecycle handle (replaces _cf_* attribute smuggling), cache=False to preserve ADR-052 notification wiring, dispatcher-aware transport health check - mcp_proxy_client gained transport= (streamablehttp/sse) and mode= options; all five ad-hoc SSE fallback sites routed through it - Compose test stack: single cfex-mcp-fast-time-server image (stale Go-era flags fixed), strict modern-only 2026-07-28 fast_time_2026 variant added, five register_* one-shots consolidated into a single 'register' container running scripts/register_mcp_servers.py 'stack' - Tests: SDK invariant spike suite, config validators, reworked registry/service suites, E2E negotiation suite (11 tests) proving auto/legacy modes against strict and legacy upstreams - Docs: MCP_CLIENT_CONNECT_MODE and legacy rollback documented - Fixes: four-parameter httpx2.Timeout in proxy-client default path, token-exchange integration test isolation from shared Redis, pre-existing get_httpx_client_factory dedent repair in gateway_service Packaging — the SDK pin lands in core dependencies, not in extras: - An earlier revision of this branch split the `mcp` pin into mutually exclusive `runtime` / `live-tests` extras backed by a `[tool.uv] conflicts` block, so the gateway could stay on `mcp` 2.x while the fastmcp-based protocol-compliance harness stayed on `mcp` < 2.0. That harness is gone, so the partition is removed: `mcp==2.0.0b2` and `mcp-types==2.0.0b2` sit in core `dependencies` alongside `cpex`, and `pip install mcp-contextforge-gateway` once again resolves a working install. - Every `--extra runtime` / `--extra live-tests` flag is dropped from the Make targets, the pytest/pytest-rust/wrapper workflows and both Containerfiles, restoring the install targets to their pre-extras form. The `granian` extra is dropped from the Containerfile install lists as well; no such extra has ever been defined. - All install documentation, the four affected ADRs and the agent-facing dependency-layout guidance revert to their extras-free wording. - Fixes a duplicate `cpex-secrets-detection` key in `[tool.uv.exclude-newer-package]` that left pyproject.toml invalid TOML. Also realigns two upstream session-factory tests that were left on the pre-migration SDK shape: they patched `streamablehttp_client` / `ClientSession` and reached for the retired `_cf_shutdown_event` / `_cf_owner_task` attributes, so they errored during setup instead of exercising the post-ready WARNING path and the timeout structured-logging fallback. Signed-off-by: Jonathan Springer <jps@s390x.com> * feat: upgrade MCP SDK from 2.0.0b2 to 2.0.0 final The 2.0.0 final release removes mcp.shared.session entirely (the b2 typing-only RequestResponder stub is gone) and changes Client(cache=) from a bool flag to CacheConfig | None, where None disables the response cache so message_handler passes through unwrapped. - pyproject/uv.lock: mcp/mcp-types >=2.0.0, cpex tracked from the 0.1.x-mcp-2.0.0 branch, drop stale exclude-newer pins - mcpgateway/utils/session_compat.py: new shim re-creating the RequestResponder typing stub so existing annotations and isinstance guards keep working (never instantiated, matching b2 semantics) - notification_service/upstream_session_registry: import from the shim; pass cache=None (was cache=False) in the pooled upstream session factory so the response cache does not wrap message_handler - tests: repoint RequestResponder imports to the shim, assert cache is None, update gateway refresh test to the mcp_proxy_client seam, refresh upstream session error category expectations, regenerate .secrets.baseline Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(services): align MCP httpx client factories on httpx2 types Signed-off-by: Jonathan Springer <jps@s390x.com> * test(live-gateway): migrate MCP e2e suites to SDK 2.0 final client API Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(compose): restore per-service REST auto-registration, drop strict 2026-07-28 probe Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(testing): drop stale Fast Time 2026 strict-probe references The fast_time_2026_server service was removed from docker-compose.yml in 5e54707, but its advertising and test wiring lingered: - scripts/register_mcp_servers.py: delete (unused anywhere in the repo) - Makefile: drop the :8887 banner and auto-registration echo from testing-up - tests/e2e/test_upstream_connect_mode_e2e.py: strip the strict-upstream behavior matrix, wire-capture proofs, and strict registration tests; keep the legacy connect-mode proofs against fast_time_server (:8888) Signed-off-by: Jonathan Springer <jps@s390x.com> * Restore Containerfile and Makefile to main state Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix(mcp-2.0): address SDK 2.0 review findings and live-stack build issues Review findings (PR IBM#5493): - transports/streamablehttp: ctx.meta is a TypedDict (RequestParamsMeta) in MCP 2.0 with no model_dump(); extract via dict() with a legacy-model fallback at the call_tool/get_prompt/read_resource sites - transports/streamablehttp: complete() now returns types.CompleteResult wrapping types.Completion so the wire shape carries the required 'completion' nesting; update test assertions to the corrected contract - utils/streamable_http_compat: build httpx2.AsyncClient (not v0 httpx) in the no-factory fallback, matching what the MCP 2.0 SDK transports expect; same-class fixes in resource_service factory and the upstream session registry (HttpxClientFactory alias + httpx/httpx2 exception matching) - services/http_client_service: add get_httpx2_timeout(); the v0 httpx.Timeout from get_http_timeout() is rejected by httpx2.AsyncClient, so point the gateway/tool/resource factory fallbacks at the new helper - transports/streamablehttp: capture server-emitted session IDs whenever stateful sessions are enabled, not only when multi-worker affinity is on, so the logical-owner claim fires on single-node deployments; unskip the affinity-disabled regression test Live stack: - Containerfile: add git to the builder stage (uv needs it to fetch the git-pinned cpex dependency) and split the monolithic venv RUN into discrete steps so build failures name the exact stage - docker-compose: repin cfex-mcp-fast-time-server to the current digest; the previous pin predates the 2026-07-28 caching directives and its tools/list responses failed ListToolsResult validation during federation Also rewrite test_invoke_tool_mcp_refuses_stored_env_filter to the v2 mcp_proxy_client mock pattern (pre-existing stale migration leftover). Signed-off-by: Jonathan Springer <jps@s390x.com> * fix: direct_proxy resource reads — SSE transport, final content shape, blob decoding Three defects left direct_proxy federated resource reads broken or silently corrupting content: 1. mcp_proxy_client was called without a transport argument, so SSE gateways were dialed with streamable-HTTP framing and the connect failed. Select the transport from the gateway's registered type. 2. The proxied read already returns final content, but execution fell through into the cache-mode pointer-resolution machinery, which assumes a DB-shaped object and crashed on content.id (AttributeError, masked to "" by the ingress). Build the final ResourceContent (with id and mime_type) in the branch and skip the machinery via a flag initialized at function scope alongside the other locals. 3. MCP transports carry blobs as base64 strings while ResourceContent.blob holds raw bytes; passing the string through meant the ingress re-encoded it, double-encoding every binary resource. Decode to raw bytes at the seam. Tests: TestReadResourceDirectProxy updated to assert the final content shape (ResourceContent) — the previously asserted intermediate shape is exactly what crashed the pointer-resolution machinery in production — and the blob test now uses valid base64 and asserts decoded raw bytes. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: let 2026-era requests through the session-less peeks so the ingress can serve modern sessions The streamable ingress wrapper peeks at session-less POSTs to give unknown-method probes a truthful -32601 (instead of the SDK's 'Missing session ID'). But 2026-era requests are self-describing and session-less BY DESIGN, and server/discover is not in the gateway's known-method list — so every modern client's discover probe was answered -32601 here, before the SDK's era routing ever ran, and the client silently fell back to the legacy handshake. The gateway could consume modern upstream sessions but never serve one. Mirror the SDK's own era-dispatch rule (MCP-Protocol-Version header present and not a handshake version => modern) and skip both session-less peeks for such requests: the SDK's modern classifier owns their validation and rejections. Also add server/discover to the known-method list. With this, a v2 client negotiates 2026-07-28 through the gateway (tools, prompts, and resources verified against both a 1.x-era SSE downstream and a 2026-era streamable downstream); legacy clients are unaffected (no header / handshake header keeps the old peek behavior, pinned by test). Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * refactor(tests): group era-aware peek-bypass tests under TestModernEraPeekBypass Groups the three behaviors pinning the modern-era peek bypass (modern header passes through, handshake header still peeked, server/discover is a known method) under one class with a shared docstring telling the era-boundary story — easier to locate and read as a unit in this large test module. No behavior change. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * rename era peek-bypass group to TestDualEra Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * feat: standing listen streams for modern servers feeding debounced refresh Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * refactor: module-level listener imports, tighter notify_list_changed docstring Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: prune stale catalog rows created via notification-driven refresh Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: answer unknown tool/prompt/resource requests in proper error shapes for both eras Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * feat: relay upstream tool progress to downstream callers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: surface tool invocation failures as isError results for both eras Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * feat: pass through 2026 MRTR elicitation for modern clients on modern servers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * port: move IBM#5934 MCP handshake probe onto the SDK 2.0 client The Test Connection endpoint's initialize-path probe was written against the SDK 1.x client API. Port it to SDK 2.0: _SniPinningTransport is now built on httpx2 (the stack the v2 client transports run on) with identical pinning logic, the local pinned-client factory returns httpx2.AsyncClient, _probe_session reads the v2 result field names, and the probe connects via the v2 sse_client / streamable_http_client. The raw server/discover probe path is unchanged. Tests updated to the v2 shapes; every pinning assertion (sni_hostname, pinned_host, TLS verify context) still asserts. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> Co-authored-by: Jonathan Springer <jps@s390x.com> * ci: run PR checks on pull requests targeting chore/mcp-sdk-v2 (IBM#6715) * ci: run PR checks on pull requests targeting chore/mcp-sdk-v2 Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: relay plugin-blocked tool calls as isError results on the /mcp path A PluginViolationError raised on the SDK-served /servers/{id}/mcp path fell through to the generic handler and was wrapped as a bare code-0 JSON-RPC error. Catch it in call_tool and return an isError CallToolResult, matching the contract the plugin e2e suites assert. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: port inherited unit tests to SDK v2 call shapes The tests still targeted the SDK 1.x API: the streamablehttp_client patch target and 3-tuple transport, camelCase result fields, session.call_tool mocks where v2 calls client.session.call_tool, and call_tool assertions without the v2 keyword arguments. Header assertions now inspect the http_client passed to the SDK instead of a headers kwarg. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: satisfy pylint and pre-commit on migration files Disable no-member on the mcp_app.request_context reads (the property is injected at runtime for the v1 handler adapters) and try-except-raise on the listener's CancelledError guard; strip a trailing space flagged by pre-commit. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: satisfy pre-commit and diff coverage for the plugin-block change Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: remove unwanted commentsn from tests Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * feat(compose): add 'ui' profile for contextforge-web-ui and document release verification (IBM#6716) * feat(compose): add 'ui' profile for contextforge-web-ui and document release verification Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix: lint issue Signed-off-by: Marek Dano <mk.dano@gmail.com> * docs: fix forbid-source-placeholders hook failure in ui.md Replace '(source: ...)' pattern on line 130 with an inline link so the pre-commit forbid-source-placeholders hook no longer rejects the file. Also refresh .secrets.baseline line numbers after the one-line change. Closes IBM#6716 Signed-off-by: Marek Dano <mk.dano@gmail.com> * chore: refresh .secrets.baseline line numbers after rebase onto chore/mcp-sdk-v2 Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix(compose): pin web_ui image to a released version and clarify BFF UI docs Address PR review feedback: pin WEB_UI_IMAGE to a specific released tag instead of `latest`, add a release-checklist step to bump and verify that version, and clarify that the built-in UI and the BFF-style contextforge-web-ui are both supported side by side. Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix(compose): pin web_ui image by digest and add ui profile config check Pin WEB_UI_IMAGE to tag@digest (docker-compose.yml, .env.example) so the supported ui profile's session/auth-handling BFF service can't be silently swapped by a retargeted registry tag. Document the resolve-and-bump-digest step in release-management.md IBM#6.4, add `make compose-ui-config-check` (docker compose --profile ui config --quiet) plus a matching CI workflow to catch profile/schema regressions on PRs, and drop the duplicated WEB_UI_IMAGE version literal from AGENTS.md so there's one source of truth. Addresses review feedback from vishu-bh and gcgoncalves on IBM#6716. Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix: refresh .secrets.baseline line numbers for docker-compose.yml The baseline committed in ef347b9 was generated before the last edit to docker-compose.yml's web_ui image comment block, leaving it 5 lines stale and failing detect-secrets in CI (Pre-commit Checks, run 34468285949) even though it passed locally against the pre-edit state. Signed-off-by: Marek Dano <mk.dano@gmail.com> --------- Signed-off-by: Marek Dano <mk.dano@gmail.com> * fix: read the remote span context through get_current_span in the request middleware Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * feat: enforce legacy-only MCP protocol negotiation for inbound and outbound paths Add MCP_INBOUND_PROTOCOL_MODE config to gate inbound client protocol versions at the middleware layer. When set to 'legacy', the gateway rejects 2026-07-28 with 400 and supported handshake versions, steering dual-era clients to retry with initialize. Change MCP_CLIENT_CONNECT_MODE default from 'auto' to 'legacy' so outbound connections use initialize-only by default. Gate the server/discover probe in test_gateway_handshake() behind the connect mode, and add a startup guard that force-disables modern_listener_service when legacy mode is active. Closes IBM#6767 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * docs: clarify inbound/outbound MCP client vs server roles in configuration reference Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: make pre-commit Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: .env.example legacy default Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: update stale tests and add coverage for legacy-mode guards - Fix test_default_protocol_version_applied to pin mode to auto (legacy case already covered by dedicated test) - Fix doctest in protocol_version.py to compare against LATEST_HANDSHAKE_VERSION matching the new legacy default - Add TestGatewayHandshakeLegacyDiscoverSkip verifying discover probe is skipped in legacy mode and attempted in auto mode - Add test_lifespan_legacy_mode_skips_modern_listeners verifying init_modern_listener_service is not called in legacy connect mode Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(tests): add configure_handshake_env fixture for discover-path handshake tests mcp_client_connect_mode defaults to "legacy", which skips the server/discover probe. Tests mocking ResilientHttpClient for discover-path behaviour were timing out (30 s health_check_timeout) while the SDK attempted initialize against a real host. - Add configure_handshake_env fixture (configure_allowlist + auto mode) so all discover-path handshake tests exercise the probe without repeating the pair - Revert 9 connectivity tests to configure_allowlist-only; mcp_client_connect_mode has no effect on check_mcp_server_connectivity - Add test_handshake_legacy_mode_skips_discover_uses_initialize: acceptance criterion for IBM#6767 — legacy mode bypasses the probe and reports negotiation_path='initialize' with no ResilientHttpClient calls Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: make pre-commit and tests Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: pin the conformance suite to cf-integration v0.5.1 Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * ci: align the pytest-rust diff-cover gate with pytest.yml at 90% Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: unpin mcp and mcp-types Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: surface upstream HTTP status when the mcp 2.x handshake fails Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: finish the SDK 2 port of the live E2E schema Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * ci: stop running PR checks on pull requests targeting chore/mcp-sdk-v2 Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> Signed-off-by: Marek Dano <mk.dano@gmail.com> Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> Co-authored-by: Jonathan Springer <jps@s390x.com> Co-authored-by: Marek Dano <marekdano@users.noreply.github.com> Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com>
…ests (IBM#6797) * fix(tests): allow real DNS passthrough for external IdP integration tests The session-scoped _deterministic_dns stub in tests/conftest.py resolves every non-localhost hostname to a hardcoded example.com address, which blackholes all Microsoft Graph / Entra ID egress from the in-process integration suite (tests/integration/test_entra_id_integration.py): every network-dependent test fails with httpx.ConnectTimeout, so the suite cannot pass under tests/ as shipped. - Add opt-in TESTS_DNS_PASSTHROUGH_HOSTS: comma-separated hosts resolved via real DNS while the stub stays active. Hermetic default unchanged. - Normalize IDNA-encoded bytes hostnames before matching: httpx/anyio resolve through anyio.getaddrinfo, which encodes hostnames to bytes; the previous string-only comparison never matched passthrough (or localhost) entries on that path. - Document TESTS_DNS_PASSTHROUGH_HOSTS and the required --with-integration flag in docs/docs/testing/entra-id-e2e.md. Verified with a live tenant: the Entra ID E2E suite passes 15/15 with the variables set; without them, DNS remains stubbed and the suite self-skips as before. Signed-off-by: Jonathan Springer <jps@s390x.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* fix(ui): handle object types in anyOf/oneOf schemas Admin UI form generator now correctly detects object types wrapped in anyOf/oneOf JSON Schema constructs. Previously, these were treated as plain text fields, causing dict parameters to be sent as strings and triggering validation errors. Updated both form generation and submission logic in tools.js to check for object types nested within anyOf/oneOf arrays. Fixes issue where MCP tools with nullable object parameters (using anyOf with object and null types) failed when invoked from Admin UI. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(admin-ui): fix anyOf/oneOf object detection in validate modal and submit paths Closes IBM#6420 - Extract isObjectSchemaType(prop) and allowsNullSchemaType(prop) helpers to eliminate duplicated 3-line detection expressions across call sites. - Fix validate-modal form generation (was only checking prop.type === 'text', rendering a single-line input for object/anyOf/oneOf params instead of a 4-row textarea). - Fix runToolValidation submit path (was still using prop.type === 'object', missing anyOf/oneOf unions entirely). - Fix null-arm regression: parsed === null is only rejected when the schema has no null arm (allowsNullSchemaType). Submitting 'null' for a schema with anyOf: [{type:'object'},{type:'null'}] no longer throws 'Value must be an object'. - Fix mixed-union regression: for anyOf/oneOf unions that include non-object arms (e.g. anyOf: [{type:'object'},{type:'string'}]), a SyntaxError from JSON.parse now falls back to passing the value as a plain scalar instead of aborting the submission. - Add unit tests covering all four paths in both runToolTest and runToolValidation: valid JSON object, null-arm allowed, mixed-union plain-string fallback, and invalid-JSON error on pure object schema. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(admin-ui): correct anyOf/oneOf object field parsing in runToolValidation and runToolTest Extract shared parseObjectFieldValue() helper to replace duplicated ~20-line try/catch blocks at the two call sites. Fix 1 (blocking): For anyOf:[{type:"object"},{type:"string"}], valid JSON scalars like "42" or "true" parse without SyntaxError, then hit the typeof !== "object" guard and throw "Value must be an object", blocking a previously valid submission. The helper now checks hasScalarUnionArm() before rejecting a successfully parsed scalar, and returns the raw string when the union has a matching scalar arm. Fix 2 (functionally-impacting): For anyOf:[{type:"object"},{type:"null"}], invalid JSON (e.g. {bad) throws SyntaxError; the old guard prop.type !== "object" is always true for union props (prop.type is undefined), so the raw invalid string passed through silently. The helper now gates the SyntaxError fallback on hasScalarUnionArm(), which returns false for object/null-only unions, so invalid JSON is correctly rejected with a user-visible error. Closes IBM#6420 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * tests: update assertions in test/unit/js/tools.test.js to align with the centralized error message formating from parseObjectFieldValue() Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
…and single-flight (IBM#6933) * fix: use shared HTTP client and add TTL cache for OpenAPI spec fetches Replace per-request get_isolated_http_client with the shared singleton get_http_client() in fetch_openapi_spec. The shared client already sets follow_redirects=False, so SSRF safety is preserved. Per-request timeout is passed via client.stream(..., timeout=timeout). Add a 60s in-process TTL cache keyed on spec_url to collapse concurrent fetches of the same OpenAPI spec to one upstream call per TTL window. Closes IBM#6835 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: harden OpenAPI spec fetch with DNS-pinning, bounded cache, cookie isolation, and single-flight - Replace validate_url with validate_url_for_connection_pinning + pin_url_to_resolved_ip to close DNS-rebinding SSRF window (CWE-918) - Bound spec cache to 64 entries with LRU eviction and active TTL expiry (CWE-400) - Use ephemeral httpx.AsyncClient per fetch instead of shared singleton to prevent cross-host cookie leakage (CWE-565/CWE-359) - Add per-URL asyncio.Lock for single-flight coordination so concurrent cache misses share one upstream fetch (CWE-400) - Update tests to match new isolated-client and connection-pinning patterns Closes IBM#6835 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: return deepcopy from spec cache and add TTL/concurrency tests - Return copy.deepcopy() from _spec_cache hits to prevent callers from mutating the shared cached dict. - Add test_cache_ttl_expiry_triggers_refetch covering the expiry branch. - Add test_concurrent_fetches_coalesce_to_single_request proving the single-flight lock produces exactly one upstream fetch. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: release single-flight lock on failed OpenAPI spec fetch A failed fetch writes no cache entry, so neither the TTL sweep nor the LRU eviction could drop its lock. A caller-controlled stream of failing spec URLs grew _spec_locks without bound. Pop the lock on the failure path. Return a deep copy on the cache-miss path so no caller can mutate the cached spec. Extract the duplicated TTL freshness check into _fresh_cached_copy(). Add debug logging for cache hit, cache miss, and single-flight coalescing. Add unit regressions for the lock leak, copy isolation, and LRU bound. Add black-box coverage for POST /v1/tools/generate-schemas-from-openapi against a running gateway. Closes IBM#6835 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: cache failed OpenAPI spec fetches for a short TTL A failed fetch wrote no cache entry, so every waiter queued behind the single-flight lock ran the failing fetch again. N waiters paid N times the timeout. Store the raised exception under a short negative TTL of 5 seconds. Waiters re-raise it from the cache instead of refetching. Cache entries now hold an expiry, so specs and failures share one bounded map with different lifetimes. Drop the manual lock cleanup on the failure path. A failure now writes a cache entry, so the TTL sweep and the LRU eviction reclaim its lock. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
* fix: preserve empty dict in structuredContent handling
Empty dicts are valid MCP responses but were being discarded by the 'or'
operator (which treats {} as falsy). This caused RuntimeError for tools
with outputSchema that return structuredContent: {}.
Changed to explicit None check to preserve empty dicts while still
falling back to snake_case variant when needed.
Fixes issue where Hubbl and other MCP servers fail with 'Tool has an
output schema but did not return structured content' error.
Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
* test: add regression test for empty dict in structuredContent
Add unit test to verify that structuredContent: {} is preserved through
the non-direct-proxy MCP tool invocation path. This prevents regression
of the bug where Python's 'or' operator treated empty dicts as falsy.
Also harmonize the same pattern in streamablehttp_transport.py to use
explicit None checks for consistency and future safety.
Addresses review findings for PR IBM#6182.
Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
* fix: preserve empty-dict structuredContent on call_tool success paths
Both success-path return sites in call_tool() gated on plain truthiness
(if structured:), which silently dropped an empty-dict structuredContent
and returned a bare list. The MCP SDK then raised:
RuntimeError: Tool has an output schema but did not return structured content
Fix both sites with an explicit is-None check so that {} is preserved:
- streamablehttp_transport.py primary path (~line 2126)
- streamablehttp_transport.py forwarded/session-affinity path (~line 1965)
The error-path branches (isError=True) already pass structured through
directly and are unaffected.
Add two regression tests covering call_tool() directly (not just
ToolService.invoke_tool) with structured_content={} for both the primary
and forwarded paths.
Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
* fixed the tests
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
* fixed pre-commit
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
* fixed tests
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
* fixed pre-commit
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
---------
Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
…le (IBM#6654) * feat: make MAX_HEADER_VALUE_LENGTH configurable via environment variable - Add max_header_value_length config field with 16KB default (up from hardcoded 4KB) - Remove hardcoded MAX_HEADER_VALUE_LENGTH constants from passthrough_headers.py and translate_header_utils.py - Update sanitize_header_value() functions to use configurable value with Mock-safe fallback - Add tests for Mock-safe fallback behavior in mocked test environments - Update .env.example with MAX_HEADER_VALUE_LENGTH documentation - Adjust test expectations for increased default limit Closes IBM#6653 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * test: add comprehensive coverage for Mock-safe fallback in header sanitization - Add unit tests for Mock object detection in sanitize_header_value() - Add tests for AttributeError and TypeError exception handling - Cover previously uncovered lines in translate_header_utils.py (lines 101-103) - Cover previously uncovered lines in passthrough_headers.py (lines 122-123) - Tests verify fallback to 16384 default when settings access fails Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * refactor: move max_header_value_length validation to config.py Move validation and default value handling for MAX_HEADER_VALUE_LENGTH from runtime fallback logic to Pydantic field validation in Settings. Changes: - Add field_validator in config.py to validate max_header_value_length is an integer, positive, and <= max_header_field_size_bytes - Change default from 16KB to 4KB to match original MAX_HEADER_VALUE_LENGTH constant (prevents regression) - Remove Mock-safe isinstance() checks from sanitize_header_value() in passthrough_headers.py and translate_header_utils.py - Update .env.example to reflect 4KB default - Remove obsolete Mock-safe fallback tests that tested the removed behavior - Update all test files to properly mock max_header_value_length = 4096 when patching settings objects - Fix unrelated ruff E712 error in gateway_service.py (use is_active instead of is_active == True) Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fix: address review feedback on max_header_value_length validator - Remove cross-field validator that rejected MAX_HEADER_VALUE_LENGTH=16384 and broke deployments with MAX_HEADER_FIELD_SIZE_BYTES < 4096; the two fields are independent — validator now only enforces positivity - Drop unreachable isinstance(v, int) dead code in the same validator - Restore test_large_header_values and test_multiple_concurrent_requests that were removed from tests/integration/test_translate_dynamic_env.py - Update test_config_validation to assert 16384 is valid and that max_header_field_size_bytes does not constrain max_header_value_length - Remove stray mock assignment line inserted mid-docstring in test_passthrough_headers.py - Restore is_active == True with noqa/pylint disable suppression in gateway_service.py transfer_gateway_ownership, consistent with all other is_active == True sites in the codebase Closes IBM#6653 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: address second-round review findings for max_header_value_length - Join config.py description string onto one line to satisfy ruff-format (CI 'Run pre-commit hooks' job was failing on this field). - Reword .env.example comment: drop false 'Must be <= MAX_HEADER_FIELD_SIZE_BYTES' and note that MAX_HEADER_FIELD_SIZE_BYTES and MAX_HEADER_TOTAL_SIZE_BYTES must also be raised for tokens near or above 8KB (HeaderSizeMiddleware enforces them first and returns 431 before sanitization runs). - Add TestVaultPluginLargeToken in test_vault_plugin.py: three tests that reproduce the issue IBM#6653 failure (4KB truncation → invalid JSON), confirm intact parse with a raised limit, and exercise the full VaultPlugin end-to-end path with an ~8KB Atlassian-style token. - Drop unreachable try/except (AttributeError, TypeError) fallback in three test files; read settings.max_header_value_length directly. Closes IBM#6653 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * update .secrets.baseline Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fixed tests and improved Changelog details Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
…ailures (IBM#6181) * fix: return MCP-compliant error responses for tool execution failures MCP protocol requires all tool results to include a 'content' field with an 'isError' flag for failures. Previously, runtime errors (server crashes, timeouts, network failures) raised exceptions that were converted to JSON-RPC errors, causing clients to fail with 'content: Field required' validation errors. Changes: - Modified invoke_tool_direct to return CallToolResult with isError=true instead of raising ToolInvocationError for connection/execution failures - Updated SSE transport timeout handler to return structured error response - Updated SSE transport exception handler to return structured error response - Updated StreamableHTTP transport timeout handler to return structured error - Updated StreamableHTTP transport exception handler to return structured error - Updated affected tests to expect structured error responses instead of exceptions All MCP execution paths now return properly structured CallToolResult objects even when the upstream server fails, ensuring protocol compliance and better error handling for MCP clients like langchain-mcp-adapters. Pre-execution errors (auth failures, validation errors, configuration issues) continue to raise exceptions as appropriate since they prevent execution from starting. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(mcp): address PR review feedback for protocol compliance - Update invoke_tool_direct docstring to reflect runtime error handling - Clarify inline comment about previous exception behavior - Simplify test assertions to use result.isError directly Addresses review feedback on PR fixing MCP protocol violation (IBM#6275) Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: address review findings for MCP tool error response structure - Fix IBM#1 (blocking/CWE-209): route invoke_tool_direct exception text through sanitize_exception_message before returning in CallToolResult; snapshot gateway.auth_query_params inside the ORM session so secrets embedded in gateway URLs are redacted before reaching the client. - Fix IBM#2 (blocking/retry_on_status): attach structured_content with status_code to CallToolResult in the SSE and StreamableHTTP inner BaseException handlers so the retry plugin can honour retry_on_status (e.g. 429, 503) without reaching the outer handler. - Fix IBM#3 (functionally-impacting): re-raise SystemExit, GeneratorExit, and KeyboardInterrupt before the BaseException catch-all in both SSE and StreamableHTTP handlers; mirrors the existing CancelledError guard (PR IBM#3202 review B7). - Should-fix: update connect_to_sse_server and connect_to_streamablehttp_server docstrings to reflect that these functions now return instead of raising for runtime failures. - Should-fix: extract _make_mcp_tool_error() static helper to eliminate five duplicate CallToolResult(isError=True) constructions; structured_ content is now threaded through a single site. - Tests: add test_invoke_tool_direct_error_is_sanitized to cover finding IBM#1; add test_sse/streamablehttp_http_status_error_sets_structured_ content_status_code to cover finding IBM#2. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: restore ToolTimeoutError raise on SSE/StreamableHTTP timeout paths and fix structuredContent kwarg - SSE and StreamableHTTP timeout handlers were returning self._make_mcp_tool_error() instead of raising ToolTimeoutError. The return value fell through into the post-process block, firing TOOL_POST_INVOKE a second time after _run_timeout_post_invoke() had already fired it once. Restore raise ToolTimeoutError() so the existing outer handler (line 7174) is the sole post-invoke callsite for the timeout path. - _make_mcp_tool_error() passed structured_content= (snake_case) to types.CallToolResult, whose field is structuredContent (camelCase) with no alias and extra='allow'. The value was stored as an unrecognised extra attribute, so _coerce_to_tool_result() dropped it and structured_content was always None on the direct-proxy path. Fix: use structuredContent= kwarg. - Update _run_timeout_post_invoke docstring: 'control returns to the caller which raises' was stale; the caller now raises itself. - Add tests: TestMcpSseTimeoutPostInvokeFiresOnce: asserts invoke_hook fires exactly once on an unretried SSE timeout. TestMcpStreamableHttpTimeoutPostInvokeFiresOnce: same for StreamableHTTP. TestMakeToolErrorStructuredContentSurvivesCoercion: asserts structuredContent from _make_mcp_tool_error survives _coerce_to_tool_result (direct-proxy regression guard). - Update two existing SSE/StreamableHTTP timeout tests that asserted the old broken return-instead-of-raise behaviour; they now expect ToolTimeoutError to be raised. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * tests: improve coverage Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: update tool_service tests for MCP SDK v2 client API Tests patched the removed tool_service.sse_client / tool_service.streamablehttp_client / tool_service.ClientSession symbols and asserted the camelCase structuredContent/isError attributes. Main's MCP SDK v2 migration (IBM#6868) replaced those with a single mcp_proxy_client context manager and renamed CallToolResult's fields to structured_content/is_error. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…BM#6967) The auto-created MCP tool for an A2A-wrapped agent validates its arguments against a schema that requires `query`, not `message` (see a2a_protocol.py's request-payload handling). The test sent {"message": "hi"}, so the gateway rejected the tools/call request at schema validation before it ever reached the upstream A2A backend, making the downstream vault-header assertion fail for an unrelated reason. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: harden server-scoped tool lookup Signed-off-by: lucarlig <luca.carlig@ibm.com> * fix: isolate tool lookup cache by invocation scope Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: update tool cache scope implementation - Modified tool_lookup_cache.py to handle scope-aware caching - Updated server_service.py and tool_service.py for cache integration - Added comprehensive test coverage for cache scoping behavior - Updated e2e tests to validate cache scope functionality Closes IBM#6959 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: update cache mocks to support get_negative async method - Add get_negative AsyncMock to cache mocks in test_mcp_apps_security.py - Add deprecated field to cache payload in test_tool_service_coverage.py - Simplify test_server_scoping_denies_unattached_tool to test DB path directly Fixes 4 failing tests related to tool cache scope implementation. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: cover stale scoped cache authorization Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * feat: implement server-scoped tool lookup cache with negative caching - Add server_id parameter to tool_lookup_cache get/set/invalidate methods - Implement caller-scoped negative caching for tool resolution failures - Update registry_cache to support server-scoped invalidation - Add comprehensive test coverage for cache scoping behavior - Update a2a_service and tool_service to use scoped cache operations This enables per-virtual-server tool caching and prevents cache pollution across different server contexts. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: address tool cache review findings Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: add missing file header to test_tool_lookup_cache_performance.py Pre-commit hook automatically added the required copyright header. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: validate negative cache key shape Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: lucarlig <luca.carlig@ibm.com> Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> Co-authored-by: lucarlig <luca.carlig@ibm.com>
* Update tool service to return MCP spec compliant list instead of dict Signed-off-by: Chris PC <chrispc@li-4dc2bf4c-325d-11b2-a85c-b68e8b1fc307.ibm.com> * Update unit test to reflect changes Signed-off-by: Chris PC <chrispc@li-4dc2bf4c-325d-11b2-a85c-b68e8b1fc307.ibm.com> * Fix coverage Signed-off-by: Chris PC <chrispc@li-4dc2bf4c-325d-11b2-a85c-b68e8b1fc307.ibm.com> * fix: limit non-JSON 2xx errors to tools with outputSchema A non-JSON 2xx body is now an error only when the tool declares an outputSchema, which is the case IBM#6199 describes. Tools without a schema keep passing the raw text through as a successful result. On the schema path the parse error is returned directly instead of being fed to extract_using_jq, where TextContent failed serialization and the truncation message was replaced by a generic jsonpath error. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: only read error key from dict JSON error bodies A valid JSON array on a 203/207 response raised AttributeError on result.get, which the parse-error handler then reported as a JSON parse error. A non-2xx array containing "error" raised TypeError on indexing. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: correct parse-error advice and bound JSON error echo The parse-error text no longer tells operators to raise REST_RESPONSE_TEXT_MAX_LENGTH. That limit applies only after parsing fails. A non-2xx JSON body without an error key now echoes the raw text, truncated to the limit. This removes an orjson TypeError on big integers and an unbounded echo. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: live gateway check for non-JSON body on outputSchema tool Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: run non-JSON outputSchema live check on its own gateway The previous test called the bare /mcp mount, which scopes lookups to the default_server_id sentinel and cannot find the tool. It also used an upstream that a docker gateway cannot reach. The test now starts its own gateway, adds the tool to a virtual server, and calls it through /servers/{id}/mcp/. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: assert the live upstream receives the GET Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: assert truncation bound and add large-valid-JSON boundary case The truncated-body test only checked for the parse-error text, so it would pass even without truncation. It now sets an explicit REST_RESPONSE_TEXT_MAX_LENGTH and asserts the echoed text stops at that exact limit. A new test proves the limit only bounds the echoed error text: a valid JSON body longer than the limit still parses and returns structured content. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: bound non-string error values to the response length limit A non-2xx or 203/207 JSON body with a non-string 'error' value is serialized fresh with orjson and was never bounded by _handle_json_parse_error. It could forward an arbitrarily large value, bypassing REST_RESPONSE_TEXT_MAX_LENGTH. The serialized text is now cut to the same limit. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Chris PC <chrispc@li-4dc2bf4c-325d-11b2-a85c-b68e8b1fc307.ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Chris PC <chrispc@li-4dc2bf4c-325d-11b2-a85c-b68e8b1fc307.ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* chore: release 1.0.11 version updates and dependency upgrades - Bump version 1.0.10 → 1.0.11 across all canonical locations - Update UBI base images to 10.2 across all Containerfiles - Upgrade rmcp 1.8.0 → 2.1.0 (fixes CVE high/medium: IBM#279, IBM#280, IBM#281) - Prune 3 stale cargo-vet exemptions (supply-chain clean) - Update Python lockfile (uv.lock): 60+ packages upgraded - Update requirements.txt files (docs, tests/load, tests/populate, data_analysis_server) - Update npm dependencies, regenerate package-lock.json (0 vulnerabilities) - Update CPEX exclude-newer dates to 2026-09-24 - Update plugin pyproject.toml gateway version pins (cedar, opa) Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: update OCI image version label to 1.0.11 and track in bumpversion org.opencontainers.image.version was missed by bump2version (not tracked). Add Containerfile entry to .bumpversion.cfg so it is auto-bumped in future releases. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(test): fix TestGatewayLifecycle ephemeral_gateway fixture 409 collision The ephemeral_gateway fixture registered at the same upstream URL as the active module-scoped streamable_http_gateway fixture. The DB-level constraint uq_team_owner_email_name_tool (team_id, owner_email, name) blocked the tool insert when both gateways were owned by admin@example.com and fetched identical fast-time-* tool names, surfacing as a 409 on every ephemeral_gateway call. Fix: add a class-scoped autouse fixture _displace_url_for_lifecycle_class to TestGatewayLifecycle that displaces any existing gateway at the upstream URL once before all tests in the class and restores it after. ephemeral_gateway now registers as public without managing displacement itself. Also fix test_duplicate_registration_conflicts to use the same visibility (public) as the ephemeral gateway so the duplicate check fires correctly. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(docs): update pymdownx.arithmatex format functions for pymdownx 12.x fence_mathjax_format and inline_mathjax_format were removed in newer pymdownx releases. Replace with the current public API: - arithmatex_fenced_format (was fence_mathjax_format) - arithmatex_inline_format (was inline_mathjax_format) Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * docs: add 1.0.11 changelog section Record the 57 pull requests merged between v1.0.10 and release/1.0.11, plus the four release-branch commits that carry no pull request. Keep every entry that was under Unreleased and append the new entries. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * docs: remove duplicate 1.0.10 entry from 1.0.11 changelog Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * update secretsbaseline Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com>
* feat: prefix federated resource names with gateway names Refs IBM#6621. Preserve resource URIs and local names; deterministic same-URI routing remains outside this change. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: separate resource name expansion from default live gate Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: integrate resource naming with consolidated live suite Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: address resource namespacing review feedback Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: make resource rename intent explicit and preserve bulk naming state Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * style: move new resource naming imports to module scope Keep existing inline imports outside this change and remove unrelated import-order churn. Validation: 834 unit tests passed, 1 existing skip. Staged hooks passed with the unchanged 99.5% docstring-coverage failure waived. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * docs: simplify resource rename comment Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: avoid SQLAlchemy case import collision Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: adapt resource namespacing to MCP SDK v2 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: update resource pagination for MCP SDK v2 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: ignore stale federated resource names Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: align federated rename expectations after rebase Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
* feat: add invitee invitations API Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: address invitation API review feedback Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh secrets baseline Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: disable redirect following in SSE gateway health-check client get_isolated_http_client defaults to follow_redirects=True. The SSE health-check call site omitted the override, making it the only client in gateway_service.py that followed redirects. Two security consequences: - SSRF: a user with gateways.create could register an SSE gateway whose URL 302-redirects the health check to an internal or cloud-metadata host, bypassing the SSRF validation that only checks the registered URL. - Credential forwarding: for authheaders gateways, httpx 0.28.1 preserves custom credential headers (e.g. X-API-Key) across redirects. The stored credential was forwarded to the attacker-controlled redirect destination. All sibling clients in gateway_service.py already pin follow_redirects=False (get_httpx_client_factory at line 4943, and lines 7396, 7563, 8131, 8540). Add the same flag to the SSE health-check client to match the documented SSRF-hardening invariant. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test: assert follow_redirects=False in SSE health-check client (GHSA-3frw-wjxx-2p6m) Add TestSSEHealthCheckNoRedirects to test_gateway_service_health_oauth.py. The test captures kwargs passed to get_isolated_http_client and asserts follow_redirects=False is present, so accidental reversion of the SSRF fix is caught immediately. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: add a same-origin and pinning gate for DCR outbound URLs Add a validation gate for URLs that the DCR flow sends requests to. The gate requires the issuer origin and pins the connection to the address resolved during validation. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: move DCR tests to the isolated HTTP client seam Point the DCR tests at the isolated client factory. Pinned requests need an isolated client, so the shared client seam no longer receives them. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: pin both AS metadata discovery requests Route both discovery requests through the outbound URL gate. Each request now dials the address resolved during validation, which closes the rebinding window on the issuer hostname. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: pin the DCR client registration request Route the registration POST through the outbound URL gate. Guard the error branch against a non-JSON upstream body, which previously escaped the handler. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: pin the DCR registration update and delete requests Route both management requests through the outbound URL gate. The registration access token no longer travels to a URL outside the issuer origin. Neither method has a caller in mcpgateway today. Both are public and both send a bearer token, so both are fixed now. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: clear the AS metadata cache around DCR integration tests The metadata cache is module-global and lives for the process, so a document cached by another test module leaked into these tests. Clear it before and after each test. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: raise DcrError on non-JSON upstream bodies and isolate the metadata cache Guard the remaining response.json() calls in dcr_service.py so each method keeps its documented DcrError contract; json.JSONDecodeError subclasses ValueError, not httpx.HTTPError, so it was escaping the surrounding except httpx.HTTPError handlers. Replace the unit test module's 15 inline _metadata_cache.clear() calls with one autouse fixture that also clears after each test, so the module stops leaking cached AS metadata into whatever test module runs next. Move the stray if __name__ == "__main__" guard to the end of the file, and bring the integration fixture's stub_dns_for_pinning docstring in line with its unit-test twin. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: cover the non-JSON-body and incomplete-pinning branches in dcr_service CI's diff-cover gate requires 90% coverage of changed lines and reported 84.1% on dcr_service.py. Every missing line was an except ValueError guard added to close a bare-ValueError escape, or an OIDC-fallback issuer-mismatch branch, or the defensive incomplete-pinning check in _prepare_pinned_request, none of which had a covering test. Add nine tests: one for the incomplete-pinning guard, three for the two discovery paths' non-JSON and issuer-mismatch branches, two for register_client's success and error non-JSON branches, and three for update_client_registration's success, JSON-error, and non-JSON-error branches. diff-cover now reports 100% on the changed lines. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: correct the dcr_service pinning docstrings Describe the origin check as the SSRF gate for metadata-document URLs and a consistency check for issuer-built URLs. Document that a URL policy refusal skips the OIDC fallback. Name the SNI hostname in the extensions attribute. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…6879) * fix(security): normalize /v1 paths for rate limit tier matching Normalize request paths in RateLimitMiddleware by stripping root path, API aliases, and /v1 prefixes before pattern matching. This prevents versioned endpoints from falling through to the LOW tier default. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * refactor(rate-limit): clean up stale /v1 branches and remove double normalization Remove dead version guards from tier patterns that were unreachable after _normalize_path_for_matching strips /v1 from every path before matching: - SESSION_REFRESH: ^(/v1)?/auth/refresh$ -> ^/auth/refresh$ - CRITICAL_INVITATION: ^/(?:v1/)?teams/... -> ^/teams/... Extract path normalization in dispatch so it runs once per request. Introduce _get_tier_for_normalized and _get_tier_name_for_normalized as the inner matching primitives; get_endpoint_tier and _get_tier_name delegate to them, preserving the public API. Update the forged-HMAC test to patch _get_tier_for_normalized, which is the method dispatch now calls directly. Add deny-path regression assertions to test_endpoint_tier_matches_v1_versioned_paths confirming that /v1/auth/email/login, /v1/tokens, /v1/oauth/token, and /v1/rbac/roles never return the 500 rpm LOW default. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…on startup (IBM#6881) * fix(security): read URL scheme allowlist at call time and scan DB on startup Closes IBM#6264. Gap 1: ALLOWED_URL_SCHEMES was frozen at class-definition time. Replace the cached class-attribute read in SecurityValidator.validate_url() with a direct read of settings.validation_allowed_url_schemes so the live LazySettingsWrapper is consulted on every call instead of the stale copy. Gap 2: pre-existing gateway, tool, and A2A agent records were never checked against the allowlist. Add _check_url_scheme_compliance() called from lifespan on startup. It queries all enabled records, logs a WARNING for each violation, and raises SystemExit when STRICT_SCHEME_ENFORCEMENT=true (default: false). Config: add strict_scheme_enforcement: bool = False. Docs: update .env.example with restart semantics and the new flag. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * test(security): add validation_allowed_url_schemes to settings mocks validate_url() now reads settings.validation_allowed_url_schemes at call time. Mocks and stub classes that patch mcpgateway.common.validators.settings must supply this attribute or the scheme check fails before the SSRF logic under test is reached. Add validation_allowed_url_schemes = ["http://", "https://", "ws://", "wss://"] to every affected mock: - tests/security/test_gateway_test_ssrf_protection.py (18 mock_settings blocks) - tests/unit/mcpgateway/routers/test_llmchat_router.py (StrictSSRFSettings) - tests/unit/mcpgateway/test_admin.py (StrictSSRFSettings) - tests/unit/mcpgateway/validation/test_validators_advanced.py (ssrf_settings mock and _settings() helper) Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: use top-level db imports in _check_url_scheme_compliance to resolve pylint reimport warnings Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * test: add unit tests for _check_url_scheme_compliance to satisfy diff-cover Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: address review findings for URL scheme compliance Sanitize URLs via sanitize_url_for_logging() before logging (CWE-532). Wrap DB scan in try/except so transient errors do not abort startup. Document restart requirement in .env.example and configuration.md. Patch get_settings() directly in test fixtures for call-time reads. Delete dead ALLOWED_URL_SCHEMES class attribute. Add test_disabled_records_excluded asserting filter call args. Reword internal comment as plain text. Add URL Scheme Allowlist section to ops docs. Closes IBM#6264 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(security): fail closed in strict mode when DB compliance scan errors _check_url_scheme_compliance() caught SQLAlchemyError and returned even when STRICT_SCHEME_ENFORCEMENT=true, letting the process start without verifying enabled records against the URL scheme allowlist (CWE-693). Raise SystemExit in strict mode when the database query fails. Retain warn-and-continue for the default non-strict path. Add tests for both strict and non-strict DB error paths, and cover the SIGHUP settings-cache-clear failure branch (signal_handlers.py:73-74). Closes IBM#6264 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * docs: clarify SIGHUP vs restart for URL scheme allowlist changes Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: resolve 5 review findings for issue IBM#6264 PR - Fix deny-path test URL to use /v1/mcp-servers/test prefix - Remove ponytail: agent-tooling comments from validators.py - SIGHUP: validate trial Settings before evicting cached instance - Per-table try/except in startup compliance scan to preserve partial results - Remove stale ALLOWED_URL_SCHEMES class attribute (zero in-tree readers) Closes IBM#6264 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(startup): harden URL scheme compliance scan Catch ValueError from sanitize_url_for_logging. urlparse raises on a malformed netloc, for example a stray bracket from an IPv6-style host. The error escaped the per-table SQLAlchemyError handler and aborted startup even when STRICT_SCHEME_ENFORCEMENT was false. Roll back the session in the per-table SQLAlchemyError handler. A ProgrammingError or DataError leaves the transaction aborted, so the next table query also failed and dropped real violations. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * chore: refresh secrets baseline after rebase Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
With stateful sessions and session affinity, tool calls on /servers/<id>/mcp are rerouted to /rpc. There, a PluginViolationError became a JSON-RPC error through the application exception handler, and a ToolInvocationError (for example "Please authorize <gateway> first") became -32603 "Internal error" with the message dropped. MCP clients do not show protocol errors to the model, so neither the policy reason nor the authorization hint reached it. Catch both in the tools/call branch of handle_rpc and return a result with isError, as call_tool in the streamable HTTP transport already does. Unknown tools, malformed requests and crashed plugins (PluginError) keep their protocol errors. Closes IBM#7055 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Alvin Gregory <alvingregory10k@gmail.com>
adgregory
requested review from
Lang-Akshay,
brian-hussey,
crivetimihai,
ja8zyjits and
msureshkumar88
as code owners
September 29, 2026 22:33
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🐛 Bug-fix PR
📌 Summary
Closes #7055.
With stateful sessions and session affinity enabled,
tools/callon/servers/<id>/mcpis rerouted to/rpc. On that path a failed or policy-blocked tool call came back as a JSON-RPC protocol error, which MCP clients do not show to the model: a plugin denial arrived as-32602 Plugin Violation: ...and aToolInvocationError(such as "Please authorize first") as-32603 Internal error, with the message dropped. In claude.ai the model saw only a generic failure.This PR reports those failures inside the result with
isError: true, the same waycall_toolin the streamable HTTP transport already does since #6868.🔁 Reproduction Steps
See #7055. In short:
USE_STATEFUL_SESSIONS=true,MCPGATEWAY_SESSION_AFFINITY_ENABLED=true, then atools/callwith anMcp-Session-Idthat atool_pre_invokeplugin denies, or that targets a per-user OAuth gateway the caller has not authorized.🐞 Root Cause
mcpgateway/main.py,handle_rpc,tools/callbranch:PluginViolationErroris re-raised toplugin_violation_exception_handler, which builds a JSON-RPC error for every method, andToolInvocationErrorfalls through to the generic handler (-32603 Internal error).💡 Fix Description
In the
tools/callbranch only, catchToolInvocationErrorandPluginViolationErroraround_execute_rpc_tools_calland return:{"content": [{"type": "text", "text": "<str(exc)>"}], "isError": true}The text is
str(exc), matching the streamable transport. Unchanged on purpose:PluginError(a plugin that crashed is not a tool-call outcome);/_internal/mcp/tools/callpath, which has its own handling and test (test_call_plugin_violation_returns_jsonrpc_format). Happy to align it in a follow-up if maintainers want both paths identical.📏 Reviewability
triage— new issue; will need triage🧪 Verification
Three new tests in
tests/unit/mcpgateway/test_rpc_tool_invocation.py(TestRPCToolsCallFailuresAreToolResults):test_plugin_violation_is_an_iserror_result: fails on unmodifiedmain(gets-32602 Plugin Violation), passes with the fix.test_tool_invocation_error_is_an_iserror_result: fails on unmodifiedmain(gets-32603 Internal error), passes with the fix.test_plugin_error_stays_a_protocol_error: guard that a crashed plugin keeps its JSON-RPC error; passes before and after.pytest tests/unit/mcpgateway/test_rpc_tool_invocation.py tests/unit/mcpgateway/test_main_error_handlers.pypytest -n auto tests/unit/mcpgateway/test_main.py tests/unit/mcpgateway/test_main_extended.py tests/unit/mcpgateway/test_mcp_method_registry.py tests/unit/mcpgateway/transportsblack --check,ruff checkmake testisort --check-onlyflagstest_rpc_tool_invocation.py, but for pre-existing imports thatmainalready has; this PR does not touch them.End to end: we run the
PluginViolationErrorhalf of this change as a patch on v1.0.10 behind claude.ai custom connectors. Policy denials and "system not connected, connect it at " denials now reach the model, which relays them to the person.📐 MCP Compliance
MCP specification, Tools, Error Handling: protocol errors for unknown tools and invalid arguments; tool execution errors in the result with
isError: true.✅ Checklist
black), lint clean (ruff) on changed files