Repository navigation
Conversation
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
IBM#5637) * Update fix_file_headers to have generic default AUTHOR, defined COPYRIGHT line, and new line fix, and add pre-commit-config.yaml to execute the fix headers check as part of the pre-commit checks so that the author can see the changes Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Update all headers so that checks pass Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Fixed all files with new fix-header rules Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Update .secrets.baseline after rebase Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Correcting straggling files after rebase Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Update CONTRIBUTING doc to be consistent Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Rebased off main and adjusted findings Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Correct Makefile to not refer to authors for fix-headers Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> --------- Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
(IBM#5664) * fix(resources): revert resource name uniqueness constraint from IBM#5158 Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore: update .secrets.baseline line numbers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore: update .secrets.baseline line numbers after code removals Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * test: update resource tests for name-uniqueness revert Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix cwe-117 improper output neutralization logs. Signed-off-by: AHERDERE <aherdere@ie.ibm.com> * cwe_117_improper_output_neutralization_logs Signed-off-by: AHERDERE <aherdere@ie.ibm.com> * fix cwe-117 improper output neutralization logs Signed-off-by: AHERDERE <aherdere@ie.ibm.com> * fix cwe-117 improper output neutralization logs Signed-off-by: AHERDERE <aherdere@ie.ibm.com> * fix cwe-117 improper output neutralization logs Signed-off-by: AHERDERE <aherdere@ie.ibm.com> --------- Signed-off-by: AHERDERE <aherdere@ie.ibm.com> Co-authored-by: AHERDERE <aherdere@ie.ibm.com>
…5669) Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
…rget (IBM#5634) * ci(pyright): add PR-scoped type check workflow and pyright-pr make target Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * ci(pyright): move pyright check into linting-full workflow Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * ci(pyright): move pyrightconfig.json into pyproject.toml [tool.pyright] Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…touched file to then become pyright compliant (IBM#5703) Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
…ion errors (IBM#5663) * fix(transport,middleware): remove manual Content-Length headers causing compression errors Fixes IBM#5457 When compression is enabled (COMPRESSION_ENABLED=true), manually setting Content-Length headers before the compression middleware runs causes "Too much data for declared Content-Length" errors because the declared size is uncompressed but the actual body is compressed. Changes: - Remove manual Content-Length from streamablehttp transport (4 locations) - Skip sanitization of compressed responses in ValidationMiddleware - Add 16 regression tests to prevent future occurrences The compression middleware now correctly sets Content-Length after compressing the response body, ensuring the header matches the actual transmitted byte count. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * addressed the comments Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed headers Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* fix: return 404 for missing server and gateway deletes Preserve 403 for denied ownership and direct scoped callers. Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed the pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed the comments Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
…#4821) * fix(auth): unify email extraction across all resource operations Replaced 23 instances of inconsistent email extraction patterns with canonical get_user_email() helper to ensure email-over-sub precedence across all resource CRUD operations. Problem: Resource update/delete operations used direct user.get('email') extraction which missed the 'sub' claim fallback, causing 403 Forbidden errors when tokens had {'sub': 'user@example.com'} structure. This created ownership check failures because create operations used get_user_email() (which extracts from 'sub') while update operations did not. Solution: Unified all resource operations to use get_user_email() with consistent email-over-sub precedence, ensuring the identity used for ownership checks matches across create/update/delete operations. Files changed: - mcpgateway/main.py: 18 instances (tools, resources, prompts, gateways, agents, servers) - mcpgateway/admin.py: 1 instance - mcpgateway/routers/runtime_admin_router.py: 4 instances - mcpgateway/routers/oauth_router.py: 2 instances - tests/unit/mcpgateway/test_email_extraction_consistency.py: New test suite Closes IBM#4800 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: address PR review findings for email extraction consistency - Remove 'unknown' sentinel filtering from oauth_router._extract_user_email() - Add proper 'unknown' rejection at enforcement layer (oauth_router.py:295) - Simplify admin.py email extraction guard (remove redundant if user check) - Update remaining user.get('email') patterns to use canonical get_user_email() - main.py: config export audit field - streamablehttp_transport.py: transport logging identity - resource_service.py: plugin context population - session_registry.py: session mapping registration - Fix test to use empty dict instead of dict with only is_admin field Ensures consistent email-over-sub precedence across entire codebase and proper identity validation before access checks. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(auth): address PR review findings for email extraction unification - Remove 'unknown' sentinel branch from oauth_router.py (fail closed with 401) - Revert admin.py null-guard to maintain isinstance pattern consistency - Move lazy import to module top-level in session_registry.py (hot path optimization) - Complete pattern sweep: fix remaining user.get('email') in llmchat_router, metadata_capture, main.py, admin.py - Add regression tests for sub-claim token scenario (exact IBM#4800 case) - Document email-over-sub precedence in 14 endpoint docstrings Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: resolve email extraction test failures in PR IBM#4800 - Filter out 'unknown' sentinel in _extract_user_email() to properly reject missing emails - Update test expectation for get_user_email() call count (2 calls: logging + authorization) Fixes test_enforce_gateway_access_requires_email and test_handle_rpc_logging_set_level_populates_email_when_missing Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * remove sentinel from _extract_user_email() function Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: address security review findings from PR IBM#4800 Finding IBM#1 (CWE-284): Fixed dead-code guard in resource_service.py - Changed 'isinstance(user, dict) and "email" in user' to 'isinstance(user, dict)' - Allows get_user_email() to handle sub-only tokens like {'sub': 'user@example.com'} Finding IBM#2 (CWE-287): Fixed sentinel mismatch in oauth_router.py - Removed _extract_user_email() helper (returned None) to use canonical get_user_email() (returns 'unknown') - Added explicit filtering: 'if requester_email == "unknown": requester_email = None' - Applied to both OAuth endpoints (initiate flow, fetch_tools_after_oauth) - Prevents 'unknown' sentinel from being used as token storage key - Removed obsolete test for deleted _extract_user_email() function Finding IBM#3 (CWE-770): Restored HeaderSizeMiddleware registration - Re-added import: 'from mcpgateway.middleware.header_size_middleware import HeaderSizeMiddleware' - Restored middleware registration with config-gated activation - Ensures RFC 6585 § 5 header-flood/OOM defense is active Finding IBM#4 (CWE-284): Fixed 'unknown' sentinel in session_registry.py - Added filtering before _register_session_mapping: 'if user_email == "unknown": user_email = None' - Prevents all unauthenticated sessions from being conflated under 'unknown' owner key - Maintains proper session isolation under MCP_REQUIRE_AUTH=false All changes maintain the canonical get_user_email() pattern from AGENTS.md while ensuring the 'unknown' sentinel is properly filtered at boundaries where None is expected for missing identity. Closes IBM#4800 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * chore: apply ruff formatting fixes - Format long strings in alembic migration - Format HeaderSizeMiddleware log message - Remove extra blank lines in oauth_router.py - Fix import order in metrics.py Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: restore unrelated main.py refactors clobbered by earlier rebase Restores safe_log_user() usage, dataplane_publisher_service naming and lifecycle, early rate-limiter Redis init, the MCPPathRewriteMiddleware _apply_mcp_rewrite helper, and metrics_aggregation_interval_seconds override handling to match origin/main. These were unintentionally reverted during a prior rebase of this branch (flagged in review as unrelated refactoring) and are unrelated to the email-extraction fix. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: preserve user_email for admin bypass in list_servers/list_gateways user_email was zeroed out on admin bypass, which per _apply_visibility_scope() semantics also hides the admin's own private servers/gateways, not just other users' rows. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: regenerate secrets baseline after rebase onto origin/main Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: correct copyright header format in email extraction test CI's correct-file-headers pre-commit hook requires the standard "Copyright contributors to the MCP-CONTEXT-FORGE project" line. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: restore exception chaining in remove_root, resources/read, prompts/get These handlers lost `from e` chaining and, in two cases, the `as e` capture itself, truncating stack traces. Also moved the remove_root success return back outside the try block. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: preserve admin identity in RPC dispatcher and normalize OAuth requester email Admin bypass in the list_gateways, resources/list, resources/read, prompts/list, and prompts/get RPC handlers cleared user_email alongside token_teams, causing BaseService._apply_visibility_scope() to treat admins as anonymous and hide their own private rows. Only token_teams should be relaxed for admin bypass; the email is still needed for owner matching. Also normalize the requester email in oauth_router._enforce_gateway_access() before comparing it against the already-normalized gateway owner, so mixed-case or whitespace-padded emails are not incorrectly denied access. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com>
…FK deletes (IBM#5659) * fix(api): use savepoint for personal team cascade delete and reorder FK deletes Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * code quality improvements Signed-off-by: Gabriel Costa <gabrielcg@proton.me> --------- Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* Update roadmap for 1.0.6 Signed-off-by: Jonathan Springer <jps@s390x.com> * uplifted the ubi10 base image tagg Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* bump-version 1.0.5 -> 1.0.6 / update ubi packages Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * Updated rust aand js Signed-off-by: cafalchio <mcafalchio@gmail.com> * Removed ui static from pyproject.toml Signed-off-by: cafalchio <mcafalchio@gmail.com> * Ran pre-commit Signed-off-by: cafalchio <mcafalchio@gmail.com> * Added Changelog Signed-off-by: cafalchio <mcafalchio@gmail.com> * Added local benchmark Signed-off-by: cafalchio <mcafalchio@gmail.com> * update changelog Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * Ran pre-commit Signed-off-by: cafalchio <mcafalchio@gmail.com> * Removed Beta Notice Signed-off-by: cafalchio <mcafalchio@gmail.com> * Fixed changelog and secrets folder Signed-off-by: cafalchio <mcafalchio@gmail.com> * fix changelog Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * Fixed the number of PR listed Signed-off-by: cafalchio <mcafalchio@gmail.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Signed-off-by: cafalchio <mcafalchio@gmail.com> Co-authored-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
) * ci: enable all arch build for workflow_dispatch manual trigger Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * ci: restrict ppc64le build to workflow_dispatch only Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * ci: fix null coercion, stale comments and wheels gate robustness Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…BM#5610) * feat(search): add versioned GET /v1/search unified search endpoint Expose unified cross-entity search at a stable, versioned, non-admin path so client-facing global search no longer depends on GET /admin/search, which is gated on admin.dashboard and slated for deprecation. - Extract the orchestration body of admin_unified_search into a shared, permission-agnostic helper perform_unified_search(); admin_unified_search becomes a thin admin.dashboard-gated wrapper (behavior unchanged). - Add mcpgateway/routers/search.py with GET /search (authentication only, no admin-panel gate). Per-entity RBAC and token scoping still enforce visibility inside the shared helper, and users remain gated on admin.user_management. - Register the router in the v1 assembly (always-on) so it serves at /v1/search and the legacy /search alias, independent of the admin API. - Add router tests covering unauthenticated access, invalid entity_types, users protection, team_id scoping, limit/limit_per_type, and response-shape parity with /admin/search. Related: IBM#5520 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * docs(search): trim verbose comments and docstrings Condense the added module/function docstrings and inline comments for the /v1/search endpoint while keeping the rationale (why it exists, security model, always-on registration). No behavior change; docstring coverage stays at 100%. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(search): add integration test for /v1/search endpoint Exercise GET /v1/search and the legacy /search alias through the full ASGI stack (routing, auth middleware, orchestration) on the real app, stubbing only the per-entity admin_search_* DB queries. Covers authenticated grouped response, legacy-prefix parity, limit_per_type forwarding, and 401 deny paths for both prefixes. Mirrors the pattern from tests/integration/test_resource_management.py. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(search): add real-data token-scoping regression for /v1/search Seed real teams, a non-admin member with a real global tools.read role, and public/team-private tools, then run the real PermissionService and admin_search_tools (nothing stubbed at the security boundary) to prove /v1/search enforces token/team visibility: - token scoped to team-B hides team-A's private tool while still returning public and team-B tools (deny with a positive control) - same user without token narrowing sees team-A too (confirms the deny was the token, not missing access) - non-admin requesting tools,users drops users but still returns tools Identity is injected; PermissionService, admin_search_tools, and DB rows are real. Full JWT-middleware validation is a separate concern. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(search): function-scope _auth_client to stop PermissionService patch leak The _auth_client fixture was module-scoped and wrapped its body in a patch(PermissionService=AlwaysGrant) context, so once the mocked wiring tests ran, the patch stayed active until module teardown and leaked into the later real-PermissionService scoping tests during a full-file run. Function scope tears the patch (and the fixture's dependency_overrides) down after each test, so the real-data tests exercise the genuine PermissionService. Verified by temporarily removing the seeded tools.read role: the real-data tests then fail on a full-file run (403 -> empty results), confirming real RBAC is now active. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix(search): register /search in _LEGACY_PREFIXES for deprecation headers The unified search router is always mounted, so its legacy unversioned alias (/search) must be listed in _LEGACY_PREFIXES to receive Sunset/Deprecation headers like every other legacy shim. Without it, the parity guard test_legacy_prefixes_match_assembled_routers fails. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(search): cover real per-entity denial not collapsing unified search Add a real-data test for the "one entity's denial must not fail the whole search" guarantee: user_b has tools.read but not servers.read, so a genuine 403 from admin_search_servers flows through _safe_entity_search and becomes empty results. Seed a public server matching the query so an empty servers list proves the denial rather than missing data, and assert tools still return (search did not collapse). Verified empirically by temporarily granting servers.read and confirming the seeded server then appears. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix(search): allow scoped tokens to reach /v1/search in TokenScopingMiddleware TokenScopingMiddleware default-denies unmapped paths, so /v1/search (normalized to /search) returned 403 for any token with non-wildcard permission scopes before the handler ran, making the endpoint unusable for granularly-scoped API tokens. Treat /search as authenticated-only at the middleware layer: it spans many entity types, each guarded by its own @require_permission, and per-entity denials are suppressed downstream by _safe_entity_search, so a single required permission cannot be chosen here without wrongly rejecting a valid token. Other token restrictions (team ownership, server_id, IP, time, usage limits) still apply. /admin/search keeps its admin.dashboard gate. Add regression tests: direct permission-map assertions plus a real signed scoped JWT through the live middleware (200 for /v1/search, 403 for an unmapped control path). Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix(search): token-scope team results in unified search (non-admin) admin_search_teams' non-admin path returned every DB membership and ignored token scope, so a token narrowed to team B still leaked team A's id, name, slug, description, visibility, and status through /v1/search (and /admin/teams/search). Tools already narrow via _get_user_team_ids; teams bypassed it. Filter the non-admin team results through the same token-aware helper (_get_user_team_ids, which honors token_teams/_cached_team_ids). The caller's own personal team stays visible; for unscoped tokens the intersection is a no-op. Uses getattr for is_personal so partial team mocks keep working. Add real-data regression tests through /v1/search: a token scoped to team B does not see team A, and without narrowing the same member sees both (confirming the deny is the token, not missing membership). Verified empirically by disabling the filter and observing team A leak. The admin branch (which intentionally returns all teams) is unchanged here and tracked separately. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(search): add real-JWT e2e for /v1/search Add an end-to-end test that drives a real signed JWT through the full auth and token-scoping middleware (no dependency overrides) against the real app on a temp DB — the path unit/integration tests don't exercise, mirroring the manual curl verification. Three minimal, high-value cases: - unauthenticated request -> 401 (real auth middleware) - non-admin JWT -> 200 and the member's tools.read role returns the seeded public tool (proves no admin needed, real RBAC, real data) - permission-scoped JWT (tools.read) -> 200 (token-scoping middleware allows /search) Verified empirically: disabling the /search allow makes the scoped case fail with 403, and the deny/happy contrast confirms auth is genuinely enforced. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * chore(search): align file headers with updated repo convention Apply the current .github/tools/fix_file_headers.py header format (Copyright contributors line, no Authors line) to the search source and test files, which the correct-file-headers pre-commit hook flags after the rebase onto main. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix(search): defer mcpgateway.admin import so it stays off the startup path The always-on search router imported mcpgateway.admin at module top level, so admin.py (and its module-level AdminStatsCache/AuthCache/MetricsBufferService/ Argon2/ContentSecurity instances, ~1.8s) loaded at startup even when MCPGATEWAY_ADMIN_API_ENABLED=false — contradicting the endpoint's own "no admin dependency" framing. Take team_id as a raw Query in the signature (instead of Depends(_validated_team_id_param)) and import _validated_team_id_param and perform_unified_search lazily inside the handler, so admin.py loads only on the first /v1/search call rather than at import. Verified: importing the search router no longer pulls mcpgateway.admin into sys.modules. Real HTTP behavior is unchanged (valid team_id passes, invalid -> 400); the unit test now uses a valid UUID since the handler runs the real validation. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * chore(search): drop unused logger and cover public-only token scope Address two non-blocking review notes: - Remove the unused module-level logger (and its logging import) from routers/search.py. - Add an integration test for the public-only scope (token_teams=[], what a token with no teams claim normalizes to): the caller, though a member of both seeded teams, sees only the public tool and not the team-private ones — the degenerate scope the explicit-team-list tests did not cover. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * chore(search): mark test hex UUID as allowlisted for detect-secrets The 32-char hex UUID literal used as a valid team_id in the router test trips the detect-secrets pre-commit hook (high-entropy hex). Suppress inline with Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> # pragma: allowlist secret per the repo convention for Python files. --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
* Add v1 catalog API Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: removing duplicate endpoint Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* Fixed lint Signed-off-by: cafalchio <mcafalchio@gmail.com> * Added skip check Signed-off-by: cafalchio <mcafalchio@gmail.com> * Corrected tags Signed-off-by: cafalchio <mcafalchio@gmail.com> * added correct version Signed-off-by: cafalchio <mcafalchio@gmail.com> * Addressed review issues Signed-off-by: cafalchio <mcafalchio@gmail.com> * Addressed plugin tmp path Signed-off-by: cafalchio <mcafalchio@gmail.com> * add override for ocp deployment so it can assign UID from its allowed range instead of hardcoded 10001 and add podSecurityContext for override to job-migration.yaml and fic for volumemount indentation in deployment-mcpgateway.yaml Signed-off-by: Claudia Gray <claudiag082004@gmail.com> * Ran pre-commit Signed-off-by: cafalchio <mcafalchio@gmail.com> * secrets Signed-off-by: cafalchio <mcafalchio@gmail.com> * Update secrets baseline Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Fixed test waiting for latest Signed-off-by: cafalchio <mcafalchio@gmail.com> * Fixed breaking test Signed-off-by: cafalchio <mcafalchio@gmail.com> --------- Signed-off-by: cafalchio <mcafalchio@gmail.com> Signed-off-by: Claudia Gray <claudiag082004@gmail.com> Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> Co-authored-by: Claudia Gray <claudiag082004@gmail.com> Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
IBM#5244) * fix: resolve authorization_code OAuth gateway offline issues Fix bug IBM#5237 where authorization_code OAuth gateways would become permanently offline due to health check assumptions and destructive token handling. Changes: - Decouple health checks from token ownership for authorization_code flow - Treat 401/403 responses as 'gateway reachable' instead of failures - Make client_secret decryption failures explicit (raise OAuthError) - Respect omit_resource flag during token refresh - Only delete tokens on invalid_grant errors (RFC 6749 §5.2) - Add gateway reactivation on 401/403 when previously unreachable - Update last_seen timestamp even on auth failures Tests: - Added test_oauth_authorization_code_health.py (8 tests) - Added test_gateway_service_401_reactivation.py (5 tests) - Updated existing OAuth tests to match new behavior - All 172 OAuth-related tests passing with no regressions Closes IBM#5237 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix pylint issues Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * strengthen authorization_code gateway health-check and token refresh logic Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed pre-commit/tests errors Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * wrapped streamable exception group and fixed client-description fail closed issue Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed headers Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed dead try/except issue Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * added documentation for oauth updates Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix secretsbaseline Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix pre-commit Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
) * feat(teams): support bulk member creation in TeamCreateRequest Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * fix(lint): suppress W0201 false positives on SQLAlchemy column assignments in _upsert_membership Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * fix: normalise seed email to lowercase before lookup and invitation creation EmailStr preserves the supplied casing (e.g. ALICE@example.com), but EmailUser records are stored in canonical lowercase. _normalize_member_seeds built _NormalizedSeed with the un-lowercased address, so: * EmailUser.email.in_([...]) missed existing users with mixed-case input * known_users.get(seed.email) returned None even when the user was found * the seed fell through to the invitation path, storing the uppercase address — making accept_invitation() always fail its exact email comparison against the lowercase authenticated identity. Fix: canonicalise the email to .strip().lower() at the point it enters _NormalizedSeed. The separate 'key' variable was made redundant and removed. Adds three regression tests in TestMixedCaseEmailNormalisation: - uppercase active user is correctly matched and added as a member - uppercase unknown address invitation is stored in lowercase - mixed-case duplicate pair is correctly rejected as a duplicate Closes IBM#5626 Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * docs+test: document POST /teams and add deny-path coverage for seeded creation Addresses blocking review findings from IBM#5626: docs/docs/manage/api-usage.md: - Add POST /teams section with TeamCreateRequest schema table - Document TeamMemberSeed fields including lowercase normalisation note - Document server-side seed routing rules (member vs invitation decision) - Document configuration flags: ALLOW_TEAM_CREATION, ALLOW_TEAM_INVITATIONS, MAX_TEAM_MEMBER_SEEDS, MAX_MEMBERS_PER_TEAM - Document TeamCreateResponse as a superset of TeamResponse - Document SeededMemberResponse and SeededInvitationResponse fields - Add curl examples for minimal create and create-with-members - Add full JSON response example with members_added and invitations_sent - Add error response table (400/403/422) with example detail messages tests/unit/mcpgateway/test_team_governance_flags.py (TestCreateTeamWithMembersDenyPaths): - test_unauthenticated_create_with_members_raises_401 - test_team_creation_disabled_with_members_raises_403 - test_invitations_disabled_with_unknown_seed_raises_400 Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * fix: secrets baseline Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * Address comments Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * fix: secrets baseline Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * refactor(teams): address review feedback on bulk member seeding Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * feat(teams): make MAX_TEAM_MEMBER_SEEDS a configurable setting The create-team seed ceiling was a hardcoded module constant but was documented in the config table alongside real environment variables, so operators could reasonably expect to tune it via env and be silently ignored. Promote it to a `Settings` field (default 500), source the schema constant from it, add it to `.env.example`, and list it in the configuration reference. The per-team `MAX_MEMBERS_PER_TEAM` limit still applies underneath. Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * fix: pre-commit check on CI Signed-off-by: Marek Dano <Marek.Dano@ibm.com> * fix: secrets baseline Signed-off-by: Marek Dano <Marek.Dano@ibm.com> --------- Signed-off-by: Marek Dano <Marek.Dano@ibm.com> Co-authored-by: Marek Dano <Marek.Dano@ibm.com>
* refactor: remove Granian HTTP server option Granian (Rust-based ASGI server) is removed as an HTTP server option. Gunicorn with Uvicorn workers is now the sole server. Code & build: - Delete run-granian.sh (420 lines) - Remove granian pip extra from pyproject.toml - Drop granian from Containerfile and infra/wheels/Containerfile - Hardcode gunicorn in docker-entrypoint.sh, remove HTTP_SERVER env var - Regenerate uv.lock (drops granian + setproctitle) Makefile: - Remove serve-granian, serve-granian-ssl, serve-granian-http2 targets - Remove container-run-granian deprecated targets - Remove CONTAINER_HTTP_SERVER variable and granian lock cleanup Docker Compose: - Remove HTTP_SERVER and all GRANIAN_* env vars from all compose files - docker-compose-performance.yml: flip default from granian to gunicorn Helm chart: - Remove HTTP_SERVER and GRANIAN_* from values.yaml, schema, and README Docs: - Supersede ADR-0025 (Granian HTTP Server) - Remove granian sections from tuning, scale, TLS, performance, and roadmap docs - Remove granian cross-references from ADR-0024 and ADR-0026 Signed-off-by: Jonathan Springer <jps@s390x.com> * Update .secrets.baseline Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
* chore: remove obsolete redis_fallback config references Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com> * test: address review on config-validation guard (anchor path, assert constructed) Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com> * fixed pre-commit Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com> * fixed pre-commit Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com> --------- Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com>
…BM#4636) * initial commit Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * down revivion Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: remove review documentation files Remove review-only documentation that should not be in the main codebase. Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * style(migration): remove unused sqlalchemy import in d21698ae4a19 Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * add pragma: allowlist secret Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: add coverage for IntegrityError race condition handling in role service Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat: add unique constraints to RBAC models for test database schema Add partial unique indexes to Role and UserRole ORM models via __table_args__ to ensure test databases created with create_all() have the same constraints as production databases created through Alembic migrations. This fixes integration test coverage by allowing IntegrityError race condition handling tests to trigger real constraint violations. Changes: - Role: Add uq_roles_name_scope_active index (one active role per name+scope) - UserRole: Add two indexes for NULL and non-NULL scope_id cases - Simplify test mocks to use natural IntegrityError from database Coverage: 100% on role_service.py changed lines (previously 62%) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * ci: add --with-integration flag to pytest workflows to enable integration tests Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: add unit tests for role service IntegrityError race condition handling Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * revert: remove --with-integration flag from workflow files Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: update alembic down_revision to w7x8y9z0a1b2 to prevent multiple heads - Changes down_revision from 9fb98535724d to w7x8y9z0a1b2 - Ensures proper migration chain after rebase with main Signed-off-by: Rakhi Dutta <rakhidutta@users.noreply.github.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(rbac): exclude expired assignments from unique constraints and use deterministic deduplication Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * rebase and down revision fix Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * soft_delete Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * pylint fix Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * added try and exception block Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test coverage Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(rbac): remap user_roles before role dedup, prefer unexpired assignments, fix W1203 logging, add migration tests Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(rbac): fix stale migration docstring/comments, drop unused pytest import, remove blank line Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(rbac): set down_revision to b6c7d8e9f0a1 to resolve duplicate alembic heads Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: fix file headers — add encoding/Location/Copyright/SPDX lines, remove stale Authors Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: refresh .secrets.baseline line numbers after rebase Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Signed-off-by: Rakhi Dutta <rakhidutta@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
… counter on recovery (IBM#4862) * fix: increase health check timeout from 5s to 30s and reset failure counter on recovery Root cause: 24x timeout mismatch between periodic health checks (5s) and Test button (120s) caused false 'Offline' status for slow gateways. Changes: - mcpgateway/config.py: health_check_timeout 5 -> 30, gateway_health_check_timeout 5.0 -> 30.0 - mcpgateway/services/gateway_service.py: reset _gateway_failure_counts on gateway recovery to prevent flickering accumulation - docs: update documented defaults in configuration.md, scale.md, tuning.md - .env.example: update commented-out examples Closes IBM#4785 Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com> * fixed documentation with proper default value Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* fix: harden JWT secret enforcement, remove hardcoded secrets, add startup validation (GHSA-8pcq-mx48-hjvj) Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * added init_secrets options for dev Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * restored changelog - addressed the comments Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: use upstream package-lock.json and .secrets.baseline Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * addressed comments and hardened helm files as well Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * added tests Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed tests Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed tests and audited secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * added setup make target to path .env file Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix error message on start up to give dev options Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * added updated meessage Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * added BASIC_AUTH to initsecrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed tests Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: quote compose :? guards, revert unreachable translate.py fallback, add logging import docker-compose.yml / docker-compose-embedded.yml: wrap all 20 unquoted \ environment entries in double-quotes. The error message contains 'Run: ' (colon-space) which YAML parses as a map key without quoting, causing 'unexpected type map' on docker compose up even when the secret is set. Quoting makes docker compose config exit 0. mcpgateway/translate.py: remove the SecurityConfigurationError try/except fallback added in the advisory fix. The fallback was permanently unreachable: LoggingService at line 150 already imports settings at module level unconditionally (logging_service.py:27 -> config.py), so validate_security_combinations fires before the guarded block. Reverted to the pre-existing ImportError-only fallback which correctly handles the only reachable failure mode (config package absent at import time). tests/unit/mcpgateway/test_translate.py: remove test_translate_security_config_error_fallback which exercised the now- deleted unreachable code path. tests/unit/mcpgateway/test_config.py: add missing 'import logging' so the four primary-worker heartbeat tests (inserted during rebase conflict resolution) resolve logging.WARNING without NameError. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * pre-commit spacing and added secrets baseline Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed pre-commit Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * removed e2e harness Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix detect secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix linting Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * Fix plugin integration tests Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
IBM#5809) The Docker Security Scan merge-queue gate (Grype, severity-cutoff high) fails on CVE-2026-6477 in libpq 16.11-3.el10_1, shipped by the runtime stage's ubi10/ubi-minimal:10.2-1784581369 base. Red Hat has since published libpq-16.14-1.el10_2 with the fix, but the pinned dated tag predates that erratum and Docker's layer cache keeps serving the vulnerable package on rebuilds even though `microdnf upgrade -y` already runs in that layer. Bump UBI_MINIMAL to the 2026-07-21 dated snapshot (ubi10/ubi-minimal:10.2-1784669047), which carries the patched libpq and busts the stale cache layer. UBI_BASE and NODEJS_IMAGE are untouched since libpq is only installed in the UBI_MINIMAL-based runtime stage. Verified locally: rebuilt the image with this pin, confirmed `rpm -q libpq` reports 16.14-1.el10_2, regenerated the SBOM (Syft v1.42.3) and re-ran Grype v0.110.0 with the repo's .grype.yaml and the same --only-fixed --fail-on high flags the CI job uses - clean, no findings. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* feat: add invitee invitations API Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: address invitation API review feedback Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh secrets baseline Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: disable redirect following in SSE gateway health-check client get_isolated_http_client defaults to follow_redirects=True. The SSE health-check call site omitted the override, making it the only client in gateway_service.py that followed redirects. Two security consequences: - SSRF: a user with gateways.create could register an SSE gateway whose URL 302-redirects the health check to an internal or cloud-metadata host, bypassing the SSRF validation that only checks the registered URL. - Credential forwarding: for authheaders gateways, httpx 0.28.1 preserves custom credential headers (e.g. X-API-Key) across redirects. The stored credential was forwarded to the attacker-controlled redirect destination. All sibling clients in gateway_service.py already pin follow_redirects=False (get_httpx_client_factory at line 4943, and lines 7396, 7563, 8131, 8540). Add the same flag to the SSE health-check client to match the documented SSRF-hardening invariant. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test: assert follow_redirects=False in SSE health-check client (GHSA-3frw-wjxx-2p6m) Add TestSSEHealthCheckNoRedirects to test_gateway_service_health_oauth.py. The test captures kwargs passed to get_isolated_http_client and asserts follow_redirects=False is present, so accidental reversion of the SSRF fix is caught immediately. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: add a same-origin and pinning gate for DCR outbound URLs Add a validation gate for URLs that the DCR flow sends requests to. The gate requires the issuer origin and pins the connection to the address resolved during validation. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: move DCR tests to the isolated HTTP client seam Point the DCR tests at the isolated client factory. Pinned requests need an isolated client, so the shared client seam no longer receives them. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: pin both AS metadata discovery requests Route both discovery requests through the outbound URL gate. Each request now dials the address resolved during validation, which closes the rebinding window on the issuer hostname. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: pin the DCR client registration request Route the registration POST through the outbound URL gate. Guard the error branch against a non-JSON upstream body, which previously escaped the handler. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: pin the DCR registration update and delete requests Route both management requests through the outbound URL gate. The registration access token no longer travels to a URL outside the issuer origin. Neither method has a caller in mcpgateway today. Both are public and both send a bearer token, so both are fixed now. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: clear the AS metadata cache around DCR integration tests The metadata cache is module-global and lives for the process, so a document cached by another test module leaked into these tests. Clear it before and after each test. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: raise DcrError on non-JSON upstream bodies and isolate the metadata cache Guard the remaining response.json() calls in dcr_service.py so each method keeps its documented DcrError contract; json.JSONDecodeError subclasses ValueError, not httpx.HTTPError, so it was escaping the surrounding except httpx.HTTPError handlers. Replace the unit test module's 15 inline _metadata_cache.clear() calls with one autouse fixture that also clears after each test, so the module stops leaking cached AS metadata into whatever test module runs next. Move the stray if __name__ == "__main__" guard to the end of the file, and bring the integration fixture's stub_dns_for_pinning docstring in line with its unit-test twin. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: cover the non-JSON-body and incomplete-pinning branches in dcr_service CI's diff-cover gate requires 90% coverage of changed lines and reported 84.1% on dcr_service.py. Every missing line was an except ValueError guard added to close a bare-ValueError escape, or an OIDC-fallback issuer-mismatch branch, or the defensive incomplete-pinning check in _prepare_pinned_request, none of which had a covering test. Add nine tests: one for the incomplete-pinning guard, three for the two discovery paths' non-JSON and issuer-mismatch branches, two for register_client's success and error non-JSON branches, and three for update_client_registration's success, JSON-error, and non-JSON-error branches. diff-cover now reports 100% on the changed lines. Refs IBM#478 Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: correct the dcr_service pinning docstrings Describe the origin check as the SSRF gate for metadata-document URLs and a consistency check for issuer-built URLs. Document that a URL policy refusal skips the OIDC fallback. Name the SNI hostname in the extensions attribute. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…6879) * fix(security): normalize /v1 paths for rate limit tier matching Normalize request paths in RateLimitMiddleware by stripping root path, API aliases, and /v1 prefixes before pattern matching. This prevents versioned endpoints from falling through to the LOW tier default. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * refactor(rate-limit): clean up stale /v1 branches and remove double normalization Remove dead version guards from tier patterns that were unreachable after _normalize_path_for_matching strips /v1 from every path before matching: - SESSION_REFRESH: ^(/v1)?/auth/refresh$ -> ^/auth/refresh$ - CRITICAL_INVITATION: ^/(?:v1/)?teams/... -> ^/teams/... Extract path normalization in dispatch so it runs once per request. Introduce _get_tier_for_normalized and _get_tier_name_for_normalized as the inner matching primitives; get_endpoint_tier and _get_tier_name delegate to them, preserving the public API. Update the forged-HMAC test to patch _get_tier_for_normalized, which is the method dispatch now calls directly. Add deny-path regression assertions to test_endpoint_tier_matches_v1_versioned_paths confirming that /v1/auth/email/login, /v1/tokens, /v1/oauth/token, and /v1/rbac/roles never return the 500 rpm LOW default. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…on startup (IBM#6881) * fix(security): read URL scheme allowlist at call time and scan DB on startup Closes IBM#6264. Gap 1: ALLOWED_URL_SCHEMES was frozen at class-definition time. Replace the cached class-attribute read in SecurityValidator.validate_url() with a direct read of settings.validation_allowed_url_schemes so the live LazySettingsWrapper is consulted on every call instead of the stale copy. Gap 2: pre-existing gateway, tool, and A2A agent records were never checked against the allowlist. Add _check_url_scheme_compliance() called from lifespan on startup. It queries all enabled records, logs a WARNING for each violation, and raises SystemExit when STRICT_SCHEME_ENFORCEMENT=true (default: false). Config: add strict_scheme_enforcement: bool = False. Docs: update .env.example with restart semantics and the new flag. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * test(security): add validation_allowed_url_schemes to settings mocks validate_url() now reads settings.validation_allowed_url_schemes at call time. Mocks and stub classes that patch mcpgateway.common.validators.settings must supply this attribute or the scheme check fails before the SSRF logic under test is reached. Add validation_allowed_url_schemes = ["http://", "https://", "ws://", "wss://"] to every affected mock: - tests/security/test_gateway_test_ssrf_protection.py (18 mock_settings blocks) - tests/unit/mcpgateway/routers/test_llmchat_router.py (StrictSSRFSettings) - tests/unit/mcpgateway/test_admin.py (StrictSSRFSettings) - tests/unit/mcpgateway/validation/test_validators_advanced.py (ssrf_settings mock and _settings() helper) Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: use top-level db imports in _check_url_scheme_compliance to resolve pylint reimport warnings Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * test: add unit tests for _check_url_scheme_compliance to satisfy diff-cover Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: address review findings for URL scheme compliance Sanitize URLs via sanitize_url_for_logging() before logging (CWE-532). Wrap DB scan in try/except so transient errors do not abort startup. Document restart requirement in .env.example and configuration.md. Patch get_settings() directly in test fixtures for call-time reads. Delete dead ALLOWED_URL_SCHEMES class attribute. Add test_disabled_records_excluded asserting filter call args. Reword internal comment as plain text. Add URL Scheme Allowlist section to ops docs. Closes IBM#6264 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(security): fail closed in strict mode when DB compliance scan errors _check_url_scheme_compliance() caught SQLAlchemyError and returned even when STRICT_SCHEME_ENFORCEMENT=true, letting the process start without verifying enabled records against the URL scheme allowlist (CWE-693). Raise SystemExit in strict mode when the database query fails. Retain warn-and-continue for the default non-strict path. Add tests for both strict and non-strict DB error paths, and cover the SIGHUP settings-cache-clear failure branch (signal_handlers.py:73-74). Closes IBM#6264 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * docs: clarify SIGHUP vs restart for URL scheme allowlist changes Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix: resolve 5 review findings for issue IBM#6264 PR - Fix deny-path test URL to use /v1/mcp-servers/test prefix - Remove ponytail: agent-tooling comments from validators.py - SIGHUP: validate trial Settings before evicting cached instance - Per-table try/except in startup compliance scan to preserve partial results - Remove stale ALLOWED_URL_SCHEMES class attribute (zero in-tree readers) Closes IBM#6264 Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(startup): harden URL scheme compliance scan Catch ValueError from sanitize_url_for_logging. urlparse raises on a malformed netloc, for example a stray bracket from an IPv6-style host. The error escaped the per-table SQLAlchemyError handler and aborted startup even when STRICT_SCHEME_ENFORCEMENT was false. Roll back the session in the per-table SQLAlchemyError handler. A ProgrammingError or DataError leaves the transaction aborted, so the next table query also failed and dropped real violations. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * chore: refresh secrets baseline after rebase Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
… fork (IBM#6688) WORKER_ID was captured once at module import, so under gunicorn's --preload every forked worker inherited the master's id and subscribed to the same Redis pub/sub channel, turning point-to-point forwarding into a per-container broadcast. Replace the constant with get_worker_id(), which derives the id from the live PID at call time, update all call sites (main.py, streamablehttp_transport.py), and drop the now-redundant post_fork rebind in gunicorn.config.py. Closes IBM#6687 Signed-off-by: Jonathan Springer <jps@s390x.com>
* initial fix to prevent conflicting tool name register Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * Fix visibility projection in update_gateway(). Add missing Admin, catalog, and import collision handling. Add focused service and endpoint tests, then live gateway test. Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * improved the coverage Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed review comments Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed precommit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed precommit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed healthcheck recovery and federated too update Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed lint issue Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * improved test coverage and health-check revovery Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
IBM#6875) * fix: enforce MCP Streamable HTTP Origin validation to prevent DNS rebinding Implements the MCP 2025-11-25 §transport-security requirement: servers MUST reject a present-but-invalid Origin header with HTTP 403. Changes: - Add _check_mcp_origin() and _check_mcp_host() in streamablehttp_transport.py. - Apply to all HTTP methods (POST, GET, DELETE) and both /mcp and /servers/{id}/mcp routes via MCPOriginHostGate in main.py. - Add mcp_allowed_origins and mcp_allowed_hosts settings (Settings). Both default to empty set so enforcement is opt-in (backward-compatible). Parsed from MCP_ALLOWED_ORIGINS / MCP_ALLOWED_HOSTS env vars via JSON array or CSV string. - Document both env vars in .env.example near ALLOWED_ORIGINS. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test: bring diff coverage for origin validation to 100% Add tests for _parse_mcp_origin_sets (all string/collection/fallback branches) and SessionManagerWrapper SDK security-settings construction to satisfy the diff-cover ≥90% gate. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(test): address PR 6875 review findings - Remove unused 'patch' import (Ruff F401) - Drop unused 'wrapper' variable in SDK security settings test (Ruff F841) - Add test: security_settings=None passed to SDK when mcp_allowed_hosts is empty - Add deny-path test: custom origin gate enforces 403 when mcp_allowed_hosts is unset and mcp_allowed_origins is configured Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: address PR 6875 review findings - Wire _check_mcp_origin() into RustMCPRuntimeProxy.handle_streamable_http() before the Rust forward; loopback-exempted; returns HTTP 403 with deprecation headers. Closes blocking finding IBM#1. - Omit allowed_origins from TransportSecuritySettings when mcp_allowed_origins is empty. The SDK treats an empty list as reject-all; Origin enforcement in that case falls solely to _check_mcp_origin(). Closes blocking finding IBM#2. - Add request-level split-config tests (mcp_allowed_hosts set, mcp_allowed_origins empty) and three Rust-proxy Origin deny-path tests. Closes functionally-impacting finding IBM#3. - Document in .env.example that MCP_ALLOWED_ORIGINS covers Rust-mode traffic via the Python proxy layer; the Rust crate has no independent validation. Closes functionally-impacting finding IBM#4. - Comment at _check_mcp_origin call site notes exact-string vs host:* wildcard-port mismatch with the SDK gate (minor IBM#5). - Remove duplicate loopback-forward detection; is_internally_forwarded derived once, reused for Origin guard and session-affinity (minor IBM#6). - Extract _parse_origin_set() static method; _parse_allowed_origins and _parse_mcp_origin_sets both delegate to it (minor IBM#7). Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: address round-2 PR 6875 review findings - Drop SDK TransportSecuritySettings entirely; implement gateway-owned _check_mcp_host() alongside _check_mcp_origin() using the same opt-in exact-match pattern. Both Python and Rust paths now use the gateway gate exclusively — no SDK allowed_origins/allowed_hosts coupling. Closes blocking finding IBM#1 (round 2). - Gate the loopback + x-forwarded-internally bypass on mcpgateway_session_affinity_enabled. When affinity is disabled (the default) the header provides no bypass, closing the post-DNS- rebind spoofing vector. Closes blocking finding IBM#2 (round 2). - Extract _parse_mcp_scope_headers() shared helper used by both streamablehttp_transport and rust_mcp_runtime_proxy, eliminating the duplicated header-extraction block. Closes minor finding IBM#5 (round 2). - Add sanitize_for_log() to the Rust proxy rejected-Origin/Host log lines (CWE-117). Closes functionally-impacting finding IBM#4 (round 2). - Update .env.example MCP_ALLOWED_HOSTS block: document exact-match semantics, gateway-gate coverage for both transport modes, remove the now-obsolete SDK HTTP 421 warning. Closes functionally-impacting finding IBM#3 (round 2). - Rename allow_empty_string -> coerce_non_iterable_to_empty in _parse_origin_set to accurately describe what the flag controls. Closes minor finding IBM#6 (round 2). - Replace stale SDK-wiring tests with _check_mcp_host unit tests, Host-gate integration tests, and affinity-bypass tests (enabled/disabled). Patch forwarded-path tests in test_streamablehttp_transport.py to set mcpgateway_session_affinity_enabled=True — these tests exercise the affinity routing path which requires affinity to be on. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: address round-3 PR 6875 review findings Blocking IBM#1/IBM#2 — move Origin/Host gate to the public /mcp mount level. Introduce MCPOriginHostGate (streamablehttp_transport.py) as an ASGI wrapper mounted at /mcp in main.py. The gate fires before any ingress dispatch (Python, rust-internal, rust-public), removing the per-transport duplication and closing the gap where InternalTrustedMCPTransportBridge and RustMCPPublicProxyApp had no gate. The /_internal/mcp/transport bridge is mounted separately and intentionally bypasses the gate (trusted Rust sidecar traffic only). Remove gate from handle_streamable_http and RustMCPRuntimeProxy.handle_streamable_http. Suggestion IBM#8 — Origin/Host header extraction now uses hdrs.get('origin') (is None semantics), not the 'or None' coercion that silently treated an empty-string header as absent. Add two unit tests per field. Suggestion IBM#9 — Drop the _parse_mcp_origin_sets validator. Extend the existing _parse_allowed_origins @field_validator to cover all three fields (allowed_origins, mcp_allowed_origins, mcp_allowed_hosts). Update the test_config.py calls from _parse_mcp_origin_sets to _parse_allowed_origins. FI IBM#3 — Fix the mcp_allowed_hosts comment in config.py: remove the stale SDK-level and host:* wildcard references. FI IBM#4 — Remove stale 'Closes #<issue>' placeholder and the mistaken 'Closes IBM#421' (bandit zero-finding issue) from earlier commit messages. FI IBM#5 — Add tests/live_gateway/mcp/test_origin_host_enforcement.py: three black-box HTTP tests against a running gateway (unlisted Origin returns 403, allowlisted Origin is accepted, missing Origin is always accepted). Tests self-skip when MCP_ALLOWED_ORIGINS is not configured. FI IBM#7 — Update .env.example: clarify that enforcement covers all /mcp ingress modes and that /_internal/mcp/transport is exempt. Fix stale host:* wildcard note. Add commented MCP_ALLOWED_ORIGINS/MCP_ALLOWED_HOSTS entries to charts/mcp-stack/values.yaml. Add [Unreleased] Security entry to CHANGELOG.md. Update test_mcp_origin_validation.py: rewrite integration tests to call MCPOriginHostGate directly instead of handle_streamable_http. Update test_rust_mcp_runtime_proxy.py: replace stale 'proxy rejects Origin' tests with a test that confirms the proxy no longer gates on Origin. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix pre-commit Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * docs: clarify MCP_ALLOWED_ORIGINS/HOSTS nginx-direct-to-Rust bypass Round-3 blocking IBM#2: .env.example claimed enforcement covers 'all runtime modes' including rust-public. That claim is true when Python is the public ingress (MCPOriginHostGate wraps /mcp before dispatch). It is false when a reverse proxy routes /mcp directly to the Rust listener on port 8787 — Python is bypassed entirely in that path. Add a NOTE to both MCP_ALLOWED_ORIGINS and MCP_ALLOWED_HOSTS blocks: state that the nginx-direct-to-8787 path is not covered and that operators must configure Origin/Host validation at the proxy layer. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: exempt loopback Host values from MCP_ALLOWED_HOSTS enforcement Round-4 blocking IBM#1: test_server_mcp_handshake dispatches in-process via internal_loopback_base_url() which produces Host: 127.0.0.1:{PORT}. With MCP_ALLOWED_HOSTS set, MCPOriginHostGate rejected the request with 403 before any session or RBAC logic ran. _check_mcp_host() now strips the port and unconditionally accepts the loopback hostname literals 127.0.0.1, localhost, [::1], and ::1 regardless of the allowlist. A DNS-rebound page carries the attacker hostname in Host, never a loopback literal, so this does not weaken the DNS-rebinding protection. Add six regression tests in TestCheckMcpHost (IPv4 with/without port, localhost, IPv6 bracketed, non-loopback still rejected) and one MCPOriginHostGate integration test that reproduces the handshake self-call path (Host: 127.0.0.1:4444 with a non-loopback allowlist). Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * update secrets baseline Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…ersions) (IBM#5811) * feat: add container image version bumper script with bats tests Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(makefile): add bats and container-bump-image-versions targets Signed-off-by: Jonathan Springer <jps@s390x.com> * fix: derive Pyxis repo path from pin; portable sed in bats test Remove the hard-coded ARG-to-repository map (repo_for_arg) that could drift from the actual image path in the Containerfile; the Pyxis repository is now derived from each pin's image reference, with a fail-fast guard for pins outside registry.access.redhat.com. Replace the macOS-only 'sed -i ""' in the bats test with the portable sed -i.bak pattern so the suite runs on GNU sed (Ubuntu CI). Add regression tests for the derived-repository lookup and the foreign-registry refusal. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix: clean sed backups via EXIT trap; deterministic apply order Install an EXIT trap removing Containerfile .bak files so a sed failure mid-apply (die path) cannot strand backup files on disk; the redundant per-file rm in the loop is folded into the trap. Iterate the apply phase over MANAGED_ARGS in declaration order instead of undefined associative-array key order, matching the plan and report loops and removing a latent ordering hazard. Add a regression test asserting no .bak files remain after an update. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix: validate wheels UBI_MINIMAL pin before applying bump The plan phase only grepped the root Containerfile for ARG lines; the wheels Containerfile's UBI_MINIMAL pin was never validated. sed exits 0 on a missing match, so a missing wheels pin would print 'updated' and exit 0 while the pins silently diverged — breaking the atomic-update invariant this tool exists to enforce. A divergent pin value was silently converged to the root pin's image path. Both states now abort during planning with a reconcile-first error and zero files modified. Locked with two regression tests. Signed-off-by: Jonathan Springer <jps@s390x.com> * chore: regenerate .secrets.baseline via detect-secrets scan Signed-off-by: Jonathan Springer <jps@s390x.com> * Bump container images UBI_BASE: 10.2-1790075508 -> 10.2-1790841271 NODEJS_IMAGE: 10.2-1790228697 -> 10.2-1790817011 UBI_MINIMAL: 10.2-1790075626 -> 10.2-1790753097 Signed-off-by: Jonathan Springer <jps@s390x.com> * fix: extend image bumper to cover infra/nginx/Dockerfile - Convert bare FROM in infra/nginx/Dockerfile to ARG NGINX_IMAGE + FROM so the script can manage the pin via the same sed pattern used for the root Containerfile. - Add NGINX_IMAGE to MANAGED_ARGS / MANAGED_FILES; the script now manages four pins across three files. - Replace declare -A associative arrays with parallel indexed arrays (bash 3.2 compatible); the script previously failed on macOS system bash and any other bash < 4 environment. - Bump page_size from 100 to 500 to avoid missing the current pin for high-churn images with more than 100 tag-update events. - Add NGINX_DOCKERFILE_PATH test hook and nginx fixture to the bats suite; extend all 15 existing integration tests with nginx assertions; add two new tests (NGINX_IMAGE bump, missing NGINX_IMAGE pin). Total test count: 11 -> 18, all passing on bash 3.2 (macOS) and bash 5.2 (Linux bats/bats container). Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: rollback on partial write failure; unify bats targets - Apply phase now uses a commit-or-rollback discipline: the EXIT trap restores .bak files when _bump_ok=0 (failure path) and deletes them when _bump_ok=1 (success path). Previously the trap always deleted backups, leaving the repo in a partial state when a later sed write failed after earlier writes had already landed. - _bump_ok is a global (not local) so the EXIT trap can read it without triggering set -u's unbound-variable error. - test-bats is now a one-line alias for bats. Both targets now run all tests/**/*.bats; the duplicated install-check and tests/bash/- only scope are removed. All 18 bats tests pass. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * update secretsbaseline Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Co-authored-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix(sso): resolve an email claim for generic OIDC providers authenticate_or_create_user keys the local user record on email and rejects any identity without one. The dedicated Keycloak, Entra and ADFS branches already fall back to other claims when a provider does not release `email`, but the generic OIDC branch read user_data["email"] directly. A provider whose /userinfo omits that claim therefore failed login with `user_creation_failed` after an otherwise successful token exchange, JWKS fetch and userinfo call -- all of which log success, so the failure looks like it happens somewhere it does not. Add _resolve_email_claim for the generic branch: * provider_metadata["email_claim"], when set, is used as-is -- explicit operator intent, no shape check. * Otherwise try email, preferred_username, upn, unique_name, mail. A fallback must contain "@", so a bare username is never silently promoted into the field used as the identity key. Using a fallback is logged, since that value becomes the account identity. When nothing qualifies, log the claim names that WERE present. The previous warning reported only the absence, and userinfo bodies are not logged anywhere else, so an operator had no way to tell which claim to configure. Signed-off-by: Sami Marreed <sami.marreed@ibm.com> * test(sso): cover the generic OIDC email claim on deny paths and against a live gateway Address review feedback on IBM#6484: tests only, no production-code change. Unit tests run the real generic-OIDC _normalize_user_info and the real authenticate_or_create_user (browser SSO path) and build_external_identity (bearer path) with no `email` claim. They assert that an identity resolved from a fallback claim still passes every gate an `email` claim passes: email_verified, trusted_domains, provider linking, auto_create_users, sso_require_admin_approval and admin-claim escalation. Each deny case asserts the resolved email first so it cannot pass vacuously. The live-gateway test drives the authorization-code flow against Keycloak with the `email` scope removed from the client, and covers provisioning from preferred_username, denial when no claim holds an address, denial when a configured email_claim is absent, and precedence of an `email` claim. Signed-off-by: Sami Marreed <sami.marreed@ibm.com> * chore: regenerate secrets baseline Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Sami Marreed <sami.marreed@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…M#6424) * fix(auth): wire external-IdP verification into get_current_user() get_current_user() (mcpgateway/auth.py), the dependency behind get_current_user_with_permissions() and therefore every Depends(get_current_user_with_permissions) REST endpoint (/tools, /gateways, /servers, /rpc, etc.), only ever called verify_jwt_token_cached() — the internal-only JWT verifier. A bearer token from a provider with trusted_for_api_auth=True and a matching SSO_API_TOKEN_AUTH_ENABLED was therefore rejected on every one of these endpoints, even though the external-IdP verification path (verify_credentials_cached -> _maybe_verify_external -> verify_external_idp_token) worked correctly when called directly. Swap the internal-only verifier for verify_credentials_cached(), which already implements exactly this fallback (try a trusted external issuer first, fall back to verify_jwt_token_cached() otherwise) and is already used by require_auth()/require_auth_header_first() elsewhere in the codebase. verify_credentials_cached() returns a superset of verify_jwt_token_cached()'s payload shape for the internal-JWT case (adds a 'token' key), so this is additive for existing callers. Refs IBM#6396 — REST/`/rpc` surface only; does not close it (/mcp remains). Signed-off-by: Sami Marreed <sami.marreed@ibm.com> * test(auth): cover external-IdP REST auth and fix mocks after get_current_user() dispatch change get_current_user() now calls verify_credentials_cached() instead of verify_jwt_token_cached() directly (previous commit), so the ~80 unit tests in tests/unit/mcpgateway/test_auth.py that patched mcpgateway.auth.verify_jwt_token_cached to stub JWT verification need to patch the new dispatch point instead. This is a mechanical rename of the patch target only -- verify_credentials_cached is a superset wrapper (falls through to verify_jwt_token_cached for internal tokens), so mocked return values and existing assertions are unchanged. Also adds a live-gateway black-box test (tests/live_gateway/sso/test_external_idp_rest_auth_e2e.py) that exercises the fix against a real Keycloak-backed gateway: a bearer token from a trusted_for_api_auth provider with a matching audience is accepted on GET /tools and correctly team-scoped via SSO team_mapping (group member sees the team's tool, a user in no mapped group does not), while a wrong-audience token, a token from a provider not opted into trusted_for_api_auth, and no token at all are all still rejected with 401. Reuses the existing Keycloak-backed live_gateway/sso fixture pattern (tests/live_gateway/sso/test_oauth_jwks_e2e.py) and realm-seeded users (infra/keycloak/realm-export.json). Wires SSO_API_TOKEN_AUTH_ENABLED through to the gateway container in docker-compose.yml's sso profile so the new test's scenario is actually reachable when the stack is started with it set; the flag already existed in config.py/.env.example but was never passed through by compose. Signed-off-by: Sami Marreed <sami.marreed@ibm.com> * test(auth): add /rpc and deny-path live tests for external-IdP auth Address review feedback on IBM#6424: cover POST /rpc (tools/list) in the live test alongside GET /tools, and add deny-path checks for a wrong audience, an untrusted issuer and a missing token on both routes. The /mcp mount is out of scope here and stays tracked in IBM#6396. Fixture fixes found while running the suite against a real Keycloak: - admin token uses teams=None so it can create team-scoped resources - an oidc-audience-mapper is added for the run and removed afterwards, because the realm's users carry no client roles and their tokens have no aud claim - the provider fixture restores trusted_for_api_auth, api_audience and team_mapping in a single PUT on teardown - response bodies are read before the request context is disposed - positive-path tests skip when the Keycloak issuer is not https, because verify_oauth_access_token requires an https jwks_uri on the issuer origin Signed-off-by: Sami Marreed <sami.marreed@ibm.com> * test(auth): document the https Keycloak setup for the external-IdP live suite The four positive tests need an https issuer, which the sso compose profile does not provide, so they skip there. Document the minimal https Keycloak and gateway setup that runs all nine tests, and pass -p playwright, which pyproject.toml disables by default. Signed-off-by: Sami Marreed <sami.marreed@ibm.com> * chore(secrets): refresh baseline after rebase Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: remove issue reference from compose comment Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Sami Marreed <sami.marreed@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* new implementation. According to the issue documentation fix is fine. So we are going ahead with that Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed doc to explicity state Platform admin prominence Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
…ests (IBM#6895) * test: eliminate pytest warnings, dead test code, and redundant auth tests Closes IBM#6894 The pytest warnings summary carried 137 avoidable warnings per CI run; most came from conditions inside test files. Fix the causes: - Replace 1-15 byte HS256 secrets with 39-byte constants across nine test files. Signature semantics stay unchanged; pyjwt stops emitting InsecureKeyLengthWarning. - Switch HTTP_422_UNPROCESSABLE_ENTITY to HTTP_422_UNPROCESSABLE_CONTENT in the A2A JSON-RPC passthrough test. - Close the coroutine that create_task-rejection mocks leaked in TestA2AInvalidationBestEffort; the "never awaited" RuntimeWarning fired at garbage-collection time in an unrelated test. - Silence ValidationMiddleware's intentional deprecation warning with a module pytestmark in its three test files; the pytest.warns assertion still passes. - Bind datetime.now(timezone.utc).isoformat(" ") on non-Postgres dialects in twelve Alembic migrations that wrote datetime objects through text(). The string is byte-identical to what the sqlite3 default adapter stores, so stored values do not change and the Python 3.12 deprecated-adapter warning disappears. Postgres branches keep binding the datetime object. - Delete PydanticA2AAgent's class Config; BaseModelWithConfigDict already sets from_attributes=True, so the block was redundant and raised PydanticDeprecatedSince20 at import. - Scope four filterwarnings entries in pyproject.toml: cpex payload headers deprecation (no extensions field exists yet), starlette httpx2 drift, the rbac.get_db() doctest, and a corrected HTTP_413_REQUEST_ENTITY_TOO_LARGE category (StarletteDeprecationWarning subclasses UserWarning, so the old DeprecationWarning filter never matched). Also remove debt the same review surfaced: - Delete the first shadowed class TestGetDb in test_auth.py; the second definition carried the same tests plus the assertions. - Collapse test_invoke_agent_with_{basic_auth,bearer_auth,custom_headers} into one parametrized test; the three copies differed only in agent name, auth type, and header dict. Every header is now asserted to round-trip. Full suite: 23,334 passed, 899 skipped, 2 xfailed, 0 failed. Warnings drop from 252 to 180; every remaining warning is a SADeprecationWarning or SAWarning from historical migration internals, documented as out of scope in IBM#6894. Pinned ruff 0.15.20 and black 26.3.1 show no new findings versus main; detect-secrets findings drop from nine to seven. The five 60-second otel control-telemetry tests from CI job 105708501375 are stale-branch artifacts; main already fixes them in eaf295e (IBM#6131), so this change leaves them alone. Signed-off-by: Jonathan Springer <jps@s390x.com> * test: remove four perpetually-skipped team invitation stub tests Closes IBM#6894 test_create_invitation_success, test_accept_invitation_success, test_deactivate_existing_invitation_before_creating_new, and test_expiry_days_from_settings carried pytest.mark.skip("Complex integration test - main functionality covered by simpler tests") since they were added and never ran. The coverage they name is provided by the simpler tests around them. Delete the stubs; 48 tests remain in the file and pass. Signed-off-by: Jonathan Springer <jps@s390x.com> * test: remove skipped gateway-registration stub from e2e suite Closes IBM#6894 test_register_gateway carried pytest.mark.skip("Requires external gateway connectivity") and an empty body since it was added. Gateway registration is already proven black-box by the live_gateway suite (docker compose testing-up plus the tests under tests/live_gateway/), so no relocation is needed. Delete the stub. Signed-off-by: Jonathan Springer <jps@s390x.com> * test: remove skipped Admin UI version-tab stub and its dead file Closes IBM#6894 test_admin_ui_contains_version_tab carried two skip markers ("Auth system changed - needs update for email auth" and skipif(not settings.mcpgateway_ui_enabled)) and never ran. The rest of test_ui_version.py consists of commented-out stubs and fixtures that serve only the skipped test. Version-check UI behavior belongs to the Playwright suites. Delete the file. Signed-off-by: Jonathan Springer <jps@s390x.com> * test: stop session-affinity test from leaking an OTel span into the suite Closes IBM#6894 test_attach_envelope_trace_context_attaches_context_with_traceparent performed a real opentelemetry.context.attach() of a span built from the W3C example traceparent, discarded the token, and never detached. pytest does not isolate contextvars per test, so every test that ran afterwards in the same serial order saw an active current span. inject_trace_context_headers() then stamped that traceparent into outbound headers and MCP _meta, breaking test_invoke_tool_mcp_streamablehttp (meta=None assertion) and test_streamablehttp_uses_registry_and_modified_payload_with_headers_none under distribution-dependent ordering. The rewritten test patches opentelemetry.context.attach and asserts the extracted context is attached; its previous assertion was the tautology "result is not None or result is None". Verified: leaker-then-victims serial run passes; the full tests/unit/mcpgateway/ serial run passes with zero failures; the full parallel suite passes with zero failures. Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
* feat(protocol): default MCP connect and inbound modes to auto MCP_CLIENT_CONNECT_MODE and MCP_INBOUND_PROTOCOL_MODE now default to auto. Outbound connections negotiate the 2026-07-28 revision through server/discover and fall back to the legacy initialize handshake. Inbound clients may send mcp-protocol-version 2026-07-28. Set either variable to legacy to restore the previous behaviour. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(protocol): update header-default doctest for auto inbound mode The doctest asserted LATEST_HANDSHAKE_VERSION for a request without the mcp-protocol-version header. Auto mode negotiates LATEST_PROTOCOL_VERSION. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * commented out the defaults in the .env.example Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(protocol): correct headerless claim and cover inbound mode live The middleware records the mode default in request state for a headerless request. Era routing reads the raw header in the SDK, so an absent header still takes the legacy handshake path. The docstring states that now. Add black-box coverage for the inbound protocol mode at /mcp: the default auto mode serves a 2026-07-28 server/discover, legacy mode answers 400 with the supported-version list, a headerless initialize negotiates a handshake-era version, an unauthenticated modern request gets 401, and a principal without servers.use gets 403 on a server-scoped path. Pass PLATFORM_ADMIN_PASSWORD and DEFAULT_USER_PASSWORD to the source-run gateway in tests/e2e/test_upstream_connect_mode_e2e.py. The curated subprocess env hid the strong passwords that conftest exports, so Settings.validate_security_combinations rejected the weak built-in default and the subprocess exited with code 1. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: make pre-commit Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
* test: establish sandbox start-method support for validation workers Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * refactor(security): extract killable sandbox pool from jq runner Move the generic pool lifecycle into mcpgateway/utils/sandbox_pool.py. The pool builds workers, proves they start, gates admission, and kills a worker that overruns its budget. Rebuild jq_runner on the new pool. The public API and the jq exception types stay the same. Read the worker count and the timeout through callables. A captured value would ignore a setting that a test changes after import. Stamp the admission gate on the executor. A gate held on the pool object would admit against a different executor after a rebuild. Tighten the start-method test to expect a TimeoutError. A broad Exception also accepts a broken pool, which passes for the wrong reason. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): read the sandbox budget before taking a permit The budget read sat between the gate acquire and the try block that releases the permit. A raising timeout function leaked the permit, and the pool then returned SandboxBusy forever. Move the read above the acquire. It needs no permit. Add a regression test. It arms a raising timeout function, then proves a later call still reaches a worker. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat(security): run regex-bearing schema validation in a killable sandbox A JSON Schema regex is attacker-influenced. Python holds the GIL while it backtracks. An unbounded match freezes the gateway worker for every tenant. Route a schema that carries a regex keyword into a killable worker pool. Bound each validation by a wall-clock limit. Never inspect pattern content. Whole-validation routing also covers the paths that reach re inside jsonschema itself, which no keyword override intercepts. Fail closed on every abnormal outcome. A timeout, a busy pool, a broken pool, an oversized instance, an unserializable input, and an absent sandbox all raise ValidationError. A truncated validation is never a pass. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(security): assert the sandbox bounds the validation, not the clock An elapsed-time ceiling passes for any fast failure. A pattern that merely rejects quickly satisfies it, so the corpus could stay green over a guard that routes every schema inline. This project has shipped that failure once. Assert the raised error carries the safety-budget wording, so a bound is told apart from an ordinary mismatch. Add a test that asserts the sandbox submission itself: present for a regex keyword, absent without one. Routing is now proved directly rather than inferred from timing. Name the real cause when the pool fails to start. The previous text blamed the platform for every failure, including a configuration fault, and sent an operator to the wrong place during an incident. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): refuse an unknown validator class and test the escape for real The two patternProperties escape tests swallowed every error and asserted only elapsed time. They passed with no sandbox at all, so the one hole that killed the previous design read as covered while nothing checked it. Assert the timeout wording instead, and cover both keywords from one case. Match the timeout phrase rather than the generic sandbox-failure phrase. The generic text also appears when the pool is broken, so the corpus went green against a pool that failed every submission in microseconds. Refuse a validator class outside the draft table. Substituting a draft gave an extended validator different semantics in the sandbox than inline, which weakens the boundary and says nothing. Name the class instead. Drop the config reference to an untracked design document. Place the new main import in sorted order. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): bound tool input and output schema validation Route _validate_with_cached_schema through validate_safely. The sandbox now bounds both tool input validation and tool output validation, the two sites that shared this helper. A catastrophic-backtracking schema no longer holds the GIL. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): cap bounded-validation tests and correct stale comments Add a 30s pytest-timeout marker to every test that asserts a validation bound. Without it, a sandbox regression hangs the test instead of failing it, and CI stalls with no named cause. Delete the now-unused _NO_RETRIEVE_REGISTRY constant and correct the SSRF-guard comment above it: validators are built against an empty registry inside validate_safely, not against this file's own registry, which no longer exists. Update _validate_with_cached_schema's docstring to describe the sandbox delegation instead of stale inline-validator mechanics. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): bound prompt argument schema validation DbPrompt.validate_arguments() called jsonschema.validate() directly, bypassing the sandboxed validation path. A crafted argument_schema with a catastrophic regex pattern could freeze prompt rendering, and this path needs only permission to render a prompt to trigger. Route the call through validate_safely() so the schema runs in the same killable, time-bounded worker as tool input and output validation. The ValueError contract for callers is unchanged. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): withhold regex output schemas from the SDK validator The vendored MCP SDK validates outputSchema with stock jsonschema at mcp/server/lowlevel/server.py:573. The gateway decorator validate_input=False gates only the input validator at server.py:536. The output call stays active. A remote MCP server controls tool.output_schema. It therefore controls a regex that runs unbounded inside site-packages, where no gateway change reaches. _to_mcp_tool now omits outputSchema when schema_uses_regex reports a regex keyword at any depth. The advertised tool drops the key, so the SDK skips the validation. The bounded check in tool_service still validates the same data. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): guard proxied output schemas on the direct-proxy path _proxy_list_tools_to_gateway returned the remote gateway's SDK tool models unchanged. Those models carry the remote's outputSchema into the SDK tool cache, where mcp/server/lowlevel/server.py:573 validates against it with stock jsonschema. The previous commit guarded only _to_mcp_tool, so a direct_proxy gateway still reached the unbounded validator. Extract _guard_output_schema and call it from both paths. _to_mcp_tool builds a dict payload; the proxy path holds constructed pydantic models, so _guard_proxied_tools adapts the shape with model_copy. The regex decision lives in one function, so the two paths cannot drift. The SDK skips output validation when outputSchema is None, so replacing the value is enough; the field need not be absent. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat(security): warn when a registered schema carries a regex keyword Registration, gateway sync and OpenAPI import now call warn_unprovable_patterns at the point each schema reaches storage. Each call logs one warning naming the schema's source and never raises, so no registration, sync or import can fail because of this check. The sandbox already bounds every regex match; this only gives an operator an inventory of which schemas carry a regex keyword. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): keep the regex-pattern warning out of the federation failure path register_gateway ran warn_unprovable_patterns before appending the new DbTool to db_tools, inside the same try/except that drops a tool from federation on any exception. The warning cannot raise today, but a diagnostic call sitting ahead of the data operation it describes turns any future change to that call into silent federation data loss. Move both calls to run after the tool is already in db_tools, so a raising warning can only fail to log, never unregister the tool. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): make warn_unprovable_patterns structurally unable to raise 74e5029 introduced warn_unprovable_patterns and stated in its commit body that no registration, sync or import could fail because of the check. That was true only because the function happened not to raise; nothing in the code enforced it. Four call sites relied on this unstated assumption without a guard: the before_insert/before_update listeners in db.py catch only jsonschema.exceptions.SchemaError, so any other exception would abort the SQLAlchemy flush and fail tool or prompt registration, and the OpenAPI import sites in openapi_schema_router.py and admin.py had no exception handling at all. Rather than wrap every call site in its own try/except, move the guarantee into the function itself. warn_unprovable_patterns now catches and logs any exception from its own body, including from schema_uses_regex, so it cannot fail its caller by construction. Every current and future call site inherits this for free. The gateway_service.py federation-loop ordering fix from the prior commit is unaffected and stays as defence in depth. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * feat(security): log operator regex patterns compiled without a time bound Four plugins compile operator-supplied regex strings at load time: regex_filter, resource_filter, code_safety_linter and harmful_content_detector. These patterns come from deploy-time configuration, not from a request, so a pathological pattern here is administrator self-denial-of-service, not an attack surface. Route them through a log line, not the validation sandbox. Add warn_unprovable_pattern_source() to safe_jsonschema.py, alongside warn_unprovable_patterns(). It never raises, so a plugin cannot fail to load because a diagnostic failed. Call it immediately before each operator-configured re.compile() site. The hardcoded default lexicons in code_safety_linter and harmful_content_detector stay silent: they are project-authored, and a warning on a pattern the project itself ships is noise. Convert the harmful_content_detector comprehension to a loop so the log call can run per pattern, keeping the existing non-string entry passthrough unchanged. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(security): prove no validation path reaches an unbounded regex engine A grep over first-party code cannot prove this. The escape found in review lived in site-packages, which a grep over mcpgateway/ never reads. Verify by execution instead. Three layers cover the tool input path, the tool output path and the prompt argument path. Layer 1 spies on the validation sandbox. It asserts each path submits a regex-bearing schema, and submits nothing for a schema without one. Only a positive assertion can prove that routing happened. Layer 2 replaces the pattern and patternProperties keyword implementations in every stock draft with a function that raises. Any pattern evaluated in this process fails the test by name. Patch the keyword tables, not jsonschema._keywords: each draft captured the function object at import time, so the module attribute is never read again. A module-attribute patch was measured and does not fire. Layer 3 makes jsonschema.validate raise. Nothing calls it today. It guards against a future change that reintroduces a direct call. The sandbox worker is a separate process, so no in-process patch reaches it. A correctly routed validation still succeeds while the tripwires are armed. Each layer carries a self-test that proves the layer can fail. One more test rejects an instance that breaks the pattern, which proves the worker evaluated the schema. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(security): cover the SDK output-schema site and assert rejection content The module proved three first-party sites and was blind to the fourth. The vendored SDK validates outputSchema in-process and unbounded at mcp/server/lowlevel/server.py:573. validate_input=False does not gate that branch. The only defense is _guard_output_schema, which withholds the schema. That guard submits nothing to the sandbox and evaluates no keyword, so the submit spy and the keyword tripwire both miss it. Removing the guard left all tests green. Add a fourth layer. Assert _to_mcp_tool and _guard_proxied_tools withhold a regex-bearing outputSchema, then drive the real SDK CallToolRequest handler with the tripwires armed. A self-test passes an unguarded schema to prove the SDK site is live. Hold the SDK input branch shut as well. inputSchema reaches the SDK unguarded, so validate_input=True would open a second unbounded site. Read the flag off the registered handler and assert it stays False. Assert what the rejection says, not that a rejection happened. Every fail-closed branch of validate_safely produces the same outcome in-process with no worker running, so the former assertion passed with the sandbox switched off. Each schema shape now names the message fragment only its own evaluation produces. Cover the shapes the corpus missed. Add a pattern under anyOf, which a router that skips JSON array nodes reports as regex-free, and a patternProperties schema, which the tripwire armed but nothing exercised. Parametrize every routing, tripwire, guard and rejection test over all three shapes. Record two limits in place. The jsonschema.validate patch cannot reach a caller that binds the function at import time, and no schema here may carry $anchor or $dynamicAnchor, because the metaschema checks both with its own pattern keyword. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(security): assert the event loop stays responsive under schema ReDoS A hostile subject against a nested-quantifier pattern stalls the event loop for about 8 seconds when the match runs in this process. Moving the match to a thread changes nothing, because CPython holds the GIL through a regex match. A test that only measures how long validation took passes against that non-fix. Measure the loop instead. The heartbeat wakes every 10 ms and records how late each wake-up was. The test asserts three things: the heartbeat produced samples, the worst wake-up stayed inside twice the sandbox budget, and the validation was stopped by that budget. The last assertion separates a working sandbox from a broken pool that refuses every regex-bearing schema and also keeps the loop free. The subject is 28 characters. Backtracking doubles per added character, so a longer subject runs for hours under a regression, and pytest.mark.timeout cannot break it: the signal is delivered between bytecodes and a regex match never yields one. The test would hang instead of fail. The ReDoS test in test_input_validation.py stored a catastrophic pattern and asserted it was stored. It now validates a hostile subject against each stored schema, and asserts the budget stopped the one pattern that backtracks. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(security): assert the timeout wording at every bounded-validation site Three sites asserted a bound with elapsed time plus a weak outcome check. That pair cannot tell "the sandbox bounded a runaway" apart from "rejected instantly for an unrelated reason". Each site now asserts the message contains "exceeded the execution time limit", which only the timeout path produces. A mismatch gives a jsonschema message, and any other sandbox fault gives the broader "could not be completed safely" text. The prompt site reads the phrase through the ValueError that validate_arguments raises. The pinning test for warn_unprovable_patterns broke only the schema_uses_regex call, so it pinned one path rather than the never-raises property. Moving the success-path warning out of the try would have kept it green while reintroducing an unguarded statement. The test is now parametrized over each raisable statement in the function body. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): make the never-raises diagnostics true to their docstrings Both warn_unprovable_patterns and warn_unprovable_pattern_source promise to swallow every failure, including a failure from logging itself. The fallback warning in each except handler was unguarded, so broken logging still raised into a SQLAlchemy before_insert listener, a federation sync loop, and a plugin load path. Those callers drop a tool registration when a diagnostic raises. Both handlers now report through _report_diagnostic_failure, which guards the fallback and drops a second failure. Nothing remains to do once logging is unusable. warn_unprovable_pattern_source had no pinning test. It now carries the same parametrized test as its sibling, one raisable statement at a time: the len(pattern) the log call evaluates, the log call itself, and the fallback log. Both tests gain a case that breaks every logging call, which is what pins the new guard. The subject-length comment in the ReDoS test claimed pytest.mark.timeout cannot interrupt a regex match. That is false; the signal arrives during the match. Measured behavior: the timeout reports on schedule, then the test call waits for the match, because a Python thread cannot be killed and closing the loop joins its executor. A 3 second timeout on a 30 character match reports at 3 seconds and returns after 42. The comment now states that. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(security): pin the event-loop budget and finish the bounded-validation wording The event-loop regression test derived its stall budget from regex_timeout_seconds, the same unpinned setting the sandbox uses for its own timeout. Raising that setting past about 3.85 seconds let the budget grow past the actual GIL stall, so the test kept passing while a thread-only sandbox stalled the loop for close to 8 seconds. Add an explicit precondition that fails when regex_timeout_seconds exceeds the value this suite was measured against, and cap the derived budget so neither the setting nor the derivation can silently widen the check. Apply the measured MAX_VALID_SUBJECT_SECONDS bound to the valid-subject test, which previously allowed a 500x regression to pass unnoticed. Correct the BOUNDED comment at the three remaining sites that claimed the timeout path is the only one producing the outer "could not be completed safely" text. validate_safely wraps every sandbox fault in that text; the inner "exceeded the execution time limit" phrase is what actually comes from SandboxTimeout alone. Add the missing validation-pool fixture to the ReDoS coverage in test_input_validation.py, scoped to the one test that needs it rather than autoused across the file's many unrelated security tests, so the pool's lifecycle no longer depends on lazy _ensure() and on no other module having left the sandbox marked down. Proved the budget fix catches the regression it was written for: swapping ThreadPoolExecutor into SandboxPool._build and setting REGEX_TIMEOUT_SECONDS=5 now fails the event-loop test where it previously passed. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(e2e): prove a hostile schema pattern cannot stall a live gateway Registers a tool whose input_schema carries a catastrophic backtracking pattern, invokes it through the real MCP path (HTTP routing, auth, RBAC, the sandboxed validator), and asserts the gateway still answers /health immediately afterwards. Unit-level coverage proves validate_safely is bounded in isolation; this proves the property survives everything between a unit test and a real request. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(test): bound the ReDoS e2e probe by the MCP client timeout _mcp_tool_call rides a ClientSession capped at _CLIENT_TIMEOUT (default 5s), which fires before a 20s elapsed assertion ever could. A genuine regression now surfaces as a diagnosed pytest.fail naming both possible causes -- an unbounded sandbox or an overloaded CI box -- instead of an uncaught McpError traceback. Also delete the throwaway server before the tool it holds, reusing the file's existing _delete_owned helper so a cleanup failure is reported rather than swallowed by a bare suppress(Exception). Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(test): catch the ExceptionGroup a bare McpError handler cannot see _mcp_tool_call runs initialize() then call_tool() on the same session, so the SDK's anyio TaskGroups wrap a real timeout in ExceptionGroup twice (per _unwrap_exception_group's own docstring). A bare 'except McpError' never catches that shape, so a regression would still surface as a raw nested traceback instead of the intended diagnostic. Widen the except to (McpError, ExceptionGroup) and unwrap with the file's existing _unwrap_exception_group helper, matching the pattern already used by TestTokenLifecycle.test_scoped_token_denied_tool_execute. Also reinstate the elapsed assertion. _CLIENT_TIMEOUT bounds a single send_request round trip, not the whole call, and this call makes two (initialize, call_tool) after session setup, so the honest ceiling is a small multiple of _CLIENT_TIMEOUT rather than the raw value or a dropped check. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs(security): record schema validation sandbox changelog Document the sandboxed schema-validation worker process in the CHANGELOG under Unreleased: pattern/patternProperties schemas now validate in a killable worker, registration behavior is unchanged, and a tool's outputSchema is withheld when it carries a regex keyword. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: offload sandboxed schema validation calls to a thread Both call sites in mcpgateway/services/tool_service.py invoked the schema validator synchronously from async methods. SandboxPool.submit() blocks on Future.result(timeout=...) while a worker process validates a regex-bearing schema, so a hostile request held the event loop for the sandbox's timeout budget on every call, not just once. Wrap both calls in asyncio.to_thread, matching the existing jq offload pattern in this file. _extract_and_validate_structured_content stays synchronous; only the call site changes, so its doctests and its architectural documentation (Validator B) are unaffected. Adds a regression test driving the real production entry point (preview_tool_invocation) with a catastrophic input_schema, measuring event-loop lateness. Proven both directions: fails at 1.01s stall with the offload removed, passes at 0.006s with it restored -- the existing sandbox test alone could not catch this, since it wraps its own call in asyncio.to_thread and so never exercises whether production offloads it. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: pass object arguments in schema ReDoS test _validate_tool_input_arguments takes a Dict argument. Wrap the hostile subject in an object and move the pattern onto a property schema. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: document the JSON Schema regex sandbox Add a securing.md section for the REGEX_* settings, fail-closed behavior, platform start method, outputSchema withholding, and the unbounded client SDK output validation path. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: name validate_safely as Validator B's mechanism Validator A in the MCP client SDK still uses stock jsonschema. List that path under known gaps. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(security): fork the schema sandbox only on Linux macOS lists fork in get_all_start_methods(), so the sandbox chose fork there. Fork is unsafe on Darwin. Pick fork on Linux only and spawn elsewhere, matching jq_runner.subprocess_mode_available(). Reuse one module-level no-retrieve registry for inline and worker validation. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: drop outputSchema withholding after the MCP SDK v2 upgrade The v2 low-level server no longer validates outputSchema, so the withholding guard defended a removed code path. After the rebase the guard also read the v1 field name and withheld nothing. Restore the transport to main, so tools advertise outputSchema unchanged. Remove the guard tests and the related changelog and docs text. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: cover output-path offload and tool insert listener; document REGEX_* settings Add a loop-stall test for the invoke_tool output-schema path. Add a Tool before_insert test for the regex warning. Pin the real outcome of each ReDoS pattern in test_regex_dos_prevention. Reset the sandbox-down flag in the safe_jsonschema test fixture teardown. Add REGEX_TIMEOUT_SECONDS, REGEX_WORKERS, and REGEX_MAX_SUBJECT_BYTES to configuration.md. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: evaluate sandbox workers once on start Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: refresh secrets and align sandbox test mocks after rebase Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: complete regex warnings and bound output validation test Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* fix: harden outbound HTTP connection handling Outbound HTTP requests now establish connections against an address that the outbound URL policy validated, and keep the original hostname for the Host header and for TLS certificate verification. A shared helper centralises this so every outbound path applies the same policy. Outbound URL validation now runs regardless of the SSRF protection setting. The scheme allowlist, dangerous-protocol, control-character, embedded-credential and XSS checks apply unconditionally; disabling SSRF protection no longer disables them as a side effect. Behaviour notes for operators are in CHANGELOG.md, including pooled MCP sessions no longer following redirects, fail-closed handling when a caller-supplied client's TLS context cannot be read, and reduced DNS address failover on some requests. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * style: apply ruff format to the changed files CI's pre-commit job runs the ruff-format hook, which is not covered by the local 'make ruff' target (check-only). Reformats five files this branch touched. Verified with an AST comparison before/after that no file's parse tree changed except one docstring collapsed from two lines to one, whose text content is identical; every other change is line-wrap and trailing- comma only. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * style: correct file headers on two new files The repo's correct-file-headers pre-commit hook expects a Location/Copyright/ SPDX docstring header on every mcpgateway/ and tests/ Python file. ssrf_pinning.py had a stale 'Copyright 2025' line instead of the current 'Copyright contributors to the MCP-CONTEXT-FORGE project'; the new test file had no header block at all. Applied via .github/tools/fix_file_headers.py --fix-all, the same tool CI runs. No code content changed. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: restore the request URL after a pinned send Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: isolate pinned OAuth and cross-gateway requests from the shared pool httpcore keys pooled connections by (scheme, host, port) and ignores sni_hostname. Outbound DNS pinning rewrites the host to a validated IP, so two different hostnames pinned to the same address collapse to one origin and can reuse a connection whose TLS handshake verified a certificate for the other hostname, while carrying this hostname's credentials. Move the OAuth token POST (oauth_manager.py _post_token_request) and the UAID cross-gateway POST (a2a_service.py _invoke_remote_agent) off the shared HTTP client onto get_isolated_http_client, matching the pattern already used by tool_service.py and the sibling A2A local invocation path. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: isolate pinned LLM provider requests from the shared pool Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: record that pinned requests require an isolated client Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: adapt sibling tests to isolated-client pinning fix Four sibling test files independently exercised the outbound pinning code paths fixed earlier on this branch, but mocked the old access points or asserted the old buggy behaviour, so a full make test run still failed 56 cases across them. - test_mcp_servers_router.py: the two SNI-pinning transport tests asserted the request URL stayed pinned to the resolved IP after a successful send. That is the bug fixed in 228d2c2 (the transport restores request.url to the original hostname in a finally block). Update both assertions to expect the restored hostname. - test_a2a_service_uaid_security.py: _invoke_remote_agent's UAID path now calls get_isolated_http_client() directly instead of the shared get_http_client() singleton. Add an autouse fixture, trimmed from the one already used in test_a2a_service.py, that bridges the isolated client call to whatever get_http_client mock each test installs. - test_oauth_manager_pkce.py and test_identity_propagation.py: _post_token_request's non-mTLS branch now calls the free function get_isolated_http_client() instead of the per-instance _get_client() method, so a method override can no longer intercept it. Convert every _get_client patch/monkeypatch site to patch get_isolated_http_client via the _isolated_client() helper already reviewed in test_oauth_manager.py, adding a local copy of the helper to each file. test_identity_propagation.py's raw attribute assignments also gained the monkeypatch fixture they needed. The mTLS branch of _post_token_request builds its own inline httpx.AsyncClient and is unchanged, so no mTLS-path test was touched. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: correct stale oauth_manager isolated-client docs, harden pinning test Update _post_token_request's docstring and the follow_redirects comment, which still described the pre-fix shared-client behavior and could mislead a future maintainer into reverting the isolated-client fix that keeps pinned token requests out of the shared connection pool. Strengthen test_oauth_token_post_uses_an_isolated_client to assert follow_redirects=False on both captured isolated clients, matching its LLM proxy sibling test. Add a regression test proving SniPinningTransport's URL-restore finally block does not corrupt or block a still-open multi-chunk response stream. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: migrate outbound DNS pinning tests to httpx2 SniPinningTransport now subclasses httpx2.AsyncHTTPTransport and the gateway/tool-service connectors route through mcp_proxy_client instead of importing sse_client/streamablehttp_client directly. Update the transport and connector-patch tests to match. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: skip TLS fail-closed check for non-pooled transports A transport with no connection pool (an in-process ASGITransport, used by tests) never dials the network, so there is no DNS-rebinding surface for _wrap_httpx_client_factory to pin. Treat that case as unpinned instead of raising, and keep the RuntimeError fail-closed path for a pooled transport that unexpectedly carries no TLS context. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * style: apply ruff format to llm_proxy_service.py Parenthesize the multi-context-manager async with, matching the ruff-format style the pre-commit CI job enforces. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: keep outbound pinning when the proxy variable does not match the URL scheme httpx mounts HTTP_PROXY on http:// only and HTTPS_PROXY on https:// only. _egress_proxy_applies() treated any proxy variable as covering every scheme. With only HTTP_PROXY set, an https:// target skipped pinning while httpx still connected directly and resolved DNS again. Select the proxy variables by the URL scheme. Clarify in the module docstring that the shared-pool constraint applies to URLs pinned by PinnedTarget.pin(), not to SniPinningTransport. Add regression tests for the proxy scheme match, NO_PROXY exemptions, unsafe Host authorities, pinned-address exhaustion, the pooled-session wrap no-op paths, the streamablehttp health check, the OAuth custom-CA branch, and the RuntimeError type on a blocked pooled session. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: assert the pinned dial address in SniPinningTransport router tests The URL-restore change dropped the check that the transport dials the pinned address. Capture the URL inside the mocked send so the tests assert both the dialled address and the restored hostname. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: name invoke_tool in the SSRF_DNS_FAIL_CLOSED description Name the REST arm of tool invocation explicitly. Remove an unrelated issue reference from two session registry test docstrings. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: expect MCP error results from tool pinning tests after rebase invoke_tool_direct and the MCP transport arms of invoke_tool now return a CallToolResult with is_error set instead of raising (IBM#6181). Assert the error result instead of an exception. The direct-proxy refusal test records dial attempts, because the error result also absorbs the AssertionError from _refuse_to_dial. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: configure URL schemes in DNS fail-open test Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* fix(oauth): preserve catalog on empty fetch Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com> * fix(oauth): keep empty-catalog guard aligned with current main Follow-up to the merge with main, which reworked fetch_tools_after_oauth around the guard added by this PR. - route gateway.name through SecurityValidator.sanitize_log_message in the new warning, matching the log hygiene the rest of the function adopted upstream - update the regression test for the EmailUser lookup fetch_tools_after_oauth now performs to build user_context, and assert that no DELETE statement is issued instead of counting db.execute calls, which is no longer a proxy for 'no stale cleanup ran' Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com> * test(oauth): cover partial catalog cleanup after authorization Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com> * refactor(oauth): reuse MCP_SYNC_CREATED_VIA_VALUES for OAuth cleanup The empty-catalog guard added a local mcp_created_via_values set. Main has since gained the module-level MCP_SYNC_CREATED_VIA_VALUES constant, used for the same purpose in _refresh_gateway_tools_resources_prompts, so the local copy is now a near-duplicate that also omits "notification_service" and therefore leaves stale notification-driven entries behind on authorization code gateways. - pass the shared constant from fetch_tools_after_oauth instead of the local set, so both catalog-cleanup paths agree on what counts as MCP-discovered - extend the partial-catalog test with a stale notification_service tool the OAuth fetch path must prune, which pins the constant in place - refresh the two .secrets.baseline line numbers shifted by the test change Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com> Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Add 1-line reminder of new UX port to Makefile
Signed-off-by: Jonathan Springer <jps@s390x.com>
* chore: use WEB_UI_PORT env var in testing-up output
Replace the hardcoded port in the testing-up service table with
${WEB_UI_PORT:-3001}, matching the pattern used in docker-compose.yml.
This ensures the displayed URL reflects any custom WEB_UI_PORT override.
Signed-off-by: Jonathan Springer <jps@s390x.com>
* fix: resolve testing-up web UI port from Compose configuration
The testing-up summary expanded ${WEB_UI_PORT:-3001} in the recipe
shell, which never reads .env. Compose reads .env for interpolation,
so the summary printed 3001 while Compose published the port from
.env (e.g. 3100). Rename the row to ContextForge Web UI: Compose
publishes web_ui directly, not through the gateway nginx listener.
Add scripts/testing-web-ui-port.sh to read the published host port
from the resolved Compose configuration. The script uses
config --format json on Docker Compose v2+ and parses canonical
config YAML on podman-compose and docker-compose v1, which have no
JSON output. A failed lookup stops the recipe instead of printing a
fallback URL.
Add tests/bash/testing_web_ui_port.bats to compare the resolver
against the Compose-resolved host port for the four WEB_UI_PORT
placements and to pin the YAML parser shape per backend.
Signed-off-by: Jonathan Springer <jps@s390x.com>
* test: supply required compose variables in web UI port fixture
The bats fixture env file supplied DEFAULT_USER_PASSWORD and
PLATFORM_ADMIN_PASSWORD only. Compose interpolation for the testing
profiles also requires JWT_SECRET_KEY and AUTH_ENCRYPTION_SECRET, so
the four resolver tests failed on shells without those variables
exported. Add recognizable test-only placeholders for both.
Signed-off-by: Jonathan Springer <jps@s390x.com>
---------
Signed-off-by: Jonathan Springer <jps@s390x.com>
IBM#7087) * fix(performance): guard psutil.cpu_freq() against SystemError on macOS psutil 7.2.2 on macOS (Apple Silicon) raises: SystemError: <built-in function cpu_freq> returned a result with an exception set The C extension does not cleanly propagate host_processor_info failures, leaving CPython's internal error flag set instead of raising OSError. Wrap the cpu_freq() call in try/except Exception so any variant from the C extension is caught and cpu_freq_mhz falls back to None. Fixes IBM#7086 Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * fix(version): guard psutil.cpu_freq() against SystemError in version.py Same psutil 7.2.2 macOS bug as performance_service.py — cpu_freq() raises SystemError via its C extension on Apple Silicon. The /version endpoint calls _psutil_metrics() which was also unguarded. Fixes the e2e failure: FAILED tests/e2e/test_main_apis.py::TestVersionAndDocs::test_get_version_with_admin_auth Related: IBM#7086 Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * test: add parametrized tests for psutil.cpu_freq() exception guard Covers SystemError and RuntimeError from cpu_freq() in both: - performance_service.get_system_metrics() - version._system_metrics() Verifies cpu_freq_mhz is None and other metrics are unaffected. Related: IBM#7086 Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * test(version): use _make_fake_psutil() in cpu_freq exception test Replace inline MagicMock stub with the existing _make_fake_psutil() helper, overriding only cpu_freq. Consistent with the rest of the test file. Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * test(performance): extract _make_mock_psutil() helper, refactor TestSystemMetrics Eliminates the duplicated 8-line inline mock setup in test_get_system_metrics_load_average_exception_sets_none and test_get_system_metrics_cpu_freq_exception_sets_none. Each test now calls _make_mock_psutil() and overrides only the attribute it cares about. Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * test(version): rename _make_fake_psutil to _make_mock_psutil Consistent naming with test_performance_service.py. Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> --------- Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
* chore: remove mcp-servers sample directory and templating Remove the unsupported sample/test MCP servers, the Go/Python scaffolding templates, and the url-to-markdown CI workflow from the monorepo. The six Python servers now live in IBM/contextforge-examples under mcp-servers/python/. The scaffolding templates are deleted without migration. - Delete mcp-servers/ (six Python servers, templates, scaffold scripts). - Delete the mcp-url-to-markdown-tests workflow. - Delete the sample-server doc pages. Rewrite the servers section index. - Remove mcp-servers references from CI, lint configs, packaging, and docs. - Switch the performance compose generator and benchmark docs to the published cfex-mcp-fast-time-server and cfex-mcp-benchmark-server images. - Remove stale Go CI setup. No Go sources remain in the repository. - Update the fast-time-server catalog entry for the published image. The /v1/mcp-servers REST API is unchanged. It remains the product-language alias for the gateways API. Closes IBM#6378 Closes IBM#6379 Closes IBM#6380 Closes IBM#6381 Closes IBM#6382 Closes IBM#6383 Closes IBM#6384 Signed-off-by: Jonathan Springer <jps@s390x.com> * test: align regression tests with generator and workflow changes Remove the obsolete JWT_SECRET_KEY escape workaround in test_issue_6083_compose_cleanup.py. The generator template is escaped at the source, so the monkeypatch double-escaped the placeholder and reintroduced the KeyError. Drop the Set up Go step assertion in test_secret_baseline_ci_decision_workflow.py. The step left pre-commit.yml with the Go toolchain removal. The shared ci-decision contract remains covered by the Run pre-commit and detect-secrets assertions. Signed-off-by: Jonathan Springer <jps@s390x.com> * ci: restore Set up Go in lint-helm-ci job make linting-full runs linting-workflow-actionlint, which executes actionlint via go run. The job needs a Go toolchain even though the repository no longer contains Go sources. Cache stays disabled: no go.sum or go.mod exists to key it. Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com>
* fix(a2a): propagate W3C trace context through A2A invocations
The request span middleware now traces POST /a2a/invoke and /a2a/{agent_name}/invoke.
The span adopts a valid inbound traceparent, for example from Envoy, as its parent.
The a2a.invoke span injects its context into the outbound agent request headers.
The downstream agent call therefore joins the same trace as the inbound request.
Unit tests cover route selection, parent adoption, missing or malformed context, and outbound injection.
Signed-off-by: Antonio Miranda <antonio.miranda@basf.com>
* fix(a2a): harden W3C trace propagation per review
Address review findings on IBM#7093:
- Gate context baggage on policy inside inject_trace_context_headers().
The composite propagator emitted raw baggage before the settings
check. Remove it unconditionally and re-add sanitized baggage only
when otel_baggage_enabled and otel_baggage_propagate_to_external
both permit it (CWE-200).
- Replace traceparent, tracestate, and baggage keys case-insensitively
before injection. Prepared headers from auth mappings or plugins can
no longer conflict with the active span, and stale tracestate no
longer survives an empty active trace state.
- Inject the active context into UAID cross-gateway forwarding headers
in _invoke_remote_agent so gateway hops continue the same trace.
- Trace the /a2a/{agent_name}/jsonrpc ingress route as well.
- Add regression tests against the real opentelemetry-api propagator.
- Add a live black-box suite that boots an isolated gateway with the
real SDK and a header-capture agent. It asserts one trace ID across
inbound parent, request span, a2a.invoke span, and downstream
traceparent, denies outbound dispatch on authentication failure, and
withholds context baggage when external propagation is disabled.
- Add a pytest (observability extra) CI job that runs the SDK-gated
modules and the live suite.
Signed-off-by: Jonathan Springer <jps@s390x.com>
---------
Signed-off-by: Antonio Miranda <antonio.miranda@basf.com>
Signed-off-by: Jonathan Springer <jps@s390x.com>
Co-authored-by: Antonio Miranda <antonio.miranda@basf.com>
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
* fix: validate x-mcp-header annotations and mirror them as Mcp-Param headers on upstream calls Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: trim the _call_upstream_tool docstring Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: collapse the x-mcp-header mirroring tests into one call-site test Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: supply the default request context in the x-mcp-header transport tests Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: assert the exact x-mcp-header validation error in the federation test Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: prove unauthorized calls are refused before the x-mcp-header mirroring path Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…sions (IBM#7137) Section 1.6 previously told maintainers to grep FROM lines and visit the Red Hat Container Catalog by hand. The new make container-bump-image-versions target automates this: it queries the Pyxis API, validates pin consistency across Containerfile, infra/wheels/Containerfile, and infra/nginx/Dockerfile, and applies updates atomically with rollback on failure. Replace the manual grep-and-table prose with the automated target, its managed ARG/file table, the minor-line policy note, and the prerequisite (curl and jq) admonition. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* chore: update Python dependencies for 2026-10-05 release cycle - Bump mcp constraint in all external plugins (opa, cedar, llmguard) from >=1.28.1,<2 to >=2.2.0 to match gateway's mcp>=2.0.0 requirement and resolve uv lock failure for Python 3.13 split - Add oauthlib>=4.0.0 transitive pin; fixes PYSEC-2026-4114 - Add oauthlib to [tool.uv.exclude-newer-package] to allow 4.0.0 (published 2026-09-28) past the global exclude-newer=10 days cutoff - Advance all [tool.uv.exclude-newer-package] dates to 2026-10-05T23:59:59Z - Fix typo: 'Transivite' -> 'Transitive' in requests comment - Bump docs/requirements.txt, tests/load/requirements.txt, tests/populate/requirements.txt, and uv.lock to latest resolved versions Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix pre-commit Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: bump RetryWithBackoffPlugin max_retries to 3 for 0.3.11 cpex-retry-with-backoff 0.3.11 changed max_retries semantics to total attempts including the original call. Bump from 2 to 3 to preserve the prior behaviour of 2 retries after the original. Also update version field to 0.3.11. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* chore: migrate Tailwind CSS v3 to v4 and fix npm CVEs - Upgrade tailwindcss from ^3.4.17 to ^4.0.0; drop autoprefixer (bundled in v4) - Add @tailwindcss/postcss devDependency; update postcss.config.js to use it - Remove tailwind.config.js; port config to CSS-native format in tailwind.src.css: - Replace @tailwind base/components/utilities with @import 'tailwindcss' - Add @custom-variant dark for .dark class dark mode - Port custom keyframes (float, pulse-soft, slide-up, fade-in) to @Keyframes - Register utilities via @Utility (animate-float, animate-pulse-soft, etc.) - Upgrade postcss-cli from ^11.0.0 to ^12.0.0 (uses chokidar 5.x, drops braces) - Fix overrides: brace-expansion >=5.0.12 (GHSA-q2hr/qhr7/6j4f), fast-uri >=3.1.8 (GHSA-qw65/58mr/hrr3), ip-address >=9.0.5 (GHSA-j6r3/h3mg), braces >=3.0.3 (partial mitigation GHSA-vfj7) - 10 remaining findings are all stylelint -> fast-glob -> micromatch -> braces; no upstream fix exists (micromatch 4.x has no patched release); dev-only linting tools with no production bundle impact Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: remove tailwind.config.js from Containerfile COPY tailwind.config.js was deleted as part of the Tailwind v4 migration (config is now CSS-native in tailwind.src.css). The Containerfile still referenced it on line 174, causing the amd64 Docker build to fail with 'not found'. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
The llmchat extra depended on langchain-mcp-adapters, which has no release that works with mcp>=2.0.0: 0.3.1 imports `mcp.shared.context.RequestContext` (removed in v2) and 0.3.2 pins `mcp<2.0.0`. Upstream closed v2 support as superseded by `langchain.mcp`, whose releases require langchain-core>=1.6 and langgraph>=1.2.11, above the versions the llmchat extra allows. LLM Chat therefore failed to start, and the guarded import reported it as "dependencies are missing". Replace MultiServerMCPClient with a small `_MCPToolSource` that lists tools with `mcp.client.Client` (following pagination) and wraps each one as a LangChain StructuredTool that calls it over a fresh session, using the gateway's existing streamable HTTP helper, `sse_client` and `StdioServerParameters`. Drop langchain-mcp-adapters from the extra. When the llmchat imports fail, report the original ImportError. Closes IBM#7090 Signed-off-by: Sena Köse <kosesena@hotmail.com>
kosesena
force-pushed
the
fix/llmchat-mcp-sdk-v2-tools
branch
from
October 6, 2026 09:41
6626456 to
21bd096
Compare
ScarAnd
pushed a commit
to ScarAnd/mcp-context-forge
that referenced
this pull request
Oct 23, 2026
Signed-off-by: Andrin Enz <Andrin.Enz@ibm.com> # Conflicts: # mcpgateway/services/mcp_client_chat_service.py
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🐛 Bug-fix PR
📌 Summary
LLM Chat cannot start on
mainafter the MCP SDK v2 upgrade (#6868): thellmchatextra still installslangchain-mcp-adapters, which has no release compatible withmcp>=2.0.0.🔁 Reproduction Steps
Closes #7090. Install with
--extra llmchat;uv.lockresolveslangchain-mcp-adapters==0.3.1next tomcp==2.2.0, its import ofmcp.shared.context.RequestContextfails,_LLMCHAT_AVAILABLEbecomesFalseandPOST /llmchat/connectreturns 500 with "dependencies are missing".🐞 Root Cause
langchain-mcp-adapters0.3.1 has no upper bound onmcpand breaks at import time with v2; 0.3.2 pinsmcp<2.0.0, so bumping the floor would make the extra unresolvable.langchain.mcp. That module ships fromlangchain1.4.0, which requireslangchain-core>=1.6andlanggraph>=1.2.11, above the<=1.5.1/<=1.2.0bounds thellmchatextra allows, and the namespace is marked beta.ImportErrorinto a "missing dependencies" message, hiding the cause.💡 Fix Description
MultiServerMCPClientis replaced by a small_MCPToolSourceinmcp_client_chat_service.py. It takes the same{"default": connection}mapping, lists tools withmcp.client.Client(followingnext_cursor), and wraps each tool as a LangChainStructuredTool(input schema = the MCP tool'sinputSchema) that calls it over a fresh session, like the adapter did. An error result raisesToolException.streamable_httpuses the gateway's existingmcpgateway.utils.streamable_http_compat.streamable_http_client(with headers),sseusesmcp.client.sse.sse_client,stdiousesStdioServerParameters.langchain-mcp-adaptersis removed from thellmchatextra;uv.lockonly loses that package.connect(),initialize()andreload_tools()now report the originalImportError.langchain-mcp-adaptersdescribe connecting external agents to the gateway and are left unchanged.📏 Reviewability
triage🧪 Verification
ruff,black,isort,pylint(10.00/10)pytest tests/unit/mcpgateway/services/test_mcp_client_chat_service_tool_source.py(in-process MCP v2 server: list tools with schema, call a tool, error →ToolException, transport selection)pytest tests/unit/mcpgateway/services/test_mcp_client_chat_service*.py tests/unit/mcpgateway/routerstest_sso_router.py::test_handle_sso_callback_keycloak_oversized_id_token_skips_hint_cookie, fails the same way onmain)pytest --doctest-modules mcpgateway/services/mcp_client_chat_service.pyExisting tests that patched
MultiServerMCPClientnow patch_MCPToolSource; three assertions on the old "dependencies are missing" text were updated to the new message.📐 MCP Compliance (if relevant)
✅ Checklist
black,isort)