Skip to content

fix(llmchat): load MCP tools with the MCP SDK v2 client - #7126

Draft
kosesena wants to merge 3257 commits into
IBM:mainfrom
kosesena:fix/llmchat-mcp-sdk-v2-tools
Draft

kosesena wants to merge 3257 commits into
IBM:mainfrom
kosesena:fix/llmchat-mcp-sdk-v2-tools

Conversation

@kosesena

@kosesena kosesena commented Oct 4, 2026

Copy link
Copy Markdown

🐛 Bug-fix PR

Draft: opened so the work on #7090 is visible. Two possible directions were discussed in the issue; this one needs no new dependency and no change to the existing langchain/langgraph pins. Happy to adjust if you'd prefer the other.

📌 Summary

LLM Chat cannot start on main after the MCP SDK v2 upgrade (#6868): the llmchat extra still installs langchain-mcp-adapters, which has no release compatible with mcp>=2.0.0.

🔁 Reproduction Steps

Closes #7090. Install with --extra llmchat; uv.lock resolves langchain-mcp-adapters==0.3.1 next to mcp==2.2.0, its import of mcp.shared.context.RequestContext fails, _LLMCHAT_AVAILABLE becomes False and POST /llmchat/connect returns 500 with "dependencies are missing".

🐞 Root Cause

  • langchain-mcp-adapters 0.3.1 has no upper bound on mcp and breaks at import time with v2; 0.3.2 pins mcp<2.0.0, so bumping the floor would make the extra unresolvable.
  • Upstream closed v2 support (Support for upcoming Python MCP SDK v2.0.0 langchain-ai/langchain-mcp-adapters#578) as superseded by langchain.mcp. That module ships from langchain 1.4.0, which requires langchain-core>=1.6 and langgraph>=1.2.11, above the <=1.5.1 / <=1.2.0 bounds the llmchat extra allows, and the namespace is marked beta.
  • The guarded import turned the ImportError into a "missing dependencies" message, hiding the cause.

💡 Fix Description

  • MultiServerMCPClient is replaced by a small _MCPToolSource in mcp_client_chat_service.py. It takes the same {"default": connection} mapping, lists tools with mcp.client.Client (following next_cursor), and wraps each tool as a LangChain StructuredTool (input schema = the MCP tool's inputSchema) that calls it over a fresh session, like the adapter did. An error result raises ToolException.
  • Transports: streamable_http uses the gateway's existing mcpgateway.utils.streamable_http_compat.streamable_http_client (with headers), sse uses mcp.client.sse.sse_client, stdio uses StdioServerParameters.
  • langchain-mcp-adapters is removed from the llmchat extra; uv.lock only loses that package.
  • When the llmchat imports fail, connect(), initialize() and reload_tools() now report the original ImportError.
  • Non-text tool content (images, resources) is not passed through yet; the previous path returned text for the agent as well. I can extend this if needed.
  • The docs pages that mention langchain-mcp-adapters describe connecting external agents to the gateway and are left unchanged.

📏 Reviewability

  • This PR has one clear purpose
  • The linked issue is not labeled triage
  • Unrelated bugs or improvements are tracked in separate issues/PRs
  • Tests are included with the code they validate
  • If AI-assisted, I understand and can explain the generated changes

🧪 Verification

Check Command Status
Lint (changed source file) ruff, black, isort, pylint (10.00/10) ✅
New tests pytest tests/unit/mcpgateway/services/test_mcp_client_chat_service_tool_source.py (in-process MCP v2 server: list tools with schema, call a tool, error → ToolException, transport selection) ✅
Chat service + routers pytest tests/unit/mcpgateway/services/test_mcp_client_chat_service*.py tests/unit/mcpgateway/routers ✅ 1792 passed, 1 failed (test_sso_router.py::test_handle_sso_callback_keycloak_oversized_id_token_skips_hint_cookie, fails the same way on main)
Doctests pytest --doctest-modules mcpgateway/services/mcp_client_chat_service.py ✅

Existing tests that patched MultiServerMCPClient now patch _MCPToolSource; three assertions on the old "dependencies are missing" text were updated to the new message.

📐 MCP Compliance (if relevant)

  • Matches current MCP spec (uses the MCP SDK v2 client)
  • No breaking change to MCP clients

✅ Checklist

  • Code formatted (black, isort)
  • No secrets/credentials committed

gandhipratik203 and others added 30 commits July 16, 2026 14:28
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
IBM#5637)

* Update fix_file_headers to have generic default AUTHOR, defined COPYRIGHT line, and new line fix, and add pre-commit-config.yaml to execute the fix headers check as part of the pre-commit checks so that the author can see the changes

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* Update all headers so that checks pass

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* Fixed all files with new fix-header rules

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* Update .secrets.baseline after rebase

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* Correcting straggling files after rebase

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* Update CONTRIBUTING doc to be consistent

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* Rebased off main and adjusted findings

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* Correct Makefile to not refer to authors for fix-headers

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

---------

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
 (IBM#5664)

* fix(resources): revert resource name uniqueness constraint from IBM#5158

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* chore: update .secrets.baseline line numbers

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* chore: update .secrets.baseline line numbers after code removals

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* test: update resource tests for name-uniqueness revert

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

---------

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix cwe-117 improper output neutralization logs.

Signed-off-by: AHERDERE <aherdere@ie.ibm.com>

* cwe_117_improper_output_neutralization_logs

Signed-off-by: AHERDERE <aherdere@ie.ibm.com>

* fix cwe-117 improper output neutralization logs

Signed-off-by: AHERDERE <aherdere@ie.ibm.com>

* fix cwe-117 improper output neutralization logs

Signed-off-by: AHERDERE <aherdere@ie.ibm.com>

* fix cwe-117 improper output neutralization logs

Signed-off-by: AHERDERE <aherdere@ie.ibm.com>

---------

Signed-off-by: AHERDERE <aherdere@ie.ibm.com>
Co-authored-by: AHERDERE <aherdere@ie.ibm.com>
…5669)

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
…rget (IBM#5634)

* ci(pyright): add PR-scoped type check workflow and pyright-pr make target

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* ci(pyright): move pyright check into linting-full workflow

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* ci(pyright): move pyrightconfig.json into pyproject.toml [tool.pyright]

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

---------

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…touched file to then become pyright compliant (IBM#5703)

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
…ion errors (IBM#5663)

* fix(transport,middleware): remove manual Content-Length headers causing compression errors

Fixes IBM#5457

When compression is enabled (COMPRESSION_ENABLED=true), manually setting
Content-Length headers before the compression middleware runs causes
"Too much data for declared Content-Length" errors because the declared
size is uncompressed but the actual body is compressed.

Changes:
- Remove manual Content-Length from streamablehttp transport (4 locations)
- Skip sanitization of compressed responses in ValidationMiddleware
- Add 16 regression tests to prevent future occurrences

The compression middleware now correctly sets Content-Length after
compressing the response body, ensuring the header matches the actual
transmitted byte count.

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* addressed the comments

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed headers

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* fix: return 404 for missing server and gateway deletes
Preserve 403 for denied ownership and direct scoped callers.

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed the pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed the comments

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
…#4821)

* fix(auth): unify email extraction across all resource operations

Replaced 23 instances of inconsistent email extraction patterns with
canonical get_user_email() helper to ensure email-over-sub precedence
across all resource CRUD operations.

Problem: Resource update/delete operations used direct user.get('email')
extraction which missed the 'sub' claim fallback, causing 403 Forbidden
errors when tokens had {'sub': 'user@example.com'} structure. This
created ownership check failures because create operations used
get_user_email() (which extracts from 'sub') while update operations
did not.

Solution: Unified all resource operations to use get_user_email() with
consistent email-over-sub precedence, ensuring the identity used for
ownership checks matches across create/update/delete operations.

Files changed:
- mcpgateway/main.py: 18 instances (tools, resources, prompts, gateways, agents, servers)
- mcpgateway/admin.py: 1 instance
- mcpgateway/routers/runtime_admin_router.py: 4 instances
- mcpgateway/routers/oauth_router.py: 2 instances
- tests/unit/mcpgateway/test_email_extraction_consistency.py: New test suite

Closes IBM#4800

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* fix: address PR review findings for email extraction consistency

- Remove 'unknown' sentinel filtering from oauth_router._extract_user_email()
- Add proper 'unknown' rejection at enforcement layer (oauth_router.py:295)
- Simplify admin.py email extraction guard (remove redundant if user check)
- Update remaining user.get('email') patterns to use canonical get_user_email()
  - main.py: config export audit field
  - streamablehttp_transport.py: transport logging identity
  - resource_service.py: plugin context population
  - session_registry.py: session mapping registration
- Fix test to use empty dict instead of dict with only is_admin field

Ensures consistent email-over-sub precedence across entire codebase and
proper identity validation before access checks.

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* fix(auth): address PR review findings for email extraction unification

- Remove 'unknown' sentinel branch from oauth_router.py (fail closed with 401)
- Revert admin.py null-guard to maintain isinstance pattern consistency
- Move lazy import to module top-level in session_registry.py (hot path optimization)
- Complete pattern sweep: fix remaining user.get('email') in llmchat_router, metadata_capture, main.py, admin.py
- Add regression tests for sub-claim token scenario (exact IBM#4800 case)
- Document email-over-sub precedence in 14 endpoint docstrings

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* fix: resolve email extraction test failures in PR IBM#4800

- Filter out 'unknown' sentinel in _extract_user_email() to properly reject missing emails
- Update test expectation for get_user_email() call count (2 calls: logging + authorization)

Fixes test_enforce_gateway_access_requires_email and test_handle_rpc_logging_set_level_populates_email_when_missing

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* remove sentinel from _extract_user_email() function

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* fix: address security review findings from PR IBM#4800

Finding IBM#1 (CWE-284): Fixed dead-code guard in resource_service.py
- Changed 'isinstance(user, dict) and "email" in user' to 'isinstance(user, dict)'
- Allows get_user_email() to handle sub-only tokens like {'sub': 'user@example.com'}

Finding IBM#2 (CWE-287): Fixed sentinel mismatch in oauth_router.py
- Removed _extract_user_email() helper (returned None) to use canonical get_user_email() (returns 'unknown')
- Added explicit filtering: 'if requester_email == "unknown": requester_email = None'
- Applied to both OAuth endpoints (initiate flow, fetch_tools_after_oauth)
- Prevents 'unknown' sentinel from being used as token storage key
- Removed obsolete test for deleted _extract_user_email() function

Finding IBM#3 (CWE-770): Restored HeaderSizeMiddleware registration
- Re-added import: 'from mcpgateway.middleware.header_size_middleware import HeaderSizeMiddleware'
- Restored middleware registration with config-gated activation
- Ensures RFC 6585 § 5 header-flood/OOM defense is active

Finding IBM#4 (CWE-284): Fixed 'unknown' sentinel in session_registry.py
- Added filtering before _register_session_mapping: 'if user_email == "unknown": user_email = None'
- Prevents all unauthenticated sessions from being conflated under 'unknown' owner key
- Maintains proper session isolation under MCP_REQUIRE_AUTH=false

All changes maintain the canonical get_user_email() pattern from AGENTS.md while ensuring
the 'unknown' sentinel is properly filtered at boundaries where None is expected for missing identity.

Closes IBM#4800

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* chore: apply ruff formatting fixes

- Format long strings in alembic migration
- Format HeaderSizeMiddleware log message
- Remove extra blank lines in oauth_router.py
- Fix import order in metrics.py

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* fix: restore unrelated main.py refactors clobbered by earlier rebase

Restores safe_log_user() usage, dataplane_publisher_service naming and
lifecycle, early rate-limiter Redis init, the MCPPathRewriteMiddleware
_apply_mcp_rewrite helper, and metrics_aggregation_interval_seconds
override handling to match origin/main. These were unintentionally
reverted during a prior rebase of this branch (flagged in review as
unrelated refactoring) and are unrelated to the email-extraction fix.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: preserve user_email for admin bypass in list_servers/list_gateways

user_email was zeroed out on admin bypass, which per
_apply_visibility_scope() semantics also hides the admin's own
private servers/gateways, not just other users' rows.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* chore: regenerate secrets baseline after rebase onto origin/main

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: correct copyright header format in email extraction test

CI's correct-file-headers pre-commit hook requires the standard
"Copyright contributors to the MCP-CONTEXT-FORGE project" line.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: restore exception chaining in remove_root, resources/read, prompts/get

These handlers lost `from e` chaining and, in two cases, the `as e`
capture itself, truncating stack traces. Also moved the remove_root
success return back outside the try block.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: preserve admin identity in RPC dispatcher and normalize OAuth requester email

Admin bypass in the list_gateways, resources/list, resources/read,
prompts/list, and prompts/get RPC handlers cleared user_email alongside
token_teams, causing BaseService._apply_visibility_scope() to treat
admins as anonymous and hide their own private rows. Only token_teams
should be relaxed for admin bypass; the email is still needed for
owner matching.

Also normalize the requester email in oauth_router._enforce_gateway_access()
before comparing it against the already-normalized gateway owner, so
mixed-case or whitespace-padded emails are not incorrectly denied
access.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

---------

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com>
…FK deletes (IBM#5659)

* fix(api): use savepoint for personal team cascade delete and reorder FK deletes

Signed-off-by: Gabriel Costa <gabrielcg@proton.me>

* code quality improvements

Signed-off-by: Gabriel Costa <gabrielcg@proton.me>

---------

Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* Update roadmap for 1.0.6

Signed-off-by: Jonathan Springer <jps@s390x.com>

* uplifted the ubi10 base image tagg

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Jonathan Springer <jps@s390x.com>
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* bump-version 1.0.5 -> 1.0.6 / update ubi packages

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* Updated rust aand js

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Removed ui static from pyproject.toml

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Ran pre-commit

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Added Changelog

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Added local benchmark

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* update changelog

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* Ran pre-commit

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Removed Beta Notice

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Fixed changelog and secrets folder

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* fix changelog

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* Fixed the number of PR listed

Signed-off-by: cafalchio <mcafalchio@gmail.com>

---------

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
Signed-off-by: cafalchio <mcafalchio@gmail.com>
Co-authored-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
)

* ci: enable all arch build for workflow_dispatch manual trigger

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* ci: restrict ppc64le build to workflow_dispatch only

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* ci: fix null coercion, stale comments and wheels gate robustness

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

---------

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…BM#5610)

* feat(search): add versioned GET /v1/search unified search endpoint

Expose unified cross-entity search at a stable, versioned, non-admin path
so client-facing global search no longer depends on GET /admin/search, which
is gated on admin.dashboard and slated for deprecation.

- Extract the orchestration body of admin_unified_search into a shared,
  permission-agnostic helper perform_unified_search(); admin_unified_search
  becomes a thin admin.dashboard-gated wrapper (behavior unchanged).
- Add mcpgateway/routers/search.py with GET /search (authentication only, no
  admin-panel gate). Per-entity RBAC and token scoping still enforce
  visibility inside the shared helper, and users remain gated on
  admin.user_management.
- Register the router in the v1 assembly (always-on) so it serves at
  /v1/search and the legacy /search alias, independent of the admin API.
- Add router tests covering unauthenticated access, invalid entity_types,
  users protection, team_id scoping, limit/limit_per_type, and response-shape
  parity with /admin/search.

Related: IBM#5520
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* docs(search): trim verbose comments and docstrings

Condense the added module/function docstrings and inline comments for the
/v1/search endpoint while keeping the rationale (why it exists, security model,
always-on registration). No behavior change; docstring coverage stays at 100%.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* test(search): add integration test for /v1/search endpoint

Exercise GET /v1/search and the legacy /search alias through the full ASGI
stack (routing, auth middleware, orchestration) on the real app, stubbing only
the per-entity admin_search_* DB queries. Covers authenticated grouped
response, legacy-prefix parity, limit_per_type forwarding, and 401 deny paths
for both prefixes. Mirrors the pattern from tests/integration/test_resource_management.py.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* test(search): add real-data token-scoping regression for /v1/search

Seed real teams, a non-admin member with a real global tools.read role, and
public/team-private tools, then run the real PermissionService and
admin_search_tools (nothing stubbed at the security boundary) to prove
/v1/search enforces token/team visibility:

- token scoped to team-B hides team-A's private tool while still returning
  public and team-B tools (deny with a positive control)
- same user without token narrowing sees team-A too (confirms the deny was the
  token, not missing access)
- non-admin requesting tools,users drops users but still returns tools

Identity is injected; PermissionService, admin_search_tools, and DB rows are
real. Full JWT-middleware validation is a separate concern.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* test(search): function-scope _auth_client to stop PermissionService patch leak

The _auth_client fixture was module-scoped and wrapped its body in a
patch(PermissionService=AlwaysGrant) context, so once the mocked wiring tests
ran, the patch stayed active until module teardown and leaked into the later
real-PermissionService scoping tests during a full-file run. Function scope
tears the patch (and the fixture's dependency_overrides) down after each test,
so the real-data tests exercise the genuine PermissionService. Verified by
temporarily removing the seeded tools.read role: the real-data tests then fail
on a full-file run (403 -> empty results), confirming real RBAC is now active.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* fix(search): register /search in _LEGACY_PREFIXES for deprecation headers

The unified search router is always mounted, so its legacy unversioned alias
(/search) must be listed in _LEGACY_PREFIXES to receive Sunset/Deprecation
headers like every other legacy shim. Without it, the parity guard
test_legacy_prefixes_match_assembled_routers fails.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* test(search): cover real per-entity denial not collapsing unified search

Add a real-data test for the "one entity's denial must not fail the whole
search" guarantee: user_b has tools.read but not servers.read, so a genuine
403 from admin_search_servers flows through _safe_entity_search and becomes
empty results. Seed a public server matching the query so an empty servers
list proves the denial rather than missing data, and assert tools still
return (search did not collapse). Verified empirically by temporarily
granting servers.read and confirming the seeded server then appears.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* fix(search): allow scoped tokens to reach /v1/search in TokenScopingMiddleware

TokenScopingMiddleware default-denies unmapped paths, so /v1/search
(normalized to /search) returned 403 for any token with non-wildcard
permission scopes before the handler ran, making the endpoint unusable for
granularly-scoped API tokens.

Treat /search as authenticated-only at the middleware layer: it spans many
entity types, each guarded by its own @require_permission, and per-entity
denials are suppressed downstream by _safe_entity_search, so a single required
permission cannot be chosen here without wrongly rejecting a valid token. Other
token restrictions (team ownership, server_id, IP, time, usage limits) still
apply. /admin/search keeps its admin.dashboard gate.

Add regression tests: direct permission-map assertions plus a real signed
scoped JWT through the live middleware (200 for /v1/search, 403 for an unmapped
control path).

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* fix(search): token-scope team results in unified search (non-admin)

admin_search_teams' non-admin path returned every DB membership and ignored
token scope, so a token narrowed to team B still leaked team A's id, name,
slug, description, visibility, and status through /v1/search (and
/admin/teams/search). Tools already narrow via _get_user_team_ids; teams
bypassed it.

Filter the non-admin team results through the same token-aware helper
(_get_user_team_ids, which honors token_teams/_cached_team_ids). The caller's
own personal team stays visible; for unscoped tokens the intersection is a
no-op. Uses getattr for is_personal so partial team mocks keep working.

Add real-data regression tests through /v1/search: a token scoped to team B
does not see team A, and without narrowing the same member sees both
(confirming the deny is the token, not missing membership). Verified
empirically by disabling the filter and observing team A leak.

The admin branch (which intentionally returns all teams) is unchanged here and
tracked separately.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* test(search): add real-JWT e2e for /v1/search

Add an end-to-end test that drives a real signed JWT through the full auth and
token-scoping middleware (no dependency overrides) against the real app on a
temp DB — the path unit/integration tests don't exercise, mirroring the manual
curl verification.

Three minimal, high-value cases:
- unauthenticated request -> 401 (real auth middleware)
- non-admin JWT -> 200 and the member's tools.read role returns the seeded
  public tool (proves no admin needed, real RBAC, real data)
- permission-scoped JWT (tools.read) -> 200 (token-scoping middleware allows
  /search)

Verified empirically: disabling the /search allow makes the scoped case fail
with 403, and the deny/happy contrast confirms auth is genuinely enforced.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* chore(search): align file headers with updated repo convention

Apply the current .github/tools/fix_file_headers.py header format (Copyright
contributors line, no Authors line) to the search source and test files, which
the correct-file-headers pre-commit hook flags after the rebase onto main.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* fix(search): defer mcpgateway.admin import so it stays off the startup path

The always-on search router imported mcpgateway.admin at module top level, so
admin.py (and its module-level AdminStatsCache/AuthCache/MetricsBufferService/
Argon2/ContentSecurity instances, ~1.8s) loaded at startup even when
MCPGATEWAY_ADMIN_API_ENABLED=false — contradicting the endpoint's own
"no admin dependency" framing.

Take team_id as a raw Query in the signature (instead of
Depends(_validated_team_id_param)) and import _validated_team_id_param and
perform_unified_search lazily inside the handler, so admin.py loads only on the
first /v1/search call rather than at import. Verified: importing the search
router no longer pulls mcpgateway.admin into sys.modules. Real HTTP behavior is
unchanged (valid team_id passes, invalid -> 400); the unit test now uses a valid
UUID since the handler runs the real validation.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* chore(search): drop unused logger and cover public-only token scope

Address two non-blocking review notes:
- Remove the unused module-level logger (and its logging import) from
  routers/search.py.
- Add an integration test for the public-only scope (token_teams=[], what a
  token with no teams claim normalizes to): the caller, though a member of both
  seeded teams, sees only the public tool and not the team-private ones —
  the degenerate scope the explicit-team-list tests did not cover.

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>

* chore(search): mark test hex UUID as allowlisted for detect-secrets

The 32-char hex UUID literal used as a valid team_id in the router test trips
the detect-secrets pre-commit hook (high-entropy hex). Suppress inline with

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
# pragma: allowlist secret per the repo convention for Python files.

---------

Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
* Add v1 catalog API

Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>

* fix: removing duplicate endpoint

Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>

---------

Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* Fixed lint

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Added skip check

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Corrected tags

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* added correct version

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Addressed review issues

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Addressed plugin tmp path

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* add override for ocp deployment so it can assign UID from its allowed range instead of hardcoded 10001 and add podSecurityContext for override to job-migration.yaml and fic for volumemount indentation in deployment-mcpgateway.yaml

Signed-off-by: Claudia Gray <claudiag082004@gmail.com>

* Ran pre-commit

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* secrets

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Update secrets baseline

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* Fixed test waiting for latest

Signed-off-by: cafalchio <mcafalchio@gmail.com>

* Fixed breaking test

Signed-off-by: cafalchio <mcafalchio@gmail.com>

---------

Signed-off-by: cafalchio <mcafalchio@gmail.com>
Signed-off-by: Claudia Gray <claudiag082004@gmail.com>
Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
Co-authored-by: Claudia Gray <claudiag082004@gmail.com>
Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
IBM#5244)

* fix: resolve authorization_code OAuth gateway offline issues

Fix bug IBM#5237 where authorization_code OAuth gateways would become
permanently offline due to health check assumptions and destructive
token handling.

Changes:
- Decouple health checks from token ownership for authorization_code flow
- Treat 401/403 responses as 'gateway reachable' instead of failures
- Make client_secret decryption failures explicit (raise OAuthError)
- Respect omit_resource flag during token refresh
- Only delete tokens on invalid_grant errors (RFC 6749 §5.2)
- Add gateway reactivation on 401/403 when previously unreachable
- Update last_seen timestamp even on auth failures

Tests:
- Added test_oauth_authorization_code_health.py (8 tests)
- Added test_gateway_service_401_reactivation.py (5 tests)
- Updated existing OAuth tests to match new behavior
- All 172 OAuth-related tests passing with no regressions

Closes IBM#5237

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* fix pylint issues

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>

* strengthen authorization_code gateway health-check and token refresh logic

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fixed pre-commit/tests errors

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* wrapped streamable exception group and fixed client-description fail closed issue

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fixed headers

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fixed dead try/except issue

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* added documentation for oauth updates

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix secretsbaseline

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix pre-commit

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
Co-authored-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
)

* feat(teams): support bulk member creation in TeamCreateRequest

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* fix(lint): suppress W0201 false positives on SQLAlchemy column assignments in _upsert_membership

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* fix: normalise seed email to lowercase before lookup and invitation creation

EmailStr preserves the supplied casing (e.g. ALICE@example.com), but
EmailUser records are stored in canonical lowercase. _normalize_member_seeds
built _NormalizedSeed with the un-lowercased address, so:

* EmailUser.email.in_([...]) missed existing users with mixed-case input
* known_users.get(seed.email) returned None even when the user was found
* the seed fell through to the invitation path, storing the uppercase
  address — making accept_invitation() always fail its exact email
  comparison against the lowercase authenticated identity.

Fix: canonicalise the email to .strip().lower() at the point it enters
_NormalizedSeed. The separate 'key' variable was made redundant and removed.

Adds three regression tests in TestMixedCaseEmailNormalisation:
- uppercase active user is correctly matched and added as a member
- uppercase unknown address invitation is stored in lowercase
- mixed-case duplicate pair is correctly rejected as a duplicate

Closes IBM#5626

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* docs+test: document POST /teams and add deny-path coverage for seeded creation

Addresses blocking review findings from IBM#5626:

docs/docs/manage/api-usage.md:
- Add POST /teams section with TeamCreateRequest schema table
- Document TeamMemberSeed fields including lowercase normalisation note
- Document server-side seed routing rules (member vs invitation decision)
- Document configuration flags: ALLOW_TEAM_CREATION, ALLOW_TEAM_INVITATIONS,
  MAX_TEAM_MEMBER_SEEDS, MAX_MEMBERS_PER_TEAM
- Document TeamCreateResponse as a superset of TeamResponse
- Document SeededMemberResponse and SeededInvitationResponse fields
- Add curl examples for minimal create and create-with-members
- Add full JSON response example with members_added and invitations_sent
- Add error response table (400/403/422) with example detail messages

tests/unit/mcpgateway/test_team_governance_flags.py (TestCreateTeamWithMembersDenyPaths):
- test_unauthenticated_create_with_members_raises_401
- test_team_creation_disabled_with_members_raises_403
- test_invitations_disabled_with_unknown_seed_raises_400

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* fix: secrets baseline

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* Address comments

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* fix: secrets baseline

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* refactor(teams): address review feedback on bulk member seeding

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* feat(teams): make MAX_TEAM_MEMBER_SEEDS a configurable setting

The create-team seed ceiling was a hardcoded module constant but was
documented in the config table alongside real environment variables,
so operators could reasonably expect to tune it via env and be silently
ignored.

Promote it to a `Settings` field (default 500), source the schema
constant from it, add it to `.env.example`, and list it in the
configuration reference. The per-team `MAX_MEMBERS_PER_TEAM` limit still
applies underneath.

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* fix: pre-commit check on CI

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

* fix: secrets baseline

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>

---------

Signed-off-by: Marek Dano <Marek.Dano@ibm.com>
Co-authored-by: Marek Dano <Marek.Dano@ibm.com>
* refactor: remove Granian HTTP server option

Granian (Rust-based ASGI server) is removed as an HTTP server option.
Gunicorn with Uvicorn workers is now the sole server.

Code & build:
- Delete run-granian.sh (420 lines)
- Remove granian pip extra from pyproject.toml
- Drop granian from Containerfile and infra/wheels/Containerfile
- Hardcode gunicorn in docker-entrypoint.sh, remove HTTP_SERVER env var
- Regenerate uv.lock (drops granian + setproctitle)

Makefile:
- Remove serve-granian, serve-granian-ssl, serve-granian-http2 targets
- Remove container-run-granian deprecated targets
- Remove CONTAINER_HTTP_SERVER variable and granian lock cleanup

Docker Compose:
- Remove HTTP_SERVER and all GRANIAN_* env vars from all compose files
- docker-compose-performance.yml: flip default from granian to gunicorn

Helm chart:
- Remove HTTP_SERVER and GRANIAN_* from values.yaml, schema, and README

Docs:
- Supersede ADR-0025 (Granian HTTP Server)
- Remove granian sections from tuning, scale, TLS, performance, and roadmap docs
- Remove granian cross-references from ADR-0024 and ADR-0026

Signed-off-by: Jonathan Springer <jps@s390x.com>

* Update .secrets.baseline

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

---------

Signed-off-by: Jonathan Springer <jps@s390x.com>
Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
* chore: remove obsolete redis_fallback config references

Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com>

* test: address review on config-validation guard (anchor path, assert constructed)

Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com>

* fixed pre-commit

Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com>

* fixed pre-commit

Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com>

---------

Signed-off-by: Yosief Eyob <yosiefogbazion@gmail.com>
…BM#4636)

* initial commit

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* down revivion

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* chore: remove review documentation files

Remove review-only documentation that should not be in the main codebase.

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* style(migration): remove unused sqlalchemy import in d21698ae4a19

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* add pragma: allowlist secret

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: add coverage for IntegrityError race condition handling in role service

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* feat: add unique constraints to RBAC models for test database schema

Add partial unique indexes to Role and UserRole ORM models via __table_args__
to ensure test databases created with create_all() have the same constraints
as production databases created through Alembic migrations.

This fixes integration test coverage by allowing IntegrityError race condition
handling tests to trigger real constraint violations.

Changes:
- Role: Add uq_roles_name_scope_active index (one active role per name+scope)
- UserRole: Add two indexes for NULL and non-NULL scope_id cases
- Simplify test mocks to use natural IntegrityError from database

Coverage: 100% on role_service.py changed lines (previously 62%)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* ci: add --with-integration flag to pytest workflows to enable integration tests

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: add unit tests for role service IntegrityError race condition handling

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* revert: remove --with-integration flag from workflow files

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: update alembic down_revision to w7x8y9z0a1b2 to prevent multiple heads

- Changes down_revision from 9fb98535724d to w7x8y9z0a1b2
- Ensures proper migration chain after rebase with main

Signed-off-by: Rakhi Dutta <rakhidutta@users.noreply.github.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(rbac): exclude expired assignments from unique constraints and use deterministic deduplication

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* rebase and down revision fix

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* soft_delete

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* pylint fix

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* added try and exception block

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test coverage

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(rbac): remap user_roles before role dedup, prefer unexpired assignments, fix W1203 logging, add migration tests

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(rbac): fix stale migration docstring/comments, drop unused pytest import, remove blank line

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(rbac): set down_revision to b6c7d8e9f0a1 to resolve duplicate alembic heads

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* chore: fix file headers — add encoding/Location/Copyright/SPDX lines, remove stale Authors

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* chore: refresh .secrets.baseline line numbers after rebase

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

---------

Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Signed-off-by: Rakhi Dutta <rakhidutta@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Signed-off-by: lucarlig <luca.carlig@ibm.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
… counter on recovery (IBM#4862)

* fix: increase health check timeout from 5s to 30s and reset failure counter on recovery

Root cause: 24x timeout mismatch between periodic health checks (5s)
and Test button (120s) caused false 'Offline' status for slow gateways.

Changes:
- mcpgateway/config.py: health_check_timeout 5 -> 30,
  gateway_health_check_timeout 5.0 -> 30.0
- mcpgateway/services/gateway_service.py: reset _gateway_failure_counts
  on gateway recovery to prevent flickering accumulation
- docs: update documented defaults in configuration.md, scale.md, tuning.md
- .env.example: update commented-out examples

Closes IBM#4785

Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com>

* fixed documentation with proper default value

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com>
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* fix: harden JWT secret enforcement, remove hardcoded secrets, add startup validation (GHSA-8pcq-mx48-hjvj)

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* added init_secrets options for dev

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* restored changelog
- addressed the comments

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* chore: use upstream package-lock.json and .secrets.baseline

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* addressed comments and hardened helm files as well

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* added tests

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fixed tests

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fixed tests and audited secrets

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* added setup make target to path .env file

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix error message on start up to give dev options

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* added updated meessage

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* added BASIC_AUTH to initsecrets

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fixed tests

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix: quote compose :? guards, revert unreachable translate.py fallback, add logging import

docker-compose.yml / docker-compose-embedded.yml: wrap all 20 unquoted
\ environment entries in double-quotes.  The error message
contains 'Run: ' (colon-space) which YAML parses as a map key without
quoting, causing 'unexpected type map' on docker compose up even when the
secret is set.  Quoting makes docker compose config exit 0.

mcpgateway/translate.py: remove the SecurityConfigurationError try/except
fallback added in the advisory fix.  The fallback was permanently
unreachable: LoggingService at line 150 already imports settings at module
level unconditionally (logging_service.py:27 -> config.py), so
validate_security_combinations fires before the guarded block.  Reverted
to the pre-existing ImportError-only fallback which correctly handles the
only reachable failure mode (config package absent at import time).

tests/unit/mcpgateway/test_translate.py: remove
test_translate_security_config_error_fallback which exercised the now-
deleted unreachable code path.

tests/unit/mcpgateway/test_config.py: add missing 'import logging' so the
four primary-worker heartbeat tests (inserted during rebase conflict
resolution) resolve logging.WARNING without NameError.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* pre-commit spacing and added secrets baseline

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fixed pre-commit

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* removed e2e harness

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

---------

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix detect secrets

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix linting

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* Fix plugin integration tests

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

---------

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
IBM#5809)

The Docker Security Scan merge-queue gate (Grype, severity-cutoff high)
fails on CVE-2026-6477 in libpq 16.11-3.el10_1, shipped by the runtime
stage's ubi10/ubi-minimal:10.2-1784581369 base. Red Hat has since
published libpq-16.14-1.el10_2 with the fix, but the pinned dated tag
predates that erratum and Docker's layer cache keeps serving the
vulnerable package on rebuilds even though `microdnf upgrade -y`
already runs in that layer.

Bump UBI_MINIMAL to the 2026-07-21 dated snapshot
(ubi10/ubi-minimal:10.2-1784669047), which carries the patched libpq
and busts the stale cache layer. UBI_BASE and NODEJS_IMAGE are
untouched since libpq is only installed in the UBI_MINIMAL-based
runtime stage.

Verified locally: rebuilt the image with this pin, confirmed
`rpm -q libpq` reports 16.14-1.el10_2, regenerated the SBOM (Syft
v1.42.3) and re-ran Grype v0.110.0 with the repo's .grype.yaml and the
same --only-fixed --fail-on high flags the CI job uses - clean, no
findings.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
vishu-bh and others added 27 commits September 29, 2026 10:24
* feat: add invitee invitations API

Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>

* test: address invitation API review feedback

Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>

* chore: refresh secrets baseline

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

---------

Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: disable redirect following in SSE gateway health-check client

get_isolated_http_client defaults to follow_redirects=True. The SSE
health-check call site omitted the override, making it the only client
in gateway_service.py that followed redirects.

Two security consequences:
- SSRF: a user with gateways.create could register an SSE gateway whose
  URL 302-redirects the health check to an internal or cloud-metadata
  host, bypassing the SSRF validation that only checks the registered URL.
- Credential forwarding: for authheaders gateways, httpx 0.28.1 preserves
  custom credential headers (e.g. X-API-Key) across redirects. The stored
  credential was forwarded to the attacker-controlled redirect destination.

All sibling clients in gateway_service.py already pin follow_redirects=False
(get_httpx_client_factory at line 4943, and lines 7396, 7563, 8131, 8540).
Add the same flag to the SSE health-check client to match the documented
SSRF-hardening invariant.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* test: assert follow_redirects=False in SSE health-check client (GHSA-3frw-wjxx-2p6m)

Add TestSSEHealthCheckNoRedirects to test_gateway_service_health_oauth.py.
The test captures kwargs passed to get_isolated_http_client and asserts
follow_redirects=False is present, so accidental reversion of the SSRF
fix is caught immediately.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

---------

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: add a same-origin and pinning gate for DCR outbound URLs

Add a validation gate for URLs that the DCR flow sends requests to. The gate
requires the issuer origin and pins the connection to the address resolved
during validation.

Refs IBM#478

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: move DCR tests to the isolated HTTP client seam

Point the DCR tests at the isolated client factory. Pinned requests need an
isolated client, so the shared client seam no longer receives them.

Refs IBM#478

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: pin both AS metadata discovery requests

Route both discovery requests through the outbound URL gate. Each request now
dials the address resolved during validation, which closes the rebinding
window on the issuer hostname.

Refs IBM#478

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: pin the DCR client registration request

Route the registration POST through the outbound URL gate. Guard the error
branch against a non-JSON upstream body, which previously escaped the handler.

Refs IBM#478

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: pin the DCR registration update and delete requests

Route both management requests through the outbound URL gate. The registration
access token no longer travels to a URL outside the issuer origin.

Neither method has a caller in mcpgateway today. Both are public and both send
a bearer token, so both are fixed now.

Refs IBM#478

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: clear the AS metadata cache around DCR integration tests

The metadata cache is module-global and lives for the process, so a document
cached by another test module leaked into these tests. Clear it before and
after each test.

Refs IBM#478

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: raise DcrError on non-JSON upstream bodies and isolate the metadata cache

Guard the remaining response.json() calls in dcr_service.py so each method
keeps its documented DcrError contract; json.JSONDecodeError subclasses
ValueError, not httpx.HTTPError, so it was escaping the surrounding
except httpx.HTTPError handlers. Replace the unit test module's 15 inline
_metadata_cache.clear() calls with one autouse fixture that also clears
after each test, so the module stops leaking cached AS metadata into
whatever test module runs next. Move the stray if __name__ == "__main__"
guard to the end of the file, and bring the integration fixture's
stub_dns_for_pinning docstring in line with its unit-test twin.

Refs IBM#478

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: cover the non-JSON-body and incomplete-pinning branches in dcr_service

CI's diff-cover gate requires 90% coverage of changed lines and reported
84.1% on dcr_service.py. Every missing line was an except ValueError guard
added to close a bare-ValueError escape, or an OIDC-fallback issuer-mismatch
branch, or the defensive incomplete-pinning check in
_prepare_pinned_request, none of which had a covering test.

Add nine tests: one for the incomplete-pinning guard, three for the two
discovery paths' non-JSON and issuer-mismatch branches, two for
register_client's success and error non-JSON branches, and three for
update_client_registration's success, JSON-error, and non-JSON-error
branches. diff-cover now reports 100% on the changed lines.

Refs IBM#478

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* docs: correct the dcr_service pinning docstrings

Describe the origin check as the SSRF gate for metadata-document URLs and a
consistency check for issuer-built URLs. Document that a URL policy refusal
skips the OIDC fallback. Name the SNI hostname in the extensions attribute.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

---------

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…6879)

* fix(security): normalize /v1 paths for rate limit tier matching

Normalize request paths in RateLimitMiddleware by stripping root path,
API aliases, and /v1 prefixes before pattern matching. This prevents
versioned endpoints from falling through to the LOW tier default.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* refactor(rate-limit): clean up stale /v1 branches and remove double normalization

Remove dead version guards from tier patterns that were unreachable after
_normalize_path_for_matching strips /v1 from every path before matching:
- SESSION_REFRESH: ^(/v1)?/auth/refresh$ -> ^/auth/refresh$
- CRITICAL_INVITATION: ^/(?:v1/)?teams/... -> ^/teams/...

Extract path normalization in dispatch so it runs once per request.
Introduce _get_tier_for_normalized and _get_tier_name_for_normalized as
the inner matching primitives; get_endpoint_tier and _get_tier_name
delegate to them, preserving the public API. Update the forged-HMAC test
to patch _get_tier_for_normalized, which is the method dispatch now calls
directly.

Add deny-path regression assertions to test_endpoint_tier_matches_v1_versioned_paths
confirming that /v1/auth/email/login, /v1/tokens, /v1/oauth/token, and
/v1/rbac/roles never return the 500 rpm LOW default.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

---------

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…on startup (IBM#6881)

* fix(security): read URL scheme allowlist at call time and scan DB on startup

Closes IBM#6264.

Gap 1: ALLOWED_URL_SCHEMES was frozen at class-definition time. Replace the
cached class-attribute read in SecurityValidator.validate_url() with a direct
read of settings.validation_allowed_url_schemes so the live LazySettingsWrapper
is consulted on every call instead of the stale copy.

Gap 2: pre-existing gateway, tool, and A2A agent records were never checked
against the allowlist. Add _check_url_scheme_compliance() called from lifespan
on startup. It queries all enabled records, logs a WARNING for each violation,
and raises SystemExit when STRICT_SCHEME_ENFORCEMENT=true (default: false).

Config: add strict_scheme_enforcement: bool = False.
Docs: update .env.example with restart semantics and the new flag.
Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* test(security): add validation_allowed_url_schemes to settings mocks

validate_url() now reads settings.validation_allowed_url_schemes at call
time. Mocks and stub classes that patch mcpgateway.common.validators.settings
must supply this attribute or the scheme check fails before the SSRF logic
under test is reached.

Add validation_allowed_url_schemes = ["http://", "https://", "ws://", "wss://"]
to every affected mock:
- tests/security/test_gateway_test_ssrf_protection.py (18 mock_settings blocks)
- tests/unit/mcpgateway/routers/test_llmchat_router.py (StrictSSRFSettings)
- tests/unit/mcpgateway/test_admin.py (StrictSSRFSettings)
- tests/unit/mcpgateway/validation/test_validators_advanced.py
  (ssrf_settings mock and _settings() helper)

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* fix: use top-level db imports in _check_url_scheme_compliance to resolve pylint reimport warnings

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* test: add unit tests for _check_url_scheme_compliance to satisfy diff-cover

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* fix: address review findings for URL scheme compliance

Sanitize URLs via sanitize_url_for_logging() before logging (CWE-532).
Wrap DB scan in try/except so transient errors do not abort startup.
Document restart requirement in .env.example and configuration.md.
Patch get_settings() directly in test fixtures for call-time reads.
Delete dead ALLOWED_URL_SCHEMES class attribute.
Add test_disabled_records_excluded asserting filter call args.
Reword internal comment as plain text.
Add URL Scheme Allowlist section to ops docs.

Closes IBM#6264

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* fix(security): fail closed in strict mode when DB compliance scan errors

_check_url_scheme_compliance() caught SQLAlchemyError and returned even
when STRICT_SCHEME_ENFORCEMENT=true, letting the process start without
verifying enabled records against the URL scheme allowlist (CWE-693).

Raise SystemExit in strict mode when the database query fails. Retain
warn-and-continue for the default non-strict path.

Add tests for both strict and non-strict DB error paths, and cover the
SIGHUP settings-cache-clear failure branch (signal_handlers.py:73-74).

Closes IBM#6264

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* docs: clarify SIGHUP vs restart for URL scheme allowlist changes

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* fix: resolve 5 review findings for issue IBM#6264 PR

- Fix deny-path test URL to use /v1/mcp-servers/test prefix
- Remove ponytail: agent-tooling comments from validators.py
- SIGHUP: validate trial Settings before evicting cached instance
- Per-table try/except in startup compliance scan to preserve partial results
- Remove stale ALLOWED_URL_SCHEMES class attribute (zero in-tree readers)

Closes IBM#6264

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* fix(startup): harden URL scheme compliance scan

Catch ValueError from sanitize_url_for_logging. urlparse raises on a
malformed netloc, for example a stray bracket from an IPv6-style host.
The error escaped the per-table SQLAlchemyError handler and aborted
startup even when STRICT_SCHEME_ENFORCEMENT was false.

Roll back the session in the per-table SQLAlchemyError handler. A
ProgrammingError or DataError leaves the transaction aborted, so the
next table query also failed and dropped real violations.

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* chore: refresh secrets baseline after rebase

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

---------

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
… fork (IBM#6688)

WORKER_ID was captured once at module import, so under gunicorn's
--preload every forked worker inherited the master's id and subscribed
to the same Redis pub/sub channel, turning point-to-point forwarding
into a per-container broadcast. Replace the constant with
get_worker_id(), which derives the id from the live PID at call time,
update all call sites (main.py, streamablehttp_transport.py), and drop
the now-redundant post_fork rebind in gunicorn.config.py.

Closes IBM#6687

Signed-off-by: Jonathan Springer <jps@s390x.com>
* initial fix to prevent conflicting tool name register

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* Fix visibility projection in update_gateway().
Add missing Admin, catalog, and import collision handling.
Add focused service and endpoint tests, then live gateway test.

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* improved the coverage

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed review comments

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed precommit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed precommit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed healthcheck recovery and federated too update

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed lint issue

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* improved test coverage and health-check revovery

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
IBM#6875)

* fix: enforce MCP Streamable HTTP Origin validation to prevent DNS rebinding

Implements the MCP 2025-11-25 §transport-security requirement: servers
MUST reject a present-but-invalid Origin header with HTTP 403.

Changes:
- Add _check_mcp_origin() and _check_mcp_host() in streamablehttp_transport.py.
- Apply to all HTTP methods (POST, GET, DELETE) and both /mcp and
  /servers/{id}/mcp routes via MCPOriginHostGate in main.py.
- Add mcp_allowed_origins and mcp_allowed_hosts settings (Settings).
  Both default to empty set so enforcement is opt-in (backward-compatible).
  Parsed from MCP_ALLOWED_ORIGINS / MCP_ALLOWED_HOSTS env vars via
  JSON array or CSV string.
- Document both env vars in .env.example near ALLOWED_ORIGINS.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* test: bring diff coverage for origin validation to 100%

Add tests for _parse_mcp_origin_sets (all string/collection/fallback
branches) and SessionManagerWrapper SDK security-settings construction
to satisfy the diff-cover ≥90% gate.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix(test): address PR 6875 review findings

- Remove unused 'patch' import (Ruff F401)
- Drop unused 'wrapper' variable in SDK security settings test (Ruff F841)
- Add test: security_settings=None passed to SDK when mcp_allowed_hosts is empty
- Add deny-path test: custom origin gate enforces 403 when mcp_allowed_hosts is
  unset and mcp_allowed_origins is configured

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix: address PR 6875 review findings

- Wire _check_mcp_origin() into RustMCPRuntimeProxy.handle_streamable_http()
  before the Rust forward; loopback-exempted; returns HTTP 403 with
  deprecation headers. Closes blocking finding IBM#1.

- Omit allowed_origins from TransportSecuritySettings when
  mcp_allowed_origins is empty. The SDK treats an empty list as
  reject-all; Origin enforcement in that case falls solely to
  _check_mcp_origin(). Closes blocking finding IBM#2.

- Add request-level split-config tests (mcp_allowed_hosts set,
  mcp_allowed_origins empty) and three Rust-proxy Origin deny-path
  tests. Closes functionally-impacting finding IBM#3.

- Document in .env.example that MCP_ALLOWED_ORIGINS covers Rust-mode
  traffic via the Python proxy layer; the Rust crate has no independent
  validation. Closes functionally-impacting finding IBM#4.

- Comment at _check_mcp_origin call site notes exact-string vs host:*
  wildcard-port mismatch with the SDK gate (minor IBM#5).

- Remove duplicate loopback-forward detection; is_internally_forwarded
  derived once, reused for Origin guard and session-affinity (minor IBM#6).

- Extract _parse_origin_set() static method; _parse_allowed_origins and
  _parse_mcp_origin_sets both delegate to it (minor IBM#7).

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix: address round-2 PR 6875 review findings

- Drop SDK TransportSecuritySettings entirely; implement gateway-owned
  _check_mcp_host() alongside _check_mcp_origin() using the same
  opt-in exact-match pattern. Both Python and Rust paths now use the
  gateway gate exclusively — no SDK allowed_origins/allowed_hosts
  coupling. Closes blocking finding IBM#1 (round 2).

- Gate the loopback + x-forwarded-internally bypass on
  mcpgateway_session_affinity_enabled. When affinity is disabled
  (the default) the header provides no bypass, closing the post-DNS-
  rebind spoofing vector. Closes blocking finding IBM#2 (round 2).

- Extract _parse_mcp_scope_headers() shared helper used by both
  streamablehttp_transport and rust_mcp_runtime_proxy, eliminating
  the duplicated header-extraction block. Closes minor finding IBM#5
  (round 2).

- Add sanitize_for_log() to the Rust proxy rejected-Origin/Host log
  lines (CWE-117). Closes functionally-impacting finding IBM#4 (round 2).

- Update .env.example MCP_ALLOWED_HOSTS block: document exact-match
  semantics, gateway-gate coverage for both transport modes, remove
  the now-obsolete SDK HTTP 421 warning. Closes functionally-impacting
  finding IBM#3 (round 2).

- Rename allow_empty_string -> coerce_non_iterable_to_empty in
  _parse_origin_set to accurately describe what the flag controls.
  Closes minor finding IBM#6 (round 2).

- Replace stale SDK-wiring tests with _check_mcp_host unit tests,
  Host-gate integration tests, and affinity-bypass tests
  (enabled/disabled). Patch forwarded-path tests in
  test_streamablehttp_transport.py to set
  mcpgateway_session_affinity_enabled=True — these tests exercise
  the affinity routing path which requires affinity to be on.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix: address round-3 PR 6875 review findings

Blocking IBM#1/IBM#2 — move Origin/Host gate to the public /mcp mount level.
Introduce MCPOriginHostGate (streamablehttp_transport.py) as an ASGI
wrapper mounted at /mcp in main.py. The gate fires before any ingress
dispatch (Python, rust-internal, rust-public), removing the per-transport
duplication and closing the gap where InternalTrustedMCPTransportBridge
and RustMCPPublicProxyApp had no gate. The /_internal/mcp/transport bridge
is mounted separately and intentionally bypasses the gate (trusted Rust
sidecar traffic only). Remove gate from handle_streamable_http and
RustMCPRuntimeProxy.handle_streamable_http.

Suggestion IBM#8 — Origin/Host header extraction now uses hdrs.get('origin')
(is None semantics), not the 'or None' coercion that silently treated an
empty-string header as absent. Add two unit tests per field.

Suggestion IBM#9 — Drop the _parse_mcp_origin_sets validator. Extend the
existing _parse_allowed_origins @field_validator to cover all three fields
(allowed_origins, mcp_allowed_origins, mcp_allowed_hosts). Update the
test_config.py calls from _parse_mcp_origin_sets to _parse_allowed_origins.

FI IBM#3 — Fix the mcp_allowed_hosts comment in config.py: remove the stale
SDK-level and host:* wildcard references.

FI IBM#4 — Remove stale 'Closes #<issue>' placeholder and the mistaken
'Closes IBM#421' (bandit zero-finding issue) from earlier commit messages.

FI IBM#5 — Add tests/live_gateway/mcp/test_origin_host_enforcement.py: three
black-box HTTP tests against a running gateway (unlisted Origin returns
403, allowlisted Origin is accepted, missing Origin is always accepted).
Tests self-skip when MCP_ALLOWED_ORIGINS is not configured.

FI IBM#7 — Update .env.example: clarify that enforcement covers all /mcp
ingress modes and that /_internal/mcp/transport is exempt. Fix stale
host:* wildcard note. Add commented MCP_ALLOWED_ORIGINS/MCP_ALLOWED_HOSTS
entries to charts/mcp-stack/values.yaml. Add [Unreleased] Security entry
to CHANGELOG.md.

Update test_mcp_origin_validation.py: rewrite integration tests to call
MCPOriginHostGate directly instead of handle_streamable_http. Update
test_rust_mcp_runtime_proxy.py: replace stale 'proxy rejects Origin'
tests with a test that confirms the proxy no longer gates on Origin.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix pre-commit

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* docs: clarify MCP_ALLOWED_ORIGINS/HOSTS nginx-direct-to-Rust bypass

Round-3 blocking IBM#2: .env.example claimed enforcement covers 'all
runtime modes' including rust-public. That claim is true when Python
is the public ingress (MCPOriginHostGate wraps /mcp before dispatch).
It is false when a reverse proxy routes /mcp directly to the Rust
listener on port 8787 — Python is bypassed entirely in that path.

Add a NOTE to both MCP_ALLOWED_ORIGINS and MCP_ALLOWED_HOSTS blocks:
state that the nginx-direct-to-8787 path is not covered and that
operators must configure Origin/Host validation at the proxy layer.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix: exempt loopback Host values from MCP_ALLOWED_HOSTS enforcement

Round-4 blocking IBM#1: test_server_mcp_handshake dispatches in-process
via internal_loopback_base_url() which produces Host: 127.0.0.1:{PORT}.
With MCP_ALLOWED_HOSTS set, MCPOriginHostGate rejected the request with
403 before any session or RBAC logic ran.

_check_mcp_host() now strips the port and unconditionally accepts the
loopback hostname literals 127.0.0.1, localhost, [::1], and ::1
regardless of the allowlist. A DNS-rebound page carries the attacker
hostname in Host, never a loopback literal, so this does not weaken
the DNS-rebinding protection.

Add six regression tests in TestCheckMcpHost (IPv4 with/without port,
localhost, IPv6 bracketed, non-loopback still rejected) and one
MCPOriginHostGate integration test that reproduces the handshake
self-call path (Host: 127.0.0.1:4444 with a non-loopback allowlist).

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* update secrets baseline

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

---------

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…ersions) (IBM#5811)

* feat: add container image version bumper script with bats tests

Signed-off-by: Jonathan Springer <jps@s390x.com>

* chore(makefile): add bats and container-bump-image-versions targets

Signed-off-by: Jonathan Springer <jps@s390x.com>

* fix: derive Pyxis repo path from pin; portable sed in bats test

Remove the hard-coded ARG-to-repository map (repo_for_arg) that could
drift from the actual image path in the Containerfile; the Pyxis
repository is now derived from each pin's image reference, with a
fail-fast guard for pins outside registry.access.redhat.com.

Replace the macOS-only 'sed -i ""' in the bats test with the portable
sed -i.bak pattern so the suite runs on GNU sed (Ubuntu CI).

Add regression tests for the derived-repository lookup and the
foreign-registry refusal.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* fix: clean sed backups via EXIT trap; deterministic apply order

Install an EXIT trap removing Containerfile .bak files so a sed failure
mid-apply (die path) cannot strand backup files on disk; the redundant
per-file rm in the loop is folded into the trap.

Iterate the apply phase over MANAGED_ARGS in declaration order instead
of undefined associative-array key order, matching the plan and report
loops and removing a latent ordering hazard.

Add a regression test asserting no .bak files remain after an update.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* fix: validate wheels UBI_MINIMAL pin before applying bump

The plan phase only grepped the root Containerfile for ARG lines; the
wheels Containerfile's UBI_MINIMAL pin was never validated. sed exits 0
on a missing match, so a missing wheels pin would print 'updated' and
exit 0 while the pins silently diverged — breaking the atomic-update
invariant this tool exists to enforce. A divergent pin value was
silently converged to the root pin's image path.

Both states now abort during planning with a reconcile-first error and
zero files modified. Locked with two regression tests.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* chore: regenerate .secrets.baseline via detect-secrets scan

Signed-off-by: Jonathan Springer <jps@s390x.com>

* Bump container images

UBI_BASE: 10.2-1790075508 -> 10.2-1790841271
NODEJS_IMAGE: 10.2-1790228697 -> 10.2-1790817011
UBI_MINIMAL: 10.2-1790075626 -> 10.2-1790753097

Signed-off-by: Jonathan Springer <jps@s390x.com>

* fix: extend image bumper to cover infra/nginx/Dockerfile

- Convert bare FROM in infra/nginx/Dockerfile to ARG NGINX_IMAGE + FROM
  so the script can manage the pin via the same sed pattern used for the
  root Containerfile.
- Add NGINX_IMAGE to MANAGED_ARGS / MANAGED_FILES; the script now
  manages four pins across three files.
- Replace declare -A associative arrays with parallel indexed arrays
  (bash 3.2 compatible); the script previously failed on macOS system
  bash and any other bash < 4 environment.
- Bump page_size from 100 to 500 to avoid missing the current pin for
  high-churn images with more than 100 tag-update events.
- Add NGINX_DOCKERFILE_PATH test hook and nginx fixture to the bats
  suite; extend all 15 existing integration tests with nginx assertions;
  add two new tests (NGINX_IMAGE bump, missing NGINX_IMAGE pin).
  Total test count: 11 -> 18, all passing on bash 3.2 (macOS) and
  bash 5.2 (Linux bats/bats container).

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix: rollback on partial write failure; unify bats targets

- Apply phase now uses a commit-or-rollback discipline: the EXIT
  trap restores .bak files when _bump_ok=0 (failure path) and
  deletes them when _bump_ok=1 (success path). Previously the trap
  always deleted backups, leaving the repo in a partial state when
  a later sed write failed after earlier writes had already landed.

- _bump_ok is a global (not local) so the EXIT trap can read it
  without triggering set -u's unbound-variable error.

- test-bats is now a one-line alias for bats. Both targets now run
  all tests/**/*.bats; the duplicated install-check and tests/bash/-
  only scope are removed.

All 18 bats tests pass.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* update secretsbaseline

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

---------

Signed-off-by: Jonathan Springer <jps@s390x.com>
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
Co-authored-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix(sso): resolve an email claim for generic OIDC providers

authenticate_or_create_user keys the local user record on email and rejects
any identity without one. The dedicated Keycloak, Entra and ADFS branches
already fall back to other claims when a provider does not release `email`,
but the generic OIDC branch read user_data["email"] directly. A provider
whose /userinfo omits that claim therefore failed login with
`user_creation_failed` after an otherwise successful token exchange, JWKS
fetch and userinfo call -- all of which log success, so the failure looks
like it happens somewhere it does not.

Add _resolve_email_claim for the generic branch:

* provider_metadata["email_claim"], when set, is used as-is -- explicit
  operator intent, no shape check.
* Otherwise try email, preferred_username, upn, unique_name, mail. A
  fallback must contain "@", so a bare username is never silently promoted
  into the field used as the identity key. Using a fallback is logged, since
  that value becomes the account identity.

When nothing qualifies, log the claim names that WERE present. The previous
warning reported only the absence, and userinfo bodies are not logged
anywhere else, so an operator had no way to tell which claim to configure.

Signed-off-by: Sami Marreed <sami.marreed@ibm.com>

* test(sso): cover the generic OIDC email claim on deny paths and against a live gateway

Address review feedback on IBM#6484: tests only, no production-code change.

Unit tests run the real generic-OIDC _normalize_user_info and the real
authenticate_or_create_user (browser SSO path) and build_external_identity
(bearer path) with no `email` claim. They assert that an identity resolved
from a fallback claim still passes every gate an `email` claim passes:
email_verified, trusted_domains, provider linking, auto_create_users,
sso_require_admin_approval and admin-claim escalation. Each deny case
asserts the resolved email first so it cannot pass vacuously.

The live-gateway test drives the authorization-code flow against Keycloak
with the `email` scope removed from the client, and covers provisioning
from preferred_username, denial when no claim holds an address, denial when
a configured email_claim is absent, and precedence of an `email` claim.

Signed-off-by: Sami Marreed <sami.marreed@ibm.com>

* chore: regenerate secrets baseline

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

---------

Signed-off-by: Sami Marreed <sami.marreed@ibm.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…M#6424)

* fix(auth): wire external-IdP verification into get_current_user()

get_current_user() (mcpgateway/auth.py), the dependency behind
get_current_user_with_permissions() and therefore every
Depends(get_current_user_with_permissions) REST endpoint (/tools,
/gateways, /servers, /rpc, etc.), only ever called
verify_jwt_token_cached() — the internal-only JWT verifier. A bearer
token from a provider with trusted_for_api_auth=True and a matching
SSO_API_TOKEN_AUTH_ENABLED was therefore rejected on every one of
these endpoints, even though the external-IdP verification path
(verify_credentials_cached -> _maybe_verify_external ->
verify_external_idp_token) worked correctly when called directly.

Swap the internal-only verifier for verify_credentials_cached(), which
already implements exactly this fallback (try a trusted external
issuer first, fall back to verify_jwt_token_cached() otherwise) and is
already used by require_auth()/require_auth_header_first() elsewhere
in the codebase. verify_credentials_cached() returns a superset of
verify_jwt_token_cached()'s payload shape for the internal-JWT case
(adds a 'token' key), so this is additive for existing callers.

Refs IBM#6396 — REST/`/rpc` surface only; does not close it (/mcp remains).

Signed-off-by: Sami Marreed <sami.marreed@ibm.com>

* test(auth): cover external-IdP REST auth and fix mocks after get_current_user() dispatch change

get_current_user() now calls verify_credentials_cached() instead of
verify_jwt_token_cached() directly (previous commit), so the ~80 unit
tests in tests/unit/mcpgateway/test_auth.py that patched
mcpgateway.auth.verify_jwt_token_cached to stub JWT verification need
to patch the new dispatch point instead. This is a mechanical rename
of the patch target only -- verify_credentials_cached is a superset
wrapper (falls through to verify_jwt_token_cached for internal
tokens), so mocked return values and existing assertions are
unchanged.

Also adds a live-gateway black-box test
(tests/live_gateway/sso/test_external_idp_rest_auth_e2e.py) that
exercises the fix against a real Keycloak-backed gateway: a bearer
token from a trusted_for_api_auth provider with a matching audience
is accepted on GET /tools and correctly team-scoped via SSO
team_mapping (group member sees the team's tool, a user in no mapped
group does not), while a wrong-audience token, a token from a
provider not opted into trusted_for_api_auth, and no token at all are
all still rejected with 401. Reuses the existing Keycloak-backed
live_gateway/sso fixture pattern (tests/live_gateway/sso/test_oauth_jwks_e2e.py)
and realm-seeded users (infra/keycloak/realm-export.json).

Wires SSO_API_TOKEN_AUTH_ENABLED through to the gateway container in
docker-compose.yml's sso profile so the new test's scenario is
actually reachable when the stack is started with it set; the flag
already existed in config.py/.env.example but was never passed
through by compose.

Signed-off-by: Sami Marreed <sami.marreed@ibm.com>

* test(auth): add /rpc and deny-path live tests for external-IdP auth

Address review feedback on IBM#6424: cover POST /rpc (tools/list) in the live
test alongside GET /tools, and add deny-path checks for a wrong audience, an
untrusted issuer and a missing token on both routes. The /mcp mount is out of
scope here and stays tracked in IBM#6396.

Fixture fixes found while running the suite against a real Keycloak:
- admin token uses teams=None so it can create team-scoped resources
- an oidc-audience-mapper is added for the run and removed afterwards, because
  the realm's users carry no client roles and their tokens have no aud claim
- the provider fixture restores trusted_for_api_auth, api_audience and
  team_mapping in a single PUT on teardown
- response bodies are read before the request context is disposed
- positive-path tests skip when the Keycloak issuer is not https, because
  verify_oauth_access_token requires an https jwks_uri on the issuer origin

Signed-off-by: Sami Marreed <sami.marreed@ibm.com>

* test(auth): document the https Keycloak setup for the external-IdP live suite

The four positive tests need an https issuer, which the sso compose profile
does not provide, so they skip there. Document the minimal https Keycloak and
gateway setup that runs all nine tests, and pass -p playwright, which
pyproject.toml disables by default.

Signed-off-by: Sami Marreed <sami.marreed@ibm.com>

* chore(secrets): refresh baseline after rebase

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* chore: remove issue reference from compose comment

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

---------

Signed-off-by: Sami Marreed <sami.marreed@ibm.com>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* new implementation. According to the issue documentation fix is fine. So we are going ahead with that

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed doc to explicity state Platform admin prominence

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
…ests (IBM#6895)

* test: eliminate pytest warnings, dead test code, and redundant auth tests

Closes IBM#6894

The pytest warnings summary carried 137 avoidable warnings per CI run;
most came from conditions inside test files. Fix the causes:

- Replace 1-15 byte HS256 secrets with 39-byte constants across nine
  test files. Signature semantics stay unchanged; pyjwt stops emitting
  InsecureKeyLengthWarning.
- Switch HTTP_422_UNPROCESSABLE_ENTITY to HTTP_422_UNPROCESSABLE_CONTENT
  in the A2A JSON-RPC passthrough test.
- Close the coroutine that create_task-rejection mocks leaked in
  TestA2AInvalidationBestEffort; the "never awaited" RuntimeWarning
  fired at garbage-collection time in an unrelated test.
- Silence ValidationMiddleware's intentional deprecation warning with a
  module pytestmark in its three test files; the pytest.warns assertion
  still passes.
- Bind datetime.now(timezone.utc).isoformat(" ") on non-Postgres
  dialects in twelve Alembic migrations that wrote datetime objects
  through text(). The string is byte-identical to what the sqlite3
  default adapter stores, so stored values do not change and the
  Python 3.12 deprecated-adapter warning disappears. Postgres branches
  keep binding the datetime object.
- Delete PydanticA2AAgent's class Config; BaseModelWithConfigDict
  already sets from_attributes=True, so the block was redundant and
  raised PydanticDeprecatedSince20 at import.
- Scope four filterwarnings entries in pyproject.toml: cpex payload
  headers deprecation (no extensions field exists yet), starlette
  httpx2 drift, the rbac.get_db() doctest, and a corrected
  HTTP_413_REQUEST_ENTITY_TOO_LARGE category (StarletteDeprecationWarning
  subclasses UserWarning, so the old DeprecationWarning filter never
  matched).

Also remove debt the same review surfaced:

- Delete the first shadowed class TestGetDb in test_auth.py; the second
  definition carried the same tests plus the assertions.
- Collapse test_invoke_agent_with_{basic_auth,bearer_auth,custom_headers}
  into one parametrized test; the three copies differed only in agent
  name, auth type, and header dict. Every header is now asserted to
  round-trip.

Full suite: 23,334 passed, 899 skipped, 2 xfailed, 0 failed. Warnings
drop from 252 to 180; every remaining warning is a SADeprecationWarning
or SAWarning from historical migration internals, documented as out of
scope in IBM#6894. Pinned ruff 0.15.20 and black 26.3.1 show no new
findings versus main; detect-secrets findings drop from nine to seven.

The five 60-second otel control-telemetry tests from CI job
105708501375 are stale-branch artifacts; main already fixes them in
eaf295e (IBM#6131), so this change leaves them alone.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* test: remove four perpetually-skipped team invitation stub tests

Closes IBM#6894

test_create_invitation_success, test_accept_invitation_success,
test_deactivate_existing_invitation_before_creating_new, and
test_expiry_days_from_settings carried pytest.mark.skip("Complex
integration test - main functionality covered by simpler tests") since
they were added and never ran. The coverage they name is provided by
the simpler tests around them. Delete the stubs; 48 tests remain in
the file and pass.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* test: remove skipped gateway-registration stub from e2e suite

Closes IBM#6894

test_register_gateway carried pytest.mark.skip("Requires external
gateway connectivity") and an empty body since it was added. Gateway
registration is already proven black-box by the live_gateway suite
(docker compose testing-up plus the tests under tests/live_gateway/),
so no relocation is needed. Delete the stub.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* test: remove skipped Admin UI version-tab stub and its dead file

Closes IBM#6894

test_admin_ui_contains_version_tab carried two skip markers ("Auth
system changed - needs update for email auth" and
skipif(not settings.mcpgateway_ui_enabled)) and never ran. The rest of
test_ui_version.py consists of commented-out stubs and fixtures that
serve only the skipped test. Version-check UI behavior belongs to the
Playwright suites. Delete the file.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* test: stop session-affinity test from leaking an OTel span into the suite

Closes IBM#6894

test_attach_envelope_trace_context_attaches_context_with_traceparent
performed a real opentelemetry.context.attach() of a span built from
the W3C example traceparent, discarded the token, and never detached.
pytest does not isolate contextvars per test, so every test that ran
afterwards in the same serial order saw an active current span.
inject_trace_context_headers() then stamped that traceparent into
outbound headers and MCP _meta, breaking
test_invoke_tool_mcp_streamablehttp (meta=None assertion) and
test_streamablehttp_uses_registry_and_modified_payload_with_headers_none
under distribution-dependent ordering.

The rewritten test patches opentelemetry.context.attach and asserts the
extracted context is attached; its previous assertion was the tautology
"result is not None or result is None".

Verified: leaker-then-victims serial run passes; the full
tests/unit/mcpgateway/ serial run passes with zero failures; the full
parallel suite passes with zero failures.

Signed-off-by: Jonathan Springer <jps@s390x.com>

---------

Signed-off-by: Jonathan Springer <jps@s390x.com>
Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
* feat(protocol): default MCP connect and inbound modes to auto

MCP_CLIENT_CONNECT_MODE and MCP_INBOUND_PROTOCOL_MODE now default to auto. Outbound connections negotiate the 2026-07-28 revision through server/discover and fall back to the legacy initialize handshake. Inbound clients may send mcp-protocol-version 2026-07-28. Set either variable to legacy to restore the previous behaviour.

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* fix(protocol): update header-default doctest for auto inbound mode

The doctest asserted LATEST_HANDSHAKE_VERSION for a request without the mcp-protocol-version header. Auto mode negotiates LATEST_PROTOCOL_VERSION.

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* commented out the defaults in the .env.example

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* fix(protocol): correct headerless claim and cover inbound mode live

The middleware records the mode default in request state for a headerless
request. Era routing reads the raw header in the SDK, so an absent header
still takes the legacy handshake path. The docstring states that now.

Add black-box coverage for the inbound protocol mode at /mcp: the default
auto mode serves a 2026-07-28 server/discover, legacy mode answers 400 with
the supported-version list, a headerless initialize negotiates a
handshake-era version, an unauthenticated modern request gets 401, and a
principal without servers.use gets 403 on a server-scoped path.

Pass PLATFORM_ADMIN_PASSWORD and DEFAULT_USER_PASSWORD to the source-run
gateway in tests/e2e/test_upstream_connect_mode_e2e.py. The curated
subprocess env hid the strong passwords that conftest exports, so
Settings.validate_security_combinations rejected the weak built-in default
and the subprocess exited with code 1.

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

* fixup: make pre-commit

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>

---------

Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
* test: establish sandbox start-method support for validation workers

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* refactor(security): extract killable sandbox pool from jq runner

Move the generic pool lifecycle into mcpgateway/utils/sandbox_pool.py.
The pool builds workers, proves they start, gates admission, and kills a
worker that overruns its budget.

Rebuild jq_runner on the new pool. The public API and the jq exception
types stay the same.

Read the worker count and the timeout through callables. A captured value
would ignore a setting that a test changes after import.

Stamp the admission gate on the executor. A gate held on the pool object
would admit against a different executor after a rebuild.

Tighten the start-method test to expect a TimeoutError. A broad Exception
also accepts a broken pool, which passes for the wrong reason.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): read the sandbox budget before taking a permit

The budget read sat between the gate acquire and the try block that
releases the permit. A raising timeout function leaked the permit, and
the pool then returned SandboxBusy forever.

Move the read above the acquire. It needs no permit.

Add a regression test. It arms a raising timeout function, then proves a
later call still reaches a worker.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* feat(security): run regex-bearing schema validation in a killable sandbox

A JSON Schema regex is attacker-influenced. Python holds the GIL while it
backtracks. An unbounded match freezes the gateway worker for every tenant.

Route a schema that carries a regex keyword into a killable worker pool.
Bound each validation by a wall-clock limit. Never inspect pattern content.
Whole-validation routing also covers the paths that reach re inside
jsonschema itself, which no keyword override intercepts.

Fail closed on every abnormal outcome. A timeout, a busy pool, a broken
pool, an oversized instance, an unserializable input, and an absent sandbox
all raise ValidationError. A truncated validation is never a pass.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test(security): assert the sandbox bounds the validation, not the clock

An elapsed-time ceiling passes for any fast failure. A pattern that merely
rejects quickly satisfies it, so the corpus could stay green over a guard
that routes every schema inline. This project has shipped that failure once.

Assert the raised error carries the safety-budget wording, so a bound is
told apart from an ordinary mismatch. Add a test that asserts the sandbox
submission itself: present for a regex keyword, absent without one. Routing
is now proved directly rather than inferred from timing.

Name the real cause when the pool fails to start. The previous text blamed
the platform for every failure, including a configuration fault, and sent
an operator to the wrong place during an incident.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): refuse an unknown validator class and test the escape for real

The two patternProperties escape tests swallowed every error and asserted
only elapsed time. They passed with no sandbox at all, so the one hole that
killed the previous design read as covered while nothing checked it. Assert
the timeout wording instead, and cover both keywords from one case.

Match the timeout phrase rather than the generic sandbox-failure phrase.
The generic text also appears when the pool is broken, so the corpus went
green against a pool that failed every submission in microseconds.

Refuse a validator class outside the draft table. Substituting a draft gave
an extended validator different semantics in the sandbox than inline, which
weakens the boundary and says nothing. Name the class instead.

Drop the config reference to an untracked design document. Place the new
main import in sorted order.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): bound tool input and output schema validation

Route _validate_with_cached_schema through validate_safely.
The sandbox now bounds both tool input validation and tool
output validation, the two sites that shared this helper.
A catastrophic-backtracking schema no longer holds the GIL.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): cap bounded-validation tests and correct stale comments

Add a 30s pytest-timeout marker to every test that asserts a
validation bound. Without it, a sandbox regression hangs the
test instead of failing it, and CI stalls with no named cause.

Delete the now-unused _NO_RETRIEVE_REGISTRY constant and correct
the SSRF-guard comment above it: validators are built against an
empty registry inside validate_safely, not against this file's
own registry, which no longer exists.

Update _validate_with_cached_schema's docstring to describe the
sandbox delegation instead of stale inline-validator mechanics.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): bound prompt argument schema validation

DbPrompt.validate_arguments() called jsonschema.validate() directly,
bypassing the sandboxed validation path. A crafted argument_schema
with a catastrophic regex pattern could freeze prompt rendering, and
this path needs only permission to render a prompt to trigger.

Route the call through validate_safely() so the schema runs in the
same killable, time-bounded worker as tool input and output
validation. The ValueError contract for callers is unchanged.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): withhold regex output schemas from the SDK validator

The vendored MCP SDK validates outputSchema with stock jsonschema at
mcp/server/lowlevel/server.py:573. The gateway decorator validate_input=False
gates only the input validator at server.py:536. The output call stays active.

A remote MCP server controls tool.output_schema. It therefore controls a regex
that runs unbounded inside site-packages, where no gateway change reaches.

_to_mcp_tool now omits outputSchema when schema_uses_regex reports a regex
keyword at any depth. The advertised tool drops the key, so the SDK skips the
validation. The bounded check in tool_service still validates the same data.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): guard proxied output schemas on the direct-proxy path

_proxy_list_tools_to_gateway returned the remote gateway's SDK tool models
unchanged. Those models carry the remote's outputSchema into the SDK tool
cache, where mcp/server/lowlevel/server.py:573 validates against it with stock
jsonschema. The previous commit guarded only _to_mcp_tool, so a direct_proxy
gateway still reached the unbounded validator.

Extract _guard_output_schema and call it from both paths. _to_mcp_tool builds a
dict payload; the proxy path holds constructed pydantic models, so
_guard_proxied_tools adapts the shape with model_copy. The regex decision lives
in one function, so the two paths cannot drift.

The SDK skips output validation when outputSchema is None, so replacing the
value is enough; the field need not be absent.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* feat(security): warn when a registered schema carries a regex keyword

Registration, gateway sync and OpenAPI import now call
warn_unprovable_patterns at the point each schema reaches storage. Each
call logs one warning naming the schema's source and never raises, so
no registration, sync or import can fail because of this check. The
sandbox already bounds every regex match; this only gives an operator
an inventory of which schemas carry a regex keyword.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): keep the regex-pattern warning out of the federation failure path

register_gateway ran warn_unprovable_patterns before appending the new
DbTool to db_tools, inside the same try/except that drops a tool from
federation on any exception. The warning cannot raise today, but a
diagnostic call sitting ahead of the data operation it describes turns
any future change to that call into silent federation data loss.

Move both calls to run after the tool is already in db_tools, so a
raising warning can only fail to log, never unregister the tool.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): make warn_unprovable_patterns structurally unable to raise

74e5029 introduced warn_unprovable_patterns and stated in its commit
body that no registration, sync or import could fail because of the
check. That was true only because the function happened not to raise;
nothing in the code enforced it. Four call sites relied on this
unstated assumption without a guard: the before_insert/before_update
listeners in db.py catch only jsonschema.exceptions.SchemaError, so
any other exception would abort the SQLAlchemy flush and fail tool or
prompt registration, and the OpenAPI import sites in
openapi_schema_router.py and admin.py had no exception handling at
all.

Rather than wrap every call site in its own try/except, move the
guarantee into the function itself. warn_unprovable_patterns now
catches and logs any exception from its own body, including from
schema_uses_regex, so it cannot fail its caller by construction. Every
current and future call site inherits this for free.

The gateway_service.py federation-loop ordering fix from the prior
commit is unaffected and stays as defence in depth.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* feat(security): log operator regex patterns compiled without a time bound

Four plugins compile operator-supplied regex strings at load time:
regex_filter, resource_filter, code_safety_linter and
harmful_content_detector. These patterns come from deploy-time
configuration, not from a request, so a pathological pattern here is
administrator self-denial-of-service, not an attack surface. Route
them through a log line, not the validation sandbox.

Add warn_unprovable_pattern_source() to safe_jsonschema.py, alongside
warn_unprovable_patterns(). It never raises, so a plugin cannot fail
to load because a diagnostic failed. Call it immediately before each
operator-configured re.compile() site.

The hardcoded default lexicons in code_safety_linter and
harmful_content_detector stay silent: they are project-authored, and
a warning on a pattern the project itself ships is noise.

Convert the harmful_content_detector comprehension to a loop so the
log call can run per pattern, keeping the existing non-string entry
passthrough unchanged.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test(security): prove no validation path reaches an unbounded regex engine

A grep over first-party code cannot prove this. The escape found in review
lived in site-packages, which a grep over mcpgateway/ never reads. Verify by
execution instead.

Three layers cover the tool input path, the tool output path and the prompt
argument path.

Layer 1 spies on the validation sandbox. It asserts each path submits a
regex-bearing schema, and submits nothing for a schema without one. Only a
positive assertion can prove that routing happened.

Layer 2 replaces the pattern and patternProperties keyword implementations in
every stock draft with a function that raises. Any pattern evaluated in this
process fails the test by name. Patch the keyword tables, not
jsonschema._keywords: each draft captured the function object at import time,
so the module attribute is never read again. A module-attribute patch was
measured and does not fire.

Layer 3 makes jsonschema.validate raise. Nothing calls it today. It guards
against a future change that reintroduces a direct call.

The sandbox worker is a separate process, so no in-process patch reaches it. A
correctly routed validation still succeeds while the tripwires are armed.

Each layer carries a self-test that proves the layer can fail. One more test
rejects an instance that breaks the pattern, which proves the worker evaluated
the schema.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test(security): cover the SDK output-schema site and assert rejection content

The module proved three first-party sites and was blind to the fourth. The
vendored SDK validates outputSchema in-process and unbounded at
mcp/server/lowlevel/server.py:573. validate_input=False does not gate that
branch. The only defense is _guard_output_schema, which withholds the schema.
That guard submits nothing to the sandbox and evaluates no keyword, so the
submit spy and the keyword tripwire both miss it. Removing the guard left all
tests green.

Add a fourth layer. Assert _to_mcp_tool and _guard_proxied_tools withhold a
regex-bearing outputSchema, then drive the real SDK CallToolRequest handler
with the tripwires armed. A self-test passes an unguarded schema to prove the
SDK site is live.

Hold the SDK input branch shut as well. inputSchema reaches the SDK unguarded,
so validate_input=True would open a second unbounded site. Read the flag off
the registered handler and assert it stays False.

Assert what the rejection says, not that a rejection happened. Every
fail-closed branch of validate_safely produces the same outcome in-process with
no worker running, so the former assertion passed with the sandbox switched
off. Each schema shape now names the message fragment only its own evaluation
produces.

Cover the shapes the corpus missed. Add a pattern under anyOf, which a router
that skips JSON array nodes reports as regex-free, and a patternProperties
schema, which the tripwire armed but nothing exercised. Parametrize every
routing, tripwire, guard and rejection test over all three shapes.

Record two limits in place. The jsonschema.validate patch cannot reach a caller
that binds the function at import time, and no schema here may carry $anchor or
$dynamicAnchor, because the metaschema checks both with its own pattern
keyword.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test(security): assert the event loop stays responsive under schema ReDoS

A hostile subject against a nested-quantifier pattern stalls the event loop for
about 8 seconds when the match runs in this process. Moving the match to a
thread changes nothing, because CPython holds the GIL through a regex match. A
test that only measures how long validation took passes against that non-fix.
Measure the loop instead.

The heartbeat wakes every 10 ms and records how late each wake-up was. The test
asserts three things: the heartbeat produced samples, the worst wake-up stayed
inside twice the sandbox budget, and the validation was stopped by that budget.
The last assertion separates a working sandbox from a broken pool that refuses
every regex-bearing schema and also keeps the loop free.

The subject is 28 characters. Backtracking doubles per added character, so a
longer subject runs for hours under a regression, and pytest.mark.timeout cannot
break it: the signal is delivered between bytecodes and a regex match never
yields one. The test would hang instead of fail.

The ReDoS test in test_input_validation.py stored a catastrophic pattern and
asserted it was stored. It now validates a hostile subject against each stored
schema, and asserts the budget stopped the one pattern that backtracks.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test(security): assert the timeout wording at every bounded-validation site

Three sites asserted a bound with elapsed time plus a weak outcome check. That
pair cannot tell "the sandbox bounded a runaway" apart from "rejected instantly
for an unrelated reason". Each site now asserts the message contains "exceeded
the execution time limit", which only the timeout path produces. A mismatch
gives a jsonschema message, and any other sandbox fault gives the broader "could
not be completed safely" text. The prompt site reads the phrase through the
ValueError that validate_arguments raises.

The pinning test for warn_unprovable_patterns broke only the schema_uses_regex
call, so it pinned one path rather than the never-raises property. Moving the
success-path warning out of the try would have kept it green while reintroducing
an unguarded statement. The test is now parametrized over each raisable
statement in the function body.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): make the never-raises diagnostics true to their docstrings

Both warn_unprovable_patterns and warn_unprovable_pattern_source promise to
swallow every failure, including a failure from logging itself. The fallback
warning in each except handler was unguarded, so broken logging still raised
into a SQLAlchemy before_insert listener, a federation sync loop, and a plugin
load path. Those callers drop a tool registration when a diagnostic raises.

Both handlers now report through _report_diagnostic_failure, which guards the
fallback and drops a second failure. Nothing remains to do once logging is
unusable.

warn_unprovable_pattern_source had no pinning test. It now carries the same
parametrized test as its sibling, one raisable statement at a time: the
len(pattern) the log call evaluates, the log call itself, and the fallback log.
Both tests gain a case that breaks every logging call, which is what pins the
new guard.

The subject-length comment in the ReDoS test claimed pytest.mark.timeout cannot
interrupt a regex match. That is false; the signal arrives during the match.
Measured behavior: the timeout reports on schedule, then the test call waits for
the match, because a Python thread cannot be killed and closing the loop joins
its executor. A 3 second timeout on a 30 character match reports at 3 seconds
and returns after 42. The comment now states that.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test(security): pin the event-loop budget and finish the bounded-validation wording

The event-loop regression test derived its stall budget from
regex_timeout_seconds, the same unpinned setting the sandbox uses for its
own timeout. Raising that setting past about 3.85 seconds let the budget
grow past the actual GIL stall, so the test kept passing while a
thread-only sandbox stalled the loop for close to 8 seconds. Add an
explicit precondition that fails when regex_timeout_seconds exceeds the
value this suite was measured against, and cap the derived budget so
neither the setting nor the derivation can silently widen the check.

Apply the measured MAX_VALID_SUBJECT_SECONDS bound to the valid-subject
test, which previously allowed a 500x regression to pass unnoticed.

Correct the BOUNDED comment at the three remaining sites that claimed the
timeout path is the only one producing the outer "could not be completed
safely" text. validate_safely wraps every sandbox fault in that text; the
inner "exceeded the execution time limit" phrase is what actually comes
from SandboxTimeout alone.

Add the missing validation-pool fixture to the ReDoS coverage in
test_input_validation.py, scoped to the one test that needs it rather than
autoused across the file's many unrelated security tests, so the pool's
lifecycle no longer depends on lazy _ensure() and on no other module
having left the sandbox marked down.

Proved the budget fix catches the regression it was written for: swapping
ThreadPoolExecutor into SandboxPool._build and setting
REGEX_TIMEOUT_SECONDS=5 now fails the event-loop test where it previously
passed.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test(e2e): prove a hostile schema pattern cannot stall a live gateway

Registers a tool whose input_schema carries a catastrophic backtracking
pattern, invokes it through the real MCP path (HTTP routing, auth, RBAC,
the sandboxed validator), and asserts the gateway still answers /health
immediately afterwards. Unit-level coverage proves validate_safely is
bounded in isolation; this proves the property survives everything
between a unit test and a real request.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(test): bound the ReDoS e2e probe by the MCP client timeout

_mcp_tool_call rides a ClientSession capped at _CLIENT_TIMEOUT (default
5s), which fires before a 20s elapsed assertion ever could. A genuine
regression now surfaces as a diagnosed pytest.fail naming both possible
causes -- an unbounded sandbox or an overloaded CI box -- instead of an
uncaught McpError traceback.

Also delete the throwaway server before the tool it holds, reusing the
file's existing _delete_owned helper so a cleanup failure is reported
rather than swallowed by a bare suppress(Exception).

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(test): catch the ExceptionGroup a bare McpError handler cannot see

_mcp_tool_call runs initialize() then call_tool() on the same session, so
the SDK's anyio TaskGroups wrap a real timeout in ExceptionGroup twice
(per _unwrap_exception_group's own docstring). A bare 'except McpError'
never catches that shape, so a regression would still surface as a raw
nested traceback instead of the intended diagnostic. Widen the except to
(McpError, ExceptionGroup) and unwrap with the file's existing
_unwrap_exception_group helper, matching the pattern already used by
TestTokenLifecycle.test_scoped_token_denied_tool_execute.

Also reinstate the elapsed assertion. _CLIENT_TIMEOUT bounds a single
send_request round trip, not the whole call, and this call makes two
(initialize, call_tool) after session setup, so the honest ceiling is a
small multiple of _CLIENT_TIMEOUT rather than the raw value or a dropped
check.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* docs(security): record schema validation sandbox changelog

Document the sandboxed schema-validation worker process in the
CHANGELOG under Unreleased: pattern/patternProperties schemas now
validate in a killable worker, registration behavior is unchanged,
and a tool's outputSchema is withheld when it carries a regex
keyword.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: offload sandboxed schema validation calls to a thread

Both call sites in mcpgateway/services/tool_service.py invoked the
schema validator synchronously from async methods. SandboxPool.submit()
blocks on Future.result(timeout=...) while a worker process validates a
regex-bearing schema, so a hostile request held the event loop for the
sandbox's timeout budget on every call, not just once.

Wrap both calls in asyncio.to_thread, matching the existing jq offload
pattern in this file. _extract_and_validate_structured_content stays
synchronous; only the call site changes, so its doctests and its
architectural documentation (Validator B) are unaffected.

Adds a regression test driving the real production entry point
(preview_tool_invocation) with a catastrophic input_schema, measuring
event-loop lateness. Proven both directions: fails at 1.01s stall with
the offload removed, passes at 0.006s with it restored -- the existing
sandbox test alone could not catch this, since it wraps its own call in
asyncio.to_thread and so never exercises whether production offloads it.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: pass object arguments in schema ReDoS test

_validate_tool_input_arguments takes a Dict argument. Wrap the hostile
subject in an object and move the pattern onto a property schema.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* docs: document the JSON Schema regex sandbox

Add a securing.md section for the REGEX_* settings, fail-closed
behavior, platform start method, outputSchema withholding, and the
unbounded client SDK output validation path.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* docs: name validate_safely as Validator B's mechanism

Validator A in the MCP client SDK still uses stock jsonschema. List
that path under known gaps.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix(security): fork the schema sandbox only on Linux

macOS lists fork in get_all_start_methods(), so the sandbox chose fork
there. Fork is unsafe on Darwin. Pick fork on Linux only and spawn
elsewhere, matching jq_runner.subprocess_mode_available().

Reuse one module-level no-retrieve registry for inline and worker
validation.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: drop outputSchema withholding after the MCP SDK v2 upgrade

The v2 low-level server no longer validates outputSchema, so the
withholding guard defended a removed code path. After the rebase the
guard also read the v1 field name and withheld nothing. Restore the
transport to main, so tools advertise outputSchema unchanged. Remove
the guard tests and the related changelog and docs text.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: cover output-path offload and tool insert listener; document REGEX_* settings

Add a loop-stall test for the invoke_tool output-schema path.
Add a Tool before_insert test for the regex warning.
Pin the real outcome of each ReDoS pattern in test_regex_dos_prevention.
Reset the sandbox-down flag in the safe_jsonschema test fixture teardown.
Add REGEX_TIMEOUT_SECONDS, REGEX_WORKERS, and REGEX_MAX_SUBJECT_BYTES to configuration.md.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: evaluate sandbox workers once on start

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* chore: refresh secrets and align sandbox test mocks after rebase

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: complete regex warnings and bound output validation test

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* fix: harden outbound HTTP connection handling

Outbound HTTP requests now establish connections against an address that the
outbound URL policy validated, and keep the original hostname for the Host
header and for TLS certificate verification. A shared helper centralises this
so every outbound path applies the same policy.

Outbound URL validation now runs regardless of the SSRF protection setting.
The scheme allowlist, dangerous-protocol, control-character, embedded-credential
and XSS checks apply unconditionally; disabling SSRF protection no longer
disables them as a side effect.

Behaviour notes for operators are in CHANGELOG.md, including pooled MCP sessions
no longer following redirects, fail-closed handling when a caller-supplied
client's TLS context cannot be read, and reduced DNS address failover on some
requests.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* style: apply ruff format to the changed files

CI's pre-commit job runs the ruff-format hook, which is not covered by the
local 'make ruff' target (check-only). Reformats five files this branch
touched. Verified with an AST comparison before/after that no file's parse
tree changed except one docstring collapsed from two lines to one, whose
text content is identical; every other change is line-wrap and trailing-
comma only.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* style: correct file headers on two new files

The repo's correct-file-headers pre-commit hook expects a Location/Copyright/
SPDX docstring header on every mcpgateway/ and tests/ Python file.
ssrf_pinning.py had a stale 'Copyright 2025' line instead of the current
'Copyright contributors to the MCP-CONTEXT-FORGE project'; the new test file
had no header block at all. Applied via .github/tools/fix_file_headers.py
--fix-all, the same tool CI runs. No code content changed.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: restore the request URL after a pinned send

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: isolate pinned OAuth and cross-gateway requests from the shared pool

httpcore keys pooled connections by (scheme, host, port) and ignores
sni_hostname. Outbound DNS pinning rewrites the host to a validated IP,
so two different hostnames pinned to the same address collapse to one
origin and can reuse a connection whose TLS handshake verified a
certificate for the other hostname, while carrying this hostname's
credentials.

Move the OAuth token POST (oauth_manager.py _post_token_request) and
the UAID cross-gateway POST (a2a_service.py _invoke_remote_agent) off
the shared HTTP client onto get_isolated_http_client, matching the
pattern already used by tool_service.py and the sibling A2A local
invocation path.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: isolate pinned LLM provider requests from the shared pool

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* docs: record that pinned requests require an isolated client

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: adapt sibling tests to isolated-client pinning fix

Four sibling test files independently exercised the outbound pinning
code paths fixed earlier on this branch, but mocked the old access
points or asserted the old buggy behaviour, so a full make test run
still failed 56 cases across them.

- test_mcp_servers_router.py: the two SNI-pinning transport tests
  asserted the request URL stayed pinned to the resolved IP after a
  successful send. That is the bug fixed in 228d2c2 (the transport
  restores request.url to the original hostname in a finally block).
  Update both assertions to expect the restored hostname.
- test_a2a_service_uaid_security.py: _invoke_remote_agent's UAID path
  now calls get_isolated_http_client() directly instead of the shared
  get_http_client() singleton. Add an autouse fixture, trimmed from the
  one already used in test_a2a_service.py, that bridges the isolated
  client call to whatever get_http_client mock each test installs.
- test_oauth_manager_pkce.py and test_identity_propagation.py:
  _post_token_request's non-mTLS branch now calls the free function
  get_isolated_http_client() instead of the per-instance _get_client()
  method, so a method override can no longer intercept it. Convert
  every _get_client patch/monkeypatch site to patch
  get_isolated_http_client via the _isolated_client() helper already
  reviewed in test_oauth_manager.py, adding a local copy of the helper
  to each file. test_identity_propagation.py's raw attribute
  assignments also gained the monkeypatch fixture they needed.

The mTLS branch of _post_token_request builds its own inline
httpx.AsyncClient and is unchanged, so no mTLS-path test was touched.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* docs: correct stale oauth_manager isolated-client docs, harden pinning test

Update _post_token_request's docstring and the follow_redirects comment,
which still described the pre-fix shared-client behavior and could mislead
a future maintainer into reverting the isolated-client fix that keeps
pinned token requests out of the shared connection pool.

Strengthen test_oauth_token_post_uses_an_isolated_client to assert
follow_redirects=False on both captured isolated clients, matching its
LLM proxy sibling test.

Add a regression test proving SniPinningTransport's URL-restore finally
block does not corrupt or block a still-open multi-chunk response stream.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: migrate outbound DNS pinning tests to httpx2

SniPinningTransport now subclasses httpx2.AsyncHTTPTransport and the
gateway/tool-service connectors route through mcp_proxy_client instead of
importing sse_client/streamablehttp_client directly. Update the transport
and connector-patch tests to match.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: skip TLS fail-closed check for non-pooled transports

A transport with no connection pool (an in-process ASGITransport, used by
tests) never dials the network, so there is no DNS-rebinding surface for
_wrap_httpx_client_factory to pin. Treat that case as unpinned instead of
raising, and keep the RuntimeError fail-closed path for a pooled transport
that unexpectedly carries no TLS context.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* style: apply ruff format to llm_proxy_service.py

Parenthesize the multi-context-manager async with, matching the ruff-format
style the pre-commit CI job enforces.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fix: keep outbound pinning when the proxy variable does not match the URL scheme

httpx mounts HTTP_PROXY on http:// only and HTTPS_PROXY on https:// only.
_egress_proxy_applies() treated any proxy variable as covering every scheme.
With only HTTP_PROXY set, an https:// target skipped pinning while httpx still
connected directly and resolved DNS again. Select the proxy variables by the
URL scheme.

Clarify in the module docstring that the shared-pool constraint applies to
URLs pinned by PinnedTarget.pin(), not to SniPinningTransport.

Add regression tests for the proxy scheme match, NO_PROXY exemptions, unsafe
Host authorities, pinned-address exhaustion, the pooled-session wrap no-op
paths, the streamablehttp health check, the OAuth custom-CA branch, and the
RuntimeError type on a blocked pooled session.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: assert the pinned dial address in SniPinningTransport router tests

The URL-restore change dropped the check that the transport dials the pinned
address. Capture the URL inside the mocked send so the tests assert both the
dialled address and the restored hostname.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* docs: name invoke_tool in the SSRF_DNS_FAIL_CLOSED description

Name the REST arm of tool invocation explicitly. Remove an unrelated issue
reference from two session registry test docstrings.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: expect MCP error results from tool pinning tests after rebase

invoke_tool_direct and the MCP transport arms of invoke_tool now return a
CallToolResult with is_error set instead of raising (IBM#6181). Assert the error
result instead of an exception. The direct-proxy refusal test records dial
attempts, because the error result also absorbs the AssertionError from
_refuse_to_dial.

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* test: configure URL schemes in DNS fail-open test

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>

* fixed pre-commit

Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>

---------

Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
* fix(oauth): preserve catalog on empty fetch

Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com>

* fix(oauth): keep empty-catalog guard aligned with current main

Follow-up to the merge with main, which reworked fetch_tools_after_oauth
around the guard added by this PR.

- route gateway.name through SecurityValidator.sanitize_log_message in the
  new warning, matching the log hygiene the rest of the function adopted
  upstream
- update the regression test for the EmailUser lookup fetch_tools_after_oauth
  now performs to build user_context, and assert that no DELETE statement is
  issued instead of counting db.execute calls, which is no longer a proxy for
  'no stale cleanup ran'

Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com>

* test(oauth): cover partial catalog cleanup after authorization

Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com>

* refactor(oauth): reuse MCP_SYNC_CREATED_VIA_VALUES for OAuth cleanup

The empty-catalog guard added a local mcp_created_via_values set. Main has
since gained the module-level MCP_SYNC_CREATED_VIA_VALUES constant, used for
the same purpose in _refresh_gateway_tools_resources_prompts, so the local
copy is now a near-duplicate that also omits "notification_service" and
therefore leaves stale notification-driven entries behind on authorization
code gateways.

- pass the shared constant from fetch_tools_after_oauth instead of the local
  set, so both catalog-cleanup paths agree on what counts as MCP-discovered
- extend the partial-catalog test with a stale notification_service tool the
  OAuth fetch path must prune, which pins the constant in place
- refresh the two .secrets.baseline line numbers shifted by the test change

Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Signed-off-by: 林SO <142557582+Linxiushen@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Add 1-line reminder of new UX port to Makefile

Signed-off-by: Jonathan Springer <jps@s390x.com>

* chore: use WEB_UI_PORT env var in testing-up output

Replace the hardcoded port in the testing-up service table with
${WEB_UI_PORT:-3001}, matching the pattern used in docker-compose.yml.
This ensures the displayed URL reflects any custom WEB_UI_PORT override.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* fix: resolve testing-up web UI port from Compose configuration

The testing-up summary expanded ${WEB_UI_PORT:-3001} in the recipe
shell, which never reads .env. Compose reads .env for interpolation,
so the summary printed 3001 while Compose published the port from
.env (e.g. 3100). Rename the row to ContextForge Web UI: Compose
publishes web_ui directly, not through the gateway nginx listener.

Add scripts/testing-web-ui-port.sh to read the published host port
from the resolved Compose configuration. The script uses
config --format json on Docker Compose v2+ and parses canonical
config YAML on podman-compose and docker-compose v1, which have no
JSON output. A failed lookup stops the recipe instead of printing a
fallback URL.

Add tests/bash/testing_web_ui_port.bats to compare the resolver
against the Compose-resolved host port for the four WEB_UI_PORT
placements and to pin the YAML parser shape per backend.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* test: supply required compose variables in web UI port fixture

The bats fixture env file supplied DEFAULT_USER_PASSWORD and
PLATFORM_ADMIN_PASSWORD only. Compose interpolation for the testing
profiles also requires JWT_SECRET_KEY and AUTH_ENCRYPTION_SECRET, so
the four resolver tests failed on shells without those variables
exported. Add recognizable test-only placeholders for both.

Signed-off-by: Jonathan Springer <jps@s390x.com>

---------

Signed-off-by: Jonathan Springer <jps@s390x.com>
IBM#7087)

* fix(performance): guard psutil.cpu_freq() against SystemError on macOS

psutil 7.2.2 on macOS (Apple Silicon) raises:
  SystemError: <built-in function cpu_freq> returned a result with an exception set

The C extension does not cleanly propagate host_processor_info failures,
leaving CPython's internal error flag set instead of raising OSError.

Wrap the cpu_freq() call in try/except Exception so any variant from
the C extension is caught and cpu_freq_mhz falls back to None.

Fixes IBM#7086

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* fix(version): guard psutil.cpu_freq() against SystemError in version.py

Same psutil 7.2.2 macOS bug as performance_service.py — cpu_freq()
raises SystemError via its C extension on Apple Silicon. The /version
endpoint calls _psutil_metrics() which was also unguarded.

Fixes the e2e failure:
  FAILED tests/e2e/test_main_apis.py::TestVersionAndDocs::test_get_version_with_admin_auth

Related: IBM#7086
Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* test: add parametrized tests for psutil.cpu_freq() exception guard

Covers SystemError and RuntimeError from cpu_freq() in both:
- performance_service.get_system_metrics()
- version._system_metrics()

Verifies cpu_freq_mhz is None and other metrics are unaffected.

Related: IBM#7086
Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* test(version): use _make_fake_psutil() in cpu_freq exception test

Replace inline MagicMock stub with the existing _make_fake_psutil()
helper, overriding only cpu_freq. Consistent with the rest of the
test file.

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* test(performance): extract _make_mock_psutil() helper, refactor TestSystemMetrics

Eliminates the duplicated 8-line inline mock setup in
test_get_system_metrics_load_average_exception_sets_none and
test_get_system_metrics_cpu_freq_exception_sets_none.
Each test now calls _make_mock_psutil() and overrides only
the attribute it cares about.

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

* test(version): rename _make_fake_psutil to _make_mock_psutil

Consistent naming with test_performance_service.py.

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>

---------

Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com>
* chore: remove mcp-servers sample directory and templating

Remove the unsupported sample/test MCP servers, the Go/Python scaffolding
templates, and the url-to-markdown CI workflow from the monorepo.
The six Python servers now live in IBM/contextforge-examples under
mcp-servers/python/. The scaffolding templates are deleted without migration.

- Delete mcp-servers/ (six Python servers, templates, scaffold scripts).
- Delete the mcp-url-to-markdown-tests workflow.
- Delete the sample-server doc pages. Rewrite the servers section index.
- Remove mcp-servers references from CI, lint configs, packaging, and docs.
- Switch the performance compose generator and benchmark docs to the
  published cfex-mcp-fast-time-server and cfex-mcp-benchmark-server images.
- Remove stale Go CI setup. No Go sources remain in the repository.
- Update the fast-time-server catalog entry for the published image.

The /v1/mcp-servers REST API is unchanged.
It remains the product-language alias for the gateways API.

Closes IBM#6378
Closes IBM#6379
Closes IBM#6380
Closes IBM#6381
Closes IBM#6382
Closes IBM#6383
Closes IBM#6384

Signed-off-by: Jonathan Springer <jps@s390x.com>

* test: align regression tests with generator and workflow changes

Remove the obsolete JWT_SECRET_KEY escape workaround in
test_issue_6083_compose_cleanup.py. The generator template is escaped
at the source, so the monkeypatch double-escaped the placeholder and
reintroduced the KeyError.

Drop the Set up Go step assertion in
test_secret_baseline_ci_decision_workflow.py. The step left
pre-commit.yml with the Go toolchain removal. The shared ci-decision
contract remains covered by the Run pre-commit and detect-secrets
assertions.

Signed-off-by: Jonathan Springer <jps@s390x.com>

* ci: restore Set up Go in lint-helm-ci job

make linting-full runs linting-workflow-actionlint, which executes
actionlint via go run. The job needs a Go toolchain even though the
repository no longer contains Go sources. Cache stays disabled: no
go.sum or go.mod exists to key it.

Signed-off-by: Jonathan Springer <jps@s390x.com>

---------

Signed-off-by: Jonathan Springer <jps@s390x.com>
* fix(a2a): propagate W3C trace context through A2A invocations

The request span middleware now traces POST /a2a/invoke and /a2a/{agent_name}/invoke.
The span adopts a valid inbound traceparent, for example from Envoy, as its parent.

The a2a.invoke span injects its context into the outbound agent request headers.
The downstream agent call therefore joins the same trace as the inbound request.

Unit tests cover route selection, parent adoption, missing or malformed context, and outbound injection.

Signed-off-by: Antonio Miranda <antonio.miranda@basf.com>

* fix(a2a): harden W3C trace propagation per review

Address review findings on IBM#7093:

- Gate context baggage on policy inside inject_trace_context_headers().
  The composite propagator emitted raw baggage before the settings
  check. Remove it unconditionally and re-add sanitized baggage only
  when otel_baggage_enabled and otel_baggage_propagate_to_external
  both permit it (CWE-200).
- Replace traceparent, tracestate, and baggage keys case-insensitively
  before injection. Prepared headers from auth mappings or plugins can
  no longer conflict with the active span, and stale tracestate no
  longer survives an empty active trace state.
- Inject the active context into UAID cross-gateway forwarding headers
  in _invoke_remote_agent so gateway hops continue the same trace.
- Trace the /a2a/{agent_name}/jsonrpc ingress route as well.
- Add regression tests against the real opentelemetry-api propagator.
- Add a live black-box suite that boots an isolated gateway with the
  real SDK and a header-capture agent. It asserts one trace ID across
  inbound parent, request span, a2a.invoke span, and downstream
  traceparent, denies outbound dispatch on authentication failure, and
  withholds context baggage when external propagation is disabled.
- Add a pytest (observability extra) CI job that runs the SDK-gated
  modules and the live suite.

Signed-off-by: Jonathan Springer <jps@s390x.com>

---------

Signed-off-by: Antonio Miranda <antonio.miranda@basf.com>
Signed-off-by: Jonathan Springer <jps@s390x.com>
Co-authored-by: Antonio Miranda <antonio.miranda@basf.com>
Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
* fix: validate x-mcp-header annotations and mirror them as Mcp-Param headers on upstream calls

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* fix: trim the _call_upstream_tool docstring

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* fix: collapse the x-mcp-header mirroring tests into one call-site test

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* fix: supply the default request context in the x-mcp-header transport tests

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* fix: assert the exact x-mcp-header validation error in the federation test

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

* fix: prove unauthorized calls are refused before the x-mcp-header mirroring path

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>

---------

Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…sions (IBM#7137)

Section 1.6 previously told maintainers to grep FROM lines and visit the
Red Hat Container Catalog by hand. The new make container-bump-image-versions
target automates this: it queries the Pyxis API, validates pin consistency
across Containerfile, infra/wheels/Containerfile, and infra/nginx/Dockerfile,
and applies updates atomically with rollback on failure.

Replace the manual grep-and-table prose with the automated target, its
managed ARG/file table, the minor-line policy note, and the prerequisite
(curl and jq) admonition.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* chore: update Python dependencies for 2026-10-05 release cycle

- Bump mcp constraint in all external plugins (opa, cedar, llmguard) from >=1.28.1,<2 to >=2.2.0
  to match gateway's mcp>=2.0.0 requirement and resolve uv lock failure for Python 3.13 split
- Add oauthlib>=4.0.0 transitive pin; fixes PYSEC-2026-4114
- Add oauthlib to [tool.uv.exclude-newer-package] to allow 4.0.0 (published 2026-09-28)
  past the global exclude-newer=10 days cutoff
- Advance all [tool.uv.exclude-newer-package] dates to 2026-10-05T23:59:59Z
- Fix typo: 'Transivite' -> 'Transitive' in requests comment
- Bump docs/requirements.txt, tests/load/requirements.txt,
  tests/populate/requirements.txt, and uv.lock to latest resolved versions

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix pre-commit

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix: bump RetryWithBackoffPlugin max_retries to 3 for 0.3.11

cpex-retry-with-backoff 0.3.11 changed max_retries semantics to
total attempts including the original call. Bump from 2 to 3 to
preserve the prior behaviour of 2 retries after the original.
Also update version field to 0.3.11.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

---------

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* chore: migrate Tailwind CSS v3 to v4 and fix npm CVEs

- Upgrade tailwindcss from ^3.4.17 to ^4.0.0; drop autoprefixer (bundled in v4)
- Add @tailwindcss/postcss devDependency; update postcss.config.js to use it
- Remove tailwind.config.js; port config to CSS-native format in tailwind.src.css:
  - Replace @tailwind base/components/utilities with @import 'tailwindcss'
  - Add @custom-variant dark for .dark class dark mode
  - Port custom keyframes (float, pulse-soft, slide-up, fade-in) to @Keyframes
  - Register utilities via @Utility (animate-float, animate-pulse-soft, etc.)
- Upgrade postcss-cli from ^11.0.0 to ^12.0.0 (uses chokidar 5.x, drops braces)
- Fix overrides: brace-expansion >=5.0.12 (GHSA-q2hr/qhr7/6j4f),
  fast-uri >=3.1.8 (GHSA-qw65/58mr/hrr3), ip-address >=9.0.5 (GHSA-j6r3/h3mg),
  braces >=3.0.3 (partial mitigation GHSA-vfj7)
- 10 remaining findings are all stylelint -> fast-glob -> micromatch -> braces;
  no upstream fix exists (micromatch 4.x has no patched release); dev-only linting
  tools with no production bundle impact

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

* fix: remove tailwind.config.js from Containerfile COPY

tailwind.config.js was deleted as part of the Tailwind v4 migration
(config is now CSS-native in tailwind.src.css). The Containerfile
still referenced it on line 174, causing the amd64 Docker build to
fail with 'not found'.

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>

---------

Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
The llmchat extra depended on langchain-mcp-adapters, which has no
release that works with mcp>=2.0.0: 0.3.1 imports
`mcp.shared.context.RequestContext` (removed in v2) and 0.3.2 pins
`mcp<2.0.0`. Upstream closed v2 support as superseded by
`langchain.mcp`, whose releases require langchain-core>=1.6 and
langgraph>=1.2.11, above the versions the llmchat extra allows. LLM Chat
therefore failed to start, and the guarded import reported it as
"dependencies are missing".

Replace MultiServerMCPClient with a small `_MCPToolSource` that lists
tools with `mcp.client.Client` (following pagination) and wraps each one
as a LangChain StructuredTool that calls it over a fresh session, using
the gateway's existing streamable HTTP helper, `sse_client` and
`StdioServerParameters`. Drop langchain-mcp-adapters from the extra.
When the llmchat imports fail, report the original ImportError.

Closes IBM#7090

Signed-off-by: Sena Köse <kosesena@hotmail.com>
@kosesena
kosesena force-pushed the fix/llmchat-mcp-sdk-v2-tools branch from 6626456 to 21bd096 Compare October 6, 2026 09:41
ScarAnd pushed a commit to ScarAnd/mcp-context-forge that referenced this pull request Oct 23, 2026
Signed-off-by: Andrin Enz <Andrin.Enz@ibm.com>

# Conflicts:
#	mcpgateway/services/mcp_client_chat_service.py

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: LLM Chat fails to start after MCP SDK v2 upgrade (langchain-mcp-adapters imports removed mcp.shared.context.RequestContext)