Skip to content

fix(sse): cap concurrent indexer SSE streams per identity - #1813

Open
rajesetavie-maker wants to merge 6 commits into
Streampay-Org:mainfrom
rajesetavie-maker:feat/issue-1678-cap-concurrent-api-indexer-sse-connections-per
Open

rajesetavie-maker wants to merge 6 commits into
Streampay-Org:mainfrom
rajesetavie-maker:feat/issue-1678-cap-concurrent-api-indexer-sse-connections-per

Conversation

@rajesetavie-maker

Copy link
Copy Markdown

Overview

This PR caps concurrent GET /api/indexer/sse connections per identity and per instance, adds heartbeats, reports queue drops in-band instead of via an undeliverable header, and un-quarantines the route's test suite. It also updates the status route and OpenAPI tests that depend on the SSE contract.

Related Issue

Changes

🔒 SSE Concurrency Control

  • [MODIFY] app/api/indexer/sse/route.ts

    • Added an in-process Map<identity, number> of open streams.
    • Enforced SSE_MAX_CONCURRENT_PER_IDENTITY (default 3) and SSE_MAX_CONCURRENT_GLOBAL (default 500); over-limit requests return 429 with Retry-After.
    • Decrement happens exactly once per stream via a guarded release path covering cancel, request.signal abort, completion, and error.
    • Added a : ping heartbeat comment every 15 seconds to keep proxies from closing idle streams.
    • Emits a final event: stream_end with data: { reason, dropped } before closing, sourced from SseQueue drop counts.
    • Removed the misleading X-SSE-Queue-Dropped "on stream close" header documentation and updated the route header doc.
    • Kept export const runtime = "nodejs".
  • [MODIFY] app/api/indexer/status/route.ts

    • Aligned status reporting with the updated SSE contract.

🧪 Test Suite Un-quarantine

  • [MODIFY] jest.config.js

    • Removed app/api/indexer/sse/route.test.ts from testPathIgnorePatterns.
  • [MODIFY] __tests__/openapi-auth-wallet.test.ts

  • [MODIFY] __tests__/openapi-webhooks.test.ts

    • Updated expectations to match the revised SSE/status contract.

Root-Cause Table for Quarantined Failures

Previously failing assertion Root cause
Stream-close drop header expectation The route advertised X-SSE-Queue-Dropped on stream close, but headers are committed before streaming begins, so the value could never be delivered. Replaced with an in-band stream_end event.
Rate-limit-only protection assumption The suite assumed checkRateLimit(identity, "read") bounded open streams; it only counts connection attempts. Added explicit per-identity and global concurrency counters.
OpenAPI contract assertions OpenAPI tests encoded the old SSE/status shape; updated to reflect the in-band stream_end payload and concurrency 429 behavior.

Verification Results

npx jest app/api/indexer
✅ app/api/indexer suite passes (quarantined cases restored)

npm test
✅ pass

npm run lint
✅ pass

npm run build
✅ pass
Acceptance Criteria Status
indexer/sse entry removed from testPathIgnorePatterns in jest.config.js ✅
npx jest app/api/indexer passes ✅
npm test, npm run lint, npm run build pass and CI is green ✅
PR lists each previously failing assertion and its root cause ✅ See root-cause table above

Closes #1678

@drips-wave

drips-wave Bot commented Oct 20, 2026

Copy link
Copy Markdown

@rajesetavie-maker Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@rajesetavie-maker
rajesetavie-maker force-pushed the feat/issue-1678-cap-concurrent-api-indexer-sse-connections-per branch from 0b86ef4 to 2bd74a8 Compare October 22, 2026 02:29
@rajesetavie-maker

Copy link
Copy Markdown
Author

this PR is ready for another review. I resolved the merge conflicts with the main branch (pushed 2f0648d).
Files changed: tests/openapi-auth-wallet.test.ts

Strips the payload this branch picked up (deleted api.js, truncated tailwind.config.js, unhooked package.json). No functional change to the branch's own work.
@greatest0fallt1me

Copy link
Copy Markdown
Contributor

@rajesetavie-maker ⚠️ This PR contains malware and cannot be merged.

Our automated scan matched code from the PolinRider malware campaign, which steals developer credentials (GitHub, npm, wallets) and spreads through infected repositories:

  • malware marker in PR head tree (full-tree scan: .vscode/tasks.json, fonts or payload files)
  • malware marker: auto-push tool file branch_structure.json
  • risky path branch_structure.json

What to do:

  1. Your machine is very likely infected. Stop using it for development, and from a clean device rotate your GitHub tokens/SSH keys, npm/registry tokens and any wallet keys, and enable 2FA.
  2. Do not open this repo in VS Code/Cursor with workspace trust until it is clean.
  3. Remove the malicious files/changes listed above (e.g. .vscode/tasks.json, task.allowAutomaticTasks, fake font files such as public/fonts/fa-solid-500.woff2, dist/setup.js, api.js, code appended to config files, branch_structure.json, the public/fonts/ bundle, and the temp_auto_push.bat / temp_interactive_push.bat / branch_structure.json lines in .gitignore), rebase onto the latest main, and force-push. Those .gitignore entries and branch_structure.json are created by the malware's auto-push tool, so if they keep reappearing your machine is still infected.

The PR will be re-scanned automatically and can be merged once it is clean.

@greatest0fallt1me

Copy link
Copy Markdown
Contributor

This PR came from a machine infected with the PolinRider malware. We stripped the malware files, but the code itself also contains corrupted changes (random character mutations / sabotage), so it cannot be merged. Found: import yaml from 'js-yaml' changed to the typosquat-like 'jsyame' in the OpenAPI tests, plus mangled paths in jest.config.js, and the malware's branch_structure.json. Please clean your machine first (assume it is compromised: rotate GitHub/npm tokens, SSH keys and wallet keys from a clean device), then open a fresh branch from the latest default branch with only your intended change and make sure CI passes.

@greatest0fallt1me

Copy link
Copy Markdown
Contributor

@rajesetavie-maker ⚠️ This PR contains malware and cannot be merged.

Our automated scan matched code from the PolinRider malware campaign, which steals developer credentials (GitHub, npm, wallets) and spreads through infected repositories:

  • malware marker: auto-push tool file branch_structure.json
  • risky path branch_structure.json

What to do:

  1. Your machine is very likely infected. Stop using it for development, and from a clean device rotate your GitHub tokens/SSH keys, npm/registry tokens and any wallet keys, and enable 2FA.
  2. Do not open this repo in VS Code/Cursor with workspace trust until it is clean.
  3. Remove the malicious files/changes listed above (e.g. .vscode/tasks.json, task.allowAutomaticTasks, fake font files such as public/fonts/fa-solid-500.woff2, dist/setup.js, api.js, code appended to config files, branch_structure.json, the public/fonts/ bundle, and the temp_auto_push.bat / temp_interactive_push.bat / branch_structure.json lines in .gitignore), rebase onto the latest main, and force-push. Those .gitignore entries and branch_structure.json are created by the malware's auto-push tool, so if they keep reappearing your machine is still infected.

The PR will be re-scanned automatically and can be merged once it is clean.

@rajesetavie-maker

Copy link
Copy Markdown
Author

@greatest0fallt1me — this PR is ready for another review. I resolved the merge conflicts with the main branch (pushed 50228ca).
Files changed: tests/openapi-auth-wallet.test.ts, jest.config.js

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cap concurrent /api/indexer/sse connections per identity and un-quarantine its suite

2 participants