You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
stop API-mode DB-log scheduling from pre-opening unused SST receivers
keep SQL-mode receiver behavior unchanged
add regression coverage for API-mode no-preopen and SQL-mode compatibility
Problem
In API mode, scheduled DB-log tasks were opening an SST receiver during scheduling, but dbjobs later requested a fresh receiver through receive-task/{taskname} and used that one instead. The first listener was left idle until its accept deadline expired, producing delayed accept tcp ... i/o timeout log noise.
Fix
For the four scheduler DB-log paths (errorlog, auditlog, sqlerrorlog, slowquery), return early in API mode and queue the task with port "0" instead of pre-opening an SST receiver. SQL mode still opens the receiver up front.
Traced the fix end-to-end against the API-mode dispatch path (jobInsertTask in cluster/srv_job.go) and the on-demand receiver (handlerMuxServerReceiveTask in server/api_database.go), and it holds up:
In API mode, jobInsertTask's if cluster.Conf.SchedulerJobsMode == "api" branch never reads the port argument at all — it only claims the task slot and writes a cookie. So swapping the real SSTRunReceiverToDBLogFile(...) call for a literal "0" is behaviorally inert from JobInsertTask's perspective; nothing downstream persists or depends on that value in API mode.
The real receiver is opened later, on demand, by handlerMuxServerReceiveTask, which for errorlog/auditlog/sqlerrorlog/slowquery calls the same SSTRunReceiverToDBLogFile with the same DBLogKindFromTaskName mapping — so API-mode and scheduler-mode still land in the same canonical log file. This confirms the orphaned first listener really was pure waste (the accept-deadline-timeout noise described in the linked issue), not a listener anything relied on.
All four touched task names (errorlog, slowquery, sqlerrorlog, auditlog) are registered as TaskCapRemoteOnly in config/config.go, so IsRemoteTask correctly routes them through the cookie-write path both before and after this change — no behavior change there.
SQL mode is untouched (the early return is gated strictly behind SchedulerJobsMode == "api"), matching the PR description that SQL-mode's polling-based dbjobs script still needs the pre-opened receiver's port written into the jobs table.
Tests: TestJobBackupDBLog_APIMode_DoesNotPreOpenReceiver and TestJobBackupDBLog_SQLMode_StillOpensReceiver are solid regression coverage — they snapshot the real SSTs.SSTconnections map before/after, assert on the diff (not "whatever's in the map now," which avoids cross-test pollution), and use a locked poll-with-timeout to wait for the listener's own deferred cleanup before asserting, rather than racing it. The SstAvailablePorts nil-map footgun in the SQL-mode test setup (SSTSenderFreePort writes into it during cleanup) is called out and handled correctly. I traced SSTGetSenderPort/SSTRunReceiverToFile/tcp_con_handle_to_file to confirm the map keying and cleanup assumptions the tests make are accurate for both the DBLogRotate and non-rotate code paths.
I wasn't able to actually execute go test in this sandboxed session (shell commands beyond read-only inspection required approval that wasn't available here), so this review is based on static tracing of the full call graph rather than an observed test run — the author's own go test ./cluster -run 'TestJobBackupDBLog_' in CI is the authoritative check.
Minor nit (non-blocking): the four call sites repeat an identical 3-4 line comment + if cluster.Conf.SchedulerJobsMode == "api" { return server.JobInsertTask(task, "0", cluster.Conf.MonitorAddress) } block verbatim. Given all four already share the task/early-return shape, this could be a small helper (e.g. server.queueAPIModeLogTask(task string) (int64, error)) to avoid four copies drifting out of sync if the API-mode dispatch logic ever changes. Purely stylistic — I would not block on it.
No security, performance, or test-coverage concerns beyond the above. The fix is narrowly scoped, matches the stated problem, and the new tests would have caught the regression this PR fixes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Problem
In API mode, scheduled DB-log tasks were opening an SST receiver during scheduling, but dbjobs later requested a fresh receiver through
receive-task/{taskname}and used that one instead. The first listener was left idle until its accept deadline expired, producing delayedaccept tcp ... i/o timeoutlog noise.Fix
For the four scheduler DB-log paths (
errorlog,auditlog,sqlerrorlog,slowquery), return early in API mode and queue the task with port"0"instead of pre-opening an SST receiver. SQL mode still opens the receiver up front.Testing
go test ./cluster -run 'TestJobBackupDBLog_'Related to #1693