Skip to content

v0.7.1 fix(jobworker): skip error reporting for jobs cancelled during shutdown - #9

Merged
ungerik merged 1 commit into
mainfrom
ungerik/skip-canceled-job-errors
Jul 2, 2026
Merged

ungerik merged 1 commit into
mainfrom
ungerik/skip-canceled-job-errors

Conversation

@ungerik

@ungerik ungerik commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

A job dispatch cancelled during shutdown was still logged as ERROR and sent to
the OnError callback, even though the job is simply reset and retried (no retry
attempt consumed). This is expected shutdown noise, not a failure. This PR guards
both reporting points so a context.Canceled interruption is handled quietly.

What changed

  • jobworker/dojob.go — in DoJob, the OnError call and the "Job error"
    ERROR/WARN log are now skipped when the worker's error is (or wraps)
    context.Canceled. The error is still returned and job.ErrorMsg/ErrorData
    are still set, so doJobAndSaveResultInDB still detects the cancellation and
    resets/retries the job exactly as before.
  • jobworker/workerthreads.go — the worker dispatch loop skips OnError and
    the "Error while dispatching the job" ERROR log when
    doJobAndSaveResultInDB returns a context.Canceled (the shutdown reset path
    returns ctx.Err()).
  • jobworker/dojob_internal_test.go — new DB-free test: OnError is skipped
    for a direct and a wrapped context.Canceled, still fires for a genuine
    failure, and the cancellation still propagates out of DoJob.

The OnError contract ("called for every error that would also be logged") is
preserved: the log and the callback are skipped together.

Why it's correct

  • errors.Is(err, context.Canceled) traverses both errs.WrapWithFuncParams
    (via withCallStack.Unwrap) and errors.Join, so the guard fires on the
    wrapped return values (verified by the new test).
  • Per-job JobTimeout (context.DeadlineExceeded) is deliberately not
    suppressed — those jobs genuinely ran too long and still consume a retry.

Known consideration (not a blocker)

The guard suppresses any error whose chain contains context.Canceled, including
the rare case where a worker returns a cancel-wrapped error while the worker
thread's context is still alive (not a real shutdown). Such a job is reset and
retried without consuming an attempt — pre-existing behavior — but now with no
log/OnError signal. This matches the intended "a cancelled dispatch is always an
expected shutdown interruption" framing. If we later want to scope it strictly to
shutdown, gate the workerthreads.go guard on ctx.Err() != nil instead.

Testing

./scripts/run-tests.sh — go vet / revive / gosec clean, all packages pass
(jobqueue, jobworker, jobworkerdb, tests), including the new guard test.

Release

Bumps VERSION to v0.7.1 and adds the CHANGELOG entry.

A dispatch cancelled during shutdown is an expected interruption: the job is
reset and retried without consuming a retry attempt, so reporting it via
OnError or logging it at ERROR/WARN is misleading noise. DoJob and the worker
dispatch loop now skip OnError and the error/warning log when the error is (or
wraps) context.Canceled. Genuine failures and per-job JobTimeout
(context.DeadlineExceeded) are unaffected. Adds a DB-free test covering the
guard. Releases v0.7.1.
@ungerik
ungerik merged commit 8408759 into main Jul 2, 2026
2 checks passed
@ungerik
ungerik deleted the ungerik/skip-canceled-job-errors branch July 2, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant