Skip to content

Upgrade warning: commits inside jobs

Jump to bottom
Guewen Baconnier edited this page Apr 9, 2026 · 3 revisions

Commits during execution of a job

TL;DR

Commits during the execution of a job are now forbidden, because they may provoke double-execution of the same job. An the latest versions, a new option on the Job Function can be enabled to allow commits, by opening a new transaction, meaning it requires a additional connection to the database.

Explanation

A long standing issue in the job queue has been that a worker getting hard killed while processing a job would leave a job stuck in a started state. This issue has been solved in versions 16.0.2.10.0, 17.0.1.2.0, 18.0.2.0.0, 19.0.1.0.0 (#716). When a worker starts a job, it acquires a lock on a record in a queue_job_lock table. The jobrunner skips jobs as long as they are locked. If the worker crashes, the lock is released so the jobrunner can reset the status and restart the aborted job.

If, during its execution, a job commits the transaction, it releases immediately this lock. The jobrunner may reset and restart the job, while the first run is still ongoing.

Since versions 16.0.2.13.2, 18.0.2.1.0, 19.0.1.1.0 (#880), commits in jobs are forbidden, considering that double execution can have severe side effects in some cases, and that in most cases, jobs do not need to commit. See also the discussion on #889

In some cases though, a commit may still be necessary. The connector addon for instance commits directly after having created a record on a external system, to ensure we do not lose the external ID and create it twice in case of error. The "Scheduled Actions as Queue Jobs" addon may run a batch and have intermediate commits. For these special cases, a new option (#899) can be enabled on the Job Function, which allow to commit inside of the job. It is important to understand what it does under the hood: it runs the whole job's method inside a new transaction. Which means that every time a worker starts a job, it will open two database connections: one for handling the job's status and the lock and a second one to isolate the job's execution. Do not enable this option if not needed.

What should you do?

Job requeuer

The job requeuer has been added in:

  • 16.0.2.10.0
  • 17.0.1.2.0
  • 18.0.2.0.0
  • 19.0.1.0.0

If your installed addon version is below, you are not using the new dead job requeuer. You may want to upgrade to benefit from it though, but otherwise you do not have any action to take.

Protection against double-execution

The protection that forbids commits has been added in versions:

  • 16.0.2.13.2
  • 17.0: unreleased
  • 18.0.2.1.0
  • 19.0.1.1.0

If your installed addon version is between the introduction of the job requeuer and these, it means you have no protection against double-execution, it is advised to upgrade as soon as possible. For version 17.0, the protection has not been released yet, it will be soon alongside the new "allow commit" option.

If you have jobs with the following error

Commit is forbidden in queue jobs. If the current job is a cron running as queue job, modify it to run as a normal cron.

You have several choices, depending on your situation:

  • If the job is a cron (with "Scheduled Actions as Queue Jobs"), modify it to run as a normal cron
  • If the commit is actually not necessary in the job, remove it
  • Upgrade to the latest version (to be released, work in progress in #899)

New allow commit option

The new option to allow commit on a job function basis has been added in

Monitor your jobs closely

  • If you do have any errors due to commits, you should have nothing to do
  • If you do have errors, and the commits cannot be removed from the code of the jobs, enable the option of the Job Function only for the concerned job

Clone this wiki locally