srun fans out multiple processes if ntasks is not set explicitly
Issue
With experiments_per_job: 1, the sbatch template runs each experiment via a bare srun (no --ntasks). If cpus-per-task is set but a task count isn't, srun's default becomes floor(alloc_cpus / cpus_per_task). On a partition that grants more CPUs than requested (e.g. OverSubscribe=EXCLUSIVE / whole-node), one experiment is launched as N concurrent processes.
Cause
seml/commands/start.py (v0.5.6, ~L203):
srun_str = '' if experiments_per_job > 1 else 'srun '
This bare 'srun ' is substituted as {maybe_srun} into seml/templates/slurm/slurm_template.sh with no -n. Per SLURM manpage on srun:
If -c is specified without -n, as many tasks will be allocated per node as possible while satisfying the -c restriction.
Suggested fix
Pin one task in the single-experiment path, while still honoring an explicit user task count:
srun_str = ""
if experiments_per_job <= 1:
_task_keys = ('ntasks', 'n', 'ntasks-per-node', 'ntasks-per-gpu', 'ntasks-per-socket')
srun_str = 'srun ' if any(k in sbatch_options for k in _task_keys) else 'srun --ntasks=1 '
Workaround
(or why I am not submitting a PR rn)
Add ntasks: 1 to sbatch_options. Explicitly setting it will populate the corresponding env var, and srun will inherit the setting.
srunfans out multiple processes ifntasksis not set explicitlyIssue
With experiments_per_job: 1, the sbatch template runs each experiment via a bare srun (no --ntasks). If cpus-per-task is set but a task count isn't, srun's default becomes floor(alloc_cpus / cpus_per_task). On a partition that grants more CPUs than requested (e.g. OverSubscribe=EXCLUSIVE / whole-node), one experiment is launched as N concurrent processes.
Cause
seml/commands/start.py(v0.5.6, ~L203):This bare 'srun ' is substituted as
{maybe_srun}intoseml/templates/slurm/slurm_template.shwith no -n. Per SLURM manpage on srun:Suggested fix
Pin one task in the single-experiment path, while still honoring an explicit user task count:
Workaround
(or why I am not submitting a PR rn)
Add
ntasks: 1tosbatch_options. Explicitly setting it will populate the corresponding env var, andsrunwill inherit the setting.