Skip to content

Commit c1cecf8

Browse files
Sebastian Andrzej SiewiorPeter Zijlstra
authored andcommitted
sched: Cache task_struct::flags in sched_submit_work()
sched_submit_work() is considered to be a hot path. The preempt_disable() instruction is a compiler barrier and forces the compiler to load task_struct::flags for the second comparison. By using a local variable, the compiler can load the value once and keep it in a register for the second comparison. Verified on x86-64 with gcc-10. Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Link: https://lkml.kernel.org/r/20200819200025.lqvmyefqnbok5i4f@linutronix.de
1 parent 01ccf59 commit c1cecf8

1 file changed

Lines changed: 5 additions & 2 deletions

File tree

kernel/sched/core.c

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4551,9 +4551,12 @@ void __noreturn do_task_dead(void)
45514551

45524552
static inline void sched_submit_work(struct task_struct *tsk)
45534553
{
4554+
unsigned int task_flags;
4555+
45544556
if (!tsk->state)
45554557
return;
45564558

4559+
task_flags = tsk->flags;
45574560
/*
45584561
* If a worker went to sleep, notify and ask workqueue whether
45594562
* it wants to wake up a task to maintain concurrency.
@@ -4562,9 +4565,9 @@ static inline void sched_submit_work(struct task_struct *tsk)
45624565
* in the possible wakeup of a kworker and because wq_worker_sleeping()
45634566
* requires it.
45644567
*/
4565-
if (tsk->flags & (PF_WQ_WORKER | PF_IO_WORKER)) {
4568+
if (task_flags & (PF_WQ_WORKER | PF_IO_WORKER)) {
45664569
preempt_disable();
4567-
if (tsk->flags & PF_WQ_WORKER)
4570+
if (task_flags & PF_WQ_WORKER)
45684571
wq_worker_sleeping(tsk);
45694572
else
45704573
io_wq_worker_sleeping(tsk);

0 commit comments

Comments
 (0)