Repository navigation
perf(sched): wake one worker per queued global EDF task - #789
Closed
atsushi421 wants to merge 3 commits into
Closed
atsushi421 wants to merge 3 commits into
atsushi421 wants to merge 3 commits into
Conversation
This was referenced Nov 13, 2026
3 tasks done
atsushi421
force-pushed
the
feat/edf-budget-global-wakes
branch
from
November 13, 2026 05:50
9f7658e to
183d771
Compare
atsushi421
changed the base branch from
feat/edf-idle-core-preemption
to
fix/sched-num-task-in-queue
November 13, 2026 05:51
Before this change, the EDF scheduler did not preempt if one core in the set of the woken task was idle. An idle core takes only one task, and it stays idle until the primary CPU wakes it. If several tasks woke in that window, all of them were enqueued, and only one of them got the idle core. The others waited behind running tasks with later deadlines. Now the scheduler skips preemption only if an idle core takes the task. An idle core takes the earliest queued task runnable on it. So it takes the woken task if the task is earlier than every queued task runnable on that core, or if the queue holds fewer tasks than the set has idle cores. Otherwise, the scheduler chooses a victim among the busy cores of the set. Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
NUM_TASK_IN_QUEUE underflowed to u32::MAX. ArcWake::wake incremented it, and get_next_task() decremented it when a non-clustered scheduler returned a task. Other paths did not match: - The re-wakes in do_preemption() and run_main(), and the re-wake of a preempted task, call wake_task() without ArcWake::wake, so their tasks were decremented but never incremented. - A task that invoke_preemption() pushed to a preemption-pending heap was incremented but runs without get_next_task(). - get_next() skips terminated and panicked tasks without a decrement. With the counter at u32::MAX, wake_workers() woke every sleeping worker on every loop of the primary CPU. Now the PrioritizedFIFO, PrioritizedRR, and Panicked schedulers increment the counter when they push a task to their queue and decrement it when they pop one. So the counter equals the number of queued non-clustered tasks on every path. These schedulers also wake the primary CPU after they enqueue a task, as the EDF scheduler does, because the re-wakes do not go through ArcWake::wake. The two target-specific definitions of the counter become one. Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
The EDF scheduler kept every task in one affinity queue, and wake_workers() woke every core in the affinity mask of that queue. One queued task whose set holds every worker core put every worker in the mask, so wake_workers() woke every sleeping worker. The extra workers found no task and slept again. Now the EDF scheduler keeps two queues. The global queue holds the tasks whose set holds every worker core, and the restricted queue holds the other tasks. wake_workers() wakes every core in the mask of the restricted queue, and one sleeping worker per task of the global queue, as it does for the tasks of the other schedulers. Scheduler::queued_cpu_mask() becomes queued_workers(), which also returns the number of global tasks. The queue key gets a sequence number, so tasks with equal deadlines and wake times still dequeue in push order across both queues. Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
romancardenas
force-pushed
the
feat/edf-budget-global-wakes
branch
from
November 16, 2026 00:20
183d771 to
302181f
Compare
romancardenas
force-pushed
the
fix/sched-num-task-in-queue
branch
from
November 16, 2026 05:11
d417ea8 to
47b6f5d
Compare
Contributor
Author
|
Closing this PR. I measured it again on current main, which includes #788 and #792. On QEMU ( |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR wakes one sleeping worker per queued EDF task whose set holds every worker core. It is the third PR of a stack on top of #787.
Before this PR, the EDF scheduler kept every task in one affinity queue, and
wake_workers()woke every core in the affinity mask of that queue. One queued task whose set holds every worker core put every worker in the mask, sowake_workers()woke every sleeping worker. The extra workers tookGLOBAL_WAKE_GET_MUTEX, found no task, and slept again. GEDF woke one worker per task, so #787 added this cost for every former GEDF task.Changes:
get_next()pops the earliest task of either queue that can run on the calling core.(absolute_deadline, wake_time, seq).seqgrows with each push, so tasks with equal deadlines and wake times still dequeue in push order across both queues.Scheduler::queued_cpu_mask()becomesqueued_workers(), which returns the mask of the restricted queue and the number of global tasks.wake_workers()wakes every core in the mask and one sleeping worker per global task, as it does for the tasks of the other schedulers.Related links
How was this PR tested?
scripts/ci.shpasses on my machine.scheduler/edf.rscheck thatpop_for_cputakes the earliest task of either queue, that equal keys are FIFO across both queues, that a restricted task stays off other cores, and that a global task stays out of the restricted mask.aarch64_virt, 4 CPUs),test_edflogs no[FAIL], every node oftest_dagruns, and every task oftest_multi_prioritized_schedulerstarts and finishes, with no panic.Notes for reviewers
With temporary counters during 30 seconds of
test_dagon QEMU (aarch64_virt, 4 CPUs),wake_workers()woke 13 workers on #792 (3 for the affinity mask and 10 for the budget) and 11 workers with this PR (0 and 11).test_dagrarely queues a global EDF task while workers sleep, so the saving is small there. It grows with the number of sleeping workers and of global EDF tasks that wait in the queue.I also ran a benchmark that wakes global EDF tasks while workers sleep. Eight
EDF(10 ms, CpuSet::all())tasks run 200 us every 10 ms, with releases spread 1.25 ms apart, for 10 seconds (8000 jobs). The table shows two runs per configuration on QEMU (aarch64_virt).wake_workers()This PR removes about 60% of the wakes by
wake_workers()with 8 CPUs. But on QEMU, these wakes are about 2% of the times that a worker woke and slept again without work. Most of those wakes come from another source, which I did not identify. The latency differences are within the run-to-run noise.Merge #792 first. This PR targets its branch.