Repository navigation
fix(sched): ignore idle cores in EDF preemption, as GEDF did - #788
Merged
Merged
Conversation
This was referenced Nov 13, 2026
3 tasks done
Before this change, the EDF scheduler did not preempt if one core in the set of the woken task was idle. An idle core takes only one task, and it stays idle until the primary CPU wakes it. If several tasks woke in that window, all of them were enqueued, and only one of them got the idle core. The others waited behind running tasks with later deadlines. Now the scheduler skips preemption only if an idle core takes the task. An idle core takes the earliest queued task runnable on it. So it takes the woken task if the task is earlier than every queued task runnable on that core, or if the queue holds fewer tasks than the set has idle cores. Otherwise, the scheduler chooses a victim among the busy cores of the set. Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
romancardenas
force-pushed
the
feat/edf-idle-core-preemption
branch
from
November 16, 2026 00:20
0d2157c to
9176f0f
Compare
If an idle core of the set took the woken task, the previous commit skipped preemption. That rule judges only the woken task. It does not judge again a queued task that a later task pushes off the idle core. For example, take one idle core and two busy cores that run tasks with deadlines 50 and 40. If a task with deadline 12 wakes before a task with deadline 10, the rule queues both without preemption. The idle core takes the task with deadline 10, and the task with deadline 12 waits while the tasks with deadlines 50 and 40 run. If the sets differ, a rule that compares the queue length with the number of idle cores also fails. For example, cores 2 and 3 are idle, and the rule queues two tasks. Core 2 can take the one that core 3 can also run. Then the other task, whose set holds cores 1 and 2, waits behind a task with a later deadline on core 1. Now the scheduler ignores idle cores, as the GEDF scheduler did before PR #787. It chooses the victim among the busy cores of the set. If the woken task has a higher priority than the task on that core, the scheduler preempts it. If an idle core can take the task, the preemption is unneeded. But a sleeping core runs a queued task only after the primary CPU wakes it. With this rule, the preempted task waits for that wake instead of the woken task. If the set of a task holds one core, the behavior does not change. That set has no busy core while its core is idle. Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
A waker reads `RUNNING` of a CPU and pins a task to the preemption-pending heap of that CPU under `GLOBAL_WAKE_GET_MUTEX`. But the CPU clears `RUNNING` and empties the heap without that mutex. If the CPU finishes its task and empties the heap between the read and the pin, it finds no queued task and sleeps. While the CPU runs no task, the preemption IPI can leave the task pinned, and `wake_workers()` does not see the heap. Then the task waits until an unrelated interrupt wakes the CPU. Now `run_main()` checks the heap again before the CPU sleeps. The check comes after `get_next_task()`, which takes `GLOBAL_WAKE_GET_MUTEX`. So the check sees the pin of every waker that read the CPU as busy. On main, `NUM_TASK_IN_QUEUE` underflows, so `wake_workers()` wakes every sleeping worker on each loop of the primary CPU. This hides the race. PR #792 fixes the underflow, so this fix must come first. With the previous commit, the EDF scheduler also pins a task while its set has an idle core. So the race can leave those idle cores asleep too. Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
- Skip a CPU whose running task is missing from the task table, as GEDF did. Before this change, such a CPU stopped the whole decision. - In the comment and the mdbook, say that the woken task can preempt even if the set has an idle core, and give the costs. The preemption can be unneeded, the victim waits for the primary CPU to wake a sleeping core, and a victim whose set holds no idle core waits for its own core. - In the mdbook, say that a preempting task goes to the preemption-pending heap of the victim core and not to the queue. Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The EDF scheduler now ignores idle cores when it decides whether a woken task preempts, as the GEDF scheduler did before #787.
Since #787, the scheduler queues a woken task without preemption if one core of its set is idle. But an idle core takes only one task, and a sleeping core runs a queued task only after the primary CPU wakes it. If several tasks wake in that window, the scheduler queues all of them, and only one gets the idle core. The others wait behind running tasks with later deadlines. A DAG node that publishes to several subscribers causes such a burst. GEDF preempted in this case, so #787 brought the problem to every former GEDF task.
Changes:
invoke_preemption()skips idle cores and chooses the victim among the busy cores of the set. If the woken task has a higher priority than the task on that core, the scheduler preempts it. If the set has no busy core, the scheduler queues the task.invoke_preemption()also skips a core whose running task is missing from the task table, as GEDF did. That task finished after the scheduler read the core.run_main()checks its preemption-pending heap again. Without this check, a task that a waker pins just after the worker empties the heap stays there until an unrelated interrupt. With the new rule, the scheduler also pins tasks while the set has idle cores, so the race can leave those cores asleep too.The rule has these costs:
If the set of a task holds one core, the behavior does not change.
Related links
How was this PR tested?
scripts/ci.shpasses on my machine.aarch64_virt, 4 CPUs),test_edflogs no[FAIL], and every node oftest_dagruns.test_multi_prioritized_schedulerstarts and finishes. No test panics.Notes for reviewers
A rule that skips preemption only if an idle core will take the woken task does not work either. It judges only the woken task. Take one idle core. If a task with deadline 12 wakes before a task with deadline 10, the rule queues both, and the task with deadline 12 waits behind running tasks with later deadlines. GEDF preempts for both. If the sets differ, the rule can also misjudge. With cores 2 and 3 idle, core 2 can take a task that core 3 can also run, and leave a task with set {1, 2} behind a later deadline on core 1.