Skip to content

fix(sched): ignore idle cores in EDF preemption, as GEDF did - #788

Merged
romancardenas merged 4 commits into
mainfrom
feat/edf-idle-core-preemption
Nov 16, 2026
Merged

romancardenas merged 4 commits into
mainfrom
feat/edf-idle-core-preemption

Conversation

@atsushi421

@atsushi421 atsushi421 commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Description

The EDF scheduler now ignores idle cores when it decides whether a woken task preempts, as the GEDF scheduler did before #787.

Since #787, the scheduler queues a woken task without preemption if one core of its set is idle. But an idle core takes only one task, and a sleeping core runs a queued task only after the primary CPU wakes it. If several tasks wake in that window, the scheduler queues all of them, and only one gets the idle core. The others wait behind running tasks with later deadlines. A DAG node that publishes to several subscribers causes such a burst. GEDF preempted in this case, so #787 brought the problem to every former GEDF task.

Changes:

  • invoke_preemption() skips idle cores and chooses the victim among the busy cores of the set. If the woken task has a higher priority than the task on that core, the scheduler preempts it. If the set has no busy core, the scheduler queues the task.
  • invoke_preemption() also skips a core whose running task is missing from the task table, as GEDF did. That task finished after the scheduler read the core.
  • Before an idle worker sleeps, run_main() checks its preemption-pending heap again. Without this check, a task that a waker pins just after the worker empties the heap stays there until an unrelated interrupt. With the new rule, the scheduler also pins tasks while the set has idle cores, so the race can leave those cores asleep too.
  • The mdbook describes the new rule, its costs, and that an EDF task preempts a task of a lower-priority scheduler regardless of its deadline.

The rule has these costs:

  • If an idle core can take the woken task, the preemption is unneeded.
  • The preempted task waits for the primary CPU to wake a sleeping core. With the feat(sched): merge GEDF into ClusteredEDF and rename it to EDF #787 rule, the woken task waited instead, although it has the earlier deadline.
  • If the set of the preempted task holds no idle core, the preempted task waits for its own core while the idle core stays idle. GEDF had no CPU sets, so it did not have this case.

If the set of a task holds one core, the behavior does not change.

Related links

How was this PR tested?

  • scripts/ci.sh passes on my machine.
  • On QEMU (aarch64_virt, 4 CPUs), test_edf logs no [FAIL], and every node of test_dag runs.
  • On QEMU, every task of test_multi_prioritized_scheduler starts and finishes. No test panics.

Notes for reviewers

A rule that skips preemption only if an idle core will take the woken task does not work either. It judges only the woken task. Take one idle core. If a task with deadline 12 wakes before a task with deadline 10, the rule queues both, and the task with deadline 12 waits behind running tasks with later deadlines. GEDF preempts for both. If the sets differ, the rule can also misjudge. With cores 2 and 3 idle, core 2 can take a task that core 3 can also run, and leave a task with set {1, 2} behind a later deadline on core 1.

@atsushi421
atsushi421 added this pull request to stack #791 November 13, 2026 05:21
@atsushi421
atsushi421 removed this pull request from stack #791 November 13, 2026 05:51
@atsushi421
atsushi421 added this pull request to stack #793 November 13, 2026 05:53
Base automatically changed from feat/merge-gedf-into-edf to main November 16, 2026 00:20
Before this change, the EDF scheduler did not preempt if one core in the
set of the woken task was idle. An idle core takes only one task, and it
stays idle until the primary CPU wakes it. If several tasks woke in that
window, all of them were enqueued, and only one of them got the idle
core. The others waited behind running tasks with later deadlines.

Now the scheduler skips preemption only if an idle core takes the task.
An idle core takes the earliest queued task runnable on it. So it takes
the woken task if the task is earlier than every queued task runnable on
that core, or if the queue holds fewer tasks than the set has idle cores.
Otherwise, the scheduler chooses a victim among the busy cores of the set.

Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
@romancardenas
romancardenas force-pushed the feat/edf-idle-core-preemption branch from 0d2157c to 9176f0f Compare November 16, 2026 00:20
If an idle core of the set took the woken task, the previous commit
skipped preemption. That rule judges only the woken task. It does not
judge again a queued task that a later task pushes off the idle core.
For example, take one idle core and two busy cores that run tasks with
deadlines 50 and 40. If a task with deadline 12 wakes before a task with
deadline 10, the rule queues both without preemption. The idle core
takes the task with deadline 10, and the task with deadline 12 waits
while the tasks with deadlines 50 and 40 run.

If the sets differ, a rule that compares the queue length with the
number of idle cores also fails. For example, cores 2 and 3 are idle,
and the rule queues two tasks. Core 2 can take the one that core 3 can
also run. Then the other task, whose set holds cores 1 and 2, waits
behind a task with a later deadline on core 1.

Now the scheduler ignores idle cores, as the GEDF scheduler did before
PR #787. It chooses the victim among the busy cores of the set. If the
woken task has a higher priority than the task on that core, the
scheduler preempts it. If an idle core can take the task, the preemption
is unneeded. But a sleeping core runs a queued task only after the
primary CPU wakes it. With this rule, the preempted task waits for that
wake instead of the woken task.

If the set of a task holds one core, the behavior does not change. That
set has no busy core while its core is idle.

Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
@atsushi421 atsushi421 changed the title fix(sched): preempt when queued EDF tasks claim every idle core fix(sched): ignore idle cores in EDF preemption, as GEDF did Nov 16, 2026
A waker reads `RUNNING` of a CPU and pins a task to the
preemption-pending heap of that CPU under `GLOBAL_WAKE_GET_MUTEX`. But
the CPU clears `RUNNING` and empties the heap without that mutex. If
the CPU finishes its task and empties the heap between the read and the
pin, it finds no queued task and sleeps. While the CPU runs no task,
the preemption IPI can leave the task pinned, and `wake_workers()` does
not see the heap. Then the task waits until an unrelated interrupt
wakes the CPU.

Now `run_main()` checks the heap again before the CPU sleeps. The check
comes after `get_next_task()`, which takes `GLOBAL_WAKE_GET_MUTEX`. So
the check sees the pin of every waker that read the CPU as busy.

On main, `NUM_TASK_IN_QUEUE` underflows, so `wake_workers()` wakes
every sleeping worker on each loop of the primary CPU. This hides the
race. PR #792 fixes the underflow, so this fix must come first. With
the previous commit, the EDF scheduler also pins a task while its set
has an idle core. So the race can leave those idle cores asleep too.

Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>
- Skip a CPU whose running task is missing from the task table, as GEDF did. Before this change, such a CPU stopped the whole decision.
- In the comment and the mdbook, say that the woken task can preempt even if the set has an idle core, and give the costs. The preemption can be unneeded, the victim waits for the primary CPU to wake a sleeping core, and a victim whose set holds no idle core waits for its own core.
- In the mdbook, say that a preempting task goes to the preemption-pending heap of the victim core and not to the queue.

Signed-off-by: atsushi421 <atsushi.yano.2@tier4.jp>

@romancardenas romancardenas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@romancardenas
romancardenas merged commit d55d756 into main Nov 16, 2026
1 check passed
@romancardenas
romancardenas deleted the feat/edf-idle-core-preemption branch November 16, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants