Repository navigation
Conversation
… tick (tier4#729) `invoke_preemption_tick()` dequeued the head of the run queues whenever the quantum of the running task had expired, without comparing priorities. A lower-priority head was sent to the target CPU, rejected by `do_preemption()`, and re-queued at the tail of its level. Nothing restarted the quantum, so this repeated on every primary loop iteration: one IPI per iteration, and the FIFO order of the lower level was lost. - Add `Scheduler::get_next_if()` and `get_next_task_if()`, which dequeue the head only if a predicate accepts it, under the run queue lock. `PriorityQueue::peek()` is added for it. - The tick dequeues a task only if its priority is at least that of the running task and no preemption is pending on the CPU. Otherwise it restarts the quantum, or sends the IPI again if one is pending. - Panicked tasks are never taken by the tick: `Task::priority` keeps the priority they had before the panic. - The tick returns the time until the quantum expires, and `scheduler::wake_task()` caps the sleep of the primary CPU with it while a task is queued, so a restarted quantum is still checked on time. - Start the quantum of a preempted task that `do_preemption()` resumes directly; it did not pass through `run_main`, so its quantum looked expired at once. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
| // A queued task may be waiting for a time quantum to expire. Nothing | ||
| // else wakes the primary CPU at that time, so do not sleep beyond it. | ||
| // If no task is queued, the next enqueue wakes the primary CPU. | ||
| if NUM_TASK_IN_QUEUE.load(Ordering::Relaxed) == 0 { |
There was a problem hiding this comment.
This check runs before guard.wake_task(), so a task that a sleep handler enqueues in this same call is not counted, and quantum_wait is already dropped. wake_cpu(0) does nothing on CPU 0, so a timer-woken task with the same priority as the running PrioritizedRR tasks then waits for the next sleeper, or up to 1 s. Reading NUM_TASK_IN_QUEUE after guard.wake_task() closes this case.
| }); | ||
|
|
||
| if let Some(next_task) = next_task { | ||
| push_preemption_pending(cpu_id, next_task); |
There was a problem hiding this comment.
get_next_task_if releases GLOBAL_WAKE_GET_MUTEX before this push, and nothing checks again that task_id still runs on cpu_id. If that task finishes in this window, the worker can see an empty queue and no pending task, and go to sleep. The task pushed here then stays in the pending heap of an idle CPU, because the preempt IPI does nothing there and wake_workers() does not count pending tasks. main has the same window, but the idle check in run_main assumes that every pin is pushed under the mutex, as the wake paths do. Adding get_current_task(cpu_id) == Some(task_id) to the predicate and pushing before the mutex is released (for example in a callback on Taken) closes it.
| if ctx.is_some() { | ||
| // `next` resumes on its own thread without passing through | ||
| // `run_main`, so start its time quantum here. | ||
| task_info.update_last_executed(); |
There was a problem hiding this comment.
This covers only the resume branch, and it runs after set_current_task(cpu_id, next.id) has made next visible to the tick. On the branch without a saved context, last_executed_time stays stale until run_main updates it, so a tick in that window sees an expired quantum and an empty pending heap, and rotates next out before it runs. One update_last_executed() call just before set_current_task covers both branches.
| /// Get the next executable task. | ||
| fn get_next(&self, execution_ensured: bool) -> Option<Arc<Task>>; | ||
|
|
||
| /// Dequeue the task that `get_next(false)` would return, but only if |
There was a problem hiding this comment.
After this change, only run_main calls get_next_task, and it always passes true, so the execution_ensured == false branches of the four schedulers are dead code. This doc, the doc of get_next_task_if, and the comment in edf.rs that the new EDF get_next_if points to still describe a get_next_task(false) path on the primary CPU. Rewording them, or removing the execution_ensured parameter, keeps them accurate.
| task::{ | ||
| get_last_executed_by_task_id, get_task, get_tasks_running, set_current_task, | ||
| set_need_preemption, State, NUM_TASK_IN_QUEUE, | ||
| get_task, get_tasks_running, set_current_task, set_need_preemption, State, |
There was a problem hiding this comment.
This removes the only use of task::get_last_executed_by_task_id. The function is pub, so the compiler does not warn about it, and it can go in this PR too.
Description
Fixes #729.
PrioritizedRRScheduler::invoke_preemption_tick()dequeued the head of the run queues whenever the quantum of the running task had expired, without comparing priorities. A lower-priority head was sent to the target CPU, rejected bydo_preemption(), and re-queued at the tail of its priority level. Nothing restarted the quantum, so this repeated on every primary loop iteration.With this change the tick dequeues a task only if it can actually replace the running one, and restarts the quantum otherwise.
Changes
Scheduler::get_next_if()andscheduler::get_next_task_if(). They dequeue the head of the run queue only if a predicate accepts it, and the check and the dequeue are done under the run queue lock.PriorityQueue::peek()is added for this. PrioritizedFIFO and PrioritizedRR share the implementation (take_head_if()).GLOBAL_WAKE_GET_MUTEX, so a wake path cannot push a pending task between the check and the dequeue.last_executed_timeof the running task, so the run queues are not examined again on every iteration.Task::prioritykeeps the priority a task had before it panicked, so the priority comparison cannot judge such a task, and it could displace a healthy PrioritizedRR task.scheduler::wake_task()caps its return value with it while a task is queued. Without this, a task of equal priority that arrives just after a quantum restart waits until something else wakes the primary CPU (up to 1 s when the timer is enabled).do_preemption()now updateslast_executed_timewhen it switches directly to the saved thread of a preempted task. That path does not pass throughrun_main, so the quantum of the resumed task looked expired at once.mdbook/src/internal/scheduler.mdand the comments onTask::preempts()anddo_preemption()are updated.Behavior changes to be aware of
last_executed_timeis also whatawkernel_shellprints as "Last Executed". For a long-running PrioritizedRR task it is now the time of the last quantum restart, not the time of the last dispatch.Not addressed
A task that the tick (or a wake path) moved to a preemption-pending heap is still re-queued at the tail of its level if the running task changes before the IPI is handled. This PR removes the case that happened on every iteration; the remaining window is the one described in #703.
Verification
make check BSP=aarch64_virtandmake clippy: no warnings or errors.awkernel_lib,awkernel_async_lib,awkernel_drivers): all pass. A test forPriorityQueue::peek()is added.test_prioritized_rr(45 s): the 24 priority-1 tasks ran 359 to 369 times each; priority-0 tasks ran only while the tasks were being spawned.test_multi_prioritized_scheduler(30 s): all 30 tasks started and finished, on this branch and onmain.Comparison with
maintest_prioritized_rralways has an equal-priority task in the queue, so it does not hit #729. I measured a dedicated scenario instead: QEMU x86_64 with 16 CPUs (15 workers), 7 CPU-boundPrioritizedRR(1)tasks and 30 CPU-boundPrioritizedRR(0)tasks, so that onlyPrioritizedRR(0)tasks wait in the run queue. Temporary counters (not part of this PR) counted the IPIs sent by the tick and the pending tasks thatdo_preemption()rejected and re-queued.Cumulative values after 20 s:
mainPrioritizedRR(1)tasks (total)PrioritizedRR(0)task (min / max)mainthe tick sent about 10,000 IPIs per second for the first 10 s, and about half of them were rejected and re-queued. With this PR the rate is steady at about 1,900 per second, close to the 2,000 per second expected for 8 CPUs rotating every 4 ms.mainat least onePrioritizedRR(0)task made no progress for the whole 20 s. With this PR the minimum and the maximum differ by about 14%.This is one run per branch on QEMU without KVM, so the absolute values will differ on real hardware. On
mainthe IPI and preemption rates dropped to a few hundred per second after about 11 s and the rotation of thePrioritizedRR(0)tasks almost stopped; I did not investigate why.Related links
How was this PR tested?
scripts/ci.shpasses on my machine.Notes for reviewers