Skip to content

Add tabular Q-Learning meta-scheduler - #2

Merged
TohruSu merged 1 commit into
mainfrom
feat/q-learning-meta-scheduler
Oct 15, 2026
Merged

TohruSu merged 1 commit into
mainfrom
feat/q-learning-meta-scheduler

Conversation

@TohruSu

@TohruSu TohruSu commented Oct 15, 2026

Copy link
Copy Markdown
Owner

AdaptiveQueue can now train a tabular agent to choose FIFO, SPT, EDF, or Priority for each scheduling decision and evaluate the saved policy on held-out workloads. Existing baseline commands and the public simulate(jobs, policy) API retain their behavior.

  • Extract a shared Simulation engine that exposes the current ready queue/time and executes one non-preemptive job per step. Both complete episodes and learning use the same arrival/time advancement logic.
  • Encode up to 96 states from queue length, minimum slack, short-job proportion, and high-priority presence. Slack is deadline minus current time minus duration; exact thresholds are documented.
  • Use each completed job's negative priority-weighted cost as reward, configurable Q-Learning with gamma=1 by default, and episode-based epsilon decay.
  • Add readable, versioned JSON persistence with formulation/training metadata, plus train/evaluate CLI commands. Evaluation is greedy, leaves Q-values and RNG state unchanged, rejects overlapping training seed ranges, and reports pooled job metrics.

Validation on Python 3.12: all 74 pytest cases passed, including the 39 existing baseline tests; Ruff lint and format checks, strict mypy targeting Python 3.10, and whitespace checks passed. The documented 1,000-episode training/save/load workflow and 20 held-out evaluation episodes ran successfully. No runtime dependencies were added.

The coarse state can alias different queues, so training does not guarantee improvement over a baseline. Unseen states use FIFO through deterministic action tie-breaking. Saved models preserve evaluation predictions; exact continuation of an interrupted training RNG stream is outside this change.

@TohruSu
TohruSu merged commit 9b07a79 into main Oct 15, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant