Reinforcement learning for LLM inference scheduling. DQN agent learns to balance throughput, TTFT, latency, and memory pressure vs FIFO/SJF/priority baselines.
reinforcement-learning pytorch dqn ppo kv-cache ml-systems gpu-scheduling llm-inference ai-infrastructure vectorized-environments inference-scheduling systems-rl
-
Updated
Jul 9, 2026 - Python