Repository navigation
feat: build the vLLM engine settings in one place - #352
Merged
Merged
Conversation
Nothing uses these changes until the switch to sampling pools. RequestStore.set_future and get_future become set_result and get_result, and SamplingStore.commit becomes set_result, so both stores name the same operation the same way. The SDK endpoint retrieve_future and the Redis key open_rl:future:<id> keep their names. engine_args returns the vLLM engine settings a sampler uses, and serve_args renders them as vllm serve flags, so both launch styles get the same settings from the same env. The deploy config adds the sampler-router ConfigMap with Envoy and llm-d EPP config for each pool's CPU pod. RBAC lets the API server create a pool Deployment and NetworkPolicy, and lets the dispatcher list its replicas. The Workload CRD keeps pod template labels, which the EPP selects replicas by.
droot
approved these changes
Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR introduces a single place to build a sampler's vLLM engine settings,
engine_argsinvllm_options.py. Today these settings are built insidevllm_sampler.py, tied to the FFT flag, and only the in-process sampler can use them. With the multi-replica design (#324) we need the same settings in two places, for an engine inside the sampler process and asvllm serveflags for each pool replica (serve_args), and building them in one place keeps the two from drifting apart.A few settings change once samplers use it. Prefix caching is turned on, so vLLM reuses work on prompts that start the same way, which RL does constantly and the router relies on. vLLM stops applying a checkpoint's default sampling settings, such as Qwen3's temperature of 0.6, so a request gets exactly what the client asked for. Sleep mode is only enabled for a sampler that shares its GPU with the trainer, since that's the only one that sleeps. This PR only adds the function, so nothing changes at runtime until the switch later in the stack.
The rest is small.
set_futureandget_futureare renamed toset_resultandget_resultso both stores use the same name, and the deploy config adds the router ConfigMap, RBAC and Workload labels the multi-replica path needs later.