Repository navigation
Clean up old vLLM compile caches on the shared volume #376
Description
Problem
With #358, samplers keep vLLM's compile cache on the shared volume at $OPEN_RL_TMP_DIR/vllm-cache. vLLM saves each compile in a folder named by a hash of the vLLM and torch versions, the GPU, the model code and the engine settings. Changing any of these creates a new folder, and nothing deletes the old ones.
It's small. In the H100 + L4 fleet test the cache came to 429 MB:
| Setup | Size |
|---|---|
| gemma-4-E2B full fine-tuning (H100) | 168 MB |
| gemma-4-E2B LoRA (H100) | 145 MB |
| Qwen3-0.6B LoRA (L4) | 60 MB |
| Qwen3-0.6B full fine-tuning (L4) | 51 MB |
On a 1 TiB Filestore, that's about 2,000 vLLM upgrades before it fills.
Options
- Per-version folders that clean themselves up. Each sampler puts its cache in
vllm-cache/<vllm version>/and touches alast_usedfile there on every start, then deletes other version folders unused for N days. It never deletes a cache in use, but it doesn't clean up folders left by settings changes within one version. - CronJob that deletes old folders. Removes hash folders older than N days. Simple, but reads don't update a folder's modified time, so it can delete a cache that's still in use. That costs one recompile.
- Size cap. Deletes the oldest folders when
vllm-cache/grows past a limit. This bounds disk use directly, with the same modified-time problem.
Activity