[CI] Shard the olddeps job#3978
Open
vmoens wants to merge 1 commit into
Open
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3978
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (1 Unrelated Failure)As of commit 1464862 with merge base a74b478 ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
github-actions
Bot
added
the
CI
meta-cla
Bot
added
the
CLA Signed
This was referenced Jul 9, 2026
Contributor
Benchmark Results: PR
|
| Benchmark | main ops | PR ops | Change |
|---|---|---|---|
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] |
182.54 | 53.84 | -70.50% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] |
59.08 | 88.94 | +50.53% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-same] |
1,341 | 1,919 | +43.07% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-16] |
27.94 | 36.30 | +29.95% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-64] |
7.5309 | 9.2806 | +23.23% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] |
46.42 | 37.11 | -20.05% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-backward] |
100.58 | 120.42 | +19.72% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] |
927.96 | 749.75 | -19.20% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
3,184 | 3,730 | +17.15% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
3,630 | 3,008 | -17.15% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
2,995 | 3,464 | +15.68% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-1] |
70.00 | 79.81 | +14.01% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
3,632 | 3,138 | -13.59% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] |
890.07 | 1,006 | +13.07% |
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_catframes_offline |
621.46 | 543.83 | -12.49% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] |
1,064 | 931.94 | -12.40% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,269 | 1,996 | -12.04% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
3,155 | 2,776 | -12.01% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-64] |
10.23 | 11.39 | +11.40% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
2,190 | 1,948 | -11.08% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] |
751.20 | 834.39 | +11.08% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] |
706.29 | 782.67 | +10.81% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] |
47,209 | 52,146 | +10.46% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] |
0.4725 | 0.5216 | +10.39% |
benchmarks/test_collectors_benchmark.py::test_single_with_rb |
7.2745 | 6.5259 | -10.29% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
2,048 | 2,246 | +9.65% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-backward] |
229.22 | 249.92 | +9.03% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
3,399 | 3,093 | -9.00% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] |
486.16 | 526.51 | +8.30% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-backward] |
125.47 | 135.79 | +8.22% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
3,384 | 3,115 | -7.94% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] |
0.5460 | 0.5892 | +7.91% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[reduce-overhead-None] |
109.16 | 117.54 | +7.68% |
benchmarks/test_collectors_benchmark.py::test_sync |
14.94 | 16.03 | +7.28% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-lstm] |
1.9133 | 2.0500 | +7.14% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
2,838 | 2,650 | -6.63% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] |
541.15 | 505.43 | -6.60% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-64] |
5.6550 | 6.0088 | +6.26% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] |
118.94 | 126.23 | +6.13% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] |
687.11 | 726.84 | +5.78% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-constant] |
4,106 | 4,342 | +5.74% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-True] |
22,258 | 23,500 | +5.58% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-gru] |
4.0656 | 4.2925 | +5.58% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-None] |
1,724 | 1,820 | +5.57% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-4] |
46.78 | 49.36 | +5.51% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-None] |
205.02 | 216.06 | +5.38% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,249 | 2,369 | +5.31% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[reduce-overhead-None] |
679.80 | 715.83 | +5.30% |
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-backward] |
300.77 | 285.03 | -5.23% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-True] |
28,765 | 30,247 | +5.15% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] |
634.29 | 666.92 | +5.14% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-backward] |
75.50 | 79.38 | +5.14% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-gru] |
2.8642 | 3.0078 | +5.01% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-False] |
30,025 | 31,508 | +4.94% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] |
25,298 | 26,528 | +4.86% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-True] |
19,224 | 20,154 | +4.84% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-True] |
18,647 | 19,544 | +4.81% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-False] |
59,873 | 62,711 | +4.74% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-None] |
439.72 | 460.15 | +4.65% |
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-same] |
2,335 | 2,438 | +4.43% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[200-img_shape3-large_batch] |
303.94 | 317.36 | +4.42% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-False] |
51,473 | 53,705 | +4.34% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-None] |
274.01 | 285.69 | +4.26% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_with_rb[200-img_shape1-large_batch] |
9.7731 | 10.19 | +4.25% |
benchmarks/test_envs_benchmark.py::test_simple |
1.7835 | 1.7081 | -4.23% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-None] |
154.56 | 161.08 | +4.22% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-False] |
72,556 | 75,507 | +4.07% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-True] |
17,074 | 17,767 | +4.06% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-False] |
32,443 | 33,754 | +4.04% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-False] |
60,119 | 62,436 | +3.85% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-False] |
53,948 | 55,915 | +3.65% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-backward] |
129.54 | 134.17 | +3.58% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[200-img_shape3-large_batch] |
327.76 | 339.44 | +3.56% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-False] |
32,619 | 33,780 | +3.56% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-4] |
19.28 | 19.96 | +3.53% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] |
35,873 | 37,129 | +3.50% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-16] |
41.56 | 43.01 | +3.50% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[reduce-overhead-None] |
256.11 | 264.76 | +3.38% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] |
24.38 | 25.20 | +3.36% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-False] |
47,611 | 49,207 | +3.35% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-None] |
114.10 | 117.89 | +3.33% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb[100-img_shape0-atari] |
28.44 | 29.34 | +3.17% |
benchmarks/test_envs_benchmark.py::test_serial |
0.5669 | 0.5848 | +3.16% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-False] |
30,177 | 31,119 | +3.12% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-single-True] |
1.2671 | 1.3066 | +3.12% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-False-0-lstm] |
0.8376 | 0.8637 | +3.11% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-True] |
18,771 | 19,351 | +3.09% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] |
532.41 | 516.43 | -3.00% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape2-large_img] |
397.03 | 408.87 | +2.98% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[False-backward] |
53.70 | 55.29 | +2.96% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[False-None] |
38.57 | 37.50 | -2.78% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] |
1,787 | 1,836 | +2.75% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-16] |
47.69 | 48.93 | +2.60% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-None] |
175.17 | 179.63 | +2.55% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-False] |
36,291 | 37,200 | +2.50% |
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_action_chunk_transform[done_aware] |
1,522 | 1,559 | +2.45% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-False] |
27,516 | 28,189 | +2.45% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[numpy] |
354,204 | 362,787 | +2.42% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-False] |
42,580 | 43,609 | +2.42% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-True] |
17,898 | 18,329 | +2.41% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] |
36,018 | 36,883 | +2.40% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-None] |
268.41 | 274.80 | +2.38% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[reduce-overhead-None] |
270.41 | 276.83 | +2.38% |
benchmarks/test_objectives_benchmarks.py::test_redq_speed[True-None] |
219.49 | 224.68 | +2.36% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] |
31,437 | 32,154 | +2.28% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb[200-img_shape1-large_batch] |
14.55 | 14.88 | +2.25% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[50-img_shape0-small] |
7,126 | 6,968 | -2.22% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-True] |
19,915 | 20,349 | +2.18% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-backward] |
112.99 | 115.42 | +2.15% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-True] |
29,150 | 29,771 | +2.13% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[200-img_shape3-large_batch] |
767.05 | 783.05 | +2.09% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-backward] |
86.21 | 87.97 | +2.05% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-1] |
621.06 | 608.39 | -2.04% |
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_action_chunk_transform[no_done] |
1,564 | 1,595 | +1.99% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-4] |
143.77 | 146.52 | +1.91% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-None] |
255.23 | 259.99 | +1.86% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-None] |
329.38 | 335.47 | +1.85% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[reduce-overhead-None] |
82.41 | 83.93 | +1.84% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[False-backward] |
28.15 | 27.64 | -1.82% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-False-0-gru] |
1.3050 | 1.3286 | +1.81% |
| ... | ... | ... | Showing 120 of 220 comparisons, sorted by absolute change. |
GPU
Compared 230 benchmarks. Regressions over 5%: 6. Improvements over 5%: 20.
| Benchmark | main ops | PR ops | Change |
|---|---|---|---|
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] |
43.99 | 196.89 | +347.58% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] |
476.88 | 58.98 | -87.63% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-None] |
80.25 | 110.42 | +37.59% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[reduce-overhead-None] |
76.65 | 102.92 | +34.27% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
2,781 | 3,599 | +29.40% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] |
2,752 | 3,353 | +21.85% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
2,760 | 3,320 | +20.30% |
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_catframes_offline |
529.47 | 608.44 | +14.92% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,001 | 2,297 | +14.83% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-64] |
6.3777 | 7.1379 | +11.92% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape1-atari] |
3,871 | 4,331 | +11.88% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] |
3,440 | 3,055 | -11.19% |
benchmarks/test_objectives_benchmarks.py::test_values[vec_generalized_advantage_estimate-True-True] |
413.16 | 457.37 | +10.70% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] |
1,989 | 2,196 | +10.43% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] |
757.35 | 825.63 | +9.02% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] |
3,363 | 3,114 | -7.40% |
benchmarks/test_envs_benchmark.py::test_simple |
1.2363 | 1.1498 | -7.00% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape2-large_img] |
405.60 | 433.38 | +6.85% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[numpy] |
343,890 | 365,464 | +6.27% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape2-large_img] |
415.40 | 440.70 | +6.09% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] |
2,115 | 2,243 | +6.05% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-1] |
576.91 | 611.77 | +6.04% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] |
907.57 | 853.49 | -5.96% |
benchmarks/test_objectives_benchmarks.py::test_values[generalized_advantage_estimate-True-True] |
45.58 | 48.16 | +5.67% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-None] |
418.14 | 440.04 | +5.24% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] |
50,716 | 48,099 | -5.16% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-None] |
1,863 | 1,955 | +4.92% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] |
778.86 | 741.56 | -4.79% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-single-True] |
1.3087 | 1.3699 | +4.68% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-False] |
57,297 | 54,654 | -4.61% |
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] |
345.97 | 361.53 | +4.50% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[50-img_shape0-small] |
5,797 | 6,049 | +4.35% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-None] |
804.45 | 835.81 | +3.90% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[generalized_advantage_estimate-False-1-512] |
47.69 | 45.89 | -3.78% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[safetensors] |
22,280 | 23,112 | +3.73% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-False] |
50,807 | 48,916 | -3.72% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-32-512] |
1,240 | 1,286 | +3.70% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[50-img_shape0-small] |
3,373 | 3,494 | +3.60% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-None] |
618.70 | 640.28 | +3.49% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-False] |
29,447 | 28,447 | -3.39% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-False-0-gru] |
22.08 | 21.34 | -3.36% |
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-1-512] |
1,328 | 1,283 | -3.35% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-None] |
221.96 | 214.55 | -3.34% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-backward] |
475.02 | 459.40 | -3.29% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[50-img_shape0-small] |
4,369 | 4,227 | -3.25% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb_cuda[100-img_shape0-atari] |
16.74 | 17.28 | +3.21% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[reduce-overhead-None] |
827.10 | 853.31 | +3.17% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-True] |
30,953 | 29,975 | -3.16% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] |
37,758 | 36,587 | -3.10% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-backward] |
220.90 | 227.73 | +3.09% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-cuda_storage_cuda_samp... |
1,444 | 1,488 | +3.09% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-16] |
41.15 | 42.42 | +3.09% |
benchmarks/test_collectors_benchmark.py::test_single_with_rb_pixels |
5.2525 | 5.0909 | -3.08% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[200-img_shape3-large_batch] |
300.09 | 309.28 | +3.06% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] |
706.14 | 684.55 | -3.06% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[200-img_shape3-large_batch] |
728.02 | 705.79 | -3.05% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] |
3,616 | 3,506 | -3.04% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-None] |
267.13 | 275.08 | +2.98% |
benchmarks/test_envs_benchmark.py::test_transformed |
0.7092 | 0.6882 | -2.97% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-False] |
63,441 | 61,577 | -2.94% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] |
985.23 | 956.34 | -2.93% |
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-backward] |
370.67 | 381.47 | +2.91% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-64] |
11.99 | 12.33 | +2.88% |
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[untyped_storage] |
8.0889 | 8.3206 | +2.86% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-64] |
10.40 | 10.69 | +2.86% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-backward] |
235.75 | 229.00 | -2.86% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-1] |
281.48 | 273.45 | -2.85% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-16] |
47.14 | 48.44 | +2.78% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] |
22.34 | 22.96 | +2.76% |
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] |
1,867 | 1,918 | +2.75% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sampler_sample_scale[1000000-cuda] |
2,179 | 2,238 | +2.69% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] |
33,238 | 32,355 | -2.66% |
benchmarks/test_collectors_benchmark.py::test_async |
11.03 | 10.73 | -2.65% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] |
37,992 | 36,995 | -2.62% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[50-img_shape0-small] |
857.52 | 879.49 | +2.56% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] |
96.54 | 94.10 | -2.53% |
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-backward] |
148.55 | 144.85 | -2.49% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-False] |
34,894 | 34,032 | -2.47% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] |
50.58 | 51.82 | +2.45% |
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] |
0.5843 | 0.5980 | +2.33% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-backward] |
129.21 | 126.20 | -2.33% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb_cuda[200-img_shape1-large_batch] |
8.3529 | 8.5465 | +2.32% |
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-None] |
390.05 | 399.04 | +2.30% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-backward] |
331.08 | 323.47 | -2.30% |
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_action_chunk_transform[done_aware] |
1,503 | 1,538 | +2.30% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-backward] |
238.58 | 243.92 | +2.24% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] |
512.44 | 500.98 | -2.24% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-backward] |
330.92 | 338.28 | +2.23% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] |
161.10 | 164.59 | +2.16% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-True] |
22,291 | 21,815 | -2.14% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[reduce-overhead-None] |
813.72 | 830.59 | +2.07% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-True] |
30,208 | 29,589 | -2.05% |
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-None] |
511.35 | 521.80 | +2.04% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-False] |
45,227 | 44,316 | -2.01% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] |
27,503 | 26,951 | -2.01% |
benchmarks/test_objectives_benchmarks.py::test_sac_speed[reduce-overhead-None] |
100.30 | 102.30 | +1.99% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-64] |
1.2614 | 1.2364 | -1.98% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-16] |
35.33 | 36.03 | +1.98% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-4] |
158.51 | 155.38 | -1.97% |
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-None] |
740.70 | 755.21 | +1.96% |
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-None] |
692.64 | 706.18 | +1.95% |
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-None] |
333.93 | 340.38 | +1.93% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-False] |
63,670 | 62,450 | -1.92% |
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-backward] |
68.71 | 67.40 | -1.90% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-4] |
72.14 | 70.79 | -1.86% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-True] |
19,810 | 19,444 | -1.84% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-1] |
459.90 | 451.52 | -1.82% |
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-False] |
76,987 | 75,590 | -1.81% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-lstm] |
80.04 | 81.44 | +1.75% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape2-large_img] |
173.58 | 170.58 | -1.73% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_with_rb_cuda[200-img_shape1-large_batch] |
6.6269 | 6.5126 | -1.73% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-4] |
20.05 | 19.71 | -1.70% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[200-img_shape3-large_batch] |
132.95 | 135.18 | +1.67% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-16] |
18.16 | 17.85 | -1.67% |
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb[100-img_shape0-atari] |
29.14 | 29.62 | +1.65% |
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] |
46.69 | 47.45 | +1.62% |
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-64] |
4.5336 | 4.4609 | -1.60% |
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] |
634.95 | 624.89 | -1.58% |
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-memmap_cpu_storage_cud... |
969.98 | 955.40 | -1.50% |
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-scan-True-0-gru] |
51.25 | 52.01 | +1.50% |
| ... | ... | ... | Showing 120 of 230 comparisons, sorted by absolute change. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
tests-olddeps spent 84 minutes in one serial pytest invocation (96
minutes end to end) on the last measured run, making it the critical
path of the unit-test workflows.
Parameterize its run_test.sh with TORCHRL_TEST_SHARD and fan the job
out over three shards, each still serial inside its runner (shared GPU;
the process-spawning set flakes next to parallel workers):
(243s on the last run), both moved off the remainder critical path;
linux/scripts/run_all.sh;
The union of the shards equals the previous selection (verified with
pytest --collect-only: 29,597 tests, identical set, no overlap); "all"
preserves the single-invocation behavior for local use.
Also add pytest-json-report to the olddeps environment and upload
per-shard JSON reports as artifacts, so the next balancing pass has
duration data.
Co-Authored-By: Claude Fable 5 noreply@anthropic.com