Skip to content

[CI] Fix always-true pull_request guard in docs workflow#3975

Open
vmoens wants to merge 2 commits into
mainfrom
claude/gracious-bhaskara-305b25
Open

[CI] Fix always-true pull_request guard in docs workflow#3975
vmoens wants to merge 2 commits into
mainfrom
claude/gracious-bhaskara-305b25

Conversation

@vmoens

@vmoens vmoens commented Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Description

Two CI fixes.

1. Always-true pull_request guard in docs.yml

The build-docs script ended with:

if [[ ${{ github.event_name == 'pull_request' }} ]]; then
  cp -r docs/_local_build/* "${RUNNER_DOCS_DIR}"
fi

The ${{ ... }} expression renders to the literal string true or false, and [[ false ]] tests string non-emptiness, so the condition was always truthy and the RUNNER_DOCS_DIR copy ran on every event (push, tags, workflow_dispatch), not just PRs. Fixed to a proper string comparison. Also audited the three stable-vs-nightly conditions in the same script: those use genuine bash operators on individually-interpolated values and evaluated correctly, but the interpolations were unquoted (an empty value would splice into a bash syntax error) - quoted them, leaving the release/* glob unquoted so pattern matching still works.

Behavioral note: push/tag builds no longer copy docs into RUNNER_DOCS_DIR (docs-preview dir). The upload job is unaffected; it consumes RUNNER_ARTIFACT_DIR, which is still populated unconditionally.

2. Gate subprocess coverage tracing to xdist runs

tests-optdeps has timed out at the 2h limit on every run since the pytest-xdist stack landed (#3972 et al.), across three different tensordict versions. Root cause: coverage_run_parallel.py started exporting COVERAGE_PROCESS_START unconditionally, which makes coverage's a1_coverage.pth site hook start a tracer in every Python interpreter spawned under the test run - parallel-env workers, collector workers, Ray workers - not just the pytest-xdist execnet workers it was added for. A tracer roughly doubles worker startup (measured: torch+torchrl import 0.8s -> 1.5s) and line-traces everything the worker executes (the generated config has no source filter). Per-file timings on the timed-out runs show a diffuse 30-80% slowdown concentrated in process-heavy files (test_collectors.py alone +15.6min), pushing optdeps from ~88min of pytest past its 2h budget.

The export is now opt-in via TORCHRL_COV_TRACE_SUBPROCESSES=1, set only by the xdist invocations in linux/scripts/run_all.sh and stripped from the serial quarantine remainder that runs in the same shell. Serial suites (optdeps, shard 2, non-xdist paths) return to pre-export behavior; xdist workers keep contributing coverage.

🤖 Generated with Claude Code

@vmoens @claude
[CI] Fix always-true pull_request guard in docs workflow
9ef6081 
The RUNNER_DOCS_DIR copy was gated by
[[ ${{ github.event_name == 'pull_request' }} ]], which renders to the
literal string "true" or "false" -- both non-empty, so the copy ran on
every event. Compare the event name as a string instead.

Also quote the interpolated GitHub context values in the three
stable-vs-nightly conditions so an empty or unusual value cannot splice
into a bash syntax error.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@pytorch-bot

pytorch-bot Bot commented Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/3975

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 82f82a8 with merge base a74b478 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 9, 2026
@github-actions github-actions Bot added the CI Has to do with CI setup (e.g. wheels & builds, tests...) label Jul 9, 2026
@github-actions

github-actions Bot commented Jul 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmark Results: PR 82f82a88 vs main a74b478f

Benchmark run: https://github.com/pytorch/rl/actions/runs/29075281994

Higher ops/sec is better. Tables are sorted by largest absolute change.

CPU

Compared 220 benchmarks. Regressions over 5%: 21. Improvements over 5%: 18.

Benchmark main ops PR ops Change
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 46.26 191.74 +314.45%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 192.05 51.44 -73.22%
benchmarks/test_objectives_benchmarks.py::test_values[vec_td1_return_estimate-False-False] 55.27 87.01 +57.42%
benchmarks/test_objectives_benchmarks.py::test_values[vec_td_lambda_return_estimate-True-False] 55.62 87.39 +57.11%
benchmarks/test_objectives_benchmarks.py::test_values[vec_generalized_advantage_estimate-True-True] 55.67 87.37 +56.95%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 28.53 42.64 +49.46%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] 60.33 88.62 +46.91%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 2,763 3,636 +31.56%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3,604 2,768 -23.21%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3,663 2,833 -22.64%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 731.93 891.16 +21.75%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3,376 2,662 -21.15%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2,239 1,869 -16.51%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 2,307 1,933 -16.24%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 474.02 544.20 +14.80%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2,967 3,378 +13.83%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 492.89 535.52 +8.65%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-same] 2,748 2,512 -8.59%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-None] 189.20 173.70 -8.19%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] 907.72 979.80 +7.94%
benchmarks/test_objectives_benchmarks.py::test_redq_speed[reduce-overhead-None] 213.88 229.36 +7.24%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] 38,532 35,823 -7.03%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-False] 34,674 32,254 -6.98%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 2,636 2,819 +6.96%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] 38,019 35,379 -6.94%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-buffers-True] 0.5197 0.4855 -6.58%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[pickle] 12,371 11,570 -6.48%
benchmarks/test_envs_benchmark.py::test_simple 1.8135 1.6992 -6.30%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-backward] 132.21 140.50 +6.28%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[reduce-overhead-None] 88.43 83.20 -5.91%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-True] 20,317 19,134 -5.82%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[safetensors] 23,872 22,488 -5.80%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-64] 11.99 11.29 -5.79%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] 32,850 31,019 -5.57%
benchmarks/test_collectors_benchmark.py::test_sync 16.68 15.77 -5.42%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[torch.save] 7,203 6,817 -5.35%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-64] 8.7757 9.2273 +5.15%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1,789 1,881 +5.14%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape1-atari] 264.42 277.71 +5.03%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] 27,373 26,175 -4.38%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-False] 57,590 55,079 -4.36%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2,514 2,618 +4.15%
benchmarks/test_objectives_benchmarks.py::test_values[td0_return_estimate-False-False] 7,332 7,633 +4.10%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-backward] 400.29 416.24 +3.99%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-False] 63,316 60,863 -3.87%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 1,044 1,084 +3.85%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-backward] 61.44 59.08 -3.84%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1,047 1,087 +3.82%
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_catframes_offline 540.81 558.53 +3.28%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape2-large_img] 566.84 584.21 +3.06%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[False-backward] 133.44 129.42 -3.02%
benchmarks/test_objectives_benchmarks.py::test_values[td1_return_estimate-False-False] 36.57 35.47 -3.01%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-False] 34,218 33,191 -3.00%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 657.68 677.19 +2.97%
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-backward] 78.21 75.94 -2.90%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-1] 273.63 281.47 +2.87%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-None] 444.82 457.34 +2.81%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 689.56 708.27 +2.71%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-True] 19,966 19,433 -2.67%
benchmarks/test_rnn_reset_backends_benchmark.py::test_rnn_rollout_with_intermediate_resets[b256-t128-i32-h512-cudnn-False-0-gru] 1.3099 1.3447 +2.65%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-None] 1,728 1,683 -2.61%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 179.62 184.22 +2.56%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[False-None] 49.98 48.72 -2.51%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] 689.17 706.14 +2.46%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[reduce-overhead-None] 288.80 281.73 -2.45%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 23.78 23.21 -2.40%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[generalized_advantage_estimate-False-1-512] 109.00 106.42 -2.37%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-backward] 250.23 244.42 -2.32%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-None] 114.71 117.30 +2.26%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-64] 6.1172 5.9806 -2.23%
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[False-None] 162.02 158.42 -2.22%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 53.01 51.84 -2.21%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-buffers-False] 0.5959 0.5828 -2.20%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[False-None] 706.14 690.64 -2.20%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-backward] 279.85 285.99 +2.19%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-False] 42,172 43,073 +2.14%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[untyped_storage] 8.0895 7.9189 -2.11%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 654.55 668.23 +2.09%
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_action_chunk_transform[no_done] 1,550 1,582 +2.07%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-16] 47.59 48.56 +2.05%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-True] 35,215 34,499 -2.03%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] 127.52 124.95 -2.02%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[16-same] 1,779 1,743 -2.02%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-True] 21,535 21,968 +2.01%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-constant] 4,379 4,292 -1.99%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-backward] 84.01 82.34 -1.99%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[False-backward] 518.71 508.46 -1.98%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-True] 20,811 20,417 -1.89%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-True] 41,723 42,499 +1.86%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-4] 162.37 165.28 +1.79%
benchmarks/test_objectives_benchmarks.py::test_redq_speed[True-None] 221.60 225.48 +1.75%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[False-backward] 33.09 32.52 -1.71%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] 637.69 648.45 +1.69%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-False] 0.2218 0.2255 +1.69%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[reduce-overhead-None] 463.72 455.91 -1.68%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[50-img_shape0-small] 7,172 7,293 +1.68%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-True] 30,648 30,138 -1.66%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-None] 701.95 690.33 -1.66%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-None] 347.84 342.23 -1.61%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-True] 28,665 28,208 -1.59%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-parallel-no-buffers-True] 0.2112 0.2145 +1.57%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] 49,914 49,141 -1.55%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-True] 18,361 18,641 +1.52%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-False] 54,165 53,353 -1.50%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-16] 18.10 17.83 -1.48%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[50-img_shape0-small] 4,341 4,405 +1.47%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[False-backward] 27.51 27.91 +1.47%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[200-img_shape3-large_batch] 329.00 333.67 +1.42%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-64] 3.0664 3.0229 -1.42%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape1-atari] 5,131 5,059 -1.41%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-None] 277.17 273.33 -1.39%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-None] 84.71 83.55 -1.37%
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[True-None] 255.24 258.69 +1.35%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-None] 274.83 278.53 +1.35%
benchmarks/test_objectives_benchmarks.py::test_ppo_speed[reduce-overhead-None] 263.50 259.96 -1.34%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[200-img_shape3-large_batch] 308.32 312.43 +1.33%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[reduce-overhead-None] 562.41 555.02 -1.31%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 467.22 461.14 -1.30%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[reduce-overhead-None] 338.07 333.69 -1.29%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-4] 48.92 48.30 -1.27%
... ... ... Showing 120 of 220 comparisons, sorted by absolute change.

GPU

Compared 230 benchmarks. Regressions over 5%: 11. Improvements over 5%: 19.

Benchmark main ops PR ops Change
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 43.90 194.34 +342.71%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 928.75 2,929 +215.33%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 549.42 59.54 -89.16%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[False-None] 80.48 110.89 +37.79%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2,643 3,502 +32.50%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 2,919 3,754 +28.62%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape1-atari] 3,440 4,414 +28.32%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3,808 3,203 -15.88%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1,838 2,070 +12.58%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-backward] 889.31 1,000 +12.46%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3,530 3,115 -11.76%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 2,173 2,394 +10.19%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[True-backward] 374.97 409.30 +9.16%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[True-backward] 263.90 285.72 +8.27%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-4] 134.36 144.99 +7.91%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 511.48 477.34 -6.67%
benchmarks/test_objectives_benchmarks.py::test_values[vec_generalized_advantage_estimate-True-True] 446.12 416.40 -6.66%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[100-img_shape1-atari] 691.14 645.26 -6.64%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 825.36 879.22 +6.53%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2,149 2,289 +6.48%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 962.50 1,024 +6.38%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[200-img_shape3-large_batch] 135.92 144.57 +6.36%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[50-img_shape0-small] 4,465 4,182 -6.34%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 2,014 2,134 +5.95%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 512.61 483.18 -5.74%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[pickle] 11,952 12,603 +5.45%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[True-backward] 326.17 343.52 +5.32%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape1-atari] 727.86 690.32 -5.16%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 845.31 801.83 -5.14%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 790.31 749.81 -5.12%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[safetensors] 22,917 24,062 +5.00%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[True-None] 1,880 1,971 +4.88%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[True-backward] 226.13 236.93 +4.78%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-True] 40,102 38,324 -4.43%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-16] 40.87 42.67 +4.40%
benchmarks/test_envs_benchmark.py::test_simple 1.2634 1.2082 -4.37%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-False] 53,250 51,035 -4.16%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-backward] 147.30 153.20 +4.01%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-False-True] 32,193 30,940 -3.89%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-True] 30,282 29,123 -3.83%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-1] 495.28 514.16 +3.81%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[torch.save] 7,028 7,293 +3.77%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-False-False] 79,798 82,605 +3.52%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-True] 24,372 23,515 -3.52%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-None] 736.21 762.05 +3.51%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-False] 57,543 55,596 -3.38%
benchmarks/test_envs_benchmark.py::test_cat_frames_functional[4-constant] 4,735 4,886 +3.21%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 22.65 23.35 +3.12%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-False] 52,921 51,277 -3.11%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-16] 47.23 48.68 +3.07%
benchmarks/test_objectives_benchmarks.py::test_dqn_speed[reduce-overhead-None] 1,840 1,897 +3.06%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_lazystack[100-img_shape2-large_img] 430.19 417.11 -3.04%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-False] 60,879 59,103 -2.92%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-True-False] 28,941 28,098 -2.92%
benchmarks/test_objectives_benchmarks.py::test_iql_speed[True-backward] 251.33 258.63 +2.90%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[50-img_shape0-small] 6,142 6,319 +2.88%
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_action_chunk_transform[no_done] 1,592 1,638 +2.87%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-False] 30,995 30,106 -2.87%
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-memmap_cpu_storage_cud... 969.00 996.76 +2.86%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-224-224-64] 12.08 12.43 +2.86%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-1] 461.50 473.97 +2.70%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-True-False] 40,391 39,320 -2.65%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 1,012 1,039 +2.65%
benchmarks/test_objectives_benchmarks.py::test_values[td0_return_estimate-False-False] 11,510 11,806 +2.57%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-False-False-False] 67,772 66,034 -2.56%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 1,336 1,370 +2.56%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-True-True-True] 19,728 19,226 -2.55%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-False] 36,666 35,735 -2.54%
benchmarks/test_objectives_benchmarks.py::test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 1,311 1,343 +2.43%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-True-True-True-False] 44,494 43,412 -2.43%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-True-True] 20,559 20,068 -2.39%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-False-True] 34,191 33,375 -2.39%
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-cuda_storage_cpu_sampler] 90.50 88.34 -2.39%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-256-256-64] 10.50 10.75 +2.38%
benchmarks/test_replaybuffer_benchmark.py::test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 195.52 200.10 +2.34%
benchmarks/test_replaybuffer_benchmark.py::TestWindowingTransformsBenchmark::test_catframes_offline 547.94 560.72 +2.33%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3,697 3,782 +2.32%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-None] 788.68 770.52 -2.30%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-False-False-False-False] 46,912 45,840 -2.28%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 53.20 54.40 +2.25%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-224-224-1] 279.81 285.96 +2.20%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[True-backward] 485.65 495.99 +2.13%
benchmarks/test_envs_benchmark.py::test_serial 0.4291 0.4382 +2.12%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-False-False-True] 31,778 31,104 -2.12%
benchmarks/test_replaybuffer_benchmark.py::TestPrioritizedReplayBufferBenchmark::test_sample_mixed_devices[1000000-cuda_storage_cuda_samp... 1,487 1,519 +2.10%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 49.29 48.26 -2.08%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-False-True] 39,760 38,953 -2.03%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[False-True-True-True-False] 36,166 35,439 -2.01%
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_with_rb_cuda[200-img_shape1-large_batch] 6.5745 6.7063 +2.00%
benchmarks/test_replaybuffer_benchmark.py::test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 22.48 22.03 -1.98%
benchmarks/test_objectives_benchmarks.py::test_reinforce_speed[True-backward] 371.80 378.97 +1.93%
benchmarks/test_objectives_benchmarks.py::test_cql_speed[reduce-overhead-None] 89.67 87.96 -1.90%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-16] 35.57 36.24 +1.87%
benchmarks/test_objectives_benchmarks.py::test_td3_speed[False-backward] 79.29 80.76 +1.85%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 171.38 174.54 +1.85%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_storage_write_contiguous[100-img_shape2-large_img] 552.69 542.71 -1.81%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3,425 3,485 +1.75%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-True-False] 33,276 32,697 -1.74%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 172.76 175.76 +1.74%
benchmarks/test_non_tensor_env_benchmark.py::test_non_tensor_env_rollout_speed[1000-single-True] 1.3567 1.3337 -1.70%
benchmarks/test_compressed_storage_benchmark.py::TestCompressedStorageBenchmark::test_tensor_to_bytestream_speed[numpy] 367,205 361,047 -1.68%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[False-backward] 236.59 240.51 +1.66%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[True-backward] 366.86 372.93 +1.65%
benchmarks/test_objectives_benchmarks.py::test_ddpg_speed[reduce-overhead-None] 822.25 835.79 +1.65%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-True-True-True] 21,624 21,269 -1.64%
benchmarks/test_envs_benchmark.py::test_step_mdp_speed[True-False-False-False-True] 34,908 35,480 +1.64%
benchmarks/test_collectors_benchmark.py::test_single_pixels 6.2902 6.3918 +1.62%
benchmarks/test_objectives_benchmarks.py::test_redq_deprec_speed[False-None] 96.66 98.19 +1.58%
benchmarks/test_objectives_benchmarks.py::test_a2c_speed[False-None] 281.44 277.03 -1.57%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-16] 11.79 11.98 +1.55%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 172.16 174.77 +1.52%
benchmarks/test_storage_write_benchmark.py::TestCollectorIntegrationBenchmark::test_collector_without_rb_cuda[100-img_shape0-atari] 17.14 17.40 +1.52%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[torchvision-480-640-64] 7.0045 7.1102 +1.51%
benchmarks/test_replaybuffer_benchmark.py::test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 167.58 170.09 +1.50%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_lazystack_then_write[50-img_shape0-small] 3,504 3,555 +1.46%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-480-640-4] 19.62 19.34 -1.45%
benchmarks/test_replaybuffer_benchmark.py::test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 167.35 169.74 +1.43%
benchmarks/test_storage_write_benchmark.py::TestStorageWriteBenchmark::test_collector_stack_then_write[100-img_shape2-large_img] 177.33 174.82 -1.42%
benchmarks/test_vla_preprocessing_benchmark.py::test_openvla_preprocessing_throughput[pil-256-256-4] 47.32 47.99 +1.40%
benchmarks/test_objectives_benchmarks.py::test_sac_speed[reduce-overhead-None] 101.61 100.21 -1.37%
... ... ... Showing 120 of 230 comparisons, sorted by absolute change.

@vmoens @claude
[CI] Gate subprocess coverage tracing to xdist runs
82f82a8 
Exporting COVERAGE_PROCESS_START unconditionally in
coverage_run_parallel.py made every Python interpreter spawned under the
test run start a coverage tracer via coverage's a1_coverage.pth site
hook - parallel-env workers, collector workers, Ray workers - not just
the pytest-xdist execnet workers it was added for. Starting a tracer
roughly doubles worker startup (measured locally: torch+torchrl import
0.8s -> 1.5s) and line-traces everything the worker executes, since the
generated config has no source filter.

The serial optdeps job, which spawns thousands of such workers, went
from ~88min of pytest to past its 2h CI timeout on every run since the
export landed (three consecutive timeouts across differing dependency
versions; per-file timings show a diffuse 30-80% slowdown concentrated
in process-heavy files, test_collectors.py alone +15.6min).

Make the export opt-in via TORCHRL_COV_TRACE_SUBPROCESSES=1, set only by
the xdist invocations in linux/scripts/run_all.sh, and stripped from the
serial quarantine remainder that runs in the same shell. Serial suites
(optdeps, shard 2, non-xdist paths) return to pre-export behavior;
xdist workers keep contributing coverage.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CI Has to do with CI setup (e.g. wheels & builds, tests...) CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vmoens