Use plus tunings for lookahead scan more widely

[bernhardmgruber]

Fixes: #9811

Performance impact:

## [0] NVIDIA B200

|  T{ct}  |  OffsetT{ct}  |  Elements{io}  |   Ref Time |   Ref Noise |   Cmp Time |   Cmp Noise |          Diff |   %Diff |  Status  |
|---------|---------------|----------------|------------|-------------|------------|-------------|---------------|---------|----------|
|   I8    |      I32      |      2^16      |  25.643 us |       2.33% |  13.315 us |       1.40% |    -12.328 us | -48.07% |  🟢 FAST  |
|   I8    |      I32      |      2^20      |  30.421 us |       3.40% |  15.184 us |       2.36% |    -15.237 us | -50.09% |  🟢 FAST  |
|   I8    |      I32      |      2^24      |  99.602 us |       5.47% |  31.559 us |       2.39% |    -68.044 us | -68.32% |  🟢 FAST  |
|   I8    |      I32      |      2^28      | 998.677 us |       0.26% | 232.261 us |       0.45% |   -766.416 us | -76.74% |  🟢 FAST  |
|   I8    |      I64      |      2^16      |  25.819 us |       2.52% |  13.355 us |       2.34% |    -12.465 us | -48.28% |  🟢 FAST  |
|   I8    |      I64      |      2^20      |  29.621 us |       2.16% |  15.240 us |       2.65% |    -14.380 us | -48.55% |  🟢 FAST  |
|   I8    |      I64      |      2^24      | 101.217 us |       5.93% |  31.566 us |       3.46% |    -69.650 us | -68.81% |  🟢 FAST  |
|   I8    |      I64      |      2^28      | 998.707 us |       0.28% | 233.716 us |       0.52% |   -764.991 us | -76.60% |  🟢 FAST  |
|   I8    |      I64      |      2^32      |  15.629 ms |       0.15% |   3.452 ms |       0.05% | -12176.626 us | -77.91% |  🟢 FAST  |
|   I16   |      I32      |      2^16      |  11.912 us |       8.03% |  11.424 us |       4.84% |     -0.487 us |  -4.09% |  🔵 SAME  |
|   I16   |      I32      |      2^20      |  15.581 us |       4.30% |  13.477 us |       4.33% |     -2.104 us | -13.50% |  🟢 FAST  |
|   I16   |      I32      |      2^24      |  44.886 us |       2.43% |  41.469 us |       3.90% |     -3.417 us |  -7.61% |  🟢 FAST  |
|   I16   |      I32      |      2^28      | 367.877 us |       0.54% | 304.846 us |       0.70% |    -63.031 us | -17.13% |  🟢 FAST  |
|   I16   |      I64      |      2^16      |  11.885 us |       7.84% |  11.510 us |       5.76% |     -0.375 us |  -3.16% |  🔵 SAME  |
|   I16   |      I64      |      2^20      |  15.493 us |       3.62% |  13.619 us |       5.48% |     -1.874 us | -12.10% |  🟢 FAST  |
|   I16   |      I64      |      2^24      |  44.885 us |       2.65% |  42.367 us |       3.80% |     -2.518 us |  -5.61% |  🟢 FAST  |
|   I16   |      I64      |      2^28      | 367.823 us |       0.57% | 318.422 us |       0.61% |    -49.401 us | -13.43% |  🟢 FAST  |
|   I16   |      I64      |      2^32      |   5.546 ms |       0.31% |   4.697 ms |       0.20% |   -849.520 us | -15.32% |  🟢 FAST  |
|   I32   |      I32      |      2^16      |  10.652 us |       8.78% |   9.881 us |       9.63% |     -0.772 us |  -7.24% |  🔵 SAME  |
|   I32   |      I32      |      2^20      |  14.630 us |       7.08% |  13.392 us |       3.20% |     -1.238 us |  -8.46% |  🟢 FAST  |
|   I32   |      I32      |      2^24      |  39.377 us |       4.02% |  37.499 us |       3.46% |     -1.878 us |  -4.77% |  🟢 FAST  |
|   I32   |      I32      |      2^28      | 333.588 us |       0.56% | 320.307 us |       0.36% |    -13.281 us |  -3.98% |  🟢 FAST  |
|   I32   |      I64      |      2^16      |  10.535 us |       9.28% |   9.587 us |       8.15% |     -0.948 us |  -9.00% |  🟢 FAST  |
|   I32   |      I64      |      2^20      |  14.451 us |       7.61% |  13.396 us |       3.27% |     -1.055 us |  -7.30% |  🟢 FAST  |
|   I32   |      I64      |      2^24      |  39.570 us |       3.90% |  37.578 us |       3.46% |     -1.993 us |  -5.04% |  🟢 FAST  |
|   I32   |      I64      |      2^28      | 332.758 us |       0.47% | 320.123 us |       0.34% |    -12.635 us |  -3.80% |  🟢 FAST  |
|   I32   |      I64      |      2^32      |   5.092 ms |       0.42% |   4.904 ms |       0.37% |   -188.202 us |  -3.70% |  🟢 FAST  |
|   I64   |      I32      |      2^16      |  11.306 us |       2.97% |  11.334 us |       3.52% |      0.028 us |   0.25% |  🔵 SAME  |
|   I64   |      I32      |      2^20      |  15.606 us |       4.35% |  15.667 us |       4.81% |      0.061 us |   0.39% |  🔵 SAME  |
|   I64   |      I32      |      2^24      |  62.449 us |       1.41% |  59.446 us |       1.86% |     -3.003 us |  -4.81% |  🟢 FAST  |
|   I64   |      I32      |      2^28      | 756.857 us |       0.21% | 660.375 us |       0.34% |    -96.482 us | -12.75% |  🟢 FAST  |
|   I64   |      I64      |      2^16      |  11.055 us |       5.56% |  11.253 us |       0.53% |      0.198 us |   1.79% |  🔴 SLOW  |
|   I64   |      I64      |      2^20      |  15.779 us |       5.35% |  15.657 us |       4.74% |     -0.122 us |  -0.77% |  🔵 SAME  |
|   I64   |      I64      |      2^24      |  62.485 us |       1.48% |  59.483 us |       1.81% |     -3.002 us |  -4.80% |  🟢 FAST  |
|   I64   |      I64      |      2^28      | 757.833 us |       0.14% | 661.140 us |       0.37% |    -96.693 us | -12.76% |  🟢 FAST  |
|   I64   |      I64      |      2^32      |  11.897 ms |       0.01% |  10.368 ms |       0.27% |  -1529.218 us | -12.85% |  🟢 FAST  |
|  I128   |      I32      |      2^16      |  13.398 us |       4.08% |  13.520 us |       4.92% |      0.122 us |   0.91% |  🔵 SAME  |
|  I128   |      I32      |      2^20      |  25.726 us |       3.03% |  23.773 us |       3.90% |     -1.953 us |  -7.59% |  🟢 FAST  |
|  I128   |      I32      |      2^24      | 182.291 us |       0.76% | 152.001 us |       0.74% |    -30.290 us | -16.62% |  🟢 FAST  |
|  I128   |      I32      |      2^28      |   2.699 ms |       0.08% |   2.108 ms |       0.13% |   -591.266 us | -21.90% |  🟢 FAST  |
|  I128   |      I64      |      2^16      |  13.327 us |       2.68% |  13.371 us |       3.31% |      0.044 us |   0.33% |  🔵 SAME  |
|  I128   |      I64      |      2^20      |  25.606 us |       3.09% |  23.815 us |       3.79% |     -1.791 us |  -6.99% |  🟢 FAST  |
|  I128   |      I64      |      2^24      | 185.067 us |       0.95% | 154.780 us |       0.75% |    -30.288 us | -16.37% |  🟢 FAST  |
|  I128   |      I64      |      2^28      |   2.700 ms |       0.08% |   2.108 ms |       0.13% |   -591.573 us | -21.91% |  🟢 FAST  |
|  I128   |      I64      |      2^32      |  43.018 ms |       0.02% |  33.373 ms |       0.02% |  -9644.961 us | -22.42% |  🟢 FAST  |
|   F32   |      I32      |      2^16      |  11.218 us |       2.92% |  11.265 us |       1.22% |      0.047 us |   0.42% |  🔵 SAME  |
|   F32   |      I32      |      2^20      |  14.592 us |       7.54% |  14.744 us |       6.88% |      0.151 us |   1.04% |  🔵 SAME  |
|   F32   |      I32      |      2^24      |  40.964 us |       3.63% |  39.314 us |       3.58% |     -1.650 us |  -4.03% |  🟢 FAST  |
|   F32   |      I32      |      2^28      | 359.671 us |       0.36% | 322.505 us |       0.41% |    -37.166 us | -10.33% |  🟢 FAST  |
|   F32   |      I64      |      2^16      |  11.066 us |       5.45% |  11.274 us |       2.05% |      0.208 us |   1.88% |  🔵 SAME  |
|   F32   |      I64      |      2^20      |  15.044 us |       6.35% |  15.063 us |       5.51% |      0.019 us |   0.13% |  🔵 SAME  |
|   F32   |      I64      |      2^24      |  41.573 us |       2.97% |  39.501 us |       3.41% |     -2.072 us |  -4.98% |  🟢 FAST  |
|   F32   |      I64      |      2^28      | 358.737 us |       0.43% | 322.929 us |       0.44% |    -35.808 us |  -9.98% |  🟢 FAST  |
|   F32   |      I64      |      2^32      |   5.444 ms |       0.44% |   4.899 ms |       0.62% |   -545.204 us | -10.01% |  🟢 FAST  |
|   F64   |      I32      |      2^16      |  11.287 us |       2.48% |  12.574 us |       7.85% |      1.287 us |  11.40% |  🔴 SLOW  |
|   F64   |      I32      |      2^20      |  15.485 us |       3.36% |  16.529 us |       6.20% |      1.044 us |   6.74% |  🔴 SLOW  |
|   F64   |      I32      |      2^24      |  64.526 us |       1.64% |  60.784 us |       1.52% |     -3.742 us |  -5.80% |  🟢 FAST  |
|   F64   |      I32      |      2^28      | 762.908 us |       0.13% | 678.825 us |       0.12% |    -84.083 us | -11.02% |  🟢 FAST  |
|   F64   |      I64      |      2^16      |  11.262 us |       1.09% |  12.958 us |       5.92% |      1.696 us |  15.06% |  🔴 SLOW  |
|   F64   |      I64      |      2^20      |  15.446 us |       2.84% |  16.554 us |       6.10% |      1.107 us |   7.17% |  🔴 SLOW  |
|   F64   |      I64      |      2^24      |  64.531 us |       1.56% |  60.742 us |       1.56% |     -3.788 us |  -5.87% |  🟢 FAST  |
|   F64   |      I64      |      2^28      | 763.059 us |       0.13% | 679.062 us |       0.13% |    -83.997 us | -11.01% |  🟢 FAST  |
|   F64   |      I64      |      2^32      |  11.948 ms |       0.01% |  10.627 ms |       0.27% |  -1321.076 us | -11.06% |  🟢 FAST  |
|   C32   |      I32      |      2^16      | 118.591 us |       1.85% | 322.200 us |       0.97% |    203.609 us | 171.69% |  🔴 SLOW  |
|   C32   |      I32      |      2^20      | 193.022 us |       3.37% | 395.092 us |       0.50% |    202.070 us | 104.69% |  🔴 SLOW  |
|   C32   |      I32      |      2^24      |   1.296 ms |       6.07% |   1.916 ms |       1.87% |    620.581 us |  47.89% |  🔴 SLOW  |
|   C32   |      I32      |      2^28      |  17.113 ms |       2.90% |  25.728 ms |       0.87% |      8.615 ms |  50.34% |  🔴 SLOW  |
|   C32   |      I64      |      2^16      | 116.076 us |       0.81% | 323.335 us |       0.43% |    207.258 us | 178.55% |  🔴 SLOW  |
|   C32   |      I64      |      2^20      | 168.746 us |       1.42% | 395.044 us |       0.58% |    226.298 us | 134.11% |  🔴 SLOW  |
|   C32   |      I64      |      2^24      |   1.330 ms |       5.27% |   1.909 ms |       3.57% |    579.131 us |  43.54% |  🔴 SLOW  |
|   C32   |      I64      |      2^28      |  17.472 ms |       3.16% |  25.591 ms |       2.01% |      8.119 ms |  46.47% |  🔴 SLOW  |
|   C32   |      I64      |      2^32      | 271.569 ms |       2.19% | 401.973 ms |       0.97% |    130.404 ms |  48.02% |  🔴 SLOW  |

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 00b8f50f-334c-4a98-8cc8-c9c079541d36

📥 Commits

Reviewing files that changed from the base of the PR and between fd41e6f and d8ba202.

📒 Files selected for processing (1)

• cub/cub/device/dispatch/tuning/tuning_scan.cuh

---

📝 Walkthrough

Summary by CodeRabbit

• Performance Improvements
  • Improved scan operation tuning on newer GPU architectures.
  • Broadened the use of optimized lookahead settings across additional operation types, which may improve scan performance and consistency.

Walkthrough

The scan tuning policy for architectures cc >= 10.0 now selects the input_value_size lookahead table for non-plus operations when the accumulator is primitive or trivially copy constructible.

Changes

Scan lookahead tuning

Layer / File(s)	Summary
Broaden lookahead policy selection cub/cub/device/dispatch/tuning/tuning_scan.cuh	The policy guard removes the operation_t == op_kind_t::plus requirement while retaining the accumulator-property requirement.

Assessment against linked issues

Objective	Addressed	Explanation
Fix the scan_exclusive_custom performance regression on Blackwell [#9811]	❓	The guard now covers the affected non-plus operations, but the provided changes do not establish that the reported benchmark regression is resolved.

---

_{Comment @coderabbitai help to get the list of available commands.}

[bernhardmgruber]

I thought a bit about what to do for complex and considered using is_fundamental_type to distinguish between complex and the ints and floats. But we probably also want to include half and bfloat in the tunings for sum, and many projects provide their own struct type for that.

I think ultimately, C32 is just not an important use case, and we may just as well accept the regression.

[pauleonix]

We have received a bug report for complex type (C64) perf regressions before (although on sort, not scan). So it is hard to say how important or not it is.

[github-actions]

🥳 CI Workflow Results

🟩 Finished in 2h 36m: Pass: 100%/287 | Total: 6d 16h | Max: 2h 34m | Hits: 48%/400481

See results here.