Add a narrow tile candidate for small ROCm INT4 matmuls

[mergennachin]

Offer a 16-column tile for ROCm queries with M=1..4 to increase parallelism. Retain every existing candidate so export-time autotuning can choose for the target GPU rather than imposing the MI300X choice. CUDA and other query sizes keep their existing candidate set; kernel arithmetic and the runtime are unchanged.

Validated with 46 focused tests, lintrunner, and full Voxtral W4/BF16 PTE exports executed by the native C++ runner on Linux MI300X. Inference on 30 seconds of audio improved from 12.270 to 7.332 seconds, and on 360 seconds from 156.649 to 97.771 seconds, with identical transcripts and matching token counts. The added candidate also compiled for representative RDNA2/3/4 targets and gfx950; those GPUs were not execution-tested. ROCm 7.1 remains unverified.

Lint was run directly because the pre-commit bootstrap cannot build its untokenize dependency under Python 3.14.

Authored with assistance from OpenAI Codex.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22850

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 Pending, 1 Unclassified Failure

As of commit 0e64460 with merge base 026ca3f ():

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

• Test CUDA Builds / test-model-cuda-e2e-Qwen3.5-35B-A3B-HQQ-INT4-quantized-int4-tile-packed / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command bash /__w/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[Copilot]

🔵 Needs a closer look

ROCm 7.1 and several target GPUs were not execution-tested, warranting final human review.

Pull request overview

Adds a ROCm-specific narrow INT4 matmul tile for small query sizes while preserving existing candidates.

Changes:

• Adds conditional autotune pruning for ROCm M=1–4.
• Adds configuration and small-shape correctness tests.

File summaries

File	Description
backends/cuda/triton/kernels/int4_matmul.py	Adds the narrow tile and ROCm-specific candidate selection.
backends/cuda/tests/test_int4_matmul.py	Tests pruning behavior and small matmul shapes.

Review details

• Files reviewed: 2/2 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟢 Approval recommended

No unresolved review issues remain, and the changes include focused tests and validation.

Review details

• Files reviewed: 3/3 changed files
• Comments generated: 0 new
• Review effort level: Lite

[mergennachin]

@claude Report the result of [test-voxtral-realtime-rocm-gfx1100-rocm7.2] when it finishes. I am especially interested in rtf result in the log (find a line that starts with "Voxtral RTF")

[claude]

Tracking test-voxtral-realtime-rocm-gfx1100-rocm7.2

• Locate the target job in CI
• Wait for the job to finish
• Report the Voxtral RTF line from the log

Status update: the sibling unittest-rocm-gfx950-rocm7.2 and test-voxtral-realtime-rocm-gfx950-rocm7.2 jobs have finished successfully, but the target test-voxtral-realtime-rocm-gfx1100-rocm7.2 / linux-job is still in progress. Continuing to poll and will report the Voxtral RTF result once it completes.

View job run

[digantdesai]

don't land this FYI, we should update this to be on manual trigger only TBH