ggml-cuda: one-time K-padded f16 dequantization for prefill GEMMs

[roberteg16]

Summary

This PR adds an opt-in path that, for large-batch (prefill) quantized dense matmuls, dequantizes the weight to f16 once and runs the GEMM through hipBLAS instead of MMQ. The weight is dequantized a single time into a persistent, K-padded f16 buffer that is cached and reused across calls, so subsequent prefill steps pay only the activation conversion and the GEMM itself. Decode (small batch) is left entirely on the existing MMQ/MMVQ path. The feature is off by default and enabled with the GGML_PREFILL_DEQUANT environment variable.

Motivation

On RDNA3.5 (gfx1151) the dense q8_0 attention projections in prefill run through MMQ (int8 WMMA) at roughly 61% of the f16 WMMA roofline. Roofline profiling and standalone hipBLAS microbenchmarks showed that the same GEMM, run as f16 through hipBLAS with a cache-line-padded K dimension, reaches roughly 80% of that roofline and is faster at the kernel level for batch sizes at and above ~512 tokens. The catch is that a naive dequantize-on-every-call approach loses this advantage because the per-call weight dequantization (q8_0 to f16) costs more than the GEMM saves. This PR removes that tax by dequantizing each weight exactly once and caching the padded f16 result.

How it works

Two changes, both confined to ggml/src/ggml-cuda/ggml-cuda.cu. First, in ggml_cuda_mul_mat, when GGML_PREFILL_DEQUANT=<min_tokens> is set and the matmul is a quantized weight with src1->ne[1] >= min_tokens (default 32), MMQ and MMVQ are disabled for that op so dispatch falls through to ggml_cuda_op_mul_mat_cublas. Second, in ggml_cuda_op_mul_mat_cublas, the f16 branch dequantizes a quantized src0 once into a persistent buffer whose row stride is padded by one cache line (GGML_CUDA_CACHE_LINE), which avoids the cache-set aliasing that otherwise halves hipBLAS throughput when the packed row size is a multiple of GGML_CUDA_ROW_ALIAS_STRIDE. The padded buffer is stored in a static map keyed by the weight device pointer; on a cache hit the stored pointer and padded leading dimension are used directly and no dequantization is performed. The GEMM leading dimension (lda) is switched from the packed ne00 to the padded stride for both the COMPUTE_32F and COMPUTE_16F code paths.

Performance

Measured on gfx1151 (Strix Halo, Qwen3.6-35B-A3B UD-Q4_K_M) with the roofline profiler and llama-bench. At the kernel level the padded f16 hipBLAS GEMM for the largest dense projection (q8_0 [2048, 8192], M=512) reaches about 34.7 TFLOPS (80.7% of the 43 TFLOPS f16 WMMA roofline) versus MMQ at about 26 TFLOPS (61%). The pure-prefill throughput improvement is roughly 3-5% at pp512 and above when combined with ROCBLAS_USE_HIPBLASLT=1. End-to-end (combined prefill plus decode) the gain is small and workload-dependent: decode is unaffected and dilutes the prefill improvement, batch sizes at or below 128 tokens are below the crossover where MMQ is still faster, and the dense projections are only a fraction of prefill time since the MoE expert GEMMs remain on MMQ. On this thermally noisy APU the e2e delta at pp128+tg128 and pp4096+tg128 is within run-to-run variance. The feature is therefore most useful for prefill-heavy, low-generation workloads (long-context prompt processing, summarization, batch scoring).

Usage

Enable at runtime with GGML_PREFILL_DEQUANT=32 (the value is the minimum token count that triggers the path; any value greater than 1 is used verbatim, otherwise it defaults to 32). Pair it with ROCBLAS_USE_HIPBLASLT=1 so the underlying cublasGemmEx call is served by hipBLASLt, whose default heuristic picks a good f32-accumulate WMMA kernel for this case; without it rocBLAS may select a slower solution. When the variable is unset the code path is identical to before.

Correctness

The padded f16 layout is bit-identical to the unpadded f16 layout (the padding only changes the row stride, not the data), and the f16 GEMM result matches MMQ within quantization error (about 1e-3 relative). Greedy generation output was verified to match the baseline for the cases tested.