Add a BM=16 qmm tile for the small-M decode dead-zone

[katlun-lgtm]

Summary

Affine quantized_matmul at small M (past the qmv batch-limit, up to 32) always pays for a full 32-row tile: the qmm path uses a single BM=32 tile and launches ceil(M/32) M-tiles, so M=10 and M=32 take the same absolute time. This shows up as a flat plateau — see #3852 (@ARahim3): qmm is flat from M=10 to M=32, then steps at 33 / 65 / 97 (one step per 32-row tile).

This adds a BM=16 tile and selects it for M <= 16 in affine mode, recovering the [qmv-limit, 16] band that was rounding up to 32.

Mechanism

qmm() launches grid = (ceil(N/32), ceil(M/32), B), so for M∈[1,32] that's one M-tile and the kernel computes 32 rows regardless of the real M. A fine M-sweep (M3 Max, 5120→17408, 4-bit) confirms the tile quantization is the whole story: flat ~0.51 ms for M=10→32, then +95% at M=33, +49% at M=65 — i.e. time = ceil(M/32) × per-tile cost. At M=32 the op runs at ~92 GB/s, far under the ~400 GB/s roofline, so it's tile/compute-bound with real headroom (not bandwidth-bound) — halving the tile actually helps rather than just re-reading weights.

Change

• qmm() (host): bm = (mode == "affine" && M <= 16) ? 16 : 32, and thread BM into the kernel name + template instantiation. The bm == 32 path is byte-identical to before, so all existing kernels resolve unchanged.
• quantized.metal: two _bm-suffixed instantiation macros, plus affine_qmm_t / affine_qmm_n instantiated at BM=16 (BK=BN=32).

The BM=16 tile reuses the existing qmm_t_impl / qmm_n_impl (already templated on BM), so there is no new kernel logic. BM=8 is intentionally not added: BM/wm = 8/2 = 4 rows per simdgroup breaks the 8-row MMA fragment tiling (TM = BM/wm/8), so BM=16 (TM=1) is the smallest valid tile.

Results (M3 Max, min of 15×30, baseline = BM=32 plateau at M=24)

shape	bits	M∈[10,16] (BM=16)	plateau (BM=32)	speedup
5120→17408	4	~0.31 ms	0.515 ms	1.63–1.68×
5120→17408	2	~0.31 ms	0.511 ms	1.63–1.66×
5120→248320 (lm_head)	4	~4.31 ms	7.16 ms	1.64–1.70×
5120→248320 (lm_head)	2	~4.28 ms	7.14 ms	1.65–1.68×

M≥17 is unchanged; the qmv / qmv_wide paths (M below the batch-limit) are untouched.

Correctness

• Exact vs dequantize-then-matmul (max|Δ| = 0) for 4-bit and 2-bit at M = 10 / 12 / 16 / 17 / 32.
• python/tests/test_quantized.py: 32 passed, 2896 subtests passed.

Scope

• Helps small continuous batches and n > 1 sampling (M in the [limit, 16] band). It deliberately does not touch the qmv_wide slope where single-stream speculative-decode verify (M = 2–6) lives — that path is already well-tuned (also noted in 2-bit quantized_matmul loses its size advantage over 4-bit at M >= 3 #3852).
• Affine only for now; the fp modes have the identical plateau and the change mirrors mechanically. Happy to fold that in here or as a follow-up.

Refs #3852.

[angeloskath]

Can you please provide the benchmark you ran? I think part of this should be handled by qmv_wide so not sure where the benefits lie.