flash-decoding

Here are 2 public repositories matching this topic...

8b-is / transformers-ultra

Ultra-High-Throughput Transformers Engine: NVIDIA FP8/TMA, Zero-Allocation Slotted Static KV Cache, Sub-5µs CUDA Graphs, Fused Logits Sampling, Tree Speculative Decoding, Split-KV Flash-Decoding & Apple Silicon Metal (MSL) Simdgroup Kernels

transformers pytorch zero-allocation tma apple-silicon fp8 llm-inference speculative-decoding flash-decoding cuda-graphs metal-msl

Updated Aug 17, 2026
Python

JohnScheuer / flash-decoding-bench

Star

Benchmark comparing Flash-Decoding-style chunked KV-parallel attention against standard SDPA at decode time (q_len=1) on RTX 2070, measuring the crossover point where KV-parallelism outperforms fused attention kernels.

benchmarking cuda pytorch transformer decode attention kv-cache long-context llm-inference flash-attention flash-decoding mlsystems

Updated Jul 19, 2026
Python

Improve this page

Add a description, image, and links to the flash-decoding topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the flash-decoding topic, visit your repo's landing page and select "manage topics."

Learn more