Repository navigation

v0.2.20: Fused NN Kernels + Flash Attention 3 SM120 + FP8 Block-Scale MMA - #193

Merged

m96-chan merged 23 commits into

mainfrom

release/v0.2.20

Jan 26, 2026

Merged

v0.2.20: Fused NN Kernels + Flash Attention 3 SM120 + FP8 Block-Scale MMA#193
m96-chan merged 23 commits into
mainfrom
release/v0.2.20

Commits on Jan 6, 2026
- feat(ops): add native Conv1d CUDA kernel (#180)
  
  m96-chan
  and
  claude
  committed
  2411032
Commits on Jan 12, 2026
- feat(examples): add Llama Guard 3 content safety classifier
  
  m96-chan
  and
  claude
  committed
  feb3304
Commits on Jan 16, 2026
- feat(llm): add LLaMA 4 native CUDA kernels
  
  m96-chan
  and
  claude
  committed
  5fcf3c3
- chore: add security benchmark scripts to gitignore
  m96-chan
  committed
  86c821b
- chore: update cutlass submodule (alignment fix)
  m96-chan
  committed
  9eb3a4b
- feat(attention): add Flash Attention 3 for SM120 (Blackwell)
  
  m96-chan
  and
  claude
  committed
  ee6b1fe
- feat(ops): add TMA utilities for SM90+ kernels
  
  m96-chan
  and
  claude
  committed
  1241b74
- wip(fa3): add TMA-enabled Flash Attention 3 kernel
  
  m96-chan
  and
  claude
  committed
  adee44b
- feat(attention): integrate TMA FA3 into SDPA dispatch
  
  m96-chan
  and
  claude
  committed
  dcd1f8e
- fix(fa3): resolve __syncthreads divergence causing kernel hang at scale
  
  m96-chan
  and
  claude
  committed
  5f763de
- refactor(fa3): parallelize softmax and fix consumer warp indexing
  
  m96-chan
  and
  claude
  committed
  028af30
- fix(fa3): resolve non-determinism in TMA FA3 attention kernel
  
  m96-chan
  and
  claude
  committed
  a7c814c
- wip(fa4): add Flash Attention 4 SM120 Phase 1 BF16 baseline
  
  m96-chan
  and
  claude
  committed
  e3214b3
- wip(fa4): Phase 2 NVFP4 Q@K^T external validation
  
  m96-chan
  and
  claude
  committed
  b676995
- bench(fa4): add Phase 3 full NVFP4 pipeline validation
  
  m96-chan
  and
  claude
  committed
  b7f66c3
- docs(fa4): add SM120 implementation report
  
  m96-chan
  and
  claude
  committed
  d22576e
- feat(fa3): add SM120 tuning configs with version selection
  
  m96-chan
  and
  claude
  committed
  7b99c62
Commits on Jan 17, 2026
- docs(fa3): document sync requirements in SM120 kernel
  
  m96-chan
  and
  claude
  committed
  1ac32c8
- feat(fp8): add native PTX inline assembly for FP8 block-scale MMA
  
  m96-chan
  and
  claude
  committed
  dc3658f
- feat(fa3): add FP8 block-scale MMA Flash Attention 3 for SM120
  
  m96-chan
  and
  claude
  committed
  a9b75e2
Commits on Jan 26, 2026
- feat(nn): add fused kernels for RMSNorm+Residual, SwiGLU, GeGLU
  
  m96-chan
  and
  claude
  committed
  ce6de2b
- fix(lint): resolve ruff lint errors in fused kernel files
  
  m96-chan
  and
  claude
  committed
  bea759c
- fix(mypy): add type annotation for scores_max in llama4.py
  
  m96-chan
  and
  claude
  committed
  0820572

Commits

Commits on Jan 6, 2026

Commits on Jan 12, 2026

Commits on Jan 16, 2026

Commits on Jan 17, 2026

Commits on Jan 26, 2026