blas-microbenchmark

How fast is each BLAS routine on this machine, with this library?
One small benchmark per routine, every result checked before it is timed,
and the libraries compared on one page of charts.

• 104 benchmarks, one executable per BLAS routine (bmb_dgemm, bmb_ddot, …), levels 1 to 3, in single, double and complex precision. Think osu-micro-benchmarks, for BLAS instead of MPI.
• Numbers you can trust: every point is checked against a reference before it is timed, so a library that computes the wrong thing stops the run instead of looking fast. The CPU, its frequency, the library, its version and the kernels it chose, and how the run was made, are recorded with every result.
• Eight BLAS libraries: OpenBLAS, BLIS, AMD AOCL-BLAS, Intel MKL, NVPL, ArmPL, Netlib, or all those FlexiBLAS can load, from one build.
• One page of charts, bmb_report: performance against size with the caches marked, libraries side by side, thread scaling, how far repeated runs spread, and, given the machine's peak, a roofline for levels 1 and 2 and the peak on level 3's charts. A single HTML file, which opens offline.
• Made for HPC sites: a Spack package, a Docker image (Apptainer too), and ReFrame tests.

Quick start

With a C compiler and a BLAS library (libopenblas-dev on Debian and Ubuntu), take blas-microbenchmark-<version>.tar.gz from the latest release, then:

tar xf blas-microbenchmark-<version>.tar.gz && cd blas-microbenchmark-<version>
./configure && make && make check && sudo make install
export PATH="/usr/local/libexec/blas-microbenchmark/level3:$PATH"
bmb_dgemm -m 512:2048

$ bmb_dgemm -m 512:2048
# blas-microbenchmark 1.9.0
# backend: openblas (OpenBLAS 0.3.26 NO_LAPACKE DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=64)
# kernels: Haswell
# verified: every point, against a reference, before it was timed
# layout: row-major
# cpu: Intel(R) Core(TM) Ultra 7 155U (14 logical CPUs, 1 NUMA node)
# caches: L1d 48K, L1i 64K, L2 2M, L3 12M
# os: Linux 6.6.87.1-microsoft-standard-WSL2 x86_64
# date: 2026-10-29T09:17:41Z
# affinity: CPUs 0-13
# command: bmb_dgemm -m 512:2048
# run: 0.2 s and 1 call of warm-up, 10 samples, batch size chosen per point, hot cache
# operands: aligned to 64 bytes, leading dimensions + 0
# arguments: trans NN, batch_count 1, batch_mode loop
# compiler: gcc 13.3.0, CFLAGS -g -O2 -fopenmp
# routine: dgemm
Thread count    Matrix dim1 (M=K)   Matrix dim2 (N)     time [s]        GFLOP/s       median [s]      IQR [%]
1               512                 512                 0.003908581     68.678        0.004154811     9.1
1               1024                1024                0.033772171     63.587        0.034789170     4.4
1               2048                2048                0.272986125     62.933        0.276045337     0.7

Every benchmark takes the same options (a routine refuses an argument it does not have, such as --trans on ddot): the sizes to sweep (-v for vectors, -m and -M for matrices), thread counts (-t 1:8), a file to save to (-o results.json, or .csv). GEMM also takes a batch of small matrices per call (--batch-count 10000 --batch-mode group), to compare a loop of calls with the library's batched one. bmb_dgemm --help lists them; man blas-microbenchmark explains everything else.

Compare libraries: bmb_report

Run the same commands against each library, then make one page of them:

bmb_ddot  -v 1024:16777216           -o results/ddot-openblas.json
bmb_dgemm -m 128:2048 -t 1:8         -o results/dgemm-openblas.json
bmb_dgemv -m 256:4096 -M 256:4096    -o results/dgemv-openblas.json
# ... the same three from a build against BLIS, as *-blis.json
# a core's peak in double precision and the memory bandwidth, as you know them:
bmb_report --peak-gflops 76.8 --peak-gbs 120 results/*.json > report.html
firefox report.html

The commands above, run on a laptop (Intel Core Ultra 7 155U, WSL2) against OpenBLAS 0.3.26 and BLIS 0.9.0, its peak taken from Intel's figures (a P-core at 4.8 GHz, 16 FLOPs a cycle; LPDDR5x-7467 on two channels): what the page draws, not a verdict on either library.

Run each command several times, into separate files, and the page also shows how far the runs spread, and which differences are beyond that noise. man bmb_report has the details.

To run every benchmark, or a selection, at a profile's sizes, into one directory that bmb_report takes whole (man bmb_suite):

bmb_suite --profile quick -o results -- -t 1,8
bmb_report results > report.html

With --blas <name>=<prefix> once a library, each built and installed on its own (with Spack, spack location -i blas-microbenchmark ^blis), it runs them all in one command, a series each in the report.

The benchmarks

Level	Double	Single	Complex
1 — vectors	dasum daxpy dcopy ddot dnrm2 dscal dswap idamax drot	sasum saxpy scopy sdot snrm2 sscal sswap isamax srot	scasum dzasum caxpy zaxpy ccopy zcopy cdotc zdotc cdotu zdotu scnrm2 dznrm2 cscal zscal cswap zswap icamax izamax
2 — matrix × vector	dgemv dgbmv dger dsymv dspmv dsyr dsyr2 dtrmv dtrsv dtpsv dtbsv	sgemv sgbmv sger ssymv sspmv ssyr ssyr2 strmv strsv stpsv stbsv	cgemv zgemv cgeru zgeru cgerc zgerc chemv zhemv cher zher cher2 zher2 ctrmv ztrmv ctrsv ztrsv
3 — matrix × matrix	dgemm dsymm dsyrk dsyr2k dtrmm dtrsm	sgemm ssymm ssyrk ssyr2k strmm strsm	cgemm zgemm csymm zsymm chemm zhemm csyrk zsyrk cherk zherk csyr2k zsyr2k cher2k zher2k ctrmm ztrmm ctrsm ztrsm

Each is bmb_<routine>. Level 1 and 2 report GFLOP/s and GB/s (copy, swap and i?amax GB/s only), level 3 GFLOP/s.

Install

From a release

The Quick start above. make install puts bmb_suite and bmb_report in /usr/local/bin, the manual pages in /usr/local/share/man, and the benchmarks by BLAS level under /usr/local/libexec/blas-microbenchmark, not on the PATH; to run them all by name:

BMB=/usr/local/libexec/blas-microbenchmark
export PATH="$BMB/level1:$BMB/level2:$BMB/level3:$PATH"

./configure --with-blas-backend=… picks the library (Backends). On a cluster, it finds one loaded with module load (OPENBLAS_ROOT, BLIS_INCDIR, …), or point at it with --with-blas-incpath=DIR and --with-blas-libpath=DIR.

Check that the download is untouched

Each release tarball is attested by the workflow that built it. With the GitHub CLI:

gh attestation verify blas-microbenchmark-<version>.tar.gz --repo MathieuCAISSA/blas-microbenchmark

Or, from 1.3.0, with cosign and the bundle attached to the release (signed by the release workflow, run for the version's tag, or from main when a release's upload is run again):

cosign verify-blob-attestation --type slsaprovenance1 \
  --bundle blas-microbenchmark-<version>.tar.gz.sigstore.json \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  --certificate-identity-regexp '^https://github\.com/MathieuCAISSA/blas-microbenchmark/\.github/workflows/release\.yml@refs/(tags/v|heads/main$)' \
  blas-microbenchmark-<version>.tar.gz

With Spack

spack repo add https://github.com/MathieuCAISSA/blas-microbenchmark.git
spack install blas-microbenchmark ^openblas
spack load blas-microbenchmark

The BLAS can be ^openblas, ^blis, ^amdblis, ^intel-oneapi-mkl, ^nvpl-blas, ^armpl-gcc, ^netlib-lapack or ^flexiblas.

In a container

From 1.6.0, each release's image is on ghcr.io, for x86_64, against OpenBLAS (latest, <version>, <version>-openblas), BLIS (<version>-blis) or Netlib (<version>-netlib). It writes to /results, here the current directory:

IMAGE=ghcr.io/mathieucaissa/blas-microbenchmark:latest
gh attestation verify oci://$IMAGE --repo MathieuCAISSA/blas-microbenchmark
docker run --rm --user "$(id -u):$(id -g)" -v "$PWD:/results" $IMAGE bmb_dgemm -m 64:1024 -o dgemm.json
apptainer exec docker://$IMAGE bmb_dgemm -m 64:1024 -o dgemm.json

Build the image yourself, with MKL, or for Apptainer

From a git clone, against OpenBLAS or the library BLAS_BACKEND names (blis, netlib, or mkl, which is not published since it would redistribute Intel's libraries):

docker build -f packaging/docker/Dockerfile --build-arg BLAS_BACKEND=openblas -t blas-microbenchmark .
docker run --rm --user "$(id -u):$(id -g)" -v "$PWD:/results" blas-microbenchmark bmb_dgemm -m 64:1024 -o dgemm.json

Apptainer takes a built image too: save it where Docker is, then convert it where Apptainer is, which writes in the current directory as you:

docker save -o blas-microbenchmark.tar blas-microbenchmark
apptainer build blas-microbenchmark.sif docker-archive://blas-microbenchmark.tar
apptainer exec blas-microbenchmark.sif bmb_dgemm -m 64:1024 -o dgemm.json

A container runs on the host's CPU, which is what the results record. Docker's --cpus and --cpuset-cpus leave the benchmark fewer CPUs than it sees, and -t beyond those gets a warning. Docker Desktop runs containers in a virtual machine, whose own limits the numbers include.

From git

Run ./autogen.sh first (it needs autoconf, automake and libtool), then as from a release. GitHub's "Source code" archives are git exports too.

Backends

Chosen when building, and each one built and tested in CI:

Library	configure flag	Spack
OpenBLAS (default)	(none)	^openblas
BLIS	--with-blas-backend=blis	^blis
AMD AOCL-BLAS	--with-blas-backend=aocl	^amdblis
Intel MKL	--with-blas-backend=mkl	^intel-oneapi-mkl
NVPL (aarch64)	--with-blas-backend=nvpl	^nvpl-blas
ArmPL (aarch64)	--with-blas-backend=armpl	^armpl-gcc
Netlib reference	--with-blas-backend=netlib	^netlib-lapack
FlexiBLAS	--with-blas-backend=flexiblas	^flexiblas

With FlexiBLAS, one build measures every library it can load (FLEXIBLAS=BLIS-OPENMP bmb_dgemm …), each its own series in the report. Fedora packages it; Debian and Ubuntu do not.

In a test suite: ReFrame

Sites that check their systems with ReFrame get every benchmark as a ReFrame test, installed with them: its result checked for sanity, its GFLOP/s and GB/s as performance variables, compared with the site's references.

R=/usr/local/share/blas-microbenchmark/reframe   # with Spack: $(spack location -i blas-microbenchmark)/share/...
reframe -c $R/blas_microbenchmark.py -n bmb_run -r
reframe -c $R/blas_microbenchmark.py -n bmb_run -S level3_size=4096 -S reference_file=refs.json -r
bmb_report output/*/*/*/*/result.json > report.html

Sizes, threads, references, and building a release in each programming environment: packaging/reframe.

Documentation

• The manual, online or as man blas-microbenchmark, man bmb_suite and man bmb_report, with what a run produces (table, CSV, JSON, the report). Before installing: man -l man/blas-microbenchmark.1 from the build directory.
• The wiki: results measured on real machines, and a FAQ.
• CHANGELOG.md, and the roadmap.
• Contributing: CONTRIBUTING.md, then AGENTS.md and doc/dev/; every test and CI job is on the development page.

Citing and license

If you use it in published work, please cite it: GitHub's Cite this repository button gives the reference, from CITATION.cff. Released under the Apache License 2.0.