Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
<!-- Modified 2026 by AMD to document process-per-GPU execution. -->
<!-- Modified 2026 by AMD to link the multi-GPU tutorial. -->

<p align="center">
<picture>
Expand Down Expand Up @@ -141,6 +142,10 @@ uv sync --extra tutorials # or: pip install -e ".[tutorials]"
uv run jupyter lab tutorials/
```

Continue with [Tutorial 2: multi-GPU quickstart](tutorials/02_multi_gpu_quickstart.ipynb)
to decode on up to eight GPUs using the process backend, check decoding
quality, and compare GPU counts with 1,024 shots per GPU.

## Benchmarking

`pelagos_benchmark.py` is a full-featured benchmark CLI. Run
Expand Down
324 changes: 324 additions & 0 deletions tutorials/02_multi_gpu_quickstart.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,324 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "multi-gpu-00",
"metadata": {},
"source": [
"<small>Copyright (c) 2026 Advanced Micro Devices, Inc. Licensed under the Apache License, Version 2.0; see the `LICENSE` file in the repository root.</small>\n",
"\n",
"# Tutorial 2 — Decoding a qLDPC code on multiple GPUs\n",
"\n",
"This notebook extends the [single-GPU quickstart](01_single_gpu_quickstart.ipynb)\n",
"to multiple GPUs:\n",
"\n",
"1. Build a decoding problem for the [[72,12,6]] bivariate bicycle code.\n",
"2. Decode a batch using one worker process per GPU.\n",
"3. Check the syndrome solutions and logical error rate.\n",
"4. Compare throughput with 1, 2, 4, and 8 GPUs, keeping 1,024 shots per GPU.\n",
"\n",
"Pelagos's process backend keeps one worker process per GPU and splits\n",
"each batch across them. Each worker runs the existing Triton decoder. Reusing the\n",
"decoder keeps those workers alive between calls.\n",
"\n",
"**Requirements:** at least two AMD GPUs with ROCm on the same machine, and a clone of\n",
"this repository with the tutorial dependencies. Use a ROCm PyTorch/Triton environment,\n",
"then run from the clone:\n",
"\n",
"```bash\n",
"pip install -e \".[tutorials]\"\n",
"jupyter lab tutorials/\n",
"```\n",
"\n",
"Run cells in order. First decodes include compilation and autotuning and may take\n",
"longer.\n"
]
},
{
"cell_type": "markdown",
"id": "multi-gpu-01",
"metadata": {},
"source": [
"## 0. Setup\n",
"\n",
"GPU IDs below refer to the devices visible to this notebook. Set any GPU visibility\n",
"variables before starting Jupyter. The default selects up to eight visible GPUs.\n",
"With eight GPUs, the scaling section runs all four counts: 1, 2, 4, and 8.\n",
"With fewer GPUs, it runs the available counts.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "multi-gpu-02",
"metadata": {},
"outputs": [],
"source": [
"import gzip\n",
"import time\n",
"from pathlib import Path\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import numpy as np\n",
"import stim\n",
"import torch\n",
"\n",
"from pelagos.process import ProcessRelayBPDecoder\n",
"from pelagos.stim.sinter.check_matrices import CheckMatrices\n",
"\n",
"assert torch.cuda.is_available(), \"Pelagos needs a GPU (CUDA or ROCm); there is no CPU fallback.\"\n",
"DEVICES = list(range(min(8, torch.cuda.device_count())))\n",
"assert len(DEVICES) >= 2 and len(set(DEVICES)) == len(DEVICES)\n",
"assert all(0 <= d < torch.cuda.device_count() for d in DEVICES), \"Allocate at least two GPUs.\"\n",
"\n",
"def find_repo_root():\n",
" for p in [Path.cwd(), *Path.cwd().parents]:\n",
" if (p / \"tests/testdata/bicycle_bivariate\").is_dir():\n",
" return p\n",
" raise FileNotFoundError(\"Run this notebook from inside the pelagos clone.\")\n",
"\n",
"REPO = find_repo_root()\n"
]
},
{
"cell_type": "markdown",
"id": "multi-gpu-03",
"metadata": {},
"source": [
"## 1. Load a circuit and sample shots\n",
"\n",
"As in Tutorial 1, we load a bundled stim circuit at physical error rate 0.003,\n",
"then construct its check matrix $H$, observable matrix $O$, and error priors.\n",
"Sampling also gives the true logical observable flips for each shot.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "multi-gpu-04",
"metadata": {},
"outputs": [],
"source": [
"paths = list((REPO / \"tests/testdata/bicycle_bivariate\").glob(\n",
" \"*bicycle_bivariate_72_12_6_memory_X,*error_rate=0.003,*.stim.gz\"\n",
"))\n",
"assert len(paths) == 1, f\"Expected one circuit, found {len(paths)}.\"\n",
"circuit = stim.Circuit(gzip.decompress(paths[0].read_bytes()).decode())\n",
"dem = circuit.detector_error_model(decompose_errors=False)\n",
"cm = CheckMatrices.from_dem(dem)\n",
"H = cm.check_matrix.tocsr()\n",
"O = cm.observables_matrix.tocsr()\n",
"print(\"Circuit:\", paths[0].name)\n",
"print(\"H:\", H.shape, \"| nonzeros:\", H.nnz, \"| logical observables:\", O.shape[0])\n",
"\n",
"PER_GPU_BATCH = 1024\n",
"NUM_SHOTS = PER_GPU_BATCH * len(DEVICES)\n",
"detectors, observables, _ = dem.compile_sampler(seed=2026).sample(NUM_SHOTS)\n",
"syndromes = np.ascontiguousarray(detectors, dtype=np.uint8)\n",
"\n",
"CONFIG = dict(\n",
" H_csr=H, error_priors=cm.error_priors,\n",
" algo=\"relay\", perf=\"throughput\", data_type=\"float32\",\n",
" pre_iter=80, num_sets=100, set_max_iter=60,\n",
" gamma0=0.65, gamma_dist_interval=(-0.24, 0.66), stop_nconv=1,\n",
" seed=1234, gamma_draw=\"broadcast\", scheduling=\"flood-fill\",\n",
")\n",
"print(\"Decoder settings:\", {k: v for k, v in CONFIG.items() if k not in (\"H_csr\", \"error_priors\")})\n"
]
},
{
"cell_type": "markdown",
"id": "multi-gpu-05",
"metadata": {},
"source": [
"## 2. Decode on multiple GPUs\n",
"\n",
"Pass the syndromes as a NumPy array or CPU tensor of shape `[batch, checks]`.\n",
"The workers decode their assigned shots on the GPUs and return the results in\n",
"the original input order.\n",
"\n",
"With 1,024 shots per GPU and eight GPUs, the global batch contains 8,192 shots.\n",
"Reuse the decoder across batches. The `with` block automatically closes its\n",
"worker processes when the block finishes.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "multi-gpu-06",
"metadata": {},
"outputs": [],
"source": [
"with ProcessRelayBPDecoder(**CONFIG, device=DEVICES) as decoder:\n",
" decoder.decode(syndromes) # Warm this batch shape; workers persist.\n",
" result = decoder.decode(syndromes)\n",
"\n",
"# Returned CPU tensors remain valid after the decoder is closed.\n",
"assert result[\"errors\"].device.type == \"cpu\"\n",
"errors = result[\"errors\"].numpy()\n",
"assert errors.shape == (NUM_SHOTS, H.shape[1])\n",
"assert np.all((errors == 0) | (errors == 1))\n",
"print(\"Decoded error patterns:\", errors.shape)\n",
"print(\"Result fields:\", sorted(result))\n"
]
},
{
"cell_type": "markdown",
"id": "multi-gpu-07",
"metadata": {},
"source": [
"The result fields have the same meaning as in Tutorial 1:\n",
"\n",
"| key | shape | meaning |\n",
"|---|---|---|\n",
"| `errors` | `[B, V]` | estimated error pattern for each shot |\n",
"| `found_count` | `[B]` | number of valid solutions found; `> 0` means the shot converged |\n",
"| `iterations` | `[B]` | BP iterations spent on each shot |\n",
"| `weights` | `[B]` | negative log-likelihood of the returned solution |\n",
"\n",
"The GPU workers return the decoded results in the original input order.\n",
"\n",
"## 3. Check the result\n",
"\n",
"A syndrome is satisfied when $H\\hat e=s\\pmod 2$. Logical decoding succeeds when\n",
"$O\\hat e$ matches the sampled observable flips. These are different checks: a valid\n",
"syndrome solution can still have a logical error. We report both, plus the fraction\n",
"of shots for which the decoder found a valid solution.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "multi-gpu-08",
"metadata": {},
"outputs": [],
"source": [
"satisfied = np.all(((H @ errors.T) % 2).T == syndromes, axis=1)\n",
"predicted_observables = ((O @ errors.T) % 2).T.astype(bool)\n",
"logical_failures = np.any(predicted_observables != observables, axis=1)\n",
"converged = result[\"found_count\"].numpy() > 0\n",
"failure_count = int(logical_failures.sum())\n",
"print(f\"Syndrome satisfied: {satisfied.mean():.3%}\")\n",
"print(f\"Converged: {converged.mean():.3%}\")\n",
"print(f\"Logical failures: {failure_count}/{NUM_SHOTS}\")\n",
"print(f\"Logical error rate: {failure_count / NUM_SHOTS:.5f} per shot (all 12 logical qubits, 6 rounds)\")\n"
]
},
{
"cell_type": "markdown",
"id": "multi-gpu-09",
"metadata": {},
"source": [
"Decoding succeeds when the predicted observable flips match the true flips.\n",
"\n",
"## 4. Scale from one to eight GPUs\n",
"\n",
"We keep 1,024 shots per GPU and increase the global batch with\n",
"GPU count. For 1, 2, 4, and 8 GPUs, the global batches are 1,024, 2,048, 4,096,\n",
"and 8,192 shots. Every run uses the process backend, including the one-GPU baseline.\n",
"\n",
"Each setting gets a warm-up at its actual batch shape, followed by three measured\n",
"repetitions. Workers are reused. The timer covers the blocking\n",
"`decode()` calls: host dispatch, transfers, GPU decoding, and returned CPU results.\n",
"Sampling, worker startup, warm-up, and quality calculations are outside the timer.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "multi-gpu-10",
"metadata": {},
"outputs": [],
"source": [
"GPU_COUNTS = [n for n in (1, 2, 4, 8) if n <= len(DEVICES)]\n",
"REPEATS = 3\n",
"# Reuse prefixes of the sampled shots as the global batch grows.\n",
"rows = []\n",
"\n",
"for count in GPU_COUNTS:\n",
" batch_size = PER_GPU_BATCH * count\n",
" batch = syndromes[:batch_size]\n",
" with ProcessRelayBPDecoder(**CONFIG, device=DEVICES[:count]) as decoder:\n",
" decoder.decode(batch) # Warm up once at this batch shape.\n",
" seconds = []\n",
" for _ in range(REPEATS):\n",
" started = time.perf_counter()\n",
" output = decoder.decode(batch)\n",
" seconds.append(time.perf_counter() - started)\n",
" rates = batch_size / np.asarray(seconds)\n",
" rows.append(dict(gpus=count, batch=batch_size,\n",
" median=float(np.median(rates)), low=float(rates.min()),\n",
" high=float(rates.max()), seconds=seconds))\n",
"\n",
"baseline = rows[0][\"median\"]\n",
"for row in rows:\n",
" row[\"speedup\"] = row[\"median\"] / baseline\n",
"\n",
"print(f\"{'GPUs':>4} {'shots/GPU':>10} {'global batch':>12} {'shots/s':>10} {'speedup':>8}\")\n",
"for row in rows:\n",
" print(f\"{row['gpus']:>4} {PER_GPU_BATCH:>10} {row['batch']:>12} \"\n",
" f\"{row['median']:>10.0f} {row['speedup']:>7.2f}x\")\n",
" print(\" measured seconds:\", [round(t, 4) for t in row[\"seconds\"]])\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "multi-gpu-11",
"metadata": {},
"outputs": [],
"source": [
"fig, ax = plt.subplots(figsize=(6, 3.5))\n",
"y = np.array([r[\"speedup\"] for r in rows])\n",
"low = np.array([r[\"low\"] / baseline for r in rows])\n",
"high = np.array([r[\"high\"] / baseline for r in rows])\n",
"ax.errorbar(GPU_COUNTS, y, yerr=[y - low, high - y],\n",
" marker=\"o\", capsize=3, label=\"multi-GPU scaling\")\n",
"ax.plot(GPU_COUNTS, GPU_COUNTS, \"k--\", alpha=0.4, label=\"ideal linear scaling\")\n",
"ax.set(xticks=GPU_COUNTS, xlabel=\"GPUs\", ylabel=\"Throughput / one-GPU median\",\n",
" title=\"Scaling: 1,024 shots per GPU\")\n",
"ax.grid(alpha=0.3)\n",
"ax.legend()\n",
"plt.tight_layout()\n",
"plt.show()\n"
]
},
{
"cell_type": "markdown",
"id": "multi-gpu-12",
"metadata": {},
"source": [
"The points show median throughput relative to the corresponding one-GPU median.\n",
"Bars show the observed minimum and maximum rates divided by that same baseline.\n",
"\n",
"Each additional GPU handles another 1,024 shots. The speedup compares the total\n",
"shots decoded per second with the one-GPU run. The dashed line shows proportional\n",
"growth: twice as many GPUs would give twice the aggregate throughput.\n",
"\n",
"For the full multi-GPU design and performance evaluation, see our paper,\n",
"*Scaling Pêlagos: Multi-GPU Orchestration for Relay Belief-Propagation Decoding*.\n",
"\n",
"## Next steps\n",
"\n",
"* To select particular GPUs, edit `DEVICES`, for example `[0, 2, 4, 6]`.\n",
" `GPU_COUNTS` selects how many of those devices each run uses.\n",
"* Lower `PER_GPU_BATCH` if GPU memory is limited. Larger codes also need smaller\n",
" per-GPU batches; keep `H.nnz × per-GPU batch` below the decoder's indexing limit.\n",
"* For the higher-level `RelayDecoder` / `ObservableDecoderRunner` API, see\n",
" [process-backend documentation](../docs/process-backend.md).\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
6 changes: 6 additions & 0 deletions tutorials/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
<!-- Modified 2026 by AMD to add the multi-GPU quickstart. -->

# Tutorials

Hands-on Jupyter notebooks for Pelagos. Run them from a clone of the repository, because they
Expand All @@ -14,5 +16,9 @@ cross-check against the `relay-bp` CPU reference.
| # | Notebook | What you learn | Runtime* |
|---|---|---|---|
| 1 | [Single-GPU quickstart](01_single_gpu_quickstart.ipynb) | Decoder API; from a stim circuit to a logical error rate on the [[72,12,6]] bivariate bicycle code; plain BP vs. Relay-BP; throughput vs. batch size; sweeping the physical error rate; comparing against the Rust reference | ~3–4 min |
| 2 | [Multi-GPU quickstart](02_multi_gpu_quickstart.ipynb) | Process-per-GPU decoding; host inputs and outputs; syndrome and logical-error checks; scaling with 1, 2, 4, and 8 GPUs at 1,024 shots per GPU | Depends on GPU count and hardware |

\* Measured on a laptop RTX 3000 Ada (8 GB). A GPU (CUDA or ROCm) is required.

Tutorial 2 uses up to eight AMD GPUs with ROCm on the same machine, with at
least two needed for the multi-GPU examples. It adapts to the available GPU count.
Loading