Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding
DSpine is a parallel drafter for speculative decoding. Each draft layer passes the predicted feature of each position to the next, allowing token dependencies to be modeled during parallel drafting.
- [2026-11] Added support for Qwen3.8-27B inference.
- [2026-11] 🍎 Added support for Apple Silicon inference via mlx-lm and SGLang-MLX.
- [2026-10] 🚀 Training code released in SpecForge.
- [2026-10] 🤗 Qwen3-8B draft weights released.
- [2026-10] 💻 Inference code released.
- [2026-10] 📄 Paper released on arXiv.
| Target model | Draft model |
|---|---|
Qwen3-8B |
tencent/Qwen3-8B-DSpine-b16 |
Qwen3.8-27B |
tencent/Qwen3.8-27B-DSpine-b16 |
Acceptance length on Qwen3-8B. DSpine, DSpark and DFlash2 are trained on the same Open-PerfectBlend data with the same training budget.
| Temp. | Method | GSM8K | MATH-500 | HumanEval | MBPP | LCB | MT-Bench | Arena-Hard | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| T = 0 | DFlash† | 6.32 | 7.99 | 6.56 | 6.29 | 6.11 | 4.26 | 3.91 | 5.92 |
| Domino† | 9.68 | 9.40 | 7.23 | 6.89 | 6.40 | 5.12 | 4.65 | 7.05 | |
| DSpark | 10.00 | 9.88 | 7.78 | 7.35 | 6.81 | 5.39 | 4.87 | 7.44 | |
| DFlash2 | 10.06 | 10.04 | 7.99 | 7.33 | 6.96 | 5.40 | 5.06 | 7.55 | |
| DSpine | 10.55 | 10.30 | 8.27 | 7.66 | 7.25 | 5.63 | 5.13 | 7.83 | |
| T = 1 | DFlash† | 5.75 | 6.51 | 5.52 | 5.23 | 4.53 | 3.77 | 3.25 | 4.94 |
| Domino† | 8.01 | 7.13 | 5.89 | 5.56 | 4.59 | 4.36 | 3.61 | 5.59 | |
| DSpark | 9.33 | 8.64 | 6.91 | 6.66 | 5.78 | 4.99 | 4.25 | 6.65 | |
| DFlash2 | 9.07 | 8.42 | 6.99 | 6.43 | 5.67 | 4.90 | 4.25 | 6.53 | |
| DSpine | 9.83 | 8.99 | 7.39 | 6.97 | 6.11 | 5.09 | 4.49 | 6.98 |
† Official checkpoints: z-lab/Qwen3-8B-DFlash-b16 and Huang2020/Qwen3-8B-Domino-b16.
We evaluate Qwen3.8-27B on WorkBuddy Bench using trajectories collected in the CodeBuddy sandbox. See the model card for acceptance length and throughput results.
The SGLang plugin uses SGLang 0.5.22 and requires Python 3.11+ and CUDA.
git clone https://github.com/tencent/dspine.git
cd dspine
python -m pip install -e .sglang serve \
--model-path Qwen/Qwen3-8B \
--speculative-algorithm DSPINE \
--speculative-draft-model-path tencent/Qwen3-8B-DSpine-b16 \
--speculative-num-draft-tokens 16 \
--tp-size 1 \
--dtype bfloat16 \
--attention-backend fa3 \
--page-size 1 \
--mem-fraction-static 0.65 \
--context-length 8192 \
--max-running-requests 64 \
--cuda-graph-bs-decode 1 2 4 8 16 32 64 \
--trust-remote-codeThe implementation uses DSpark-based dynamic verification and ReplaySSM.
Run SPS tuning with your serving settings to tune dynamic verification.
Add --speculative-dspark-sps-table-path sps.json to the command below to load the result.
sglang serve \
--model-path /path/to/Qwen3.8-27B \
--speculative-algorithm DSPINE \
--speculative-draft-model-path tencent/Qwen3.8-27B-DSpine-b16 \
--speculative-num-draft-tokens 16 \
--tp-size 1 \
--dtype bfloat16 \
--attention-backend fa3 \
--speculative-draft-attention-backend fa3 \
--linear-attn-backend triton \
--enable-linear-replayssm-spec \
--mamba-ssm-dtype float32 \
--mamba-radix-cache-strategy extra_buffer \
--max-mamba-cache-size 256 \
--mem-fraction-static 0.80 \
--context-length 131072 \
--max-running-requests 32 \
--cuda-graph-max-bs-decode 32 \
--min-free-slots-delay 1 \
--sampling-backend flashinfer \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder \
--trust-remote-codeUse --tp-size N for tensor parallelism and
--speculative-draft-attention-backend triton for Triton draft attention.
Install with pip install -e dspine_mlx. See the MLX guide
for mlx-lm and SGLang-MLX usage.
See the SpecForge training guide and DSpine online training example.
- Training code for DSpine
- Qwen3.8-27B inference
- Qwen3.8-27B draft weight release
We thank DFlash, SpecForge, SGLang and FlashInfer for their open-source work.
@article{zhang2026dspine,
title = {Draft in Parallel, Condition Through Depth: Adjacent Causal Injection for Speculative Decoding},
author = {Zhang, Haohui and Chen, Keyu and Sun, Haocheng and Gu, Weibo and Qiao, Ruizhi and Sun, Xing and Jiang, Bo},
journal = {arXiv preprint arXiv:2609.36173},
year = {2026}
}