Skip to content

docs(training-hub): add QLoRA & CPT tutorials for Ascend NPU#279

Draft
typhoonzero wants to merge 1 commit into
masterfrom
docs/npu-cpt-qlora
Draft

docs(training-hub): add QLoRA & CPT tutorials for Ascend NPU#279
typhoonzero wants to merge 1 commit into
masterfrom
docs/npu-cpt-qlora

Conversation

@typhoonzero

Copy link
Copy Markdown
Contributor

Summary

training_hub / bitsandbytes / unsloth all target CUDA and don't run on Huawei Ascend, so the existing QLoRA and CPT tutorials (#270) don't cover NPU users. This PR adds two companion notebooks that reach the same result on Ascend using only mainline transformers + torch_npu (+ the community bitsandbytes-npu-beta fork for QLoRA), and matching e2e cases.

Files added

Docs

  • docs/en/training_guides/qlora-npu-tutorial.ipynb — true 4-bit NF4 QLoRA via BitsAndBytesConfig(load_in_4bit=True, ...) + peft LoRA on torch_npu. Uses bitsandbytes-npu-beta==0.45.3 (SlightwindSec/bitsandbytes) — the community fork that adds Ascend support; upstream re-add tracked in bitsandbytes-foundation/bitsandbytes#1695.
  • docs/en/training_guides/cpt-npu-tutorial.ipynb — continued pre-training with plain transformers.Trainer on torch_npu. No MindSpeed-LLM, no HF↔MCore conversion — the light-weight single-node alternative to qwen25_pretrain_verify.ipynb.

E2E

  • e2e/cases/c15_qlora_npu.sh — NPU counterpart of C13. Same in-Pod synthetic Qwen2 + chat JSONL pattern; installs bitsandbytes-npu-beta at runtime and runs QLoRA. Skips (rc=77) on unschedulable NPU slice, no PyPI egress, or a bnb-npu wheel that can't load against the workbench's CANN/torch_npu combo.
  • e2e/cases/c16_cpt_npu.sh — NPU counterpart of C14. Same in-Pod synthetic Qwen2 + raw-text corpus; runs CPT via Trainer on torch_npu. Skips (rc=77) on unschedulable NPU slice or failed NPU sanity check.
  • run_all.sh — C15/C16 wired in with the NPU: prefix so they're gated by SKIP_NPU=1 and by require_env NPU_NAMESPACE inside each case.

Doc mdx

  • training-hub-fine-tuning.mdx — link the new notebooks; replace the "bitsandbytes 4-bit is not available on NPU" QLoRA note with the community-fork path (with the compat caveat); split the CPT-on-NPU note into light-weight (transformers.Trainer) vs distributed (MindSpeed-LLM) paths.

Caveats

  • E2E was not executed this session — the dedicated NPU cluster is offline (join token expired 2026-06-18) and the fallback cluster's 8× Ascend910 are fully held by another team's qwen3-27b-w8a8 serving workload. Both new cases are self-contained and follow the same skip/scheduling contract as C7/C8, so they will skip gracefully until an NPU host becomes available. Marking this PR as draft until at least a smoke run completes on real NPU hardware.
  • bitsandbytes-npu-beta on CANN 8.5.0 + torch_npu 2.9.0 is untested by the fork's author (the wheel was built against roughly torch_npu 2.1–2.4 / CANN 8.0–8.1). Both the notebook and the e2e case flag this and skip cleanly on import failure so it is discoverable but non-fatal. If the fork breaks on our current stack, we can either pin to a CANN 8.1-era PyTorch CANN workbench image or wait for upstream PR #1695 to land.
  • QLoRA is aarch64-only — the fork ships only a manylinux2014_aarch64 wheel. All Ascend hosts are aarch64, so this is fine, but the notebook + e2e case both call it out.

Test plan

  • bash e2e/cases/c16_cpt_npu.sh PASS on an NPU cluster with NPU_NAMESPACE + NPU_RESOURCE_NAME set.
  • bash e2e/cases/c15_qlora_npu.sh PASS (or SKIP=77 with a captured "bnb import failed" reason) on the same cluster.
  • yarn lint clean (verified locally — 0 errors, 0 warnings).
  • Both notebooks render in doom dev without stray HTML.

🤖 Generated with Claude Code

training_hub / bitsandbytes / unsloth all target CUDA and don't run on
Huawei Ascend, so the existing QLoRA and CPT tutorials don't cover NPU
users. This PR adds two companion notebooks that reach the same result
on Ascend using only mainline transformers + torch_npu (+ the community
bitsandbytes-npu-beta fork for QLoRA), plus matching e2e cases.

Docs
- docs/en/training_guides/qlora-npu-tutorial.ipynb — true 4-bit NF4
  QLoRA via BitsAndBytesConfig(load_in_4bit=True, ...) and peft LoRA
  on torch_npu. Uses the community bitsandbytes-npu-beta==0.45.3 fork
  (SlightwindSec/bitsandbytes), pinned deliberately; upstream re-add
  tracked in bitsandbytes-foundation/bitsandbytes#1695. Documents
  known compat caveats (fork built for torch_npu 2.1–2.4 / CANN 8.0–8.1,
  aarch64-only wheel), import-order requirement (torch_npu before
  bitsandbytes), NPU-specific gotchas (adamw_torch not paged_adamw_8bit,
  no CPU offload, dataloader_pin_memory=False, PYTORCH_NPU_ALLOC_CONF).
- docs/en/training_guides/cpt-npu-tutorial.ipynb — continued
  pre-training with transformers.Trainer on torch_npu (no MindSpeed-LLM,
  no HF↔MCore conversion needed). Bf16 + adamw_torch; pack raw text to
  BLOCK_SIZE with labels=input_ids. Positioned as the light-weight
  single-node alternative to the MindSpeed-LLM TP/PP recipe.
- training-hub-fine-tuning.mdx — link the two new notebooks; replace
  the "not available on NPU" QLoRA note with the community-fork path;
  add the light-weight vs distributed CPT-on-NPU split.

E2E
- e2e/cases/c15_qlora_npu.sh — NPU counterpart of C13. Same in-Pod
  synthetic Qwen2 + chat JSONL pattern; installs bitsandbytes-npu-beta
  at runtime and runs QLoRA. Skips (rc=77) on no schedulable NPU slice,
  no PyPI egress, or a bnb-npu wheel that can't load against the
  workbench's CANN/torch_npu combo.
- e2e/cases/c16_cpt_npu.sh — NPU counterpart of C14. Same in-Pod
  synthetic base model + raw-text corpus pattern; runs CPT via
  transformers.Trainer on torch_npu. Skips (rc=77) on no schedulable
  NPU slice or failed NPU sanity check.
- run_all.sh — C15/C16 wired in with NPU: prefix so they're gated by
  SKIP_NPU=1 and the require_env NPU_NAMESPACE inside each case.

Caveats
- E2E for C15/C16 was not run this session — the dedicated NPU cluster
  is offline (join token expired 2026-06-18) and the fallback cluster's
  8× Ascend910 are fully held by another team's serving workload.
  Both cases are self-contained and follow the same skip/scheduling
  contract as C7/C8, so they will skip gracefully until an NPU host
  becomes available.
- bitsandbytes-npu-beta on CANN 8.5.0 + torch_npu 2.9.0 is untested
  by the fork's author; the notebook and e2e both flag this and skip
  cleanly on import failure so it is discoverable but non-fatal.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying alauda-ai with  Cloudflare Pages  Cloudflare Pages

Latest commit: 561c916
Status: ✅  Deploy successful!
Preview URL: https://ed06f41b.alauda-ai.pages.dev
Branch Preview URL: https://docs-npu-cpt-qlora.alauda-ai.pages.dev

View logs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant