docs(training-hub): add QLoRA & CPT tutorials for Ascend NPU#279
Draft
typhoonzero wants to merge 1 commit into
Draft
docs(training-hub): add QLoRA & CPT tutorials for Ascend NPU#279typhoonzero wants to merge 1 commit into
typhoonzero wants to merge 1 commit into
Conversation
training_hub / bitsandbytes / unsloth all target CUDA and don't run on Huawei Ascend, so the existing QLoRA and CPT tutorials don't cover NPU users. This PR adds two companion notebooks that reach the same result on Ascend using only mainline transformers + torch_npu (+ the community bitsandbytes-npu-beta fork for QLoRA), plus matching e2e cases. Docs - docs/en/training_guides/qlora-npu-tutorial.ipynb — true 4-bit NF4 QLoRA via BitsAndBytesConfig(load_in_4bit=True, ...) and peft LoRA on torch_npu. Uses the community bitsandbytes-npu-beta==0.45.3 fork (SlightwindSec/bitsandbytes), pinned deliberately; upstream re-add tracked in bitsandbytes-foundation/bitsandbytes#1695. Documents known compat caveats (fork built for torch_npu 2.1–2.4 / CANN 8.0–8.1, aarch64-only wheel), import-order requirement (torch_npu before bitsandbytes), NPU-specific gotchas (adamw_torch not paged_adamw_8bit, no CPU offload, dataloader_pin_memory=False, PYTORCH_NPU_ALLOC_CONF). - docs/en/training_guides/cpt-npu-tutorial.ipynb — continued pre-training with transformers.Trainer on torch_npu (no MindSpeed-LLM, no HF↔MCore conversion needed). Bf16 + adamw_torch; pack raw text to BLOCK_SIZE with labels=input_ids. Positioned as the light-weight single-node alternative to the MindSpeed-LLM TP/PP recipe. - training-hub-fine-tuning.mdx — link the two new notebooks; replace the "not available on NPU" QLoRA note with the community-fork path; add the light-weight vs distributed CPT-on-NPU split. E2E - e2e/cases/c15_qlora_npu.sh — NPU counterpart of C13. Same in-Pod synthetic Qwen2 + chat JSONL pattern; installs bitsandbytes-npu-beta at runtime and runs QLoRA. Skips (rc=77) on no schedulable NPU slice, no PyPI egress, or a bnb-npu wheel that can't load against the workbench's CANN/torch_npu combo. - e2e/cases/c16_cpt_npu.sh — NPU counterpart of C14. Same in-Pod synthetic base model + raw-text corpus pattern; runs CPT via transformers.Trainer on torch_npu. Skips (rc=77) on no schedulable NPU slice or failed NPU sanity check. - run_all.sh — C15/C16 wired in with NPU: prefix so they're gated by SKIP_NPU=1 and the require_env NPU_NAMESPACE inside each case. Caveats - E2E for C15/C16 was not run this session — the dedicated NPU cluster is offline (join token expired 2026-06-18) and the fallback cluster's 8× Ascend910 are fully held by another team's serving workload. Both cases are self-contained and follow the same skip/scheduling contract as C7/C8, so they will skip gracefully until an NPU host becomes available. - bitsandbytes-npu-beta on CANN 8.5.0 + torch_npu 2.9.0 is untested by the fork's author; the notebook and e2e both flag this and skip cleanly on import failure so it is discoverable but non-fatal. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Deploying alauda-ai with
|
| Latest commit: |
561c916
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://ed06f41b.alauda-ai.pages.dev |
| Branch Preview URL: | https://docs-npu-cpt-qlora.alauda-ai.pages.dev |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
training_hub/bitsandbytes/unslothall target CUDA and don't run on Huawei Ascend, so the existing QLoRA and CPT tutorials (#270) don't cover NPU users. This PR adds two companion notebooks that reach the same result on Ascend using only mainlinetransformers+torch_npu(+ the communitybitsandbytes-npu-betafork for QLoRA), and matching e2e cases.Files added
Docs
docs/en/training_guides/qlora-npu-tutorial.ipynb— true 4-bit NF4 QLoRA viaBitsAndBytesConfig(load_in_4bit=True, ...)+peftLoRA ontorch_npu. Usesbitsandbytes-npu-beta==0.45.3(SlightwindSec/bitsandbytes) — the community fork that adds Ascend support; upstream re-add tracked in bitsandbytes-foundation/bitsandbytes#1695.docs/en/training_guides/cpt-npu-tutorial.ipynb— continued pre-training with plaintransformers.Trainerontorch_npu. NoMindSpeed-LLM, no HF↔MCore conversion — the light-weight single-node alternative toqwen25_pretrain_verify.ipynb.E2E
e2e/cases/c15_qlora_npu.sh— NPU counterpart of C13. Same in-Pod synthetic Qwen2 + chat JSONL pattern; installsbitsandbytes-npu-betaat runtime and runs QLoRA. Skips (rc=77) on unschedulable NPU slice, no PyPI egress, or a bnb-npu wheel that can't load against the workbench's CANN/torch_npu combo.e2e/cases/c16_cpt_npu.sh— NPU counterpart of C14. Same in-Pod synthetic Qwen2 + raw-text corpus; runs CPT viaTrainerontorch_npu. Skips (rc=77) on unschedulable NPU slice or failed NPU sanity check.run_all.sh— C15/C16 wired in with theNPU:prefix so they're gated bySKIP_NPU=1and byrequire_env NPU_NAMESPACEinside each case.Doc mdx
training-hub-fine-tuning.mdx— link the new notebooks; replace the "bitsandbytes 4-bit is not available on NPU" QLoRA note with the community-fork path (with the compat caveat); split the CPT-on-NPU note into light-weight (transformers.Trainer) vs distributed (MindSpeed-LLM) paths.Caveats
qwen3-27b-w8a8serving workload. Both new cases are self-contained and follow the same skip/scheduling contract as C7/C8, so they will skip gracefully until an NPU host becomes available. Marking this PR as draft until at least a smoke run completes on real NPU hardware.bitsandbytes-npu-betaon CANN 8.5.0 + torch_npu 2.9.0 is untested by the fork's author (the wheel was built against roughlytorch_npu 2.1–2.4/ CANN 8.0–8.1). Both the notebook and the e2e case flag this and skip cleanly on import failure so it is discoverable but non-fatal. If the fork breaks on our current stack, we can either pin to a CANN 8.1-eraPyTorch CANNworkbench image or wait for upstream PR #1695 to land.manylinux2014_aarch64wheel. All Ascend hosts are aarch64, so this is fine, but the notebook + e2e case both call it out.Test plan
bash e2e/cases/c16_cpt_npu.shPASS on an NPU cluster withNPU_NAMESPACE+NPU_RESOURCE_NAMEset.bash e2e/cases/c15_qlora_npu.shPASS (or SKIP=77 with a captured "bnb import failed" reason) on the same cluster.yarn lintclean (verified locally — 0 errors, 0 warnings).doom devwithout stray HTML.🤖 Generated with Claude Code