Skip to content

feat: add extensible Ascend A5 / Ascend 950 runtime support#260

Open
fxp19950422 wants to merge 1 commit into
modelscope:mainfrom
fxp19950422:feat/ascend-platform-main
Open

feat: add extensible Ascend A5 / Ascend 950 runtime support#260
fxp19950422 wants to merge 1 commit into
modelscope:mainfrom
fxp19950422:feat/ascend-platform-main

Conversation

@fxp19950422

Copy link
Copy Markdown

Overview

This PR ports the Ascend work from the v1-based #259 implementation onto the current mainline architecture without merging the unrelated v1 history. It introduces reusable platform, attention, and model-runtime abstractions so future Ascend model families can be added without scattering NPU-specific branches through shared pipelines.

Changes

  • Add explicit platform resolution for CPU, CUDA, ROCm, MPS, and Ascend; the default remains CUDA and detected NPUs never take over existing paths.
  • Add lazy Ascend capability probes for torch-npu, MindIE attention, compile, MXFP8/W4A4 linear quantization, and FP8 attention.
  • Refactor attention dispatch into registered backends while preserving the existing CUDA/ROCm AUTO priority; NPU AUTO resolves MindIE -> SDPA -> eager.
  • Add Qwen-Image/Qwen-Image-Edit runtime adapters for component preparation, denoise timestep hooks, MindIE compile, and native quantization.
  • Add a hardware-independent Qwen attention processor hook for MindIE FP8 attention injection.
  • Add public QuantizationConfig support while preserving use_fp8_linear compatibility and existing CUDA/Nunchaku behavior.
  • Fail early for unsupported combinations including multi-NPU, NPU Nunchaku checkpoints, compile plus offload/dynamic adapters, and native quantization plus offload.
  • Add Ascend installation documentation, compatibility guidance, examples, unit/mock tests, and opt-in hardware integration tests.

Compatibility

  • Existing CUDA, ROCm, MPS, and CPU defaults and explicit backend behavior remain unchanged.
  • torch-npu and MindIE-SD remain optional and are not added to core package dependencies.
  • Ordinary NPU inference may fall back to SDPA/eager when MindIE is absent; explicitly requested MindIE features fail with actionable errors.
  • The initial NPU scope is a single A5/Ascend 950 device. HCCL and multi-device extension points are reserved for a later phase.

Validation completed

  • git diff HEAD^ --check
  • compileall for diffsynth_engine, tests, and examples
  • 38 relevant unit/mock tests: 34 passed, 4 hardware-only tests skipped
  • CPU AUTO attention numerically matched eager with output shape (2, 7, 3, 5)
  • Capability probing safely reported unavailable features in a non-mapped runtime

Ascend hardware follow-up

The earlier v1 implementation in #259 was validated on Ascend hardware. The currently restarted container exposes /dev/davinci_manager and /dev/hisi_hdc but not /dev/davinci0 or /dev/devmm_svm; npu-smi therefore fails DCMI initialization and torch_npu reports zero devices. Mainline A5 hardware regression is pending restoration of the compute device mapping, after which the opt-in operator tests and the Qwen Image/Edit BF16, ControlNet/LoRA, FB-cache, offload, compile, MXFP8, W4A4, and FP8-attention matrix will be run.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces comprehensive support for running Qwen-Image models on Ascend NPU platforms (such as Ascend A5 and 950). It establishes a robust platform abstraction layer, implementing runtime adapters for CUDA, ROCm, and Ascend to handle platform-specific validation, component preparation, and compilation. Key additions include a new modular attention backend registry supporting MindIE-SD attention, MindIE-SD native quantization (MXFP8, W4A4, and FP8 attention), and MindIE compilation. Additionally, the Qwen DiT FB-cache mechanism has been enhanced to support multiple cache streams and sequence parallelism. Comprehensive unit and integration tests, along with documentation and examples, have been added to verify these platform integrations. No review comments were provided, so there is no feedback to address.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant