Skip to content

Missing the MTP module when exporting the Qwen3-Next HF model to an MScore model. #6436

@155394551lzk

Description

@155394551lzk

Describe the feature
The MTP module is not seen when exporting the Qwen3-Next HF model to an MScore model. Does the ms-swift 3.9.2 support MTP?

script
export MEGATRON_LM_PATH=$MEGA_PATH
swift export
--model $MODEL
--model-type qwen3_next
--torch_dtype bfloat16
--output_dir $OUT_DIR
--test_convert_precision true
--to_mcore true

Additional context
ms-swift: 3.9.2
transformers: 4.57.1
megatron-core: 0.13.2
transformer_engine: 2.7.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions