fix(quantization): fix nvfp4 availability check - #2331
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review. 📝 WalkthroughWalkthroughThe change adds a shared attribute validation helper and replaces duplicated checks in NVFP4 availability validation for input and weight quantizers. ChangesQuantizer Validation
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to NVFP4 availability checks now recognize underscored quantizer attributes after compression, allowing the compatible GEMM path to be selected without changing mismatch handling. The change is ready to merge. 🚥 Pre-merge checks | ✅ 6✅ Passed checks (6 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
What does this PR do?
Type of change: Bug fix
As described in #2330,
_nvfp4_availability_checkdoes not account for attributes prefixed with an underscore when checking quantizer attributes, resulting in a failed check. Consequently, after applying compression to an NVFP4-quantized model, the corresponding GEMM implementation cannot be located during inference, and the warning "RealQuantLinear: No real-quant GEMM found" is raised.Usage
Testing
Before your PR is "Ready for review"
Make sure you read and follow Contributor guidelines and your commits are signed (
git commit -s -S).Make sure you read and follow the Security Best Practices (e.g. avoiding hardcoded
trust_remote_code=True,torch.load(..., weights_only=False),pickle, etc.).CONTRIBUTING.md: N/AAdditional Information
Related issue: #2330
Summary by CodeRabbit