fix: use dtype instead of deprecated torch_dtype - #735
Conversation
…4.56 config.torch_dtype and the torch_dtype keyword argument were deprecated in transformers 4.56 (PR #39782). Pass dtype based on the installed transformers version (packaging.version), falling back to torch_dtype on older versions.
| from packaging.version import Version | ||
|
|
||
| def _dtype_kwargs(dtype): | ||
| """`dtype` keyword of `from_pretrained` exists since transformers 4.56 (PR #39782); |
There was a problem hiding this comment.
should we import transformers here instead, rather than on top of the file?
Address review feedback: import transformers and packaging.version inside _dtype_kwargs instead of the module top level. Fix ruff findings (line-too-long, docstring formatting, blank lines).
There was a problem hiding this comment.
🟢 Approval recommended
The change is small, targeted, and aligns with the stated deprecation-avoidance goal while preserving compatibility with older transformers versions.
Pull request overview
This PR updates the HQQ quantization path to avoid the torch_dtype deprecation introduced in transformers 4.56 by selecting the appropriate from_pretrained keyword (dtype vs torch_dtype) based on the installed transformers version.
Changes:
- Added a small helper (
_dtype_kwargs) that returns{"dtype": ...}for transformers >= 4.56 and{"torch_dtype": ...}otherwise. - Updated the
AutoModelForCausalLM.from_pretrained(...)call to pass the selected keyword via**_dtype_kwargs(...).
File summaries
| File | Description |
|---|---|
| src/pruna/algorithms/hqq.py | Adds version-gated dtype kwarg selection and updates the HQQ fallback from_pretrained call to use it. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| import transformers | ||
| from packaging.version import Version | ||
|
|
||
| if Version(transformers.__version__) >= Version("4.56"): | ||
| return {"dtype": dtype} | ||
| return {"torch_dtype": dtype} |
|
Done in 3097d26: moved the |
Summary
config.torch_dtypeand thetorch_dtypekeyword argument were deprecated in transformers 4.56 (PR #39782) and replaced bydtype. This PR updates theAutoModelForCausalLM.from_pretrainedcall insrc/pruna/algorithms/hqq.py(line 272, HQQ quantization) to select the keyword from the installed transformers version usingpackaging.version, so transformers < 4.56 keeps working. Thequantization_configargument is unchanged.Changes
src/pruna/algorithms/hqq.py: add a_dtype_kwargshelper that returns{"dtype": ...}on transformers >= 4.56 and{"torch_dtype": ...}otherwise; thefrom_pretrainedcall passes it as**_dtype_kwargs(torch.float16 if ... else ...).Test
python -m py_compile).dtypeand emits no deprecation warning; on older versions it passestorch_dtypeunchanged.Fixes #734