Skip to content

Commit 017c11b

Browse files
authored
[BUG FIX 5616904] Add transformers version restoration after PTQ for VILA (#525)
## What does this PR do? **Type of change:** ? <!-- Use one of the following: Bug fix, new feature, new example, new tests, documentation. --> Bug fix **Overview:** ? Prompt user to manually install correct transformers version for VILA ## Usage <!-- You can potentially add a usage example below. --> ```python # Add a code snippet demonstrating how to use this ``` ## Testing <!-- Mention how have you tested your change if applicable. --> ``` CUDA_VISIBLE_DEVICES=0 bash -e scripts/huggingface_example.sh --model /models/VILA1.5-3b --quant fp8 --tp 1 --pp 1 --trust_remote_code --kv_cache_free_gpu_memory_fraction 0.5 ``` ## Before your PR is "*Ready for review*" <!-- If you haven't finished some of the above items you can still open `Draft` PR. --> - **Make sure you read and follow [Contributor guidelines](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CONTRIBUTING.md)** and your commits are signed. - **Is this change backward compatible?**: Yes/No <!--- If No, explain why. --> - **Did you write any new necessary tests?**: Yes/No - **Did you add or update any necessary documentation?**: Yes/No - **Did you update [Changelog](https://github.com/NVIDIA/TensorRT-Model-Optimizer/blob/main/CHANGELOG.rst)?**: Yes/No <!--- Only for new features, API changes, critical bug fixes or bw breaking changes. --> ## Additional Information <!-- E.g. related issue. --> Signed-off-by: Yue <yueshen@nvidia.com>
1 parent 90e6638 commit 017c11b

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

examples/vlm_ptq/scripts/huggingface_example.sh

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -74,7 +74,16 @@ if [ -n "$KV_CACHE_QUANT" ]; then
7474
fi
7575

7676
if [[ "${MODEL_NAME,,}" == *"vila"* ]]; then
77-
# Install required dependency for VILA
77+
# Check transformers version - must be <= 4.50.0
78+
CURRENT_TRANSFORMERS_VERSION=$(pip show transformers | grep Version | cut -d' ' -f2)
79+
if [ "$(printf '%s\n' "4.50.0" "$CURRENT_TRANSFORMERS_VERSION" | sort -V | head -n1)" = "4.50.0" ] && [ "$CURRENT_TRANSFORMERS_VERSION" != "4.50.0" ]; then
80+
echo "ERROR: transformers version $CURRENT_TRANSFORMERS_VERSION is not supported." >&2
81+
echo "VILA requires transformers<=4.50.0" >&2
82+
echo "Please refer to examples/vlm_ptq/requirements-vila.txt for the supported versions." >&2
83+
echo "You also need to download VILA repository from https://github.com/Efficient-Large-Model/VILA.git and checkout ec7fb2c264920bf004fd9fa37f1ec36ea0942db5" >&2
84+
exit 1
85+
fi
86+
7887
pip install -r ../vlm_ptq/requirements-vila.txt
7988
# Clone original VILA repo
8089
if [ ! -d "$(dirname "$MODEL_PATH")/VILA" ]; then

0 commit comments

Comments
 (0)