Skip to content

Commit 723bf67

Browse files
committed
update key and value for the None condition
Signed-off-by: apinge <Tong.Qiu2@amd.com>
1 parent 91e4a58 commit 723bf67

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

vllm/v1/attention/backends/rocm_aiter_fa.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -704,8 +704,11 @@ def forward(
704704

705705
# decode:extend:prefill
706706
query = query[:num_actual_tokens]
707-
key = key[:num_actual_tokens] if key is not None else key_cache[:num_actual_tokens]
708-
value = value[:num_actual_tokens] if value is not None else value_cache[:num_actual_tokens]
707+
if key is not None:
708+
key = key[:num_actual_tokens]
709+
if value is not None:
710+
value = value[:num_actual_tokens]
711+
709712

710713
output_actual_tokens = output[:num_actual_tokens]
711714

0 commit comments

Comments
 (0)