Skip to content

Commit 63586b4

Browse files
committed
update key and value for the None condition
Signed-off-by: apinge <Tong.Qiu2@amd.com>
1 parent 3351f83 commit 63586b4

File tree

1 file changed

+5
-2
lines changed

1 file changed

+5
-2
lines changed

vllm/v1/attention/backends/rocm_aiter_fa.py

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -729,8 +729,11 @@ def forward(
729729

730730
# decode:extend:prefill
731731
query = query[:num_actual_tokens]
732-
key = key[:num_actual_tokens] if key is not None else key_cache[:num_actual_tokens]
733-
value = value[:num_actual_tokens] if value is not None else value_cache[:num_actual_tokens]
732+
if key is not None:
733+
key = key[:num_actual_tokens]
734+
if value is not None:
735+
value = value[:num_actual_tokens]
736+
734737

735738
output_actual_tokens = output[:num_actual_tokens]
736739

0 commit comments

Comments
 (0)