Skip to content

Commit eae34a4

Browse files
authored
[benchmark]add qwen3-235b pd+ep yaml (#5225)
1 parent df2be1c commit eae34a4

File tree

3 files changed

+27
-1
lines changed

3 files changed

+27
-1
lines changed
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
max_model_len: 32768
22
max_num_seqs: 96
3-
gpu_memory_utilization: 0.85
3+
gpu_memory_utilization: 0.8
44
kv_cache_ratio: 0.71
55
tensor_parallel_size: 4
66
quantization: wint4
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
max_model_len: 32768
2+
max_num_seqs: 32
3+
data_parallel_size: 4
4+
tensor_parallel_size: 1
5+
enable_expert_parallel: True
6+
enable_prefix_caching: False
7+
splitwise_role: decode
8+
cache_transfer_protocol: "rdma"
9+
rdma_comm_ports: "7671,7672,7673,7674"
10+
pd_comm_port: "2335"
11+
engine_worker_queue_port: "4582,4583,4584,4585"
12+
graph_optimization_config:
13+
use_cudagraph: False
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
max_model_len: 32768
2+
max_num_seqs: 32
3+
data_parallel_size: 4
4+
tensor_parallel_size: 1
5+
enable_expert_parallel: True
6+
enable_prefix_caching: False
7+
splitwise_role: prefill
8+
cache_transfer_protocol: "rdma"
9+
rdma_comm_ports: "7675,7676,7677,7678"
10+
pd_comm_port: "2334"
11+
engine_worker_queue_port: "4368,4369,4360,4361"
12+
graph_optimization_config:
13+
use_cudagraph: False

0 commit comments

Comments
 (0)