Add RTX 6000 speed

[d3y4n]

Based on the output of default benchmark cmd.

2048,2048,313.21,128,35.66,52184460
4096,2048,317.81,128,35.12,80373132
8192,2048,317.33,128,34.36,136750476
16384,2048,310.08,128,32.78,249505164
32768,2048,296.95,128,31.72,475014540
65536,2048,273.55,128,29.62,926033292

[tao12345666333]

Could you please share the complete operating steps and information about the machine resources?

This result differs significantly from the speed I tested.🤔

https://x.com/i/status/2054161265577308453

[d3y4n]

@tao12345666333 good callout, it did indeed look a bit suspicious, I'm running this on g7e.2xlarge AWS instance at the moment. I will the card in the upcoming weeks in my PC.

This is what I ran:

./ds4-bench \
  -m ds4flash.gguf \
  --prompt-file speed-bench/promessi_sposi.txt \
  --ctx-start 2048 \
  --ctx-max 65536 \
  --step-incr 2048 \
  --gen-tokens 128