Skip to content

perf(uniq): -w benchmark #13199

Description

@HackingRepo

Hi, uutils mainteners

we did a benchmark for the uu uniq utility, with -w 512 we see 5x slower than gnu uniq

Benchmark 1: ./uniq -w 512 /usr/share/dict/words
  Time (mean ± σ):      32.2 ms ±   2.8 ms    [User: 30.5 ms, System: 1.5 ms]
  Range (min … max):    29.8 ms …  43.6 ms    89 runs
 
  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
 
Benchmark 2: gnuuniq -w 512 /usr/share/dict/words
  Time (mean ± σ):       6.3 ms ±   0.9 ms    [User: 5.0 ms, System: 1.1 ms]
  Range (min … max):     5.4 ms …  11.2 ms    351 runs
 
Summary
  gnuuniq -w 512 /usr/share/dict/words ran
    5.13 ± 0.89 times faster than ./uniq -w 512 /usr/share/dict/words

so there was some slowdown that needs be fixed

eevn just with a small value 1 that slowdown still exist so reveals a serious perfomance issue with the -w that needs be fixed

relunsec@relunsec:~/software/coreutils/target/release$ hyperfine --shell=none --warmup 3 "./uniq -w 1 /usr/share/dict/words" "gnuuniq -w 1 /usr/share/dict/words"
Benchmark 1: ./uniq -w 1 /usr/share/dict/words
  Time (mean ± σ):      31.1 ms ±   2.5 ms    [User: 28.8 ms, System: 1.9 ms]
  Range (min … max):    28.2 ms …  39.0 ms    92 runs
 
Benchmark 2: gnuuniq -w 1 /usr/share/dict/words
  Time (mean ± σ):       5.7 ms ±   0.7 ms    [User: 4.6 ms, System: 1.0 ms]
  Range (min … max):     4.9 ms …   9.0 ms    436 runs
 
Summary
  gnuuniq -w 1 /usr/share/dict/words ran
    5.42 ± 0.76 times faster than ./uniq -w 1 /usr/share/dict/words
relunsec@relunsec:~/software/coreutils/target/release$ 

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions