Functions to count uniqs

Functions to count uniqs.

Functions to count uniqs

Function Function(State) StateSize Result QPS
uniqExact uniqExactState 1600003 100000 59.23
uniq uniqState 200804 100315 85.55
uniqCombined uniqCombinedState 98505 100314 108.09
uniqCombined(12) uniqCombinedState(12) 3291 98160 151.64
uniqCombined(15) uniqCombinedState(15) 24783 100768 110.18
uniqCombined(18) uniqCombinedState(18) 196805 100332 101.56
uniqCombined(20) uniqCombinedState(20) 786621 100088 65.05
uniqCombined64(12) uniqCombined64State(12) 3291 98160 164.96
uniqCombined64(15) uniqCombined64State(15) 24783 100768 133.96
uniqCombined64(18) uniqCombined64State(18) 196805 100332 110.85
uniqCombined64(20) uniqCombined64State(20) 786621 100088 66.48
uniqHLL12 uniqHLL12State 2651 101344 177.91
uniqTheta uniqThetaState 32795 98045 144.05
uniqUpTo(100) uniqUpToState(100) 1 101 222.93

Stats collected via script below on 22.2

funcname=( "uniqExact" "uniq" "uniqCombined" "uniqCombined(12)" "uniqCombined(15)" "uniqCombined(18)" "uniqCombined(20)" "uniqCombined64(12)" "uniqCombined64(15)" "uniqCombined64(18)" "uniqCombined64(20)" "uniqHLL12" "uniqTheta" "uniqUpTo(100)")
funcname2=( "uniqExactState" "uniqState" "uniqCombinedState" "uniqCombinedState(12)" "uniqCombinedState(15)" "uniqCombinedState(18)" "uniqCombinedState(20)" "uniqCombined64State(12)" "uniqCombined64State(15)" "uniqCombined64State(18)" "uniqCombined64State(20)" "uniqHLL12State" "uniqThetaState" "uniqUpToState(100)")

length=${#funcname[@]}
 

for (( j=0; j<length; j++ ));
do
  f1="${funcname[$j]}"
  f2="${funcname2[$j]}"
  size=$( clickhouse-client -q "select ${f2}(toString(number)) from numbers_mt(100000) FORMAT RowBinary" | wc -c )
  result="$( clickhouse-client -q "select ${f1}(toString(number)) from numbers_mt(100000)" )"
  time=$( rm /tmp/clickhouse-benchmark.json; echo "select ${f1}(toString(number)) from numbers_mt(100000)" | clickhouse-benchmark -i200 --json=/tmp/clickhouse-benchmark.json &>/dev/null; cat /tmp/clickhouse-benchmark.json | grep QPS  )

  printf "|%s|%s,%s,%s,%s\n" "$f1" "$f2" "$size" "$result" "$time"
done
Last modified 2022.03.10 : Create uniq-functions.md (2d5d843b5)