Following are affinity clusters of similar benchmarks based on on_cpu (# of cores used) and topdown metrics (retirement, fronted stall, backend stall, speculation)

Cluster 0 (16 entries): 500.perlbench_r 508.namd_r 511.povray_r 525.x264_r 544.nab_r brl-cad c-ray john-the-ripper namd openssl povray qe quicksilver specfem3d svt-hevc vvenc
Cluster 1 (23 entries): 503.bwaves_r 507.cactuBSSN_r 510.parest_r 519.lbm_r 520.omnetpp_r 521.wrf_r 549.fotonik3d_r 554.roms_r ai-benchmark cloverleaf easywave kripke libxsmm minibud
e mt-dgemm ncnn oidn onednn openvino stream tensorflow xmrig y-cruncher
Cluster 2 (4 entries): 541.leela_r compress-7zip m-queens n-queens
Cluster 3 (5 entries): aobench compress-lz4 gnupg lammps lzbench
Cluster 4 (7 entries): 538.imagick_r 548.exchange2_r avifenc helsing hmmer rays1bench uvg266
Cluster 5 (6 entries): build-eigen build-python compress-gzip hadoop inkscape tscp
Cluster 6 (8 entries): 505.mcf_r 531.deepsjeng_r appleseed asmfish blender primesieve stockfish v-ray
Cluster 7 (14 entries): build2 build-erlang build-ffmpeg build-gcc build-gdb build-gem5 build-godot build-imagemagick build-linux-kernel build-llvm build-mesa build-mplayer build-php
 build-wasmer
Cluster 8 (10 entries): apache cassandra compilebench ctx-clock dbench fast-cli ipc-benchmark memcached sqlite wireguard
Cluster 9 (8 entries): cp2k graphics-magick qmcpack rodinia smallpt stargate vpxenc x264
Cluster 10 (6 entries): blosc clickhouse core-latency daphne gimp mbw
Cluster 11 (6 entries): arrayfire dragonflydb nginx pgbench pjsip rbenchmark
Cluster 12 (5 entries): espeak phpbench pybench securemark smhasher
Cluster 13 (7 entries): apache-iotdb encode-wavpack fftw gcrypt java-scimark2 polybench-c synthmark
Cluster 14 (12 entries): 523.xalancbmk_r 526.blender_r 527.cam4_r 557.xz_r embree graph500 lczero openvkl ospray-studio quadray sysbench tensorflow-lite
Cluster 15 (10 entries): amg askap heffte hpcg incompact3d onnx openfoam parboil pytorch ramspeed
Cluster 16 (9 entries): compress-zstd cpp-perf-bench draco indigobench jpegxl jpegxl-decode polyhedron scimark2 z3
Cluster 17 (12 entries): clomp darktable deepsparse deepspeech ffte himeno llama.cpp llamafile lulesh ngspice npb palabos
Cluster 18 (6 entries): 502.gcc_r build-nodejs ebizzy faiss minife mnn
Cluster 19 (5 entries): botan cachebench glibc-bench gnuradio nettle
Cluster 20 (8 entries): blake2 build-apache build-clash octave-benchmark openjpeg selenium tungsten vkpeak
Cluster 21 (5 entries): mpcbench node-octane openscad pyperformance rav1e
Cluster 22 (7 entries): bork byte libreoffice numpy perl-benchmark rsvg sudokut
Cluster 23 (9 entries): compress-rar dacapobench duckdb ffmpeg gegl node-web-tooling pyhpc renaissance spark-tpcds
Cluster 24 (11 entries): aircrack-ng astcenc basis coremark cpuminer-opt java-jmh kvazaar mrbayes quantlib toybrot webp2
Cluster 25 (6 entries): dav1d opencv ospray schbench svt-av1 svt-vp9
Cluster 26 (9 entries): cryptopp dolfyn encode-flac gmpbench mutex nwchem rnnoise simdjson spark
Cluster 27 (6 entries): cockroach hackbench rocksdb scylladb speedb stress-ng
Cluster 28 (13 entries): aom-av1 financebench gpaw gromacs liquid-dsp neat openradioss pennant rawtherapee srsran tnn whisper.cpp x265
Cluster 29 (11 entries): bullet compress-pbzip2 crafty cython-bench encode-mp3 encode-opus etcpak fhourstones git libraw webp

Experimented some and settled on the following approach for clustering.

Attributes of interest

First question was “clustering based on what”. Following is a more complete set of metrics that range anywhere from the amount of I/O to floating density to counter-based instrumentation. These also have rather different ranges though can be normalized using the mean and standard deviation. I experimented first using a set of 10 metrics (on_cpu, retire, frontend, backend, speculation, IPC, GHz, float-density, branch-density, smt-contention) before settling on just the first five.

Some of these metrics are correlated and why I figured it wouldn’t add value to use both AMD and Intel. Some such as I/O metrics are very orthogonal and likely make sense in broader context but I don’t have enough study to clearly characterize.

metric	count	min	max	median	mean	stddev
elapsed	247	3.53	8.76e+03	532	1.15e+03	1.53e+03
on_cpu	247	0	15.9	6.34	6.68	5.42
inblock	247	0	4.46e+05	0	3.65e+03	3.92e+04
onblock	247	0.46	8.97e+05	131	1.57e+04	7.91e+04
page-fault	247	2.6	1.31e+05	2.1e+03	1.17e+04	2.1e+04
context-switch	247	2.88	4.77e+04	66	2.49e+03	7.58e+03
IPC	247	0.01	4.59	1.51	1.64	0.888
GHz	247	0	5.23	1.87	1.82	1.4
retire-rate	247	0.7	76.2	29.2	32.1	16
frontend-stall	247	0.1	72.8	16.3	21.7	18
backend-stall	247	4.1	97.4	38.2	42.9	21.9
spec-stall	247	0	21.3	2	3.28	3.78
retire-ucode	247	0	1.2	0	0.0769	0.141
retire-fastpath	247	0.7	76.2	25.5	27.7	14.5
float-density	247	0.016	676	62.9	129	151
frontend-latency	247	0.1	57.8	8.4	13.4	13.6
frontend-bandwidth	247	0	31.7	4.8	5.83	4.8
opcache-miss	247	0	65.9	6.1	13.4	15.1
icache-miss	247	0.1	69	13.4	16.8	11.5
backend-cpu	247	0.7	64	9.4	12.8	11.4
backend-memory	247	0.2	95.9	19.9	24.6	18.1
amd-l2-miss	247	0.05	67.5	17.1	18.3	13
amd-l2-density	247	0.022	4.29e+04	35.1	229	2.72e+03
spec-branch	247	0	21.2	1.7	2.82	3.6
spec-pipeline	247	0	2	0	0.113	0.249
branch-miss	247	0	14.8	1.85	2.72	2.99
branch-density	247	4.9	317	128	130	61.8
branch-cond	247	4.5	311	92.3	98.3	48.8
branch-ind	247	0.003	28.7	2.85	4.44	5.12
smt-contention	247	0	48.4	9.6	12.7	13.3
elapsed	238	16	1.69e+04	655	1.66e+03	2.5e+03
on_cpu	238	0	15.8	6.77	7	5.54
inblock	238	0.01	3.86e+05	65.8	6.31e+03	3.54e+04
onblock	238	0.37	6.35e+05	19.9	1.1e+04	5.18e+04
page-fault	238	4.07	1.21e+05	1.68e+03	1.06e+04	2.02e+04
context-switch	238	2.23	6.44e+04	62.4	2.49e+03	8.92e+03
IPC	238	0.01	5.53	1.89	2.02	1.01
GHz	238	0	3.04	1.32	1.2	0.88
retire-rate	238	3.7	87.3	43.2	43.4	15.4
frontend-stall	238	0.5	52	15.9	17.8	11
backend-stall	238	1.3	95.3	26.2	30.9	20.2
spec-stall	238	0	46.7	6.1	8.38	8.3
retire-ucode	238	0	16.7	2.9	3.26	2.26
retire-fastpath	238	2.4	83.2	39.6	40.2	14.9
frontend-latency	238	0.3	39.2	8.6	9.57	6.73
frontend-bandwidth	238	0.1	25.8	7.3	8.2	5.84
backend-cpu	238	0.6	76.1	11.5	15.5	12.4
backend-memory	238	0.3	89.9	10.7	15.4	16.1
l1-stall	238	0	29.6	4.1	5.53	5.66
l2-stall	238	0	58.8	7.1	7.84	8.18
l3-stall	238	0	35	2.3	3.98	5.42
dram-stall	238	0	86.9	3.3	7.46	11.7
store-stall	238	0	28.3	0.8	1.7	3.11
intel-l2-miss	238	0.35	92.7	29.1	30.1	18.2
intel-l2-density	238	0.019	2.12e+04	21.9	126	1.37e+03
spec-branch	238	0	46.7	5.7	7.95	8.3
spec-pipeline	238	0	6.2	0.3	0.427	0.652
branch-miss	238	0	20.4	0.83	1.57	2.32
branch-density	238	6.24	320	129	129	61
branch-cond	238	6.24	320	129	129	61
branch-ind	238	0.027	83	20.8	22.1	17.4

Clustering algorithm

After a web search, I ended up with a variation of Lloyd’s Algorithm. This seems relatively straightforward and settles fairly quickly on a set of clusters. As a summary, this goes through the following steps:

• Start with a set of randomly chosen cluster points. I picked every Nth benchmark as my starting point
• Now assign each benchmark to the closest cluster point. I used a simple Nth dimensional distance as sqrt(distance1 ^2 + distance2 ^2 + distance3 ^2 … distanceN^2).
• Now recompute the cluster points based on a center of the points assigned to the cluster
• Iterate the last two steps until it converges on a set of clusters.

This takes ~8 iterations when I tried it on ~240 phoronix tests along with 23 SPEC CPU 2017 benchmarks.

Now with these clusters, I can both use them to spread out a benchmark analysis across a wide range of different benchmarks and also to think of more similar benchmarks that might substitute for each other. For example cluster #7 from the list above collects a set of build benchmarks while cluster #1 looks like a set of backend bound benchmarks running on all cores.