Category Archives: experiment

Virtualization comparisons

Performance analysis, tools and experiments Posted on by

Installing and reinstalling Operating Systems can be easier to do if I maintain several virtual machines with each configuration. While this lets me compare VMs and OSs against each other, there is also a question on how the virtual environment compares against the host environment. So I’ve created a few configurations I can use for these comparisons. In particular:

NameThreadsMemoryNotes
boulder1632GBHost: 7840HS, Zen4; ubuntu 24.04
niwot2432GBHost: RX 370, Zen5; ubuntu 24.04
boulder-ubuntu816GBubuntu 24.04 guest
boulder-cachyos816GBcachyos guest
boulder “constrainted”832 GBhost with taskset –cpulist
niwot-ubuntu1216GBubuntu 24.04 guest
niwot-cachyos1216GBcachyos guest
niwot “constrained”1232GBhost with taskset –cpulist

Since I can’t dedicate the entire machine to the VM, I instead bind the VM to run with one thread bound per (hyper-threaded host) core. I also define half the memory. I can then compare this against a host “constrained” configuration that also runs on those same cores, e.g.

taskset --cpu-list 0-15:2 phoronix-test-suite batch-run coremark

The first benchmark I pick for such a comparison is coremark.

NameThreadsScore
boulder16412415
niwot24563857
boulder-ubuntu8296640
boulder-cachyos8310674
boulder “constrained”8317576
niwot-ubuntu12356810
niwot-cachyos12369503
niwot “constrained”12401518

First thing to note is that the 7840 “constrained” configuration runs at 77% of the full host configuration (317576/412415) while the 370 “constrained” configuration runs at 71% (401518/563857) so running half the cores isn’t quite as large for the 370.

Next thing to notice is the Ubuntu virtual machine performance of 7840 is 93% of constrained while 370 is 88% of constrained. The net effect is the host only benchmark is 1.37x faster on 370 than 7840 but the virtual machine is only 1.20x faster. CachyOS is faster and hence it is 98% of host on 7840 and 93% of host on 370.

This is only one benchmark so will also be useful to cross-check how much these trends also apply to other workloads. I can probably also separate this to see how much the “constrained” matches the full system and then see what the virtualization overhead as two separate comparisons.

SPEC CPU2017 Ryzen AI HX 370 vs. Ryzen 7840 HS

Performance analysis, tools and experiments Posted on by

As a follow up to previous posting looking at Ryzen AI HX 370, I have also done some SPEC CPU2017 experiments. My general idea is to compare the two processors with a few caveats:

  • I have used a configuration file roughly based on AMD Server configuration files and using the AMD AOCC compiler. However, because I am not trying to publish the absolute best results for hardware (and haven’t tuned to do so) – I will report relative comparison results rather than absolute numbers.
  • I expect AMD to release a new version of AMD AOCC for the Zen5 core. I didn’t have it when I did these comparisons and like using the same flags on both systems so these comparisons used the same flags for both Zen4 and Zen5 systems.
  • SPEC CPU2017 guidelines give a requirement of 2 GB of memory per core. My Ryzen 370 system has 24 cores and only 32 GB of memory. So I expect some benchmarks might run out of memory. For this reason and trying to get an overall comparison I’ve thus done two runs:
    • A 16-copy run on both systems. This uses all (hyperthreaded) cores on the Ryzen 7840 HS and a mix of hyperthreading of Zen5 cores + non-hyperthreading of Zen5C cores.
    • A 24-copy run on the Ryzen 370 system.

Relative results are shown in the tables below. This gives me some opportunities to drill a little deeper on why some benchmarks have larger gains than others.

Overall the differences between 16 threads and 24 threads are interesting. Using 24 threads seems to mostly help the intrate benchmarks with the geomean going from +12% to +21% and every benchmark improving vs 7840. Overall, using 24 threads seems to be more mixed with fprate. On average slightly slower than 16-threads. In both cases, the individual benchmarks also differ.

16-thread24-thread
500.perlbench_r1.121.24
502.gcc_r1.171.15
505.mcf_r1.091.21
520.omnetpp_r1.071.16
523.xalancbmk_r1.351.23
525.x264_r1.191.31
531.deepsjeng_r1.111.18
541.leela_r0.941.07
548.exchange_r1.241.38
557.xz_r0.961.16
geomean1.121.21

My intrate comparisons range from -6% to +35% with a geometric mean of +12%

16-thread24-thread
503.bwaves_r1.111.09
507.cactuBSSN_r1.301.25
508.namd_r1.221.34
510.parest_r1.531.10
511.povray_r1.191.30
519.lbm_r1.631.59
521.wrf_r1.321.17
526.blender_r1.241.27
527.cam4_r1.611.45
538.imagick_r1.191.32
544.nab_r1.191.31
549.fotonik_r1.111.09
554.roms_r1.431.15
geomean1.301.26

My fprate comparisons range from +11% to +63% with a geometric mean of +30%

phoronix – Ryzen AI HX 370 vs Ryzen 7840 HS

Performance analysis, tools and experiments Posted on by

As a follow up comparison of Ryzen AI HX 370 processor compared to Ryzen 7840 HS, this posting looks at some Phoronix benchmarks.

I’ve run more than 200 Phoronix benchmarks in analysis using performance counters. I use these clusters to guide the benchmarks chosen trying to pick one from each cluster. In some cases where the benchmark didn’t easily run on Ubuntu 24.04, I skipped to another benchmark rather than debug the original issue. A cluster list from September 2024 below:

Cluster 0 (10 entries): 505.mcf_r 531.deepsjeng_r appleseed asmfish avifenc blender ospray primesieve stockfish v-ray
Cluster 1 (3 entries): 520.omnetpp_r amg compress-xz
Cluster 2 (7 entries): 500.perlbench_r 525.x264_r 544.nab_r brl-cad quicksilver smallpt vvenc
Cluster 3 (10 entries): aom-av1 cp2k neat openradioss qmcpack srsran svt-av1 vpxenc x264 x265
Cluster 4 (14 entries): 538.imagick_r 548.exchange2_r astcenc basis coremark cpuminer-opt kvazaar mrbayes quantlib rav1e rays1bench toybrot uvg266 webp2
Cluster 5 (10 entries): blake2 build-apache build-clash build-eigen octave-benchmark openjpeg selenium tscp tungsten vkpeak
Cluster 6 (19 entries): build2 build-ffmpeg build-gcc build-gdb build-gem5 build-godot build-linux-kernel build-llvm build-mesa build-mplayer build-nodejs build-\
wasmer hackbench helsing mnn rocksdb scylladb speedb stress-ng
Cluster 7 (5 entries): bork byte openscad phpbench sudokut
Cluster 8 (10 entries): aobench compress-lz4 crafty fhourstones git gnupg lammps lzbench tjbench webp
Cluster 9 (11 entries): apache-iotdb compress-zstd core-latency cpp-perf-bench draco encode-wavpack fftw jpegxl polybench-c polyhedron z3
Cluster 10 (10 entries): botan cachebench cryptsetup gcrypt glibc-bench gnuradio java-scimark2 nettle simdjson synthmark
Cluster 11 (7 entries): duckdb inkscape libreoffice node-web-tooling numpy perl-benchmark rsvg
Cluster 12 (15 entries): bullet compress-pbzip2 cython-bench encode-flac encode-mp3 encode-opus etcpak ffmpeg hmmer libraw node-octane pyperformance rnnoise scim\
ark2 stargate
Cluster 13 (7 entries): build-python compress-gzip compress-rar dacapobench gegl hadoop spark-tpcds
Cluster 14 (10 entries): 508.namd_r 511.povray_r aircrack-ng c-ray graphics-magick java-jmh namd povray rodinia svt-hevc
Cluster 15 (7 entries): askap hpcg incompact3d onnx parboil pytorch whisperfile
Cluster 16 (15 entries): 503.bwaves_r 507.cactuBSSN_r 510.parest_r 519.lbm_r 521.wrf_r 549.fotonik3d_r 554.roms_r cloverleaf easywave kripke mt-dgemm ncnn stream\
 tensorflow xmrig
Cluster 17 (8 entries): darktable deepsparse ffte llama.cpp llamafile npb openfoam palabos
Cluster 18 (4 entries): 541.leela_r compress-7zip m-queens n-queens
Cluster 19 (7 entries): clomp deepspeech heffte himeno lulesh ngspice ramspeed
Cluster 20 (11 entries): 523.xalancbmk_r ai-benchmark libxsmm minibude oidn onednn openvino quadray tensorflow-lite xnnpack y-cruncher
Cluster 21 (4 entries): 502.gcc_r 527.cam4_r ebizzy faiss
Cluster 22 (5 entries): blosc dragonflydb mbw minife pjsip
Cluster 23 (4 entries): john-the-ripper openssl qe specfem3d
Cluster 24 (6 entries): arrayfire build-erlang build-imagemagick build-php nginx rbenchmark
Cluster 25 (11 entries): cryptopp dolfyn espeak gmpbench mpcbench mutex nwchem pybench securemark smhasher spark
Cluster 26 (12 entries): apache cassandra cockroach compilebench ctx-clock dbench fast-cli ipc-benchmark memcached pgbench sqlite wireguard
Cluster 27 (11 entries): clickhouse daphne dav1d gimp indigobench jpegxl-decode opencv pyhpc renaissance schbench svt-vp9
Cluster 28 (9 entries): 526.blender_r 557.xz_r embree graph500 lczero openvkl ospray-studio sysbench ttsiod-renderer
Cluster 29 (8 entries): financebench gpaw gromacs liquid-dsp pennant rawtherapee tnn whisper.cpp

Following is a summary of the benchmarks followed by some observations

clusterbenchmarkmetric ratio7840 metrichx 370 metric7840 on cpuhx 370 on cpu7840 retirehx 370 retire7840 frontendhx 370 frontend7840 backendhx 370 backend7840 speculationhx 370 speculation
0ospray1.583.87314 / second6.07719 /sec14.4621.2829.3%30.7%27.3%11.8%41.1%54.2%2.3%2.4%
1compress-xz0.9628.665 seconds29.736 seconds11.0412.458.2%7.3%10.2%17.3%76.5%68.2%5.1%7.1%
2quicksilver1.4112610000 fom1776333 fom15.3819.9%49.8%15.9%6.9%15.9%38.9%59.5%4.4%2.7%
3x2651.6513.79 frames/second22.81 frames/sec7.7211.6235.0%26.9%14.3%22.5%48.0%47.4%2.7%3.0%
4coremark1.37411227 iterations/sec561065 iterations/sec11.9814.4345.7%37.0%39.7%42.0%14.2%20.2%0.3%0.8%
5build-eigen0.7763.356 seconds82.516 seconds0.930.9425.2%20.4%50.5%52.8%18.6%21.9%5.6%4.8%
6build-gcc1.061038.166 seconds976.243 seconds9.9810.9124.1%18.3%51.5%60.0%19.7%18.2%4.7%3.1%
7phpbench0.771159425 score900908 score0.800.8361.2%48.6%23.0%30.1%15.0%20.1%0.8%1.1%
8lzbench0.58192 MB/s111 MB/s0.800.8234.1%22.7%26.3%36.5%21.5%21.2%18.1%19.4%
9compress-zstd1.011534.8 MB/s1556.6 MB/s4.233.4521.4%18.3%9.5%17.8%62.8%55.7%6.3%0.2%
10simdjson0.795.58 GB/s4.41 GB/s0.930.9450.4%42.7%13.1%27.0%33.2%28.0%3.3%1.5%
11perl-benchmark0.780.068363375 seconds0.08713901 seconds0.930.9243.0%35.5%41.8%41.7%11.1%18.0%4.2%4.6%
12ffmpeg0.99252.66 fps251.11 fps3.672.6132.3%29.1%18.4%30.3%29.0%33.8%5.6%6.7%
13compress-gzip0.6928.116 seconds40.597 seconds0.960.9519.9%15.1%26.4%29.1%42.0%43.0%11.7%12.7%
14povray1.3438.681 seconds28.778 seconds13.3218.8331.8%40.1%3.5%16.3%25.5%41.5%1.3%2.0%
15whisperfile1.1154.13398 seconds48.57337 seconds7.4410.8120.0%15.2%2.2%15.5%77.3%68.9%0.3%0.3%
16easywave1.268.809 seconds7.005 seconds14.6020.534.5%4.8%3.1%15.1%83.6%74.6%0.1%0.1%
17darktable1.345.711 seconds4.267 seconds3.425.5027.9%19.1%7.2%15.2%63.5%60.9%1.3%1.0%
18compress-7zip1.0176676 MIPS77409 MIPS12.0317.2721.5%13.0%38.6%53.5%29.1%19.7%10.8%13.8%
19himeno1.074447 MFLOPS4769 MFLOPS0.910.9126.4%33.3%2.5%2.7%71.0%63.7%0.2%0.3%
20minibude1.36537.395 GFinst/s733.427 GFInst/s15.3620.5119.8%18.7%0.3%1.6%79.8%79.0%0.1%0.4%
21ebizzy0.18774839 records/s140179 records/s12.8719.827.3%0.6%35.3%63.1%57.3%36.3%0.0%0.0%
22pjsip0.794613 response/sec3665 response/sec2.402.2312.2%11.3%38.4%33.9%48.4%51.3%1.1%1.1%
23openssl1.6315219867520 bytes/s17696663040 bytes/s15.5123.2546.5%33.4%4.9%13.3%48.7%53.2%0.0%0.0%
24build-php1.1667.052 seconds65.354 seconds8.3010.2020.8%15.1%50.4%57.0%24.8%24.1%3.9%3.4%
25pybench0.84554 ms663 ms0.750.7970.1%63.9%15.9%17.0%11.4%17.0%2.6%2.1%
26dbench3.74687.037 MB/s2573 MB/s1.052.0619.4%22.2%70.0%38.3%9.9%37.5%0.7%0.9%
27indigobench1.402.090 samples/sec2.917 samples/sec14.1421.2525.8%19.9%14.8%29.3%54.0%44.9%5.4%5.4%
28lczero1.41108 nodes/sec152 nodes/sec13.2318.3416.8%14.3%4.4%3.8%78.7%81.6%0.1%0.1%
29rawtherapee1.0554.194 seconds51.600 seconds7.7110.1929.0%18.5%12.6%27.1%57.0%44.8%1.5%1.3%

The first observation is most all single-threaded benchmarks run faster on the 7840 than on the Strix 370. In contrast the largest differences are among those with largest number of “on_cpu” threads.

There are two outliers that deserve a second look:

  • ebizzy is over 5x faster on 7840 than hx 370. This benchmark runs quickly so need to make sure it is running correctly in both instances. I don’t see these ratios in the two SPEC CPU2017 benchmarks also part of this group.
  • dbench runs over 3x faster on hx370 than 7840. The on_cpu is almost twice. Again useful to understand if there is another influence affecting this benchmark. Perhaps this one testing something else.

clustering

Performance analysis, tools and experiments Posted on by

Following are affinity clusters of similar benchmarks based on on_cpu (# of cores used) and topdown metrics (retirement, fronted stall, backend stall, speculation) Experimented some and settled on the following approach for clustering. Attributes of interest First question was “clustering … Continue reading →

graphics-magick sharpen, compiler improvements

Performance analysis, tools and experiments Posted on by

The following Phoronix Article – https://www.phoronix.com/review/nvidia-gh200-compilers compares GCC 13.2 with Clang 17.0.2 on an ARM platform. On the discussions attached the improvement for graphics-magick sharpen benchmark particularly stand out. So I thought I would see if I could see a … Continue reading →