{"id":1889,"date":"2024-03-01T12:39:21","date_gmt":"2024-03-01T12:39:21","guid":{"rendered":"https:\/\/mvermeulen.org\/perf\/?page_id=1889"},"modified":"2024-03-02T01:24:07","modified_gmt":"2024-03-02T01:24:07","slug":"ipc-benchmark","status":"publish","type":"page","link":"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/ipc-benchmark\/","title":{"rendered":"ipc-benchmark"},"content":{"rendered":"\n<p>Testing an interprocess communication primitives. There are 24 different subtests including<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>TCP socket with different sized messages<\/li>\n\n\n\n<li>Pipes with different sized messages<\/li>\n\n\n\n<li>FIFO pipes with different sized messages<\/li>\n\n\n\n<li>UNIX domain sockets with different sized messages<\/li>\n<\/ul>\n\n\n\n<p>Topdown summary shows mostly two processes running but not spending much time and also a moderate level of interrupts. For example the initial section is TCP sockets with increasing message sizes.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/03\/systemtime-7.png\" alt=\"\" class=\"wp-image-1899\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/03\/systemtime-7.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/03\/systemtime-7-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/03\/systemtime-7-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Topdown profile shows frontend time as the largest portion.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/03\/amdtopdown-7.png\" alt=\"\" class=\"wp-image-1900\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/03\/amdtopdown-7.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/03\/amdtopdown-7-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/03\/amdtopdown-7-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>AMD metrics show little time is spent on the CPU, approximately 1\/2 of one core. This is not floating point code. There is a high opcache miss rate.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              1020.621\non_cpu               0.032          # 0.51 \/ 16 cores\nutime                44.062\nstime                474.548\nnvcsw                289753         # 99.13%\nnivcsw               2532           # 0.87%\ninblock              48             # 0.05\/sec\nonblock              16584          # 16.25\/sec\ncpu-clock            1001082661388  # 1001.083 seconds\ntask-clock           1002538150096  # 1002.538 seconds\npage faults          196593         # 196.095\/sec\ncontext switches     7091548        # 7073.594\/sec\ncpu migrations       518            # 0.517\/sec\nmajor page faults    2              # 0.002\/sec\nminor page faults    196591         # 196.093\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             741624850927   # 203.483 branches per 1000 inst\nbranch misses        96937340773    # 13.07% branch miss\nconditional          335512707016   # 92.056 conditional branches per 1000 inst\nindirect             8794435030     # 2.413 indirect branches per 1000 inst\ncpu-cycles           3938011140824  # 0.27 GHz\ninstructions         3335477922804  # 0.85 IPC\nslots                7849808157270  #\nretiring             1395394811656  # 17.8% (17.8%)\n-- ucode             10805555686    #     0.1%\n-- fastpath          1384589255970  #    17.6%\nfrontend             5233357070230  # 66.7% (66.7%) high\n-- latency           4401810815286  #    56.1%\n-- bandwidth         831546254944   #    10.6%\nbackend              1170512139286  # 14.9% (14.9%) low\n-- cpu               308721940644   #     3.9%\n-- memory            861790198642   #    11.0%\nspeculation          49986128184    #  0.6% ( 0.6%) low\n-- branch mispredict 49821138730    #     0.6%\n-- pipeline restart  164989454      #     0.0%\nsmt-contention       556982258      #  0.0% ( 0.0%)\ncpu-cycles           5573850138090  # 0.29 GHz\ninstructions         4746679838611  # 0.85 IPC\ninstructions         1578587483573  # 45.278 l2 access per 1000 inst\nl2 hit from l1       49877915552    # 26.93% l2 miss\nl2 miss from l1      11601954748    #\nl2 hit from l2 pf    13953015145    #\nl3 hit from l2 pf    6995462328     #\nl3 miss from l2 pf   649138092      #\ninstructions         1578938514540  # 11.084 float per 1000 inst\nfloat 512            180            # 0.000 AVX-512 per 1000 inst\nfloat 256            604            # 0.000 AVX-256 per 1000 inst\nfloat 128            17500402570    # 11.084 AVX-128 per 1000 inst\nfloat MMX            0              # 0.000 MMX per 1000 inst\nfloat scalar         0              # 0.000 scalar per 1000 inst\ninstructions         3387121496957  #\nopcache              1290539350027  # 381.014 opcache per 1000 inst\nopcache miss         694249272052   # 53.8% opcache miss rate\nl1 dTLB miss         571148055      # 0.169 L1 dTLB per 1000 inst\nl2 dTLB miss         24800609       # 0.007 L2 dTLB per 1000 inst\ninstructions         3652008286708  #\nicache               1811405175336  # 496.002 icache per 1000 inst\nicache miss          33218736875    #  1.8% icache miss rate\nl1 iTLB miss         8661876        # 0.002 L1 iTLB per 1000 inst\nl2 iTLB miss         0              # 0.000 L2 iTLB per 1000 inst\ntlb flush            27911          # 0.000 TLB flush per 1000 inst\n<\/code><\/pre>\n\n\n\n<p>Intel metrics show L3 as taking the largest amount of memory and surprisingly a lot more backend bound than AMD.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              1411.002\non_cpu               0.030          # 0.48 \/ 16 cores\nutime                46.769\nstime                630.786\nnvcsw                521847         # 99.48%\nnivcsw               2734           # 0.52%\ninblock              536            # 0.38\/sec\nonblock              6192           # 4.39\/sec\ncpu-clock            1351750378635  # 1351.750 seconds\ntask-clock           1351993088667  # 1351.993 seconds\npage faults          211379         # 156.346\/sec\ncontext switches     2150650        # 1590.726\/sec\ncpu migrations       1780           # 1.317\/sec\nmajor page faults    5              # 0.004\/sec\nminor page faults    211374         # 156.343\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             854112014822   # 164.960 branches per 1000 inst\nbranch misses        2537777092     # 0.30% branch miss\nconditional          854112041478   # 164.960 conditional branches per 1000 inst\nindirect             13809268745    # 2.667 indirect branches per 1000 inst\nslots                31318484719550 #\nretiring             7356339156164  # 23.5% (23.5%)\n-- ucode             1450912164180  #     4.6%\n-- fastpath          5905426991984  #    18.9%\nfrontend             5359358888516  # 17.1% (17.1%)\n-- latency           2698193464442  #     8.6%\n-- bandwidth         2661165424074  #     8.5%\nbackend              18030808250048 # 57.6% (57.6%)\n-- cpu               6672071938072  #    21.3%\n-- memory            11358736311976 #    36.3%\nspeculation          587508667689   #  1.9% ( 1.9%)\n-- branch mispredict 445657118471   #     1.4%\n-- pipeline restart  141851549218   #     0.5%\nsmt-contention       0              #  0.0% ( 0.0%)\ncpu-cycles           6457245889546  # 0.23 GHz\ninstructions         7275449365505  # 1.13 IPC\nl2 access            143093585864   # 19.677 l2 access per 1000 inst\nl2 miss              85516676526    # 59.76% l2 miss\ncpu-cycles           6129827611475  # 40.2% memory latency\nload stalls          2378116952695  #  7.6% l1 bound\nl1 miss              1910108431750  #  7.2% l2 bound\nl2 miss              1468707911024  # 23.9% l3 bound\nl3 miss              1043305120     #  0.0% dram bound\nstore_stalls         84585107510    #  1.4% store bound\n<\/code><\/pre>\n\n\n\n<p>Process summary shows different programs for processes<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>688 processes\n\t 64 pipe                    27.56   274.22\n\t 60 fifo                    26.79   263.40\n\t 36 tcp                     20.19   245.40\n\t 36 socketpair              19.53   326.57\n\t 68 clinfo                  17.53     5.31\n\t 38 vulkaninfo               0.96     1.53\n\t  6 php                      0.18     0.56\n\t  4 vulkani:disk$0           0.10     0.17\n\t  6 clang                    0.07     0.05\n\t  6 glxinfo:gdrv0            0.05     0.13\n\t  6 glxinfo:gl0              0.05     0.13\n\t  2 llvmpipe-0               0.05     0.09\n\t  2 llvmpipe-1               0.05     0.09\n\t  2 llvmpipe-10              0.05     0.09\n\t  2 llvmpipe-11              0.05     0.09\n\t  2 llvmpipe-12              0.05     0.09\n\t  2 llvmpipe-13              0.05     0.09\n\t  2 llvmpipe-14              0.05     0.09\n\t  2 llvmpipe-2               0.05     0.09\n\t  2 llvmpipe-3               0.05     0.09\n\t  2 llvmpipe-4               0.05     0.09\n\t  2 llvmpipe-5               0.05     0.09\n\t  2 llvmpipe-6               0.05     0.09\n\t  2 llvmpipe-7               0.05     0.09\n\t  2 llvmpipe-9               0.05     0.09\n\t  2 llvmpipe-15              0.05     0.08\n\t  2 llvmpipe-8               0.05     0.08\n\t  2 glxinfo                  0.03     0.05\n\t  2 glxinfo:cs0              0.03     0.05\n\t  2 glxinfo:disk$0           0.03     0.05\n\t  2 glxinfo:sh0              0.03     0.05\n\t  2 glxinfo:shlo0            0.03     0.05\n\t  3 rocminfo                 0.03     0.00\n\t  1 lspci                    0.00     0.02\n\t128 sh                       0.00     0.00\n\t 98 ipc-benchmark            0.00     0.00\n\t 13 gcc                      0.00     0.00\n\t 10 gsettings                0.00     0.00\n\t  8 stat                     0.00     0.00\n\t  8 systemd-detect-          0.00     0.00\n\t  6 llvm-link                0.00     0.00\n\t  5 phoronix-test-s          0.00     0.00\n\t  3 dconf worker             0.00     0.00\n\t  3 gmain                    0.00     0.00\n\t  2 cc                       0.00     0.00\n\t  2 lscpu                    0.00     0.00\n\t  2 uname                    0.00     0.00\n\t  2 which                    0.00     0.00\n\t  2 xset                     0.00     0.00\n\t  1 date                     0.00     0.00\n\t  1 dirname                  0.00     0.00\n\t  1 dmesg                    0.00     0.00\n\t  1 dmidecode                0.00     0.00\n\t  1 grep                     0.00     0.00\n\t  1 ifconfig                 0.00     0.00\n\t  1 ip                       0.00     0.00\n\t  1 lsmod                    0.00     0.00\n\t  1 mktemp                   0.00     0.00\n\t  1 ps                       0.00     0.00\n\t  1 qdbus                    0.00     0.00\n\t  1 readlink                 0.00     0.00\n\t  1 realpath                 0.00     0.00\n\t  1 sed                      0.00     0.00\n\t  1 sort                     0.00     0.00\n\t  1 stty                     0.00     0.00\n\t  1 systemctl                0.00     0.00\n\t  1 template.sh              0.00     0.00\n\t  1 wc                       0.00     0.00\n\t  1 xrandr                   0.00     0.00\n0 processes running\n47 maximum processes\n<\/code><\/pre>\n\n\n\n<p>An example computation block<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>      829619) ipc-benchmark    cpu=4 start=26.57 finish=32.98\n        829620) tcp              cpu=5 start=26.57 finish=32.98\n          829621) tcp              cpu=14 start=26.57 finish=32.98\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Testing an interprocess communication primitives. There are 24 different subtests including Topdown summary shows mostly two processes running but not spending much time and also a moderate level of interrupts. For example the initial section is TCP sockets with increasing <span class=\"excerpt-dots\">&hellip;<\/span> <a class=\"more-link\" href=\"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/ipc-benchmark\/\"><span class=\"more-msg\">Continue reading &rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":58,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1889","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1889","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/comments?post=1889"}],"version-history":[{"count":2,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1889\/revisions"}],"predecessor-version":[{"id":1901,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1889\/revisions\/1901"}],"up":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/58"}],"wp:attachment":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/media?parent=1889"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}