{"id":1401,"date":"2024-02-03T20:51:50","date_gmt":"2024-02-03T20:51:50","guid":{"rendered":"https:\/\/mvermeulen.org\/perf\/?page_id=1401"},"modified":"2024-02-03T22:41:39","modified_gmt":"2024-02-03T22:41:39","slug":"amg","status":"publish","type":"page","link":"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/amg\/","title":{"rendered":"amg"},"content":{"rendered":"\n<p>A parallel multigrid solver for linear systems. This is a quick running benchmark with one workload. It runs in parallel.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-22.png\" alt=\"\" class=\"wp-image-1421\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-22.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-22-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-22-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Topdown profile shows backend stalls are high and retirement is low.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-22.png\" alt=\"\" class=\"wp-image-1423\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-22.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-22-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-22-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>AMD metrics show a moderate floating point and some L2 misses.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              65.367\non_cpu               0.683          # 10.93 \/ 16 cores\nutime                684.184\nstime                30.516\nnvcsw                34261          # 78.49%\nnivcsw               9387           # 21.51%\ninblock              0              # 0.00\/sec\nonblock              62128          # 950.45\/sec\ncpu-clock            715137381360   # 715.137 seconds\ntask-clock           715190259831   # 715.190 seconds\npage faults          8807369        # 12314.722\/sec\ncontext switches     43781          # 61.216\/sec\ncpu migrations       849            # 1.187\/sec\nmajor page faults    239            # 0.334\/sec\nminor page faults    8807130        # 12314.388\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             169782624025   # 175.891 branches per 1000 inst\nbranch misses        4319878460     # 2.54% branch miss\nconditional          156304404289   # 161.928 conditional branches per 1000 inst\nindirect             666734996      # 0.691 indirect branches per 1000 inst\ncpu-cycles           3172142083659  # 3.01 GHz\ninstructions         963009042257   # 0.30 IPC low\nslots                6345218376984  #\nretiring             296090878240   #  4.7% ( 5.7%) low\n-- ucode             539766775      #     0.0%\n-- fastpath          295551111465   #     4.7%\nfrontend             884404390044   # 13.9% (17.1%)\n-- latency           689529138882   #    10.9%\n-- bandwidth         194875251162   #     3.1%\nbackend              3967267557245  # 62.5% (76.5%) high\n-- cpu               339701703779   #     5.4%\n-- memory            3627565853466  #    57.2%\nspeculation          37459376987    #  0.6% ( 0.7%) low\n-- branch mispredict 37031973884    #     0.6%\n-- pipeline restart  427403103      #     0.0%\nsmt-contention       1159992696623  # 18.3% ( 0.0%)\ncpu-cycles           3171306945071  # 3.02 GHz\ninstructions         970302048523   # 0.31 IPC low\ninstructions         323788798692   # 61.769 l2 access per 1000 inst\nl2 hit from l1       12459651257    # 38.69% l2 miss\nl2 miss from l1      1594283255     #\nl2 hit from l2 pf    1396247525     #\nl3 hit from l2 pf    82285311       #\nl3 miss from l2 pf   6061785869     #\ninstructions         322826762239   # 117.391 float per 1000 inst\nfloat 512            66             # 0.000 AVX-512 per 1000 inst\nfloat 256            648            # 0.000 AVX-256 per 1000 inst\nfloat 128            37896810350    # 117.391 AVX-128 per 1000 inst\nfloat MMX            0              # 0.000 MMX per 1000 inst\nfloat scalar         0              # 0.000 scalar per 1000 inst\ninstructions         2686030        #\nopcache              1004915        # 374.126 opcache per 1000 inst\nopcache miss         540008         # 53.7% opcache miss rate\nl1 dTLB miss         6463           # 2.406 L1 dTLB per 1000 inst\nl2 dTLB miss         1151           # 0.429 L2 dTLB per 1000 inst\ninstructions         2696784        #\nicache               1305399        # 484.058 icache per 1000 inst\nicache miss          107358         #  8.2% icache miss rate\nl1 iTLB miss         8              # 0.003 L1 iTLB per 1000 inst\nl2 iTLB miss         0              # 0.000 L2 iTLB per 1000 inst\ntlb flush            19             # 0.007 TLB flush per 1000 inst\n<\/code><\/pre>\n\n\n\n<p>Intel metrics show backend CPU stalls and also dram memory stalls.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              130.375\non_cpu               0.815          # 13.04 \/ 16 cores\nutime                1661.904\nstime                37.999\nnvcsw                78342          # 60.06%\nnivcsw               52092          # 39.94%\ninblock              30352          # 232.81\/sec\nonblock              50968          # 390.93\/sec\ncpu-clock            1700050318910  # 1700.050 seconds\ntask-clock           1700071060304  # 1700.071 seconds\npage faults          13107552       # 7710.002\/sec\ncontext switches     130884         # 76.987\/sec\ncpu migrations       19282          # 11.342\/sec\nmajor page faults    879            # 0.517\/sec\nminor page faults    13106673       # 7709.485\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             1266941103776  # 184.143 branches per 1000 inst\nbranch misses        7059189438     # 0.56% branch miss\nconditional          1266941120608  # 184.143 conditional branches per 1000 inst\nindirect             449036973724   # 65.265 indirect branches per 1000 inst\nslots                8910379799354  #\nretiring             3624326751784  # 40.7% (40.7%)\n-- ucode             373643885674   #     4.2%\n-- fastpath          3250682866110  #    36.5%\nfrontend             1015307188635  # 11.4% (11.4%)\n-- latency           629549610931   #     7.1%\n-- bandwidth         385757577704   #     4.3%\nbackend              4111358559413  # 46.1% (46.1%)\n-- cpu               2398629919885  #    26.9%\n-- memory            1712728639528  #    19.2%\nspeculation          446666376364   #  5.0% ( 5.0%)\n-- branch mispredict 399480325933   #     4.5%\n-- pipeline restart  47186050431    #     0.5%\nsmt-contention       0              #  0.0% ( 0.0%)\ncpu-cycles           5541763745828  # 2.68 GHz\ninstructions         7359688889580  # 1.33 IPC\nl2 access            41605258580    # 11.616 l2 access per 1000 inst\nl2 miss              23999846272    # 57.68% l2 miss\ncpu-cycles           2695525220705  # 27.5% memory latency\nload stalls          728696875722   #  1.4% l1 bound\nl1 miss              691796917416   #  4.9% l2 bound\nl2 miss              560621251095   #  5.2% l3 bound\nl3 miss              419917879058   # 15.6% dram bound\nstore_stalls         12075917152    #  0.4% store bound\n<\/code><\/pre>\n\n\n\n<p>Process overview shows the amg process accumulating the overall time.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>465 processes\n\t 99 amg                   2730.94   114.56\n\t 68 clinfo                  16.86     5.98\n\t 38 vulkaninfo               0.75     1.34\n\t 18 mpirun                   0.71     2.20\n\t  4 vulkani:disk$0           0.08     0.14\n\t  6 clang                    0.08     0.04\n\t  6 php                      0.07     0.06\n\t  6 glxinfo:gdrv0            0.05     0.08\n\t  6 glxinfo:gl0              0.05     0.08\n\t  2 llvmpipe-0               0.04     0.07\n\t  2 llvmpipe-1               0.04     0.07\n\t  2 llvmpipe-10              0.04     0.07\n\t  2 llvmpipe-11              0.04     0.07\n\t  2 llvmpipe-12              0.04     0.07\n\t  2 llvmpipe-13              0.04     0.07\n\t  2 llvmpipe-14              0.04     0.07\n\t  2 llvmpipe-15              0.04     0.07\n\t  2 llvmpipe-2               0.04     0.07\n\t  2 llvmpipe-3               0.04     0.07\n\t  2 llvmpipe-4               0.04     0.07\n\t  2 llvmpipe-5               0.04     0.07\n\t  2 llvmpipe-6               0.04     0.07\n\t  2 llvmpipe-7               0.04     0.07\n\t  2 llvmpipe-8               0.04     0.07\n\t  2 llvmpipe-9               0.04     0.07\n\t  2 glxinfo                  0.03     0.04\n\t  2 glxinfo:cs0              0.03     0.04\n\t  2 glxinfo:disk$0           0.03     0.04\n\t  2 glxinfo:sh0              0.03     0.04\n\t  2 glxinfo:shlo0            0.03     0.04\n\t  3 rocminfo                 0.00     0.03\n\t  1 lspci                    0.00     0.01\n\t 82 sh                       0.00     0.00\n\t 13 gcc                      0.00     0.00\n\t 10 gsettings                0.00     0.00\n\t  8 stat                     0.00     0.00\n\t  8 systemd-detect-          0.00     0.00\n\t  6 llvm-link                0.00     0.00\n\t  5 phoronix-test-s          0.00     0.00\n\t  3 dconf worker             0.00     0.00\n\t  3 gmain                    0.00     0.00\n\t  2 cc                       0.00     0.00\n\t  2 lscpu                    0.00     0.00\n\t  2 uname                    0.00     0.00\n\t  2 which                    0.00     0.00\n\t  2 xset                     0.00     0.00\n\t  1 date                     0.00     0.00\n\t  1 dirname                  0.00     0.00\n\t  1 dmesg                    0.00     0.00\n\t  1 dmidecode                0.00     0.00\n\t  1 grep                     0.00     0.00\n\t  1 ifconfig                 0.00     0.00\n\t  1 ip                       0.00     0.00\n\t  1 lsmod                    0.00     0.00\n\t  1 mktemp                   0.00     0.00\n\t  1 ps                       0.00     0.00\n\t  1 qdbus                    0.00     0.00\n\t  1 readlink                 0.00     0.00\n\t  1 realpath                 0.00     0.00\n\t  1 sed                      0.00     0.00\n\t  1 sort                     0.00     0.00\n\t  1 stty                     0.00     0.00\n\t  1 systemctl                0.00     0.00\n\t  1 template.sh              0.00     0.00\n\t  1 wc                       0.00     0.00\n\t  1 xrandr                   0.00     0.00\n0 processes running\n47 maximum processes\n<\/code><\/pre>\n\n\n\n<p>Computation structure shows this as a MPI program<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>      354246) amg              cpu=5 start=5.51  finish=22.04\n        354247) mpirun           cpu=1 start=5.51  finish=22.02\n          354250) mpirun           cpu=15 start=6.09  finish=22.02\n          354251) mpirun           cpu=8 start=6.09  finish=6.09 \n          354252) mpirun           cpu=9 start=6.11  finish=22.02\n          354253) mpirun           cpu=2 start=6.59  finish=22.02\n          354254) mpirun           cpu=2 start=6.60  finish=22.02\n          354255) amg              cpu=3 start=6.63  finish=22.00\n            354257) amg              cpu=7 start=6.64  finish=21.99\n            354260) amg              cpu=10 start=6.64  finish=21.99\n            354285) amg              cpu=10 start=6.88  finish=22.00\n          354256) amg              cpu=7 start=6.63  finish=22.00\n            354259) amg              cpu=14 start=6.64  finish=21.99\n            354262) amg              cpu=4 start=6.65  finish=21.99\n            354283) amg              cpu=11 start=6.88  finish=22.00\n          354258) amg              cpu=6 start=6.64  finish=22.00\n            354263) amg              cpu=14 start=6.65  finish=21.99\n            354266) amg              cpu=9 start=6.65  finish=21.99\n            354279) amg              cpu=14 start=6.88  finish=22.00\n          354261) amg              cpu=13 start=6.64  finish=21.99\n            354265) amg              cpu=10 start=6.65  finish=21.99\n            354269) amg              cpu=5 start=6.66  finish=21.99\n            354282) amg              cpu=4 start=6.88  finish=21.99\n          354264) amg              cpu=12 start=6.65  finish=21.99\n            354268) amg              cpu=9 start=6.66  finish=21.99\n            354272) amg              cpu=11 start=6.66  finish=21.99\n            354286) amg              cpu=11 start=6.88  finish=21.99\n          354267) amg              cpu=1 start=6.65  finish=21.99\n            354271) amg              cpu=5 start=6.66  finish=21.99\n            354275) amg              cpu=14 start=6.67  finish=21.99\n            354284) amg              cpu=9 start=6.88  finish=21.99\n          354270) amg              cpu=2 start=6.66  finish=21.99\n            354274) amg              cpu=12 start=6.66  finish=21.99\n            354277) amg              cpu=13 start=6.67  finish=21.99\n            354281) amg              cpu=5 start=6.88  finish=22.00\n          354273) amg              cpu=0 start=6.66  finish=21.99\n            354276) amg              cpu=10 start=6.67  finish=21.99\n            354278) amg              cpu=15 start=6.67  finish=21.99\n            354280) amg              cpu=10 start=6.88  finish=21.99\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>A parallel multigrid solver for linear systems. This is a quick running benchmark with one workload. It runs in parallel. Topdown profile shows backend stalls are high and retirement is low. AMD metrics show a moderate floating point and some <span class=\"excerpt-dots\">&hellip;<\/span> <a class=\"more-link\" href=\"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/amg\/\"><span class=\"more-msg\">Continue reading &rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":58,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1401","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1401","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/comments?post=1401"}],"version-history":[{"count":5,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1401\/revisions"}],"predecessor-version":[{"id":1438,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1401\/revisions\/1438"}],"up":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/58"}],"wp:attachment":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/media?parent=1401"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}