{"id":1042,"date":"2024-01-28T23:52:56","date_gmt":"2024-01-28T23:52:56","guid":{"rendered":"https:\/\/mvermeulen.org\/perf\/?page_id=1042"},"modified":"2024-01-28T23:52:56","modified_gmt":"2024-01-28T23:52:56","slug":"ncnn-2","status":"publish","type":"page","link":"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/ncnn-2\/","title":{"rendered":"ncnn"},"content":{"rendered":"\n<p>ncnn is a neural network framework from Tencent. This test works through a dozen different neural networks first on CPU and then on Vulkan GPU. Each individual case runs quickly so that ~60 second period only runs on average a neural net for five seconds before going to the next one. It looks like we are taking advantage of all cores.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/systemtime-77.png\" alt=\"\" class=\"wp-image-1043\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/systemtime-77.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/systemtime-77-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/systemtime-77-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Topdown profile shows on average we are dominated by backend stalls, except for a few of the last network being slightly less so. The CPU and GPU profiles are similar, so perhaps my &#8220;Vulkan GPU&#8221; tests really are CPU tests as well.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/amdtopdown-114.png\" alt=\"\" class=\"wp-image-1044\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/amdtopdown-114.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/amdtopdown-114-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/amdtopdown-114-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>AMD metrics show a workload high in backend stalls and low in both retiring rate and frontend stalls. There are few floating point instructions (using int8?).<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              1946.111\non_cpu               0.924          # 14.78 \/ 16 cores\nutime                28665.305\nstime                97.446\nnvcsw                1458053        # 82.31%\nnivcsw               313322         # 17.69%\ninblock              0              # 0.00\/sec\nonblock              17848          # 9.17\/sec\ncpu-clock            28772710292348 # 28772.710 seconds\ntask-clock           28773501421019 # 28773.501 seconds\npage faults          17841468       # 620.066\/sec\ncontext switches     1780869        # 61.893\/sec\ncpu migrations       715            # 0.025\/sec\nmajor page faults    2079           # 0.072\/sec\nminor page faults    17839389       # 619.994\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             7400706707040  # 129.148 branches per 1000 inst\nbranch misses        22112539866    # 0.30% branch miss\nconditional          7306720045682  # 127.508 conditional branches per 1000 inst\nindirect             9788450464     # 0.171 indirect branches per 1000 inst\ncpu-cycles           24435176519386 # 3.85 GHz\ninstructions         11449051374995 # 0.47 IPC low\nslots                48868759359042 #\nretiring             4017392153178  #  8.2% (11.1%) low\n-- ucode             52206341384    #     0.1%\n-- fastpath          3965185811794  #     8.1%\nfrontend             1521688467294  #  3.1% ( 4.2%) low\n-- latency           584270471004   #     1.2%\n-- bandwidth         937417996290   #     1.9%\nbackend              30426520761785 # 62.3% (84.4%) high\n-- cpu               15311691089077 #    31.3%\n-- memory            15114829672708 #    30.9%\nspeculation          83894665674    #  0.2% ( 0.2%) low\n-- branch mispredict 64506051329    #     0.1%\n-- pipeline restart  19388614345    #     0.0%\nsmt-contention       12819183408457 # 26.2% ( 0.0%)\ncpu-cycles           24319189058245 # 3.85 GHz\ninstructions         11439573868940 # 0.47 IPC low\ninstructions         3817148970093  # 129.638 l2 access per 1000 inst\nl2 hit from l1       313108322322   # 28.43% l2 miss\nl2 miss from l1      15187002493    #\nl2 hit from l2 pf    56261875456    #\nl3 hit from l2 pf    109448230684   #\nl3 miss from l2 pf   16030874460    #\ninstructions         3814302904658  # 15.070 float per 1000 inst\nfloat 512            62             # 0.000 AVX-512 per 1000 inst\nfloat 256            8826           # 0.000 AVX-256 per 1000 inst\nfloat 128            57480143661    # 15.070 AVX-128 per 1000 inst\nfloat MMX            0              # 0.000 MMX per 1000 inst\nfloat scalar         0              # 0.000 scalar per 1000 inst\n<\/code><\/pre>\n\n\n\n<p>Intel metrics<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              2809.052\non_cpu               0.949          # 15.18 \/ 16 cores\nutime                42567.071\nstime                76.882\nnvcsw                2200133        # 85.22%\nnivcsw               381476         # 14.78%\ninblock              8432           # 3.00\/sec\nonblock              4912           # 1.75\/sec\ncpu-clock            42647720312779 # 42647.720 seconds\ntask-clock           42648149083383 # 42648.149 seconds\npage faults          18818807       # 441.257\/sec\ncontext switches     2595429        # 60.857\/sec\ncpu migrations       61584          # 1.444\/sec\nmajor page faults    1942           # 0.046\/sec\nminor page faults    18816865       # 441.212\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             4804748523665  # 49.759 branches per 1000 inst\nbranch misses        22716229759    # 0.47% branch miss\nconditional          4804748550129  # 49.759 conditional branches per 1000 inst\nindirect             2009633101318  # 20.812 indirect branches per 1000 inst\nslots                235806293789726 #\nretiring             48622809170505 # 20.6% (20.6%)\n-- ucode             3412287001594  #     1.4%\n-- fastpath          45210522168911 #    19.2%\nfrontend             17158999336433 #  7.3% ( 7.3%)\n-- latency           13837393028606 #     5.9%\n-- bandwidth         3321606307827  #     1.4%\nbackend              168582773381581 # 71.5% (71.5%) high\n-- cpu               124294100887241 #    52.7%\n-- memory            44288672494340 #    18.8%\nspeculation          1609099917873  #  0.7% ( 0.7%) low\n-- branch mispredict 1470207468290  #     0.6%\n-- pipeline restart  138892449583   #     0.1%\nsmt-contention       0              #  0.0% ( 0.0%)\ncpu-cycles           90957267456764 # 2.96 GHz\ninstructions         63982684727203 # 0.70 IPC\nl2 access            2072680943817  # 59.690 l2 access per 1000 inst\nl2 miss              855274244675   # 41.26% l2 miss\n<\/code><\/pre>\n\n\n\n<p>Process overview is simile with 96 benchcnn processes.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>452 processes\n\t 96 benchncnn            91381.12   284.48\n\t 68 clinfo                  16.28     6.65\n\t 38 vulkaninfo               0.95     1.34\n\t  6 glxinfo:gdrv0            0.15     0.00\n\t  6 glxinfo:gl0              0.15     0.00\n\t  6 php                      0.14     0.08\n\t  4 vulkani:disk$0           0.10     0.15\n\t  2 glxinfo                  0.08     0.00\n\t  2 glxinfo:cs0              0.08     0.00\n\t  2 glxinfo:disk$0           0.08     0.00\n\t  2 glxinfo:sh0              0.08     0.00\n\t  2 glxinfo:shlo0            0.08     0.00\n\t  6 clang                    0.06     0.06\n\t  2 llvmpipe-0               0.05     0.08\n\t  2 llvmpipe-10              0.05     0.08\n\t  2 llvmpipe-11              0.05     0.08\n\t  2 llvmpipe-12              0.05     0.08\n\t  2 llvmpipe-13              0.05     0.08\n\t  2 llvmpipe-14              0.05     0.08\n\t  2 llvmpipe-15              0.05     0.08\n\t  2 llvmpipe-2               0.05     0.08\n\t  2 llvmpipe-3               0.05     0.08\n\t  2 llvmpipe-4               0.05     0.08\n\t  2 llvmpipe-5               0.05     0.08\n\t  2 llvmpipe-6               0.05     0.08\n\t  2 llvmpipe-7               0.05     0.08\n\t  2 llvmpipe-8               0.05     0.08\n\t  2 llvmpipe-9               0.05     0.08\n\t  2 llvmpipe-1               0.05     0.07\n\t  3 rocminfo                 0.03     0.00\n\t  1 lspci                    0.00     0.02\n\t  1 ps                       0.00     0.01\n\t 84 sh                       0.00     0.00\n\t 13 gcc                      0.00     0.00\n\t  9 gsettings                0.00     0.00\n\t  8 stat                     0.00     0.00\n\t  8 systemd-detect-          0.00     0.00\n\t  6 llvm-link                0.00     0.00\n\t  6 ncnn                     0.00     0.00\n\t  5 phoronix-test-s          0.00     0.00\n\t  4 gmain                    0.00     0.00\n\t  3 dconf worker             0.00     0.00\n\t  2 cc                       0.00     0.00\n\t  2 lscpu                    0.00     0.00\n\t  2 uname                    0.00     0.00\n\t  2 which                    0.00     0.00\n\t  2 xset                     0.00     0.00\n\t  1 date                     0.00     0.00\n\t  1 dirname                  0.00     0.00\n\t  1 dmesg                    0.00     0.00\n\t  1 dmidecode                0.00     0.00\n\t  1 grep                     0.00     0.00\n\t  1 ifconfig                 0.00     0.00\n\t  1 ip                       0.00     0.00\n\t  1 lsmod                    0.00     0.00\n\t  1 mktemp                   0.00     0.00\n\t  1 qdbus                    0.00     0.00\n\t  1 readlink                 0.00     0.00\n\t  1 realpath                 0.00     0.00\n\t  1 sed                      0.00     0.00\n\t  1 sort                     0.00     0.00\n\t  1 stty                     0.00     0.00\n\t  1 systemctl                0.00     0.00\n\t  1 template.sh              0.00     0.00\n\t  1 wc                       0.00     0.00\n\t  1 xrandr                   0.00     0.00\n0 processes running\n47 maximum processes\n<\/code><\/pre>\n\n\n\n<p>The core computation blocks<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>      1129043) ncnn             cpu=12 start=5.50  finish=66.00\n        1129044) benchncnn        cpu=6 start=5.50  finish=66.00\n          1129045) benchncnn        cpu=0 start=5.51  finish=66.00\n          1129046) benchncnn        cpu=11 start=5.51  finish=66.00\n          1129047) benchncnn        cpu=9 start=5.51  finish=66.00\n          1129048) benchncnn        cpu=10 start=5.51  finish=66.00\n          1129049) benchncnn        cpu=15 start=5.51  finish=66.00\n          1129050) benchncnn        cpu=14 start=5.51  finish=66.00\n          1129051) benchncnn        cpu=4 start=5.51  finish=66.00\n          1129052) benchncnn        cpu=13 start=5.51  finish=66.00\n          1129053) benchncnn        cpu=7 start=5.51  finish=66.00\n          1129054) benchncnn        cpu=8 start=5.51  finish=66.00\n          1129055) benchncnn        cpu=3 start=5.51  finish=66.00\n          1129056) benchncnn        cpu=1 start=5.51  finish=66.00\n          1129057) benchncnn        cpu=2 start=5.51  finish=66.00\n          1129058) benchncnn        cpu=12 start=5.51  finish=66.00\n          1129059) benchncnn        cpu=5 start=5.51  finish=66.00\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>ncnn is a neural network framework from Tencent. This test works through a dozen different neural networks first on CPU and then on Vulkan GPU. Each individual case runs quickly so that ~60 second period only runs on average a <span class=\"excerpt-dots\">&hellip;<\/span> <a class=\"more-link\" href=\"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/ncnn-2\/\"><span class=\"more-msg\">Continue reading &rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":58,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1042","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1042","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/comments?post=1042"}],"version-history":[{"count":1,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1042\/revisions"}],"predecessor-version":[{"id":1045,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1042\/revisions\/1045"}],"up":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/58"}],"wp:attachment":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/media?parent=1042"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}