{"id":177,"date":"2024-01-02T14:35:51","date_gmt":"2024-01-02T14:35:51","guid":{"rendered":"https:\/\/mvermeulen.org\/perf\/?page_id=177"},"modified":"2024-01-02T14:36:27","modified_gmt":"2024-01-02T14:36:27","slug":"pytorch","status":"publish","type":"page","link":"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/pytorch\/","title":{"rendered":"pytorch"},"content":{"rendered":"\n<p>Measurements on 64-bit resnet 50 model<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/amdtopdown-5.png\" alt=\"\" class=\"wp-image-179\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/amdtopdown-5.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/amdtopdown-5-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/01\/amdtopdown-5-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>AMD metrics show a backend dominated benchmark with both memory and cpu contributing also showing both floating point and l2.  It seemed to take multiple iterations to get to a converged result, contributing to higher elapsed time.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              888.295\non_cpu               0.453          # 7.25 \/ 16 cores\nutime                6368.016\nstime                73.376\nnvcsw                6468           # 5.12%\nnivcsw               119800         # 94.88%\ninblock              205464\nonblock              6536\ncpu-clock            6441494326501  # 6441.494 seconds\ntask-clock           6441511010764  # 6441.511 seconds\npage faults          28365221       # 4403.504\/sec\ncontext switches     130370         # 20.239\/sec\ncpu migrations       6329           # 0.983\/sec\nmajor page faults    1083           # 0.168\/sec\nminor page faults    28364137       # 4403.336\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             1019213674222  # 47.300 branches per 1000 inst\nbranch misses        25970276858    # 2.55% branch miss\nconditional          863766043503   # 40.086 conditional branches per 1000 inst\nindirect             29448001923    # 1.367 indirect branches per 1000 inst\ncpu-cycles           6626976423794  # 2.01 GHz\ninstructions         4933090739402  # 0.74 IPC\nslots                13246150286592 #\nretiring             1704776212319  # 12.9% (13.6%)\n-- ucode             9812416382     #     0.1%\n-- fastpath          1694963795937  #    12.8%\nfrontend             763514723827   #  5.8% ( 6.1%)\n-- latency           428385176334   #     3.2%\n-- bandwidth         335129547493   #     2.5%\nbackend              9833869333399  # 74.2% (78.6%)\n-- cpu               5375257224574  #    40.6%\n-- memory            4458612108825  #    33.7%\nspeculation          201506860099   #  1.5% ( 1.6%)\n-- branch mispredict 188020274940   #     1.4%\n-- pipeline restart  13486585159    #     0.1%\nsmt-contention       741610871065   #  5.6% ( 0.0%)\ncpu-cycles           17451458172063 # 2.02 GHz\ninstructions         13133820486750 # 0.75 IPC\ninstructions         4385559673728  # 102.068 l2 access per 1000 inst\nl2 hit from l1       330912826682   # 19.66% l2 miss\nl2 miss from l1      34680239069    #\nl2 hit from l2 pf    63389477231    #\nl3 hit from l2 pf    41640256311    #\nl3 miss from l2 pf   11684225833    #\ninstructions         4372833455503  # 42.240 float per 1000 inst\nfloat 512            74             # 0.000 AVX-512 per 1000 inst\nfloat 256            5854           # 0.000 AVX-256 per 1000 inst\nfloat 128            184710215223   # 42.240 AVX-128 per 1000 inst\nfloat MMX            0              # 0.000 MMX per 1000 inst\nfloat scalar         192            # 0.000 scalar per 1000 inst<\/code><\/pre>\n\n\n\n<p>Intel metrics<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              345.548\non_cpu               0.703          # 11.25 \/ 16 cores\nutime                3845.621\nstime                42.453\nnvcsw                1954           # 22.18%\nnivcsw               6857           # 77.82%\ninblock              33480\nonblock              2536\ncpu-clock            3888106007754  # 3888.106 seconds\ntask-clock           3888114503534  # 3888.115 seconds\npage faults          17596747       # 4525.779\/sec\ncontext switches     10341          # 2.660\/sec\ncpu migrations       2785           # 0.716\/sec\nmajor page faults    306            # 0.079\/sec\nminor page faults    17596441       # 4525.700\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             346992734665   # 33.484 branches per 1000 inst\nbranch misses        5921009494     # 1.71% branch miss\nconditional          346992747081   # 33.484 conditional branches per 1000 inst\nindirect             122858659034   # 11.856 indirect branches per 1000 inst\nslots                122967048504506 #\nretiring             19592997128790 # 15.9% (15.9%)\n-- ucode             1405051710420  #     1.1%\n-- fastpath          18187945418370 #    14.8%\nfrontend             3831999650714  #  3.1% ( 3.1%)\n-- latency           2414419492941  #     2.0%\n-- bandwidth         1417580157773  #     1.2%\nbackend              98025673596023 # 79.7% (79.7%)\n-- cpu               83254741022304 #    67.7%\n-- memory            14770932573719 #    12.0%\nspeculation          2625657274843  #  2.1% ( 2.1%)\n-- branch mispredict 2298959158532  #     1.9%\n-- pipeline restart  326698116311   #     0.3%\nsmt-contention       0              #  0.0% ( 0.0%)\ncpu-cycles           5989513323133  # 1.11 GHz\ninstructions         5532177304949  # 0.92 IPC\nl2 access            180309976145   # 48.391 l2 access per 1000 inst\nl2 miss              62476631641    # 34.65% l2 miss<\/code><\/pre>\n\n\n\n<p>As expected, time is spent in python process<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>386 processes\n\t 86 python3              36006.91   113.28\n\t 38 vulkaninfo               0.76     1.14\n\t  4 vulkani:disk$0           0.08     0.12\n\t  6 php                      0.06     0.06\n\t  2 llvmpipe-0               0.04     0.06\n\t  2 llvmpipe-1               0.04     0.06\n\t  2 llvmpipe-10              0.04     0.06\n\t  2 llvmpipe-11              0.04     0.06\n\t  2 llvmpipe-12              0.04     0.06\n\t  2 llvmpipe-13              0.04     0.06\n\t  2 llvmpipe-14              0.04     0.06\n\t  2 llvmpipe-15              0.04     0.06\n\t  2 llvmpipe-2               0.04     0.06\n\t  2 llvmpipe-3               0.04     0.06\n\t  2 llvmpipe-4               0.04     0.06\n\t  2 llvmpipe-5               0.04     0.06\n\t  2 llvmpipe-6               0.04     0.06\n\t  2 llvmpipe-7               0.04     0.06\n\t  2 llvmpipe-8               0.04     0.06\n\t  2 llvmpipe-9               0.04     0.06\n\t  6 clang                    0.04     0.05\n\t  1 lspci                    0.00     0.03\n\t 88 sh                       0.00     0.00\n\t 12 gcc                      0.00     0.00\n\t  9 stty                     0.00     0.00\n\t  8 gsettings                0.00     0.00\n\t  8 stat                     0.00     0.00\n\t  8 systemd-detect-          0.00     0.00\n\t  8 uname                    0.00     0.00\n\t  6 file                     0.00     0.00\n\t  6 llvm-link                0.00     0.00\n\t  5 glxinfo                  0.00     0.00\n\t  5 gmain                    0.00     0.00\n\t  5 lscpu                    0.00     0.00\n\t  5 phoronix-test-s          0.00     0.00\n\t  4 dmesg                    0.00     0.00\n\t  3 cat                      0.00     0.00\n\t  3 dconf worker             0.00     0.00\n\t  3 pytorch                  0.00     0.00\n\t  3 sysctl                   0.00     0.00\n\t  2 grep                     0.00     0.00\n\t  2 setterm                  0.00     0.00\n\t  2 which                    0.00     0.00\n\t  1 cc                       0.00     0.00\n\t  1 date                     0.00     0.00\n\t  1 dirname                  0.00     0.00\n\t  1 dmidecode                0.00     0.00\n\t  1 ifconfig                 0.00     0.00\n\t  1 ip                       0.00     0.00\n\t  1 lsmod                    0.00     0.00\n\t  1 mktemp                   0.00     0.00\n\t  1 ps                       0.00     0.00\n\t  1 readlink                 0.00     0.00\n\t  1 realpath                 0.00     0.00\n\t  1 sed                      0.00     0.00\n\t  1 sort                     0.00     0.00\n\t  1 systemctl                0.00     0.00\n\t  1 template.sh              0.00     0.00\n\t  1 wc                       0.00     0.00<\/code><\/pre>\n\n\n\n<p>The core section seems to run several processes per core.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>      496689) pytorch start=71.27 finish=133.98\n        496690) python3 start=71.27 finish=133.94\n          496691) python3 start=71.53 finish=133.94\n          496692) python3 start=71.53 finish=133.94\n          496693) python3 start=71.53 finish=133.94\n          496694) python3 start=71.53 finish=133.94\n          496695) python3 start=71.53 finish=133.94\n          496696) python3 start=71.53 finish=133.94\n          496697) python3 start=71.53 finish=133.94\n          496698) python3 start=71.53 finish=133.94\n          496699) python3 start=71.53 finish=133.94\n          496700) python3 start=71.53 finish=133.94\n          496701) python3 start=71.53 finish=133.94\n          496702) python3 start=71.53 finish=133.94\n          496703) python3 start=71.53 finish=133.94\n          496704) python3 start=71.53 finish=133.94\n          496705) python3 start=71.53 finish=133.94\n          496706) file start=72.23 finish=72.23\n          496707) uname start=72.23 finish=72.23\n          496708) python3 start=72.41 finish=73.48\n            496709) file start=72.44 finish=72.44\n            496710) uname start=72.44 finish=72.44\n            496711) cat start=72.44 finish=72.45\n            496712) lscpu start=72.45 finish=72.45\n            496713) sysctl start=72.45 finish=72.45\n            496714) dmesg start=72.46 finish=72.46\n            496715) python3 start=72.46 finish=73.47\n              496716) python3 start=73.47 finish=73.47\n          496717) python3 start=73.48 finish=73.48\n          496718) python3 start=73.49 finish=133.71\n          496719) python3 start=73.49 finish=133.94\n          496720) python3 start=73.49 finish=133.94\n          496721) python3 start=73.49 finish=133.94\n          496722) python3 start=73.49 finish=133.94\n          496723) python3 start=73.49 finish=133.94\n          496724) python3 start=73.49 finish=133.94\n          496725) python3 start=73.49 finish=133.94\n          496726) python3 start=75.59 finish=133.90<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Measurements on 64-bit resnet 50 model AMD metrics show a backend dominated benchmark with both memory and cpu contributing also showing both floating point and l2. It seemed to take multiple iterations to get to a converged result, contributing to <span class=\"excerpt-dots\">&hellip;<\/span> <a class=\"more-link\" href=\"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/pytorch\/\"><span class=\"more-msg\">Continue reading &rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":58,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-177","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/177","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/comments?post=177"}],"version-history":[{"count":1,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/177\/revisions"}],"predecessor-version":[{"id":180,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/177\/revisions\/180"}],"up":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/58"}],"wp:attachment":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/media?parent=177"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}