{"id":2544,"date":"2024-09-05T01:32:22","date_gmt":"2024-09-05T01:32:22","guid":{"rendered":"https:\/\/mvermeulen.org\/perf\/?page_id=2544"},"modified":"2024-09-05T14:21:59","modified_gmt":"2024-09-05T14:21:59","slug":"whisperfile","status":"publish","type":"page","link":"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/whisperfile\/","title":{"rendered":"whisperfile"},"content":{"rendered":"\n<p>Audio to text translation using the OpenAI Whisper model and Whisper.cpp.  Runs mostly on physical cores.  There are three workloads: tiny, small and medium.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/09\/systemtime.png\" alt=\"\" class=\"wp-image-2551\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/09\/systemtime.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/09\/systemtime-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/09\/systemtime-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Topdown profile is dominated by backend stalls with not much frontend stalls.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/09\/amdtopdown.png\" alt=\"\" class=\"wp-image-2552\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/09\/amdtopdown.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/09\/amdtopdown-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/09\/amdtopdown-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>AMD metrics confirm a backend bound application with twice as many memory stalls as CPU stalls. Frontend stalls are also very low.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              3281.110\non_cpu               0.469          # 7.50 \/ 16 cores\nutime                24529.851\nstime                71.720\nnvcsw                789353         # 67.13%\nnivcsw               386482         # 32.87%\ninblock              272            # 0.08\/sec\nonblock              2000952        # 609.84\/sec\ncpu-clock            24748102587968 # 24748.103 seconds\ntask-clock           24748961686997 # 24748.962 seconds\npage faults          11397536       # 460.526\/sec\ncontext switches     1191423        # 48.140\/sec\ncpu migrations       32105          # 1.297\/sec\nmajor page faults    2              # 0.000\/sec\nminor page faults    11397534       # 460.526\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             4572575710604  # 46.718 branches per 1000 inst\nbranch misses        14777271136    # 0.32% branch miss\nconditional          4115973292256  # 42.053 conditional branches per 1000 inst\nindirect             17517967739    # 0.179 indirect branches per 1000 inst\ncpu-cycles           102097628650614 # 1.94 GHz\ninstructions         97507810361983 # 0.96 IPC\nslots                205099946303142 #\nretiring             37214286254367 # 18.1% (18.2%)\n-- ucode             48968915545    #     0.0%\n-- fastpath          37165317338822 #    18.1%\nfrontend             4284690737113  #  2.1% ( 2.1%) low\n-- latency           3315556843362  #     1.6%\n-- bandwidth         969133893751   #     0.5%\nbackend              162775146273176 # 79.4% (79.4%) high\n-- cpu               47232427007135 #    23.0%\n-- memory            115542719266041 #    56.3%\nspeculation          616667516467   #  0.3% ( 0.3%) low\n-- branch mispredict 409532861682   #     0.2%\n-- pipeline restart  207134654785   #     0.1%\nsmt-contention       209032937338   #  0.1% ( 0.0%)\ncpu-cycles           101963426481978 # 1.94 GHz\ninstructions         97517015919776 # 0.96 IPC\ninstructions         32634325078974 # 131.264 l2 access per 1000 inst\nl2 hit from l1       2265760655143  # 24.68% l2 miss\nl2 miss from l1      283693287681   #\nl2 hit from l2 pf    1244526315275  #\nl3 hit from l2 pf    667413553522   #\nl3 miss from l2 pf   106025961425   #\ninstructions         32607938489954 # 134.540 float per 1000 inst\nfloat 512            76             # 0.000 AVX-512 per 1000 inst\nfloat 256            637404         # 0.000 AVX-256 per 1000 inst\nfloat 128            4387069981977  # 134.540 AVX-128 per 1000 inst\nfloat MMX            0              # 0.000 MMX per 1000 inst\nfloat scalar         0              # 0.000 scalar per 1000 inst\ninstructions         97825072073572 #\nopcache              11470979872220 # 117.260 opcache per 1000 inst\nopcache miss         293294675378   #  2.6% opcache miss rate\nl1 dTLB miss         448855689540   # 4.588 L1 dTLB per 1000 inst\nl2 dTLB miss         48011900496    # 0.491 L2 dTLB per 1000 inst\ninstructions         97843884531235 #\nicache               458280019597   # 4.684 icache per 1000 inst\nicache miss          76535669308    # 16.7% icache miss rate\nl1 iTLB miss         35830986       # 0.000 L1 iTLB per 1000 inst\nl2 iTLB miss         0              # 0.000 L2 iTLB per 1000 inst\ntlb flush            676808         # 0.000 TLB flush per 1000 inst\n<\/code><\/pre>\n\n\n\n<p>Intel metrics show most memory stalls at the L2 level.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              6401.384\non_cpu               0.241          # 3.85 \/ 16 cores\nutime                24581.149\nstime                58.903\nnvcsw                554296         # 80.38%\nnivcsw               135330         # 19.62%\ninblock              1965872        # 307.10\/sec\nonblock              1992264        # 311.22\/sec\ncpu-clock            24794386618554 # 24794.387 seconds\ntask-clock           24795763519490 # 24795.764 seconds\npage faults          7532655        # 303.788\/sec\ncontext switches     720042         # 29.039\/sec\ncpu migrations       94201          # 3.799\/sec\nmajor page faults    91             # 0.004\/sec\nminor page faults    7532564        # 303.784\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             10341718642383 # 54.805 branches per 1000 inst\nbranch misses        9565794658     # 0.09% branch miss\nconditional          10341734517295 # 54.805 conditional branches per 1000 inst\nindirect             156368123592   # 0.829 indirect branches per 1000 inst\nslots                532475805237926 #\nretiring             251720082877249 # 47.3% (47.3%)\n-- ucode             45341165034866 #     8.5%\n-- fastpath          206378917842383 #    38.8%\nfrontend             39863638341533 #  7.5% ( 7.5%)\n-- latency           2430803551860  #     0.5%\n-- bandwidth         37432834789673 #     7.0%\nbackend              239695063602645 # 45.0% (45.0%)\n-- cpu               184700781665297 #    34.7%\n-- memory            54994281937348 #    10.3%\nspeculation          2347427794172  #  0.4% ( 0.4%) low\n-- branch mispredict 826280176476   #     0.2%\n-- pipeline restart  1521147617696  #     0.3%\nsmt-contention       0              #  0.0% ( 0.0%)\ncpu-cycles           89422295522279 # 0.87 GHz\ninstructions         188568355555583 # 2.11 IPC\nl2 access            9948057087090  # 53.102 l2 access per 1000 inst\nl2 miss              4720943665861  # 47.46% l2 miss\ncpu-cycles           88972264948894 # 18.1% memory latency\nload stalls          15534009333939 #  0.0% l1 bound\nl1 miss              35174852346207 # 24.3% l2 bound\nl2 miss              13537304518777 #  5.0% l3 bound\nl3 miss              9109399710512  # 10.2% dram bound\nstore_stalls         533063233128   #  0.6% store bound\n<\/code><\/pre>\n\n\n\n<p>Process summary shows almost all time in a .ape-1.10 program with many processes launched<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>651721 processes\n\t648384 .ape-1.10            16390354.57 2833993.32\n\t 36 clinfo                   4.10     2.25\n\t 38 vulkaninfo               1.33     0.96\n\t  6 php                      0.15     0.89\n\t  4 vulkani:disk$0           0.14     0.11\n\t  2 llvmpipe-0               0.07     0.05\n\t  2 llvmpipe-1               0.07     0.05\n\t  2 llvmpipe-10              0.07     0.05\n\t  2 llvmpipe-11              0.07     0.05\n\t  2 llvmpipe-12              0.07     0.05\n\t  2 llvmpipe-13              0.07     0.05\n\t  2 llvmpipe-14              0.07     0.05\n\t  2 llvmpipe-15              0.07     0.05\n\t  2 llvmpipe-2               0.07     0.05\n\t  2 llvmpipe-3               0.07     0.05\n\t  2 llvmpipe-4               0.07     0.05\n\t  2 llvmpipe-5               0.07     0.05\n\t  2 llvmpipe-6               0.07     0.05\n\t  2 llvmpipe-7               0.07     0.05\n\t  2 llvmpipe-8               0.07     0.05\n\t  2 llvmpipe-9               0.07     0.05\n\t  6 clang                    0.06     0.06\n\t  3 rocminfo                 0.03     0.00\n\t  1 lspci                    0.00     0.02\n\t  1 ps                       0.00     0.01\n\t 97 sh                       0.00     0.00\n\t 12 gcc                      0.00     0.00\n\t  9 whisperfile              0.00     0.00\n\t  8 gsettings                0.00     0.00\n\t  8 stat                     0.00     0.00\n\t  8 systemd-detect-          0.00     0.00\n\t  6 llvm-link                0.00     0.00\n\t  5 glxinfo                  0.00     0.00\n\t  5 gmain                    0.00     0.00\n\t  5 phoronix-test-s          0.00     0.00\n\t  3 dconf worker             0.00     0.00\n\t  2 dmesg                    0.00     0.00\n\t  2 grep                     0.00     0.00\n\t  2 lscpu                    0.00     0.00\n\t  2 setterm                  0.00     0.00\n\t  2 uname                    0.00     0.00\n\t  2 which                    0.00     0.00\n\t  1 cc                       0.00     0.00\n\t  1 date                     0.00     0.00\n\t  1 dirname                  0.00     0.00\n\t  1 dmidecode                0.00     0.00\n\t  1 ifconfig                 0.00     0.00\n\t  1 ip                       0.00     0.00\n\t  1 lsmod                    0.00     0.00\n\t  1 mktemp                   0.00     0.00\n\t  1 qdbus                    0.00     0.00\n\t  1 readlink                 0.00     0.00\n\t  1 realpath                 0.00     0.00\n\t  1 sed                      0.00     0.00\n\t  1 sort                     0.00     0.00\n\t  1 stty                     0.00     0.00\n\t  1 systemctl                0.00     0.00\n\t  1 template.sh              0.00     0.00\n\t  1 wc                       0.00     0.00\n3015 processes running\n3062 maximum processes\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Audio to text translation using the OpenAI Whisper model and Whisper.cpp. Runs mostly on physical cores. There are three workloads: tiny, small and medium. Topdown profile is dominated by backend stalls with not much frontend stalls. AMD metrics confirm a <span class=\"excerpt-dots\">&hellip;<\/span> <a class=\"more-link\" href=\"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/whisperfile\/\"><span class=\"more-msg\">Continue reading &rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":58,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-2544","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/2544","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/comments?post=2544"}],"version-history":[{"count":2,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/2544\/revisions"}],"predecessor-version":[{"id":2553,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/2544\/revisions\/2553"}],"up":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/58"}],"wp:attachment":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/media?parent=2544"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}