{"id":1358,"date":"2024-02-03T15:10:45","date_gmt":"2024-02-03T15:10:45","guid":{"rendered":"https:\/\/mvermeulen.org\/perf\/?page_id=1358"},"modified":"2024-02-03T21:43:41","modified_gmt":"2024-02-03T21:43:41","slug":"scikit-learn","status":"publish","type":"page","link":"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/scikit-learn\/","title":{"rendered":"scikit-learn"},"content":{"rendered":"\n<p>A python tool for machine learning<\/p>\n\n\n\n<p>This does not install on my system, some type of conflict between numpy and openBLAS with errors shown below. It does run on my 7950x processor where I don&#8217;t have scikit-learn, so report below on those metrics.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code> \u00d7 Preparing metadata (pyproject.toml) did not run successfully.\n  \u2502 exit code: 1\n  \u2570\u2500&gt; &#91;72 lines of output]\n      Partial import of sklearn during the build process.\n      Traceback (most recent call last):\n        File \"\/usr\/lib\/python3\/dist-packages\/numpy\/core\/__init__.py\", line 22, in &lt;module&gt;\n          from . import multiarray\n        File \"\/usr\/lib\/python3\/dist-packages\/numpy\/core\/multiarray.py\", line 12, in &lt;module&gt;\n          from . import overrides\n        File \"\/usr\/lib\/python3\/dist-packages\/numpy\/core\/overrides.py\", line 7, in &lt;module&gt;\n          from numpy.core._multiarray_umath import (\n      ImportError: \/lib\/x86_64-linux-gnu\/libblas.so.3: undefined symbol: gotoblas\n\n      During handling of the above exception, another exception occurred:\n\n      Traceback (most recent call last):\n        File \"setup.py\", line 218, in check_package_status\n          module = importlib.import_module(package)\n        File \"\/usr\/lib\/python3.10\/importlib\/__init__.py\", line 126, in import_module\n          return _bootstrap._gcd_import(name&#91;level:], package, level)\n        File \"&lt;frozen importlib._bootstrap&gt;\", line 1050, in _gcd_import\n        File \"&lt;frozen importlib._bootstrap&gt;\", line 1027, in _find_and_load\n        File \"&lt;frozen importlib._bootstrap&gt;\", line 1006, in _find_and_load_unlocked\n        File \"&lt;frozen importlib._bootstrap&gt;\", line 688, in _load_unlocked\n        File \"&lt;frozen importlib._bootstrap_external&gt;\", line 883, in exec_module\n        File \"&lt;frozen importlib._bootstrap&gt;\", line 241, in _call_with_frames_removed\n        File \"\/usr\/lib\/python3\/dist-packages\/numpy\/__init__.py\", line 150, in &lt;module&gt;\n          from . import core\n        File \"\/usr\/lib\/python3\/dist-packages\/numpy\/core\/__init__.py\", line 48, in &lt;module&gt;\n          raise ImportError(msg)\n      ImportError:\n\n      IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!\n\n      Importing the numpy C-extensions failed. This error can happen for\n      many reasons, often due to issues with your setup or how NumPy was\n      installed.\n\n      We have compiled some common reasons and troubleshooting tips at:\n\n          https:&#47;&#47;numpy.org\/devdocs\/user\/troubleshooting-importerror.html\n\n      Please note and check the following:\n\n        * The Python version is: Python3.10 from \"\/usr\/bin\/python3\"\n        * The NumPy version is: \"1.21.5\"\n\n      and make sure that they are the versions you expect.\n      Please carefully study the documentation linked above for further help.\n\n      Original error was: \/lib\/x86_64-linux-gnu\/libblas.so.3: undefined symbol: gotoblas\n\n      Traceback (most recent call last):\n        File \"\/usr\/lib\/python3\/dist-packages\/pip\/_vendor\/pep517\/in_process\/_in_process.py\", line 363, in &lt;module&gt;\n          main()\n        File \"\/usr\/lib\/python3\/dist-packages\/pip\/_vendor\/pep517\/in_process\/_in_process.py\", line 345, in main\n          json_out&#91;'return_val'] = hook(**hook_input&#91;'kwargs'])\n        File \"\/usr\/lib\/python3\/dist-packages\/pip\/_vendor\/pep517\/in_process\/_in_process.py\", line 164, in prepare_metadata_for_build_wheel\n          return hook(metadata_directory, config_settings)\n        File \"\/usr\/lib\/python3\/dist-packages\/setuptools\/build_meta.py\", line 174, in prepare_metadata_for_build_wheel\n          self.run_setup()\n        File \"\/usr\/lib\/python3\/dist-packages\/setuptools\/build_meta.py\", line 267, in run_setup\n          super(_BuildMetaLegacyBackend,\n        File \"\/usr\/lib\/python3\/dist-packages\/setuptools\/build_meta.py\", line 158, in run_setup\n          exec(compile(code, __file__, 'exec'), locals())\n        File \"setup.py\", line 669, in &lt;module&gt;\n          setup_package()\n        File \"setup.py\", line 659, in setup_package\n          check_package_status(\"numpy\", min_deps.NUMPY_MIN_VERSION)\n        File \"setup.py\", line 245, in check_package_status\n          raise ImportError(\n      ImportError: numpy is not installed.\n      scikit-learn requires numpy &gt;= 1.17.3.\n      Installation instructions are available on the scikit-learn website: http:\/\/scikit-learn.org\/stable\/install.html\n\n      &#91;end of output]\n\n  note: This error originates from a subprocess, and is likely not a problem with pip.\nerror: metadata-generation-failed\n\u00d7 Encountered error while generating package metadata.\n\u2570\u2500&gt; See above for output.\n<\/code><\/pre>\n\n\n\n<p>System profile running on 7950X, set up with three of the ~40 workloads.  Overall goes to the 32 available cores.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-21.png\" alt=\"\" class=\"wp-image-1412\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-21.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-21-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-21-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Topdown profile shows interesting patterns over time. The first workload has declining retirement while backend stalls climb and backend stalls drop. The second workload shows retirement rising as backend stalls decline and frontend stalls are low. The third workload is too short to have significant influence.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-21.png\" alt=\"\" class=\"wp-image-1413\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-21.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-21-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-21-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Overall AMD metrics have a low IPC, relatively low reitrment and higher backend stalls particularly memory. This code has little floating point.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              2787.714\non_cpu               0.400          # 12.80 \/ 32 cores\nutime                20527.167\nstime                15146.884\nnvcsw                756968         # 0.59%\nnivcsw               127597917      # 99.41%\ninblock              0              # 0.00\/sec\nonblock              53152          # 19.07\/sec\ncpu-clock            35670069017218 # 35670.069 seconds\ntask-clock           35670975886690 # 35670.976 seconds\npage faults          25700348       # 720.483\/sec\ncontext switches     128368636      # 3598.686\/sec\ncpu migrations       28530          # 0.800\/sec\nmajor page faults    11             # 0.000\/sec\nminor page faults    25700337       # 720.483\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             18887219082311 # 159.328 branches per 1000 inst\nbranch misses        2313958692726  # 12.25% branch miss\nconditional          8777226338480  # 74.043 conditional branches per 1000 inst\nindirect             329145203716   # 2.777 indirect branches per 1000 inst\ncpu-cycles           99230626523793 # 1.56 GHz\ninstructions         65075440865576 # 0.66 IPC low\nslots                198236732951058 #\nretiring             25568481119361 # 12.9% (15.3%)\n-- ucode             255973174061   #     0.1%\n-- fastpath          25312507945300 #    12.8%\nfrontend             57549990940657 # 29.0% (34.5%)\n-- latency           46925611578966 #    23.7%\n-- bandwidth         10624379361691 #     5.4%\nbackend              83129330239826 # 41.9% (49.8%)\n-- cpu               20954381468456 #    10.6%\n-- memory            62174948771370 #    31.4%\nspeculation          746737651643   #  0.4% ( 0.4%) low\n-- branch mispredict 741386192372   #     0.4%\n-- pipeline restart  5351459271     #     0.0%\nsmt-contention       31242000604238 # 15.8% ( 0.0%)\ncpu-cycles           101570060560375 # 1.59 GHz\ninstructions         65171978024022 # 0.64 IPC low\ninstructions         21720794613767 # 67.958 l2 access per 1000 inst\nl2 hit from l1       814382917898   # 35.43% l2 miss\nl2 miss from l1      177238781689   #\nl2 hit from l2 pf    315944062717   #\nl3 hit from l2 pf    307229019796   #\nl3 miss from l2 pf   38544125588    #\ninstructions         21710795462798 # 31.462 float per 1000 inst\nfloat 512            26             # 0.000 AVX-512 per 1000 inst\nfloat 256            657136608      # 0.030 AVX-256 per 1000 inst\nfloat 128            682416484723   # 31.432 AVX-128 per 1000 inst\nfloat MMX            0              # 0.000 MMX per 1000 inst\nfloat scalar         85             # 0.000 scalar per 1000 inst\ninstructions         2598433        #\nopcache              1000197        # 384.923 opcache per 1000 inst\nopcache miss         536955         # 53.7% opcache miss rate\nl1 dTLB miss         5294           # 2.037 L1 dTLB per 1000 inst\nl2 dTLB miss         1421           # 0.547 L2 dTLB per 1000 inst\ninstructions         2608361        #\nicache               1317012        # 504.919 icache per 1000 inst\nicache miss          109199         #  8.3% icache miss rate\nl1 iTLB miss         16             # 0.006 L1 iTLB per 1000 inst\nl2 iTLB miss         0              # 0.000 L2 iTLB per 1000 inst\ntlb flush            19             # 0.007 TLB flush per 1000 inst\n<\/code><\/pre>\n\n\n\n<p>Process overview shows this as a python driven workload.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>1170 processes\n\t789 python3              784329.27 470030.91\n\t 13 gmain                11536.58  6584.02\n\t  8 gdbus                11536.58  6584.02\n\t  8 pool                 11536.58  6584.02\n\t 70 vulkaninfo               5.19     1.73\n\t  6 vulkani:disk$0           0.45     0.15\n\t  6 clinfo                   0.19     0.14\n\t  2 llvmpipe-0               0.15     0.05\n\t  2 llvmpipe-1               0.15     0.05\n\t  2 llvmpipe-10              0.15     0.05\n\t  2 llvmpipe-11              0.15     0.05\n\t  2 llvmpipe-12              0.15     0.05\n\t  2 llvmpipe-13              0.15     0.05\n\t  2 llvmpipe-14              0.15     0.05\n\t  2 llvmpipe-15              0.15     0.05\n\t  2 llvmpipe-16              0.15     0.05\n\t  2 llvmpipe-17              0.15     0.05\n\t  2 llvmpipe-18              0.15     0.05\n\t  2 llvmpipe-19              0.15     0.05\n\t  2 llvmpipe-2               0.15     0.05\n\t  2 llvmpipe-20              0.15     0.05\n\t  2 llvmpipe-21              0.15     0.05\n\t  2 llvmpipe-22              0.15     0.05\n\t  2 llvmpipe-23              0.15     0.05\n\t  2 llvmpipe-24              0.15     0.05\n\t  2 llvmpipe-25              0.15     0.05\n\t  2 llvmpipe-26              0.15     0.05\n\t  2 llvmpipe-27              0.15     0.05\n\t  2 llvmpipe-28              0.15     0.05\n\t  2 llvmpipe-29              0.15     0.05\n\t  2 llvmpipe-3               0.15     0.05\n\t  2 llvmpipe-30              0.15     0.05\n\t  2 llvmpipe-31              0.15     0.05\n\t  2 llvmpipe-4               0.15     0.05\n\t  2 llvmpipe-5               0.15     0.05\n\t  2 llvmpipe-6               0.15     0.05\n\t  2 llvmpipe-7               0.15     0.05\n\t  2 llvmpipe-8               0.15     0.05\n\t  2 llvmpipe-9               0.15     0.05\n\t  6 glxinfo:gdrv0            0.13     0.00\n\t  6 glxinfo:gl0              0.13     0.00\n\t  6 php                      0.06     0.25\n\t  2 glxinfo                  0.06     0.00\n\t  2 glxinfo:cs0              0.05     0.00\n\t  2 glxinfo:disk$0           0.05     0.00\n\t  2 glxinfo:sh0              0.05     0.00\n\t  2 glxinfo:shlo0            0.05     0.00\n\t 13 dconf worker             0.00 11536.58\n\t  8 &#91;pango] FcInit           0.00 11492.10\n\t 74 sh                       0.00     0.00\n\t 13 gcc                      0.00     0.00\n\t 12 scikit-learn             0.00     0.00\n\t  8 stat                     0.00     0.00\n\t  8 systemd-detect-          0.00     0.00\n\t  6 gsettings                0.00     0.00\n\t  5 phoronix-test-s          0.00     0.00\n\t  2 cc                       0.00     0.00\n\t  2 lscpu                    0.00     0.00\n\t  2 uname                    0.00     0.00\n\t  2 which                    0.00     0.00\n\t  2 xset                     0.00     0.00\n\t  1 date                     0.00     0.00\n\t  1 dirname                  0.00     0.00\n\t  1 dmesg                    0.00     0.00\n\t  1 dmidecode                0.00     0.00\n\t  1 grep                     0.00     0.00\n\t  1 ifconfig                 0.00     0.00\n\t  1 ip                       0.00     0.00\n\t  1 lsmod                    0.00     0.00\n\t  1 lspci                    0.00     0.00\n\t  1 mktemp                   0.00     0.00\n\t  1 ps                       0.00     0.00\n\t  1 python                   0.00     0.00\n\t  1 readlink                 0.00     0.00\n\t  1 realpath                 0.00     0.00\n\t  1 sed                      0.00     0.00\n\t  1 sort                     0.00     0.00\n\t  1 stty                     0.00     0.00\n\t  1 systemctl                0.00     0.00\n\t  1 template.sh              0.00     0.00\n\t  1 wc                       0.00     0.00\n\t  1 xrandr                   0.00     0.00\n0 processes running\n80 maximum processes\n<\/code><\/pre>\n\n\n\n<p>Computation blocks are straightforward<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>      305954) scikit-learn     cpu=27 start=4.92  finish=164.59\n        305955) python3          cpu=17 start=4.92  finish=164.58\n          305956) python3          cpu=4 start=4.99  finish=164.57\n          305957) python3          cpu=2 start=4.99  finish=164.57\n          305958) python3          cpu=19 start=4.99  finish=164.57\n          305959) python3          cpu=1 start=4.99  finish=164.57\n          305960) python3          cpu=27 start=4.99  finish=164.57\n          305961) python3          cpu=7 start=4.99  finish=164.57\n          305962) python3          cpu=10 start=4.99  finish=164.57\n          305963) python3          cpu=25 start=4.99  finish=164.57\n          305964) python3          cpu=16 start=4.99  finish=164.57\n          305965) python3          cpu=0 start=4.99  finish=164.57\n          305966) python3          cpu=24 start=4.99  finish=164.57\n          305967) python3          cpu=5 start=4.99  finish=164.57\n          305968) python3          cpu=28 start=4.99  finish=164.57\n          305969) python3          cpu=6 start=4.99  finish=164.57\n          305970) python3          cpu=18 start=4.99  finish=164.57\n          305971) python3          cpu=22 start=4.99  finish=164.57\n          305972) python3          cpu=31 start=4.99  finish=164.57\n          305973) python3          cpu=8 start=4.99  finish=164.57\n          305974) python3          cpu=12 start=4.99  finish=164.57\n          305975) python3          cpu=3 start=4.99  finish=164.57\n          305976) python3          cpu=29 start=4.99  finish=164.57\n          305977) python3          cpu=26 start=4.99  finish=164.57\n          305978) python3          cpu=9 start=4.99  finish=164.57\n          305979) python3          cpu=23 start=4.99  finish=164.57\n          305980) python3          cpu=21 start=4.99  finish=164.57\n          305981) python3          cpu=20 start=4.99  finish=164.57\n          305982) python3          cpu=15 start=4.99  finish=164.57\n          305983) python3          cpu=17 start=4.99  finish=164.57\n          305984) python3          cpu=30 start=4.99  finish=164.57\n          305985) python3          cpu=13 start=4.99  finish=164.57\n          305986) python3          cpu=14 start=4.99  finish=164.57\n          305987) python3          cpu=18 start=5.31  finish=164.58\n          305988) python3          cpu=0 start=5.31  finish=164.58\n          305989) python3          cpu=1 start=5.31  finish=164.58\n          305990) python3          cpu=20 start=5.31  finish=164.58\n          305991) python3          cpu=8 start=5.32  finish=164.58\n          305992) python3          cpu=13 start=5.32  finish=164.58\n          305993) python3          cpu=19 start=5.32  finish=164.58\n          305994) python3          cpu=5 start=5.32  finish=164.58\n          305995) python3          cpu=23 start=5.32  finish=164.58\n          305996) python3          cpu=16 start=5.32  finish=164.58\n          305997) python3          cpu=21 start=5.32  finish=164.58\n          305998) python3          cpu=29 start=5.32  finish=164.58\n          305999) python3          cpu=25 start=5.32  finish=164.58\n          306000) python3          cpu=10 start=5.32  finish=164.58\n          306001) python3          cpu=22 start=5.32  finish=164.58\n          306002) python3          cpu=28 start=5.32  finish=164.58\n          306003) python3          cpu=3 start=5.32  finish=164.58\n          306004) python3          cpu=4 start=5.32  finish=164.58\n          306005) python3          cpu=12 start=5.32  finish=164.58\n          306006) python3          cpu=6 start=5.32  finish=164.58\n          306007) python3          cpu=27 start=5.32  finish=164.58\n          306008) python3          cpu=7 start=5.32  finish=164.58\n          306009) python3          cpu=2 start=5.32  finish=164.58\n          306010) python3          cpu=31 start=5.32  finish=164.58\n          306011) python3          cpu=1 start=5.32  finish=164.58\n          306012) python3          cpu=26 start=5.32  finish=164.58\n          306013) python3          cpu=9 start=5.32  finish=164.58\n          306014) python3          cpu=15 start=5.32  finish=164.58\n          306015) python3          cpu=30 start=5.32  finish=164.58\n          306016) python3          cpu=14 start=5.32  finish=164.58\n          306017) python3          cpu=24 start=5.32  finish=164.58\n          306069) &#91;pango] FcInit   cpu=-1 start=164.49 finish=164.49\n          306070) gmain            cpu=11 start=164.49 finish=164.58\n          306071) dconf worker     cpu=-1 start=164.49 finish=164.58\n            306072) gdbus            cpu=0 start=164.49 finish=164.58\n          306073) pool             cpu=29 start=164.50 finish=164.58\n          306074) python3          cpu=31 start=164.51 finish=164.51\n          306075) python3          cpu=20 start=164.51 finish=164.51\n          306076) python3          cpu=31 start=164.51 finish=164.51\n          306077) python3          cpu=21 start=164.51 finish=164.51\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>A python tool for machine learning This does not install on my system, some type of conflict between numpy and openBLAS with errors shown below. It does run on my 7950x processor where I don&#8217;t have scikit-learn, so report below <span class=\"excerpt-dots\">&hellip;<\/span> <a class=\"more-link\" href=\"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/scikit-learn\/\"><span class=\"more-msg\">Continue reading &rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":58,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1358","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1358","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/comments?post=1358"}],"version-history":[{"count":4,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1358\/revisions"}],"predecessor-version":[{"id":1414,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1358\/revisions\/1414"}],"up":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/58"}],"wp:attachment":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/media?parent=1358"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}