A python tool for machine learning
This does not install on my system, some type of conflict between numpy and openBLAS with errors shown below. It does run on my 7950x processor where I don’t have scikit-learn, so report below on those metrics.
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [72 lines of output]
Partial import of sklearn during the build process.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 22, in <module>
from . import multiarray
File "/usr/lib/python3/dist-packages/numpy/core/multiarray.py", line 12, in <module>
from . import overrides
File "/usr/lib/python3/dist-packages/numpy/core/overrides.py", line 7, in <module>
from numpy.core._multiarray_umath import (
ImportError: /lib/x86_64-linux-gnu/libblas.so.3: undefined symbol: gotoblas
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "setup.py", line 218, in check_package_status
module = importlib.import_module(package)
File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/usr/lib/python3/dist-packages/numpy/__init__.py", line 150, in <module>
from . import core
File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 48, in <module>
raise ImportError(msg)
ImportError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.10 from "/usr/bin/python3"
* The NumPy version is: "1.21.5"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: /lib/x86_64-linux-gnu/libblas.so.3: undefined symbol: gotoblas
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/usr/lib/python3/dist-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 174, in prepare_metadata_for_build_wheel
self.run_setup()
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 267, in run_setup
super(_BuildMetaLegacyBackend,
File "/usr/lib/python3/dist-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 669, in <module>
setup_package()
File "setup.py", line 659, in setup_package
check_package_status("numpy", min_deps.NUMPY_MIN_VERSION)
File "setup.py", line 245, in check_package_status
raise ImportError(
ImportError: numpy is not installed.
scikit-learn requires numpy >= 1.17.3.
Installation instructions are available on the scikit-learn website: http://scikit-learn.org/stable/install.html
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
System profile running on 7950X, set up with three of the ~40 workloads. Overall goes to the 32 available cores.
Topdown profile shows interesting patterns over time. The first workload has declining retirement while backend stalls climb and backend stalls drop. The second workload shows retirement rising as backend stalls decline and frontend stalls are low. The third workload is too short to have significant influence.
Overall AMD metrics have a low IPC, relatively low reitrment and higher backend stalls particularly memory. This code has little floating point.
elapsed 2787.714
on_cpu 0.400 # 12.80 / 32 cores
utime 20527.167
stime 15146.884
nvcsw 756968 # 0.59%
nivcsw 127597917 # 99.41%
inblock 0 # 0.00/sec
onblock 53152 # 19.07/sec
cpu-clock 35670069017218 # 35670.069 seconds
task-clock 35670975886690 # 35670.976 seconds
page faults 25700348 # 720.483/sec
context switches 128368636 # 3598.686/sec
cpu migrations 28530 # 0.800/sec
major page faults 11 # 0.000/sec
minor page faults 25700337 # 720.483/sec
alignment faults 0 # 0.000/sec
emulation faults 0 # 0.000/sec
branches 18887219082311 # 159.328 branches per 1000 inst
branch misses 2313958692726 # 12.25% branch miss
conditional 8777226338480 # 74.043 conditional branches per 1000 inst
indirect 329145203716 # 2.777 indirect branches per 1000 inst
cpu-cycles 99230626523793 # 1.56 GHz
instructions 65075440865576 # 0.66 IPC low
slots 198236732951058 #
retiring 25568481119361 # 12.9% (15.3%)
-- ucode 255973174061 # 0.1%
-- fastpath 25312507945300 # 12.8%
frontend 57549990940657 # 29.0% (34.5%)
-- latency 46925611578966 # 23.7%
-- bandwidth 10624379361691 # 5.4%
backend 83129330239826 # 41.9% (49.8%)
-- cpu 20954381468456 # 10.6%
-- memory 62174948771370 # 31.4%
speculation 746737651643 # 0.4% ( 0.4%) low
-- branch mispredict 741386192372 # 0.4%
-- pipeline restart 5351459271 # 0.0%
smt-contention 31242000604238 # 15.8% ( 0.0%)
cpu-cycles 101570060560375 # 1.59 GHz
instructions 65171978024022 # 0.64 IPC low
instructions 21720794613767 # 67.958 l2 access per 1000 inst
l2 hit from l1 814382917898 # 35.43% l2 miss
l2 miss from l1 177238781689 #
l2 hit from l2 pf 315944062717 #
l3 hit from l2 pf 307229019796 #
l3 miss from l2 pf 38544125588 #
instructions 21710795462798 # 31.462 float per 1000 inst
float 512 26 # 0.000 AVX-512 per 1000 inst
float 256 657136608 # 0.030 AVX-256 per 1000 inst
float 128 682416484723 # 31.432 AVX-128 per 1000 inst
float MMX 0 # 0.000 MMX per 1000 inst
float scalar 85 # 0.000 scalar per 1000 inst
instructions 2598433 #
opcache 1000197 # 384.923 opcache per 1000 inst
opcache miss 536955 # 53.7% opcache miss rate
l1 dTLB miss 5294 # 2.037 L1 dTLB per 1000 inst
l2 dTLB miss 1421 # 0.547 L2 dTLB per 1000 inst
instructions 2608361 #
icache 1317012 # 504.919 icache per 1000 inst
icache miss 109199 # 8.3% icache miss rate
l1 iTLB miss 16 # 0.006 L1 iTLB per 1000 inst
l2 iTLB miss 0 # 0.000 L2 iTLB per 1000 inst
tlb flush 19 # 0.007 TLB flush per 1000 inst
Process overview shows this as a python driven workload.
1170 processes
789 python3 784329.27 470030.91
13 gmain 11536.58 6584.02
8 gdbus 11536.58 6584.02
8 pool 11536.58 6584.02
70 vulkaninfo 5.19 1.73
6 vulkani:disk$0 0.45 0.15
6 clinfo 0.19 0.14
2 llvmpipe-0 0.15 0.05
2 llvmpipe-1 0.15 0.05
2 llvmpipe-10 0.15 0.05
2 llvmpipe-11 0.15 0.05
2 llvmpipe-12 0.15 0.05
2 llvmpipe-13 0.15 0.05
2 llvmpipe-14 0.15 0.05
2 llvmpipe-15 0.15 0.05
2 llvmpipe-16 0.15 0.05
2 llvmpipe-17 0.15 0.05
2 llvmpipe-18 0.15 0.05
2 llvmpipe-19 0.15 0.05
2 llvmpipe-2 0.15 0.05
2 llvmpipe-20 0.15 0.05
2 llvmpipe-21 0.15 0.05
2 llvmpipe-22 0.15 0.05
2 llvmpipe-23 0.15 0.05
2 llvmpipe-24 0.15 0.05
2 llvmpipe-25 0.15 0.05
2 llvmpipe-26 0.15 0.05
2 llvmpipe-27 0.15 0.05
2 llvmpipe-28 0.15 0.05
2 llvmpipe-29 0.15 0.05
2 llvmpipe-3 0.15 0.05
2 llvmpipe-30 0.15 0.05
2 llvmpipe-31 0.15 0.05
2 llvmpipe-4 0.15 0.05
2 llvmpipe-5 0.15 0.05
2 llvmpipe-6 0.15 0.05
2 llvmpipe-7 0.15 0.05
2 llvmpipe-8 0.15 0.05
2 llvmpipe-9 0.15 0.05
6 glxinfo:gdrv0 0.13 0.00
6 glxinfo:gl0 0.13 0.00
6 php 0.06 0.25
2 glxinfo 0.06 0.00
2 glxinfo:cs0 0.05 0.00
2 glxinfo:disk$0 0.05 0.00
2 glxinfo:sh0 0.05 0.00
2 glxinfo:shlo0 0.05 0.00
13 dconf worker 0.00 11536.58
8 [pango] FcInit 0.00 11492.10
74 sh 0.00 0.00
13 gcc 0.00 0.00
12 scikit-learn 0.00 0.00
8 stat 0.00 0.00
8 systemd-detect- 0.00 0.00
6 gsettings 0.00 0.00
5 phoronix-test-s 0.00 0.00
2 cc 0.00 0.00
2 lscpu 0.00 0.00
2 uname 0.00 0.00
2 which 0.00 0.00
2 xset 0.00 0.00
1 date 0.00 0.00
1 dirname 0.00 0.00
1 dmesg 0.00 0.00
1 dmidecode 0.00 0.00
1 grep 0.00 0.00
1 ifconfig 0.00 0.00
1 ip 0.00 0.00
1 lsmod 0.00 0.00
1 lspci 0.00 0.00
1 mktemp 0.00 0.00
1 ps 0.00 0.00
1 python 0.00 0.00
1 readlink 0.00 0.00
1 realpath 0.00 0.00
1 sed 0.00 0.00
1 sort 0.00 0.00
1 stty 0.00 0.00
1 systemctl 0.00 0.00
1 template.sh 0.00 0.00
1 wc 0.00 0.00
1 xrandr 0.00 0.00
0 processes running
80 maximum processes
Computation blocks are straightforward
305954) scikit-learn cpu=27 start=4.92 finish=164.59
305955) python3 cpu=17 start=4.92 finish=164.58
305956) python3 cpu=4 start=4.99 finish=164.57
305957) python3 cpu=2 start=4.99 finish=164.57
305958) python3 cpu=19 start=4.99 finish=164.57
305959) python3 cpu=1 start=4.99 finish=164.57
305960) python3 cpu=27 start=4.99 finish=164.57
305961) python3 cpu=7 start=4.99 finish=164.57
305962) python3 cpu=10 start=4.99 finish=164.57
305963) python3 cpu=25 start=4.99 finish=164.57
305964) python3 cpu=16 start=4.99 finish=164.57
305965) python3 cpu=0 start=4.99 finish=164.57
305966) python3 cpu=24 start=4.99 finish=164.57
305967) python3 cpu=5 start=4.99 finish=164.57
305968) python3 cpu=28 start=4.99 finish=164.57
305969) python3 cpu=6 start=4.99 finish=164.57
305970) python3 cpu=18 start=4.99 finish=164.57
305971) python3 cpu=22 start=4.99 finish=164.57
305972) python3 cpu=31 start=4.99 finish=164.57
305973) python3 cpu=8 start=4.99 finish=164.57
305974) python3 cpu=12 start=4.99 finish=164.57
305975) python3 cpu=3 start=4.99 finish=164.57
305976) python3 cpu=29 start=4.99 finish=164.57
305977) python3 cpu=26 start=4.99 finish=164.57
305978) python3 cpu=9 start=4.99 finish=164.57
305979) python3 cpu=23 start=4.99 finish=164.57
305980) python3 cpu=21 start=4.99 finish=164.57
305981) python3 cpu=20 start=4.99 finish=164.57
305982) python3 cpu=15 start=4.99 finish=164.57
305983) python3 cpu=17 start=4.99 finish=164.57
305984) python3 cpu=30 start=4.99 finish=164.57
305985) python3 cpu=13 start=4.99 finish=164.57
305986) python3 cpu=14 start=4.99 finish=164.57
305987) python3 cpu=18 start=5.31 finish=164.58
305988) python3 cpu=0 start=5.31 finish=164.58
305989) python3 cpu=1 start=5.31 finish=164.58
305990) python3 cpu=20 start=5.31 finish=164.58
305991) python3 cpu=8 start=5.32 finish=164.58
305992) python3 cpu=13 start=5.32 finish=164.58
305993) python3 cpu=19 start=5.32 finish=164.58
305994) python3 cpu=5 start=5.32 finish=164.58
305995) python3 cpu=23 start=5.32 finish=164.58
305996) python3 cpu=16 start=5.32 finish=164.58
305997) python3 cpu=21 start=5.32 finish=164.58
305998) python3 cpu=29 start=5.32 finish=164.58
305999) python3 cpu=25 start=5.32 finish=164.58
306000) python3 cpu=10 start=5.32 finish=164.58
306001) python3 cpu=22 start=5.32 finish=164.58
306002) python3 cpu=28 start=5.32 finish=164.58
306003) python3 cpu=3 start=5.32 finish=164.58
306004) python3 cpu=4 start=5.32 finish=164.58
306005) python3 cpu=12 start=5.32 finish=164.58
306006) python3 cpu=6 start=5.32 finish=164.58
306007) python3 cpu=27 start=5.32 finish=164.58
306008) python3 cpu=7 start=5.32 finish=164.58
306009) python3 cpu=2 start=5.32 finish=164.58
306010) python3 cpu=31 start=5.32 finish=164.58
306011) python3 cpu=1 start=5.32 finish=164.58
306012) python3 cpu=26 start=5.32 finish=164.58
306013) python3 cpu=9 start=5.32 finish=164.58
306014) python3 cpu=15 start=5.32 finish=164.58
306015) python3 cpu=30 start=5.32 finish=164.58
306016) python3 cpu=14 start=5.32 finish=164.58
306017) python3 cpu=24 start=5.32 finish=164.58
306069) [pango] FcInit cpu=-1 start=164.49 finish=164.49
306070) gmain cpu=11 start=164.49 finish=164.58
306071) dconf worker cpu=-1 start=164.49 finish=164.58
306072) gdbus cpu=0 start=164.49 finish=164.58
306073) pool cpu=29 start=164.50 finish=164.58
306074) python3 cpu=31 start=164.51 finish=164.51
306075) python3 cpu=20 start=164.51 finish=164.51
306076) python3 cpu=31 start=164.51 finish=164.51
306077) python3 cpu=21 start=164.51 finish=164.51