{"id":1727,"date":"2024-02-11T12:55:53","date_gmt":"2024-02-11T12:55:53","guid":{"rendered":"https:\/\/mvermeulen.org\/perf\/?page_id=1727"},"modified":"2024-02-11T15:42:16","modified_gmt":"2024-02-11T15:42:16","slug":"cockroach","status":"publish","type":"page","link":"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/cockroach\/","title":{"rendered":"cockroach"},"content":{"rendered":"\n<p>A test of a cloud-native distributed SQL database.  Looks like a version from cockroachlabs is <a href=\"https:\/\/www.cockroachlabs.com\/\">here<\/a>. This runs on a local node trying different workloads. I picked 60% read and 256 concurrency. Looks like a variable number of runable processes that sometimes exceeds the available cores.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-53.png\" alt=\"\" class=\"wp-image-1730\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-53.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-53-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-53-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Topdown profile looks to be dominated by frontend stalls with a low retirement rate.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-55.png\" alt=\"\" class=\"wp-image-1732\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-55.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-55-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-55-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Topdown metrics show a little over half the cores busy. The IPC and retirement rate is low with frontend latency high. There is still a moderate L2 access rate and L2 miss rate<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              334.639\non_cpu               0.597          # 9.56 \/ 16 cores\nutime                2696.582\nstime                502.219\nnvcsw                18549411       # 76.74%\nnivcsw               5620919        # 23.26%\ninblock              0              # 0.00\/sec\nonblock              4620296        # 13806.79\/sec\ncpu-clock            4520258374771  # 4520.258 seconds\ntask-clock           4522943060763  # 4522.943 seconds\npage faults          1432061        # 316.621\/sec\ncontext switches     27854844       # 6158.566\/sec\ncpu migrations       2728206        # 603.193\/sec\nmajor page faults    4              # 0.001\/sec\nminor page faults    1432057        # 316.621\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             1974030303012  # 201.912 branches per 1000 inst\nbranch misses        102407170177   # 5.19% branch miss\nconditional          1284689450537  # 131.404 conditional branches per 1000 inst\nindirect             40873144438    # 4.181 indirect branches per 1000 inst\ncpu-cycles           19435892478859 # 3.63 GHz\ninstructions         9809598549648  # 0.50 IPC low\nslots                38801128676808 #\nretiring             3318868095039  #  8.6% ( 9.4%) low\n-- ucode             9022406545     #     0.0%\n-- fastpath          3309845688494  #     8.5%\nfrontend             22335021009203 # 57.6% (63.2%) high\n-- latency           18761127031386 #    48.4%\n-- bandwidth         3573893977817  #     9.2%\nbackend              9023155560220  # 23.3% (25.5%)\n-- cpu               1160790156481  #     3.0%\n-- memory            7862365403739  #    20.3%\nspeculation          658238820427   #  1.7% ( 1.9%)\n-- branch mispredict 649146024898   #     1.7%\n-- pipeline restart  9092795529     #     0.0%\nsmt-contention       3464824277011  #  8.9% ( 0.0%)\ncpu-cycles           19420934695654 # 3.63 GHz\ninstructions         9799859808336  # 0.50 IPC low\ninstructions         3263133452367  # 116.177 l2 access per 1000 inst\nl2 hit from l1       337971873728   # 46.20% l2 miss\nl2 miss from l1      148712430714   #\nl2 hit from l2 pf    14700493681    #\nl3 hit from l2 pf    17772088522    #\nl3 miss from l2 pf   8655223043     #\ninstructions         3257890384531  # 45.130 float per 1000 inst\nfloat 512            51             # 0.000 AVX-512 per 1000 inst\nfloat 256            594            # 0.000 AVX-256 per 1000 inst\nfloat 128            147029392456   # 45.130 AVX-128 per 1000 inst\nfloat MMX            0              # 0.000 MMX per 1000 inst\nfloat scalar         0              # 0.000 scalar per 1000 inst\ninstructions         2397986        #\nopcache              896597         # 373.896 opcache per 1000 inst\nopcache miss         473659         # 52.8% opcache miss rate\nl1 dTLB miss         4325           # 1.804 L1 dTLB per 1000 inst\nl2 dTLB miss         1045           # 0.436 L2 dTLB per 1000 inst\ninstructions         2408738        #\nicache               1178593        # 489.299 icache per 1000 inst\nicache miss          107606         #  9.1% icache miss rate\nl1 iTLB miss         13             # 0.005 L1 iTLB per 1000 inst\nl2 iTLB miss         0              # 0.000 L2 iTLB per 1000 inst\ntlb flush            19             # 0.008 TLB flush per 1000 inst\n<\/code><\/pre>\n\n\n\n<p>Intel metrics<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              335.440\non_cpu               0.086          # 1.37 \/ 16 cores\nutime                242.311\nstime                217.011\nnvcsw                15997404       # 98.96%\nnivcsw               168694         # 1.04%\ninblock              16             # 0.05\/sec\nonblock              1608           # 4.79\/sec\ncpu-clock            4503641628988  # 4503.642 seconds\ntask-clock           4505774179785  # 4505.774 seconds\npage faults          1379992        # 306.272\/sec\ncontext switches     26579581       # 5899.004\/sec\ncpu migrations       3427600        # 760.713\/sec\nmajor page faults    0              # 0.000\/sec\nminor page faults    1379992        # 306.272\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             1675102884730  # 199.003 branches per 1000 inst\nbranch misses        57253165540    # 3.42% branch miss\nconditional          1675102907834  # 199.003 conditional branches per 1000 inst\nindirect             412498493591   # 49.005 indirect branches per 1000 inst\nslots                21972129346064 #\nretiring             5215647780921  # 23.7% (23.7%)\n-- ucode             538637393473   #     2.5%\n-- fastpath          4677010387448  #    21.3%\nfrontend             11430702200923 # 52.0% (52.0%) high\n-- latency           8623027187045  #    39.2%\n-- bandwidth         2807675013878  #    12.8%\nbackend              3256786565006  # 14.8% (14.8%) low\n-- cpu               1180244257814  #     5.4%\n-- memory            2076542307192  #     9.5%\nspeculation          2494472949221  # 11.4% (11.4%) high\n-- branch mispredict 2423272684439  #    11.0%\n-- pipeline restart  71200264782    #     0.3%\nsmt-contention       0              #  0.0% ( 0.0%)\ncpu-cycles           14285580800826 # 2.67 GHz\ninstructions         9581793809048  # 0.67 IPC low\nl2 access            586092773541   # 124.393 l2 access per 1000 inst\nl2 miss              287650250348   # 49.08% l2 miss\ncpu-cycles           7019364076935  # 44.0% memory latency\nload stalls          3009622724986  #  3.4% l1 bound\nl1 miss              2769044358629  # 11.7% l2 bound\nl2 miss              1950347371896  # 13.5% l3 bound\nl3 miss              1005857251457  # 14.3% dram bound\nstore_stalls         77504801801    #  1.1% store bound\n<\/code><\/pre>\n\n\n\n<p>Process overview shows the cockroach process with the most time.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>632 processes\n\t381 cockroach            125949.13 34009.39\n\t 38 vulkaninfo               1.50     1.14\n\t  6 glxinfo:gdrv0            0.15     0.12\n\t  4 vulkani:disk$0           0.15     0.12\n\t  2 llvmpipe-0               0.08     0.06\n\t  2 llvmpipe-1               0.08     0.06\n\t  2 llvmpipe-10              0.08     0.06\n\t  2 llvmpipe-11              0.08     0.06\n\t  2 llvmpipe-12              0.08     0.06\n\t  2 llvmpipe-13              0.08     0.06\n\t  2 llvmpipe-14              0.08     0.06\n\t  2 llvmpipe-15              0.08     0.06\n\t  2 llvmpipe-2               0.08     0.06\n\t  2 llvmpipe-3               0.08     0.06\n\t  2 llvmpipe-4               0.08     0.06\n\t  2 llvmpipe-5               0.08     0.06\n\t  2 llvmpipe-6               0.08     0.06\n\t  2 llvmpipe-7               0.08     0.06\n\t  2 llvmpipe-8               0.08     0.06\n\t  2 llvmpipe-9               0.08     0.06\n\t  2 glxinfo                  0.08     0.05\n\t  2 glxinfo:cs0              0.08     0.05\n\t  2 glxinfo:disk$0           0.07     0.04\n\t  2 glxinfo:sh0              0.07     0.04\n\t  2 glxinfo:shlo0            0.07     0.04\n\t  6 php                      0.06     0.08\n\t  1 lspci                    0.01     0.02\n\t  3 rm                       0.00     0.11\n\t  1 ps                       0.00     0.01\n\t 66 sh                       0.00     0.00\n\t 13 gcc                      0.00     0.00\n\t  8 stat                     0.00     0.00\n\t  8 systemd-detect-          0.00     0.00\n\t  6 gsettings                0.00     0.00\n\t  6 sleep                    0.00     0.00\n\t  5 dconf worker             0.00     0.00\n\t  5 gmain                    0.00     0.00\n\t  5 phoronix-test-s          0.00     0.00\n\t  2 cc                       0.00     0.00\n\t  2 lscpu                    0.00     0.00\n\t  2 uname                    0.00     0.00\n\t  2 which                    0.00     0.00\n\t  2 xset                     0.00     0.00\n\t  1 date                     0.00     0.00\n\t  1 dirname                  0.00     0.00\n\t  1 dmesg                    0.00     0.00\n\t  1 dmidecode                0.00     0.00\n\t  1 grep                     0.00     0.00\n\t  1 ifconfig                 0.00     0.00\n\t  1 ip                       0.00     0.00\n\t  1 lsmod                    0.00     0.00\n\t  1 mktemp                   0.00     0.00\n\t  1 readlink                 0.00     0.00\n\t  1 realpath                 0.00     0.00\n\t  1 sed                      0.00     0.00\n\t  1 sort                     0.00     0.00\n\t  1 stty                     0.00     0.00\n\t  1 systemctl                0.00     0.00\n\t  1 template.sh              0.00     0.00\n\t  1 wc                       0.00     0.00\n\t  1 xrandr                   0.00     0.00\n0 processes running\n144 maximum processes\n<\/code><\/pre>\n\n\n\n<p>Process structure for computation<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>      88678) cockroach        cpu=1 start=4.80  finish=111.41\n        88679) cockroach        cpu=9 start=4.80  finish=112.07\n          88681) cockroach        cpu=1 start=4.80  finish=112.07\n            88791) cockroach        cpu=8 start=60.34 finish=112.07\n            88792) cockroach        cpu=0 start=62.27 finish=112.07\n          88682) cockroach        cpu=10 start=4.80  finish=112.07\n            88685) cockroach        cpu=5 start=4.80  finish=112.07\n            88687) cockroach        cpu=5 start=4.81  finish=112.07\n            88689) cockroach        cpu=3 start=4.81  finish=112.07\n          88683) cockroach        cpu=0 start=4.80  finish=112.07\n          88684) cockroach        cpu=12 start=4.80  finish=112.07\n            88688) cockroach        cpu=1 start=4.81  finish=112.07\n              88691) cockroach        cpu=6 start=4.81  finish=112.07\n                88695) cockroach        cpu=11 start=4.83  finish=112.07\n                  88696) cockroach        cpu=15 start=4.83  finish=112.07\n                88697) cockroach        cpu=14 start=4.83  finish=112.07\n                88787) cockroach        cpu=5 start=11.85 finish=112.07\n              88788) cockroach        cpu=2 start=13.10 finish=112.07\n              88789) cockroach        cpu=0 start=18.24 finish=112.07\n          88686) cockroach        cpu=12 start=4.81  finish=112.07\n          88690) cockroach        cpu=13 start=4.81  finish=112.07\n            88692) cockroach        cpu=8 start=4.82  finish=112.07\n              88693) cockroach        cpu=9 start=4.82  finish=112.07\n                88694) cockroach        cpu=10 start=4.82  finish=112.07\n              88698) cockroach        cpu=7 start=4.85  finish=112.07\n                88699) cockroach        cpu=3 start=4.85  finish=112.07\n                88790) cockroach        cpu=1 start=23.34 finish=112.07\n          88700) cockroach        cpu=4 start=4.87  finish=112.07\n        88680) sleep            cpu=13 start=4.80  finish=9.80 \n        88701) cockroach        cpu=9 start=9.80  finish=110.36\n          88702) cockroach        cpu=4 start=9.80  finish=110.36\n            88722) cockroach        cpu=4 start=10.22 finish=110.36\n            88723) cockroach        cpu=10 start=10.22 finish=110.36\n            88724) cockroach        cpu=5 start=10.22 finish=110.36\n            88725) cockroach        cpu=11 start=10.22 finish=110.36\n            88726) cockroach        cpu=15 start=10.22 finish=110.36\n            88727) cockroach        cpu=6 start=10.22 finish=110.36\n            88728) cockroach        cpu=2 start=10.22 finish=110.36\n            88729) cockroach        cpu=12 start=10.22 finish=110.36\n            88730) cockroach        cpu=9 start=10.22 finish=110.36\n            88731) cockroach        cpu=4 start=10.22 finish=110.36\n            88732) cockroach        cpu=0 start=10.22 finish=110.36\n            88733) cockroach        cpu=6 start=10.23 finish=110.36\n            88734) cockroach        cpu=7 start=10.23 finish=110.36\n            88735) cockroach        cpu=5 start=10.23 finish=110.36\n            88736) cockroach        cpu=4 start=10.23 finish=110.36\n            88737) cockroach        cpu=1 start=10.23 finish=110.35\n            88738) cockroach        cpu=5 start=10.23 finish=110.35\n            88739) cockroach        cpu=2 start=10.23 finish=110.35\n            88740) cockroach        cpu=10 start=10.23 finish=110.35\n            88741) cockroach        cpu=2 start=10.23 finish=110.35\n            88742) cockroach        cpu=2 start=10.23 finish=110.35\n            88743) cockroach        cpu=15 start=10.23 finish=110.35\n            88744) cockroach        cpu=9 start=10.23 finish=110.35\n            88745) cockroach        cpu=10 start=10.23 finish=110.35\n            88746) cockroach        cpu=5 start=10.23 finish=110.35\n            88747) cockroach        cpu=3 start=10.23 finish=110.35\n            88748) cockroach        cpu=1 start=10.23 finish=110.35\n            88749) cockroach        cpu=5 start=10.23 finish=110.35\n            88750) cockroach        cpu=11 start=10.23 finish=110.35\n            88751) cockroach        cpu=0 start=10.23 finish=110.35\n            88752) cockroach        cpu=10 start=10.23 finish=110.35\n            88753) cockroach        cpu=7 start=10.23 finish=110.35\n            88754) cockroach        cpu=5 start=10.23 finish=110.35\n            88755) cockroach        cpu=10 start=10.23 finish=110.35\n            88756) cockroach        cpu=3 start=10.23 finish=110.35\n            88757) cockroach        cpu=9 start=10.23 finish=110.35\n            88758) cockroach        cpu=7 start=10.23 finish=110.35\n            88759) cockroach        cpu=5 start=10.23 finish=110.35\n            88760) cockroach        cpu=2 start=10.23 finish=110.35\n            88761) cockroach        cpu=7 start=10.23 finish=110.35\n            88762) cockroach        cpu=5 start=10.23 finish=110.35\n            88763) cockroach        cpu=0 start=10.23 finish=110.35\n            88764) cockroach        cpu=10 start=10.23 finish=110.35\n            88765) cockroach        cpu=12 start=10.23 finish=110.35\n            88766) cockroach        cpu=3 start=10.23 finish=110.35\n            88767) cockroach        cpu=0 start=10.23 finish=110.35\n            88768) cockroach        cpu=7 start=10.23 finish=110.35\n            88769) cockroach        cpu=12 start=10.23 finish=110.35\n            88770) cockroach        cpu=7 start=10.23 finish=110.35\n            88771) cockroach        cpu=14 start=10.23 finish=110.35\n            88772) cockroach        cpu=7 start=10.23 finish=110.35\n            88773) cockroach        cpu=5 start=10.23 finish=110.35\n            88774) cockroach        cpu=9 start=10.23 finish=110.35\n            88775) cockroach        cpu=7 start=10.23 finish=110.35\n            88776) cockroach        cpu=8 start=10.23 finish=110.35\n            88777) cockroach        cpu=9 start=10.23 finish=110.35\n            88778) cockroach        cpu=14 start=10.23 finish=110.35\n            88779) cockroach        cpu=0 start=10.23 finish=110.35\n            88780) cockroach        cpu=2 start=10.23 finish=110.35\n            88781) cockroach        cpu=8 start=10.23 finish=110.35\n            88782) cockroach        cpu=12 start=10.23 finish=110.35\n            88783) cockroach        cpu=4 start=10.23 finish=110.35\n            88784) cockroach        cpu=11 start=10.23 finish=110.35\n          88703) cockroach        cpu=2 start=9.80  finish=110.36\n            88705) cockroach        cpu=4 start=9.81  finish=110.36\n            88708) cockroach        cpu=9 start=9.81  finish=110.36\n              88714) cockroach        cpu=0 start=9.82  finish=110.36\n            88710) cockroach        cpu=5 start=9.81  finish=110.36\n              88715) cockroach        cpu=15 start=9.82  finish=110.36\n                88716) cockroach        cpu=9 start=9.82  finish=110.36\n              88717) cockroach        cpu=10 start=9.83  finish=110.36\n                88718) cockroach        cpu=4 start=9.83  finish=110.36\n                  88785) cockroach        cpu=1 start=10.29 finish=110.35\n          88704) cockroach        cpu=14 start=9.81  finish=110.36\n          88706) cockroach        cpu=11 start=9.81  finish=110.36\n            88709) cockroach        cpu=8 start=9.81  finish=110.36\n              88711) cockroach        cpu=0 start=9.81  finish=110.36\n                88712) cockroach        cpu=5 start=9.81  finish=110.36\n                  88719) cockroach        cpu=3 start=9.85  finish=110.36\n                    88720) cockroach        cpu=7 start=9.85  finish=110.36\n              88713) cockroach        cpu=7 start=9.81  finish=110.36\n          88707) cockroach        cpu=8 start=9.81  finish=110.36\n            88721) cockroach        cpu=12 start=9.87  finish=110.36\n        88793) sleep            cpu=8 start=110.37 finish=111.37\n        88794) rm               cpu=2 start=111.37 finish=111.41\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>A test of a cloud-native distributed SQL database. Looks like a version from cockroachlabs is here. This runs on a local node trying different workloads. I picked 60% read and 256 concurrency. Looks like a variable number of runable processes <span class=\"excerpt-dots\">&hellip;<\/span> <a class=\"more-link\" href=\"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/cockroach\/\"><span class=\"more-msg\">Continue reading &rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":58,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1727","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1727","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/comments?post=1727"}],"version-history":[{"count":4,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1727\/revisions"}],"predecessor-version":[{"id":1742,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1727\/revisions\/1742"}],"up":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/58"}],"wp:attachment":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/media?parent=1727"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}