{"id":1765,"date":"2024-02-23T01:17:32","date_gmt":"2024-02-23T01:17:32","guid":{"rendered":"https:\/\/mvermeulen.org\/perf\/?page_id=1765"},"modified":"2024-02-27T00:38:50","modified_gmt":"2024-02-27T00:38:50","slug":"clickhouse","status":"publish","type":"page","link":"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/clickhouse\/","title":{"rendered":"clickhouse"},"content":{"rendered":"\n<p>Query processing time for queries of an OLAP data management system.  Appears that each test had distinct phases of more and less parallel activity<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-59.png\" alt=\"\" class=\"wp-image-1794\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-59.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-59-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/systemtime-59-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>Topdown profile also shows the phases with an interesting blend of frontend and backend stages<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"960\" src=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-61.png\" alt=\"\" class=\"wp-image-1796\" srcset=\"https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-61.png 1280w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-61-1024x768.png 1024w, https:\/\/mvermeulen.org\/perf\/wp-content\/uploads\/sites\/7\/2024\/02\/amdtopdown-61-768x576.png 768w\" sizes=\"auto, (max-width: 1280px) 100vw, 1280px\" \/><\/figure>\n\n\n\n<p>AMD metrics overall shows on average only 1.3 cores busy, more backend\/memory stalls and relatively low floating point.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              960.074\non_cpu               0.083          # 1.33 \/ 16 cores\nutime                1050.202\nstime                223.703\nnvcsw                182921         # 96.17%\nnivcsw               7293           # 3.83%\ninblock              428117168      # 445920.81\/sec\nonblock              14064          # 14.65\/sec\ncpu-clock            9062722205373  # 9062.722 seconds\ntask-clock           9085391684374  # 9085.392 seconds\npage faults          125632193      # 13827.934\/sec\ncontext switches     80772494       # 8890.370\/sec\ncpu migrations       7056537        # 776.690\/sec\nmajor page faults    87             # 0.010\/sec\nminor page faults    125632106      # 13827.924\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             7589246834681  # 172.854 branches per 1000 inst\nbranch misses        196395901361   # 2.59% branch miss\nconditional          5507529190694  # 125.441 conditional branches per 1000 inst\nindirect             356469729953   # 8.119 indirect branches per 1000 inst\ncpu-cycles           37036879736615 # 2.43 GHz\ninstructions         44072109601039 # 1.19 IPC\nslots                73730064178218 #\nretiring             14063016807866 # 19.1% (22.1%)\n-- ucode             71601765390    #     0.1%\n-- fastpath          13991415042476 #    19.0%\nfrontend             15752422857947 # 21.4% (24.8%)\n-- latency           10417400144580 #    14.1%\n-- bandwidth         5335022713367  #     7.2%\nbackend              31122778866231 # 42.2% (48.9%)\n-- cpu               4439924407384  #     6.0%\n-- memory            26682854458847 #    36.2%\nspeculation          2650850523671  #  3.6% ( 4.2%)\n-- branch mispredict 2622235580370  #     3.6%\n-- pipeline restart  28614943301    #     0.0%\nsmt-contention       10135178456104 # 13.7% ( 0.0%)\ncpu-cycles           37080085833937 # 2.44 GHz\ninstructions         44036533320560 # 1.19 IPC\ninstructions         14570618208965 # 30.513 l2 access per 1000 inst\nl2 hit from l1       317014346372   # 23.13% l2 miss\nl2 miss from l1      35934414613    #\nl2 hit from l2 pf    60663061403    #\nl3 hit from l2 pf    23003739509    #\nl3 miss from l2 pf   43907858942    #\ninstructions         14556681449273 # 44.261 float per 1000 inst\nfloat 512            242            # 0.000 AVX-512 per 1000 inst\nfloat 256            1264           # 0.000 AVX-256 per 1000 inst\nfloat 128            644292753045   # 44.261 AVX-128 per 1000 inst\nfloat MMX            0              # 0.000 MMX per 1000 inst\nfloat scalar         1123           # 0.000 scalar per 1000 inst\ninstructions         2646816        #\nopcache              981664         # 370.885 opcache per 1000 inst\nopcache miss         524554         # 53.4% opcache miss rate\nl1 dTLB miss         4803           # 1.815 L1 dTLB per 1000 inst\nl2 dTLB miss         1023           # 0.387 L2 dTLB per 1000 inst\ninstructions         2672615        #\nicache               1295488        # 484.727 icache per 1000 inst\nicache miss          110556         #  8.5% icache miss rate\nl1 iTLB miss         13             # 0.005 L1 iTLB per 1000 inst\nl2 iTLB miss         0              # 0.000 L2 iTLB per 1000 inst\ntlb flush            19             # 0.007 TLB flush per 1000 inst\n<\/code><\/pre>\n\n\n\n<p>Intel metrics<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>elapsed              1374.763\non_cpu               0.043          # 0.68 \/ 16 cores\nutime                796.202\nstime                145.509\nnvcsw                424427         # 91.93%\nnivcsw               37273          # 8.07%\ninblock              437802064      # 318456.44\/sec\nonblock              2664           # 1.94\/sec\ncpu-clock            9889890146722  # 9889.890 seconds\ntask-clock           9901821096493  # 9901.821 seconds\npage faults          145004155      # 14644.191\/sec\ncontext switches     64420418       # 6505.916\/sec\ncpu migrations       3817346        # 385.520\/sec\nmajor page faults    12835          # 1.296\/sec\nminor page faults    144991320      # 14642.894\/sec\nalignment faults     0              # 0.000\/sec\nemulation faults     0              # 0.000\/sec\nbranches             7494281421376  # 170.479 branches per 1000 inst\nbranch misses        131701279317   # 1.76% branch miss\nconditional          7494281482688  # 170.479 conditional branches per 1000 inst\nindirect             1701207301554  # 38.699 indirect branches per 1000 inst\nslots                117679341351368 #\nretiring             45920805761942 # 39.0% (39.0%)\n-- ucode             2071606390469  #     1.8%\n-- fastpath          43849199371473 #    37.3%\nfrontend             15208522024469 # 12.9% (12.9%)\n-- latency           7486087889483  #     6.4%\n-- bandwidth         7722434134986  #     6.6%\nbackend              39841002160783 # 33.9% (33.9%)\n-- cpu               8222436983285  #     7.0%\n-- memory            31618565177498 #    26.9%\nspeculation          21118708940051 # 17.9% (17.9%) high\n-- branch mispredict 20590473635436 #    17.5%\n-- pipeline restart  528235304615   #     0.4%\nsmt-contention       0              #  0.0% ( 0.0%)\ncpu-cycles           95249425375812 # 1.42 GHz\ninstructions         161207153689135 # 1.69 IPC\nl2 access            1882074708088  # 22.041 l2 access per 1000 inst\nl2 miss              800276946596   # 42.52% l2 miss\ncpu-cycles           16776072624718 # 43.4% memory latency\nload stalls          6771705251692  #  3.6% l1 bound\nl1 miss              6174069101775  #  6.3% l2 bound\nl2 miss              5110846415218  #  4.0% l3 bound\nl3 miss              4442788325447  # 26.5% dram bound\nstore_stalls         508058730601   #  3.0% store bound\n<\/code><\/pre>\n\n\n\n<p>Process summary shows mix of processes in the application<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>1896 processes\n\t384 BgSchPool            832938.24 132876.96\n\t279 ThreadPool           532398.14 85425.54\n\t 48 MergeMutate          104117.22 16609.76\n\t429 clickhouse           66127.54 10586.94\n\t 24 Fetch                52058.64  8304.85\n\t 24 Common               52058.64  8304.83\n\t 24 Move                 52058.62  8304.88\n\t  6 ExterLdrReload       13014.66  2076.22\n\t  6 ConfigReloader       13014.64  2076.20\n\t  3 BgSchPool\/D           6507.33  1038.11\n\t  3 TCPHandler            6507.33  1038.11\n\t  3 AsyncMetrics          6507.32  1038.10\n\t 68 clinfo                  16.79     6.00\n\t 38 vulkaninfo               1.70     0.95\n\t  4 vulkani:disk$0           0.18     0.10\n\t  6 php                      0.15     0.15\n\t  6 glxinfo:gdrv0            0.12     0.06\n\t  6 glxinfo:gl0              0.12     0.06\n\t  2 llvmpipe-0               0.09     0.05\n\t  2 llvmpipe-1               0.09     0.05\n\t  2 llvmpipe-10              0.09     0.05\n\t  2 llvmpipe-11              0.09     0.05\n\t  2 llvmpipe-12              0.09     0.05\n\t  2 llvmpipe-13              0.09     0.05\n\t  2 llvmpipe-14              0.09     0.05\n\t  2 llvmpipe-15              0.09     0.05\n\t  2 llvmpipe-2               0.09     0.05\n\t  2 llvmpipe-3               0.09     0.05\n\t  2 llvmpipe-4               0.09     0.05\n\t  2 llvmpipe-5               0.09     0.05\n\t  2 llvmpipe-6               0.09     0.05\n\t  2 llvmpipe-7               0.09     0.05\n\t  2 llvmpipe-8               0.09     0.05\n\t  2 llvmpipe-9               0.09     0.05\n\t 48 rm                       0.06    11.24\n\t  2 glxinfo                  0.06     0.02\n\t  2 glxinfo:cs0              0.06     0.02\n\t  2 glxinfo:disk$0           0.06     0.02\n\t  2 glxinfo:sh0              0.06     0.02\n\t  2 glxinfo:shlo0            0.06     0.02\n\t  6 clang                    0.04     0.08\n\t  3 clckhouse-watch          0.03     0.03\n\t  3 rocminfo                 0.00     0.03\n\t  1 lspci                    0.00     0.02\n\t  1 ps                       0.00     0.01\n\t129 seq                      0.00     0.00\n\t129 sync                     0.00     0.00\n\t 81 sh                       0.00     0.00\n\t 14 gsettings                0.00     0.00\n\t 12 gcc                      0.00     0.00\n\t  8 stat                     0.00     0.00\n\t  8 systemd-detect-          0.00     0.00\n\t  6 llvm-link                0.00     0.00\n\t  6 sleep                    0.00     0.00\n\t  5 phoronix-test-s          0.00     0.00\n\t  3 cat                      0.00     0.00\n\t  2 lscpu                    0.00     0.00\n\t  2 uname                    0.00     0.00\n\t  2 which                    0.00     0.00\n\t  2 xset                     0.00     0.00\n\t  1 cc                       0.00     0.00\n\t  1 date                     0.00     0.00\n\t  1 dconf worker             0.00     0.00\n\t  1 dirname                  0.00     0.00\n\t  1 dmesg                    0.00     0.00\n\t  1 dmidecode                0.00     0.00\n\t  1 gmain                    0.00     0.00\n\t  1 grep                     0.00     0.00\n\t  1 ifconfig                 0.00     0.00\n\t  1 ip                       0.00     0.00\n\t  1 lsmod                    0.00     0.00\n\t  1 mktemp                   0.00     0.00\n\t  1 qdbus                    0.00     0.00\n\t  1 readlink                 0.00     0.00\n\t  1 realpath                 0.00     0.00\n\t  1 sed                      0.00     0.00\n\t  1 sort                     0.00     0.00\n\t  1 stty                     0.00     0.00\n\t  1 systemctl                0.00     0.00\n\t  1 template.sh              0.00     0.00\n\t  1 wc                       0.00     0.00\n\t  1 xrandr                   0.00     0.00\n0 processes running\n280 maximum processes\n<\/code><\/pre>\n\n\n\n<p>Example computation blocks show a backend set of processes such as thread pools and then a sequence of frontend application threads.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>      26414) clickhouse       cpu=9 start=5.74  finish=315.77\n        26415) rm               cpu=10 start=5.74  finish=5.74 \n        26416) rm               cpu=13 start=5.74  finish=5.74 \n        26417) rm               cpu=14 start=5.74  finish=5.74 \n        26418) rm               cpu=10 start=5.75  finish=5.75 \n        26419) rm               cpu=3 start=5.75  finish=5.75 \n        26420) rm               cpu=13 start=5.75  finish=5.75 \n        26421) rm               cpu=14 start=5.75  finish=5.75 \n        26422) rm               cpu=10 start=5.75  finish=5.75 \n        26423) clckhouse-watch  cpu=3 start=5.75  finish=309.88\n          26425) clickhouse       cpu=12 start=5.78  finish=309.90\n            26426) clickhouse       cpu=7 start=5.78  finish=309.90\n            26427) TCPHandler       cpu=0 start=5.83  finish=309.90\n              26667) ThreadPool       cpu=2 start=180.26 finish=309.89\n                26702) ThreadPool       cpu=6 start=180.52 finish=309.89\n              26673) ThreadPool       cpu=13 start=180.36 finish=309.89\n              26674) ThreadPool       cpu=4 start=180.36 finish=309.89\n                26692) ThreadPool       cpu=14 start=180.37 finish=309.89\n              26675) ThreadPool       cpu=7 start=180.36 finish=309.89\n                26688) ThreadPool       cpu=10 start=180.37 finish=309.89\n              26676) ThreadPool       cpu=11 start=180.36 finish=309.89\n                26714) ThreadPool       cpu=1 start=180.74 finish=309.89\n              26677) ThreadPool       cpu=13 start=180.36 finish=309.89\n                26695) ThreadPool       cpu=13 start=180.38 finish=309.89\n                26720) ThreadPool       cpu=7 start=181.11 finish=309.89\n                26721) ThreadPool       cpu=1 start=181.11 finish=309.89\n                26722) ThreadPool       cpu=4 start=181.11 finish=309.89\n                26723) ThreadPool       cpu=9 start=181.11 finish=309.89\n                26724) ThreadPool       cpu=3 start=181.11 finish=309.89\n                26725) ThreadPool       cpu=14 start=181.11 finish=309.89\n                26726) ThreadPool       cpu=5 start=181.11 finish=309.89\n                26727) ThreadPool       cpu=6 start=181.11 finish=309.89\n                26728) ThreadPool       cpu=12 start=181.11 finish=309.89\n                26729) ThreadPool       cpu=6 start=181.11 finish=309.89\n                26730) ThreadPool       cpu=11 start=181.11 finish=309.89\n                26731) ThreadPool       cpu=0 start=181.11 finish=309.89\n                26732) ThreadPool       cpu=10 start=181.12 finish=309.89\n                26733) ThreadPool       cpu=5 start=181.12 finish=309.89\n                26734) ThreadPool       cpu=11 start=181.12 finish=309.89\n                26735) ThreadPool       cpu=0 start=181.12 finish=309.89\n              26678) ThreadPool       cpu=7 start=180.36 finish=309.89\n                26696) ThreadPool       cpu=12 start=180.38 finish=309.89\n              26679) ThreadPool       cpu=6 start=180.36 finish=309.89\n                26693) ThreadPool       cpu=3 start=180.38 finish=309.89\n                26713) ThreadPool       cpu=12 start=180.74 finish=309.89\n              26680) ThreadPool       cpu=14 start=180.36 finish=309.89\n              26681) ThreadPool       cpu=4 start=180.36 finish=309.89\n                26705) ThreadPool       cpu=2 start=180.52 finish=309.89\n              26682) ThreadPool       cpu=2 start=180.36 finish=309.89\n                26689) ThreadPool       cpu=11 start=180.37 finish=309.89\n                26704) ThreadPool       cpu=8 start=180.52 finish=309.89\n              26683) ThreadPool       cpu=11 start=180.36 finish=309.89\n              26684) ThreadPool       cpu=9 start=180.36 finish=309.89\n                26694) ThreadPool       cpu=7 start=180.38 finish=309.89\n                26712) ThreadPool       cpu=10 start=180.74 finish=309.89\n              26685) ThreadPool       cpu=8 start=180.36 finish=309.89\n                26690) ThreadPool       cpu=13 start=180.37 finish=309.89\n                26711) ThreadPool       cpu=1 start=180.70 finish=309.89\n              26686) ThreadPool       cpu=0 start=180.36 finish=309.89\n                26691) ThreadPool       cpu=12 start=180.37 finish=309.89\n              26687) ThreadPool       cpu=11 start=180.36 finish=309.89\n                26703) ThreadPool       cpu=14 start=180.52 finish=309.89\n            26428) clickhouse       cpu=14 start=5.83  finish=309.90\n            26429) clickhouse       cpu=1 start=5.83  finish=309.90\n            26430) ExterLdrReload   cpu=10 start=6.03  finish=309.90\n            26431) ExterLdrReload   cpu=9 start=6.03  finish=309.90\n            26432) Move             cpu=15 start=6.04  finish=309.90\n            26433) Move             cpu=11 start=6.04  finish=309.90\n            26434) MergeMutate      cpu=0 start=6.04  finish=309.90\n            26435) Move             cpu=5 start=6.04  finish=309.90\n            26436) MergeMutate      cpu=2 start=6.04  finish=309.90\n            26437) MergeMutate      cpu=13 start=6.04  finish=309.90\n            26438) MergeMutate      cpu=4 start=6.04  finish=309.90\n            26439) MergeMutate      cpu=6 start=6.04  finish=309.90\n            26440) MergeMutate      cpu=8 start=6.04  finish=309.90\n            26441) MergeMutate      cpu=0 start=6.04  finish=309.90\n            26442) MergeMutate      cpu=11 start=6.04  finish=309.90\n            26443) MergeMutate      cpu=4 start=6.04  finish=309.90\n            26444) Move             cpu=10 start=6.04  finish=309.90\n            26445) Move             cpu=15 start=6.04  finish=309.90\n            26446) MergeMutate      cpu=13 start=6.04  finish=309.90\n            26447) MergeMutate      cpu=12 start=6.04  finish=309.90\n            26448) MergeMutate      cpu=6 start=6.04  finish=309.90\n            26449) MergeMutate      cpu=11 start=6.04  finish=309.90\n            26450) MergeMutate      cpu=9 start=6.04  finish=309.90\n              26658) ThreadPool       cpu=11 start=88.55 finish=309.89\n            26451) MergeMutate      cpu=1 start=6.04  finish=309.90\n              26661) ThreadPool       cpu=4 start=137.03 finish=309.89\n            26452) Move             cpu=1 start=6.04  finish=309.90\n            26453) MergeMutate      cpu=7 start=6.04  finish=309.90\n            26454) Move             cpu=4 start=6.04  finish=309.90\n            26455) Move             cpu=0 start=6.04  finish=309.90\n            26456) Fetch            cpu=13 start=6.04  finish=309.90\n            26457) BgSchPool        cpu=1 start=6.04  finish=309.90\n            26458) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26459) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26460) Fetch            cpu=0 start=6.04  finish=309.90\n            26461) Fetch            cpu=0 start=6.04  finish=309.90\n            26462) Common           cpu=9 start=6.04  finish=309.90\n            26463) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26464) BgSchPool        cpu=5 start=6.04  finish=309.90\n            26465) ThreadPool       cpu=14 start=6.04  finish=309.90\n            26466) Fetch            cpu=2 start=6.04  finish=309.90\n            26467) ThreadPool       cpu=2 start=6.04  finish=309.90\n            26468) BgSchPool        cpu=8 start=6.04  finish=309.90\n            26469) BgSchPool        cpu=1 start=6.04  finish=309.90\n            26470) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26471) BgSchPool        cpu=13 start=6.04  finish=309.90\n            26472) BgSchPool\/D      cpu=2 start=6.04  finish=309.90\n            26473) Fetch            cpu=0 start=6.04  finish=309.90\n            26474) Common           cpu=3 start=6.04  finish=309.90\n            26475) ThreadPool       cpu=1 start=6.04  finish=309.90\n            26476) ThreadPool       cpu=1 start=6.04  finish=309.90\n            26477) BgSchPool        cpu=1 start=6.04  finish=309.90\n            26478) BgSchPool        cpu=4 start=6.04  finish=309.90\n            26479) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26480) BgSchPool        cpu=12 start=6.04  finish=309.90\n            26481) BgSchPool        cpu=10 start=6.04  finish=309.90\n            26482) BgSchPool        cpu=11 start=6.04  finish=309.90\n            26483) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26484) Common           cpu=4 start=6.04  finish=309.90\n            26485) BgSchPool        cpu=14 start=6.04  finish=309.90\n            26486) BgSchPool        cpu=12 start=6.04  finish=309.90\n            26487) BgSchPool        cpu=6 start=6.04  finish=309.90\n            26488) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26489) BgSchPool        cpu=7 start=6.04  finish=309.90\n            26490) BgSchPool        cpu=2 start=6.04  finish=309.90\n            26491) BgSchPool        cpu=8 start=6.04  finish=309.90\n            26492) BgSchPool        cpu=9 start=6.04  finish=309.90\n            26493) BgSchPool        cpu=10 start=6.04  finish=309.90\n            26494) BgSchPool        cpu=4 start=6.04  finish=309.90\n            26495) BgSchPool        cpu=14 start=6.04  finish=309.90\n            26496) BgSchPool        cpu=12 start=6.04  finish=309.90\n            26497) BgSchPool        cpu=11 start=6.04  finish=309.90\n            26498) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26499) BgSchPool        cpu=5 start=6.04  finish=309.90\n            26500) BgSchPool        cpu=6 start=6.04  finish=309.90\n            26501) ThreadPool       cpu=2 start=6.04  finish=309.90\n            26502) ThreadPool       cpu=4 start=6.04  finish=309.90\n            26503) Common           cpu=6 start=6.04  finish=309.90\n            26504) ThreadPool       cpu=14 start=6.04  finish=309.90\n            26505) BgSchPool        cpu=1 start=6.04  finish=309.90\n            26506) BgSchPool        cpu=7 start=6.04  finish=309.90\n            26507) BgSchPool        cpu=15 start=6.04  finish=309.90\n            26508) BgSchPool        cpu=5 start=6.04  finish=309.90\n            26509) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26510) BgSchPool        cpu=14 start=6.04  finish=309.90\n            26511) BgSchPool        cpu=12 start=6.04  finish=309.90\n            26512) BgSchPool        cpu=10 start=6.04  finish=309.90\n            26513) BgSchPool        cpu=5 start=6.04  finish=309.90\n            26514) ThreadPool       cpu=13 start=6.04  finish=309.90\n            26515) BgSchPool        cpu=9 start=6.04  finish=309.90\n            26516) BgSchPool        cpu=6 start=6.04  finish=309.90\n            26517) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26518) BgSchPool        cpu=4 start=6.04  finish=309.90\n            26519) BgSchPool        cpu=1 start=6.04  finish=309.90\n            26520) BgSchPool        cpu=9 start=6.04  finish=309.90\n            26521) BgSchPool        cpu=15 start=6.04  finish=309.90\n            26522) BgSchPool        cpu=14 start=6.04  finish=309.90\n            26523) BgSchPool        cpu=7 start=6.04  finish=309.90\n            26524) BgSchPool        cpu=8 start=6.04  finish=309.90\n            26525) BgSchPool        cpu=9 start=6.04  finish=309.90\n            26526) BgSchPool        cpu=12 start=6.04  finish=309.90\n            26527) BgSchPool        cpu=6 start=6.04  finish=309.90\n            26528) BgSchPool        cpu=15 start=6.04  finish=309.90\n            26529) BgSchPool        cpu=6 start=6.04  finish=309.90\n            26530) BgSchPool        cpu=11 start=6.04  finish=309.90\n            26531) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26532) BgSchPool        cpu=13 start=6.04  finish=309.90\n            26533) BgSchPool        cpu=5 start=6.04  finish=309.90\n            26534) BgSchPool        cpu=10 start=6.04  finish=309.90\n            26535) BgSchPool        cpu=6 start=6.04  finish=309.90\n            26536) BgSchPool        cpu=4 start=6.04  finish=309.90\n            26537) BgSchPool        cpu=9 start=6.04  finish=309.90\n            26538) ThreadPool       cpu=14 start=6.04  finish=309.90\n            26539) ThreadPool       cpu=15 start=6.04  finish=309.90\n            26540) Common           cpu=10 start=6.04  finish=309.90\n            26541) Fetch            cpu=10 start=6.04  finish=309.90\n            26542) ThreadPool       cpu=12 start=6.04  finish=309.90\n            26543) ThreadPool       cpu=13 start=6.04  finish=309.90\n            26544) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26545) BgSchPool        cpu=6 start=6.04  finish=309.90\n            26546) BgSchPool        cpu=4 start=6.04  finish=309.90\n            26547) BgSchPool        cpu=9 start=6.04  finish=309.90\n            26548) BgSchPool        cpu=3 start=6.04  finish=309.90\n            26549) BgSchPool        cpu=10 start=6.04  finish=309.90\n            26550) BgSchPool        cpu=15 start=6.04  finish=309.90\n            26551) BgSchPool        cpu=1 start=6.04  finish=309.90\n            26552) BgSchPool        cpu=9 start=6.04  finish=309.90\n            26553) BgSchPool        cpu=10 start=6.05  finish=309.90\n            26554) BgSchPool        cpu=8 start=6.05  finish=309.90\n            26555) BgSchPool        cpu=0 start=6.05  finish=309.90\n            26556) BgSchPool        cpu=14 start=6.05  finish=309.90\n            26557) BgSchPool        cpu=5 start=6.05  finish=309.90\n            26558) BgSchPool        cpu=10 start=6.05  finish=309.90\n            26559) BgSchPool        cpu=9 start=6.05  finish=309.90\n            26560) BgSchPool        cpu=11 start=6.05  finish=309.90\n            26561) BgSchPool        cpu=12 start=6.05  finish=309.89\n            26562) BgSchPool        cpu=6 start=6.05  finish=309.89\n            26563) BgSchPool        cpu=3 start=6.05  finish=309.89\n            26564) BgSchPool        cpu=4 start=6.05  finish=309.89\n            26565) BgSchPool        cpu=14 start=6.05  finish=309.89\n            26566) BgSchPool        cpu=3 start=6.05  finish=309.89\n            26567) BgSchPool        cpu=2 start=6.05  finish=309.89\n            26568) BgSchPool        cpu=3 start=6.05  finish=309.89\n            26569) BgSchPool        cpu=7 start=6.05  finish=309.89\n            26570) BgSchPool        cpu=7 start=6.05  finish=309.89\n            26571) BgSchPool        cpu=8 start=6.05  finish=309.89\n            26572) BgSchPool        cpu=5 start=6.05  finish=309.89\n            26573) BgSchPool        cpu=7 start=6.05  finish=309.89\n            26574) BgSchPool        cpu=11 start=6.05  finish=309.89\n            26575) BgSchPool        cpu=5 start=6.05  finish=309.89\n            26576) BgSchPool        cpu=8 start=6.05  finish=309.89\n            26577) BgSchPool        cpu=2 start=6.05  finish=309.89\n            26578) BgSchPool        cpu=8 start=6.05  finish=309.89\n            26579) BgSchPool        cpu=2 start=6.05  finish=309.89\n            26580) BgSchPool        cpu=9 start=6.05  finish=309.89\n            26581) BgSchPool        cpu=14 start=6.05  finish=309.89\n            26582) BgSchPool        cpu=15 start=6.05  finish=309.89\n            26583) BgSchPool        cpu=15 start=6.05  finish=309.89\n            26584) BgSchPool        cpu=13 start=6.05  finish=309.89\n            26585) BgSchPool        cpu=3 start=6.05  finish=309.89\n            26586) BgSchPool        cpu=5 start=6.05  finish=309.89\n            26587) BgSchPool        cpu=12 start=6.05  finish=309.89\n            26588) BgSchPool        cpu=11 start=6.05  finish=309.89\n            26589) BgSchPool        cpu=13 start=6.05  finish=309.89\n            26590) BgSchPool        cpu=6 start=6.05  finish=309.89\n            26591) BgSchPool        cpu=9 start=6.05  finish=309.89\n            26592) BgSchPool        cpu=3 start=6.05  finish=309.89\n            26593) BgSchPool        cpu=9 start=6.05  finish=309.89\n            26594) BgSchPool        cpu=10 start=6.05  finish=309.89\n            26595) BgSchPool        cpu=1 start=6.05  finish=309.89\n            26596) BgSchPool        cpu=8 start=6.05  finish=309.89\n            26597) BgSchPool        cpu=1 start=6.05  finish=309.89\n            26598) BgSchPool        cpu=1 start=6.05  finish=309.89\n            26599) BgSchPool        cpu=5 start=6.05  finish=309.89\n            26600) BgSchPool        cpu=10 start=6.05  finish=309.89\n            26601) BgSchPool        cpu=2 start=6.05  finish=309.89\n            26602) BgSchPool        cpu=15 start=6.05  finish=309.89\n            26603) BgSchPool        cpu=13 start=6.05  finish=309.89\n            26604) BgSchPool        cpu=12 start=6.05  finish=309.89\n            26605) BgSchPool        cpu=5 start=6.05  finish=309.89\n            26606) BgSchPool        cpu=6 start=6.05  finish=309.89\n            26607) BgSchPool        cpu=2 start=6.05  finish=309.89\n            26608) ThreadPool       cpu=0 start=6.05  finish=309.89\n            26609) ThreadPool       cpu=11 start=6.05  finish=309.89\n            26610) ThreadPool       cpu=9 start=6.05  finish=309.89\n            26611) ThreadPool       cpu=11 start=6.05  finish=309.89\n            26612) Common           cpu=11 start=6.05  finish=309.89\n            26613) Common           cpu=8 start=6.05  finish=309.89\n            26614) Common           cpu=5 start=6.05  finish=309.89\n            26615) Fetch            cpu=9 start=6.05  finish=309.89\n            26616) Fetch            cpu=13 start=6.05  finish=309.89\n            26617) ThreadPool       cpu=11 start=6.17  finish=309.89\n            26618) ThreadPool       cpu=5 start=6.17  finish=309.89\n            26619) ThreadPool       cpu=8 start=6.17  finish=309.89\n            26620) ThreadPool       cpu=15 start=6.17  finish=309.89\n            26621) ThreadPool       cpu=5 start=6.17  finish=309.89\n            26622) ThreadPool       cpu=7 start=6.17  finish=309.89\n            26623) ThreadPool       cpu=7 start=6.17  finish=309.89\n            26624) ThreadPool       cpu=11 start=6.17  finish=309.89\n            26625) ConfigReloader   cpu=3 start=6.17  finish=309.89\n            26626) ThreadPool       cpu=5 start=6.17  finish=309.89\n            26627) ConfigReloader   cpu=11 start=6.17  finish=309.89\n            26628) ThreadPool       cpu=7 start=6.17  finish=309.89\n            26629) ThreadPool       cpu=13 start=6.17  finish=309.89\n            26630) ThreadPool       cpu=4 start=6.17  finish=309.89\n            26631) ThreadPool       cpu=12 start=6.17  finish=309.89\n            26632) AsyncMetrics     cpu=1 start=6.17  finish=309.89\n            26633) clickhouse       cpu=8 start=6.34  finish=309.89\n            26634) clickhouse       cpu=0 start=6.34  finish=309.89\n            26635) clickhouse       cpu=7 start=6.34  finish=309.89\n            26636) clickhouse       cpu=5 start=6.34  finish=309.89\n            26637) clickhouse       cpu=7 start=6.34  finish=309.89\n            26638) clickhouse       cpu=6 start=6.34  finish=309.89\n        26424) sleep            cpu=13 start=5.75  finish=10.76\n        26641) clickhouse       cpu=9 start=10.76 finish=10.81\n        26642) clickhouse       cpu=4 start=10.81 finish=179.97\n          26643) ThreadPool       cpu=1 start=10.85 finish=179.95\n            26644) ThreadPool       cpu=3 start=10.85 finish=179.97\n              26645) ThreadPool       cpu=3 start=10.88 finish=179.97\n              26646) ThreadPool       cpu=9 start=10.96 finish=179.96\n              26647) ThreadPool       cpu=14 start=10.97 finish=179.94\n              26648) ThreadPool       cpu=7 start=11.21 finish=179.97\n              26649) ThreadPool       cpu=8 start=11.23 finish=179.97\n              26650) ThreadPool       cpu=6 start=11.25 finish=179.97\n              26651) ThreadPool       cpu=2 start=11.82 finish=179.97\n              26652) ThreadPool       cpu=13 start=13.10 finish=179.97\n              26653) ThreadPool       cpu=11 start=34.79 finish=179.97\n              26654) ThreadPool       cpu=15 start=34.80 finish=179.96\n              26655) ThreadPool       cpu=15 start=34.81 finish=179.97\n              26656) ThreadPool       cpu=5 start=34.84 finish=179.97\n        26662) cat              cpu=12 start=179.98 finish=179.98\n        26663) clickhouse       cpu=0 start=179.98 finish=309.88\n          26664) sync             cpu=12 start=179.98 finish=180.22\n          26665) seq              cpu=10 start=180.22 finish=180.23\n          26666) clickhouse       cpu=4 start=180.23 finish=180.26\n          26668) clickhouse       cpu=1 start=180.26 finish=180.29\n          26669) clickhouse       cpu=0 start=180.29 finish=180.32\n          26670) sync             cpu=6 start=180.32 finish=180.33\n          26671) seq              cpu=0 start=180.33 finish=180.33\n          26672) clickhouse       cpu=9 start=180.33 finish=180.39\n          26697) clickhouse       cpu=3 start=180.40 finish=180.44\n          26698) clickhouse       cpu=14 start=180.44 finish=180.48\n          26699) sync             cpu=5 start=180.48 finish=180.49\n          26700) seq              cpu=11 start=180.49 finish=180.49\n          26701) clickhouse       cpu=0 start=180.49 finish=180.55\n          26706) clickhouse       cpu=11 start=180.55 finish=180.61\n          26707) clickhouse       cpu=9 start=180.61 finish=180.66\n          26708) sync             cpu=15 start=180.66 finish=180.67\n          26709) seq              cpu=2 start=180.67 finish=180.67\n          26710) clickhouse       cpu=10 start=180.67 finish=180.75\n          26715) clickhouse       cpu=12 start=180.75 finish=180.81\n          26716) clickhouse       cpu=13 start=180.81 finish=180.87\n          26717) sync             cpu=15 start=180.87 finish=180.88\n          26718) seq              cpu=12 start=180.88 finish=180.88\n          26719) clickhouse       cpu=0 start=180.88 finish=181.27\n          26736) clickhouse       cpu=1 start=181.28 finish=181.64\n          26737) clickhouse       cpu=10 start=181.65 finish=182.01\n          26738) sync             cpu=15 start=182.01 finish=182.03\n          26739) seq              cpu=10 start=182.03 finish=182.03\n          26740) clickhouse       cpu=1 start=182.03 finish=182.58\n          26741) clickhouse       cpu=4 start=182.58 finish=183.08\n          26742) clickhouse       cpu=15 start=183.09 finish=183.58\n          26743) sync             cpu=1 start=183.58 finish=183.59\n          26744) seq              cpu=14 start=183.59 finish=183.59\n          26745) clickhouse       cpu=8 start=183.59 finish=183.63\n          26746) clickhouse       cpu=15 start=183.63 finish=183.67\n          26747) clickhouse       cpu=12 start=183.68 finish=183.71\n          26748) sync             cpu=13 start=183.72 finish=183.72\n          26749) seq              cpu=14 start=183.72 finish=183.72\n          26750) clickhouse       cpu=15 start=183.72 finish=183.76\n          26751) clickhouse       cpu=11 start=183.77 finish=183.81\n          26752) clickhouse       cpu=13 start=183.81 finish=183.85\n          26753) sync             cpu=2 start=183.86 finish=183.87\n          26754) seq              cpu=2 start=183.87 finish=183.87\n          26755) clickhouse       cpu=5 start=183.87 finish=184.44\n          26756) clickhouse       cpu=7 start=184.44 finish=184.99\n          26757) clickhouse       cpu=1 start=185.00 finish=185.53\n          26758) sync             cpu=2 start=185.54 finish=185.57\n          26759) seq              cpu=11 start=185.57 finish=185.57\n          26760) clickhouse       cpu=1 start=185.57 finish=186.19\n          26761) clickhouse       cpu=2 start=186.20 finish=186.82\n          26762) clickhouse       cpu=14 start=186.82 finish=187.43\n          26763) sync             cpu=14 start=187.44 finish=187.47\n          26764) seq              cpu=13 start=187.47 finish=187.47\n          26765) clickhouse       cpu=0 start=187.47 finish=187.65\n          26766) clickhouse       cpu=8 start=187.66 finish=187.84\n          26767) clickhouse       cpu=15 start=187.84 finish=188.01\n          26768) sync             cpu=9 start=188.01 finish=188.03\n          26769) seq              cpu=11 start=188.03 finish=188.03\n          26770) clickhouse       cpu=6 start=188.03 finish=188.23\n          26771) clickhouse       cpu=0 start=188.23 finish=188.43\n          26772) clickhouse       cpu=8 start=188.44 finish=188.63\n          26773) sync             cpu=12 start=188.63 finish=188.65\n          26774) seq              cpu=0 start=188.66 finish=188.66\n          26775) clickhouse       cpu=12 start=188.66 finish=189.24\n          26776) clickhouse       cpu=2 start=189.25 finish=189.83\n          26777) clickhouse       cpu=6 start=189.83 finish=190.41\n          26778) sync             cpu=13 start=190.42 finish=190.45\n          26779) seq              cpu=0 start=190.46 finish=190.46\n          26780) clickhouse       cpu=5 start=190.46 finish=191.38\n          26781) clickhouse       cpu=5 start=191.38 finish=192.29\n          26782) clickhouse       cpu=13 start=192.29 finish=193.19\n          26783) sync             cpu=11 start=193.20 finish=193.27\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>Query processing time for queries of an OLAP data management system. Appears that each test had distinct phases of more and less parallel activity Topdown profile also shows the phases with an interesting blend of frontend and backend stages AMD <span class=\"excerpt-dots\">&hellip;<\/span> <a class=\"more-link\" href=\"https:\/\/mvermeulen.org\/perf\/workloads\/phoronix\/clickhouse\/\"><span class=\"more-msg\">Continue reading &rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":58,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1765","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1765","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/comments?post=1765"}],"version-history":[{"count":3,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1765\/revisions"}],"predecessor-version":[{"id":1827,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/1765\/revisions\/1827"}],"up":[{"embeddable":true,"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/pages\/58"}],"wp:attachment":[{"href":"https:\/\/mvermeulen.org\/perf\/wp-json\/wp\/v2\/media?parent=1765"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}