{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ClickHouse自從2016年開源以來便備受關注，主要應用於數據分析（OLAP）領域，各個大廠紛紛跟進大規模使用。百分點科技在某國家級項目建設中完成了多數據中心的ClickHouse集羣建設，日增千億數據量，在此基礎上進行優化與性能調優，能夠更好地解決部署規模擴大和數據量擴容等問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文結合項目的數據規模及業務場景，重點介紹了百分點大數據技術團隊在ClickHouse國家級項目建設中的性能優化實踐。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、概覽","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2020年4月，百分點大數據技術團隊結合某國家級多數據中心的Clickhouse集羣建設，發表了“","attrs":{}},{"type":"link","attrs":{"href":"http://mp.weixin.qq.com/s?__biz=MjM5MzI5NjY2MA==&mid=2653785649&idx=1&sn=41eb89c759e3e66db099e86b154e9d0e&chksm=bd40164a8a379f5c9227c645c6f953bbf68f6e3d3fa6625294504ba9b5ccda7cada0e36dbbe6&scene=21#wechat_redirect","title":null,"type":null},"content":[{"type":"text","text":"ClickHouse國家級項目最佳實踐","attrs":{}}]},{"type":"text","text":"”，該文介紹了Clickhouse的特點、適用場景與核心概念，以及百分點科技在大規模Clickhouse集羣建設與運營方面的實踐經驗。爲部署到更多的城市，解決當前城市數據量擴容問題，Clickhouse集羣的設計優化與性能調優成爲重點工作：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據量增加了5倍多：數據量達到每天千億的級別；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"查詢週期更長：即時查詢週期從7天增加到30天；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據種類更多：業務數據類型增加了1倍；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"集羣規模增加了400多臺，聚合節點壓力變大；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"聚合查詢值枚舉空間大：部分字段的值枚舉空間在幾百萬和幾十億之間，聚合可能內存溢出。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在做了大規模的測試後，我們發現沿用集羣版本和規劃不能滿足需求。Clickhouse的很多新特性V1.1.5版本都不支持。經過一系列的版本對比測試，集羣規劃變更及參數調優，成功將版本升級到V20.1.16，同時也升級了Clickhouse指標採集與監控系統，滿足了數據規模的增長和業務需求的變化。整個升級涉及很多內容，本文僅就Clickhouse性能調優實踐進行重點介紹。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a6/a60bdd1c11b964330a87831695f05b05.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二、調優思路","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Clickhouse性能主要由CPU、IO、Memory驅動，無論是索引優化、列編碼與壓縮算法，還是配置的調優，主要是圍繞着這三個方面進行。索引改變了數據存儲和讀取的順序，編碼和壓縮改變數據存儲與傳輸的大小，服務配置影響着ClickHouse的資源分配和執行行爲。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先通過版本升級，利用ClickHouse的新特性來提升單條SQL的查詢速度，如採用跳數索引、列編碼及新的壓縮算法，這些都是在新版本中出的新特性。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/06/06f47bef2984a6df0a4d8cf740ec6910.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其次，改變了整個集羣的寫入方式，在數據寫入的時候對集羣按照業務概念進入邏輯劃分，減少了集羣寫入壓力。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"三、實踐經驗","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. 主鍵的選擇","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.1 生產中如何建主鍵索引","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實踐中，時間對於業務是必查字段，因此選用時間字段作爲主鍵，同時將幾個重要字段也加入了主鍵。總體來說，ClickHouse索引的長度沒有明確的限制，需要根據實際業務和數據的結構來綜合考慮。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"提升查詢性能","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"加在索引中的列如果能跳過比較長的一段數據，則能很好的提升查詢性能。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"提高數據壓縮率","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據主鍵排序的時候，數據一致性越高，壓縮比越高，這兩點在下面的測試用例中會很明顯的看到。另外，較長的索引會影響索引的內存使用量，不過對於目前普遍的大數據機器配置來說，以及ClickHouse稀疏索引的特性，四五個字段的長索引，對內存的佔用還是很有限的。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.2 測試表結構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在項目建設前期，表的主鍵基本上只有業務時間一個字段，於是我們做了下面的測試，使用其他比較重要的字段作爲主鍵開頭。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面四張測試表字段一致，只有主鍵不同。其中表2和表3數據一致，各有70億數據；表28和表29數據一致，各有643575000數據，爲了避免分區part數量的影響，所有表的分區都merge比較徹底，測試表的分區數也一致。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLElog_local2(\n `found_time` UInt32,\n `s_ip` String,\n `s_city` String,\n `s_long` Float32,\n`s_lat`Float32,\nINDEXs_ip_idx1 s_ip TYPE bloom_filter() GRANULARITY 4\n)ENGINE = MergeTree\nPARTITION BYtoRelativeDayNum(toDateTime(found_time))\nORDER BYfound_time;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLElog_local3(\n `found_time` UInt32,\n `s_ip` String,\n `s_city` String,\n `s_long` Float32,\n`s_lat`Float32,\nINDEXs_ip_idx1 s_ip TYPE bloom_filter() GRANULARITY 4\n)ENGINE = MergeTree\nPARTITION BYtoRelativeDayNum(toDateTime(found_time))\nORDER BY(found_time, s_ip, s_city, s_long, s_lat);","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLElog_local28(`found_time` UInt32,\n`d_ip` IPv6CODEC(LZ4HC(9)), `s_ip` IPv6 CODEC(LZ4HC(9)), `d_port` UInt16 CODEC(NONE),`s_port` UInt16 CODEC(NONE), `s_country` LowCardinality(String),\n`s_city` StringCODEC(ZSTD(9)), s_geo String, s_long Float32 , s_lat Float32, d_countryLowCardinality(String), d_city String CODEC(ZSTD(9)),\nd_geo String,d_long Float32 , d_lat Float32\n)ENGINE =MergeTree() PARTITION BYtoRelativeHourNum(toDateTime(found_time)) ORDER BY(found_time,s_city,s_long,s_lat)\nSETTINGSindex_granularity = 8192;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLE log_local29(`found_time`UInt32,\n`d_ip` IPv6CODEC(LZ4HC(9)), `s_ip` IPv6 CODEC(LZ4HC(9)), `d_port` UInt16 CODEC(NONE),`s_port` UInt16 CODEC(NONE), `s_country` LowCardinality(String),\n`s_city` StringCODEC(ZSTD(9)), s_geo String, s_long Float32 , s_lat Float32, d_countryLowCardinality(String), d_city String CODEC(ZSTD(9)),\nd_geo String,d_long Float32 , d_lat Float32\n)ENGINE =MergeTree() PARTITION BYtoRelativeHourNum(toDateTime(found_time)) ORDER BY(s_city,s_long,s_lat,found_time)\nSETTINGSindex_granularity = 8192;","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.3 log_local2和log_local3兩張測試表壓縮效果對比","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從表2和表3各列數據大小可看出，s_ip加入到主鍵後壓縮得到一定的優化。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/76/766dd77702361e229fc006fd6cd99f7f.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/31/31e93fb75a8c59306df6d203ec410617.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.4 log_local2和log_local3兩張測試表s_ip分頁查詢效果對比","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看出s_ip加入主鍵後查詢性能有一定的提升。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/90/90e679cd39613525a86efa96e650d8dd.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3e/3e93ed334867b6f6941bf875dc803d39.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.5 found_time和s_city開頭主鍵壓縮效果對比","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/99/991305c34b9cc06703a3331d43c8591a.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.6 看上去log_local29的壓縮效果更好，但是爲什麼不用這種組合索引呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實際業務中是多字段開放式查詢，found_time是必查字段，無論查什麼字段，都要帶上found_time字段，使用s_city開頭的主鍵看上去這幾個字段得到極大的優化效果，但是在不查詢這幾個字段的時候，會因爲found_time沒有得到較好的優化而查詢效果變差。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果是要對每天的數據做類似於數倉的ETL，原始數據層使用這種索引組合則會因爲降低比較多的存儲而好很多。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2. 跳數索引","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.1 ClickHouse跳數索引類型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"生產中，主要是使用MergeTree系列的引擎，目前，MergeTree系列引擎共支持5種跳數索引，分別是minmax、set和ngrambf_v1和tokenbf_v1、bloom_filter。實際生產中我們只選用了bloom_filter跳數索引，這個是由百分點科技實際業務的數據分佈所決定的，後面會分析爲什麼只用了bloom_filter索引。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.2 如何創建","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了演示，此處創建一張測試表，只有兩個字段，found_time和ip，爲了下文描述方便，類型都採用String，並給ip字段建了5種索引以展示索引用法。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLE ip_test(found_timeString,ip String,\nINDEX ip_idx1(ip) TYPE minmax GRANULARITY 1,\nINDEX ip_idx2 ip TYPE bloom_filter(0.025) GRANULARITY 1,\nINDEX ip_idx3 ip TYPE ngrambf_v1(3, 512, 5, 0) GRANULARITY 1,\nINDEX ip_idx4 ip TYPE tokenbf_v1(512, 5, 0) GRANULARITY 1,\nINDEX ip_idx5 ip TYPEset(0) GRANULARITY 4\n)ENGINE= MergeTree() order by found_time\npartitionby toYYYYMMDD(toDateTime(found_time)) SETTINGS index_granularity = 4;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"minmax—存儲相應索引區間的大小極值；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"bloom_filter(誤差概率)—布隆過濾器，裏面的參數爲自己設定的誤差概率，區間爲(0,1),默認爲0.025；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ngrambf_v1(n,布隆過濾器字節大小，hash函數個數，隨機種子)—；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"tokenbf_v1(布隆過濾器字節大小，hash函數個數，隨機種子)—；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"set(max_rows)—存儲相應索引區間的唯一值並不超過max_rows的個數，默認值0意味着沒有限制。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"樣例數據如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\ninsertinto ip_test values('1620756379','178.237.20.123');\ninsertinto ip_test values('1620756380','204.79.197.200');\ninsertinto ip_test values('1620756445','180.163.151.161');\ninsertinto ip_test values('1620756449','72.247.116.168');\ninsertinto ip_test values('1620759973','172.24.20.17');\ninsertinto ip_test values('1620759977','101.89.39.120');\ninsertinto ip_test values('1620760033','192.168.162.96');\ninsertinto ip_test values('1620760035','123.6.2.171');\ninsertinto ip_test values('1620760036','172.217.24.74');\ninsertinto ip_test values('1620760036','101.89.39.120');\ninsertinto ip_test values('1620760047','192.168.162.96');\ninsertinto ip_test values('1620760138','123.6.2.171');\ninsertinto ip_test values('1620760239','172.217.24.74');","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.3 index_granularity和GRANULARITY","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"index_granularity：","attrs":{}},{"type":"text","text":"指的是ClickHouse的索引粒度，多少行數據產生一條索引，默認是8192行數據產生一條索引；","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"GRANULARITY：","attrs":{}},{"type":"text","text":"跳數數據根據指的表達式聚合數據塊上的信息，聚合信息的粒度是由創建索引的時候指定GRANULARITY的值決定的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖所示，index_granularity = 4意味着每4條數據產生一條索引，GRANULARITY1則意味着minmax索引聚合信息的粒度是1個index_granularity,也是4條數據，minmax索引會從每4條數據中心聚合出最大最小的值來。GRANULARITY n=n*index_granularity。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d1/d1b74681e8e0bce241daa144de2ebc27.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再如下圖所示，set索引聚合的信息是整個樣例數據的枚舉值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6b/6b9cc2934032d60b82e5d66a3b4e96ed.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.4 如何對歷史數據建索引","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"添加索引：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nALTER TABLE ip_test add INDEXip_idx1(ip) TYPE minmax GRANULARITY 1;\nALTER TABLE ip_test add INDEXip_idx2 ip TYPE bloom_filter(0.025) GRANULARITY 1;\nALTER TABLE ip_test add INDEXip_idx3 ip TYPE ngrambf_v1(3, 512, 5, 0) GRANULARITY 1;\nALTER TABLE ip_test add INDEXip_idx4 ip TYPE tokenbf_v1(512, 5, 0) GRANULARITY 1;\nALTER TABLE ip_test add INDEXip_idx5 ip TYPE set(0) GRANULARITY 4;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"刪除索引：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"ALTER TABLE ip_test DROP INDEXip_idx1;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重建索引(對歷史數據add索引的時候，只是改變了表的schema，實際的索引文件並沒有生成，需要再使用重建索引的語句對歷史數據建立索引)：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nALTER TABLE ip_test MATERIALIZEINDEX name[ IN PARTITION partition_name]","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.5 bloom_filter 查詢效果","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"黃線爲沒有建bloom_filter索引的查詢數據，藍色爲建了bloom_filter索引的查詢數據，二者數據內容一致。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d6/d6e3d77d172d781c680aee65ff6494fd.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/34/34b65bb04aa597be14db904da717e90e.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.6 跳數索引需要額外存儲","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面表格是對70億ip字段建立索引後，原始數據和索引數據大小的對比，ip枚舉值在3億以上，對於這類數據建索引，索引文件會非常大。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/86/8690777ee45de489dbc0f9c120a14d0b.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在生產中只對枚舉值比較多的字段用了bloom_filter跳數索引，其他索引沒有使用，因爲bloom_filter的索引文件不至於太大，同時對於值比較多的列又能起到比較好的過濾效果。因爲經過測試發現對於枚舉值較少的字段，不建索引查詢速度就已經很快，因爲枚舉少，本身在壓縮的時候壓縮比很高，讀取速度很快，但是建了索引之後，要麼不能過濾掉數據，要麼確實過濾掉一部分數據，但是因爲讀取的數據塊不一致又會導致從原先的大塊讀取的順序IO退化爲隨機IO，反而得不償失。而枚舉值較多的情況比如ip字段，在使用ngrambf_v1索引的時候，ngram size和bloom filter大小的選擇對索引的大小和效果影響很大。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，通過上面的表格也能看出來，有些字段的bloom_filter跳數索引還是非常佔用存儲的，查詢的時候跳數索引的IO也需要考慮。因此，實際中才會出現日誌中確實過濾掉了數據，但是查詢速度反而慢了的情況。最後，選擇什麼索引以及索引的參數怎麼設置，都要結合業務數據的分佈特點也進行，數據的排序方式、散列程度都會影響索引效果。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3. 壓縮和編碼","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 通用編碼","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/76/7658d672c4496bc7844bc11fd38d4532.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.2 特殊編碼類型","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8f/8facdb7010a1e046f476c5c58d00b9a0.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"特殊編碼與通用的壓縮算法相比，區別在於，通用的LZ4和ZSTD壓縮算法是普適行的，不關心數據的分佈特點，而特殊編碼類型對於特定場景下的數據會有更好的壓縮效果。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.3 如何使用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLE k19_ods.test8\n(\n `found_time` UInt32,\n `recv_time1` UInt32 CODEC(NONE),\n `recv_time2` UInt32,\n `recv_time3` UInt32 CODEC(LZ4),\n `recv_time4` UInt32 CODEC(LZ4HC(9)),\n `recv_time5` UInt32 CODEC(ZSTD(9)),\n `recv_time6` UInt32 CODEC(T64()),\n `name0` String CODEC(Delta()，LZ4),\n `name1` String CODEC(DoubleDelta()),\n `name2` String CODEC(Gorilla()),\n `name4` String CODEC(Gorilla(), LZ4)\n)\nENGINE = MergeTree()\nPARTITION BYtoYYYYMMDD(toDateTime(found_time))\nORDER BY found_time","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"壓縮算法和特殊編碼兩者可以結合起來使用。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.4 壓縮效果舉例對比","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"需要注意一點就是，對於LZ4HC和ZSTD選擇的壓縮level越高，壓縮效果越好，但是CPU的使用率也會相應的越高。如果插入的數據量很大，會明顯看到較高的CPU使用率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/98/984bd9e109970fc2a91443b24d8fb09d.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在生產中，我們調整了一些字段的類型，有的從String變爲Int，有的從較大取值範圍的類型調整爲較小範圍的類型；還有一些不常用和壓縮後size非常大的字段採取了較高的壓縮算法；對於一些枚舉值較少的String使用了LowCardinality(String)，但由於這個特性在所用版本中有bug，因此在生產中改成了String。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4. 數據分區","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"4.1 數據分區規則","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"不指定分區鍵","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果建表時不指定分區鍵，則數據默認不分區，所有數據寫到一個默認分區all裏面。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d1/d1de2e924c1dc1c0cdaf3faa8a7761f4.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"使用整型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果分區鍵取值屬於整型且無法轉換爲日期類型YYYYMMDD格式，則直接按照該整型的字符形式輸出作爲分區ID的取值。 ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5d/5db0a02447ee049f01cf9d729478f4a7.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"使用日期類型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果分區鍵取值屬於日期類型，或者是能夠轉換爲YYYYMMDD日期格式的整型，則按照分區表達式邏輯格式化後作爲分區ID的取值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b0/b0a830a1079cd0cc1c9d77b6c9d452e0.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"使用其它類型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果分區鍵取值既不屬於整型，也不屬於日期類型，例如String、Float等，則通過128位Hash算法取其Hash值作爲分區ID的取值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/50/508b8aa1b56dbcee9a3f1f5b4dcef1fc.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":" 4.2 分區目錄命名規則","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7e/7e7ea2cdf0b5440f7b8d717df71d259a.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"partition_id：20210524:，見上頁的生成規則；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"min_block_number:1,,最小塊編號，MergeTree引擎從1開始計數，每次+1；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"max_block_number:1，最大塊編號，新插入的數據，最小與最大編號一致；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"level:0，這個可以理解爲合併的次數，新插入的數據都是0，每合併1次+1。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"4.3 分區目錄合併過程","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c9/c932c379923a90d30bc5f171c701935b.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/fd/fdcb03575e2abeda1649a82b8e525c96.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"4.4 分區設計","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分區選擇考慮merge (寫入放大)、查詢(slelect part數量)、啓動等方面考慮。在實際生產中選擇時間作爲分區鍵，根據表數據的大小按天或者按周進行了分區。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6d/6dba7194039068e06867b38c00bcdaab.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這是之前做過的一個part數量與啓動速度測試，從中可以明顯的看出part數量很影響啓動速度。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d1/d13b8c27873f715a399a9a472bd68e4d.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"5. 配置參數調優","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"5.1 現場使用的配置參數","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關於配置參數，大部分在之前的分享中都描述過，這裏着重講一個新加的配置：distributed_group_by_no_merge。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a1/a1da8096ee180cbe47a23368fa5368d6.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面是這個參數的解釋，從官方給出的說明中，有兩個關鍵點：一是分佈式查詢的時候，不會從不同服務器merge聚合狀態；二是不同分片上有不同的key是起比較好的應用場景。爲什麼不同的分片上要有不同的key，下文中將做出解釋。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/66/66e00df5bab8c75fd4563fd2336077b6.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"5.2 不開啓distributed_group_by_no_merge","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以1臺分佈式節點、4個數據節點的集羣來做舉例，建表語句和樣例數據如下（表裏只有兩個字段：id和其對應的數量）：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\ncreate table local.server(id Int32,cntInt32)ENGINE=Memory();\ncreate table local.server(id Int32,cntInt32)ENGINE=Memory();\ncreate table local.server(id Int32,cntInt32)ENGINE=Memory();\ncreate table local.server(id Int32,cntInt32)ENGINE=Memory();\ncreatetable dis.server(id Int32,cntInt32)ENGINE=Distributed('test',local,'server');\ninsert into server values(1,10) ,(2,9),(3,8) ,(4,5) ,(5,4) ,(6,3); \ninsert into server values (1,11), (3,12),(5,13), (7,14), (9,10); \ninsert into server values (2,15 ), (4,16 ),(6,17 ), (8,18 ), (10,19); \ninsert into server values (3,14 ), (6,15 ),(9,16 ), (11,17), (12,18), (13,20);","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正常情況下，要統計整個集羣的id機器數量並取Top3，sql如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nselect id,sum(cnt)counts from server groupby id order by counts desc limit 3 ;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ClickHouse的聚合過程大致如下圖(這裏只畫了數據返回流程)。在ClickHouse的聚合查詢中，每個機器都會把自己的聚合的中間狀態返回給分佈式節點，也就是說，即使你只是想要Top100，每臺機器也會把自己所擁有的所有枚舉值都返回給分佈式節點進行進一步的聚合。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"返回給分佈式節點22條數據，分佈式節點對着22條數據進行進一步的聚合，最後得出想要的結果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5e/5ee2305d129dc9ff492221281a3d2270.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"5.3 開啓distributed_group_by_no_merge","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nselect id,sum(counts)counts from( \n select id,sum(cnt)counts from servergroup by id order by counts desc limit 3\n settingsdistributed_group_by_no_merge=1)\n group by id order by counts desc limit 3;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這條SQL在聚合的時候每個節點只只會返回各自節點的Top3給分佈式節點，分佈式節點最終聚合3*4=12條數據。不過可以看到數據有一定誤差，在生產中對於比較靠前的topN來說，實際統計的誤差較小，如果不是要求百分百精確的話，完全可以接受。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f1/f15c83d8c87cd3874583738b1edfadb8.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"5.4 如果數據ip hash存儲","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於上面的統計誤差，如果數據是按照一定規則一類數據存儲到一個分片上，則可以避免這個問題，如下圖所示。所以上文裏提到官方說要不同的key分佈在不同的shard上面，不過，這同時會引進一個新的問題——數據傾斜問題，這裏就不做討論了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2a/2a38392a2d70b32110c008dea5b48f3e.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"6. 物化視圖","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"6.1 如何創建","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE MATERIALIZED VIEW db.viewname\n(\n field TYPE...\n)\n[ENGINE = engine] [POPULATE] AS SELECT ...","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"POPULATE","attrs":{}},{"type":"text","text":"關鍵字可以創建物化視圖的時候對歷史數據起作用，否則只會對新插入的數據生效。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"6.2 物化視圖與普通視圖的區別","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"普通視圖不保存數據，保存的僅僅是查詢語句，查詢的時候還是從原表讀取數據，可以將普通視圖理解爲是個子查詢。物化視圖則是把查詢的結果根據相應的引擎存入到了磁盤或內存中，對數據重新進行了組織，你可以理解物化視圖是完全的一張新表。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"物化視圖示例","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們準備了一份1547145102數據量的ClickHouse的text_log數據，表結構如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLE system.text_log(`event_date` Date, `event_time` DateTime, `microseconds` UInt32, `thread_name`LowCardinality(String), `thread_number` UInt32, `os_thread_id` UInt32, `level`Enum8('Fatal' = 1, 'Critical' = 2, 'Error' = 3, 'Warning' = 4, 'Notice' = 5,'Information' = 6, 'Debug' = 7, 'Trace' = 8), `query_id` String, `logger_name`LowCardinality(String), `message` String, `revision` UInt32, `source_file`LowCardinality(String), `source_line` UInt64) ENGINE = MergeTree PARTITION BYtoYYYYMM(event_date) ORDER BY (event_date, event_time) SETTINGSindex_granularity = 8192。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果要統計每天各種日誌級別的log數量，查詢語句：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nselectevent_date,level,countMerge(counts)counts from test.log_level_count group byevent_date,level order by event_date,level;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"查詢耗時：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e6/e67ea2ba5c54cac362b6712700e81637.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現在對該查詢建一個物化視圖看看效果：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE MATERIALIZED VIEW test.log_level_count\nENGINE = AggregatingMergeTree() PARTITION BY toYYYYMMDD(event_date)ORDER BY (event_date, level)\nPOPULATE AS\nselect event_date,level,countState(level)counts from system.text_loggroup by event_date,level order by event_date,level;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再次查詢：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nselectevent_date,level,countMerge(counts)counts fromtest.log_level_count group by event_date,level order by event_date,level;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ba/ba0fb9e0b5609a604a3703cfff75deab.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以明顯的看出兩者查詢速度的巨大差異，相差100多倍，看到這裏有沒有兩眼發光，很興奮，覺得自己學到了一個屠龍技！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那兩者爲什麼同樣的結果使用物化視圖會快這麼多呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2e/2ee65f9c2cb32aec47f8eb8b9ad0b17a.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/64/64482b6b98a98cb1405d63ffda6ade6c.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再看兩個查詢的不同之處，能明顯看出查詢處理的數據量不一樣，原查詢直接查詢原表15.5億的數據，使用物化視圖後，物化視圖查詢的是存在自己視圖表裏的數據，數據重組後的數據量差異是兩者查詢速度差異的根本原因。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"6.4 物化視圖在實踐中的使用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲業務查詢條件的不固定，所以在實際業務中沒有使用物化視圖，不過在對ClickHouse的監控中使用到了，比如每天新增、每小時新增數據量，每天的流量情況以及慢SQL的查詢等，這裏就不詳細展開了。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結與展望","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過調優，在生產環境中，95%的查詢小於1S響應、98%查詢小於3秒響應、99%查詢小於5秒響應。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文中的優化思路和方法總體上非常的通用，但具體怎麼設計和使用需要結合實際的業務場景來進行。比如，在百分點科技項目中數據的存儲是隨機寫到一定數量機器上的，在瞭解的不少公司實踐中，比較推薦的都是對數據按照一定規則進行hash進行讓相同的數據落到同一臺機器上，我們沒有這樣做，主要有三點原因：一是數據從源頭進來的時候就沒有進行hash，二是如果在數據流中進行hash會大大的降低數據流的處理速度，三是數據傾斜問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，對於項目中存在的一些問題我們也進行了反思，比如選用的版本，還不是很穩定和比較優的版本，選擇一個版本需要花費時間和精力進行各種測試，這個版本也只是在當時匆忙的時間與多個版本中綜合妥協的選擇。現在，ClickHouse的大版本已經更新到21.X，在此版本後，又出現了很多新功能，比如對於小表的wide part功能、SQL的explain功能、mysql同步數據到ClickHouse、開窗函數，以及使用S3等作爲ClickHouse底層存儲進行存算分離等，需要學習和實踐的還有很多。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接：","attrs":{}},{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/46tvZYjy-IiWdwoJhp1KDQ","title":"","type":null},"content":[{"type":"text","text":"百分點大數據技術團隊：ClickHouse國家級項目性能優化實踐","attrs":{}}]}]}]}
百分點大數據技術團隊：ClickHouse國家級項目性能優化實踐
{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ClickHouse自從2016年開源以來便備受關注，主要應用於數據分析（OLAP）領域，各個大廠紛紛跟進大規模使用。百分點科技在某國家級項目建設中完成了多數據中心的ClickHouse集羣建設，日增千億數據量，在此基礎上進行優化與性能調優，能夠更好地解決部署規模擴大和數據量擴容等問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文結合項目的數據規模及業務場景，重點介紹了百分點大數據技術團隊在ClickHouse國家級項目建設中的性能優化實踐。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、概覽","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2020年4月，百分點大數據技術團隊結合某國家級多數據中心的Clickhouse集羣建設，發表了“","attrs":{}},{"type":"link","attrs":{"href":"http://mp.weixin.qq.com/s?__biz=MjM5MzI5NjY2MA==&mid=2653785649&idx=1&sn=41eb89c759e3e66db099e86b154e9d0e&chksm=bd40164a8a379f5c9227c645c6f953bbf68f6e3d3fa6625294504ba9b5ccda7cada0e36dbbe6&scene=21#wechat_redirect","title":null,"type":null},"content":[{"type":"text","text":"ClickHouse國家級項目最佳實踐","attrs":{}}]},{"type":"text","text":"”，該文介紹了Clickhouse的特點、適用場景與核心概念，以及百分點科技在大規模Clickhouse集羣建設與運營方面的實踐經驗。爲部署到更多的城市，解決當前城市數據量擴容問題，Clickhouse集羣的設計優化與性能調優成爲重點工作：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據量增加了5倍多：數據量達到每天千億的級別；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"查詢週期更長：即時查詢週期從7天增加到30天；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據種類更多：業務數據類型增加了1倍；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"集羣規模增加了400多臺，聚合節點壓力變大；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"聚合查詢值枚舉空間大：部分字段的值枚舉空間在幾百萬和幾十億之間，聚合可能內存溢出。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在做了大規模的測試後，我們發現沿用集羣版本和規劃不能滿足需求。Clickhouse的很多新特性V1.1.5版本都不支持。經過一系列的版本對比測試，集羣規劃變更及參數調優，成功將版本升級到V20.1.16，同時也升級了Clickhouse指標採集與監控系統，滿足了數據規模的增長和業務需求的變化。整個升級涉及很多內容，本文僅就Clickhouse性能調優實踐進行重點介紹。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a6/a60bdd1c11b964330a87831695f05b05.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二、調優思路","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Clickhouse性能主要由CPU、IO、Memory驅動，無論是索引優化、列編碼與壓縮算法，還是配置的調優，主要是圍繞着這三個方面進行。索引改變了數據存儲和讀取的順序，編碼和壓縮改變數據存儲與傳輸的大小，服務配置影響着ClickHouse的資源分配和執行行爲。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先通過版本升級，利用ClickHouse的新特性來提升單條SQL的查詢速度，如採用跳數索引、列編碼及新的壓縮算法，這些都是在新版本中出的新特性。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/06/06f47bef2984a6df0a4d8cf740ec6910.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其次，改變了整個集羣的寫入方式，在數據寫入的時候對集羣按照業務概念進入邏輯劃分，減少了集羣寫入壓力。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"三、實踐經驗","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. 主鍵的選擇","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.1 生產中如何建主鍵索引","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實踐中，時間對於業務是必查字段，因此選用時間字段作爲主鍵，同時將幾個重要字段也加入了主鍵。總體來說，ClickHouse索引的長度沒有明確的限制，需要根據實際業務和數據的結構來綜合考慮。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"提升查詢性能","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"加在索引中的列如果能跳過比較長的一段數據，則能很好的提升查詢性能。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"提高數據壓縮率","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據主鍵排序的時候，數據一致性越高，壓縮比越高，這兩點在下面的測試用例中會很明顯的看到。另外，較長的索引會影響索引的內存使用量，不過對於目前普遍的大數據機器配置來說，以及ClickHouse稀疏索引的特性，四五個字段的長索引，對內存的佔用還是很有限的。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.2 測試表結構","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在項目建設前期，表的主鍵基本上只有業務時間一個字段，於是我們做了下面的測試，使用其他比較重要的字段作爲主鍵開頭。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面四張測試表字段一致，只有主鍵不同。其中表2和表3數據一致，各有70億數據；表28和表29數據一致，各有643575000數據，爲了避免分區part數量的影響，所有表的分區都merge比較徹底，測試表的分區數也一致。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLElog_local2(\n `found_time` UInt32,\n `s_ip` String,\n `s_city` String,\n `s_long` Float32,\n`s_lat`Float32,\nINDEXs_ip_idx1 s_ip TYPE bloom_filter() GRANULARITY 4\n)ENGINE = MergeTree\nPARTITION BYtoRelativeDayNum(toDateTime(found_time))\nORDER BYfound_time;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLElog_local3(\n `found_time` UInt32,\n `s_ip` String,\n `s_city` String,\n `s_long` Float32,\n`s_lat`Float32,\nINDEXs_ip_idx1 s_ip TYPE bloom_filter() GRANULARITY 4\n)ENGINE = MergeTree\nPARTITION BYtoRelativeDayNum(toDateTime(found_time))\nORDER BY(found_time, s_ip, s_city, s_long, s_lat);","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLElog_local28(`found_time` UInt32,\n`d_ip` IPv6CODEC(LZ4HC(9)), `s_ip` IPv6 CODEC(LZ4HC(9)), `d_port` UInt16 CODEC(NONE),`s_port` UInt16 CODEC(NONE), `s_country` LowCardinality(String),\n`s_city` StringCODEC(ZSTD(9)), s_geo String, s_long Float32 , s_lat Float32, d_countryLowCardinality(String), d_city String CODEC(ZSTD(9)),\nd_geo String,d_long Float32 , d_lat Float32\n)ENGINE =MergeTree() PARTITION BYtoRelativeHourNum(toDateTime(found_time)) ORDER BY(found_time,s_city,s_long,s_lat)\nSETTINGSindex_granularity = 8192;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLE log_local29(`found_time`UInt32,\n`d_ip` IPv6CODEC(LZ4HC(9)), `s_ip` IPv6 CODEC(LZ4HC(9)), `d_port` UInt16 CODEC(NONE),`s_port` UInt16 CODEC(NONE), `s_country` LowCardinality(String),\n`s_city` StringCODEC(ZSTD(9)), s_geo String, s_long Float32 , s_lat Float32, d_countryLowCardinality(String), d_city String CODEC(ZSTD(9)),\nd_geo String,d_long Float32 , d_lat Float32\n)ENGINE =MergeTree() PARTITION BYtoRelativeHourNum(toDateTime(found_time)) ORDER BY(s_city,s_long,s_lat,found_time)\nSETTINGSindex_granularity = 8192;","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.3 log_local2和log_local3兩張測試表壓縮效果對比","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從表2和表3各列數據大小可看出，s_ip加入到主鍵後壓縮得到一定的優化。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/76/766dd77702361e229fc006fd6cd99f7f.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/31/31e93fb75a8c59306df6d203ec410617.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.4 log_local2和log_local3兩張測試表s_ip分頁查詢效果對比","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看出s_ip加入主鍵後查詢性能有一定的提升。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/90/90e679cd39613525a86efa96e650d8dd.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3e/3e93ed334867b6f6941bf875dc803d39.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.5 found_time和s_city開頭主鍵壓縮效果對比","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/99/991305c34b9cc06703a3331d43c8591a.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1.6 看上去log_local29的壓縮效果更好，但是爲什麼不用這種組合索引呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實際業務中是多字段開放式查詢，found_time是必查字段，無論查什麼字段，都要帶上found_time字段，使用s_city開頭的主鍵看上去這幾個字段得到極大的優化效果，但是在不查詢這幾個字段的時候，會因爲found_time沒有得到較好的優化而查詢效果變差。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果是要對每天的數據做類似於數倉的ETL，原始數據層使用這種索引組合則會因爲降低比較多的存儲而好很多。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2. 跳數索引","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.1 ClickHouse跳數索引類型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"生產中，主要是使用MergeTree系列的引擎，目前，MergeTree系列引擎共支持5種跳數索引，分別是minmax、set和ngrambf_v1和tokenbf_v1、bloom_filter。實際生產中我們只選用了bloom_filter跳數索引，這個是由百分點科技實際業務的數據分佈所決定的，後面會分析爲什麼只用了bloom_filter索引。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.2 如何創建","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了演示，此處創建一張測試表，只有兩個字段，found_time和ip，爲了下文描述方便，類型都採用String，並給ip字段建了5種索引以展示索引用法。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLE ip_test(found_timeString,ip String,\nINDEX ip_idx1(ip) TYPE minmax GRANULARITY 1,\nINDEX ip_idx2 ip TYPE bloom_filter(0.025) GRANULARITY 1,\nINDEX ip_idx3 ip TYPE ngrambf_v1(3, 512, 5, 0) GRANULARITY 1,\nINDEX ip_idx4 ip TYPE tokenbf_v1(512, 5, 0) GRANULARITY 1,\nINDEX ip_idx5 ip TYPEset(0) GRANULARITY 4\n)ENGINE= MergeTree() order by found_time\npartitionby toYYYYMMDD(toDateTime(found_time)) SETTINGS index_granularity = 4;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"minmax—存儲相應索引區間的大小極值；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"bloom_filter(誤差概率)—布隆過濾器，裏面的參數爲自己設定的誤差概率，區間爲(0,1),默認爲0.025；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ngrambf_v1(n,布隆過濾器字節大小，hash函數個數，隨機種子)—；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"tokenbf_v1(布隆過濾器字節大小，hash函數個數，隨機種子)—；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"set(max_rows)—存儲相應索引區間的唯一值並不超過max_rows的個數，默認值0意味着沒有限制。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"樣例數據如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\ninsertinto ip_test values('1620756379','178.237.20.123');\ninsertinto ip_test values('1620756380','204.79.197.200');\ninsertinto ip_test values('1620756445','180.163.151.161');\ninsertinto ip_test values('1620756449','72.247.116.168');\ninsertinto ip_test values('1620759973','172.24.20.17');\ninsertinto ip_test values('1620759977','101.89.39.120');\ninsertinto ip_test values('1620760033','192.168.162.96');\ninsertinto ip_test values('1620760035','123.6.2.171');\ninsertinto ip_test values('1620760036','172.217.24.74');\ninsertinto ip_test values('1620760036','101.89.39.120');\ninsertinto ip_test values('1620760047','192.168.162.96');\ninsertinto ip_test values('1620760138','123.6.2.171');\ninsertinto ip_test values('1620760239','172.217.24.74');","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.3 index_granularity和GRANULARITY","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"index_granularity：","attrs":{}},{"type":"text","text":"指的是ClickHouse的索引粒度，多少行數據產生一條索引，默認是8192行數據產生一條索引；","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"GRANULARITY：","attrs":{}},{"type":"text","text":"跳數數據根據指的表達式聚合數據塊上的信息，聚合信息的粒度是由創建索引的時候指定GRANULARITY的值決定的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖所示，index_granularity = 4意味着每4條數據產生一條索引，GRANULARITY1則意味着minmax索引聚合信息的粒度是1個index_granularity,也是4條數據，minmax索引會從每4條數據中心聚合出最大最小的值來。GRANULARITY n=n*index_granularity。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d1/d1b74681e8e0bce241daa144de2ebc27.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再如下圖所示，set索引聚合的信息是整個樣例數據的枚舉值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6b/6b9cc2934032d60b82e5d66a3b4e96ed.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.4 如何對歷史數據建索引","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"添加索引：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nALTER TABLE ip_test add INDEXip_idx1(ip) TYPE minmax GRANULARITY 1;\nALTER TABLE ip_test add INDEXip_idx2 ip TYPE bloom_filter(0.025) GRANULARITY 1;\nALTER TABLE ip_test add INDEXip_idx3 ip TYPE ngrambf_v1(3, 512, 5, 0) GRANULARITY 1;\nALTER TABLE ip_test add INDEXip_idx4 ip TYPE tokenbf_v1(512, 5, 0) GRANULARITY 1;\nALTER TABLE ip_test add INDEXip_idx5 ip TYPE set(0) GRANULARITY 4;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"刪除索引：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"ALTER TABLE ip_test DROP INDEXip_idx1;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重建索引(對歷史數據add索引的時候，只是改變了表的schema，實際的索引文件並沒有生成，需要再使用重建索引的語句對歷史數據建立索引)：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nALTER TABLE ip_test MATERIALIZEINDEX name[ IN PARTITION partition_name]","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.5 bloom_filter 查詢效果","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"黃線爲沒有建bloom_filter索引的查詢數據，藍色爲建了bloom_filter索引的查詢數據，二者數據內容一致。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d6/d6e3d77d172d781c680aee65ff6494fd.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/34/34b65bb04aa597be14db904da717e90e.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2.6 跳數索引需要額外存儲","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面表格是對70億ip字段建立索引後，原始數據和索引數據大小的對比，ip枚舉值在3億以上，對於這類數據建索引，索引文件會非常大。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/86/8690777ee45de489dbc0f9c120a14d0b.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在生產中只對枚舉值比較多的字段用了bloom_filter跳數索引，其他索引沒有使用，因爲bloom_filter的索引文件不至於太大，同時對於值比較多的列又能起到比較好的過濾效果。因爲經過測試發現對於枚舉值較少的字段，不建索引查詢速度就已經很快，因爲枚舉少，本身在壓縮的時候壓縮比很高，讀取速度很快，但是建了索引之後，要麼不能過濾掉數據，要麼確實過濾掉一部分數據，但是因爲讀取的數據塊不一致又會導致從原先的大塊讀取的順序IO退化爲隨機IO，反而得不償失。而枚舉值較多的情況比如ip字段，在使用ngrambf_v1索引的時候，ngram size和bloom filter大小的選擇對索引的大小和效果影響很大。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，通過上面的表格也能看出來，有些字段的bloom_filter跳數索引還是非常佔用存儲的，查詢的時候跳數索引的IO也需要考慮。因此，實際中才會出現日誌中確實過濾掉了數據，但是查詢速度反而慢了的情況。最後，選擇什麼索引以及索引的參數怎麼設置，都要結合業務數據的分佈特點也進行，數據的排序方式、散列程度都會影響索引效果。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3. 壓縮和編碼","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 通用編碼","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/76/7658d672c4496bc7844bc11fd38d4532.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.2 特殊編碼類型","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8f/8facdb7010a1e046f476c5c58d00b9a0.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"特殊編碼與通用的壓縮算法相比，區別在於，通用的LZ4和ZSTD壓縮算法是普適行的，不關心數據的分佈特點，而特殊編碼類型對於特定場景下的數據會有更好的壓縮效果。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.3 如何使用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLE k19_ods.test8\n(\n `found_time` UInt32,\n `recv_time1` UInt32 CODEC(NONE),\n `recv_time2` UInt32,\n `recv_time3` UInt32 CODEC(LZ4),\n `recv_time4` UInt32 CODEC(LZ4HC(9)),\n `recv_time5` UInt32 CODEC(ZSTD(9)),\n `recv_time6` UInt32 CODEC(T64()),\n `name0` String CODEC(Delta()，LZ4),\n `name1` String CODEC(DoubleDelta()),\n `name2` String CODEC(Gorilla()),\n `name4` String CODEC(Gorilla(), LZ4)\n)\nENGINE = MergeTree()\nPARTITION BYtoYYYYMMDD(toDateTime(found_time))\nORDER BY found_time","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"壓縮算法和特殊編碼兩者可以結合起來使用。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3.4 壓縮效果舉例對比","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"需要注意一點就是，對於LZ4HC和ZSTD選擇的壓縮level越高，壓縮效果越好，但是CPU的使用率也會相應的越高。如果插入的數據量很大，會明顯看到較高的CPU使用率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/98/984bd9e109970fc2a91443b24d8fb09d.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在生產中，我們調整了一些字段的類型，有的從String變爲Int，有的從較大取值範圍的類型調整爲較小範圍的類型；還有一些不常用和壓縮後size非常大的字段採取了較高的壓縮算法；對於一些枚舉值較少的String使用了LowCardinality(String)，但由於這個特性在所用版本中有bug，因此在生產中改成了String。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4. 數據分區","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"4.1 數據分區規則","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"不指定分區鍵","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果建表時不指定分區鍵，則數據默認不分區，所有數據寫到一個默認分區all裏面。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d1/d1de2e924c1dc1c0cdaf3faa8a7761f4.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"使用整型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果分區鍵取值屬於整型且無法轉換爲日期類型YYYYMMDD格式，則直接按照該整型的字符形式輸出作爲分區ID的取值。 ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5d/5db0a02447ee049f01cf9d729478f4a7.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"使用日期類型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果分區鍵取值屬於日期類型，或者是能夠轉換爲YYYYMMDD日期格式的整型，則按照分區表達式邏輯格式化後作爲分區ID的取值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b0/b0a830a1079cd0cc1c9d77b6c9d452e0.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"使用其它類型","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果分區鍵取值既不屬於整型，也不屬於日期類型，例如String、Float等，則通過128位Hash算法取其Hash值作爲分區ID的取值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/50/508b8aa1b56dbcee9a3f1f5b4dcef1fc.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":" 4.2 分區目錄命名規則","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7e/7e7ea2cdf0b5440f7b8d717df71d259a.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"partition_id：20210524:，見上頁的生成規則；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"min_block_number:1,,最小塊編號，MergeTree引擎從1開始計數，每次+1；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"max_block_number:1，最大塊編號，新插入的數據，最小與最大編號一致；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"level:0，這個可以理解爲合併的次數，新插入的數據都是0，每合併1次+1。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"4.3 分區目錄合併過程","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c9/c932c379923a90d30bc5f171c701935b.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/fd/fdcb03575e2abeda1649a82b8e525c96.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"4.4 分區設計","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分區選擇考慮merge (寫入放大)、查詢(slelect part數量)、啓動等方面考慮。在實際生產中選擇時間作爲分區鍵，根據表數據的大小按天或者按周進行了分區。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6d/6dba7194039068e06867b38c00bcdaab.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這是之前做過的一個part數量與啓動速度測試，從中可以明顯的看出part數量很影響啓動速度。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d1/d13b8c27873f715a399a9a472bd68e4d.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"5. 配置參數調優","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"5.1 現場使用的配置參數","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關於配置參數，大部分在之前的分享中都描述過，這裏着重講一個新加的配置：distributed_group_by_no_merge。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a1/a1da8096ee180cbe47a23368fa5368d6.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面是這個參數的解釋，從官方給出的說明中，有兩個關鍵點：一是分佈式查詢的時候，不會從不同服務器merge聚合狀態；二是不同分片上有不同的key是起比較好的應用場景。爲什麼不同的分片上要有不同的key，下文中將做出解釋。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/66/66e00df5bab8c75fd4563fd2336077b6.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"5.2 不開啓distributed_group_by_no_merge","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以1臺分佈式節點、4個數據節點的集羣來做舉例，建表語句和樣例數據如下（表裏只有兩個字段：id和其對應的數量）：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\ncreate table local.server(id Int32,cntInt32)ENGINE=Memory();\ncreate table local.server(id Int32,cntInt32)ENGINE=Memory();\ncreate table local.server(id Int32,cntInt32)ENGINE=Memory();\ncreate table local.server(id Int32,cntInt32)ENGINE=Memory();\ncreatetable dis.server(id Int32,cntInt32)ENGINE=Distributed('test',local,'server');\ninsert into server values(1,10) ,(2,9),(3,8) ,(4,5) ,(5,4) ,(6,3); \ninsert into server values (1,11), (3,12),(5,13), (7,14), (9,10); \ninsert into server values (2,15 ), (4,16 ),(6,17 ), (8,18 ), (10,19); \ninsert into server values (3,14 ), (6,15 ),(9,16 ), (11,17), (12,18), (13,20);","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正常情況下，要統計整個集羣的id機器數量並取Top3，sql如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nselect id,sum(cnt)counts from server groupby id order by counts desc limit 3 ;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ClickHouse的聚合過程大致如下圖(這裏只畫了數據返回流程)。在ClickHouse的聚合查詢中，每個機器都會把自己的聚合的中間狀態返回給分佈式節點，也就是說，即使你只是想要Top100，每臺機器也會把自己所擁有的所有枚舉值都返回給分佈式節點進行進一步的聚合。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"返回給分佈式節點22條數據，分佈式節點對着22條數據進行進一步的聚合，最後得出想要的結果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/5e/5ee2305d129dc9ff492221281a3d2270.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"5.3 開啓distributed_group_by_no_merge","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nselect id,sum(counts)counts from( \n select id,sum(cnt)counts from servergroup by id order by counts desc limit 3\n settingsdistributed_group_by_no_merge=1)\n group by id order by counts desc limit 3;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這條SQL在聚合的時候每個節點只只會返回各自節點的Top3給分佈式節點，分佈式節點最終聚合3*4=12條數據。不過可以看到數據有一定誤差，在生產中對於比較靠前的topN來說，實際統計的誤差較小，如果不是要求百分百精確的話，完全可以接受。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f1/f15c83d8c87cd3874583738b1edfadb8.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"5.4 如果數據ip hash存儲","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於上面的統計誤差，如果數據是按照一定規則一類數據存儲到一個分片上，則可以避免這個問題，如下圖所示。所以上文裏提到官方說要不同的key分佈在不同的shard上面，不過，這同時會引進一個新的問題——數據傾斜問題，這裏就不做討論了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2a/2a38392a2d70b32110c008dea5b48f3e.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"6. 物化視圖","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"6.1 如何創建","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE MATERIALIZED VIEW db.viewname\n(\n field TYPE...\n)\n[ENGINE = engine] [POPULATE] AS SELECT ...","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"POPULATE","attrs":{}},{"type":"text","text":"關鍵字可以創建物化視圖的時候對歷史數據起作用，否則只會對新插入的數據生效。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"6.2 物化視圖與普通視圖的區別","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"普通視圖不保存數據，保存的僅僅是查詢語句，查詢的時候還是從原表讀取數據，可以將普通視圖理解爲是個子查詢。物化視圖則是把查詢的結果根據相應的引擎存入到了磁盤或內存中，對數據重新進行了組織，你可以理解物化視圖是完全的一張新表。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"物化視圖示例","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們準備了一份1547145102數據量的ClickHouse的text_log數據，表結構如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE TABLE system.text_log(`event_date` Date, `event_time` DateTime, `microseconds` UInt32, `thread_name`LowCardinality(String), `thread_number` UInt32, `os_thread_id` UInt32, `level`Enum8('Fatal' = 1, 'Critical' = 2, 'Error' = 3, 'Warning' = 4, 'Notice' = 5,'Information' = 6, 'Debug' = 7, 'Trace' = 8), `query_id` String, `logger_name`LowCardinality(String), `message` String, `revision` UInt32, `source_file`LowCardinality(String), `source_line` UInt64) ENGINE = MergeTree PARTITION BYtoYYYYMM(event_date) ORDER BY (event_date, event_time) SETTINGSindex_granularity = 8192。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果要統計每天各種日誌級別的log數量，查詢語句：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nselectevent_date,level,countMerge(counts)counts from test.log_level_count group byevent_date,level order by event_date,level;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"查詢耗時：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e6/e67ea2ba5c54cac362b6712700e81637.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現在對該查詢建一個物化視圖看看效果：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nCREATE MATERIALIZED VIEW test.log_level_count\nENGINE = AggregatingMergeTree() PARTITION BY toYYYYMMDD(event_date)ORDER BY (event_date, level)\nPOPULATE AS\nselect event_date,level,countState(level)counts from system.text_loggroup by event_date,level order by event_date,level;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再次查詢：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nselectevent_date,level,countMerge(counts)counts fromtest.log_level_count group by event_date,level order by event_date,level;","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ba/ba0fb9e0b5609a604a3703cfff75deab.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以明顯的看出兩者查詢速度的巨大差異，相差100多倍，看到這裏有沒有兩眼發光，很興奮，覺得自己學到了一個屠龍技！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那兩者爲什麼同樣的結果使用物化視圖會快這麼多呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2e/2ee65f9c2cb32aec47f8eb8b9ad0b17a.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/64/64482b6b98a98cb1405d63ffda6ade6c.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再看兩個查詢的不同之處，能明顯看出查詢處理的數據量不一樣，原查詢直接查詢原表15.5億的數據，使用物化視圖後，物化視圖查詢的是存在自己視圖表裏的數據，數據重組後的數據量差異是兩者查詢速度差異的根本原因。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"6.4 物化視圖在實踐中的使用","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲業務查詢條件的不固定，所以在實際業務中沒有使用物化視圖，不過在對ClickHouse的監控中使用到了，比如每天新增、每小時新增數據量，每天的流量情況以及慢SQL的查詢等，這裏就不詳細展開了。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結與展望","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過調優，在生產環境中，95%的查詢小於1S響應、98%查詢小於3秒響應、99%查詢小於5秒響應。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文中的優化思路和方法總體上非常的通用，但具體怎麼設計和使用需要結合實際的業務場景來進行。比如，在百分點科技項目中數據的存儲是隨機寫到一定數量機器上的，在瞭解的不少公司實踐中，比較推薦的都是對數據按照一定規則進行hash進行讓相同的數據落到同一臺機器上，我們沒有這樣做，主要有三點原因：一是數據從源頭進來的時候就沒有進行hash，二是如果在數據流中進行hash會大大的降低數據流的處理速度，三是數據傾斜問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另外，對於項目中存在的一些問題我們也進行了反思，比如選用的版本，還不是很穩定和比較優的版本，選擇一個版本需要花費時間和精力進行各種測試，這個版本也只是在當時匆忙的時間與多個版本中綜合妥協的選擇。現在，ClickHouse的大版本已經更新到21.X，在此版本後，又出現了很多新功能，比如對於小表的wide part功能、SQL的explain功能、mysql同步數據到ClickHouse、開窗函數，以及使用S3等作爲ClickHouse底層存儲進行存算分離等，需要學習和實踐的還有很多。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接：","attrs":{}},{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/46tvZYjy-IiWdwoJhp1KDQ","title":"","type":null},"content":[{"type":"text","text":"百分點大數據技術團隊：ClickHouse國家級項目性能優化實踐","attrs":{}}]}]}]}
發表評論
所有評論
還沒有人評論，想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章