如何對 ElasticSearch 集羣進行壓力測試

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當 ElasticSearch 的業務量足夠大，比如每天都會產生數百 GB 數據的時候，你就會自然而然的需要一個性能更強的 ElasticSearch 集羣。特別是當你使用的場景是一些典型的大量數據進入的場景，比如網站日誌、用戶行爲記錄、大型電商網站的站內搜索時，一個強勁的 ElasticSearch 是必不可少的組件。在這樣的場景下，如何找到一組合適的 ElasticSearch 集羣？如何評估 ElasticSearch 集羣的性能，就成爲了一個十分重要的因素。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"用什麼 ElasticSearch 進行壓力測試？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於 ElasticSearch 性能測試和壓力測試方面，其實有很多不同的方案，比如："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://github.com/elastic/rally","title":""},"content":[{"type":"text","text":"Rally"}]},{"type":"text","text":"：Rally 是 Elastic 官方針對於 ElasticSearch 的宏觀測試工具。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://github.com/danielfireman/esperf","title":""},"content":[{"type":"text","text":"ESPerf"}]},{"type":"text","text":"：一個基於 Golang 編寫的 ElasticSerch 性能測試工具"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://github.com/logzio/elasticsearch-stress-test","title":""},"content":[{"type":"text","text":"Elasticsearch Stress Test"}]},{"type":"text","text":"：由著名的 ElasticSearch 服務提供商 Logzio 開發的性能測試工具"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了這些定製化的工具意外以外，ElasticSearch 也可以藉由其 Restful API 來使用Load Runner、JMeter等老牌工具進行測試，這些工具零零散散，各有各的用法和用途，不過，對於廣大開發者而言，還是官方出的 Rally 更令人滿意。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本篇文章中，將會使用 Rally 來完成 ElasticSearch 集羣的壓力測試"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Rally 作爲官方出品的工具，來自官方的信任加成讓他成爲各家進行壓力測試的首選工具。其次， Rally 官方給出了多種默認的數據集（Tracks）。如果你的使用場景覆蓋在這些數據集(比如HTTP 訪問事件數據、地理名稱數據、地理座標點數據、HTTP 請求日誌、問答場景、打車記錄等場景)中，可以直接使用已有的數據集，來完成測試，而無需製造測試數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b6/b6da98cb64c896b41cb80317d030d161.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即使你的場景比較特殊，無法被官方的數據集所覆蓋，也依然可以根據自己的線上數據，來創建數據集，確保測試效果的準確。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/da/dae82599ac18e034140dd15b67e830b3.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"如何使用 Rally 進行測試？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在瞭解了 Rally 後，來具體看一看 Rally 的使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"測試前的準備"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關於 Rally 的基本安裝，這裏就不再介紹，總的來說十分簡單，在配置好了 JDK 和 Python 環境以後，只需要執行"},{"type":"codeinline","content":[{"type":"text","text":"pip install rally"}]},{"type":"text","text":" 就可以完成安裝。如果你的環境複雜，希望以一個更簡單的方式來運行，你也可以選擇使用 Docker 來運行 Rally ，進行測試。關於更多的安裝方式，你可以"},{"type":"link","attrs":{"href":"https://esrally.readthedocs.io/en/latest/install.html#","title":""},"content":[{"type":"text","text":"參考 Rally 的安裝文檔"}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在安裝完成了 Rally 後，就可以開始進行 ElasticSearch 的測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在測試前，你需要先了解一些基本概念"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"race"},{"type":"text","text":"：在 Rally 中，每一次測試都可以稱之爲 race"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"car"},{"type":"text","text":"： 在 Rally 中，每一個參與測試的集羣，都可以稱之爲 car ，不同的集羣就是不同的 car，如果你是在選配置，則可以通過切換 car 來設置不同配置的測試。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"track: 在 Rally 中，每一次測試用的數據，都可以稱之爲 Track，不同的 Track 意味着不同的測試數據。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"challange： 在 Rally 中，每一個 challange 意味着一個不同的測試場景，具體的場景則代表着 ElasticSearch 所執行的操作。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在瞭解測試的基本概念後，就可以開始進行壓力測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"進行壓力測試"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"測試設備"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於本次測試實際上是一次選型的過程，在測試之前，我曾閱讀了 Elastic 官方的權威指南中的"},{"type":"link","attrs":{"href":"https://www.elastic.co/guide/cn/elasticsearch/guide/current/hardware.html","title":""},"content":[{"type":"text","text":"硬件部分"}]},{"type":"text","text":".在其文檔中提供了對於運轉 ElasticSearch 集羣設備的推薦配置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"64 GB 內存的機器是非常理想的， 但是32 GB 和16 GB 機器也是很常見的。少於8 GB 會適得其反"},{"type":"text","text":"（你最終需要很多很多的小機器），大於64 GB 的機器也會有問題， 我們將在 堆內存:大小和交換 中討論。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"如果你要在更快的 CPUs 和更多的核心之間選擇，選擇更多的核心更好"},{"type":"text","text":"。多個內核提供的額外併發遠勝過稍微快一點點的時鐘頻率。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"如果你負擔得起 SSD，它將遠遠超出任何旋轉介質（注：機械硬盤，磁帶等）。 "},{"type":"text","marks":[{"type":"strong"}],"text":"基於 SSD 的節點，查詢和索引性能都有提升"},{"type":"text","text":"。如果你負擔得起，SSD 是一個好的選擇。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"通常，選擇中配或者高配機器更好。避免使用低配機器"},{"type":"text","text":"， 因爲你不會希望去管理擁有上千個節點的集羣，而且在這些低配機器上運行 Elasticsearch 的開銷也是顯著的。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因此，我選擇了原本打算購買的三家廠商（阿里雲、騰訊雲、UCloud）的設備進行測試，在具體的配置層面，則在各家購買四臺 8C64G 100GBSSD磁盤的的雲主機來進行測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/79/7933c661cbe0cce9c90bbda0fe6341af.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"測試環節"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"1. 構建集羣"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在進行測試前，需要先對已有的三臺設備搭建集羣，並配置集羣鏈接，確保集羣正常工作。集羣搭建的部分，你可以參考官方文檔中的"},{"type":"link","attrs":{"href":"https://www.elastic.co/guide/en/elasticsearch/reference/current/add-elasticsearch-nodes.html","title":""},"content":[{"type":"text","text":"Add and remove nodes in your cluster"}]},{"type":"text","text":"部分。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"2. 執行測試命令"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在完成了集羣的建設後，測試就簡單很多，只需要執行命令，便可以對已經建設好的集羣進行測試，比如:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"bash"},"content":[{"type":"text","text":"esrally --track=pmc --target-hosts=10.5.5.10:9200,10.5.5.11:9200,10.5.5.12:9200 --pipeline=benchmark-only"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"執行完命令後，接下來就是漫長的等待，根據機器的配置和集羣等"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"3. 獲得測試結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在執行了測試命令後，測試完成後，你就可以獲得相應的測試結果。不過，爲了方便進行對比和查看，你可以在測試的命令中加入參數，從而實現將測試結果導出爲特定的格式。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"比如,執行這樣的測試命令，就可以獲得 CSV 格式的測試數據"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"esrally --track=pmc --target-hosts=10.5.5.10:9200,10.5.5.11:9200,10.5.5.12:9200 --pipeline=benchmark-only -report-format=csv --report-file=~/benchmarks/result.csv"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當你得到了 csv 的結果後，就可以對數據進行分析和對比了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"如何查看 Rally 的測試結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/38/3893e4838217dffbd18fcaf86d899ac2.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當我們對 Rally 執行測試以後，我們會拿到大量的數據，而具體這些數據應該如何來看呢？接下來我們一一來看。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"如何理解 Rally 的數據"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Rally 導出的數據共有 4 列，分別是 "},{"type":"text","marks":[{"type":"strong"}],"text":"Metric（維度）"},{"type":"text","text":"、*"},{"type":"text","marks":[{"type":"italic"}],"text":"Task（任務）"},{"type":"text","text":"*、"},{"type":"text","marks":[{"type":"strong"}],"text":"Unit（單位）"},{"type":"text","text":"和*"},{"type":"text","marks":[{"type":"italic"}],"text":"‌Result（結果）"},{"type":"text","text":"*。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們可以將數據按照 Task 進行切分，你可以得到若干組結果，如："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"沒有 Task 的宏觀數據"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"index-append 組數據"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"index-stats 組數據"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"node-stats 組數據"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"phrase 組數據"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"..."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不同的組意味着 ElasticSearch 在不同場景下的應用，因此，你在對比時，需要按照分組來看數據。而分組內部的數據，就簡單了許多，除了宏觀數據以外，其他各組的數據基本上都是一個模式的，具體可以分爲以下 14 組數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Min/Median/Max"},{"type":"text","text":"：本組測試的最小吞吐率、中位吞吐率和最大吞吐率，單位爲 ops/s ,越大越好。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"50th/90th/99th/100th percentile latency"},{"type":"text","text":": 提交請求和收到完整回覆之間的時間段，越小越好"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"50th/90th/99th/99.9th/100th percentile service time"},{"type":"text","text":"：請求處理開始和接收完整響應之間的時間段，越小越好"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"error rate"},{"type":"text","text":"：錯誤率，錯誤響應相對於響應總數的比例。任何被 Elasticsearch Python 客戶端拋出的異常都被認爲是錯誤響應(例如，HTTP 響應碼 4xx、5xx或者網絡錯誤，如網絡不可達)。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當你能夠理解數據的劃分方式後，對於 Rally 返回的衆多數據就比較好理解了。接下來，我們以實際例子來看 Rally 返回的數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"如何理解 Rally 返回的宏觀測試數據"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏以我測試的數據爲例。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ce/ced3d4c30384189c13f4acf8a887642d.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據每組數據對應的不同操作，我們可以將其數據分爲若干組，這裏我將數據按照顏色進行了一個基礎的劃分，從上到下依次爲："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"索引時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"索引節流時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"合併時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"合併節流時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"刷新時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重刷時間"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，先看第一組的索引時間，索引時間共有四個指標:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative indexing time of primary shards: 主分片累計索引時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative indexing time across primary shards：跨分片累計索引時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative indexing throttle time of primary shards：主分片累計節流索引時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative indexing throttle time across primary shards：跨分片累計節流索引時間"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這四個指標說明了 ElasticSearch 在進行數據處理所需要的索引時間，因此，時間越短越好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/17/17ca2914b5c3df500e67c061ba2916e8.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/72/720a65b66c6b24fa5de9a5dc229c524f.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本組測試結果中，"},{"type":"text","marks":[{"type":"strong"}],"text":"騰訊雲的計算時間耗費最長，阿里雲可以在騰訊的基礎之上，有約 16 % 的性能提升，UCloud 則在騰訊雲的基礎之上提升了 24%"},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在測試過程中，會因爲機器的性能等因素，而造成實際測試時間的不同。在這三組中，阿里雲是最神奇的一個，因爲它測試時間達到了6個小時。而 UCloud 和騰訊雲則分別在 3 小時和 4 小時左右，實際的測試體驗還是有很大的差距。如果你要復現相應的實驗，建議你在早上進行，這樣出測試結果的時候剛好還在白天。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下來看合併時間的數據"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative merge throttle time of primary shards：主分片累計節流合併時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Min cumulative merge throttle time across primary shards：主分片累計節流合併時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Median cumulative merge throttle time across primary shards：主分片累計節流中位合併時間"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Max cumulative merge throttle time across primary shards：主分片累計節流最大合併時間"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"合併時間組結果類似於索引時間組，不同的是測量的數據 Merge 時間。和 index 類似，時間越短越好，合併數量越大越好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/96/96ddccac0e3d015613520c0c9ee87f48.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e1/e1f075ddaa7e795b6a2fca3fe8a4479b.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本組測試結果中，"},{"type":"text","marks":[{"type":"strong"}],"text":"騰訊雲的計算時間耗費最長，阿里雲可以在騰訊的基礎之上，有約 9 % 的性能提升，UCloud 則在騰訊雲的基礎之上提升了 30%～40%"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其他幾組類似的數據，這裏就不再一一介紹，大家可以自行下載文章最後的數據進行分析，評估，得出結論。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"如何理解 Rally 返回的項目測試數據"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了宏觀數據以外，Rally 返回數據中極大比例的是各種不同場景下的數據，這裏，我們也選兩組數據進行對比分析，看一看這些數據應該如何理解。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，我們看一下 node-stats 組的結果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"node-stats 組的結果是針對 node-stats 命令的數據分析結果。這裏的吞吐量越大越好，時延則越小越好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c4/c4b4742f88d0cf23f3d9f2b7a5f4c1bf.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本組測試結果中，"},{"type":"text","marks":[{"type":"strong"}],"text":"阿里雲、騰訊雲、UCloud 在性能方面沒有較大的差距。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"類似的，我們可以對比 country"},{"type":"text","marks":[{"type":"italic"}],"text":"agg"},{"type":"text","text":"uncached 組的結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/04/04a135eb7ab56240346fc52784a2b5ac.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本組測試結果中，"},{"type":"text","marks":[{"type":"strong"}],"text":"阿里雲、騰訊雲、UCloud 在吞吐量方面沒有較大的差距，時延方面 UCloud 會更好一點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關於更多的數據，我們不在這裏一一介紹，我將測試數據已經附在了文章的最後，你可以通過下載數據，自行分析，來練習 Rally 的使用和數據分析。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一些使用 Rally 時的小技巧"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"如何解決 Rally 網絡不好的問題？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Rally 在執行時，需要下載相應的數據進行真實場景下的模擬，在這種情況下，Rally 需要從 AWS S3 上下載一些數據包，用於本地的測試。但國內的設備對於 S3 的訪問不算太友好，經常會在下載的時候出現問題，因此，你可以選擇"},{"type":"text","marks":[{"type":"strong"}],"text":"前置下載數據"},{"type":"text","text":"，這樣在測試的時候可以直接使用本地的數據來完成測試，減少失敗的可能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"想要前置下載數據也很簡單隻需要執行如下命令:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"bash"},"content":[{"type":"text","text":"curl -O https://raw.githubusercontent.com/elastic/rally-tracks/master/download.sh\nchmod u+x download.sh\n./download.sh geonames\ncd ~\ntar -xf rally-track-data-geonames.tar"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在執行 download.sh 時加入 track 名，就可以下載對應 track 數據的壓縮包到本地，你可以根據自己的實際使用情況，來決定使用什麼樣的 track 模擬真實場景。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在業務真正使用 ElasticSearch 之前，你可以像我一樣，藉助於 Rally 對備選的 ElasticSearch 集羣進行壓力測試，並通過對壓測結果進行分析，從而獲得明確的選型建議。比如，在我這次測試中，顯然 UCloud 優於其他兩家，是一個性價比 更高的選擇。當然，這是一個實驗環境下的壓測，還是需要根據具體的業務場景做測試來進行選型，更爲科學，不過希望壓測方法能給予大家參考，也歡迎大家在後續實際的測試過程中給予我反饋。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"附錄"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文涉及到的測試數據均可以在"},{"type":"link","attrs":{"href":"https://postimg.aliavv.com/mbp/3tf6y.zip","title":""},"content":[{"type":"text","text":"下載地址"}]},{"type":"text","text":"找到。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}