一杯茶的功夫就把日誌搜索引擎性能調優了？！

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者 | 小涵Q"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"來源 | 日誌易"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"策劃 | 劉燕"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Beaver 是由日誌易自主研發、安全可控的搜索引擎，由 Master、Broker 和 Datanode 三部分組成，已廣泛應用於存儲和分析大型分佈式系統生成的日誌。Beaver 擁有大量與性能相關的配置項，由於手動配置費時費力，並且有時需要修改相關配置以適配特定環境，所以自動調整配置參數優化性能是當前迫切需要解決的問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景調研"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前業界有許多自動調參的項目和算法實現，例如 CMU 開源的關係型數據庫自動調參工具 OtterTune[1]、PingCAP 仿作的 TiKV 自動調參工具[2]等，都爲我們提供了大量的論文以及開源算法代碼。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、 OtterTune"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫有很多參數，比如 MySQL 有幾百個參數，Oracle 有上千個參數。這些參數控制着數據庫的方方面面，很大程度上影響了如緩存容量和檢查點頻次等數據庫性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於不同的硬件配置，不同的工作負載，對應的最優參數文件都是不同的，這些複雜性令數據庫調優變得更加困難。DBA（Database Administrator，即數據庫管理員）不能簡單地重複使用之前調好的參數文件，他們需要花大量時間根據經驗來調優數據庫的參數，而公司則需要花大價錢來僱傭資深 DBA。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲解決上述問題，卡內基梅隆大學數據庫小組的教授、學生和研究人員開發了一個數據庫自動調參工具 OtterTune，它能利用機器學習對數據庫的參數文件進行自動化調優，利用已有的數據訓練機器學習模型，進而實現自動化地推薦最優參數。它能很好地幫助 DBA 進行數據庫調優，將 DBA 從複雜繁瑣的調參工作中解放出來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"OtterTune 的目的是爲了幫助 DBA，讓數據庫部署和調優更加容易，用機器學習代替人工來完成數據庫調參這個冗繁但又很重要的工作，讓技術人員甚至不需要專業知識也能順利完成。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"OtterTune 分爲客戶端和服務端，目標數據庫是用戶需要調優參數的數據庫："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"客戶端安裝在目標數據庫所在的機器上，收集目標數據庫的統計信息，並上傳到服務端。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"服務端一般配置在雲上，獲取到客戶端的數據之後，開始訓練機器學習模型並推薦配置文件。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"客戶端接收到推薦的配置文件後，配置到目標數據庫上，並測量其性能。以上步驟可重複進行，直到用戶對其推薦的配置文件滿意爲止。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2、AutoTiKV"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AutoTikv 是一個用於對 TiKV 數據庫進行自動調優的工具。它是根據 SIGMOD 2017年發表的一篇論文所設計[3]，能夠使用機器學習模型對數據庫參數進行自動調優。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AutoTiKV 吸取了 OtterTune 的設計理念，並簡化了相關結構。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"設計的調優過程如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/a8\/a8f442cbed4119383cf177f195339294.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"圖一：整個過程會循環跑200個 round（用戶可自定義），或者定義爲直到結果收斂爲止。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3、ML 模型"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AutoTikv 使用了和 OtterTune 一樣的高斯過程迴歸（Gaussian Process Regression，以下簡稱 GP）來推薦新的 knob，它是基於高斯分佈的一種非參數模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在還沒有利用機器學習模型對參數文件的效果進行預測的時候，OtterTune 使用的是隨機採樣的方式來收集初始數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當有足夠的數據 (X,Y) 時，OtterTune 訓練機器學習模型進行迴歸，即估計出函數 f:X→Y，表示對於參數文件 X，用 f(X) 來估計數據庫延遲 Y 的值。如此，問題則變爲尋找合適的 X，使 f(X) 的值儘量小。這樣在 f 上面做梯度下降即可找出合適的 X[4]。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如圖二所示，橫座標是兩個參數：緩存大小和日誌文件大小，縱座標是數據庫延遲（越低越好）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/60\/60f1b58e9e33951b311f56f9f3e5acc8.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"圖二：OtterTune 高斯過程迴歸模型"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"OtterTune 用高斯過程迴歸模型估計出了 f，即給定這兩個參數值，估計出對應的數據庫延遲。接着用梯度下降找到最合適的參數值，使延遲儘可能低。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"高斯過程迴歸的好處："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"和神經網絡之類的方法相比，GP 屬於無參數模型，算法計算量相對較低，而且在訓練樣本很少的情況下，GP 表現比神經網絡算法（Neural Network ） 更好。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"它能估計樣本的分佈情況，即 X 的均值 m(X) 和標準差 s(X)。若 X 周圍的數據不多，則它被估計出的標準差 s(X) 會偏大（表示這個樣本 X 和其他數據點的差異大）。直觀的理解是若數據不多，則不確定性會大，體現在標準差偏大；反之，數據足夠多時，不確定性減少，標準差會偏小。這個特性後面會用到。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但 GP 本身其實只能估計樣本的分佈，爲了得到最終的預測值，我們需要把它應用到貝葉斯優化（Bayesian Optimization）中[5]。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"貝葉斯優化算法大致可分爲兩步："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"通過 GP 估計出函數的分佈情況。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"通過採集函數（Acquisition Function）指導下一步的採樣（也就是給出推薦值）。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"採集函數（Acquisition Function）的特性是在尋找新的推薦值的時候，能夠進行平衡探索（Exploration）和利用（Exploitation）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Exploration：在目前數據量較少的未知區域探索新的點。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Exploitation：對於數據量足夠多的已知區域，利用這些數據訓練模型進行估計，找出最優值。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在推薦的過程中，需要平衡上述兩種指標。Exploitation 過多會導致結果陷入局部最優值（即重複推薦目前已知最好的點，但可能還有更好的點沒被發現），而 Exploration 過多又會導致搜索效率太低（即一直在探索新區域，而沒有對當前比較好的區域進行深入嘗試）。而平衡二者的核心思想，是當數據足夠多時，利用現有的數據推薦；當缺少數據時，在點最少的區域進行探索，探索最未知的區域能夠提供最大的信息量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"貝葉斯優化的第二步就可以幫我們實現這一思想。前面提到， GP 可以幫我們估計 X 的均值 m(X) 和標準差 s(X)，其中均值 m(X) 可以作爲 Exploitation 的表徵值，而標準差 s(X) 可以作爲 Exploration 的表徵值[6]，這樣就可以用貝葉斯優化方法來求解了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用置信區間上界（Upper Confidence Bound）作爲採集函數。假設我們需要找 X 使 Y 值儘可能大，則 U(X) = m(X) + k*s(X)，其中 k > 0 是可調的係數，我們只要找 X 使 U(X) 儘可能大即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若 U(X) 大，則可能 m(X) 大，也可能 s(X) 大。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若 s(X) 大，則說明 X 周圍數據不多，需要探索未知區域新的點。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"若 m(X) 大，說明估計的 Y 值均值大，則需要利用已知數據找到效果好的點。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注意：其中係數 k 影響着探索和利用的比例，即 k 越大，越鼓勵探索新的區域。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在具體實現中，一開始隨機生成若干個 candidate knobs，然後用上述模型計算出它們的 U(X)，找出 U(X) 最大的那一個作爲本次推薦的結果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"可行性分析"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前所有開源的自動調參工具實現原理基本上都是通過機器學習算法推薦配置參數，應用至數據庫或者其他引擎上，在不同的工作負載模式下，不斷收集 metric 信息，豐富訓練模型，直至推薦出最優的配置，以此替代頻繁的手動修改配置工作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由調研可以發現，OtterTune 是通用模型框架，在業界許多場景都能應用，不僅能調優數據庫的參數，還能夠調優操作系統內核的參數，即只要能獲取指標信息，大部分軟件都可以用此模型進行調優。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時可以借鑑 AutoTiKV 的測試代碼，將目標 DB 替換爲 Beaver_datanode，通過修改不同的配置，測試 baimi 數據集，收集 search 的性能數據，經過模型訓練後不斷推薦最優配置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"baimi 數據集，即 Apache 訪問日誌，總日誌行數7078124，日誌文件原始大小2374265761 Byte，測試 Beaver 和 ES 的搜索性能對比中用到的數據集。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AutoTiKV 代碼分析："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1．pipeline.py"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自動調參腳本入口，定義執行 round 數，自動推薦參數配置，修改配置文件並重啓相關 DB，收集 metric 數據訓練算法模型，以文件形式持久化保存對象。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2．settings.py"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"腳本參數配置，需要測試的 knobs、metrics 及 workload，數據庫連接配置。需要優化的 metric（僅支持優化一項目標 metric）、ansible 和 deploy目錄。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3．controller.py"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"knob 配置和 metric 獲取相關函數，每一個需要修改的參數都需要在 knob_set 中定義，聲明參數類型和取值範圍，修改配置文件和重啓數據庫函數等。另外包括一些工作負載相關的 workload 函數。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4．datamodel.py"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"初始化數據設置，存放數據模型，每次測試的配置參數和獲取到的指標數據都會存放在此模型中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"5．gpmodel.py"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"調用高斯過程迴歸類算法，傳遞並訓練數據模型，根據算法推薦返回最佳配置參數，用於下輪測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注意：前十輪爲隨機生成 knob。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"6．gpclass.py"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即高斯過程迴歸算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"7．showres.py"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"展示過往測試結果，將持久化保存的對象文件反序列化，調用 datamodel.py 函數中 GPDataSet 類輸出測試結果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"具體實現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本着不重複造輪子的原則，本次測試決定使用 AutoTiKV 的算法代碼，並修改其中關於數據庫的代碼，使其適用於 Beaver_datanode。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，TiKV 數據庫使用的配置文件是 yaml 格式，而 Beaver 使用的是 flags 參數的形式（–max_concurrency_tasks_per_search=4），代碼中使用的 ruamel.yaml 庫文件並不適用於 Beaver。因此，對 controller.py 中 set_tikvyml 函數進行修改，以“=”爲分隔符，讀取舊配置文件並將參數以鍵值對形式寫入字典中，對需要修改的配置項進行替換，最後把修改過後的配置參數寫入到新配置文件中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"需要修改的配置參數應在 settings.py 中提前聲明，更新 target_knob_set 列表中的參數，新增 wl_metrics 中 avgsearch 列表，並設置期望的 metric。在 controller.py 中補充參數的類型和取值範圍，配置好 knob_set 和 metric_set。修改metric 數據獲取函數，其中 read_search_latency() 函數是基於已經索引好的 baimi 數據集，測試某個場景的 search 性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"參考了 Esrally 的性能壓測代碼[7]，調用 Beaver_broker 的 API 接口，通過傳遞 pb 格式的搜索語句，來獲取不同場景下的 latnecy。本次自動調參測試中，使用的搜索場景是從數據集中獲取 apache.resp_len 字段的平均值，可以根據自己的實際環境自定義場景。因爲每次得到的性能數據可能受到各種因素影響，或產生較大誤差，爲了降低誤差值，搜索請求預熱20次，壓測100次，並計算前90th的平均值作爲最終的 metric 數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"settings.py 中需要修改的配置："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"makefile"},"content":[{"type":"text","text":"# beaver集羣的broker地址和端口，主要用來測試搜索性能\nbeaver_broker_ip=\"172.21.16.16\"\nbeaver_broker_port=\"50061\"\n# 測試搜索性能需要的索引\nindex_forsearch=\"ops-http_baimi-20210507\"\n# pb類型搜索語句，求apache.resp_len字段的平均值\npb_forsearch='search_info {query {type: kQueryMatchAll}fetch_source {fetch: true}size {value: 0} aggregations { aggs { type: kAggAvg name: \"av(apache.resp_len)\" body { field: \"apache.resp_len__l__\" } } } query_time_range {time_range {min: 0 max: 1620374828405}}}'\n# 不同工作負載模式下相關的指標，可以隨意命名，workload\nwl_metrics={\n \"avgsearch\": [\"search_latency\",\"compaction_mem\",\"compaction_cpu\"],\n}\n# workload to be load\nloadtype = \"avgsearch\"\n# workload to be run\nwltype = \"avgsearch\"\n# 需要優化的目標指標\ntarget_metric_name=\"search_latency\"\n# 待調整的配置項\ntarget_knob_set=['--enable_query_cache', # 啓用query cache\n '--max_concurrency_tasks_per_search', # 每個Search允許同時執行的數目\n '--max_per_search_ram', # 單個Search最大佔用的內存\n '--max_per_sub_search_ram', # 單個SubSearch最大佔用的內存\n '--block_ids_per_batch'] # 每個SubSearch的Block數目\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以下是 knob 和 metric 在 controller.py 中的聲明樣板："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"cs"},"content":[{"type":"text","text":"knob_set=\\\n{\"--max_concurrency_tasks_per_search\": # 配置項\n {\n \"changebyyml\": True, # True表示通過修改配置文件來調節\n \"set_func\": None, # 若changebyyml==False，則在此指定修改參數的函數名（在controller.py中定義函數），一般是不需要重啓beaver的配置\n \"minval\": 0, # if type==int, indicate min possible value\n \"maxval\": 0, # if type==int, indicate max possible value\n \"enumval\": [4, 6, 8], # if type==enum, list all valid values\n \"type\": \"enum\", # int \/ enum\n \"default\": 0 # default value\n }\n}\nmetric_set=\\ \n{\"search_latency\":\n {\n \"read_func\": read_search_latency, # 聲明查看該指標的函數（函數也定義在controller.py裏）\n \"lessisbetter\": 1, # whether less value of this metric is better(1: yes)\n \"calc\": \"ins\", # ins表示該參數的值就是benchmark之後查看的結果。inc表示該參數是incremental的，需要把benchmark之後和之前的值相減作爲結果。\n },\n}\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Beaver 重啓操作比較直接，使用 os.popen 管道命令直接在服務器上執行 kill 命令後更新配置文件重新啓動，具體操作在 controller.py 文件的 restart_beaver_datanode() 函數中。在本次測試中，腳本直接運行在 Beaver 所運行的服務器中，首先需要在 settings.py 文件中聲明 Beaver_datanode 的啓動命令和各項配置文件路徑，指定配置文件臨時存放路徑。使用 “ps -ef|grep beaver_datanode” 即可查看 Beaver_datanode 的啓動命令。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"具體配置示例如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"ini"},"content":[{"type":"text","text":"autotestdir=\"\/tmp\/auto_beaver_datanode\"\nbeaver_datanode_file=\"\/opt\/rizhiyi\/parcels\/beaver_datanode-3.7.0.0\/bin\/beaver_datanode\"\ngflagsfile=\"\/run\/rizhiyi_manager_agent\/process\/2002-beaver_datanode\/config\/beaver_datanode.gflags\"\nconfig_path=\"\/run\/rizhiyi_manager_agent\/process\/2002-beaver_datanode\/config\/beaver_datanode.pb\"\nlog_dir=\"\/data\/rizhiyi\/logs\/beaver_datanode\"\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"測試結果"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1、knobs"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"此次測試主要測試了以下配置項："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Options"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Description"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Valid range"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Confile"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"enable_query_cache"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"啓用 query cache"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[‘false’, ‘true’]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"beaver_datanode.gflags"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"max_concurrency_tasks_per_search"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每個 Search 允許同時執行的數目"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[4, 6, 8]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"max_per_search_ram"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"單個 Search 最大佔用的內存"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[198]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"max_per_sub_search_ram"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"單個 SubSearch 最大佔用的內存"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[99]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"block_ids_per_batch"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每個 SubSearch 的 Block 數目"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[16, 18, 20]"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2、metrics"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們選擇瞭如下幾個 metrics 作爲優化指標。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"search_latency：搜索延遲（ms）"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"compaction_mem：佔用內存比例（%）"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"compaction_cpu：佔用 CPU 比例（%）"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注意：knobs 和 metrics 均在 contorller.py 文件中定義。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3、腳本具體使用步驟"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"安裝 Python 3.6"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"shell"},"content":[{"type":"text","text":"$ tar -xf Python-3.6.12.tgz -C \/tmp\n# 編譯前檢查系統openssl版本是否爲1.0.2k以上版本，並安裝所需依賴包，否則編譯Python時會報錯。\n$ yum install -y openssl* bzip2* xz-devel zlib-devel libffi-devel gcc gcc-c++ sqlite sqlite-devel\n$ cd \/tmp\/Python-3.6.12\/\n$ .\/configure\n# 編譯無報錯後可以執行make安裝，默認會安裝到\/usr\/local\/lib\/python3.6\n# 使用該命令參數可以安裝到指定位置：.\/configure --prefix=\/xxx\n$ make && make install\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"安裝虛擬環境，切換到 Python 3.6 環境，下載自動化調參工具，並使用 pip 安裝相關依賴包"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"shell"},"content":[{"type":"text","text":"$ python3.6 -m venv autoenv\n$ source autoenv\/bin\/activate\n(autoenv)$ git clone https:\/\/github.com\/woxiang-H\/auto-beaver.git\n(autoenv)$ cd auto-beaver\n(autoenv)$ pip install -r requirements.txt \n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按照之前具體實現中提到的需要修改的配置，修改 auto-beaver 下 settings.py 文件。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"調整好 settings.py 之後，執行 python pipeline.py"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"等待結果收斂，查看推薦配置"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"properties"},"content":[{"type":"text","text":"################## data ##################\n------------------------------previous:------------------------------\nrowlabels, finish_time, knobs, metrics\n1 , 2021-05-11 16:41:02 , [0. 1. 0. 0. 1.] , [124.71111111 1.4 31.8 ]\n2 , 2021-05-11 16:43:41 , [0. 1. 0. 0. 0.] , [127.38888889 1.4 31.6 ]\n3 , 2021-05-11 16:46:10 , [1. 1. 0. 0. 0.] , [127.54444444 1.4 33.3 ]\n4 , 2021-05-11 16:48:48 , [0. 1. 0. 0. 2.] , [126.81111111 1.4 31.6 ]\n5 , 2021-05-11 16:51:27 , [1. 1. 0. 0. 2.] , [126.23333333 1.4 31.5 ]\n6 , 2021-05-11 16:54:06 , [0. 1. 0. 0. 2.] , [124.63333333 1.4 31.8 ]\n7 , 2021-05-11 16:56:35 , [1. 2. 0. 0. 1.] , [125.13333333 1.4 32.6 ]\n8 , 2021-05-11 16:59:14 , [1. 0. 0. 0. 1.] , [125.5 1.4 31.8]\n9 , 2021-05-11 17:01:53 , [1. 0. 0. 0. 2.] , [131.18888889 1.4 31.9 ]\n10 , 2021-05-11 17:04:31 , [0. 2. 0. 0. 1.] , [124.22222222 1.4 31.5 ]\n11 , 2021-05-11 17:07:53 , [0. 2. 0. 0. 2.] , [128.48888889 1.4 31.4 ]\n12 , 2021-05-11 17:11:14 , [0. 2. 0. 0. 1.] , [123.74444444 1.4 31.9 ]\n13 , 2021-05-11 17:14:35 , [0. 1. 0. 0. 1.] , [123.34444444 1.4 31.1 ]\n14 , 2021-05-11 17:17:56 , [0. 2. 0. 0. 1.] , [127.08888889 1.4 31.5 ]\n15 , 2021-05-11 17:21:17 , [0. 0. 0. 0. 1.] , [124.66666667 1.4 31.8 ]\n16 , 2021-05-11 17:24:38 , [0. 1. 0. 0. 1.] , [123.34444444 1.4 31.2 ]\n17 , 2021-05-11 17:27:59 , [0. 2. 0. 0. 1.] , [126.15555556 1.4 31.2 ]\n18 , 2021-05-11 17:31:21 , [0. 1. 0. 0. 1.] , [126.26666667 1.4 31.6 ]\n19 , 2021-05-11 17:34:41 , [0. 0. 0. 0. 1.] , [123.67777778 1.4 31.3 ]\n20 , 2021-05-11 17:38:04 , [0. 1. 0. 0. 1.] , [127.13333333 1.4 31.5 ]\n21 , 2021-05-11 17:41:15 , [0. 0. 0. 0. 1.] , [126.35555556 1.4 33.1 ]\n22 , 2021-05-11 17:44:36 , [0. 1. 0. 0. 1.] , [124.56666667 1.4 31.5 ]\n23 , 2021-05-11 17:47:57 , [0. 0. 0. 0. 1.] , [124.36666667 1.4 31.2 ]\n24 , 2021-05-11 17:51:19 , [0. 1. 0. 0. 1.] , [129.04444444 1.4 31.4 ]\n25 , 2021-05-11 17:54:39 , [0. 0. 0. 0. 1.] , [122.94444444 1.4 31.2 ]\n26 , 2021-05-11 17:58:01 , [0. 1. 0. 0. 1.] , [125.03333333 1.4 31.1 ]\n27 , 2021-05-11 18:01:22 , [0. 0. 0. 0. 1.] , [128.91111111 1.4 31.7 ]\n28 , 2021-05-11 18:04:42 , [0. 1. 0. 0. 1.] , [125.67777778 1.4 31.4 ]\n29 , 2021-05-11 18:08:04 , [0. 0. 0. 0. 1.] , [124.78888889 1.4 31.2 ]\n30 , 2021-05-11 18:11:15 , [1. 1. 0. 0. 1.] , [125.37777778 1.4 32.7 ]\n31 , 2021-05-11 18:14:36 , [0. 0. 0. 0. 1.] , [126.62222222 1.4 31.3 ]\n32 , 2021-05-11 18:17:57 , [0. 2. 0. 0. 1.] , [123.52222222 1.4 31.4 ]\n33 , 2021-05-11 18:21:19 , [0. 0. 0. 0. 1.] , [125.54444444 1.4 31.3 ]\n34 , 2021-05-11 18:24:31 , [0. 2. 0. 0. 1.] , [126.58888889 1.4 32.9 ]\n35 , 2021-05-11 18:27:52 , [0. 0. 0. 0. 1.] , [123.44444444 1.4 31.4 ]\n36 , 2021-05-11 18:31:12 , [0. 2. 0. 0. 1.] , [125.65555556 1.4 31.4 ]\n37 , 2021-05-11 18:34:34 , [0. 0. 0. 0. 1.] , [125.83333333 1.4 31.5 ]\n38 , 2021-05-11 18:37:45 , [1. 1. 0. 0. 1.] , [128.74444444 1.4 32.8 ]\n39 , 2021-05-11 18:41:06 , [0. 0. 0. 0. 1.] , [123.74444444 1.4 31.4 ]\n40 , 2021-05-11 18:44:28 , [1. 2. 0. 0. 1.] , [127.13333333 1.4 31.7 ]\n41 , 2021-05-11 18:47:49 , [0. 0. 0. 0. 1.] , [127.9 1.4 31.4]\n42 , 2021-05-11 18:51:00 , [0. 1. 0. 0. 1.] , [124.71111111 1.4 32.9 ]\n43 , 2021-05-11 18:54:21 , [0. 0. 0. 0. 1.] , [125.51111111 1.4 31.1 ]\n44 , 2021-05-11 18:57:42 , [0. 1. 0. 0. 1.] , [123.42222222 1.4 31.4 ]\n45 , 2021-05-11 19:01:03 , [0. 0. 0. 0. 1.] , [124.04444444 1.4 31.4 ]\n46 , 2021-05-11 19:04:24 , [0. 1. 0. 0. 1.] , [124.28888889 1.4 31.2 ]\n47 , 2021-05-11 19:07:35 , [0. 0. 0. 0. 1.] , [125.75555556 1.4 32.7 ]\n48 , 2021-05-11 19:10:57 , [0. 1. 0. 0. 1.] , [127.17777778 1.4 31.3 ]\n49 , 2021-05-11 19:14:17 , [0. 0. 0. 0. 1.] , [123.52222222 1.4 31.5 ]\n50 , 2021-05-11 19:17:38 , [0. 1. 0. 0. 1.] , [125.33333333 1.4 31.5 ]\n51 , 2021-05-11 19:20:58 , [0. 0. 0. 0. 1.] , [126.82222222 1.4 31.3 ]\n52 , 2021-05-11 19:24:19 , [0. 1. 0. 0. 1.] , [124.15555556 1.4 31.4 ]\n53 , 2021-05-11 19:27:40 , [0. 0. 0. 0. 1.] , [126.34444444 1.4 31.5 ]\n54 , 2021-05-11 19:31:01 , [0. 1. 0. 0. 1.] , [124.68888889 1.4 31.4 ]\n55 , 2021-05-11 19:34:23 , [0. 0. 0. 0. 0.] , [125.5 1.4 31.4]\n56 , 2021-05-11 19:37:44 , [0. 1. 0. 0. 1.] , [125.61111111 1.4 31.4 ]\n57 , 2021-05-11 19:40:55 , [0. 0. 0. 0. 0.] , [125.16666667 1.4 32.8 ]\n58 , 2021-05-11 19:44:16 , [0. 1. 0. 0. 1.] , [125.06666667 1.4 31.5 ]\n59 , 2021-05-11 19:47:37 , [0. 0. 0. 0. 0.] , [125.54444444 1.4 31.2 ]\n60 , 2021-05-11 19:50:58 , [0. 1. 0. 0. 1.] , [125.55555556 1.4 31.4 ]\n61 , 2021-05-11 19:54:20 , [0. 0. 0. 0. 0.] , [128.92222222 1.4 31.5 ]\n62 , 2021-05-11 19:57:31 , [0. 1. 0. 0. 1.] , [125.9 1.4 32.9]\n------------------------------new:------------------------------\nknobs: [[0. 0. 0. 0. 0.]]\nmetrics: [[124.1 1.4 31.5]]\nrowlabels: [1]\ntimestamp: 2021-05-11 20:00:52\n------------------------------TARGET:------------------------------\nknob: ['--enable_query_cache' '--max_concurrency_tasks_per_search'\n '--max_per_search_ram' '--max_per_sub_search_ram' '--block_ids_per_batch']\nmetric: search_latency\nmetric_lessisbetter: 1\n------------------------------------------------------------\nnum of knobs == 5\nknobs: ['--enable_query_cache' '--max_concurrency_tasks_per_search'\n '--max_per_search_ram' '--max_per_sub_search_ram' '--block_ids_per_batch']\nnum of metrics == 3\nmetrics: ['search_latency' 'compaction_mem' 'compaction_cpu']\n------------------------------------------------------------\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看到最佳推薦配置爲[0. 0. 0. 0. 0.]和[0. 1. 0. 0. 1.]（此數字含義爲 list 中的下標索引），具體配置參數如下。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"cpp"},"content":[{"type":"text","text":"set_beaver_datanode_gflags:: --enable_query_cache false\nset_beaver_datanode_gflags:: --max_concurrency_tasks_per_search 4\nset_beaver_datanode_gflags:: --max_per_search_ram 198m\nset_beaver_datanode_gflags:: --max_per_sub_search_ram 99m\nset_beaver_datanode_gflags:: --block_ids_per_batch 16\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"或者"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"cpp"},"content":[{"type":"text","text":"set_beaver_datanode_gflags:: --enable_query_cache false\nset_beaver_datanode_gflags:: --max_concurrency_tasks_per_search 6\nset_beaver_datanode_gflags:: --max_per_search_ram 198m\nset_beaver_datanode_gflags:: --max_per_sub_search_ram 99m\nset_beaver_datanode_gflags:: --block_ids_per_batch 18\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"結果顯示，適當提高 search 併發數，或提高 SubSearch 的 block 數會優化search 性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"存在的問題"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過修改相關代碼，目前自動調參工具能正常運行，但依然存在不足。本次測試方案利用事前存儲好的索引 baimi 數據集，僅測試影響 search 性能的參數，因此可修改的配置項也相對較少。雖然捨棄了 AutoTiKV 的 workload，但代碼依然保留此功能，待後續有針對 Beaver 的工作負載方案之後，再添加相關 workload。此外 Beaver_datanode 的重啓方式也並不優雅，有待提升。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"後續可優化："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"增加不同的 workload 模式，測試 index 性能和 search 性能（需要 Beaver支持新的 index 方式），同時測試 index 相關的配置參數。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"優雅地重啓 Beaver_datanode。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前重啓等待 Beaver_datanode 可用的 wait 時間爲200s，在實際運行的 Beaver_datanode 中，索引恢復時間相對較長，需根據不同環境靈活變化。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"指標數據獲取的準確度，會受到網絡等因素的影響，有一定的波動。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"[1] OtterTune. "},{"type":"link","attrs":{"href":"https:\/\/github.com\/cmu-db\/ottertune","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/github.com\/cmu-db\/ottertune"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"[2] AutoTiKV. "},{"type":"link","attrs":{"href":"https:\/\/github.com\/tikv\/auto-tikv","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/github.com\/tikv\/auto-tikv"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"[3] Automatic Database Management System Tuning Through Large-scale Machine Learning."},{"type":"text","text":" "},{"type":"link","attrs":{"href":"https:\/\/www.cs.cmu.edu\/~ggordon\/van-aken-etal-parameters.pdf","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/www.cs.cmu.edu\/~ggordon\/van-aken-etal-parameters.pdf"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"[4] "},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/y8VIieK0LO37SjRRyPhtrw","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/mp.weixin.qq.com\/s\/y8VIieK0LO37SjRRyPhtrw"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"[5] "},{"type":"link","attrs":{"href":"https:\/\/blog.csdn.net\/Leon_winter\/article\/details\/86604553","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/blog.csdn.net\/Leon_winter\/article\/details\/86604553"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"[6] "},{"type":"link","attrs":{"href":"https:\/\/blog.csdn.net\/a769096214\/article\/details\/80920304","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/blog.csdn.net\/a769096214\/article\/details\/80920304"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"[7] Esrally. "},{"type":"link","attrs":{"href":"https:\/\/github.com\/elastic\/rally","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/github.com\/elastic\/rally"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"拓展內容"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/www.cnblogs.com\/pdev\/p\/11318880.html","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/www.cnblogs.com\/pdev\/p\/11318880.html"}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/gitee.com\/opengauss\/openGauss-server\/tree\/master\/src\/gausskernel\/dbmind\/tools","title":"","type":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/gitee.com\/opengauss\/openGauss-server\/tree\/master\/src\/gausskernel\/dbmind\/tools"}]}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}