針對 MySQL IO 特點進行的存儲優化揭祕

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#00A5B4","name":"user"}},{"type":"strong","attrs":{}}],"text":"性能優化，是存儲工程師們永遠的追求，在我們看來，除了調整存儲架構、優化IO路徑，能對應用做出有針對性的優化，也是非常重要和有意義的事情，這意味着，除了要了解存儲本身，還需要對上層應用或中間件有足夠的認識。這次，我們就來看看 MySQL 的 IO 特點和存儲針對 MySQL 的優化思路。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#00A5B4","name":"user"}},{"type":"strong","attrs":{}}],"text":"MySQL 架構組件說明","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MySQL 及其延續的 MariaDB 是目前市場上最流行的關係型數據庫管理系統，在很多應用場景中，MySQL 都是用戶首選的 RDBMS（Relational Database Management System關係數據庫管理系統）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#000000","name":"black"}},{"type":"strong","attrs":{}}],"text":"MySQL大致包括如下幾大基礎模塊組件：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MySQL客戶端連接組件（Connectors）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"系統管理和控制工具組件（Management Service & Utilities）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"連接池組件（Connection Pool）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SQL組件（ SQL Interface）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"解析器組件（Parser）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"查詢優化器組件（Optimizer）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"緩存組件（Caches & Buffers）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"存儲引擎（Pluggable Storage Engines）","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文件系統（File System）","attrs":{}}]}]}],"attrs":{}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/77/77d6240a53cd4b7758a32af10a537f94.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#00A5B4","name":"user"}},{"type":"strong","attrs":{}}],"text":"InnoDB 存儲引擎","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"存儲引擎在 MySQL 的體系架構中位於第三層，負責對 MySQL 中的數據進行存儲和提取，是與文件打交道的子系統，它是根據底層提供的文件訪問層抽象接口定製的一種文件訪問機制，這個機制就叫作 MySQL 存儲引擎。從 MySQL 5.5 開始，默認採用 InnoDB 作爲存儲引擎。因此，優化底層存儲對 MySQL 業務的的性能，就要從瞭解和分析存儲引擎如何與底層的存儲系統進行交互開始。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下圖是官方的 InnoDB 引擎架構圖，InnoDB 存儲引擎主要分爲內存結構和磁盤結構兩大部分。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/81/8136a2e2b847b091432275ad37ce029c.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"InnoDB 磁盤主要包含 Tablespaces、InnoDB Data Dictionary、Doublewrite Buffer、Redo Log和 Undo Logs。Redo Log和 Binlog 是 MySQL 日誌系統中非常重要的兩種機制，本文主要談一下對 Redo Log 和 Binlog 進行的分析及存儲優化。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#00A5B4","name":"user"}},{"type":"strong","attrs":{}}],"text":"MySQL IO 模型和特點","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MySQL 寫數據過程中，有兩個重要的日誌文件，Redo Log 和 Binlog。Redo Log 記錄了對InnoDB 存儲引擎的事務日誌，Redo Log 的寫 IO 是固定文件範圍內的循環寫，IO 大小是 512 字節對齊（存在部分 offset 相等，執行的是覆蓋寫）。Binlog 記錄了對 MySQL 數據庫執行更改的所有操作，Binlog 的寫 IO 是文件 append 寫，IO 不對齊。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MySQL 寫請求時的存儲行爲：單線程執行 MySQL insert 寫數據時，一個 insert 對應一個write 操作；多線程併發執行 insert，MySQL 會將部分 IO 合併，然後下發到文件系統（如果是使用遠程文件系統，這個 IO 會被遠程文件系統的客戶端捕獲，例如 YRCloudFile 的客戶端），調用 write 請求寫入到 /MySQL/ib_logfile 文件中。一次 write IO 之後，立即調用fsync。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在開啓 Binlog 的場景下，寫完一次 Redo Log 後會再寫一次 Binlog，然後對 Binlog 做一次 fsync，以保證數據安全。當日志數據寫入一定量之後，MySQL 後臺另外一個線程會將所有的寫入，以每個 IO 16K 的大小進行整理，並以 aio 方式寫入到 /MySQL/ibdata 表文件中。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"MySQL讀請求時的存儲行爲：MySQL讀時，會從MySQL的緩存中查找數據，緩存命中，就不會實際下發 read IO 到底層文件系統中。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#00A5B4","name":"user"}},{"type":"strong","attrs":{}}],"text":"YRCloudFile 針對 MySQL 日誌 IO 行爲優化","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於 Redo Log 、Binlog 都是一次 IO 寫入伴隨着一次 fsync，而根據實際測試發現，fsync 對於存儲的開銷比較大。所以，對 MySQL 性能的優化，我們需要在完全確保這兩個日誌文件數據安全的前提下調整這兩個 Log的 IO 行爲。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 YRCloudFile 數據服務端，Redo Log 寫文件以 direct 方式寫入，寫入之後我們自動做一次 fsync。Binlog 由於 IO 不對齊，不可以採用 direct 方式寫，需要先寫入系統緩存，然後做一次 fsync。這樣，其實客戶端就不用再對 Redo Log 和 Binlog 文件做 remote_fsync 了，省去了客戶端調用 fsync 的開銷影響。下面是一組實測的數據對比：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/64/64c0941171f031d39ac7406128e19523.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從實測結果上，我們可以看出，在調整了 YRCloudFile 後端針對性的寫邏輯後後，MySQL 單線程寫入的性能得到了翻倍的提升。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"存儲的研發工程師們就是這樣，不但要掌握存儲的核心技能，還要關注和分析上層應用的業務行爲，才能對應用做出針對性的優化。以後，我們還會帶來更多面嚮應用的優化過程和分析，大家敬請期待。","attrs":{}}]}]}