分佈式文件存儲QoS硬核黑科技，真香

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"存儲服務質量（QoS）是用於爲應用保證指定存儲性能的技術，它確保特定應用程序或工作負載始終獲得特定的訪問性能。QoS是解決存儲資源爭用的有效手段，主要用於不同業務之間的資源爭用和存儲內部資源爭用（如故障情況下，內部數據恢復和正常業務訪問之間產生的資源搶佔）。我們這裏談的QoS，主要用於保障不同業務的數據訪問性能質量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在企業內，IT部門需要爲高優先級的業務提供更高的訪問性能（包括IOPS、帶寬、延時等維度）。對任何已建成的系統而言，服務能力都是有上限的，這就需要爲高優先級業務配置更高的QoS，爲低優先級業務配置較低的QoS，合理分配資源，滿足不同級別業務的需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"外部存儲出現之初，是爲了解決本地存儲容量不足的問題，最早的外部存儲一般只服務於一個應用系統，不存在爭用的問題。隨着存儲技術的發展，單盤容量的提升，存儲系統容量不再是系統管理員最關心的問題，一套存儲可以服務於多個業務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"管理員們開始關注如何給不同的應用系統分配最佳的性能，如何保證關鍵業務的資源。例如，主業務數據庫應用和備份應用共享使用一套存儲，數據庫服務器顯然需要更高的IOPS纔能有效穩定運行，系統管理員就可以通過QoS爲數據庫服務應用指定適當的性能級別。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"企業對文件存儲性能的定義不僅僅是IOPS、帶寬和延遲，更細粒度的元數據操作也逐漸被管理員們關注。例如，海量數據場景下，某個計算節點頻繁執行ls、du等查詢操作，更會影響到整個存儲系統的正常運行。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着IT技術的發展，企業數據類型從結構化數據爲主轉化成非結構化數據爲主，業務環境也逐漸由單臺服務器（或一組雙機系統）對接一個存儲分區的模式，轉化爲一組計算節點（多個客戶端）對接一個存儲分區的模式。一組計算集羣共同使用一個文件系統時，存儲QoS不但要保證該文件目錄的性能QoS，還要均衡不同計算節點之間的性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"基於客戶端的QoS和基於目錄的QoS區別"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多個分佈式文件存儲客戶端在共享訪問目錄時，管理員由於無法預知共享訪問的客戶端數量，因此經常希望能夠針對目錄來指定QoS。如果只能針對目錄設置客戶端的QoS，可能會出現這樣的情況，每個客戶端的QoS可以按照管理員設定進行限制，但由於接入這個目錄的客戶端太多，而導致這個目錄的訪問性能搶佔了其它目錄的資源。因此，管理員更多是希望能夠將QoS限定到目錄級別，無論多少個客戶端接入了這個目錄，這些客戶端累計的訪問性能，不會超過設定的QoS，從而保護其它目錄（即其它應用）的訪問性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"分佈式文件存儲業界在目錄級QoS的整體狀況"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式文件存儲QoS由於客戶端和數據訪問都是分散在大量不同的服務器上，通過傳統的方式很難實現各個客戶端之間的性能協同，從而實現基於目錄的QoS。業界開源以及商業存儲，例如CephFS、GlusterFS、MooseFS、BeeGFS、OceanStor 9000、X10000、Spectrum Scale（GPFS）、Isilon等，都沒有提供基於目錄的QoS功能。Lustre的一些核心開發者，實現了一定程度的QoS功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"焱融科技研發團隊基於實際用戶場景和迫切需求，率先實現了基於目錄的QoS功能"},{"type":"text","text":"。YRCloudFile分佈式文件存儲的流量控制功能包括：帶寬QoS（讀帶寬、寫帶寬、總帶寬）、IOPS（讀IOPS、寫IOPS、總IOPS），對總性能的限制，即讀+寫的總數（IOPS、帶寬），不限制比例。用戶可以組合適合的控制方式，從而達到滿足業務要求的性能要求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"元數據操作分爲查詢操作、attr操作、文件操作三大類，"},{"type":"text","marks":[{"type":"strong"}],"text":"YRCloudFile元數據 QoS限制能夠對目錄的元數據操作數進行控制，從而避免元數據集羣資源緊張，影響關鍵業務的正常運行"},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於YRCloudFile的QoS功能，管理員可以方便地爲不同應用指定對應的目錄，隨後爲應用的目錄分配合理的性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"視頻爲實際測試場景，我們爲APP1、APP2、APP3分別分配三個目錄/APP1、/APP2、/APP3；爲這三個目錄分別分配總IOPS QoS爲10，100，1000。在測試場景下，我們觀察這三個目錄的QoS設置快速生效且準確。"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/62/62aa12db75aa2489b4c72f9844a11f73.png","alt":null,"title":"","style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"none"}],"href":" https://www.zhihu.com/zvideo/1300490021137559552","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看到，通過YRCloudFile對應用目錄的QoS限制，各個應用的訪問性能被限制在管理員設定的範圍內，很好地解決了應用之間對存儲性能的爭用問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"YRCloudFile基於目錄的分佈式QoS，相比單機QoS，不僅能夠在存儲目錄上進行QoS控制；在該文件目錄的多個客戶端之間，還會通過深度優化的dmClock算法（可參考“"},{"type":"link","attrs":{"href":"https://link.zhihu.com/?target=https%3A//mp.weixin.qq.com/s%3F__biz%3DMzIzMzY1NTM4Mw%3D%3D%26mid%3D2247485230%26idx%3D1%26sn%3Dd2d3f5a254764cea8438d54dbba6fc86%26chksm%3De8831a16dff49300dc5ea28a2b6517aaa99d47f7d6958995548087677629cb6725ec976afa6e%26token%3D917121150%26lang%3Dzh_CN%26scene%3D21%23wechat_redirect","title":null},"content":[{"type":"text","text":"分佈式QoS算法解析"}],"marks":[{"type":"underline"},{"type":"strong"}]},{"type":"text","marks":[{"type":"strong"}],"text":"”）進行均衡，從而滿足更大規模、更復雜場景的業務需要"},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"視頻爲實際測試場景，目錄/APP1設置了總帶寬QoS值，YRCloudFile的QoS功能對這幾個客戶端的功能進行了自動均衡和調節，這些客戶端之間的性能可以做到動態的調節。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8d/8d6a6c44baa31270c806225f63f65d88.png","alt":null,"title":"","style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"none"}],"href":" https://www.zhihu.com/zvideo/1300491344524169216","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於AI、HPC、半導體EDA、渲染、生命科學等大規模客戶端計算和分析場景來說，分佈式文件存儲的QoS具有重要意義，QoS可以幫助管理員更好地分配存儲性能資源，使各個應用客戶端充分、平穩地使用後端存儲性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"焱融科技是一家以軟件定義存儲技術爲核心競爭力的高新技術企業，在分佈式存儲等關鍵技術上擁有自主知識產權，是高性能文件存儲和容器存儲的領導者。焱融科技核心產品YRCloudFile在全球IO500性能測試中，進入世界前六。同時，YRCloudFile是國內首個進入CNCF Landscape的容器存儲產品。焱融科技針對各行業業務特性，打造個性化行業解決方案，提供一站式的產品與服務。焱融科技系列產品已服務於人工智能、金融、政府、製造業、互聯網等行業的衆多客戶。瞭解更多焱融科技信息，請訪問官網"},{"type":"link","attrs":{"href":"https://link.zhihu.com/?target=http%3A//www.yanrongyun.com","title":null},"content":[{"type":"text","text":"http://www.yanrongyun.com"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}