Jellyfish：爲Uber最大的存儲系統提供更節省成本的數據分層

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Jellyfish 項目成功地降低了 Uber 的運營費用，並且未來可以節省更多的存儲資源。這裏介紹的分層概念可以通過多種方式進行擴展，進一步提高效率並降低成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"問    題"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Uber利用一些存儲技術基於其應用模型來存儲業務數據。其中一項技術是 Schemaless，它能夠對相關條目進行建模，然後存儲在一個包含多個列的行中，並對每列進行版本管理。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Schemaless 已經存在了多年，其中積累了 Uber 的大量數據。雖然 Uber 正在整合 Docstore 上的所有用例，但 Schemaless 仍然是先前已經存在的不同客戶管道的事實來源。爲此，Schemaless 使用快速（但昂貴）的底層存儲技術來實現高 QPS 下的毫秒級延遲。此外，Schemaless 還在每個區域都部署了一些副本，以確保不同故障模式下數據的持久性和可用性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於積累的數據越來越多，同時又使用了昂貴的存儲，所以 Schemaless 已日益成爲關鍵的成本問題，需要特別關注。因此，爲了瞭解數據訪問的模式，我們做了一些度量。我們發現，在一段時間內數據會被頻繁地訪問，之後訪問頻率會降低。確切的時期段因用例不同而異，不過，舊數據仍然必須根據要求隨時可用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"要  求"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了勾勒出問題的恰當解決方案，我們提出了以下 4 個主要要求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"向後兼容"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Schemaless 已經存在了很久，它是 Uber 許多服務甚至是服務分層不可或缺的組成部分。因此，改變現有 API 的行爲或引入一套新的 API 都不是合適的選項，因爲它們需要對 Uber 產品服務做一連串的修改，導致方案落地延期。爲此，向後兼容就成了一項必然要求——消費者應該不需要修改代碼，就能享受到方案所帶來的所有效率提升。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"延遲"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"低延遲對於及時獲得數據至關重要，因此，我們與消費者團隊合作，調研不同的用例。研究表明，對於使用舊數據的用例，幾百毫秒的 P99 延遲是可以接受的，而對於使用新數據的用例，延遲必須保持在幾十毫秒之內。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"效率"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對現有 API 實現的任何改變都應該儘可能保持高效率，這不僅是爲了保證低延遲，也是爲了防止資源過度使用，如 CPU 和內存。這就需要進行優化，以減少讀取 \/ 寫入放大，關於這一點，我們將稍後進行說明。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"可配置性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如前所述，Schemaless 在 Uber 有許多用例，這些用例在訪問模式和延遲容忍度等方面不盡相同。這就要求我們的解決方案在一些關鍵點上可參數化，以便可以針對不同的用例進行配置和調整。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"解決方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"該解決方案的一個重要理念是，根據數據的訪問模式來處理數據，讓我們可以獲得相稱的投資回報率。也就是說，頻繁訪問的數據成本相對較高，而不頻繁訪問的數據成本必須相對較低。這正是數據分層所要達到的目的——類似於內存分層的概念。不過，我們需要一種方法來保持向後的兼容性，以確保對我們的消費者可以不做任何更改。這就要求我們把數據放在與複雜操作相同的層級中，這對跨層協同來說是個不小的挑戰。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們研究了減少舊數據空間佔用的方法。我們嘗試對數據單元（如一個行程）進行批處理，並在應用層面應用不同的壓縮方法，這樣我們就可以根據應用的預期性能來調整壓縮係數。這同時也降低了回填作業的讀取 \/ 寫入放大率，我們將在後面討論。我們探索了不同的壓縮方法，針對不同的用例做了不同的配置。我們發現，當我們批量壓縮若干單元時，ZSTD 壓縮算法整體可以節省高達 40% 的存儲空間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這一點上，我們意識到，藉助壓縮和批處理，我們可以在同一層中對數據進行內部分層，進一步減少舊數據的空間佔用。這使我們能夠把延遲控制在所要求的幾百毫秒之內。由於批次大小和 ZSTD 是可配置的，我們可以針對目前由 Schemaless 提供服務的不同用例調整我們的解決方案。通過恰當的實現，我們也可以滿足效率要求，達成上面討論的所有 4 個要求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們把這個項目稱爲 Jellyfish，因爲它是海洋中最高效的游泳選手，它在一定距離內消耗的能量比其他任何水生動物都少，包括強大的鮭魚。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"概念驗證"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現在，解決方案的大框架已經有了，我們需要快速評估其價值。爲此，我們進行了一系列的實驗，並做了一個快速的概念驗證。我們的目標是評估總體能節省多少空間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Jellyfish 主要使用 2 個參數來控制總體的空間節省，以及對 CPU 利用率的影響："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"批次大小：控制批處理的行數"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"壓縮等級：控制速度 vs. ZSTD 壓縮"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據概念驗證的度量結果，我們將批次大小設爲 100 行，ZSTD 等級設爲 7，這對 CPU 來說壓力應該不大。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這種設置下，總體壓縮率約爲 40%，如下圖所示。該圖還顯示了我們嘗試過的其他配置，這些配置出現了收益遞減或空間節省降低的情況。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/5b\/5bb4b352118d9b483afb7eccdde0dd8b.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們還沒有在大規模情況下觀察的一個關鍵指標是“批處理”請求的延遲。在實現的早期，我們通過一些壓力測試跟蹤過，確定可以滿足延遲 SLA，即數百毫秒。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"架  構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雖然我們考慮了幾個備選方案，但在這裏我們只討論最終設計。整體架構如下圖所示。後端有批處理表和實時表。批處理後端存儲從實時後端遷移過來的舊數據。實時後端與舊的後端完全一樣，但只用來存儲最近的數據。也就是說，新數據總是被寫進實時後端，就像以前一樣。一旦數據在一段時間後變冷，就會被遷移出來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下圖是一個高級視圖，顯示了在實現 Jellyfish 之後前端（查詢層）和後端（存儲引擎）組件的新架構。簡單起見，我們將主要關注新增部分，即以綠色顯示的部分。新架構的核心是 2 個表：（1）標準的“實時”表和（2）新增的批處理表。還是和以前一樣，客戶數據首先會被寫入實時表。經過一定的時間後（可根據用例進行配置），數據在經過分批和壓縮後被移到批處理表中。分批是由單元格完成的，它是 Schemaless 的基本單位。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/6f\/6fbcade65e895dac22fce59cfc3d789a.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如圖所示，Schemaless 用了一個批處理索引，它從單格元的 UUID 映射到相應的批次 UUID（UUID 到 BID）。在讀取舊數據的過程中，批處理索引用來快速檢索出正確的批次，解壓，並對其進行索引以提取所請求的單元格。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"請 求 流"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"新架構對用戶請求流產生了一些影響，我們將從讀取和寫入兩個方面進行說明。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"讀取"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"單個單元格的讀取還是和平常一樣進到實時表，因爲大多數請求（>90%）都是針對最近的數據。如果成功，請求之後就會終止。如果不成功，請求會“溢出到”批處理索引，找到批處理表，並在同一查詢中獲取它。下圖顯示了這個流程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/49\/498d6ae9405b9d31a30ff7b9ba5968c0.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"還有一種類型的讀取，它請求一個完整的行（構成一個邏輯業務實體的若干單元格，如行程）。這種請求的數據可能跨越了實時表和批處理表的界限。對於這樣的請求，我們調用兩個後端，並根據用戶定義的一些順序合併結果，如下圖所示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/97\/9720145648b6cd6e773b363a0bb25e61.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"寫入"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着數據被分割到兩個表中，主鍵的唯一性不復存在。爲了應對這種情況，我們需要擴展寫入查詢，以檢查數據在批處理索引中是否存在，並作爲同一事務的一個組成部分。我們發現，由於批處理索引比較小，所以查找的速度很快。下圖顯示了寫入路徑的新流程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/7c\/7c55c76b87bebd1f5cad2a1eafdc07ef.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"上  線"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於 Uber 而言，Schemaless 是一項關鍵任務，因此，Jellyfish 的上線需要做到絕對完美。爲此，上線過程需要通過多個驗證階段，而且最後是分階段推廣到實際的生產實例上。爲了保證功能行爲的正常，我們對所有新增的和調整過的端點進行了驗證，也包括一些邊緣情況。此外，爲了度量其時間特性，我們還對端點的非功能方面做了微基準測試。我們對啓用了 Jellyfish 的測試實例做了宏基準測試，以度量它們在各種工作負載下的性能特性。爲了找到吞吐量和延遲之間的關係，我們也進行了壓力測試。可以確定，啓用 Jellyfish 後，幾百毫秒的延遲服務協議是可以滿足的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着 Jellyfish 準備就緒，我們開始將其推廣到生產系統中。Uber 的行程存儲系統 Mezzanine 佔用的空間特別大。我們對如何分階段推出 Jellyfish 進行了討論。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"階段"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"向生產實例的推廣要經歷幾個階段，如下圖所示。下文大概介紹了我們使用單個分片推廣的情況。然後，我們逐步推廣到各分片和區域。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/5e\/5e59acfb4440406eec31046881ec614f.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"啓用 Jellyfish："},{"type":"text","text":"針對實例配置 Jellyfish 和遷移範圍，並允許創建批處理後端。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"遷移："},{"type":"text","text":"從實時後端讀取舊數據並將其複製到批處理後端。這個階段最耗時也最耗資源，並隨要遷移的數據量而伸縮。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"一致性驗證："},{"type":"text","text":"對發送到實時表的流量做了投影處理，以便可以在批處理後端進行數據驗證。它針對請求的舊數據計算摘要，並將其與來自 Jellyfish 的數據進行比較。我們會報告兩種類型的一致性：內容和計數。對於成功的遷移，兩者都必須爲零。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"預刪除："},{"type":"text","text":"實際上是逆向投影，只有在一致性達到 100% 時纔會啓用。請求舊數據的流量實際上是由 Jellyfish 提供的，不過我們仍然從實時後端計算摘要並與之比較。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"邏輯刪除："},{"type":"text","text":"會在請求舊數據時關閉實時後端讀取路徑，因此，也就不再對選擇的分片做摘要計算。這個階段完全模擬了舊數據從實時後端消失的情況。它有助於測試分層邏輯以及數據被真正刪除後的新數據流。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":6,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"物理刪除："},{"type":"text","text":"是在確認邏輯刪除成功後真正地刪除數據，與之前的階段不同，這個階段是不可逆的。此外，不同副本的刪除是交錯進行的，這樣可以確保在遇到意料之外的運行時問題時數據的可用性和業務的連續性。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"挑戰"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對任何正在使用的生產系統做更改都會面臨不小的挑戰。爲了確保數據的安全性和可用性，我們非常謹慎地採用了分階段的方法。而且，在從一個階段轉入下一個階段時，我們會確保客戶有足夠的時間進行監控和測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們面臨的一項挑戰是，有一個特定的服務導致了高負載，該服務主要是搜索舊數據來重新計算摘要。高負載導致了無法接受的延遲，所以我們與正要棄用該管道的客戶展開了合作。另一個更嚴重的挑戰是，用戶在請求單元格最近有更新的舊數據行時得到的是不完整的數據。我們需要推出一個修復方案，將其從實時後端和批量後端返回的結果合併後再返回給用戶。第三個挑戰和其他數據密集型任務的遷移工作有關，如重建用戶定義的索引和回填（backfill）作業。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們得到的啓示是，生產環境總是會向我們提出一些挑戰，不僅會影響項目的時間表，也會影響解決方案的適用性。爲了克服這些挑戰，我們需要仔細診斷，並與客戶密切協作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"優化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Jellyfish 項目的整個實施過程中，對於 Jellyfish 會明顯改變數據訪問模型的部分，我們一直在進行延遲或吞吐量方面的優化，其中包括："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"對請求的單元格進行解碼："},{"type":"text","text":"當用戶請求一個單元格時，會一次性獲取整個批次。我們只對所請求單元格的 JSON 部分進行解碼，而不對其他 99 個單元格進行解碼。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"只刪除元數據："},{"type":"text","text":"當就地刪除單元格時（由於 TTL 等原因），我們只從批處理索引中刪除該單元格的條目，這樣用戶就無法訪問它了。單元格實際的刪除工作是由一個後臺作業完成的，該作業通過一個 read-modify-write 操作更新批處理單元格。我們將被刪除的單元格的信息存儲在一個日誌表中，供後臺作業使用。這樣，我們就避免了在前臺運行這個昂貴的操作，在線讀 \/ 寫路徑就不會受到影響，用戶感知到的延遲也會相應降低。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"按批次整理更新："},{"type":"text","text":"當就地更新單元格時，一個批處理單元格可能會多次更新。使用 read-modify-write，更新過程既耗費資源又耗費時間。通過對更新按批次進行分組，我們能夠將一個作業的總更新時間降爲 1\/4。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"收穫"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Jellyfish 全面推出並確認可以滿足我們的要求後，我們就準備開始收穫了。爲此，我們開始分階段地從舊的後端中刪除數據。下圖顯示了在開始刪除後的幾天內，實際佔用的存儲空間減少的情況。在我們的情況下，Jellyfish 節省了 33% 的存儲空間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/f0\/f0eadecbfb988a68c74a70476297fd2e.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"未來展望"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Jellyfish 項目成功地降低了 Uber 的運營費用，並且未來可以節省更多的存儲資源。這裏介紹的分層概念可以通過多種方式進行擴展，進一步提高效率並降低成本。我們正考慮將 Jellyfish 應用於 Docstore、顯式分層以及使用不同的物理層等一些方向上。要實現這一目標，其中一部分工作是向用戶開放一套新的 API 用於訪問舊數據，並優化不同層級的軟件和硬件棧。Uber 非常歡迎有才華的工程師加入這項工作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"查看英文原文："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/eng.uber.com\/jellyfish-cost-effective-data-tiering\/","title":"","type":null},"content":[{"type":"text","text":"https:\/\/eng.uber.com\/jellyfish-cost-effective-data-tiering\/"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}