源碼解析 | 萬字長文詳解 Flink 中的 CopyOnWriteStateTable

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現如今想閱讀 HashMap 源碼實際上比較簡單，因爲網上一大堆博客去分析 HashMap 和 ConcurrentHashMap。而本文是全網首篇詳細分析 CopyOnWriteStateTable 源碼的博客，閱讀複雜集合類源碼的過程是相當有挑戰的，筆者在剛開始閱讀也遇到很多疑問，最後一一解決了。本文有一萬兩千多字加不少的配圖，實屬不易。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"詳細閱讀完本文，無論是針對面試還是開闊視野一定會對大家有幫助的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"聲明：筆者的源碼分析都是基於 flink-1.9.0 release 分支，其實閱讀源碼不用非常在意版本的問題，各版本的主要流程基本都是類似的。如果熟悉了某個版本的源碼，之後新版本有變化，我們重點看一下變化之處即可。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文主要講述 Flink 中 CopyOnWriteStateTable 相關的知識，當使用 MemoryStateBackend 和 FsStateBackend 時，默認情況下會將狀態數據保存到 CopyOnWriteStateTable 中。CopyOnWriteStateTable 中保存多個 KeyGroup 的狀態，每個 KeyGroup 對應一個 CopyOnWriteStateMap。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 是一個類似於 HashMap 的結構，但支持了兩個非常有意思的功能："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、hash 結構爲了保證讀寫數據的高性能，都需要有擴容策略，CopyOnWriteStateMap 的擴容策略是一個漸進式 rehash 的策略，即：不是一下子將數據全遷移的新的 hash 表，而是慢慢去遷移數據到新的 hash 表中。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2、Checkpoint 時 CopyOnWriteStateMap 支持異步快照，即：Checkpoint 時可以在做快照的同時，仍然對 CopyOnWriteStateMap 中數據進行修改。問題來了：數據修改了，怎麼保證快照數據的準確性呢？"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"瞭解 Redis 的同學應該知道 Redis 也是一個大的 hash 結構，擴容策略也是漸進式 rehash。Redis 的 RDB 在持久化數據的過程中同時也是對外服務的，對外服務意味着數據可能被修改，那麼 RDB 如何保證持久化好的數據一定是正確的呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"舉個例子：17 點00分00秒 RDB 開始持久化數據，過了 1 秒 Redis 中某條數據被修改了，過了一分鐘 RDB 才持久化結束。RDB 預期的持久化結果應該是 17 點00分00秒那一刻 Redis 的完整快照，請問持久化過程中那些修改操作是否會影響 Redis 的快照。答：當然可以做到不影響。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Flink 在 Checkpoint 時的快照與 Redis 類似，都是想在快照時依然對外提供服務，減少服務停頓時間。Flink 具體如何實現上述功能的呢？帶着問題詳細閱讀下文。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"1.StateTable 簡介"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"StateTable 有兩個實現：CopyOnWriteStateTable 和 NestedMapsStateTable。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateTable 屬於 Flink 自己定製化的數據結構，Checkpoint 時支持異步 Snapshot。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"NestedMapsStateTable 直接嵌套 Java 的兩層 HashMap 來存儲數據，Checkpoint 時需要同步快照。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面詳細介紹 CopyOnWriteStateTable。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"2.CopyOnWriteStateTable"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"StateTable 中持有 "},{"type":"text","marks":[{"type":"strong"}],"text":"StateMap[] keyGroupedStateMaps"},{"type":"text","text":" 真正的存儲數據。StateTable 會爲每個 KeyGroup 的數據初始化一個 StateMap 來對 KeyGroup 做數據隔離。對狀態進行操作時，StateTable 會先根據 key 計算對應的 KeyGroup，拿到相應的 StateMap，才能對狀態進行操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateTable 中使用 CopyOnWriteStateMap 存儲數據，這裏主要介紹 CopyOnWriteStateMap 的實現。CopyOnWriteStateMap 中就是一個數組 + 鏈表構成的 hash 表。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 中元素類型都是是：StateMapEntry。hash 表的第一層先是一個 StateMapEntry 類型的數組，即：StateMapEntry[]。在 StateMapEntry 類中有個 "},{"type":"text","marks":[{"type":"strong"}],"text":"StateMapEntry next "},{"type":"text","text":"指針構成鏈表。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 相比普通的 hash 表，有以下幾點需要重點關注："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 的擴容策略是漸進式 rehash，而不是一下子擴容完"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了支持異步的 Snapshot，需要將 Snapshot 時 StateMap 的快照保存下來，具體的保存策略怎麼實現的？"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了支持 CopyOnWrite 功能，所以在修改數據時，要進行一系列 copy 的操作，不能修改原始數據，否則會影響 Snapshot。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Snapshot 異步快照流程及 Snapshot 完成時，如何 release 掉舊版本數據？"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"3.CopyOnWriteStateMap 的漸進式 rehash 策略"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"漸進式 rehash 策略表示 CopyOnWriteStateMap 中當前有一個 hash 表對外服務，但是當前 hash 表中元素太多需要擴容了，需要將數據遷移到一個容量更大的 hash 表中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Java 的 HashMap 在擴容時會一下子將舊 hash 表中所有數據都移動到大 hash 表中，這樣的策略存在的問題是如果 HashMap 當前存儲了 1 G 的數據，那麼瞬間需要將 1 G 的數據遷移完，可能會比較耗時。而 CopyOnWriteStateMap 在擴容時，不會一下子將數據全部遷移完，而是在每次操作 CopyOnWriteStateMap 時，慢慢去遷移數據到大的 hash 表中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如：可以在每次 get、put 操作時，遷移 4 條數據到大 hash 表中，這樣經過一段時間的 get 和 put 操作，所有的數據就能遷移完成。所以漸進式 rehash 策略，會分很多次將所有的數據遷移到新的 hash 表中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3.1 擴容簡述"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在內存中有兩個 hash 表，一個是 primaryTable 作爲主桶，一個是 rehashTable 作爲擴容期間用的桶。初始階段只有 primaryTable，當 primaryTable 中元素個數大於設定的閾值時，就要開始擴容。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"擴容過程：申請一個相比 primaryTable 容量大一倍的 hash 表保存到 rehashTable 中，慢慢地將 primaryTable 中的元素遷移到 rehashTable 中。對應到源碼中：putEntry 方法中判斷 size() > threshold 時，會調用 doubleCapacity 方法申請新的 hash 表賦值給 rehashTable。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖所示 primaryTable 中桶的個數爲 4，rehashTable 中桶的個數爲 8。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/68/6861d97aec2f907f9988c640a09c1aef.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"擴容時 primaryTable 中 0 位置上的元素會遷移到 rehashTable 的 0 和 4 位置上，同理 primaryTable 中 1 位置上的元素會遷移到 rehashTable 的 1 和 5 位置上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3.2 選擇 Table 的策略"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假設 primaryTable 中 0 桶的數據已經遷移到 rehashTable 桶了，那麼之後無論是 put 還是 get 操作 0 桶的數據，那麼都會去操作 rehashTable。而 1、2、3 桶還未遷移，所以 1、2、3 桶還需要操作 primaryTable 桶。對應到源碼中會有一個選桶的操作，選擇到底使用 primaryTable 還是 rehashTable。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"源碼實現如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// 選擇當前元素到底使用 primaryTable 還是 incrementalRehashTable\nprivate StateMapEntry[] selectActiveTable(int hashCode) {\n // 計算 hashCode 應該被分到 primaryTable 的哪個桶中\n int curIndex = hashCode & (primaryTable.length - 1);\n // 大於等於 rehashIndex 的桶還未遷移，應該去 primaryTable 中去查找。\n // 小於 rehashIndex 的桶已經遷移完成，應該去 incrementalRehashTable 中去查找。\n return curIndex >= rehashIndex ? primaryTable : incrementalRehashTable;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先通過 int curIndex = hashCode & (primaryTable.length - 1); 計算當前 hashCode 應該分到 primaryTable 的哪個桶中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"rehashIndex 用來標記當前 rehash 遷移的進度，即：rehashIndex 之前的數據已經從 primaryTable 遷移到 rehashTable 桶中。假設 rehashIndex = 1，表示 primaryTable 1 桶之前的數據全部遷移完成了，即：0 桶數據全部遷移完了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"策略：大於等於 rehashIndex 的桶還未遷移，應該去 primaryTable 中去查找。小於 rehashIndex 的桶已經遷移完成，應該去 incrementalRehashTable 中去查找。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3.3 遷移過程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每次有 get、put、containsKey、remove 操作時，都會調用 computeHashForOperationAndDoIncrementalRehash 方法觸發遷移操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"computeHashForOperationAndDoIncrementalRehash 方法作用："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"檢測是否處於 rehash 中，如果正在 rehash 就會調用 incrementalRehash 遷移一波數據"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"計算 key 和 namespace 對應的 hashCode"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重點關注 incrementalRehash 方法實現："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"private void incrementalRehash() {\n\n StateMapEntry[] oldMap = primaryTable;\n StateMapEntry[] newMap = incrementalRehashTable;\n\n int oldCapacity = oldMap.length;\n int newMask = newMap.length - 1;\n int requiredVersion = highestRequiredSnapshotVersion;\n int rhIdx = rehashIndex;\n // 記錄本次遷移了幾個元素\n int transferred = 0;\n\n // 每次至少遷移 MIN_TRANSFERRED_PER_INCREMENTAL_REHASH 個元素到新桶、\n // MIN_TRANSFERRED_PER_INCREMENTAL_REHASH 默認爲 4\n while (transferred < MIN_TRANSFERRED_PER_INCREMENTAL_REHASH) {\n\n // 遍歷 oldMap 的第 rhIdx 個桶\n StateMapEntry e = oldMap[rhIdx];\n\n // 每次 e 都指向 e.next，e 不爲空，表示當前桶中還有元素未遍歷，需要繼續遍歷\n // 每次遷移必須保證，整個桶被遷移完，不能是某個桶遷移到一半\n while (e != null) {\n // 遇到版本比 highestRequiredSnapshotVersion 小的元素，則 copy 一份\n if (e.entryVersion < requiredVersion) {\n e = new StateMapEntry<>(e, stateMapVersion);\n }\n // 保存下一個要遷移的節點節點到 n\n StateMapEntry n = e.next;\n\n // 遷移當前元素 e 到新的 table 中，插入到鏈表頭部\n int pos = e.hash & newMask;\n e.next = newMap[pos];\n newMap[pos] = e;\n\n // e 指向下一個要遷移的節點\n e = n;\n // 遷移元素數 +1\n ++transferred;\n }\n\n oldMap[rhIdx] = null;\n // rhIdx 之前的桶已經遷移完，rhIdx == oldCapacity 就表示遷移完成了\n // 做一些初始化操作\n if (++rhIdx == oldCapacity) {\n XXX\n return;\n }\n }\n\n // primaryTableSize 中減去 transferred，增加 transferred\n primaryTableSize -= transferred;\n incrementalRehashTableSize += transferred;\n rehashIndex = rhIdx;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"incrementalRehash 方法中第一層 while 循環用於控制每次遷移的最小元素個數。然後遍歷 oldMap 的第 rhIdx 個桶，e 指向當前遍歷的元素，每次 e 都指向 e.next，e 不爲空，表示當前桶中還有元素未遍歷，需要繼續遍歷。每次遷移必須保證，整個桶被遷移完，不能是某個桶遷移到一半。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"遷移過程中，將當前元素 e 重新計算 hash 值，插入到 newMap 相應桶的頭部（頭插法）。其中 e.entryVersion < requiredVersion 時，需要創建一個新的 Entry，這裏是爲了支持 CopyOnWrite 功能，下面會介紹。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"4.StateMap 的 Snapshot 策略"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"StateMap 的 Snapshot 策略是指：爲了支持異步的 Snapshot，需要將 Snapshot 時 StateMap 的快照保存下來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"傳統的方法就是將 StateMap 的全量數據在內存中深拷貝一份，然後拷貝的這一份數據去慢慢做快照，原始的數據可以對外服務。但是深拷貝需要拷貝所有的真實數據，所以效率會非常低。爲了提高效率，Flink 只是對數據進行了淺拷貝。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4.1 淺拷貝原理分析"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"淺拷貝就是隻拷貝引用，不拷貝數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假如 StateMap 沒有處於擴容中，Snapshot 流程相對比較簡單，創建一個新的 snapshotData，直接將 primaryTable 的數據拷貝到 snapshotData 中即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e8/e8c7423ecd1cfc7ac26280ad15541f5f.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如圖所示，對於淺拷貝可以理解爲兩個 Table 的 0 號桶中都引用的同一個鏈表，也就是將 snapshotData 指向圖中的 Entry a 即可。其他桶的淺拷貝也是類似，就不一一畫圖了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假如 StateMap 當前處於擴容中，Snapshot 流程相對比較繁瑣，創建一個新的 snapshotData，需要將 primaryTable 和 rehashTable 的數據都拷貝到 snapshotData 中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/07/07269cc7ca7e387a50dbae084e461e56.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如圖所示，將原始兩個 Table 數據拷貝到 snapshotData 中，但是 snapshotData 數組的長度並不是 primaryTable 的長度 + rehashTable 的長度。而是分別計算 primaryTable 和 rehashTable 中有幾個桶中有數據。例如上圖案例所示，primaryTable 中有 3 個桶中有元素，rehashTable 中有 2 個桶中有元素，所以snapshotData 的桶數量爲 5 即可，沒必要 4 + 8 = 12 個桶。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上圖中也是省略了 Entry，Entry 引用的淺拷貝與之前沒有擴容的情況類似。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4.2 淺拷貝源碼詳解"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先調用 CopyOnWriteStateTable 的 stateSnapshot 方法對整個 StateTable 進行快照。stateSnapshot 方法會創建 CopyOnWriteStateTableSnapshot，CopyOnWriteStateTableSnapshot 的構造器中會調用 CopyOnWriteStateTable 的 getStateMapSnapshotList 方法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"getStateMapSnapshotList 方法源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"List> getStateMapSnapshotList() {\n List> snapshotList = \n new ArrayList<>(keyGroupedStateMaps.length);\n // 調用所有 CopyOnWriteStateMap 的 stateSnapshot 方法\n // 生成 CopyOnWriteStateMapSnapshot 保存到 list 中\n for (int i = 0; i < keyGroupedStateMaps.length; i++) {\n CopyOnWriteStateMap stateMap = \n (CopyOnWriteStateMap) keyGroupedStateMaps[i];\n snapshotList.add(stateMap.stateSnapshot());\n }\n return snapshotList;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateTable 中爲每個 KeyGroup 維護了一個 StateMap 到 keyGroupedStateMaps 中，getStateMapSnapshotList 方法會調用所有 CopyOnWriteStateMap 的 stateSnapshot 方法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 的 stateSnapshot 方法相關源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public CopyOnWriteStateMapSnapshot stateSnapshot() {\n return new CopyOnWriteStateMapSnapshot<>(this);\n}\n\nCopyOnWriteStateMapSnapshot(CopyOnWriteStateMap owningStateMap) {\n super(owningStateMap);\n\n // 對 StateMap 的數據進行淺拷貝，生成 snapshotData\n this.snapshotData = owningStateMap.snapshotMapArrays();\n // 記錄當前的 StateMap 版本到 snapshotVersion 中\n this.snapshotVersion = owningStateMap.getStateMapVersion();\n this.numberOfEntriesInSnapshotData = owningStateMap.size();\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 的 stateSnapshot 方法會創建 CopyOnWriteStateMapSnapshot，CopyOnWriteStateMapSnapshot 的構造器中會調用 StateMap 的 snapshotMapArrays 方法對 StateMap 的數據進行淺拷貝生成 snapshotData。且將當前的 StateMap 版本到 snapshotVersion 中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"StateMap 的 snapshotMapArrays 方法對淺拷貝原理進行了代碼實現，代碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public class CopyOnWriteStateMap extends StateMap { \n // 當前 StateMap 的 version\n private int stateMapVersion;\n // 所有 正在進行中的 snapshot 的 version\n private final TreeSet snapshotVersions;\n // 正在進行中的那些 snapshot 的最大版本號\n private int highestRequiredSnapshotVersion;\n\n StateMapEntry[] snapshotMapArrays() {\n // 1、stateMapVersion 版本 + 1，賦值給 highestRequiredSnapshotVersion，\n // 並加入snapshotVersions\n synchronized (snapshotVersions) {\n ++stateMapVersion;\n highestRequiredSnapshotVersion = stateMapVersion;\n snapshotVersions.add(highestRequiredSnapshotVersion);\n }\n\n // 2、 將現在 primary 和 Increment 的元素淺拷貝一份到 copy 中\n // copy 策略：copy 數組長度爲 primary 中剩餘的桶數 + Increment 中有數據的桶數\n // primary 中剩餘的數據放在 copy 數組的前面，Increment 中低位數據隨後，\n // Increment 中高位數據放到 copy 數組的最後\n StateMapEntry[] table = primaryTable;\n\n final int totalMapIndexSize = rehashIndex + table.length;\n final int copiedArraySize = Math.max(totalMapIndexSize, size());\n final StateMapEntry[] copy = new StateMapEntry[copiedArraySize];\n\n if (isRehashing()) {\n final int localRehashIndex = rehashIndex;\n final int localCopyLength = table.length - localRehashIndex;\n // for the primary table, take every index >= rhIdx.\n System.arraycopy(table, localRehashIndex, copy, 0, localCopyLength);\n\n table = incrementalRehashTable;\n System.arraycopy(table, 0, copy, localCopyLength, localRehashIndex);\n System.arraycopy(table, table.length >>> 1, copy, \n localCopyLength + localRehashIndex, localRehashIndex);\n } else {\n System.arraycopy(table, 0, copy, 0, table.length);\n }\n\n return copy;\n }\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 中三個比較重要的屬性："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"stateMapVersion：表示當前 StateMap 的版本，每次 Snapshot 時版本號加一"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"snapshotVersions：存放所有正在進行中的 snapshot 的版本號（因爲可能存在多個同時進行的 Snapshot）"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"highestRequiredSnapshotVersion：表示正在進行中的那些 snapshot 的最大版本號，如果當前沒有正在進行中的 Snapshot，那麼賦值爲 0"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"snapshotMapArrays 方法第一步按照上述規則更新這三個屬性，第二步將現在 primaryTable 和 rehashTable 的元素淺拷貝一份到 copy 數組中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注：copy 數組的長度與上述原理分析不完全一致，原理分析時應該是 copiedArraySize = totalMapIndexSize；實際上 copiedArraySize = Math.max(totalMapIndexSize, size())。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"源碼註釋寫到：理論上 totalMapIndexSize 就夠了，這裏考慮 size 主要是爲了兼容 StateMap 的 TransformedSnapshotIterator 功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"5.CopyOnWrite 實現原理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上一部分得出結論，每次 Snapshot 時僅僅是淺拷貝一份，所以 Snapshot 和 StateMap 共同引用真實的數據。假如 Snapshot 還沒將數據 flush 到磁盤，但是 StateMap 中對數據進行了修改，那麼 Snapshot 最後 flush 的數據就是錯誤的。Snapshot 的目標是：將 Snapshot 快照中原始的數據刷到磁盤，既然叫快照，所以不允許被修改。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"5.1 CopyOnWrite 原理簡述"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那 StateMap 如何來保證修改數據的時候，不會修改 Snapshot 的數據呢？其實原理很簡單：StateMap 和 Snapshot 共享了一大堆數據，既然 Snapshot 要求數據不能修改，那麼 StateMap 在修改某條數據時可以將這條數據複製一份產生一個副本，所以 Snapshot 和 StateMap 就會各自擁有自己的副本，所以 StateMap 對數據的修改就不會影響 Snapshot 的快照。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然爲了節省內存和提高效率，StateMap 只會拷貝那些要改變的數據，儘量多的實現共享，不能實現共享的數據只能 Copy 一份再修改了，這就是類名用 CopyOnWrite 修飾的原因。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"5.2 CopyOnWrite 原理詳解"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上一部分 Snapshot 時，僅僅對 Table 做了一份淺拷貝，而且可以看到拷貝前後，桶內的數據不變，且桶跟桶之間是沒有交集的，所以這裏的原理詳解主要就分析一個桶中的鏈表如何實現 CopyOnWrite。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.2.1 修改鏈表頭部節點的場景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/61/61e20739e1cb5f14c15d862c9059d59a.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上圖所示，primaryTable 和 snapshotTable 的 0 號桶都指向 Entry a，假設現在應用層要修改 Entry a 的數據，整體流程："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深拷貝一個 Entry a 對象爲 Entry a copy"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將 Entry a copy 放到 primaryTable 的鏈表中，且 next 指向 Entry b"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用層修改 Entry a copy 的 data，將 data1 修改爲設定的 data2"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏 Entry b 和 c 沒有修改，所以不用拷貝，屬於 primaryTable 和 snapshotTable 共享的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏就引出了 CopyOnWriteStateMap 的設計目標（自己的理解，並不是官方觀點）：在保證 Snapshot 數據正確性的前提下，儘量的少拷貝數據提高性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.2.2 修改鏈表中間節點的場景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/63/63b87bc1647f6fe76914331d700f450d.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上圖所示，primaryTable 和 snapshotTable 的 0 號桶都指向 Entry a，假設現在應用層要修改 Entry b 的數據，整體流程："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深拷貝一個 Entry b 對象爲 Entry b copy"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將 Entry b copy 串在 primaryTable 的鏈表中，且 next 指向 Entry c"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用層修改 Entry b copy 的 data，將 data 修改爲設定的 data2"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是上述流程成立嗎？如上圖所示 Entry a 和 c 是 primaryTable 和 snapshotTable 共享的。每個 Entry 只有一個 next 指針，所以 Entry a 可以同時指向 Entry b 和 b copy 嗎？肯定是不可以的，所以 Entry a 不可以共享。下圖是正確流程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d1/d103699049f578e432dd51cd1d945098.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖所示，在修改 Entry b 時，不僅僅要將 Entry b 拷貝一份，而且還要將鏈表中 Entry b 之前的 Entry 必須全部 copy 一份，這樣才能保證在滿足正確性的前提下修改 Entry b，畢竟正確性是第一位。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正確整體流程："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深拷貝 Entry a 和 b 對象爲 Entry a copy 和 b copy"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將 Entry a copy 和 b copy 串在 primaryTable 的鏈表中，且 Entry b 的 next 指向 Entry c"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用層修改 Entry b copy 的 data，將 data 修改爲設定的 data2"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"總結："},{"type":"text","text":"假設要修改 Entry b，那麼要將 Entry b 以及鏈表中 Entry b 之前的 Entry 必須全部 copy 一份，Entry b 之後的 Entry 可以共享。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.2.3 插入新數據的場景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f3/f30a04737a4bd4e6951aa5fec89517e4.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上圖所示是插入新數據的場景，會使用頭插法插入 Entry d，頭插法不需要拷貝原始鏈表的任何數據，只需要插入最新的數據到鏈表頭部即可。這樣 primaryTable 可以訪問到插入的數據，且不影響 SnapshotData 訪問原始快照的數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注：這裏必須是插入新數據的場景，對於 Map 類型，插入舊數據對應的可能是修改操作"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"■ 5.2.4 鏈表頭部有新節點再修改鏈表中間節點的場景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7e/7e3e53c21a190997ea72f6d9a18fb6e9.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上圖所示是鏈表頭部有新節點 Entry d 再修改 Entry b 的場景，此時正確的流程是："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深拷貝 Entry a 和 b 對象爲 Entry a copy 和 b copy"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將 Entry a copy 和 b copy 串在 Entry d 的鏈表中，且 Entry b 的 next 指向 Entry c"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"應用層修改 Entry b copy 的 data，將 data 修改爲設定的 data2"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"之前說過要修改 Entry b 需要將 Entry b 之前的 Entry 全部 copy 一份，但是此時並不需要對 Entry d 進行 copy。之前 copy 是因爲 Entry b 之前的元素有被 snapshotData 引用，但是這裏 Entry d 並不被 snapshotData 引用，只有 primaryTable 只有 Entry d，所以不需要 copy。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"修改 Entry b 時，Entry b 之前的 Entry 哪些需要 copy，哪些不需要 copy，具體如何區分會在後續的源碼環節詳細介紹。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.2.5 get 鏈表中間節點的場景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"理論來講，訪問中間節點的場景數據數據是非常安全的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2c/2c2507b6d72a9466c9ff506176a0a904.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下圖所示 Flink 應用層通過 primaryTable 訪問 Entry b，理論來講只是讀取的場景就不需要 copy 副本了。因爲之前 copy 副本都是因爲應用層修改了數據，爲了保證 Snapshot 數據的不可變特性，所以專門 copy 一個副本讓 primaryTable 去修改。但神奇的是 CopyOnWriteStateMap 在 get 操作時，也需要將 Entry b 以及 Entry b 之前的所有 Entry 拷貝一個副本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲什麼呢？雖然是 get 訪問操作，但是應用層拿到了 Entry b 中的 data 對象，萬一應用層修改了 data 對象裏的屬性怎麼辦呢？例如 Entry 中的 data 是 Person 對象，Person 對象可能有一些 setter 方法，可以修改其 name 和 age。如果應用層修改了 name 或 age，那麼在 Snapshot 的過程中，還是出現了數據修改的情況。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"所以 CopyOnWriteStateMap 把 get 操作跟 put 操作同等對待，無論是 get 還是 put 都需要將 Entry 及其之前的 Entry copy 一份。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.2.6 remove 數據的場景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"需要區分兩種 case：remove 的 Entry 是鏈表頭節點；remove 的 Entry 不是鏈表頭節點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Case1："},{"type":"text","text":"remove 的 Entry 是鏈表頭節點的場景比較簡單，將桶直接指向 Entry a 的 next Entry b 即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ec/ec4fc6b455f7332d96171b9371651447.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Case 2："},{"type":"text","text":"remove 的 Entry 不是鏈表頭節點，需要將 Entry b 之前的所有 Entry 拷貝一份（新插入的 Entry 不需要拷貝），且 Entry b 前一個節點的副本直接指向 Entry b 的下一個節點。具體爲什麼 Entry a 需要拷貝一份與 put 和 get 操作類似，因爲 Entry a 的 next 指針沒辦法指向兩個節點，所以 primaryTable 和 snapshotTable 要有各自的頭結點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c7/c7be3e71307a3c84a69ffa337d042dfb.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.2.7 COW 原理小結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上述 case 基本覆蓋到了各種場景，這裏做一個總結："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"插入新的 Entry 使用頭插法插入到鏈表中"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假設要修改 Entry b，那麼要將 Entry b 以及鏈表中 Entry b 之前的 Entry 必須全部 copy 一份（新插入的數據不需要拷貝），Entry b 之後的 Entry 可以共享"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"訪問 Entry b 的場景與修改 Entry b 的場景類似"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假如修改或訪問的數據是 copy 後的數據，那麼實際上不需要再 copy 了，因爲 copy 後的數據已經保證是 primaryTable 獨佔的數據，不與 Snapshot 共享"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"remove 數據的場景，分爲兩種 case："}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果 remove 的 Entry 是鏈表頭節點，將桶直接指向頭結點的 next 節點即可。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":1,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果 remove 的 Entry 不是鏈表頭節點，需要將目標 Entry 之前的所有 Entry 拷貝一份，且目標 Entry 前一個節點的副本直接指向目標 Entry 的下一個節點。當然如果前繼節點已經是新版本了，則不需要拷貝，直接修改前繼 Entry 的 next 指針即可。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"5.3 CopyOnWriteStateMap 各種操作源碼詳解"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.3.1 CopyOnWriteStateMap 介紹"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 類用於存儲數據，支持了 CopyOnWrite 的功能，先介紹 CopyOnWriteStateMap 中一些相對重要的字段，相關源碼如下所示（重點看一下每個字段的註釋）："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public class CopyOnWriteStateMap extends StateMap {\n // 默認容量 128，即：hash 表中桶的個數默認 128\n public static final int DEFAULT_CAPACITY = 128;\n\n // hash 擴容遷移數據時，每次最少要遷移 4 條數據\n private static final int MIN_TRANSFERRED_PER_INCREMENTAL_REHASH = 4;\n\n // State 的序列化器\n protected final TypeSerializer stateSerializer;\n\n // 空表：提前創建好\n private static final StateMapEntry, ?, ?>[] EMPTY_TABLE = \n new StateMapEntry[MINIMUM_CAPACITY >>> 1];\n\n // 當前 StateMap 的 version，每次創建一個 Snapshot 時，StateMap 的版本號加一\n private int stateMapVersion;\n\n // 所有 正在進行中的 snapshot 的 version\n // 每次創建出一個 Snapshot 時，都需要將 Snapshot 的 version 保存到該 Set 中\n private final TreeSet snapshotVersions;\n\n // 正在進行中的那些 snapshot 的最大版本號\n // 這裏保存的就是 TreeSet snapshotVersions 中最大的版本號\n private int highestRequiredSnapshotVersion;\n\n // 主表：用於存儲數據的 table\n private StateMapEntry[] primaryTable;\n\n // 擴容時的新表，擴容期間數組長度爲 primaryTable 的 2 倍。\n // 非擴容期間爲 空表\n private StateMapEntry[] incrementalRehashTable;\n\n // primaryTable 中元素個數\n private int primaryTableSize;\n\n // incrementalRehashTable 中元素個數\n private int incrementalRehashTableSize;\n\n // primary table 中增量 rehash 要遷移的下一個 index\n // 即：primaryTable 中 rehashIndex 之前的數據全部搬移完成\n private int rehashIndex;\n\n // 擴容閾值，與 HashMap 類似，當元素個數大於 threshold 時，就會開始擴容。\n // 默認 threshold 爲 StateMap 容量 * 0.75\n private int threshold;\n\n // 用於記錄元素修改的次數，遍歷迭代過程中，發現 modCount 修改了，則拋異常\n private int modCount;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其中  primaryTable 字段是真正存儲數據的 hash 表，primaryTable 是 StateMapEntry 類型的數據，StateMapEntry 用於存儲 StateMap 中的一條數據，下面介紹 StateMapEntry。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.3.2 StateMapEntry"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"StateMapEntry 是 CopyOnWriteStateMap 中真正存儲數據的實體。在 Java 的 HashMap 中也是將數據封裝在 Entry 中，HashMap 的 Entry 源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"static class Node implements Map.Entry {\n // 當前 key 對應的 hash 值\n final int hash;\n final K key;\n V value;\n // next 指向當前桶中下一個 Node\n Node next;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HashMap 中的靜態內部類 Node 實現 Map.Entry，類中有四個字段：hash、key、value、next。key 和 value 不同解釋，hash 表示當前 key 對應的 hash 值，next 指向當前桶中下一個 Node。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HashMap 在 get(key) 查找數據流程："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據 key 計算 hash 值，定位到具體的桶"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"遍歷當前桶的一個個 Entry，先比較 hash 值是否相同，在比較 key 是否相同(使用 equals 判斷 key 是否相同)"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果 hash 值和 key 的 equals 方法都能匹配，表示找到了對應的 Entry，返回 Entry 中的 value 即可"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"StateMapEntry 源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"protected static class StateMapEntry implements StateEntry {\n final K key;\n final N namespace;\n S state;\n final int hash;\n StateMapEntry next;\n // new entry 時的版本號\n int entryVersion;\n // state （數據）更新時的 版本號\n int stateVersion;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"StateMapEntry 與 HashMap 的 Entry 相似度較高，其他 key、hash、next 這三個屬性完全相同，StateMapEntry 中的 state 表示 HashMap 中的 value，即：具體存儲的數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"StateMapEntry 相比 HashMap 的 Entry，多了三個字段："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"namespace：namespace 是 Flink 中的概念，用於區分不同的 Window，在 StateMapEntry 中 key 和 namespace 組合起來作爲共同的主鍵，state 作爲 value"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"entryVersion：表示創建 entry 時的版本號"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"stateVersion：表示當前 StateMapEntry 中 state （數據）更新時的版本號"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於 key 和 namespace 共同作爲主鍵，因此在 CopyOnWriteStateMap 的 get 或 put 操作中，判斷是否找到了匹配的 Entry，不僅要判斷 hash 值，還要通過 equals 方法對 key 和 namespace 進行判斷。三個參數都校驗通過才能表示找到了相應的 Entry。這一點是與 HashMap 區別較大的，要注意理解。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.3.3 插入新數據源碼流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 類的 put 方法如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public void put(K key, N namespace, S value) {\n // putEntry 用於找到對應的 Entry，\n // 包括了修改數據或插入新數據的場景\n final StateMapEntry e = putEntry(key, namespace);\n\n // 將 value set 到 Entry 中\n e.state = value;\n // state 更新了，所以要更新 stateVersion\n e.stateVersion = stateMapVersion;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"put 方法直接調用 putEntry 方法，putEntry 用於找到對應的 Entry，putEntry 包括了修改數據或插入新數據的場景。找到 Entry 後，將 value set 到 Entry 中。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"putEntry 方法源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"private StateMapEntry putEntry(K key, N namespace) {\n // 計算當前對應的 hash 值，選擇 primaryTable 或 incrementalRehashTable\n final int hash = computeHashForOperationAndDoIncrementalRehash(key, namespace);\n final StateMapEntry[] tab = selectActiveTable(hash);\n int index = hash & (tab.length - 1);\n\n // 遍歷當前桶中鏈表的一個個 Entry\n for (StateMapEntry e = tab[index]; e != null; e = e.next) {\n // 如果根據 key 和 namespace 找到了對應的 Entry，則認爲是修改數據\n // 普通的 HashMap 結構有一個 Key ，而這裏 key 和 namespace 的組合當做 key\n if (e.hash == hash && key.equals(e.key) && namespace.equals(e.namespace)) {\n // 修改數據邏輯（暫時忽略）\n if (e.entryVersion < highestRequiredSnapshotVersion) {\n e = handleChainedEntryCopyOnWrite(tab, index, e);\n }\n\n // 修改數據，直接返回對應的 Entry\n return e;\n }\n }\n\n // 代碼走到這裏，說明原始的鏈表中沒找到對應 Entry，即：插入新數據的邏輯\n ++modCount;\n if (size() > threshold) {\n doubleCapacity();\n }\n\n // 鏈中沒有找到 key 和 namespace 的數據\n return addNewStateMapEntry(tab, key, namespace, hash);\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"putEntry 方法首先會計算當前 key 和 namespace 對應的 hash 值，使用 selectActiveTable 選擇使用 primaryTable 或 incrementalRehashTable，然後計算當前元素對應桶的 index。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏注意，普通的 HashMap 結構有一個 Key 一個 value。而這裏 key 和 namespace 的組合當做 Map 的 key，value 仍然是原來的 value。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"遍歷當前桶中鏈表的一個個 Entry，如果通過 hash 值、 key 和 namespace 的 equals 方法進行匹配，如果匹配成功，表示找到了對應的 Entry，則認爲是修改數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果遍歷完當前桶中鏈表的所有元素還沒找到匹配的 Entry，說明是插入一條新數據，則執行 addNewStateMapEntry 方法往鏈表頭部插入一個新的 Entry 返回（頭插法）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.3.4 修改數據源碼流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 putEntry 中，修改數據場景的源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// 如果根據 key 和 namespace 找到了對應的 Entry，則認爲是修改數據\n// 普通的 HashMap 結構有一個 Key ，而這裏 key 和 namespace 的組合當做 key\nif (e.hash == hash && key.equals(e.key) && namespace.equals(e.namespace)) {\n // entryVersion 表示 entry 創建時的版本號\n // highestRequiredSnapshotVersion 表示 正在進行中的那些 snapshot 的最大版本號\n // entryVersion 小於 highestRequiredSnapshotVersion，說明 Entry 的版本小於當前某些 Snapshot 的版本號，\n // 即：當前 Entry 是舊版本的數據，當前 Entry 被其他 snapshot 持有。\n // 爲了保證 Snapshot 的數據正確性，這裏必須爲 e 創建新的副本，且 e 之前的某些元素也需要 copy 副本\n // handleChainedEntryCopyOnWrite 方法將會進行相應的 copy 操作，並返回 e 的新副本\n // 然後將返回 handleChainedEntryCopyOnWrite 方法返回的 e 的副本返回給上層，進行數據的修改操作。\n if (e.entryVersion < highestRequiredSnapshotVersion) {\n e = handleChainedEntryCopyOnWrite(tab, index, e);\n }\n\n // 反之，entryVersion >= highestRequiredSnapshotVersion\n // 說明當前 Entry 創建時的版本比所有 Snapshot 的版本高\n // 即：當前 Entry 是新版本的數據，不被任何 Snapshot 持有\n // 注：Snapshot 不可能引用高版本的數據\n // 此時，e 是新的 Entry，不存在共享問題，所以直接修改當前 Entry 即可，所以返回當前 e \n return e;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏是上一部分插入新數據的部分源碼，現在重點講述修改數據的過程。如果根據 key 和 namespace 找到了相應的 Entry，則認爲是對老數據的修改，走相應的修改邏輯。然後判斷當前 Entry 的 entryVersion 是否小於 highestRequiredSnapshotVersion。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"entryVersion 表示 entry 創建時的版本號，highestRequiredSnapshotVersion 表示正在進行中的那些 snapshot 的最大版本號。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"entryVersion 小於 highestRequiredSnapshotVersion，說明 Entry 創建時的版本小於當前某些 Snapshot 的版本號，即：當前 Entry 是舊版本的數據，當前 Entry 被其他 Snapshot 持有。爲了保證 Snapshot 的數據正確性，這裏必須爲 e 創建新的副本，且 e 之前的某些元素也需要 copy 副本，handleChainedEntryCopyOnWrite 方法將會進行相應的 copy 操作，並返回 e 的新副本。最後將 e 的副本返回給上層，進行數據的修改操作。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"反之，entryVersion >= highestRequiredSnapshotVersion，說明當前 Entry 創建時的版本比所有 Snapshot 的版本高。Snapshot 不可能引用高版本的數據，所以當前 Entry 是新版本的數據不被任何 Snapshot 持有。此時 e 是新的 Entry，不存在共享問題，所以直接修改當前 Entry 即可，所以返回當前 e。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"handleChainedEntryCopyOnWrite 方法的作用：爲 Entry e 創建新的副本，且鏈表中 Entry e 之前某些元素也需要 copy 副本，最後返回 e 的副本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那哪些元素應該拷貝，哪些元素不應該拷貝呢？Snapshot 之後新創建的 Entry 就不需要再拷貝了，Snapshot 之前創建的 Entry 會被 Snapshot 引用所以需要再拷貝。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"handleChainedEntryCopyOnWrite 的源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"private StateMapEntry handleChainedEntryCopyOnWrite(\n StateMapEntry[] tab,\n int mapIdx,\n StateMapEntry untilEntry) {\n\n // current 指向當前桶的頭結點\n StateMapEntry current = tab[mapIdx];\n StateMapEntry copy;\n\n // 判斷頭結點創建時的版本是否低於 highestRequiredSnapshotVersion\n // 如果低於，則 current 節點被 Snapshot 引用，所以需要 new 一個新的 Entry\n if (current.entryVersion < highestRequiredSnapshotVersion) {\n copy = new StateMapEntry<>(current, stateMapVersion);\n tab[mapIdx] = copy;\n } else {\n copy = current;\n }\n\n // 依次遍歷當前桶的元素，直到遍歷到 untilEntry 節點，也就是我們要修改的 Entry 節點\n while (current != untilEntry) {\n current = current.next;\n // current 版本小於 highestRequiredSnapshotVersion，則需要拷貝，\n // 否則不用拷貝\n if (current.entryVersion < highestRequiredSnapshotVersion) {\n // entryVersion 表示創建 Entry 時的 version，\n // 所以新創建的 Entry 對應的 entryVersion 要更新爲當前 StateMap 的 version\n copy.next = new StateMapEntry<>(current, stateMapVersion);\n copy = copy.next;\n } else {\n copy = current;\n }\n }\n return copy;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從源碼可以看到，，從頭結點到要修改的 Entry 節點依次遍歷桶中元素，都是使用 "},{"type":"text","marks":[{"type":"strong"}],"text":"current.entryVersion < highestRequiredSnapshotVersion "},{"type":"text","text":"來判斷當前節點的創建創建時的版本是否低於 highestRequiredSnapshotVersion。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果低於則 current 節點被 Snapshot 引用，所以需要 new 一個新的 Entry，也就是所謂的拷貝一個副本。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"否則不用拷貝。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在新創建 Entry 時，新 Entry 的 entryVersion 要更新爲當前 StateMap 的 version，表示這是一個新版本的 Entry，並沒有被 Snapshot 引用。這樣之後再要修改該 Entry 時直接修改該 Entry 即可，不需要再拷貝一份副本了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.3.5 訪問數據源碼流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWriteStateMap 類的 get 方法與 putEntry 類似，都是依次遍歷相應桶的元素，直到根據 key 和 namespace 找到了相應的 Entry，則返回相應的 Entry。如果遍歷完相應桶的所有 Entry，都沒有與 key 和 namespace 相匹配的 Entry，則表示 StateMap 中沒有指定的元素則返回 null。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果找到了相應 Entry，爲了保證 Snapshot 引用的數據不被修改，所以也要進行拷貝操作。除了拷貝其他源碼比較簡單與 putEntry 完成類似，所以重點分析找到 Entry 後的相關源碼。相關源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"if ((e.hash == hash && key.equals(eKey) && namespace.equals(eNamespace))) {\n // 一旦 get 當前數據，爲了防止應用層修改數據內部的屬性值，\n // 所以必須保證這是一個最新的 Entry，並更新其 stateVersion\n\n // 首先檢查當前的 State，也就是 value 值是否是舊版本數據，\n // 如果 value 是舊版本，則必須深拷貝一個 value\n // 否則 value 是新版本，直接返回給應用層\n if (e.stateVersion < requiredVersion) {\n // 此時還有兩種情況，\n // 1、如果當前 Entry 是舊版本的，則 Entry 也需要拷貝一份，\n // 按照之前分析過的 handleChainedEntryCopyOnWrite 策略拷貝即可\n // 2、當前 Entry 是新版本數據，則不需要拷貝，直接修改其 State 即可\n if (e.entryVersion < requiredVersion) {\n e = handleChainedEntryCopyOnWrite(tab, hash & (tab.length - 1), e);\n }\n // 更新其 stateVersion\n e.stateVersion = stateMapVersion;\n // 通過序列化器，深拷貝一個數據\n e.state = getStateSerializer().copy(e.state);\n }\n\n return e.state;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一旦 get 當前數據，爲了防止應用層修改數據內部的屬性值，所以必須保證這是一個最新的 Entry，並更新其 stateVersion。首先檢查當前的 State，也就是 value 值是否是舊版本數據："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果 value 是舊版本，則必須深拷貝一個 value"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"否則 value 是新版本，直接返回給應用層"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果 value 值是還區分兩種情況："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、如果當前 Entry 是舊版本的，則 Entry 也需要拷貝一份，按照之前分析過的 handleChainedEntryCopyOnWrite 策略拷貝即可"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2、當前 Entry 是新版本數據，則不需要拷貝，直接修改其 State 即可"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"case 1 容易理解，如下圖所示訪問 Entry b 就是 case 1 的場景，需要使用 handleChainedEntryCopyOnWrite 方法對 Entry b 和 a 進行拷貝操作，然後再對 Entry b 的 value 對象進行一次深拷貝，所以 Entry b 和 b copy 不會共享 data 對象。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2c/2c2507b6d72a9466c9ff506176a0a904.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雖然 Entry a 也拷貝了一份生成 Entry a copy，但是 Entry a 中的 value 對象並沒有深拷貝一份，而是共享 data1 對象。get Entry b 後 Entry a 和 a copy 引用 data 1 的圖示用下圖會更形象一些，即：Entry a 和 a copy 的 state 會共同引用 data1 對象。對於修改 Entry a 如果下次再有 get 操作，就會對應上述的 case 2 場景：stateVersion 是老版本，但是 Entry a copy 屬於新版本。此時不需要再對 Entry 進行復制操作，只需要對 State 進行一次深拷貝，保證不會將 Entry a 的 State 返回給應用層。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3f/3f7cc843c0734e2ceed1b216086c55ba.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"■ 5.3.6 remove 數據源碼流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"removeEntry 源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"private StateMapEntry removeEntry(K key, N namespace) {\n\n final int hash = computeHashForOperationAndDoIncrementalRehash(key, namespace);\n final StateMapEntry[] tab = selectActiveTable(hash);\n int index = hash & (tab.length - 1);\n\n for (StateMapEntry e = tab[index], prev = null; \n e != null; prev = e, e = e.next) {\n if (e.hash == hash && key.equals(e.key) && namespace.equals(e.namespace)) {\n // 如果要刪除的 Entry 不存在前繼節點，說明要刪除的 Entry 是頭結點，\n // 直接將桶直接指向頭結點的 next 節點即可。\n if (prev == null) {\n tab[index] = e.next;\n } else {\n // 如果 remove 的 Entry 不是鏈表頭節點，需要將目標 Entry 之前的所有 Entry 拷貝一份，\n // 且目標 Entry 前一個節點的副本直接指向目標 Entry 的下一個節點。\n // 當然如果前繼節點已經是新版本了，則不需要拷貝，直接修改前繼 Entry 的 next 指針即可。\n // copy-on-write check for entry\n if (prev.entryVersion < highestRequiredSnapshotVersion) {\n prev = handleChainedEntryCopyOnWrite(tab, index, prev);\n }\n prev.next = e.next;\n }\n // 修改一些計數器\n ++modCount;\n if (tab == primaryTable) {\n --primaryTableSize;\n } else {\n --incrementalRehashTableSize;\n }\n return e;\n }\n }\n return null;\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"remove 數據的場景，分爲兩種 case："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果 remove 的 Entry 是鏈表頭節點，將桶直接指向頭結點的 next 節點即可。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果 remove 的 Entry 不是鏈表頭節點，需要將目標 Entry 之前的所有 Entry 拷貝一份，且目標 Entry 前一個節點的副本直接指向目標 Entry 的下一個節點。當然如果前繼節點已經是新版本了，則不需要拷貝，直接修改前繼 Entry 的 next 指針即可。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"源碼比較清晰加上已經詳細分析了 put 和 get 源碼，所以 remove 源碼直接結合原理看註釋即可。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"6.Snapshot 流程及完成後的 release 操作"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面已經分析了 CopyOnWriteStateMap 的擴容 rehash 原理和源碼、Snapshot 時淺拷貝原理和源碼以及CopyOnWrite 實現的原理和源碼。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CopyOnWrite 的實現主要爲了減少 Checkpoint 同步階段的停頓時間，將數據的快照過程儘量放到異步流程。下面分析 Snapshot 異步快照流程及 Snapshot 完成後 release 相關操作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"HeapSnapshotStrategy 類的 AsyncSnapshotCallable 匿名內部類的 callInternal 方法中會調用 AbstractStateTableSnapshot 的 writeStateInKeyGroup 方法，並依次將每個 KeyGroupId 當做參數傳入。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"writeStateInKeyGroup 方法源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"public void writeStateInKeyGroup(@Nonnull DataOutputView dov, int keyGroupId) {\n // 獲取 KeyGroupId 對應的 CopyOnWriteStateMapSnapshot\n StateMapSnapshot> stateMapSnapshot = \n getStateMapSnapshotForKeyGroup(keyGroupId);\n // 將 stateMapSnapshot 中的 State 數據進行序列化輸出\n stateMapSnapshot.writeState(localKeySerializer, localNamespaceSerializer, \n localStateSerializer, dov, stateSnapshotTransformer);\n // stateMapSnapshot 對應的數據已經遍歷完了，所以可以釋放該快照\n stateMapSnapshot.release();\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"writeStateInKeyGroup 方法拿到 KeyGroupId 對應的 CopyOnWriteStateMapSnapshot，然後將 stateMapSnapshot 中的 State 數據進行序列化輸出，這一步就會依次遍歷 stateMapSnapshot 所有引用的數據序列化輸出到外部存儲中。序列化完成就可以釋放該快照了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"release 最後會調用 CopyOnWriteStateMap 的 releaseSnapshot 方法，releaseSnapshot 方法源碼如下所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"void releaseSnapshot(int snapshotVersion) {\n synchronized (snapshotVersions) {\n // 將 相應的 snapshotVersion 從 snapshotVersions 中 remove\n snapshotVersions.remove(snapshotVersion);\n // 將 snapshotVersions 的最大值更新到 highestRequiredSnapshotVersion，\n // 如果snapshotVersions 爲空，則 highestRequiredSnapshotVersion 更新爲 0\n highestRequiredSnapshotVersion = snapshotVersions.isEmpty() ? \n 0 : snapshotVersions.last();\n }\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"releaseSnapshot 方法將相應的 snapshotVersion 從 snapshotVersions 中 remove，並將 snapshotVersions 的最大值更新到 highestRequiredSnapshotVersion，如果snapshotVersions 爲空，則 highestRequiredSnapshotVersion 更新爲 0。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有個小疑問：根據之前的流程分析，Snapshot 過程中如果 Flink 應用層發生了大量 get 和 put 操作，那麼很多 Entry 和 State 都會出現多個副本。Snapshot 結束後，就應該把那些舊版本的數據清理掉。可是沒有看到對舊版本數據進行清理操作呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2c/2c2507b6d72a9466c9ff506176a0a904.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上圖所示，Entry b 和 a 都存在副本，當 Snapshot 結束後，因爲新數據在 Entry a copy 和 b copy 中，所以 Entry a 和 b 都應該被清理掉，留着 Entry a copy 和 b copy 即可。但是代碼中沒有看到去清理 Entry a 和 b。那麼會不會出現內存泄漏的問題呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其實並不會，Snapshot 結束後 snapshotData 對應的 hash 表不會再被異步快照的線程引用，所以 Entry a 和 b 就會變成不可達對象，會被 JVM 的 GC 回收掉。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"7.總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文詳細介紹了 CopyOnWriteStateTable 的設計原理及相關源碼，主要從 rehash 和 CopyOnWrite 兩個點進行深入剖析，希望對大家能有所幫助。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文涉及的 github 倉庫，都在 feature/source-code-read-1-9-0 分支，之後也會持續更新："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"https://github.com/1996fanrui/flink/tree/feature/source-code-read-1-9-0註釋"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}