Redis 緩存的三大問題及其解決方案

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis經常用於系統中的緩存，這樣可以解決目前IO設備無法滿足互聯網應用海量的讀寫請求的問題。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"一、緩存穿透","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"緩存穿透是指緩存和數據庫中都沒有的數據，而用戶不斷髮起請求，如發起id爲-1的數據或者特別大的不存在的數據。有可能是黑客利用漏洞攻擊從而去壓垮應用的數據庫。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1. 常見解決方案","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於緩存穿透問題，常見的解決方案有以下三種：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"驗證攔截：接口層進行校驗，如鑑定用戶權限，對ID之類的字段做基礎的校驗，如id<=0的字段直接攔截；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"緩存空數據：當數據庫查詢到的數據爲空時，也將這條數據進行緩存，但緩存的有效性設置得要較短，以免影響正常數據的緩存；","attrs":{}}]}]}],"attrs":{}},{"type":"codeblock","attrs":{"lang":"java"},"content":[{"type":"text","text":"Copypublic Student getStudentsByID(Long id) {\n\n // 從Redis中獲取學生信息\n Student student = redisTemplate.opsForValue()\n .get(String.valueOf(id));\n if (student != null) {\n return student;\n }\n\n // 從數據庫查詢學生信息，並存入Redis\n student = studentDao.selectByStudentId(id);\n if (student != null) {\n redisTemplate.opsForValue()\n .set(String.valueOf(id), student, 60, TimeUnit.MINUTES);\n } else {\n // 即使不存在，也將其存入緩存中\n redisTemplate.opsForValue()\n .set(String.valueOf(id), null, 60, TimeUnit.SECONDS);\n }\n\n return student;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用布隆過濾器：布隆過濾器是一種比較獨特數據結構，有一定的誤差。當它指定一個數據存在時，它不一定存在，但是當它指定一個數據不存在時，那麼它一定是不存在的。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2. 布隆過濾器","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"布隆過濾器是一種比較特殊的數據結構，有點類似與HashMap，在業務中我們可能會通過使用HashMap來判斷一個值是否存在，它可以在O(1)時間複雜度內返回結果，效率極高，但是受限於存儲容量，如果可能需要去判斷的值超過億級別，那麼HashMap所佔的內存就很可觀了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而BloomFilter解決這個問題的方案很簡單。首先用多個bit位去代替HashMap中的數組，這樣的話儲存空間就下來了，之後就是對 Key 進行多次哈希，將 Key 哈希後的值所對應的 bit 位置爲1。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當判斷一個元素是否存在時，就去判斷這個值哈希出來的比特位是否都爲1，如果都爲1，那麼可能存在，也可能不存在（如下圖F）。但是如果有一個bit位不爲1，那麼這個Key就肯定不存在。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/85/851a1cbd67e8f4d1bb43179b363982dd.webp","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注意：BloomFilter並不支持刪除操作，只支持添加操作。這一點很容易理解，因爲你如果要刪除數據，就得將對應的bit位置爲0，但是你這個Key對應的bit位可能其他的Key也對應着。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3. 緩存空數據與布隆過濾器的比較","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面對這兩種方案都進行了簡單的介紹，緩存空數據與布隆過濾器都能有效解決緩存穿透問題，但使用場景有着些許不同；","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當一些惡意攻擊查詢查詢的key各不相同，而且數量巨多，此時緩存空數據不是一個好的解決方案。因爲它需要存儲所有的Key，內存空間佔用高。並且在這種情況下，很多key可能只用一次，所以存儲下來沒有意義。所以對於這種情況而言，使用布隆過濾器是個不錯的選擇；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而對與空數據的Key數量有限、Key重複請求效率較高的場景而言，可以選擇緩存空數據的方案。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"二、緩存擊穿緩","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"存擊穿是指當前熱點數據存儲到期時，多個線程同時併發訪問熱點數據。因爲緩存剛過期，所有併發請求都會到數據庫中查詢數據。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"解決方案","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將熱點數據設置爲永不過期；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"加互斥鎖：互斥鎖可以控制查詢數據庫的線程訪問，但這種方案會導致系統的吞吐量下降，需要根據實際情況使用。","attrs":{}}]}]}],"attrs":{}},{"type":"codeblock","attrs":{"lang":"java"},"content":[{"type":"text","text":"Copypublic String get(key) {\n String value = redis.get(key);\n if (value == null) { // 代表緩存值過期\n // 設置3min的超時，防止del操作失敗的時候，下次緩存過期一直不能load db\n if (redis.setnx(key_mutex, 1, 3 * 60) == 1) { // 代表設置成功\n value = db.get(key);\n redis.set(key, value, expire_secs);\n redis.del(key_mutex);\n } else { // 這個時候代表同時候的其他線程已經load db並回設到緩存了，這時候重試獲取緩存值即可\n sleep(50);\n get(key); // 重試\n }\n } else {\n return value;\n }\n}\n","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"三、緩存雪崩","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"緩存雪崩發生有幾種情況，比如大量緩存集中在或者緩存同時在大範圍中失效，出現了大量請求去訪問數據庫，從而導致CPU和內存過載，甚至停機。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個簡單的雪崩過程：","attrs":{}}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"Redis 集羣產生了大面積故障；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"緩存失敗，此時仍有大量請求去訪問 Redis 緩存服務器；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"在大量 Redis 請求失敗後，這些請求將會去訪問數據庫；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"由於應用的設計依賴於數據庫和 Redis 服務，很快就會造成服務器集羣的雪崩，最終導致整個系統的癱瘓。","attrs":{}}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"解決方案","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"【事前】高可用緩存：高可用緩存是防止出現整個緩存故障。即使個別節點，機器甚至機房都關閉，系統仍然可以提供服務，Redis 哨兵(Sentinel) 和 Redis 集羣(Cluster) 都可以做到高可用；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"【事中】緩存降級（臨時支持）：當訪問次數急劇增加導致服務出現問題時，我們如何確保服務仍然可用。在國內使用比較多的是 Hystrix，它通過熔斷、降級、限流三個手段來降低雪崩發生後的損失。只要確保數據庫不死，系統總可以響應請求，每年的春節 12306 我們不都是這麼過來的嗎？只要還可以響應起碼還有搶到票的機會；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"【事後】Redis備份和快速預熱：Redis數據備份和恢復、快速緩存預熱。","attrs":{}}]}]}],"attrs":{}}]}