Redis 緩存性能實踐及總結

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"一、前言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在互聯網應用中，緩存成爲高併發架構的關鍵組件。這篇博客主要介紹緩存使用的典型場景、實操案例分析、Redis使用規範及常規 Redis 監控。"}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"二、常見緩存對比"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"常見的緩存方案，有本地緩存，包括HashMap/ConcurrentHashMap、Ehcache、Memcache、Guava Cache等，緩存中間件包括Redis、Tair等。"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/46/46d073c60452425a45a10811115f4612.webp","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"三、Redis使用場景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"underline"},{"type":"strong"}],"text":"1. 計數"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis實現快速計數及緩存功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如：視頻或直播在線觀看人數，用戶每播放一次，就會自增1。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"underline"},{"type":"strong"}],"text":"2. Session集中管理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Session可以存儲在應用服務是JVM中，但這一種方案會有一致性的問題，還有高併發下，會引發JVM內存溢出。Redis將用戶的Session集中管理，這種情況下只要保證Redis的高可用和擴展性，每次用戶更新或查詢登錄都直接從Redis中信息獲取。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"underline"},{"type":"strong"}],"text":"3.限速"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如：業務要求用戶一分鐘內，只能獲取5次驗證碼。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"underline"},{"type":"strong"}],"text":"4.排行榜"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關係型數據庫在排行榜方面查詢速度普遍偏慢，所以可以藉助redis的SortedSet進行熱點數據的排序。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"比如在項目中，如果需要統計主播的吸金排行榜，可以以主播的id作爲member, 當天打賞的活動禮物對應的熱度值作爲 score, 通過zrangebyscore就可以獲取主播活動日榜。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"underline"},{"type":"strong"}],"text":"5.分佈式鎖"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在實際的多進程併發場景下，使用分佈式鎖來限制程序的併發執行。多用於防止高併發場景下，緩存被擊穿的可能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分佈式鎖的實際就是\"佔坑\"，當另一個進程來執行setnx時，發現標識位已經爲1，只好放棄或者等待。"}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"四、案例解析"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1、【案例】過期設置- set命令會去掉過期時間"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis所有的數據結構，都可以設置過期時間。如果一個字符串已經設置了過期時間，然後重新設置它，會導致過期時間消失。所以在項目中需要合理評估Redis容量，避免因爲頻繁set導致沒有過期策略，間接導致內存被佔滿。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如下是 Redis 源碼截圖："}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/78/788e492c3f8d5d132f8b82035ab1835a.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2、【案例】關於 Jedis 2.9.0 及以下版本過期設置 BUG"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"發現Jedis在進行expiredAt命令調用時有bug，最終調用的是pexpire命令，這個bug會導致key過期時間很長，導致Redis內存溢出等問題。建議升級到Jedis 2.9.1及以上版本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BinaryJedisCluster.java源碼如下："}]},{"type":"codeblock","attrs":{"lang":"java"},"content":[{"type":"text","text":"\n@Override\n public Long pexpireAt(final byte[] key, final long millisecondsTimestamp) {\n return new JedisClusterCommand(connectionHandler, maxAttempts) {\n @Override\n public Long execute(Jedis connection) {\n return connection.pexpire(key, millisecondsTimestamp); //此處pexpire應該是pexpireAt\n }\n }.runBinary(key);\n }"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對比pexpire和pexpireAt："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/96/96c154109d01f350ec658b23da9d855f.webp","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"比如我們當前使用的時間是2018-06-14 17:00:00，它的unix時間戳爲1528966800000毫秒，當我們使用PEXPIREAT命令時，由於是過去的時間，相應的key會立即過期。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而我們誤用了PEXPIRE命令時，key不會立即過期，而是等到1528966800000毫秒後才過期，key過期時間會相當長，約幾W天后，從而可能導致Redis內存溢出、服務器崩潰等問題。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3、【案例】緩存被擊穿"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"緩存的key有過期策略，如果恰好在這個時間點對這個Key有大量的併發請求，這些請求發現緩存過期一般都會從後端DB回源數據並回設到緩存，這個時候大併發的請求可能會瞬間把後端DB壓掛。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"業界常用優化方案有"},{"type":"text","marks":[{"type":"strong"}],"text":"兩種"},{"type":"text","text":"："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"第一種："},{"type":"text","text":"使用分佈式鎖，保證高併發下，僅有一個線程能回源後端DB。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"第二種："},{"type":"text","text":"保證高併發的請求到的Redis key始終是有效的，使用非用戶請求回源後端，改成主動回源。一般可以使用異步任務進行緩存的主動刷新。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4、【案例】Redis-standalone架構禁止使用非0庫"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis執行命令select 0和select 1切換，造成性能損耗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"RedisTemplate在執行execute方法的時候會先獲取鏈接。"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/fc/fc837c1a3e914a20f28c756fde1f9411.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"執行到RedisConnectionUtils.java，會有一段獲取鏈接的方法。"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a9/a9f76da2c25b3de0fcd27c6f22638d14.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JedisConnectionFactory.java 會調用JedisConnection構造器，注意這邊的dbIndex就是數據庫編號，如：1"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/fb/fb39b22bd09a3282dac7a91694ab9eff.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"繼續跟進JedisConnection代碼，當選擇庫大於1時，會有select db操作。如果一直使用0庫是不需要額外執行切庫命令的。知道了第一個切庫select 1的地方，那麼select 0是哪來的呢？"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/72/72c5c0e00dfd8cac3553f4ca5ba8919b.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其實客戶端使用Redis也會是要釋放鏈接的，只不過RedisTemplate已經幫我們自動釋放了，讓我們再回到一開始RedisTemplate執行execute(...)方法的地方。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/49/49c3cf96dc606b96d356ab9c525ef57f.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面還是RedisConnectionUtils.java，執行鏈接關閉的代碼。"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f4/f4295d1b849684ec72c2a0a5414137c6.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按代碼註釋的意思，如果選擇庫編號不爲0，spring-data-redis框架每次都會執行重置select 0！"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/23/23ca3a4196da725c7222f39e594a33b5.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"筆者在vivo商城業務中，商品詳情頁接口經過上面的調優，性能提高了3倍多。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"進一步驗證數據庫切換至少影響性能3倍左右（視具體業務而定）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Rediscluster集羣數據庫，默認0庫，無法選擇其他數據庫，也就避免了這個問題。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"5、【案例】當心時間複雜度o(n)Redis命令"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis是單線程的，所以線程安全的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis使用非阻塞IO，並且大部分命令的時間複雜度O(1)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用高耗時的命令是非常危險的，會佔用唯一的一個線程的大量處理時間，導致所有的請求都被拖慢。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如：獲取所有set集合中的元素 smembers myset，返回指定Hash中所有的member，時間複雜度O(N)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"緩存的Value集合變大，當高並接口請求時，會從Redis讀取相關數據，每個請求讀取的時間變長，不斷的疊加，導致出現熱點KEY情況，Redis某個分片處於阻塞，CPU使用率達到100%。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"6、【案例】緩存熱key"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Redis中，訪問頻率高的key稱爲熱點key，當某一熱點key的請求到Server主機時，由於請求量特別大，導致主機資源不足，甚至宕機，影響正常的服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"熱key問題的產生，有如下"},{"type":"text","marks":[{"type":"strong"}],"text":"兩種原因："}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"用戶消費的數據遠大於生產的數據，比如熱賣商品或秒殺商品、熱點新聞、熱點評論等，這些典型的讀多寫少的場景會產生熱點問題。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"請求分片集中，超過單Server的性能極限，比如 固定名稱key，哈希落入一臺Server，訪問量極大的情況，超過Server極限時，就會導致熱點Key問題的產生。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼在實際業務中，"},{"type":"text","marks":[{"type":"strong"}],"text":"如何識別到熱點key呢？"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"憑藉業務經驗，進行預估哪些是熱key；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"客戶端統計收集，本地統計或者上報；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"如果服務端有代理層，可以在代理層進行收集上報；"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當我們識別到熱key，"},{"type":"text","marks":[{"type":"strong"}],"text":"如何解決熱key問題"},{"type":"text","text":"？"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"Redis集羣擴容：增加分片副本，均衡讀流量；"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"進一步對熱key進行散列，比如將一個key備份爲key1,key2……keyN，同樣的數據N個備份，N個備份分佈到不同分片，訪問時可隨機訪問N個備份中的一個，進一步分擔讀流量。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"使用二級緩存，即本地緩存。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當發現熱key後，將熱key對應數據首先加載到應用服務器本地緩存中，減少對Redis的讀請求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"五、Redis 規範"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1、禁止使用非database 0"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"說明："}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis-standalone架構，禁止使用Redis中的其他database。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原由："}]},{"type":"blockquote","content":[{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲以後業務遷移 Redis Cluster 保持兼容性."}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多個 database 用 select 切換時，更消耗CPU資源。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"更易自動化運維管理，如 scan/dbsize 命令只用於當database。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"部分 Redis Clients 因線程安全問題，不支持單實例多 database。"}]}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2、Key設計規範"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按業務功能命名key前綴，防止key衝突覆蓋，推薦 用冒號分隔，例如，業務名:表名:id:，如 live:rank:user:weekly:1:202003。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Key的長度小於30個字符，Key名字本身是String對象，Redis硬編碼限制最大長度512MB。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Redis緩存場景，推薦Key都設置TTL值，保證不使用的Key能被及時清理或淘汰。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Key設計時禁止包含特殊字符，如空格、換行、單雙引號以及其他轉義字符。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3、Value設計規範"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"單個Value大小必須控制10KB以內，單實例鍵個數過大，可能導致過期鍵的回收不及時。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"set、hash、list等複雜數據類型，要儘量降低數據結構中的元素個數，建議個數不要超過1000。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4、關注命令時間複雜度"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"推薦使用O(1)命令，如get scard等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"O(N)命令關注N的數量，如下命令需要對N值在業務層面做控制。"}]},{"type":"blockquote","content":[{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"hgetall"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"lrange"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"smembers"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"zrange"}]}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如：smember命令時間複雜度爲O(n)，當n持續增加時，會導致 Redis CPU 持續飆高，阻塞其他命令的執行；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"5、Pipeline使用"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"說明："}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Pipeline是Redis批量提交的一種方式,也就是把多個命令操作建立一次連接發給Redis去執行,會比循環的單次提交性能更優。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis客戶端執行一條命令分4個過程：發送命令 -> 命令排隊 ->命令執行 -> 返回結果。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/89/89440d5bee9a77a5baa84ff0b696f998.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"常用的mget、mset命令，有效節約RTT（命令執行往返時間），但hgetall並沒有mhgetall，是不支持批量操作的。此時，需要使用Pipeline命令"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f9/f95b118000ffc42a34ef265995f74a22.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"boxShadow"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如：直播中臺項目中，需要同時查詢主播日、周、月排行榜，使用PIPELINE一次提交多個命令，同時返回三個榜單數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"6、線上禁用命令"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"禁止使用Monitor"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"禁止生產環境使用monitor命令，monitor命令在高併發條件下，會存在內存暴增和影響Redis性能的隱患"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"禁止使用Keys"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"keys操作是遍歷所有的key，如果key非常多的情況下導致慢查詢，會阻塞其他命令。所以禁止使用keys及keys pattern命令。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"建議線上使用scan命令代替keys命令。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"禁止使用Flushall、Flushdb"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"刪除Redis中所有數據庫中的所有記錄，並且該命令是原子性的，不會終止執行，一旦執行，將不會執行失敗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"禁止使用Save"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"阻塞當前redis服務器，直到持久化操作完成爲止，對於內存較大的實例會造成長時間的阻塞。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"BGREWRITEAOF"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"手動AOF，手動持久化對於內存較大的實例會造成長時間的阻塞。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Config"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Config是客戶端配置方式，不利於Redis運維。建議在Redis配置文件中設置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"六、Redis 監控"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1、慢查詢"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"方法一："},{"type":"text","text":"slowlog獲取慢查詢日誌"}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"127.0.0.1:{port}> slowlog get 5"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1) 1) (integer) 47"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"   2) (integer) 1533810300"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"   3) (integer) 175833"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"   4) 1) \"DEL\""}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"      2) \"spring:session:expirations:1533810300000\""}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2) 1) (integer) 46"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"   2) (integer) 1533810300"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"   3) (integer) 117400"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"   4) 1) \"SMEMBERS\""}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"方法二："},{"type":"text","text":"更全面的慢查詢可以通過CacheCloud工具監控。"}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"路徑：\"應用列表\"-點擊相關應用名-點擊\"慢查詢\"Tab頁。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"點擊\"慢查詢\"，重點關注慢查詢個數及相關命令。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2、監控Redis實例綁定的CPU核心使用率"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於Redis是單線程，重點監控Redis實例綁定的CPU核心使用率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一般CPU資源使用率爲10%左右，如果使用率高於20%時，考慮是否使用了RDB持久化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3、Redis分片負載均衡"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當前redis-cluster架構模式，3個master和3個Slave組成的集羣，關注 Redis-cluster每個分片requests流量均衡情況；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過命令獲取："},{"type":"text","marks":[{"type":"bgcolor","attrs":{"color":"#FDED8A","name":"yellow"}}],"text":"redis-cli -p{port} -h{host} --stat"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一般情況，超過12W需要告警。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"4、關注大key BigKey"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過Redis提供的工具，redis-cli定時掃描相應Redis大Key，進行優化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"具體命令如下："},{"type":"text","marks":[{"type":"bgcolor","attrs":{"color":"#FDED8A","name":"yellow"}}],"text":"redis-cli -h 127.0.0.1 -p {port} --bigkeys"},{"type":"text","text":"或 "},{"type":"text","marks":[{"type":"bgcolor","attrs":{"color":"#FDED8A","name":"yellow"}}],"text":"redis-memory-for-key -s {IP} -p {port} XXX_KEY"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一般超過10K爲大key，需要重點關注，建議從業務層面優化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"5、監控Redis佔用內存大小"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Info memory 命令查看，避免在高併發場景下，由於分配的MaxMemory被耗盡，帶來的性能問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重點關注 used_memory_human 配置項對應的value值，增量過高時，需要重點評估。"}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"七、總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"結合具體業務特性，合理評估Redis所需內存容量、選擇數據類型、設置單key大小，才能更好地服務於業務，爲業務提供高性能的保障。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule"},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：Jessica Chen"}]}]}]}