【融雲技術】超大規模併發下自定義屬性的設置與分發

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一. ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0099FF","name":"user"}},{"type":"strong","attrs":{}}],"text":"自定義屬性介紹和應用場景","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在移動互聯網高速發展的新時代，IM 即時通訊得到了快速的發展。IM 中除了傳統的圖文聊天，在線秀場、在線教育、直播帶貨、遊戲互動等場景應用也越來越廣泛。特別是一些一線主播的直播中，直播間動輒幾萬甚至幾十萬上百萬人同時觀看，這對 IM 的通訊併發能力提出了更高的要求。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"融雲作爲 IM 即時通訊和音視頻通訊雲服務的代表，提供了直播聊天室 SDK 和音視頻直播 SDK。而在提供這些基礎功能的時候，業務層可能也需要設置一些自己的屬性，比如在語音直播聊天室場景的會場屬性同步，主播麥位信息、角色管理等等，亦或者狼人殺等卡牌類遊戲場景中記錄用戶的角色和牌局狀態等。這些自定義屬性都可以豐富整個 IM 的業務形態。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本篇文章中我重點介紹以聊天室爲背景的超大併發規模下，是如何實現這些自定義屬性的設置以及分發的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0099FF","name":"user"}},{"type":"strong","attrs":{}}],"text":"二. 海量用戶下面臨的技術挑戰","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"聊天室和普通的 IM 羣相比最大的不同點在於它是一個虛擬的組織，聊天室是無界的，所有的人都可以隨意的進出聊天室；而羣更像一個房間，它是一個有界的、有上限的私密組織。而隨着直播行業的蓬勃發展和觀看直播人數的與日劇增，直播房間的聊天室更容易出現那種超高併發場景，例如一線主播直播間的聊天室。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自定義屬性在進行了設置後，需要及時的將這些屬性同步到聊天室中的各個端上。在這個過程中最大的挑戰是在分發上。因爲如果聊天室人數巨多，而且很多場景自定義屬性是具有時效性的，所以及時的將自定義屬性同步到各端就很考驗服務的高併發場景下的分發能力了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#0099FF","name":"user"}},{"type":"strong","attrs":{}}],"text":"三. 高併發下自定義屬性設置與分發實現","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在分佈式系統架構中，爲了統一聊天室的行爲，我們會利用一致性 hash 將同一個聊天室的所有信令匯聚到一個實例上。這樣做是有一定好處的，比如能提升服務的緩存命中率，像聊天室人員進出黑/白名單設置和判斷等都可以利用內存中的緩存，不用每次都訪問第三方緩存，從而提高了聊天室的響應速度。但是這樣帶來的問題就是，如果聊天室人數過大，聊天室的分發能力會是系統的瓶頸，而且一旦單臺主機下發能力達到上限，是無法通過擴容來解決問題的，因爲不管怎麼擴容，一個聊天室的所有信令還是會匯聚到這個實例中。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了提升服務的分發能力，我們獨立出一套單獨的分發服務（聊天室消息服務），具體如下圖所示：","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/19/198c05aca11e20f2c2ac42bc58278027.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":"在這套架構系統中，我們將聊天室系統分爲聊天室服務以及聊天室消息服務，聊天室服務的作用依然是接收聊天室相關的所有上行信令，比如人員進出，自定義屬性設置等等。而聊天室消息服務更專注聊天室消息以及自定義屬性等的分發。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#333333","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用戶在加入聊天室後，需要根據用戶自身的 ID 進行一致性 hash 來算出需要落在哪個聊天室消息服務中，並進行加入。這樣一個聊天室的用戶就會被平均分散到了聊天室消息服務中。這樣做也解決了系統遇到壓力後不能擴容的問題。在自定義屬性同步到聊天室消息服務上後，分發節點需要將屬性放置到內存中，然後給當前節點的所屬用戶發送一個通知拉取的信令，客戶端在收到通知拉取信令後進行自定義屬性的拉取。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#0099FF","name":"user"}},{"type":"strong","attrs":{}}],"text":"四. 自定義屬性的內存存儲和客戶端快速拉取","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在聊天室消息服務接收到自定義屬性變更請求時，會將這份數據保存到全量屬性集合裏，這個集合裏存的是所有的自定義屬性集合，如果這份數據被 LRU 淘汰掉或者服務重啓的話，會從第三方緩存中重新構建一份數據出來。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"內存中的全量數據，比較適合於剛加入聊天室的人，他們加入聊天室後直接拉取這些全量數據即可。但是如果已經在聊天室了也已經拉取過全量屬性的話，這時聊天室再設置了新的自定義屬性或者刪除了某一個自定義屬性的話，如果客戶端想知道剛纔的自定義屬性的行爲，就需要對客戶端的全量自定義屬性與服務器端自定義屬性進行比對。不管比對行爲是放在服務器端還是放在客戶端都會增加一定的計算壓力。爲了解決增量數據的同步，我們構建了一份屬性變更記錄的集合，具體如圖所示： ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/cb/cb9fc6655e31d37576a9a73d9b879692.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"屬性變更記錄採用的是一個有序的 map 集合 .key 爲變更時間戳，value 裏存變更的類型以及自定義屬性內容，變更的類型主要爲設置和刪除。這個有序的 map 提供了這段時間內所有的自定義屬性的動作，客戶端在收到自定義屬性變更拉取的通知後，帶着自己本地最大自定義屬性的時間戳上來拉取，例如圖中場景，客戶端傳的如果是時間戳  6，則會拉取到時間戳爲 7 和時間戳爲 8 的兩條記錄。客戶端拉取到增量的內容後在本地進行回放，然後對自己本地的內容進行相關的操作即可。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#0099FF","name":"user"}},{"type":"strong","attrs":{}}],"text":"五. 結束語","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文章大體解析了融雲聊天室的基礎架構以及在此架構下，如何做到了超大規模併發下的自定義屬性的設置和分發。而隨着互聯網的高速發展，自定義屬性的設置應用會越來越廣泛，而併發規模或許會繼續提升，屆時我們的架構也會迭代，繼續爲客戶提供低延時高可靠的服務。","attrs":{}}]}]}