如何有效縮短閒魚消息處理時長

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着用戶數的快速增長，閒魚的IM也迎來了前所未有的挑戰。多年的業務迭代，端側IM的代碼已經因爲多年的迭代層次結構不足夠清晰，之前消息一些隱藏起來的數據同步問題，也隨着用戶數的增大而被放大。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏面的具體流程在於，後臺需要同步到用戶端側的數據包，後臺會根據數據包的業務類型劃分成不同的數據域，數據包在對應域裏面存在唯一且連續的編號，每一個數據包發送到端側並且被成功消費後，端側會記錄當前每一個數據域已經同步過的版本編號，下一次數據同步就以本地數據域的編號開始，不斷的同步到客戶端。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然用戶不會一直在線等待消息，所以之前這裏端側採用了推拉結合的方式保證數據的同步。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在線時使用ACCS實時的將最新的數據內容推送到客戶端。ACCS是淘寶無線向開發者提供全雙工、低延時、高安全的通道服務。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"離線啓動後，根據本地的數據域編號，拉取不在線時候的數據差。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當數據獲取出現黑洞（數據包Version不連續的狀態）時，觸發數據同步拉取。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"分析"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現存的一個同步策略是可以基本保障IM的數據同步的，但是也伴隨着一些隱含的問題："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"短時間密集數據推送時，會快速的觸發多次數據域同步。域同步回來的數據如果存在問題，又會觸發新一輪的同步，造成網絡資源的浪費。冗餘數據包\/無效的數據內容會佔用有效內容的處理資源，又對CPU和內存資源造成浪費。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"數據域中的數據包客戶端是否正常消費，服務端側無感知，只能被動地根據當前數據域信息返回數據。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"數據收取\/消息數據體解析\/存儲落庫邏輯拆分不夠清晰，無法針對性的對某一層的代碼拆分替換進行ABTest。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對遇見的這些問題，對閒魚IM進行分層改造，抽離數據同步層。除了希望以後這個數據的同步內容可以用在IM之外，也希望隨着穩定性的增加，賦能其他的業務場景。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文重點來看下端側閒魚IM數據同步的一些解決思路。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"數據同步優化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"拆分&分層"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於服務端來說，業務側產出數據包後，會拼接上當前的數據域信息，然後通過數據同步層將數據推送到端側。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}