騎士卡：基於Kafka搭建消息中心，上億消息推送輕鬆完成

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"全球購騎士卡是國內領先的會員制特權電商平臺，匯聚國內外“喫喝玩樂買”超 300 項會員專屬優惠特權。全球購騎士卡基於移動互聯生活方式，打通線上、線下消費場景，彙集時下熱門、高頻的商品及服務優惠。會員可享全國超萬家大型商超購物8折起、全國加油7折起、熱門電商平臺專屬4折起、大牌美食餐飲 5 折起等，滿足用戶喫、喝、玩、樂、買各場景的消費需求。截至2020年，全球購騎士卡已累計服務用戶超 5000 萬名。2020年4月，全球購騎士卡完成 A 輪數千萬美元融資；同年 5 月，全球購騎士卡完成數千萬美元 A+輪融資。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"新的需求"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"全球購騎士特權業務的飛速發展，當前每天平均發送的短信量達到了約 200 萬+，需要 PUSH 的推送量達到了約 1 億+，通過微信推送量達到了5000 萬+。因此，如何構造建設一個"},{"type":"text","marks":[{"type":"strong"}],"text":"高性能、高穩定性、可擴展的消息中心"},{"type":"text","text":"迫在眉睫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"消息中心技術選型主要參考以下因素："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"削峯填谷能力"},{"type":"text","text":"：消息中心需要處理各條業務線的通知和營銷任務的信息，而這些信息根據轉化的需要，很大可能會集中化地在短期內進行推送，所以需要系統有削峯填谷的能力。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"接口通用能力"},{"type":"text","text":"：消息中心的接入方不希望被綁定在某個接口上，不需要對該接口進行維護可以供多個業務方進行發送處理。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"靈活類型劃分"},{"type":"text","text":"：消息中心需要支持靈活的業務分類配置， 因爲我們消息中心這裏的業務配置非常多，大類就有短信、PUSH、微信推送，短信裏又分通知、驗證碼和營銷類別，而 PUSH 又區分 APNS、渠道服務商等第三方通道，以及 Android 廠商通道。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"穩定處理能力"},{"type":"text","text":"：所依賴的技術產品運行穩定，因爲處於消息中心的通道位置，不能忍受產品本身的穩定性波動帶來的業務損失。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"集羣擴展能力"},{"type":"text","text":"：所依賴的技術產品沒有擴容瓶頸，對於我們的業務繼續發展有擴展的足夠空間，可以快速進行業務擴容訴求。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"新的解法"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/9d\/9d37a261a2358c64f57c3a6c6f8f60f1.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用消息中間件來做消息中心的通道是顯現而見的目標選項，綜合對比多種消息的產品，由於騎士卡並沒有需要順序消息、事務消息等高階功能，而是重點關注以下這些功能點："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"隊列的擴展能力"},{"type":"text","text":"：在這方面，RabbitMQ 的單 Queue 的處理能力不容易擴展；而 RocketMQ 的 Topic 是有 ConsumerQueue 的參數來進行配置擴容的，在 Broker 的配置文件裏指定，但是對 Broker 層面生效的；而 Kafka 的 Partition 可以每個 Topic 擁有不同的取值。這樣在分類靈活性方面，Kafka 是最優的選擇，RocketMQ 次之。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"通用的接入方式"},{"type":"text","text":"：本質上 RabbitMQ、RocketMQ、Kafka 都是私有協議的方式接入，比較雲上商業版本的接入方式，對於 Kafka 支持最純粹友好，可以使用官方的接入方式進行接入。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"消息的吞吐能力"},{"type":"text","text":"：在各類消息的對比測試中， 因爲 Kafka 本身的處理機制原因，都是由客戶端進行拉消息，整個 Broker 的處理方式比別的消息中間件要簡潔，而 Kafka 的讀寫能力\/吞吐量都是最大的。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"集羣穩定性能力"},{"type":"text","text":"：雲上的消息產品都很友好地保持業務的連續性來進行升配操作，並且對於商業版本的 Kafka 做了 Broker 上的優化，存儲上的優化，運維上的優化後，不需要擔心自建集羣出現的不穩定問題，完全滿足騎士卡的需求。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"業務價值"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用 Kafka 構建消息中心，對騎士卡來說最重要的是"},{"type":"text","marks":[{"type":"strong"}],"text":"保障了業務的穩健"},{"type":"text","text":"。利用 Kafka 的吞吐能力，自定義的 partition 設定（擴展），通過彈性擴展消費者實例的方式，自消息中心上線以來，一直運行平穩，沒有出現過影響業務的故障。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時，"},{"type":"text","marks":[{"type":"strong"}],"text":"系統運維起來十分簡單"},{"type":"text","text":"。利用雲上的Kafka能力，避免了測試期自建集羣莫名其妙的 Broker 故障，不需要投入額外的資源來保障消息中間件正常工作。並且可以通過白屏化的升級操作來匹配騎士卡的業務發展，也可以按需要來快速調整實例數。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"值得一提的是，使用雲產品 Kafka，無論在生產環境還是本地開發測試環境，都可以直接使用雲產品，最大限度減少通用產品依賴，讓團隊專注於業務的開拓實現，"},{"type":"text","marks":[{"type":"strong"}],"text":"極大的提升了團隊工作效率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“在全球購騎士卡消息中心的搭建過程中，我們使用阿里雲的Kafka完成了消息中心高吞吐量，穩定以及可擴展的目標。目前，消息中心作爲業務運營推廣的基石，發揮着重要作用，對於新業務的接入，通過消息隊列的配置修改即可完成，對現有業務可以做到無侵入，儘可能的減少了故障發生的可能。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"——騎士卡CTO"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：阿里巴巴中間件（ID：Aliware_2018）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/yU6Fpy_4DYTOiPf7gg4pBg","title":"xxx","type":null},"content":[{"type":"text","text":"騎士卡：基於Kafka搭建消息中心，上億消息推送輕鬆完成"}]}]}]}