骑士卡：基于Kafka搭建消息中心，上亿消息推送轻松完成

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"全球购骑士卡是国内领先的会员制特权电商平台，汇聚国内外“吃喝玩乐买”超 300 项会员专属优惠特权。全球购骑士卡基于移动互联生活方式，打通线上、线下消费场景，汇集时下热门、高频的商品及服务优惠。会员可享全国超万家大型商超购物8折起、全国加油7折起、热门电商平台专属4折起、大牌美食餐饮 5 折起等，满足用户吃、喝、玩、乐、买各场景的消费需求。截至2020年，全球购骑士卡已累计服务用户超 5000 万名。2020年4月，全球购骑士卡完成 A 轮数千万美元融资；同年 5 月，全球购骑士卡完成数千万美元 A+轮融资。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"新的需求"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"全球购骑士特权业务的飞速发展，当前每天平均发送的短信量达到了约 200 万+，需要 PUSH 的推送量达到了约 1 亿+，通过微信推送量达到了5000 万+。因此，如何构造建设一个"},{"type":"text","marks":[{"type":"strong"}],"text":"高性能、高稳定性、可扩展的消息中心"},{"type":"text","text":"迫在眉睫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"消息中心技术选型主要参考以下因素："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"削峰填谷能力"},{"type":"text","text":"：消息中心需要处理各条业务线的通知和营销任务的信息，而这些信息根据转化的需要，很大可能会集中化地在短期内进行推送，所以需要系统有削峰填谷的能力。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"接口通用能力"},{"type":"text","text":"：消息中心的接入方不希望被绑定在某个接口上，不需要对该接口进行维护可以供多个业务方进行发送处理。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"灵活类型划分"},{"type":"text","text":"：消息中心需要支持灵活的业务分类配置， 因为我们消息中心这里的业务配置非常多，大类就有短信、PUSH、微信推送，短信里又分通知、验证码和营销类别，而 PUSH 又区分 APNS、渠道服务商等第三方通道，以及 Android 厂商通道。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"稳定处理能力"},{"type":"text","text":"：所依赖的技术产品运行稳定，因为处于消息中心的通道位置，不能忍受产品本身的稳定性波动带来的业务损失。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"集群扩展能力"},{"type":"text","text":"：所依赖的技术产品没有扩容瓶颈，对于我们的业务继续发展有扩展的足够空间，可以快速进行业务扩容诉求。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"新的解法"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/9d\/9d37a261a2358c64f57c3a6c6f8f60f1.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用消息中间件来做消息中心的通道是显现而见的目标选项，综合对比多种消息的产品，由于骑士卡并没有需要顺序消息、事务消息等高阶功能，而是重点关注以下这些功能点："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"队列的扩展能力"},{"type":"text","text":"：在这方面，RabbitMQ 的单 Queue 的处理能力不容易扩展；而 RocketMQ 的 Topic 是有 ConsumerQueue 的参数来进行配置扩容的，在 Broker 的配置文件里指定，但是对 Broker 层面生效的；而 Kafka 的 Partition 可以每个 Topic 拥有不同的取值。这样在分类灵活性方面，Kafka 是最优的选择，RocketMQ 次之。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"通用的接入方式"},{"type":"text","text":"：本质上 RabbitMQ、RocketMQ、Kafka 都是私有协议的方式接入，比较云上商业版本的接入方式，对于 Kafka 支持最纯粹友好，可以使用官方的接入方式进行接入。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"消息的吞吐能力"},{"type":"text","text":"：在各类消息的对比测试中， 因为 Kafka 本身的处理机制原因，都是由客户端进行拉消息，整个 Broker 的处理方式比别的消息中间件要简洁，而 Kafka 的读写能力\/吞吐量都是最大的。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"集群稳定性能力"},{"type":"text","text":"：云上的消息产品都很友好地保持业务的连续性来进行升配操作，并且对于商业版本的 Kafka 做了 Broker 上的优化，存储上的优化，运维上的优化后，不需要担心自建集群出现的不稳定问题，完全满足骑士卡的需求。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"业务价值"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用 Kafka 构建消息中心，对骑士卡来说最重要的是"},{"type":"text","marks":[{"type":"strong"}],"text":"保障了业务的稳健"},{"type":"text","text":"。利用 Kafka 的吞吐能力，自定义的 partition 设定（扩展），通过弹性扩展消费者实例的方式，自消息中心上线以来，一直运行平稳，没有出现过影响业务的故障。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同时，"},{"type":"text","marks":[{"type":"strong"}],"text":"系统运维起来十分简单"},{"type":"text","text":"。利用云上的Kafka能力，避免了测试期自建集群莫名其妙的 Broker 故障，不需要投入额外的资源来保障消息中间件正常工作。并且可以通过白屏化的升级操作来匹配骑士卡的业务发展，也可以按需要来快速调整实例数。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"值得一提的是，使用云产品 Kafka，无论在生产环境还是本地开发测试环境，都可以直接使用云产品，最大限度减少通用产品依赖，让团队专注于业务的开拓实现，"},{"type":"text","marks":[{"type":"strong"}],"text":"极大的提升了团队工作效率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“在全球购骑士卡消息中心的搭建过程中，我们使用阿里云的Kafka完成了消息中心高吞吐量，稳定以及可扩展的目标。目前，消息中心作为业务运营推广的基石，发挥着重要作用，对于新业务的接入，通过消息队列的配置修改即可完成，对现有业务可以做到无侵入，尽可能的减少了故障发生的可能。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"——骑士卡CTO"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文转载自：阿里巴巴中间件（ID：Aliware_2018）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文链接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/yU6Fpy_4DYTOiPf7gg4pBg","title":"xxx","type":null},"content":[{"type":"text","text":"骑士卡：基于Kafka搭建消息中心，上亿消息推送轻松完成"}]}]}]}