RocketMQ 在金融企业技术中台的落地实践

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"线上各位的朋友们大家好，很荣幸能和大家交流RocketMQ使用的相关经验。我今天和大家分享的主题是《RocketMQ 在金融企业技术中台的落地实践》。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先做下简单的自我介绍，我叫陈培新，来着国信证券。目前参与国信证券技术中台建设相关工作，专注的技术有微服务、Serverless。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本次分享的内容包含如下两个部分。首先介绍国信证券技术中台的背景、发展历史、架构等情况，然后再介绍RocketMQ在技术中台的各种落地场景情况。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们先看下技术中台的简介。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先是技术中台的背景，在当前数字化转型的浪潮中，券商行业面临有三个挑战：1是长久以来，券商信息系统建设模式以厂商为主，导致大量烟囱式的异构系统，数据集成困难，系统运维压力大；2是行业竞争和业务复杂化，对信息技术提出更高的要求，存在业务需求响应不及时，跨部门协作成本高的问题。3是缺乏自主掌控，创新后劲不足。那么为了应对上述挑战，国信证券提出建设全栈式敏捷研发技术中台作为数字化转型重点，通过架构支撑、研发赋能，实现统一开发框架、快速交付、自主掌控的目标。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下来我们再看下国信技术中台架构进化路线图。我们可以看到从早期的总线结构、到微服务、到容器化、到DevOps、以及正在进行的Serverless。国信技术架构工作，每1-2年都会上一个台阶，应用范围也越来越广。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e1\/e1665567c2aab1b6dad2e71480a7c35f.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面是整个技术中台的技术全景图。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/38\/38a330bb53b5762b1b3e56094cf0f837.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"包括互联互通、共享服务、基础架构、持续交付等四个子域。其中互联互通解决安全高性能接入的问题，支持消息推送，其中互联网关为基于Netty自研的API网关。共享服务提供基础服务的复用能力，抽象提炼了自选股中心、用户中心、行情中心、扩展交易中心等公共基础服务，可实现业务应用的快速建设。基础架构包括自研的Zebra微服务、消息队列、分布式技术、服务网格等，其中Zebra微服务已经Gitee社区开源，欢迎大家使用和多提意见。持续交付子域主要包含Simba研发效能平台，微服务自动发布平台，实现开发运维端到端的自动化，可显著提升开发、测试、运维研发流程的一体化程度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最后看下技术中台在国信证券的应用效果。技术中台全面支撑国信证券各类应用，如手机金太阳、邮件系统、企业微信等。金太阳手机证券APP支撑了包括交易、理财、业务办理、资讯、开户、扩展行情等业务，承担了超过50%的互联网流量，其中接入的国信金太阳手机证券APP总注册用户超过1600万，月活数据324万，动态有效用户超200万，日并发峰值超过27000+TPS，日推送个性化信息量达到1840万笔，接入以来未出现任何平台级的生产事件，为国信证券信息系统的稳定、高效、安全迭代提供有力的支撑。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/52\/520cb97f3443bbb3895cdca8667bdb2a.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以上就是国信技术中台的相关介绍，那么接下来我们来看看RocketMQ在国信中台是如何落地应用的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先看下集群搭建方面。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/a5\/a500fa9c56169eeb61f4eee82fae06f7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们知道，如果是通过手工的搭建RocketMQ，或多或少存在误配置的问题，同时对于RocketMQ相关性能调优的经验也难以复用沉淀。为此在基础架构方面，我们在开发测试网段、生产网段都提供了RocketMQ的集群自动构建流程，将构建流程自动化。开发或运维可根据需要填入所需的集群信息，待流程审批通过后，平台会自动做集群构建。这样一来可大大减少集群搭建的时间，减少出错概率。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下来看RocketMQ在推送平台的应用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/b3\/b3fa31586cea096c3ed5fdcb4ae4648e.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"推送平台支撑国信网金、营业部、总部分支机构等业务部门的信息推送，和当前主流手机厂商都做了对接，如苹果、华为、小米和OV等，目前日均推送信息数量为200w。推送平台提供了多维度的统计查询功能，可方便让推送者查询其推送信息的各个状态情况。这样就要求推送平台对消息推送的状态做记录，对每个人的每条消息都要记录推送状态日志，每条消息大概有10个状态，那么总计每天日志记录量为2000w左右。同时消息推送特点是集中在交易日的开市 9点到10点左右。如果直接采用JDBC入库的话，那么会对数据库造成较大的压力，同时也可能影响到消息推送的速度。为了避免上述问题，推送平台使用RocketMQ发送推送状态日志，推送统计模块拉取RocketMQ推送日志，然后对日志做批量入库。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在这个场景中，我们使用RocketMQ来做削峰。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第二个是在API网关中的应用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/c3\/c30a6de9d9089874e4a985fa77e6291d.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"国信API网关是基于Netty自主研发的网关。它支持了多协议（HTTP、TCP、WebSocket），提供路由、协议转换、安全认证等特性，是国信Zebra微服务的总入口。我们知道，API网关有一个重要的特性是路由，简单的说就是进来一个请求，网关要根据URL等信息，将请求分发到对应微服务的某个接口。和常见的Zuul或者SpringCloud Gateway不同，Zebra后端微服务是基于gRPC协议，这里的API网关是将URL映射为微服务的gRPC接口。在API网关运行过程中，后端微服务不断地做的迭代升级，提供的接口在不停变化。为了不对客户造成影响，要求API网关要支持做到不停服务更新路由规则。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"API网关的路由规则放在自研的配置中心，当对应的API网关规则有变化时，会将变更的路由规则通过RocketMQ发送出来。API网关在收取到路由更新RocketMQ消息后，实时更新内存中的路由规则，从而达到路由规则的热更新。下面的API网关的路由规则配置样例，在方法映射中，左边是URL前缀，右边是微服务接口名。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/0e\/0ef2189ae1acdeeca1f3104bb2d95a3e.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于微服务的配置热更新也是类似，微服务配置的热更新一般用于开关、阈值、熔断限流更新等场景。配置中心和微服务框架使用另外一个RocketMQ Topic。使用广播的模式，Tag为对应的微服务名称，这样不同的微服务可根据Tag对消息做过滤，只处理本微服务的配置变更消息。在配置持久化后，配置中心将更新的配置通过RocketMQ消息发送，对应微服务接收到对应消息后，更新对应内存中的配置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上述两个场景中，我们使用RocketMQ来做消息通知。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下来再看看RocketMQ在分布式任务监控中的应用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/ba\/ba001bb7e4a5f583a8e7413f675297ad.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"国信采用ElasticJob作为分布式任务框架，同时联合运维开发了一个对任务运行业务状态做的监控组件。可以通过这个监控组件对执行出问题的的任务做告警，还可以查询任务历史执行记录。举个例子，比如任务在指定的时间触发了，但是所依赖的资源（比如数据，或者微服务挂掉了），从业务角度来看，任务执行结果为失败，此时应该做告警。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们使用了2张表作为任务的监控（一张任务状态表、一张任务历史表）。一开始我们提供的分布式任务SDK通过直接连接数据库的方式写入任务的监控信息，技术上虽然可行。但是不优雅，和监控数据库的耦合性太强，每个任务都必须配置和数据库的连接信息。如果数据库密码修改或者字段有变化，得通知到各个微服务的运维做修改，工作量庞大且容易遗漏。所以在后面我们通过RocketMQ将微服务监控和对应的监控数据库做了解耦，SDK使用RocketMQ发送任务结果，同时新增一个“分布式任务状态处理”组件来拉取监控消息然后统一入库。这样监控数据库有变更时，只需变更这个新增的组件即可，变更工作量和风险大大降低。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看到我们使用RocketMQ来做系统组件的之间的解耦。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下来我们再来看看RocketMQ最后一个落地场景-异步任务。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/7e\/7e741f6f74e569860c3bbc71252cc5f4.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在国信内部开发测试环境和生产环境隔离，安全规定不允许开放端口，也就是说两个环境的应用之间无法通过接口相互调用。我们可以看到DevOps平台部署在开发测试网段（比如Jira、Gitlab等），ITIL、容器、微服务配置中心等在生产环境部署。但是去年我们平台支撑的项目在做持续交付三级的事情，标准里面又间接地要求技术中台将上诉两个环境打通。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"举几个例子：比如“部署与发布管理”中“过程可视化”对上线流程的要求，在开发侧中Jira可以自动创建ITIL（要求开发环境的Jira可以通过生产的ITIL创建工单），ITIL里面各类流转信息要在开发侧可见（我们通过回写到Jira上线单），对于部署平台比如容器平台，数据库发布平台的日志在开发侧也要可见。第二个是配置的管理，比如配置晋级，微服务的配置要从开发、测试最后一路晋级到生产环境。也要打通到生产。最后和安全组达成一个办法，将两个网段直接的系统调用系统通过公司的安全文件通过来存放。然后两边各部署一套通道服务来定期获取文件，解析并调用对应的系统接口。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由于对接系统和接口较多，我们部署对每个系统都部署对应的任务处理器，启用任务获取任务定期获取文件，做简单解析后，然后通过RocketMQ发送任务消息，对应任务处理器接收到RocketMQ再做相应的处理。使用RocketMQ可保持任务处理器之间的独立性和可扩展性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最后再简单看下中台微服务框架对RocketMQ的支持，微服务提供了集成RocketMQ的文档说明，使用方法等，同时微服务的脚手架也支持可视化引入RocketMQ依赖。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/dd\/ddb09b55efefde586ed2c2809cff54a1.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/2a\/2ad24a518766808af03b6d412fd60b5f.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以上就是今天分析的国信技术中台落地实践RocketMQ的场景，希望能够对大家在日常开发的方案设计有所帮助，谢谢大家。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]}]}