【得物技术】得物社区实践

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2021年是我司着重关注生产稳定性的一年，得物社区服务早期是由PHP语言构建的单体应用支撑着日活百万用户，随着高速的发展在性能跟业务上已逐渐不能满足未来的需求与规划，在第一阶段上社区与架构团队同学提供了php + yaf、Java + spring cloud、Go + grpc + K8S的技术选型方案，考虑到服务性能与迁移成本，最终选择了 Go + grpc + K8S 作为此项工程的首选为社区微服务构建建立起了里程碑。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"随着业务的发展，对稳定性要求越来越高，为增强业务服务的自治能力，提高集群的稳定性与可控性，且考虑最低成本的接入方式，同时考虑社区与交易系统(Dubbo技术栈)有着千丝万缕的关系，最终希望能完成两个集群系统的轻松融合，故选用应用层框架Dubbo-go来实Golang服务的注册与发现。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Golang微服务架构，大家可能比较熟悉的是","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/micro/micro","title":null,"type":null},"content":[{"type":"text","text":"Go Micro","attrs":{}}]},{"type":"text","text":"和","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/go-kit/kit","title":null,"type":null},"content":[{"type":"text","text":"Go Kit","attrs":{}}]},{"type":"text","text":"（还有","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/NYTimes/gizmo","title":null,"type":null},"content":[{"type":"text","text":"Gizmo","attrs":{}}]},{"type":"text","text":"），确实，Go Micro的社区活跃度，Go Kit的GitHub Star数也18k以上，但这里并没选择，主要是Go Micro提供了许多的功能，开箱即用，但灵活性受限；go Kit虽被追捧，但是我们并非是重新起Golang服务，应用层框架限制过于严格，代码迁移成本将会非常高。考虑以上困难，最终选了一个还在成长期的Dubbo-go。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Dubbo-go介绍","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo-go是目前Dubbo多语言生态最火热的项目之一，已随着Dubbo生态加入Apache基金会，截止目前已有不少一二线互联网公司使用(钉钉、携程、涂鸦、开课吧等)，社区活跃度较高，响应开发者需求较快，有较快且贴合开发着需求的版本迭代速度。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/74/74b7c44d9ae5702f94cfcc87a795b5bb.png","alt":null,"title":"图1","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo-go主项目，主要是基于Dubbo的分层代码设计，上图是Dubbo-go的代码分层，基本上与 Java版本Dubbo现有的分层一致，所以Dubbo-go也继承了Dubbo的一些优良特性，比如整洁的代码架构、易于扩展、完善的服务治理功能。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前Dubbo-go已经实现了Dubbo的常用功能（如负责均衡、集群策略、服务多版本多实现、服务多注册中心多协议发布、泛化调用、服务降级熔断等），其中服务注册发现已经支持zookeeper/etcd/consul/nacos 主流注册中心。这里不展开详细介绍。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"方案实现","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根据现有项目以gRPC为服务调用的背景前提下，考虑到对业务代码侵入程度，且做到兼容原有方案正常使用，两套gRPC实现下可切换自由，做到生产环境切换Rpc治理框架的实时性与可控性，降低生产环境风险，故结合Dubbo-go自身支持gRPC协议入手满足以上需求。注册中心选型为Nacos，与目前现有中间件保持统一，同时满足配置部分配置项需求。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/77/771ef4138d31b8bfef5acaf2f30c62d9.png","alt":null,"title":"图2","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这里我们要先思考两个问题，一个是Dubbo-go的集成如何兼容原有gRPC方案，保持两套方案可同时在线支持生产，第二个问题是两套gRPC之间如何实现实时切换。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"兼容性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"在实现此项需求前，我们先来谈谈gRPC自身特性，gRPC是谷歌开源的一个 RPC 框架，面向移动和HTTP/2设计，内容交换格式采用ProtoBuf(Google Protocol Buffers)，开源已久，提供了一种灵活、高效、自动序列化结构数据的机制，作用与XML，Json类似，但使用二进制，（反）序列化速度快，压缩效率高，传输协议采用http2，性能比http1.1提升很多。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"在根据","attrs":{}},{"type":"text","text":"介绍的 gRPC 的相关特性可以看出来，gRPC已经解决了 codec 和 transport 两层的问题，结合图一看，从cluster层往上，是没有gRPC相关涉及的地方，从","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"图1","attrs":{}},{"type":"text","text":"里面可以看出要做gRPC相关适配，在 protocol 这一层是最合适的，我们可以如同 DubboProtocol 一样，扩展出来一个gRPCProtocol ，这个 gRPC protocol 大体上相当于一个Adapter，将底层的gRPC的实现和我们自身的Dubbo-go结合在一起。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/84/8418b6cfa51437c600e5edda557c2910.png","alt":null,"title":"图三","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基于上述，Dubbo-go帮助我们解决了gRPC的相关整合，相当于在gRPC基础之上包装了Dubbo-go治理层，而我们从gRPC的","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"ProtoBuf修改作为切入点开始，Dubbo-go官方基于Google ","attrs":{}},{"type":"text","text":"protobuf","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":" ","attrs":{}},{"type":"text","text":"扩展插件定义了Dubbo-go gRPC所使用的 protobuf 自定义逻辑代码，","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"完成兼容性问题即可。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// HelloWorldServiceClientImpl is the client API for HelloWorldService service.\ntype HelloWorldServiceClientImpl struct {\n\tSayHello func(ctx context.Context, in *SayHelloReq, out *SayHelloResp) error\n\t//...\n}\n\n// service Reference\nfunc (c *HelloWorldServiceClientImpl) Reference() string {\n\treturn \"helloWorldServiceImpl\"\n}\n\n// GetDubboStub\nfunc (c *HelloWorldServiceClientImpl) GetDubboStub(cc *grpc.ClientConn) HelloWorldServiceClient {\n\treturn NewHelloWorldServiceClient(cc)\n}\n\n// Server interface\ntype HelloWorldServiceProviderBase struct {\n\tproxyImpl protocol.Invoker\n}\n\n// set invoker proxy\nfunc (s *HelloWorldServiceProviderBase) SetProxyImpl(impl protocol.Invoker) {\n\ts.proxyImpl = impl\n}\n\n// get invoker proxy\nfunc (s *HelloWorldServiceProviderBase) GetProxyImpl() protocol.Invoker {\n\treturn s.proxyImpl\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"实时切换开关","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"实时切换，起初是为了在压测环境方便两套不同实现的gRPC方案可实时切换做压测数据收集，后期是抱着敬畏生产的态度，在生产环境刚接入Dubbo-go时将Rpc框架切换支持服务、方法自由切换，从稳定性出发，选择性的切换观测服务稳定性状态。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此项需求的接入，同样是从gRPC的","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"ProtoBuf修改作为切入点开始，同时基于Nacos配置中心实现，我们将原有gRPC的客户端调用和Dubbo-go的客户端调用封装成一个统一实例化入口（这里简称ClinetDubbo），客户端所有方法新增一份继承 ClinetDubbo 具体实现（由protobuf扩展插件统一脚本生成），实现内容大致为获取ClinetDubbo中的两套gRPC客户端，此时通过Nacos配置获取配置中心所打开的客户端是哪一套，根据判断实现走具体的gRPC链路。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f5/f5e13e26d4fb9e95aa8c96a1182ad72d.png","alt":null,"title":"图四","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"左侧gRPC常规链路，右侧Dubbo-go治理模型链路。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// ClientDubbo\ntype HelloWorldServiceClientDubbo struct {\n\tGrpcClient HelloWorldServiceClient\n\tDubboClient *HelloWorldServiceClientImpl\n\tOpen bool\n\tCaller string\n}\n\n// 具体的方法实现\nfunc (c *HelloWorldServiceClientDubbo) SayHello(ctx context.Context, req *SayHelloReq, opts ...grpc.CallOption) (*SayHelloResp, error) {\n\tserverName := c.DubboClient.Reference()\n //获取 nacos配置源数据\n\tserverCfg := nacosCfg.GetServerCfg()\n\tif !c.Open {\n\t\tc.Open = serverCfg.AllOpen\n\t}\n\tcfg := serverCfg.ServiceCfg\n\n // 判断调用链路\n\tif !c.Open &&\n\t\t!cfg[serverName].Open &&\n\t\t(cfg[serverName].Consumers == nil || !cfg[serverName].Consumers[c.Caller]) &&\n\t\t!cfg[serverName].Method[\"SayHello\"].Open &&\n\t\t(cfg[serverName].Method[\"SayHello\"].Consumer == nil || !cfg[serverName].Method[\"SayHello\"].Consumer[c.Caller]) {\n\t\t\n // 原gRPC链路\n return c.GrpcClient.SayHello(ctx, req, opts...)\n\t}\n\n\t// Dubbo-go治理链路\n\tout := new(SayHelloResp)\n\terr := c.DubboClient.SayHello(ctx, req, out)\n\treturn out, err\n}\n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"项目集成","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以下是基于现有项目结构集成Dubbo-go框架示例：","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"provider","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"type HelloWorldService struct {\n\t*pb.UnimplementedHelloWorldServiceServer\n\t*pb.HelloWorldServiceClientImpl\n\t*pb.HelloWorldServiceProviderBase\n}\n\nfunc NewHelloWorldService() *HelloWorldService {\n\treturn &HelloWorldService{\n\t\tHelloWorldServiceProviderBase: &pb.HelloWorldServiceProviderBase{},\n\t}\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基于原有服务提供的基础之上加入Dubbo-go扩展部分，提供服务注册。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"consumer","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"//原有Grpc\nvar HelloWorldCli HelloWorldServiceClient \n//Dubbo-go\nvar HelloWorldProvider = &HelloWorldServiceClientImpl{} \n\nfunc GetHelloWorldCli() HelloWorldServiceClient {\n\tif HelloWorldCli == nil {\n\t\tHelloWorldCli = NewHelloWorldClient(grpc_client.GetGrpcClient(...))\n\t}\n\treturn &HelloWorldServiceClientDubbo{\n\t\tGrpcClient: HelloWorldCli,\n\t\tDubboClient: HelloWorldProvider,\n\t\tCaller: dubboCfg.Caller,\n\t\tOpen: false,\n\t}\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"GetHelloWorldCli()","attrs":{}}],"attrs":{}},{"type":"text","text":"简单封装了客户端调用，此方法最终返回 ","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"HelloWorldServiceClientDubbo","attrs":{}}],"attrs":{}},{"type":"text","text":" 结构体的方法，客户端发起调用进入以 ","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"HelloWorldServiceClientDubbo","attrs":{}}],"attrs":{}},{"type":"text","text":" 实现的具体方法中，根据配置项判断执行具体gRPC调用链路。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Main","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"func main() {\n\t//provider\n\tconfig.SetProviderService(rpc.NewHelloWorldService(), ...)\n \n // 设置服务消费者\n\tconfig.SetConsumerService(api..., ....)\n \n // 加载dubbo\n\tconfig.Load()\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以上就是社区服务集成Dubbo-go的整体思路与方案，我们会发现在现有项目中需要改动的代码量很少，且对业务方方代码无任何侵入。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"小结","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo-go集成，增强了业务服务gRPC调用过程中治理能力，基于cluster增加了服务集群的容错能力，实现了应用服务之间容错能力的可配置性；完善且统一了社区服务原始新老架构服务的全链路监控和服务指标监控告警；增强了业务伙伴对集群内服务的透明化、可控性，在遇到问题后整体的链路梳理上有了更多的可参考信息。基于Dubbo整体生态，可轻松支持Golang与Java的Rpc互通，做到跨语言Rpc调用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1a/1afbca1b104655606416a67da2dabe75.png","alt":null,"title":"图五","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"最后","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo-go作为一个微服务框架，自身包含治理能力，这部分能力如何与K8S融洽结合。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"K8S 提供了pod/endpoint/service 三层维度的资源，可以通过监听pod/endpoint/service三层维度资源的事件，作出合理的处理以达到服务治理的目的，不需要引入额外组件，通过监听 k8s 中最细粒度资源 pod 的事件，通过k8s apiserver获取pod列表，只是通过apiserver使用etcd的服务注册和服务通知能力，其他继续使用Dubbo-go的服务治理能力，模型简单，不需要实现额外的模块，几乎不需要对Dubbo作出改动。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文｜小柯","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"关注得物技术，携手走向技术的云端","attrs":{}}]}]}