【得物技術】得物社區實踐

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2021年是我司着重關注生產穩定性的一年，得物社區服務早期是由PHP語言構建的單體應用支撐着日活百萬用戶，隨着高速的發展在性能跟業務上已逐漸不能滿足未來的需求與規劃，在第一階段上社區與架構團隊同學提供了php + yaf、Java + spring cloud、Go + grpc + K8S的技術選型方案，考慮到服務性能與遷移成本，最終選擇了 Go + grpc + K8S 作爲此項工程的首選爲社區微服務構建建立起了里程碑。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着業務的發展，對穩定性要求越來越高，爲增強業務服務的自治能力，提高集羣的穩定性與可控性，且考慮最低成本的接入方式，同時考慮社區與交易系統(Dubbo技術棧)有着千絲萬縷的關係，最終希望能完成兩個集羣系統的輕鬆融合，故選用應用層框架Dubbo-go來實Golang服務的註冊與發現。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Golang微服務架構，大家可能比較熟悉的是","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/micro/micro","title":null,"type":null},"content":[{"type":"text","text":"Go Micro","attrs":{}}]},{"type":"text","text":"和","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/go-kit/kit","title":null,"type":null},"content":[{"type":"text","text":"Go Kit","attrs":{}}]},{"type":"text","text":"（還有","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/NYTimes/gizmo","title":null,"type":null},"content":[{"type":"text","text":"Gizmo","attrs":{}}]},{"type":"text","text":"），確實，Go Micro的社區活躍度，Go Kit的GitHub Star數也18k以上，但這裏並沒選擇，主要是Go Micro提供了許多的功能，開箱即用，但靈活性受限；go Kit雖被追捧，但是我們並非是重新起Golang服務，應用層框架限制過於嚴格，代碼遷移成本將會非常高。考慮以上困難，最終選了一個還在成長期的Dubbo-go。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Dubbo-go介紹","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo-go是目前Dubbo多語言生態最火熱的項目之一，已隨着Dubbo生態加入Apache基金會，截止目前已有不少一二線互聯網公司使用(釘釘、攜程、塗鴉、開課吧等)，社區活躍度較高，響應開發者需求較快，有較快且貼合開發着需求的版本迭代速度。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/74/74b7c44d9ae5702f94cfcc87a795b5bb.png","alt":null,"title":"圖1","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo-go主項目，主要是基於Dubbo的分層代碼設計，上圖是Dubbo-go的代碼分層，基本上與 Java版本Dubbo現有的分層一致，所以Dubbo-go也繼承了Dubbo的一些優良特性，比如整潔的代碼架構、易於擴展、完善的服務治理功能。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前Dubbo-go已經實現了Dubbo的常用功能（如負責均衡、集羣策略、服務多版本多實現、服務多註冊中心多協議發佈、泛化調用、服務降級熔斷等），其中服務註冊發現已經支持zookeeper/etcd/consul/nacos 主流注冊中心。這裏不展開詳細介紹。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"方案實現","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根據現有項目以gRPC爲服務調用的背景前提下，考慮到對業務代碼侵入程度，且做到兼容原有方案正常使用，兩套gRPC實現下可切換自由，做到生產環境切換Rpc治理框架的實時性與可控性，降低生產環境風險，故結合Dubbo-go自身支持gRPC協議入手滿足以上需求。註冊中心選型爲Nacos，與目前現有中間件保持統一，同時滿足配置部分配置項需求。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/77/771ef4138d31b8bfef5acaf2f30c62d9.png","alt":null,"title":"圖2","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏我們要先思考兩個問題，一個是Dubbo-go的集成如何兼容原有gRPC方案，保持兩套方案可同時在線支持生產，第二個問題是兩套gRPC之間如何實現實時切換。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"兼容性","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"在實現此項需求前，我們先來談談gRPC自身特性，gRPC是谷歌開源的一個 RPC 框架，面向移動和HTTP/2設計，內容交換格式採用ProtoBuf(Google Protocol Buffers)，開源已久，提供了一種靈活、高效、自動序列化結構數據的機制，作用與XML，Json類似，但使用二進制，（反）序列化速度快，壓縮效率高，傳輸協議採用http2，性能比http1.1提升很多。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"在根據","attrs":{}},{"type":"text","text":"介紹的 gRPC 的相關特性可以看出來，gRPC已經解決了 codec 和 transport 兩層的問題，結合圖一看，從cluster層往上，是沒有gRPC相關涉及的地方，從","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"圖1","attrs":{}},{"type":"text","text":"裏面可以看出要做gRPC相關適配，在 protocol 這一層是最合適的，我們可以如同 DubboProtocol 一樣，擴展出來一個gRPCProtocol ，這個 gRPC protocol 大體上相當於一個Adapter，將底層的gRPC的實現和我們自身的Dubbo-go結合在一起。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/84/8418b6cfa51437c600e5edda557c2910.png","alt":null,"title":"圖三","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於上述，Dubbo-go幫助我們解決了gRPC的相關整合，相當於在gRPC基礎之上包裝了Dubbo-go治理層，而我們從gRPC的","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"ProtoBuf修改作爲切入點開始，Dubbo-go官方基於Google ","attrs":{}},{"type":"text","text":"protobuf","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":" ","attrs":{}},{"type":"text","text":"擴展插件定義了Dubbo-go gRPC所使用的 protobuf 自定義邏輯代碼，","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"完成兼容性問題即可。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// HelloWorldServiceClientImpl is the client API for HelloWorldService service.\ntype HelloWorldServiceClientImpl struct {\n\tSayHello func(ctx context.Context, in *SayHelloReq, out *SayHelloResp) error\n\t//...\n}\n\n// service Reference\nfunc (c *HelloWorldServiceClientImpl) Reference() string {\n\treturn \"helloWorldServiceImpl\"\n}\n\n// GetDubboStub\nfunc (c *HelloWorldServiceClientImpl) GetDubboStub(cc *grpc.ClientConn) HelloWorldServiceClient {\n\treturn NewHelloWorldServiceClient(cc)\n}\n\n// Server interface\ntype HelloWorldServiceProviderBase struct {\n\tproxyImpl protocol.Invoker\n}\n\n// set invoker proxy\nfunc (s *HelloWorldServiceProviderBase) SetProxyImpl(impl protocol.Invoker) {\n\ts.proxyImpl = impl\n}\n\n// get invoker proxy\nfunc (s *HelloWorldServiceProviderBase) GetProxyImpl() protocol.Invoker {\n\treturn s.proxyImpl\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"實時切換開關","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實時切換，起初是爲了在壓測環境方便兩套不同實現的gRPC方案可實時切換做壓測數據收集，後期是抱着敬畏生產的態度，在生產環境剛接入Dubbo-go時將Rpc框架切換支持服務、方法自由切換，從穩定性出發，選擇性的切換觀測服務穩定性狀態。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此項需求的接入，同樣是從gRPC的","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"ProtoBuf修改作爲切入點開始，同時基於Nacos配置中心實現，我們將原有gRPC的客戶端調用和Dubbo-go的客戶端調用封裝成一個統一實例化入口（這裏簡稱ClinetDubbo），客戶端所有方法新增一份繼承 ClinetDubbo 具體實現（由protobuf擴展插件統一腳本生成），實現內容大致爲獲取ClinetDubbo中的兩套gRPC客戶端，此時通過Nacos配置獲取配置中心所打開的客戶端是哪一套，根據判斷實現走具體的gRPC鏈路。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f5/f5e13e26d4fb9e95aa8c96a1182ad72d.png","alt":null,"title":"圖四","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#262626","name":"user"}}],"text":"左側gRPC常規鏈路，右側Dubbo-go治理模型鏈路。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// ClientDubbo\ntype HelloWorldServiceClientDubbo struct {\n\tGrpcClient HelloWorldServiceClient\n\tDubboClient *HelloWorldServiceClientImpl\n\tOpen bool\n\tCaller string\n}\n\n// 具體的方法實現\nfunc (c *HelloWorldServiceClientDubbo) SayHello(ctx context.Context, req *SayHelloReq, opts ...grpc.CallOption) (*SayHelloResp, error) {\n\tserverName := c.DubboClient.Reference()\n //獲取 nacos配置源數據\n\tserverCfg := nacosCfg.GetServerCfg()\n\tif !c.Open {\n\t\tc.Open = serverCfg.AllOpen\n\t}\n\tcfg := serverCfg.ServiceCfg\n\n // 判斷調用鏈路\n\tif !c.Open &&\n\t\t!cfg[serverName].Open &&\n\t\t(cfg[serverName].Consumers == nil || !cfg[serverName].Consumers[c.Caller]) &&\n\t\t!cfg[serverName].Method[\"SayHello\"].Open &&\n\t\t(cfg[serverName].Method[\"SayHello\"].Consumer == nil || !cfg[serverName].Method[\"SayHello\"].Consumer[c.Caller]) {\n\t\t\n // 原gRPC鏈路\n return c.GrpcClient.SayHello(ctx, req, opts...)\n\t}\n\n\t// Dubbo-go治理鏈路\n\tout := new(SayHelloResp)\n\terr := c.DubboClient.SayHello(ctx, req, out)\n\treturn out, err\n}\n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"項目集成","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以下是基於現有項目結構集成Dubbo-go框架示例：","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"provider","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"type HelloWorldService struct {\n\t*pb.UnimplementedHelloWorldServiceServer\n\t*pb.HelloWorldServiceClientImpl\n\t*pb.HelloWorldServiceProviderBase\n}\n\nfunc NewHelloWorldService() *HelloWorldService {\n\treturn &HelloWorldService{\n\t\tHelloWorldServiceProviderBase: &pb.HelloWorldServiceProviderBase{},\n\t}\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於原有服務提供的基礎之上加入Dubbo-go擴展部分，提供服務註冊。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"consumer","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"//原有Grpc\nvar HelloWorldCli HelloWorldServiceClient \n//Dubbo-go\nvar HelloWorldProvider = &HelloWorldServiceClientImpl{} \n\nfunc GetHelloWorldCli() HelloWorldServiceClient {\n\tif HelloWorldCli == nil {\n\t\tHelloWorldCli = NewHelloWorldClient(grpc_client.GetGrpcClient(...))\n\t}\n\treturn &HelloWorldServiceClientDubbo{\n\t\tGrpcClient: HelloWorldCli,\n\t\tDubboClient: HelloWorldProvider,\n\t\tCaller: dubboCfg.Caller,\n\t\tOpen: false,\n\t}\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"GetHelloWorldCli()","attrs":{}}],"attrs":{}},{"type":"text","text":"簡單封裝了客戶端調用，此方法最終返回 ","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"HelloWorldServiceClientDubbo","attrs":{}}],"attrs":{}},{"type":"text","text":" 結構體的方法，客戶端發起調用進入以 ","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"HelloWorldServiceClientDubbo","attrs":{}}],"attrs":{}},{"type":"text","text":" 實現的具體方法中，根據配置項判斷執行具體gRPC調用鏈路。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Main","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"func main() {\n\t//provider\n\tconfig.SetProviderService(rpc.NewHelloWorldService(), ...)\n \n // 設置服務消費者\n\tconfig.SetConsumerService(api..., ....)\n \n // 加載dubbo\n\tconfig.Load()\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以上就是社區服務集成Dubbo-go的整體思路與方案，我們會發現在現有項目中需要改動的代碼量很少，且對業務方方代碼無任何侵入。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"小結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo-go集成，增強了業務服務gRPC調用過程中治理能力，基於cluster增加了服務集羣的容錯能力，實現了應用服務之間容錯能力的可配置性；完善且統一了社區服務原始新老架構服務的全鏈路監控和服務指標監控告警；增強了業務夥伴對集羣內服務的透明化、可控性，在遇到問題後整體的鏈路梳理上有了更多的可參考信息。基於Dubbo整體生態，可輕鬆支持Golang與Java的Rpc互通，做到跨語言Rpc調用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1a/1afbca1b104655606416a67da2dabe75.png","alt":null,"title":"圖五","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"最後","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Dubbo-go作爲一個微服務框架，自身包含治理能力，這部分能力如何與K8S融洽結合。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"K8S 提供了pod/endpoint/service 三層維度的資源，可以通過監聽pod/endpoint/service三層維度資源的事件，作出合理的處理以達到服務治理的目的，不需要引入額外組件，通過監聽 k8s 中最細粒度資源 pod 的事件，通過k8s apiserver獲取pod列表，只是通過apiserver使用etcd的服務註冊和服務通知能力，其他繼續使用Dubbo-go的服務治理能力，模型簡單，不需要實現額外的模塊，幾乎不需要對Dubbo作出改動。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文｜小柯","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關注得物技術，攜手走向技術的雲端","attrs":{}}]}]}