微服務架構設計與最佳實踐

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文有『Go開源說』第三期 go-zero 直播內容修改整理而成，視頻內容較長，拆分成上下篇，本文內容有所刪減和重構。","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大家好，很高興來到“GO開源說” 跟大家分享開源項目背後的一些故事、設計思想以及使用方法，今天分享的項目是 go-zero，一個集成了各種工程實踐的 web 和 rpc 框架。我是Kevin，go-zero 作者，我的 github id 是 ","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/kevwan","title":""},"content":[{"type":"text","text":"kevwan","attrs":{}}]},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"go-zero 概覽","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"go-zero 雖然是20年8月7號纔開源，但是已經經過線上大規模檢驗了，也是我近20年工程經驗的積累，開源後得到社區的積極反饋，在6個多月的時間裏，獲得了6.4k star。多次登頂github Go語言日榜、周榜、月榜榜首，並獲得了gitee最有價值項目（GVP），開源中國年度最佳人氣項目。同時微信社區極爲活躍，3500+人的社區羣，go-zero愛好者們一起交流go-zero使用心得和討論使用過程中的問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/60/60e3ec8d232b818984b4a18ef3616570.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下圖中間三層是 go-zero 內建支持的服務治理相關組件，基本涵蓋了微服務治理主要的能力，而且是基本不需要開發者自己配置的，默認方案已經是經過大規模線上項目調優的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/04/048d0a65f69fdfe908bac88d1f8d0bf2.png","alt":"arch","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"微服務系統設計中的痛點","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. 微服務系統如何拆分？","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"先粗後細，不要過細，切忌一個接口一個服務","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"橫向拆分，而非縱向，我們儘量不要超過三層調用","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"單向調用，嚴禁循環調用","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"禁止接口類型透傳，在不同的層之間不要共享同一個數據定義，避免一處修改，影響其它","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"沒有依賴關係的串行調用改爲並行，可以通過 core/mr 包降低響應延遲而不增加系統負載","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a1/a1175f9d507a474e57e23d6082a18b85.png","alt":"mr","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2. 如何保障高併發高可用？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"良好的數據邊界","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 數據邊界是微服務拆分的核心，不同的服務之間不要顯示共享數據，而應該通過 rpc 共享。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"高效的緩存管理","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 服務能否支撐高併發，緩存很關鍵。緩存機制不光要設計好，還需要通過工具儘可能讓業務開發人員避免出錯，因爲緩存代碼的編寫有相當的難度，goctl 很好的生成了自動管理緩存的代碼。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"優雅的熔斷降載保護","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 微服務系統一般都是由大量服務共同組合而成的，服務多了自然會有某個服務出現故障的風險，我們不能讓某個服務的故障導致整個系統不可用，這就是服務雪崩，要避免雪崩，我們就需要有效隔離有故障的服務，從而降級可用。熔斷和降載是防止服務雪崩的最有效手段之一。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"彈性伸縮能力","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 對於高併發的系統來說，突發流量洪峯的情況下，系統需要能夠及時水平擴容，go-zero 的自適應降載可以很好的配合 kubernetes 集羣的自動水平伸縮能力。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"清晰的資源使用定義","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 要想讓系統保持穩定，我們一定要對資源使用做清晰的定義，比如我們到底在什麼時候考慮擴充資源，是資源佔用率達到了50%還是70%，必須對系統資源的使用狀況有足夠清晰的定義。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"高效的監控報警","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 我在團隊內部一直強調：沒有度量就沒有優化。我們必須對系統有高效的監控和及時的報警機制。這樣才能讓我們對整個系統的運行狀態有足夠的瞭解。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"大型微服務項目從何下手？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2b/2bc28e523a6281eb1512b679c92aa9c4.jpeg","alt":"overview","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"微服務系統大體上看起來如上圖，但是我們並不是一定要業務一開始就上微服務，我們看看一個典型的微服務系統是怎麼進化而來的，下面粗略的講解一下大型微服務系統的進化過程。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從單體服務開始","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 項目剛開始，我們不要一味的追求技術的先進性，因爲大部分項目是走不到高併發的那一天的，我們需要的是第一時間滿足業務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"業務優先，技術支撐","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 我在團隊裏講：架構是從業務中來，到業務中去。任何脫離了業務的技術都是自嗨，我們需要把滿足業務需要作爲第一優先級，技術需要支撐業務現在和可預期發展的需要即可。當然，有滿足自身業務的現成的框架和最佳實踐那是最好了，但不要讓技術棧過於複雜，避免重心從業務轉移到技術本身來。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"服務指標監控","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 隨着業務的發展，我們可能需要對技術做一定的升級和改造了，但是我們一直強調：沒有度量就沒有優化。所以我們需要及時加上對整個服務的關鍵指標的監控，從而讓我們在瞭解系統的前提下進行必要的改造。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據拆分+緩存管理","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 當業務發展到一定程度之後，基於監控，我們發現服務必須要做拆分了。那麼我們第一步要做的是先把數據拆分清楚，數據拆分後，我們就可以加上對應的緩存管理，從而保障數據層面的穩定性。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"服務拆分","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 相對於數據拆分，服務的拆分相對是比較容易的，基於拆分好的數據，對應出上層的服務，因爲服務是無狀態的，所以這個拆分就比較容易。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支撐系統建設","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 隨着業務的發展，日常的系統維護工作就顯得比較繁瑣和容易出錯了。此時，我們需要建設支撐系統，如何部署新服務，如何更新老服務，是不是要上 kubernetes，等等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自動化+工程建設","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 當業務發展到一定程度，工程效率就是一個很大的問題了。goctl 就是爲了解決自動化和工程效率問題而生，其中內置的 api, rpc, model, Dockerfile, k8s部署文件等的自動生成節省了我們大量時間，也避免了業務開發中的錯誤。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"更多內容","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分享中的 go-zero 緩存最佳實踐部分在","attrs":{}},{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/aafdbf584a5d4111a2324adf8","title":""},"content":[{"type":"text","text":"這裏","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你想要更好的瞭解 go-zero 項目，歡迎前往官方網站上學習具體的示例。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"視頻回放地址","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://www.bilibili.com/video/BV1Jy4y127Xu","title":""},"content":[{"type":"text","text":"https://www.bilibili.com/video/BV1Jy4y127Xu","attrs":{}}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"項目地址","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://github.com/tal-tech/go-zero","title":""},"content":[{"type":"text","text":"https://github.com/tal-tech/go-zero","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"歡迎使用 go-zero 並 ","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"star","attrs":{}},{"type":"text","text":" 支持我們！","attrs":{}}]}]}