基於 Golang 構建高可擴展的雲原生 PaaS(附 PPT 下載)

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/4a/4a26b5498ac4f0b97e549a42db8244c9.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者|劉浩楊來源|","attrs":{}},{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/5u3W8kPAUo6FE98qlk5OAA","title":"","type":null},"content":[{"type":"text","text":"爾達 Erda 公衆號","attrs":{}}]},{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文整理自劉浩楊在 GopherChina 2021 北京站主會場的演講,微信添加:Erda202106,聯繫小助手即可獲取講師 PPT。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"前言","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當今時代,數字化轉型概念在 ToB 領域非常火熱,越來越多的企業需要數字化轉型,因此越來越多的廠商湧入了 ToB 市場,端點就是其中一員。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"端點的核心業務是給企業提供從採購、交易、履約、倉儲到零售全鏈路的數字化轉型解決方案。圍繞這個目的,我們建立了一整套面向雲原生的企業數字化軟件產品,其中:​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PaaS 平臺 Erda 是技術底座。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Trantor 是具有 HPAPaaS 能力的研發底座,運行在 Erda 之上,提供低代碼和高效開發的能力。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Gaia 是完整的業務能力平臺,覆蓋計劃、尋源、採購、營銷、服務、銷售、交易、履約、結算等企業供應鏈業務領域,提供會員營銷、全渠道運營、SRM 採購和在線銷售平臺等業務產品。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過這幾層能力的組合,使我們具備了生產可定製 SaaS 化產品的能力,爲企業客戶提供端到端的解決方案,滿足各行業不同客戶的各類需求。所以可以看到在這個分層的企業數字化研發體系中,我們通過 PaaS 平臺向下屏蔽複雜的基礎設施,使整個業務系統擁有架構多雲之上的能力;同時,PaaS 平臺也向上提供微服務研發和治理的技術中臺能力,讓上層用戶能夠更專注於業務系統自身的構建。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下來,簡單介紹一下我們在構建 Erda 過程中的思考,文章後半部分將回到技術本身,來分享一下我們如何使用 Golang 實現雲原生的 PaaS 平臺。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c3/c3daac16c9b0a1c134e937a0b27427f6.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"軟件交付面臨的問題和挑戰","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"端點從 2012 年成立到現在,一直聚焦在企業級軟件開發和交付上,我們也從最開始的軟件定製交付,到今天使用標準軟件 + 二開機制解決定製需求的方式交付。在這個過程中,我們遇到的問題可以總結爲以下 4 點:​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"隨着交付規模的快速增長,開發和交付團隊要如何提效?","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"業務軟件系統如何適配複雜的客戶部署環境?","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"大規模交付的部署過程需要可被標準化。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"交付上線後需要持續保障業務穩定性。","attrs":{}}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如今,敏捷開發是一個相對普及的概念,我們在做軟件交付的時候,也需要把開發流程從傳統的瀑布式開發轉變爲敏捷開發,比如一個理想的開發方式是:使用項目協同工具來管理研發過程的需求、任務、缺陷;使用 gitflow 來管理功能需求的開發;使用自動化的流水線實現持續集成;然後基於容器化來部署我們的軟件;部署之後觸發自動化測試來驗證功能的正確性。同時,像 API 網關、微服務治理、容器服務、流計算等這些在互聯網公司流行的技術,也逐漸開始受到大型傳統企業的青睞,這對我們的要求也是進一步提高。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了解決這些問題,我們就需要一個完整的 DevOps 平臺來支撐整個研發交付的流程,來達到提高研發效能的目的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/0b/0beacecba9134a6c698a7600850ce9d3.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"剛纔講了在研發過程中的問題,接下來聊一下在需求開發完成後,去企業提供的環境進行交付的環節。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我們做過的客戶裏面,有自建機房用 zstack 來提供虛擬化,也有采購阿里雲、騰訊雲、或者微軟雲 azure 的 ECS,還有的會提供現成的容器環境但管理接口不同,比如 Rancher 和 OpenShift,這些情況我們都碰到過。甚至最近的一個項目裏,客戶提供的是 arm 架構的運行環境。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲這些企業提供的環境千差萬別,所以交付就變成了一件很痛苦的事情。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"回顧計算機軟件的發展歷史,我們知道 java 是現在很流行的一個開發語言和平臺,java 能流行的很大一個原因是因爲它提出了 Java 虛擬機,通過編譯 Java 代碼到通用的字節碼,再由運行時去即時編譯字節碼到不同平臺的彙編方式來實現“一次編寫,到處運行”。那麼,我們構建的軟件系統是否也可以像 java 一樣 “一次打包,到處運行”?說到這裏,可能很多人都會想到:基於 Docker 和 Kubernetes 作爲軟件的運行時也可以讓我們的軟件製品屏蔽掉不同的部署環境。但是,管理不同環境下的 Kubernetes 也是一件很複雜的事情,我們需要一個平臺來向上承載不同的業務系統,向下管理資源和應用調度。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們今天的做法是通過一個多雲管理平臺來解決這個問題。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/30/30ff4bb7023c4747fa86f8573f527d8c.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"很多軟件系統在驗收交付上線後,如果沒有一個完整的平臺實時監控、洞察系統的健康狀態,將會導致各種問題(穩定性、異常、錯誤、攻擊)時有發生,嚴重影響業務的正常運行。保證業務穩定性也是我們可以提供給客戶的核心服務之一。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在之前的案例裏,我們利用 PaaS 上監控的高度擴展性,幫助一家排名世界 500 強的外企搭建了一套雲上的統一監控運維平臺,真正實現了從雲產品,比如 SLB、WAF、ECS、RDS,到平臺層、再到 PaaS 上運行的業務系統的全鏈路監控和日誌數據的統一分析,再通過 PaaS 內置的可視化和告警入口,使得客戶只需要投入很少的 IT 運維人員就能夠保證系統的穩定性。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另一個場景是我們的 PaaS 平臺自身。我們現在同時在運維着 100+ 的 Kubernetes 集羣,但全職負責運維的同事大概只有三四人,而且可預見規模擴大的同時,也不需要在運維上投入更多的人力,這也得益於我們對平臺可觀測性的全面覆蓋。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"所以可以看到,研發、交付、運維遇到的很多問題,我們都是通過 PaaS 很好地解決了。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a9/a963acaebb261dcadba9fad57359b59f.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"端點一站式 PaaS : Erda","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面提到,我們構建 PaaS 很大一部分是來自端點自身的需求。早在 2016 年我們就開始搭建端點的部署和應用監控平臺,當時的選擇是使用 Docker 和 Jenkins 等開源工具、讓業務應用手寫 dockerfile 的方式來進行發佈;在監控的選型上,使用 Zipkin 作爲分佈式應用的追蹤系統、使用 ELK 作爲日誌平臺,這可能也是很多公司常見的做法。這套系統搭建起來以後,在一段時間裏滿足了公司內部的開發使用,但是隨着業務對平臺要求越來越多,我們也引入了更多的開源組件,維護這套系統的成本也越來越高。同時,有客戶希望我們能把技術平臺也一起對外輸出,很顯然使用開源組件混搭的這臺系統並不能滿足我們的要求。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 2017 年,我們想要重新設計一個以應用爲中心的研發平臺產品,於是選擇了以 DC/OS 爲底座,在上層使用 Java 構建 DevOps 和服務治理平臺,並且在 2018 年初交付了第一個客戶。現在來看,選擇 mesos 而不是 K8s,當時是走了彎路的,所以從 2018 年下半年開始,我們也全面擁抱 K8s 生態。同時因爲 Java 的性能和資源佔用問題,我們做了第二次的 PaaS 架構升級,開始使用 Golang 重寫了大部分的平臺組件,也因爲要同時支持 DC/OS 和 K8s 兩個容器平臺,我們拆分出了異構的調度器模塊和流水線模塊。時至今日,我們正在交付的所有客戶都已經在使用 K8s 架構。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"越往後走,我們開發的功能就越多,平臺覆蓋的範圍也越大。我們陸陸續續在平臺上實現了微服務治理、快數據、移動開發、邊緣計算等諸多平臺級應用,演變成了今天 Erda 的一站式 PaaS 產品矩陣。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3e/3e5abf0c246c340668d1d5eac1e0aa6c.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上圖是 Erda 現今的產品大圖,其中以多雲管理、 DevOps、微服務治理和快數據四個平臺爲核心。我們在多雲管理平臺裏面實現了兩個比較重要的能力:​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個是對異構資源環境的支持,支持管理物理機、虛擬機、支持原生的 K8s 集羣、DC/OS 集羣等。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"另一個能力是原生的多集羣架構,可以通過不同的集羣對環境進行物理隔離,比如實現開發、測試和生產環境的資源互不影響。基於容器化,我們也可以很容易地實現對資源池的精細管理,按 CPU 和內存的粒度精確控制不同應用的資源分配等。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在多雲平臺之上,DevOps、微服務和快數據平臺分別來管理業務和數據。在業務側提供軟件開發的需求、開發、測試、部署、運維全生命週期的功能,在數據側也通過批流一體實現真正快數據治理能力。除此之外,業務和數據的統一協同,業務可以更大程度地通過數字進行描述,從而實現更爲精細化的運營。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7d/7d74b8a50625e0dd35fe3bd020f31494.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"Erda 架構的思考","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面介紹了 Erda 提供的的一站式能力,那麼我們是怎麼去實現在 PaaS 上實現構建不同的平臺服務,實現同一個內核支持微服務開發和數據治理的能力呢?首先來看我們對 PaaS 擴展架構的思考。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在最初的版本里,我們只有簡單的應用部署功能,把所有的代碼放在一起構建了一個單體服務。隨着平臺功能逐漸增多,我們也開始把系統拆分成更多的微服務,在拆分的過程中,嚴格遵守一些設計原則,進行合理的架構分層來避免服務爆炸。在 Erda 當前的系統架構中,我們把系統分爲三層:平臺應用層,服務層和 PaaS 的內核層,並且約定每一層之間不允許產生橫向調用,只允許調用下一層服務提供的接口。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"內核層","attrs":{}},{"type":"text","text":"提供了 PaaS 和基礎設施交互的能力**,**它包含三個核心組件和相對獨立的監控系統:​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"cluster manager 通過和每個集羣的 cluster agent 通信,實現多集羣管理的能力。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"pipeline 服務實現了 workflow 的編排和 job 調度能力,包括短時任務、數據的批任務、流任務等。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"orchestrator 服務實現了 workload 的編排和有狀態/無狀態的服務調度能力。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"監控系統負責收集、計算和展示每個集羣上報的可觀察性數據。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"服務層","attrs":{}},{"type":"text","text":"類似於業務系統中提到的中臺概念,提供平臺中的公共功能模塊的能力,如用戶、租戶、權限、通知、報表、事件、審計、版本控制、項目管理等。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"應用層","attrs":{}},{"type":"text","text":"則是每個平臺調用下層的能力進行自身的業務邏輯封裝,同時提供和前端交互的接口。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過這樣一個分層架構,可以很容易地去實現一個新的平臺應用。例如在實現移動開發平臺時,代碼管理、構建流水線、審計可以複用 DevOps 原有的能力,用戶行爲分析、異常分析等功能依賴監控系統的能力得以實現。​","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3e/3e6930d0f7b5eecd20fe3092d79df28a.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"衆所周知,Kubernetes 作爲雲原生應用的最佳部署平臺,開放了容器運行時接口(CRI)、容器網絡接口(CNI)和容器存儲接口(CSI),這些接口讓 Kubernetes 的開放性變得最大化,而 Kubernetes 本身則專注於容器調度。同樣,在以應用爲中心的 DevOps 平臺中,CI/CD 是最爲核心的能力,而我們也認爲 CI/CD 的核心是對 workflow 和 workload 的管理,所以我們也開放了 workflow 接口和 workload 接口,讓 DevOps 平臺專注於應用開發管理,而不必關注 JOB 和應用被部署到何種平臺上。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ad/ad7a6ceeb93141cedf428116296a9858.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏我們也在實踐基礎設施即代碼的思想。在 Erda 上,你可以使用 pipeline.yaml 來定義應用的 CI/CD 全流程,比如代碼拉取、質量掃描、單測執行、應用編譯、構建和部署。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個 pipeline.yaml 的示例如下:","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"yaml"},"content":[{"type":"text","text":"stages:\n stage:\n - git-checkout:\n stage:\n - buildpack:\n params:\n context: ${git-checkout}\n stage:\n - release:\n params:\n services:\n trade-server: ${buildpack:image}\n erda_yaml: ${git-checkout}/erda.yaml\n stage:\n - deploy:\n params:\n release_id: ${release:id}\n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時,還可以使用聲明式的 erda.yaml 來描述你的應用微服務架構,包括微服務之間的依賴關係、對中間件的依賴等,任何標準的 erda.yml 都可以被 DevOps 平臺所部署。一個 erda.yaml 的示例如下:​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"yaml"},"content":[{"type":"text","text":"services:\n trade-server:\n resources:\n cpu: 2\n mem: 2048\n deployments:\n replicas: 2\n ports:\n - port: 8080\n envs:\n ENABLE_ALIPAY: true\naddons:\n mysql:\n plan: mysql:basic\n options:\n version: 5.7.23\n create_dbs: trade_db \n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"值得一提的是,基於 pipeline(流水線)和 erda(部署)兩個定義之上,我們構建了豐富的 addon 和 action 市場,你可以把自己的服務打包成一個開箱即用的 addon,也可以來實現自己的流水線 action,這些機制讓 DevOps 平臺的擴展性得到極大的增強。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們再來看一下 Erda 微服務平臺的設計。共分爲控制面和微服務引擎兩個部分,它們之間通過我們設計的協議進行通信。通常,我們會使用 Erda 內置的微服務引擎,這裏的選型是使用了 Spring Cloud Alibaba 組件作爲內置實現,可以兼容企業原有的 Spring Cloud & Dubbo 的業務應用,不需要改動任何代碼即可接入。同時,對於有特殊需求使用雲服務的用戶,也支持直接對接雲廠商的微服務引擎,我們已經實現了阿里雲 MSE 的可選實現。在這樣的設計下,現在很流行的 ServiceMesh 系統比如 Istio 也可以很容易的作爲 Erda 微服務引擎的一種實現被接入。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/37/37b2f3cd8297002e04908fb521494d5a.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後,我們再來講一下監控系統的設計。在 Erda 的場景裏,監控系統同時支撐基礎監控、APM 和瀏覽器端的監控,所以我們沒有直接使用社區流行的監控系統,比如 Prometheus 或者 Skywalking,而是選擇了自研一套監控系統,在採集端適配社區的標準化協議。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先我們在每個集羣中安裝基於 Telegraf 深度定製的 Cluster Agent,負責採集 Kubernetes、Erda 在集羣中部署組件的可觀察性數據,同時這個 Agent 也可以模擬成一個 Prometheus Server,拉取集羣中被自動發現的 Exporters 數據。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼在每個節點上,也會部署一個 Node Agent,可以自動發現節點上的 Pod,並把 Pod 識別爲 Erda 平臺定義的 Service、Job 或者 Addon 組件。同時 Node Agent 也會作爲一個本地 Proxy,接收 Pod 中的業務應用 Java Agent 探針和 NodeJS 探針上報的應用請求和 Trace 數據,轉發給後端的 Metrics Gateway 組件。在未來,我們計劃支持更多的第三方協議,如 SkyWalking 和 OpenTelemetry,來更多地提高 Erda 在可觀察性上的擴展。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/4f/4f17468f37ae713856948bd87b52d95c.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"Erda Infra 模塊化開發框架","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面從架構的角度講了我們對 PaaS 擴展的一些思考,也提到通過微服務的方式來構建整個平臺,實現微服務化的重點就是要對系統功能進行合理拆解,把系統拆解成諸多邊界清晰、可以內聚的業務模塊。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雖然在 golang 社區中有很多 web 開發框架,如 gin、go-zero、beego,但深思熟慮後,我們發現需要的不僅僅是一個 Web 框架和微服務框架,而且還需要一個以模塊化開發爲核心,可以通過接口對系統進行擴展的開發體系。基於此我們設計了 Erda Infra,一套輕量級 Go 微服務框架,包含大量現成的模塊和工具,能夠快速構建起以模塊化驅動的應用程序。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Erda-Infra 的核心特性包括:​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以模塊化設計方式來驅動應用系統實現,支持模塊可插拔。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"統一配置讀取,支持默認值、支持從文件、環境變量、命令行參數讀取。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"統一模塊的初始化、啓動、關閉。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"統一管理模塊間的依賴關係。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支持模塊間的依賴注入。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"包含大量現成的微模塊 Provider,包括 etcd、docker、cassandra 、es 的 sdk、Http Server、gRPC Server等。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用 Protocol Buffer 定義接口,支持自動生成 gRPC 和 HTTP 接口和客戶端調用代碼。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"提供快速構建模塊的代碼生成工具。","attrs":{}}]}]}],"attrs":{}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b9/b9027bf8a7cece251dd3946d77529eed.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Erda 現在有數千個功能點包含在幾十個服務裏面,一旦服務功能拆分設計的不合理,很容易引發整個項目的維護成本問題。我們在從架構層面審視模塊拆分的同時,也期望使用開發框架來對開發進行一些約束。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"第一個原則","attrs":{}},{"type":"text","text":"是對模塊的拆分粒度要儘可能小,讓每個模塊的邏輯能高度內聚,對外的依賴可以松耦合。拿應用監控這個子系統來說,我們拆解成了拓撲、服務列表、服務監控、瀏覽器監控、錯誤分析、鏈路追蹤等相對獨立的模塊,然後使用 Erda Infra 提供的能力拼裝到一起組成監控的服務。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"第二個原則","attrs":{}},{"type":"text","text":"是模塊的開發者需要面向接口開發,而不是面向實現;模塊之間依賴接口,而不是具體的實現類。這樣做的好處是需要對模塊進行擴展的時候,可以不需要修改已有的代碼。我們也在 Erda Infra 中實現了依賴注入容器,通過類似 Spring 中 IoC 的方式,讓開發模塊的人更關注自己負責的邏輯單元。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面來看一個 Erda Infra 中最簡單的模塊示例,我們把模塊定義爲 provider 結構,可以把需要注入的接口聲明在 provider 的字段中,同時還需要實現 Init 、Run 函數來控制模塊的生命週期,最後把定義好的模塊註冊到全局的服務容器中,即可完成一個微模塊的實現。​","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/35/35254e9a9560a8c66e330f1dfb6e7851.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼我們如何在服務容器中定位到實現的接口呢?Erda Infra 提供了一個 Provide 接口,在這個接口中我們可以顯示聲明接口的具體實現,下面是 mysql-client 模塊的聲明示例:​","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/99/9910de99b7490053af4419c522c5a2bd.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於接口的獲取,Erda Infra 提供了自動注入和使用服務定位器來解析兩種方式。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自動注入和 Spring 的使用一樣,通過在字段上標記 autowired 標籤即可生效:​","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/aa/aa62af0be258e491701d1b4e1491bf93.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"服務定位器模式則可以使用在後臺任務中,通過框架的服務上下文來獲取需要的接口具體實現:​","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ae/ae25aeeff1827a4601f904a0de891b1d.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後再來聊一聊 Erda 中的 API 定義方式,我們把 API 統一都定義在 Protobuf 文件中,這樣可以統一生成 gRPC、HTTP、OpenAPI 接口,以及 Client 的代碼。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假設系統中有一個服務叫 Greeter ,定義如下所示。其中:通過 google.api.http 選項,來定義 http 相關的信息,可以自動把請求的數據解析到 Path 參數和 Query 參數上。還可以通過在對應的接口上聲明 +publish 特殊的註釋,來表示發佈該接口到 OpenAPI 上,如果想要把某個接口強制標記爲內部接口,那麼也可以通過 +private 註釋來實現。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"protobuf"},"content":[{"type":"text","text":"// +publish prefix:\"/api/greeter\" backend-prefix:\"/api\" service:\"user-center\"\nservice GreeterService {\n // say hello\n // +publish\n rpc SayHello (HelloRequest) returns (HelloResponse) {\n option (google.api.http) = {\n get: \"/api/hello/{name}\",\n };\n }\n }\n \n message HelloRequest {\n string name = 1;\n }\n \n message HelloResponse {\n string data = 1;\n }\n","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Erda Infra 作爲我們使用的基礎框架,現在還有一些不完善的地方,我們後續計劃繼續添加服務註冊發現的支持,來實現運行時的服務模塊管理等,以此持續增強 Erda 的平臺擴展性。​","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"總結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們從 3 月開始計劃 Erda 的開源,到現在已經構建了第一個穩定的開源版本,第二個版本也將在近期發佈。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Erda 本身提供的一站式服務開發治理和數據治理功能已經非常豐富,我們接下來將會重點對用戶體驗和功能細節進行打磨,對開源代碼重構以提高代碼質量,簡化平臺的安裝部署方式等。歡迎大家關注 Erda 開源項目,一起探討、交流雲原生 PaaS 相關的技術,共建開源社區。​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Erda Github 地址:","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/erda-project/erda","title":"","type":null},"content":[{"type":"text","text":"https://github.com/erda-project/erda","attrs":{}}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Erda Cloud 官網:","attrs":{}},{"type":"link","attrs":{"href":"https://www.erda.cloud/","title":"","type":null},"content":[{"type":"text","text":"https://www.erda.cloud/","attrs":{}}]}]}]}],"attrs":{}}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章