爲什麼你家的 K8s 應用平臺不好用?

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"圍繞 K8s 構建應用平臺的常見問題","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#434343","name":"user"}}],"text":"問題一,用戶訴求與 Kubernetes API 之間的鴻溝","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"大家都知道 Kubernetes 這個項目的定位是 “Platform for Platforms”,它提供了一系列統一的概念和擴展方式,屏蔽了底層存儲、網絡、計算等基礎設施的實現細節,使得平臺構建者基於 K8s 可以快速構建出一個場景化能力/平臺出來。但是這往往也使得這些平臺的構建者陷入了誤區,直接把 K8s 的概念暴露給了業務用戶,還美其名曰“原生 K8s 體驗”,這帶來的最直接的問題就是用戶學不會,不願意遷移到這些“先進”的雲原生平臺上來。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/0d/0dc2b2b4f02705a849dbac2294279d6b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"圖1 業務和 K8s 的概念差異","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"事實上這就是平臺構建者們需要解決的問題。大家可以看下圖1,業務用戶關心的應用管理概念裏面,在研發層面是代碼、鏡像、應用、CI/CD 流程等,而 K8s 的對應概念是 Pod、Deployment、CRD等等;在運維層面是彈性策略、灰度發佈策略等,而 K8s 對應的概念是 Ingress, MetricsPodAutoScaler 等等;在用戶操作方式層面是 GUI、CLI 等,而 K8s 就是寫 YAML 文件。毫無疑問這裏面存在大量差異導致產生了一個很難逾越的概念鴻溝,研發人員需要花大量時間學習業務無關的知識,這更加使得業務壓力雪上加霜。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"所以做的好的平臺針對這一問題,通常會在 K8s 的基礎上包一層,針對業務場景提供“業務能夠理解的”概念和服務。但是這樣做,就往往會出現另外兩方面的問題。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"問題二,針對不同場景構建的應用平臺形成大量“穀倉”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"針對大一點的公司會出現“穀倉”問題,因爲大公司包含衆多的業務線,很容易形成不同場景的業務平臺,雖然大家都是基於 K8s 構建的,但是在上面形成了風格迥異的概念和接口。如圖2所示,這些不同的平臺就像一個個“穀倉”,用戶體驗和技術問題依然存在。 ","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/86/866b059624e6ae3ec5cd4070031a8d83.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"圖2 不同場景應用平臺形成穀倉問題","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"用戶體驗上,在使用單個平臺的情況下還好,但是實際用戶需求往往並不單一。比如一個業務用戶在“無狀態應用平臺 - A”這邊創建了一個微服務,然後又不得不去“有狀態應用平臺 - B”創建一箇中間件,使得用戶在對接不同場景的應用時缺乏統一性,更談不上橫向交互能力了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在技術上,不同場景的平臺都基於 K8s 構建了獨立的能力模塊,往往容易出現重複造輪子的現象,每個獨立的能力都缺少橫向複用和可移植性,導致碎片化比較嚴重,技術上也很難積累沉澱,本質上就造成人力資源的浪費。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"問題三,直接基於 K8s 構建 PaaS 跟不上生態發展","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"針對小一點的公司,雖然沒有“穀倉”問題,但是構建一個 PaaS 去蓋住 K8s 的複雜概念並不是“銀彈”。大家可以想象,爲了把 K8s 的複雜概念轉換成用戶能理解的業務概念,常規的做法就是做一層平臺專屬的 API,通過這層 API 做轉換,如圖所示。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/36/36717099f263fb46bd40b73aae341d36.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"圖3 構建 PaaS 蓋住 K8s 問題","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這層 API 在最初的時候沒有問題,但是隨着用戶需求的增長,以及 K8s 生態的快速發展,這層 API 就會成爲瓶頸,擋住平臺用戶使用繁榮的生態能力。這個問題在這兩年已經成爲了共識,導致平臺構建者疲於奔命地努力封裝 API, 到頭來發現用戶已經學會了社區 K8s 上某個新功能的玩法,還不如直接把 K8s API 暴露出去。然後大量的平臺又退化到直接暴露 K8s API 這一層次上,這又回到了第一個問題上面。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"構建好用的“K8s應用平臺”核心問題在哪?","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在研究觀察了大量開源和商業化的應用管理產品以後,我們得出了一些有意思的結論。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"結論一:抽象可以有效降低平臺的學習和使用門檻","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c0/c0c32540650b9d61a6450fce0c306060.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"圖4 不同項目或產品的抽象程度","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"如圖4 所示,以業內比較典型的","attrs":{}},{"type":"link","attrs":{"href":"https://kubernetes.io/","title":null}},{"type":"link","attrs":{"href":"https://kubernetes.io/","title":null},"content":[{"type":"text","text":"Kubernetes","attrs":{}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"、","attrs":{}},{"type":"link","attrs":{"href":"https://knative.dev/","title":null},"content":[{"type":"text","text":"Knative","attrs":{}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"、","attrs":{}},{"type":"link","attrs":{"href":"https://rio.io/","title":null},"content":[{"type":"text","text":"Racher Rio","attrs":{}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 以及","attrs":{}},{"type":"link","attrs":{"href":"https://www.heroku.com/","title":null}},{"type":"link","attrs":{"href":"https://www.heroku.com/","title":null},"content":[{"type":"text","text":"Heroku","attrs":{}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 這四個平臺爲例。","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"其中,Kubernetes 的定位是 Platform for Platform,它只針對基礎設施做封裝來賦能平臺團隊,不面向最終用戶做抽象。所以業界共識和最佳實踐,都是需要基於 Kubernetes 做二次開發,至少也得用一個圖形界面來進行二次封裝,才能給用戶使用。在這裏,我們把 Kubernetes 作爲抽象程度最底的基準。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Knative,只面向 Serverless 場景(函數模式,按流量擴容計費,無狀態,事件驅動)設計。它通過自己的 CRD 在 Kubernetes 的基礎上做了一層模型限制,把入口緊緊限制在 Service/Route/Configuration/Event 這幾個 Serverless 獨有的概念上,從而在 Kubernetes 之上提供了 Serverless 的使用體驗,相比於 Kubernetes 已經大大降低了用戶的使用心智,但 Knative 的定位依然不是面向最終用戶的,而是爲了構建 Cloud Run 這樣面向用戶的產品而設計的。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"RIO 是一個基於 Kubernetes 實現的 PaaS,它提供了一個 CLI,圍繞應用生命週期管理將很多底層 K8s 的功能包裝成了簡單的 CLI 命令,如 rio run 代表應用運行,rio route 增加一條訪問路由,這個使用體驗的心智負擔比 Knative 要低很多,基本上已經迴歸到了業務研發人員能夠理解的層面上。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Heroku 是業內最知名的 PaaS,它並非構建在 Kubernetes 之上,但它早在十幾年前就開始推崇“以應用爲中心”的理念,易用性相較於 RIO 更進一步。其抽象也已經是大家熟知的應用(heroku apps),域名(heroku domains),發佈,工作流等用戶級的概念,使用起來基本不需要針對這些概念做額外學習,在整個業界的反饋中,Heroku 的使用體驗也是標杆級的。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"結論二:抽象程度越高,平臺的擴展性和被平臺團隊的認可度顯著降低","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/71/71fa0119e01a6238c76fa8230984711a.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"圖5 抽象程度越高擴展性卻越差","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"如圖 5 所示,剛剛我們看到抽象程度越高越易用的四個平臺,在擴展性上也呈現出了顯著的差異。","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"同樣,依然拿 Kubernetes 作爲基準,它不提供用戶層抽象,天然就是一個面向平臺團隊的工具定位,通過 CRD + Controller 的方式來進行擴展。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Knative 雖然也在讓用戶操作 K8s,但是因爲其模型限制在了少數幾個概念對象上,所以大家雖然可以在背後實現 Controller 做擴展,但是擴展的字段卻只能侷限在原先 Knative 的模型對象上,只能通過在對象上加 annotation(類似於打標籤加備註)的方式來 workaround。所以說,Knative 作爲 Serverless 場景下的平臺構建工具尚具備一定的可擴展性,但要服務更廣泛的場景,幾乎不可能。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"RIO 的定位是基於 K8s 的 “Micro PaaS”,它在擴展性上沒有給出方案,也就是說除了通過修改代碼添加功能之外,我們找不到其他擴展方式。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Heroku 作爲一個經典 PaaS,自己定義了一系列的規範來提供擴展性,同時開創了自己的插件生態。但這個生態相比於 Kubernetes 的今天的生態來說,就完全“不值一提”了。可以說,Heroku 的可擴展性問題,是它最終從主流平臺淡出的一個重要原因,感興趣的朋友可以閱讀筆者之前的文章","attrs":{}},{"type":"link","attrs":{"href":"https://www.infoq.cn/article/zhbbauekoowfk3skvahu","title":null},"content":[{"type":"text","text":"《Heroku 的得與失》","attrs":{}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"總體來看 ,抽象程度越高,最終用戶的使用體驗就越好,但平臺的可擴展性就越差,就越不可能在平臺團隊中得到採納,這就解釋了今天 Kubernetes 的被採納度,遠遠高於 Heroku。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" ","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"抽象和應用本身的標準化模型缺一不可","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"那麼是不是把 K8s 的對象保留,通過構建模板的方式把抽象做出來,然後用戶只需要填模板需要渲染的值\b,就好用了呢? 答案並非如此,","attrs":{}},{"type":"link","attrs":{"href":"https://helm.sh/","title":null},"content":[{"type":"text","text":"Helm","attrs":{}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 就是這種模式的典型代表。 雖然 helm 也受到了廣泛的歡迎,但是它的問題也客觀存在:","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong","attrs":{}}],"text":" ","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"狀態迴流:helm 把對象下發下去以後,並沒有統一的狀態迴流機制。舉例來說,你把 Deployment 和 Service 組裝成一個 helm 應用安裝下去後,那麼 Deployment 和 Service 是否創建成功了,有沒有出錯這些狀態,helm 是無法直接查看這些細節的,你又得下沉到 K8s 這些概念裏去。尤其是當這些對象不是 K8s 內置對象的時候,這個問題就更爲明顯了。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"註冊、發現、衝突管理:除了圍繞擴展的能力如何使用,能力的註冊與發現同樣重要。如何讓用戶自助式的註冊以及發現平臺層的能力,以及這些能力哪些是互補、哪些是衝突的,也是應用管理需要解決的問題。而 helm 的定位本質上是一個打包部署的工具,並沒有考慮這方面的問題。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"發佈與版本化:雖然 helm 也有版本的概念,也有 upgrade 命令,但是熟悉 helm 的朋友一定知道 helm 的升級其實是違背發佈原則的,因爲它完全沒有任何的灰度,而是直接粗暴的使用新版本替代老版本。一個生產級別的應用管理平臺一定是有着豐富的發佈策略保證穩定可靠的平臺。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"通用能力建設:除了發佈以外,其實還有許多能力是通用的,比如彈性擴縮、訪問路由、日誌監控等等,這些通用能力圍繞 helm 模式也很難建設。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"答案","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"KubeCon NA 正式發佈的 ","attrs":{}},{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/LauydAy1ngcDuZ3lhqrL6Q","title":""},"content":[{"type":"text","text":"KubeVela","attrs":{}}]},{"type":"text","text":"項目(github地址:http://kubevela.io/)","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 正是基於上述這些問題的探索得到的答案,","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong","attrs":{}}],"text":"一個簡單易用且高度可擴展的應用管理平臺與核心引擎","attrs":{}},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章