雲原生領域再添重磅開源項目：Clusternet、Caelus正式開源

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Kubernetes多集羣管理項目Clusternet正式開源"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"11月4日，在"},{"type":"link","attrs":{"href":"https:\/\/des.cloud.tencent.com\/2021\/\/?utm_source=baidu&utm_medium=sem-pc&utm_campaign=%E5%AA%92%E4%BB%8B%E5%B9%BF%E5%91%8A&CampaignType=%E5%B9%BF%E5%91%8A%E8%90%A5%E9%94%80#\/","title":"xxx","type":null},"content":[{"type":"text","text":"2021騰訊數字生態大會"}]},{"type":"text","text":"上，騰訊宣佈了雲原生領域一項重磅開源進展—— Kubernetes多集羣管理項目 Clusternet 正式開源。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"據瞭解，Clusternet 由騰訊聯合多點生活、QQ音樂、富途證券、微衆銀行、酷狗音樂、三七互娛等共同發起，專注Kubernetes多集羣管理和應用治理方向。作爲未來分佈式雲的技術基石，Clusternet 通過組件化方式擴展 Kubernetes，將 Kubernetes 強大的集羣、應用和服務能力擴展至分佈式雲，兼容所有 Kubernetes 生態資源和軟件。騰訊在本次大會中發佈的騰訊雲原生分佈式雲中心 TDCC 產品正是以 Clusternet 作爲核心技術。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過 Clusternet，無論集羣是運行在公有云、私有云、混合雲還是邊緣雲上，都可以獲得統一的管理和一致的訪問體驗，實現 Kubernetes API 集中部署和多集羣的應用程序和服務的協同調度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"具體來說，Clusternet 具備以下特點："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最輕量化架構。採用了 Kubernetes  AA （Aggregated APIServer）方式部署，不依賴額外的存儲和端口，無需額外學習和維護不同平臺軟件和系統的差異，大大降低了運維複雜度。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一站式連接各類集羣。同時支持 Pull 和 Push 模式管理 Kubernetes 集羣，首創 Dual 模式自動化一站式管理海量集羣。即使在無專網通道的情況下，仍可實現跨集羣路由訪問。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"零成本升級多雲架構。原有單雲單集羣架構業務，可以零學習和改造成本升級至多雲多集羣架構，完全兼容 Kubernetes 的標準 API、Helm Chart 以及自定義的 CRD，無需額外學習複雜的多集羣 API。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"豐富、靈活的策略配置。提供了多種類型的配置策略，包括分發策略、差異化策略等，通過靈活的搭配即可滿足複雜的業務場景，減少人工干預和重複勞動。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"便捷接入。Clusternet 提供了完善的對接能力，支持 kubectl plugin 以及 client-go，方便業務一鍵接入，具備管理多集羣的能力。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"大數據全場景在離線混部系統Caelus正式開源"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在2021騰訊數字生態大會上，騰訊還正式開源了全場景在離線混部系統"},{"type":"link","attrs":{"href":"https:\/\/github.com\/Tencent\/Caelus","title":"xxx","type":null},"content":[{"type":"text","text":"Caelus"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Caelus 由騰訊大數據團隊聯合騰訊多個業務部門共同研發，旨在解決大數據資源缺口問題，通過Caelus在離線混部，擴充大數據任務可用資源，發揮空閒資源更大價值。據介紹，在騰訊內部，Caelus混部方案已經被大規模應用到廣告、存儲、大數據、機器學習等多個業務，平均提升30% 資源利用率，節省了上億成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，在線作業容器化已成爲主流，但絕大公司的在線業務還有相當大未容器化的存量，或有些業務（如存儲類服務）不適合容器化，這些沒有經過容器化的在線應用，沒有很好的資源隔離方式，但資源卻非常可觀，因此Caelus通過以後置容器的方式來管理在線應用，將這部分資源也充分利用起來，也減少了混部在很多公司、很多場景的限制。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於離線任務，Caelus可以支持大數據任務，也可以支持任何非大數據類的任務。但由於大數據任務佔到了離線任務的極高比例，並且離線大數據任務資源碎片化特點比較明顯，而大數據又有很多是Hadoop的生態，因此騰訊做了很多的兼容工作。同時，也看到有很多公司已經在雲原生大數據領域有了比較不錯的進展，對於使用Caelus來做混部更加自然。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Caelus挖掘的是在線業務的空閒資源，混部要優先保障在線服務的正常運行，當在線業務需要資源時，大數據業務要及時歸還資源。Caelus內部集成了指標收集、資源畫像、資源隔離、干擾檢測、離線驅逐等模塊，從調度、隔離、檢測等方面全方位保障在線的服務質量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於資源隔離，Caelus採用全維度彈性資源隔離，包括CPU、內存、磁盤IO、磁盤空間、網絡IO等。同時可以配合騰訊OS版本，進一步提升資源隔離效果。在干擾檢測方面，通過RDT、perf、eBPF等技術深度採集硬件、內核及業務相關指標數據，通過豐富的異常檢測算法，判斷在線服務是否受到干擾。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，Caelus 已經在騰訊內部經過長時間規模化的考驗，未來還將在更多網絡模式下的容器熱遷移、對機器學習等長時間運行的離線任務混部質量提升等方面進行持續迭代優化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"騰訊公司副總裁王巨宏表示，隨着雲計算和雲原生的發展，也逐漸形成了"},{"type":"link","attrs":{"href":"https:\/\/www.infoq.cn\/news\/Ufv0MCPgfxFAWWctsXMS","title":"xxx","type":null},"content":[{"type":"text","text":"雲開源生態"}]},{"type":"text","text":"，雲則爲開源的發展提供了最佳的載體。騰訊雲也致力於通過雲原生開源項目，與行業共同探索開源的社會價值和商業價值。“我們希望未來不管是一個人，還是三五個人的小團隊，或者是大多數中小企業，都能用上和大廠一樣的框架和技術，進而助力整個行業技術的創新和進步。”"}]}]}