是的，我們不用 Kubernetes

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我們看來，目前許多公司全力投入Kubernetes都是沒有意義的，但選擇權在他們。如果你讀到了這篇文章，而且你所在的組織目前正在設法確定自己有多需要Kubernetes，那麼我希望本文的觀點可以幫助你的團隊做出正確的決定。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#333333","name":"user"}},{"type":"strong"}],"text":"本文最初發佈於Ably工程博客，由InfoQ中文站翻譯並分享。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"前言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Ably，我們運行着一個大規模的生產基礎設施，支撐着 Ably 全球客戶的實時消息應用程序。和大多數科技公司一樣，這個基礎設施在很大程度上是基於軟件的；這些軟件大部分是部署並運行在 Docker 容器上的，這也和大多數科技公司一樣。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你一直在關注技術領域，那麼你可能會想到，下面這個問題經常會有人問："}]},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“那麼……你們用 Kubernetes 嗎？”"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們當前的客戶，我們潛在的客戶，對我們平臺感興趣的開發人員，應聘"},{"type":"link","attrs":{"href":"https:\/\/jobs.ably.com\/","title":null,"type":null},"content":[{"type":"text","text":"Ably 職位"}]},{"type":"text","text":"的人，都問過這個問題。我們甚至還碰到過一些有趣的應聘者，他們放棄我們提供的職位，理由是我們沒有使用 Kubernetes。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在規劃基礎設施長期路線圖時，我們也問過自己這個問題：我們應該在某個時候將 Kubernetes 作爲主要的部署平臺嗎？"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"爲什麼選擇Kubernetes？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/kubernetes.io\/","title":null,"type":null},"content":[{"type":"text","text":"Kubernetes"}],"marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}],"text":" "},{"type":"text","text":"是最知名的容器化軟件大規模編排系統。雖然"},{"type":"link","attrs":{"href":"https:\/\/www.lastweekinaws.com\/podcast\/aws-morning-brief\/kubernetes-is-the-most-expensive-way-to-run-a-service\/","title":null,"type":null},"content":[{"type":"text","text":"存在不同的聲音"}]},{"type":"text","text":"，但在很大程度上，它仍然處在炒作週期的頂峯——你經常會覺得每個人都在 Kubernetes 上運行軟件，或者至少最近聽人說，他們希望將所有東西都移到 Kubernetes 上。"},{"type":"link","attrs":{"href":"https:\/\/mesos.apache.org\/","title":"xxx","type":null},"content":[{"type":"text","text":"Mesos"}]},{"type":"text","text":" 的流行度迅速降低，"},{"type":"link","attrs":{"href":"https:\/\/thenewstack.io\/kubernetes-vs-docker-swarm-whats-the-difference\/","title":null,"type":null},"content":[{"type":"text","text":"Docker Swarm"}],"marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}],"text":" "},{"type":"text","text":"也是少說爲妙，而如果你還在往普通的 EC2 實例上部署，那不如索性用穿孔卡得了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Kubernetes 有很多優點。它是由一些具有大規模生產負載經驗的工程師開發的，而且是在現實世界中經過谷歌 Borg 編排系統證明了的技術。它獲得了供應商持續不斷的支持，現如今，你不會發布一款軟件產品而不提供官方 Docker 鏡像和 Kubernetes 部署指南。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多少有點令人驚訝的是，Kubernetes 從其靈感來源 Borg 繼承下來的東西已經所剩無幾，即將一個大型裸金屬機器池變成一個私有云環境。現如今的大多數 Kubernetes 部署似乎都是在虛擬機上，而這些虛擬機多數時候是在公有云提供商那。公有云是現在許多組織部署 Kubernetes 的動因；它被視爲一個統一的 API 層，使多雲部署對 DevOps 人員透明。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即使在一朵雲中，這種 Kubernetes 開發體驗也經常被提及。開發人員已經習慣使用 Docker 了，而 Kubernetes 可以讓相同的容器輕鬆在生產環境中運行。此外，藉助命名空間、內置資源管理以及內置的虛擬網絡特性，或服務網格，或兩者兼而有之，它還允許部署由不同團隊維護的多個交互式任務，並且彼此互不影響。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Ably 用什麼來代替 Kubernetes？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Ably 是一個公有云客戶。我們的整個生產環境都在"},{"type":"link","attrs":{"href":"https:\/\/aws.amazon.com\/","title":null,"type":null},"content":[{"type":"text","text":"AWS"}]},{"type":"text","text":" 上，目前沒有其他地方。我們運行在 EC2 實例上。一天之中，機器總數是自動變化的，但不管怎麼變化，總數至少也有幾千臺，分佈在 10 個 AWS 區域。這些機器確實是在運行 Docker，我們的大部分軟件也是部署在容器中的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/0b\/8d\/0b0a8c8e869eca9e910f500fe711ef8d.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們沒有使用任何知名的運行時編排層。在創建時，根據所在的"},{"type":"link","attrs":{"href":"https:\/\/docs.aws.amazon.com\/autoscaling\/ec2\/userguide\/","title":null,"type":null},"content":[{"type":"text","text":"自動縮放組"}]},{"type":"text","text":"，每個實例就已經知道自己運行哪個容器。每個實例上都有一個小小的自定義引導服務，這是引導鏡像的一部分，它會查找實例配置，拉取合適的容器鏡像，並啓動容器。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"容器集在實例的生命週期裏都不會變。不會有一個調度服務將一個實例從“核心”轉成“前端”或其他類型的實例：要改變集羣的結構，實例會整體創建或銷燬，而不是讓它們運行一組不同的容器。每個實例上都有輕量級的監控服務，如果一個容器死掉，它會將其復活，而如果一個實例運行了任何集羣不再需要的軟件版本，那麼它會將其終止。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於進入的流量，我們使用"},{"type":"link","attrs":{"href":"https:\/\/ably.com\/blog\/limits-aws-network-load-balancers","title":null,"type":null},"content":[{"type":"text","text":"AWS 網絡負載均衡器"}],"marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}]},{"type":"text","text":"。由於一個縮放組對應一個生產服務，所以我們可以使用常規的 AWS 方法，將一個 NLB 指定到一個縮放組作爲目標組，不需要額外的抽象層。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於內部服務間通信，或者是作爲“服務網格”（你願意的話），我們使用……網絡。由於機器上的服務不是任意混在一起的，所以機器自動分配IP地址對於我們來說已經完全足夠了。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"不用 Kubernetes 爲什麼還要用 Docker？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不管怎樣，Docker 仍然是一種非常方便的軟件部署格式，尤其是在使用依賴項很多的語言（Node、Python、Ruby……）編寫時。在這些情況下，可部署單元是一個複雜目錄樹下的成千上萬且相互依賴的文件，再加上一個執行運行時，而且該運行時的版本必須是與源樹快照相對應的版本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們以前是通過簡單的tar包分發軟件構建（和 Heroku 一樣，我們稱之爲 slug），每個實例上的管理服務會下載並解壓。從功能上講，我們現在做的還是一樣的事，因爲實際上，Docker 鏡像只是"},{"type":"link","attrs":{"href":"https:\/\/github.com\/opencontainers\/image-spec\/blob\/master\/layer.md","title":"xxx","type":null},"content":[{"type":"text","text":"一組 tar 包"}]},{"type":"text","text":"和與之綁定的 JSON blob 元數據，但 "},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0a0a0a","name":"user"}},{"type":"strong"}],"text":"curl "},{"type":"text","text":"和 "},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0a0a0a","name":"user"}},{"type":"strong"}],"text":"tar "},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0a0a0a","name":"user"}}],"text":"被 "},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0a0a0a","name":"user"}},{"type":"strong"}],"text":"docker pull "},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0a0a0a","name":"user"}}],"text":"所取代。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（並不是工程團隊中的每個人都認爲這是一項改進，但關於這一點，我們會在後續的博文中討論。）"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"這種設置夠靈活嗎？"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"資源管理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在資源管理方面，我們可以根據服務的需求確定要使用的 "},{"type":"link","attrs":{"href":"https:\/\/instances.vantage.sh\/","title":"xxx","type":null},"content":[{"type":"text","text":"EC2 實例類型"}]},{"type":"text","text":"。我們不需要知道如何將較小的服務打包到較大的實例上。關於如何將小型VM打包到大型物理機上，亞馬遜的經驗至少比我們多10年，因此，還是由他們處理那些細節吧。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這就是"},{"type":"link","attrs":{"href":"https:\/\/aws.amazon.com\/aws-cost-management\/aws-cost-optimization\/right-sizing\/","title":null,"type":null},"content":[{"type":"text","text":"Right Sizing"}],"marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}]},{"type":"text","text":"：大多數服務只能有效使用一定數量的資源。一個有兩個線程的進程不需要 16 顆 CPU；一個一分鐘寫一次盤的進程不需要每秒可以完成 9 萬次寫操作的 SSD 存儲；亞馬遜提供的 "},{"type":"link","attrs":{"href":"https:\/\/aws.amazon.com\/ec2\/graviton\/","title":"xxx","type":null},"content":[{"type":"text","text":"CPU 架構選項"}]},{"type":"text","text":"可以帶來最高性價比。從衆多 AWS 服務中選擇合適的組件可以幫助我們儘可能縮減開銷，控制成本，最終降低客戶的每條消息費率。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"自動縮放"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"EC2 實例組知道如何自動增加或減少組中實例的數量來滿足需求。可用的工具和 Kubernetes 類似。顯然，我們針對 AWS 所做的設計並不能直接在其他雲提供商那裏使用，但那時我們沒有使用任何其他雲提供商。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然，我們還是可以手動管理容量。每個縮放組中的期望實例數隨時都可以手動設置，設置完成後，自動縮放策略會再次接管這項工作，根據系統負載增加和減少實例數量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們向客戶收費是根據他們實際使用服務的情況。任何針對冗餘容量的預算都要我們自掏腰包，因此，我們要儘可能地提高資源利用率，同時又要爲我們的客戶提供良好的服務級別，即使是在出現意外峯值負載的情況下。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"流量入口"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果撇開"},{"type":"link","attrs":{"href":"https:\/\/ably.com\/blog\/limits-aws-network-load-balancers","title":"xxx","type":null},"content":[{"type":"text","text":"我們遇到的實現 Bug"}]},{"type":"text","text":" 不談，那麼流量入口在所有主流雲提供商那裏都是一個已經解決了的問題 。也就是說，你可以持續地將接收外部流量的服務映射到運行該服務的機器集上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"客戶流量既可以直接到達 NLB，也可以先繞道 CloudFront。在運行在每個區域中的服務看來，這沒什麼區別。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"無論何種情況，在每個區域中都使用一個負載均衡器是實現"},{"type":"link","attrs":{"href":"https:\/\/ably.com\/resources\/datasheets\/using-ably-at-scale","title":null,"type":null},"content":[{"type":"text","text":"橫向可擴展性"}]},{"type":"text","text":"和靈活性的另一種方法——可以應對連接數出現重大變化的情況，如"},{"type":"link","attrs":{"href":"https:\/\/ably.com\/case-studies\/tennis-australia","title":null,"type":null},"content":[{"type":"text","text":"大型體育賽事開幕"}]},{"type":"text","text":"時吸引了數十萬甚至更多的觀衆。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"DevOps"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了確保我們的工程師可以有意義地參與到我們生產系統的管理，開發人員可以設置一個配置值——“該集羣現在可以運行這個組件的"},{"type":"text","marks":[{"type":"italic"}],"text":"這個"},{"type":"text","text":"版本，謝謝”——隨着時間推移，該服務的所有實例都將被運行新版本的實例所代替。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"使用 Kubernetes 的話，情況如何？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果將生產環境遷移到 Kubernetes 上，我們的環境和進程會有什麼變化（希望是有所改進）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於我們的基礎設施團隊規模有限，所以唯一值得考慮的選項就是"},{"type":"link","attrs":{"href":"https:\/\/docs.aws.amazon.com\/eks\/latest\/userguide\/what-is-eks.html","title":"xxx","type":null},"content":[{"type":"text","text":"與 AWS 完全集成的託管 Kubernetes"}]},{"type":"text","text":"。我們的產品需要全球部署，我們至少需要 10 個集羣（每個區域一個）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/43\/44\/439cb66d283a385f6e1eb2dffb8ee744.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"資源管理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當通過 Kubernetes 管理資源時，我們不是針對每個服務使用大小適中的 EC2 實例，而是使用大實例（"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0a0a0a","name":"user"}},{"type":"strong"}],"text":".metal "},{"type":"text","marks":[{"type":"color","attrs":{"color":"#0a0a0a","name":"user"}}],"text":"類的一種，"},{"type":"text","text":"也許是最大的），並將它們用容器打包。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"打包服務器有個小優點，就是可以使用現有服務器上的空閒資源，而不必爲資源需求不大的服務額外分配機器。但這也有一個很大的缺點，就是在同一臺機器上運行異構服務，導致資源競爭。這不是什麼新問題：雲提供商在使用虛擬機時也面臨同樣的問題，即“吵鬧的鄰居”。不過，雲提供商的系統中有 10 幾年積累下的祕密武器，可以在最大程度上爲客戶減輕這個問題。但在 Kubernetes 上，你得自己解決所有問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一種可能的方法是設法在使用 Kubernetes 時保留“一個虛擬機，一個服務“模型。Kubernetes minions 不必都相同，它們可以是不同大小的虛擬機，Kubernetes 的調度條件可以確保每個 minion 只運行一個邏輯服務。這樣就有一個問題：如果要在特定的 EC2 實例組上運行特定的容器集，爲什麼還要有一個 Kubernetes 層，而不直接那樣做？"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"自動縮放"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在使用 Kubernetes 時，服務的自動縮放看上去也很類似："},{"type":"link","attrs":{"href":"https:\/\/kubernetes.io\/docs\/tasks\/run-application\/horizontal-pod-autoscale\/#support-for-custom-metrics","title":"xxx","type":null},"content":[{"type":"text","text":"暴露一個自定義的“當前使用率”指標"}]},{"type":"text","text":"，然後設置"},{"type":"link","attrs":{"href":"https:\/\/kubernetes.io\/docs\/tasks\/run-application\/horizontal-pod-autoscale\/#scaling-policies","title":null,"type":null},"content":[{"type":"text","text":"規則"}]},{"type":"text","text":"根據需要增加或移除容器。當然，只有在集羣節點上還有容量時，Kubernetes 集羣才能啓動額外的服務 pod。這樣說來，我們在部署時就需要"},{"type":"link","attrs":{"href":"https:\/\/www.itpro.co.uk\/development\/containers\/360052\/kubernetes-costs-spiralling-as-businesses-fail-to-monitor-spend","title":"xxx","type":null},"content":[{"type":"text","text":"留下相當數量的空閒容量"}]},{"type":"text","text":"，並添加"},{"type":"link","attrs":{"href":"https:\/\/docs.aws.amazon.com\/eks\/latest\/userguide\/cluster-autoscaler.html","title":null,"type":null},"content":[{"type":"text","text":"Cluster Autoscaler"}],"marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}],"text":" "},{"type":"text","text":"來根據需要增加節點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於 Cluster Autoscaler 來說，向上擴展集羣相當簡單——“當空閒容量低於期望值時增加節點”。不過，收縮集羣就比較複雜了：可能你最終找到了最空閒的節點，但不是空節點。這就需要把剩餘的 pod 遷移到其他節點上，騰空這個節點後再停止該節點，完成集羣收縮。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關於自動縮放，我們的結論是，工作方式和現在類似，但我們要解決兩個自動縮放問題，而不是一個，而且這兩個問題都比我們現在要解決的問題更復雜。按需或按計劃容量管理（需人工干預）都更復雜一點，因爲我們必須首先保證有足夠的 Kubernetes 節點，只有那樣纔有足夠的 pod 來提供所需的服務。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"流量入口"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用 Kubernetes 時，流量入口就非常簡單了。EKS 團隊提供了一些非常值得稱道的設計選項：如果集羣那樣配置，"},{"type":"link","attrs":{"href":"https:\/\/docs.aws.amazon.com\/eks\/latest\/userguide\/pod-networking.html","title":"xxx","type":null},"content":[{"type":"text","text":"每個 pod 都會收到一個 AWS 託管的 IP 地址"}]},{"type":"text","text":"，該地址與 EC2 的虛擬網絡層 VPC 完全集成。在集羣內運行的東西可以直接訪問這些 IP，運行在 AWS 上同一個 VPC 族內但不在集羣裏的東西，則需要通過 AWS 提供的兩種虛擬負載均衡器訪問。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當在服務的 Kubernetes 規範中添加 Ingress 或 Service 小節時，有一個"},{"type":"link","attrs":{"href":"https:\/\/docs.aws.amazon.com\/eks\/latest\/userguide\/aws-load-balancer-controller.html","title":null,"type":null},"content":[{"type":"text","text":"控制器"}]},{"type":"text","text":"會自動創建 AWS 負載均衡器，並將它們直接指向對應的 pod 集。總的來說，這不會比我們現在暴露流量路由實例的方式更復雜。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然，這裏有一個隱藏的缺點，就是這種出色的集成完全是 AWS 特有的。因此，如果你想在多雲環境下使用 Kubernetes，這就不是很有幫助了。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"DevOps"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Kubernetes 的世界中，軟件版本管理和我們現在的方式非常類似。現在，我們是在自定義配置系統中指定一個新的目標版本，所有 EC2 實例會自動滾動替換，使用 Kubernetes 的話，我們需要在"},{"type":"link","attrs":{"href":"https:\/\/kubernetes.io\/docs\/concepts\/workloads\/controllers\/deployment\/#updating-a-deployment","title":null,"type":null},"content":[{"type":"text","text":"Kubernetes Deployment"}],"marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}],"text":" "},{"type":"text","text":"中指定一個新的目標版本，然後 pod 會滾動替換。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"但是還有其他好處嗎？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"總的來說，我們做的事基本不變，但做法更復雜了。在探討如何移植現有的基礎設施時，如果在 Kubernetes 上運行能提供其他我們沒有考慮到的好處，那或許值得這樣做。讓我們看下人們經常提到的 Kubernetes 的其他優點，看看它們對我們是否有幫助。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"多雲就緒"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/www.gartner.com\/smarterwithgartner\/why-organizations-choose-a-multicloud-strategy\/","title":null,"type":null},"content":[{"type":"text","text":"每個人都應該有一個多雲戰略！"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/www.lastweekinaws.com\/blog\/multi-cloud-is-the-worst-practice\/","title":null,"type":null},"content":[{"type":"text","text":"除非他們不應該那樣做。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們發現自己屬於“不應該”的那部分，當前也就沒有執行這樣的戰略。我們並不是完全贊成，Kubernetes 是達成這一戰略的好方法。當服務需要訪問集羣外的服務時，或者需要從集羣外訪問時，或者使用某種持久化存儲時，不同的主流雲提供商之間確實存在差別，而 Kubernetes 並不足以完全屏蔽這種差異，讓開發完全不用考慮。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你仍然需要一個 Kubernetes on AWS 部署策略，以及另一個相似但不同的 Kubernetes on GCP. 部署策略，那麼與不使用 Kubernetes 相比，採用兩個相似但不同的 AWS 和 GCP 部署策略不是會困難許多嗎？"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"混合雲就緒"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我們看來，管理混合雲或內部環境（或者換個說法，管理你自己的物理機）是一個部署 Kubernetes 的有效理由。無獨有偶，這也是 Borg 的設計初衷。如果我們已經計劃好建設自己的物理數據中心，而不是購買公有云資源，我們將在其中安裝的幾乎可以肯定是裸機 Kubernetes 集羣。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"還有一個選項是設法基於虛擬機構建私有云，許多組織選擇了這種方式。不過，根據已有的經驗，構建那種環境絕不是一個實惠或簡單的選項。我們還沒有準備好創建自己的數據中心。要等到我們的業務翻幾番，擁有自己的硬件所帶來的好處值得我們在工程部門保有一個物理基礎設施小組。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"基礎設施即代碼"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基礎設施即代碼是我們已經在做的工作，用了"},{"type":"link","attrs":{"href":"https:\/\/aws.amazon.com\/cloudformation\/getting-started\/","title":null,"type":null},"content":[{"type":"text","text":"CloudFormation"}],"marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}],"text":" "},{"type":"text","text":"和"},{"type":"link","attrs":{"href":"https:\/\/www.terraform.io\/intro\/index.html","title":null,"type":null},"content":[{"type":"text","text":"Terraform"}],"marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#50b4f3","name":"user"}}],"text":" "},{"type":"text","text":"以及我們自定義的工具。編寫 Kubernetes YAML 文件並不是管理“基礎設施即代碼”的唯一方式，在許多情況下，也不是最合適的方式。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"有一個很大而且很活躍的社區"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用 Kubernetes，人們常說的一個好處是，有一個很大的用戶社區可以分享問題和建議。但是，並不是運行了 Kubernetes 才能參與到這個由用戶和開發者組成的龐大社區。雲計算還涉及許多其他的方面，我們積極參與了其中許多社區。AWS 用戶的技術社區比 Kubernetes 開發者和用戶社區更大，也有相當大的重疊。我們部署的許多其他技術，如 "},{"type":"link","attrs":{"href":"https:\/\/cassandra.apache.org\/","title":"xxx","type":null},"content":[{"type":"text","text":"Cassandra"}]},{"type":"text","text":"，也非常流行。我們並沒有因爲身處 Kubernetes 社區之外而感到孤單。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個讓人更不願意承認的事實是，在許多情況下，產品龐大而且發展迅速的用戶社區並沒有什麼實際的幫助。Ably SRE 有多名工程師之前曾在 Kubernetes 團隊工作，他們發現，數量龐大的初學者使得他們很難就 Kubernetes 相關的問題找到有用的信息。許多人使用 Kubernetes，但根據我們的經驗，很少有人對它有深入的瞭解，因此，對於這樣一個擁有龐大用戶羣體的東西，通過社區解決問題比你想象的要難得多。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大量的供應商湧入這一領域，造成了嚴重的技術攪動，爲了適應新出現的第三方插件，Kubernetes 的核心特性頻繁增加或變化。一個龐大而活躍的社區既是一種祝福，也是一種詛咒。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Kubernetes 的附加成本"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/www.theregister.com\/2021\/02\/25\/google_kubernetes_autopilot\/","title":null,"type":null},"content":[{"type":"text","text":"複雜性"}]},{"type":"text","text":"。遷到 Kubernetes，組織需要一個完整的工程團隊才能保證 Kubernetes 集羣的運行，這還是假設使用的是託管 Kubernetes 服務，而且要有其他基礎設施團隊工程師維護上面的其他支持服務以及組織實際的產品或服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如，上面提到的那些做得很好的 AWS EKS 網絡和流量入口服務。那不是 EKS 自帶的。你需要創建一個 EKS 集羣，然後在上面安裝並配置那些服務。然後還有"},{"type":"link","attrs":{"href":"https:\/\/docs.aws.amazon.com\/eks\/latest\/userguide\/managing-coredns.html","title":"xxx","type":null},"content":[{"type":"text","text":"一些其他的服務"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果我們沿着服務提供商提供的道路走下去，那顯然可以減輕我們的工作，但那是有代價的。我們前面提到過這樣一個生機勃勃的市場所造成的技術攪動，因此，"},{"type":"link","attrs":{"href":"https:\/\/www.itprotoday.com\/hybrid-cloud\/8-problems-kubernetes-architecture","title":"xxx","type":null},"content":[{"type":"text","text":"從一個供應商轉到另一個供應商遠非易事"}]},{"type":"text","text":"。選擇供應商並不是一件輕巧的事，那本身就是一種架構承諾。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這個問題並不是只有 AWS 提供的託管 Kubernetes 纔有——整個行業都是如此。重點不是 EKS 團隊的工作不夠出色，而是說，即使真有一個維護得很好的託管 Kubernetes 設置，也仍然需要做很多工作才能將其從一個基本的集羣變成一個可以可靠運行服務的生產環境。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"小結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雖然我們知道 Kubernetes 是一個設計得很好的產品，但我們目前沒有使用它，或者說沒有計劃使用它。不是說部署 Kubernetes 沒意義，只是說到目前爲止，對我們來說沒意義。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在經過仔細的成本收益評估後，我們發現，引入這樣一個成本高昂的組件並不能真正解決我們的問題，而只是將問題轉移到了其他地方。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在我們看來，目前許多公司全力投入 Kubernetes 都是沒有意義的，但選擇權在他們。如果你讀到了這篇文章，而且你所在的組織目前正在設法確定自己有多需要 Kubernetes，那麼我希望自己的觀點可以幫助你的團隊做出正確的決定。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"和其他任何組織的系統一樣，我們的生產系統無疑也存在問題和技術債務。我們有一個改進清單，我們也會額外"},{"type":"link","attrs":{"href":"https:\/\/jobs.ably.com\/","title":null,"type":null},"content":[{"type":"text","text":"招聘工程師"}]},{"type":"text","text":"來幫助我們實現這些改進。不過，歸根結底，“遷移到 Kubernetes ”不在這份清單上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"查看英文原文："},{"type":"link","attrs":{"href":"https:\/\/ably.com\/blog\/no-we-dont-use-kubernetes","title":null,"type":null},"content":[{"type":"text","text":"No, we don’t use Kubernetes"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}