硬核干货|使用GitLab CI部署Rancher集群

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在当今瞬息万变的DevOps世界中，遵循最佳实践至关重要。这些最佳实践涉及安全性、访问控制、资源限制等方面。在DevOps中最为重要的事情之一是持续集成（CI）和持续交付（CD）。而且对于一个有效部署来说，持续集成是极为关键的部分。但是在集成的过程中我们总是一次又一次地重复手动步骤——尤其是在节点配置方面。此时，我们需要“万物自动化”的思维方式来保证我们工作的正常运转，以便我们可以高效执行并确保我们的应用程序得以有效部署和运行。通过GitLab CI\/CD，你会获得一个对用户友好的UI，它可以配置构建（build）并根据需要对其进行自定义。它还包括了设置流水线触发器、构建变量、license合规性等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从统一的控制台查看构建步骤极为有益，特别当你正在试图排除构建故障时。每个构建步骤也会显示运行命令的CLI输出。这可以让你从一个视角了解构建过程中发生的事情，而无需SSH进入runner节点。CI\/CD工具通常与构建文件一起工作，它决定了构建步骤。当使用GitLab CI\/CD时，构建文件被称为.gitlab-ci.yaml。在本文中，你将会了解到构建文件的组合方式及其作用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GitLab CI工具如何与AWS进行通信以触发新资源的启动是我们部署的另一个重要部分。我们的部署还包括Terraform、RKE和Rancher2。主要目标是产生一个按需部署和销毁基础设施的流水线。最终结果是我们可以通过点击一个按钮来触发，或者用一条（或两条）CLI命令来获得一个高可用的、一致的部署。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"你可以访问以下链接查看本文的源代码："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"underline"}],"text":"https:\/\/gitlab.com\/iby.autometa\/rancher-deploy"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"部署流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在这个部署中每个组件都有其特定的目的，部署的目标是按照安全、低成本和高可用的最佳实践以部署所需的最少资源。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是首先我们要了解CI\/CD流水线是什么？从概念上来说，CI\/CD流水线应该有3个阶段——source、build和deploy："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Source："},{"type":"text","text":"每个部署都需要一个代码管理工具，常见的工具包括Github和GitLab。Bitbucket也是一个不错的选择。在本文的场景中，我们选择GitLab，因为它除了作为我们的源码管理工具之外，还可以提供内置的CI\/CD功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Build："},{"type":"text","text":"在构建文件.gitlab-ci.yaml中提到的步骤（stages）将定义构建步骤。在这个阶段中，GitLab平台将验证代码并运行一个terraform plan。在各个步骤中，可以传递命令、设置变量、构建Docker镜像、创建文件等。这使得我们可以将步骤解耦，也就是说如果我们选择移除步骤或添加新的步骤将更加容易。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Deploy："},{"type":"text","text":"在这一步骤中，有两个手动操作。我们采取的第一个手动操作是deploy。这个选项会使用Terraform代码启动基础设施的创建。一旦执行了这个手动步骤，GitLab就会联系到AWS，用访问权限和秘钥进行认证，并开始将基础设施部署到公有云中（本例为AWS）。另一个发挥重要作用的组件是 provider.tf 文件。这个文件定义了部署的云提供程序。我们的第二个手动选项是destroy。就像deploy一样，它是手动触发的。在某些情况下此步骤可以自动化，但在大多数情况下，我们在执行部署或销毁部署时都要小心谨慎。同时建议执行这些步骤时限制访问权限，因为安全的最佳实践包括使用用户数据库，并为这些手动步骤的执行申请权限。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"基础设施图解"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此图展示了此部署中使用的所有工具及其在本次部署中提供的功能："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"GitLab：代码管理和CI\/CD"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AWS：弹性计算机云(EC2)、简单存储服务(S3)、Route 53(R53)、安全组、弹性负载均衡(ELB)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"S3：在本次部署中，我们需要手动创建S3 bucket。在你开始你的部署之前，确保你已经创建你的bucket并在变量部分指定了它。S3 bucket将维护terraform.tfstate文件。如果你想了解更多关于管理Terraform状态欢迎查阅以下链接："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"underline"}],"text":"https:\/\/www.terraform.io\/docs\/state\/index.html"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Terraform：基础设施即代码（IaC）；RKE提供程序-允许配置Kubernetes集群；Rancher 2.x提供程序-允许从Terraform代码中配置Rancher管理的集群；Helm提供程序-可以安装Helm chart并最终在创建的基础设施上安装Rancher。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/14\/14fc2bfec2bd0a1f27a3251bb539f77b.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"CI\/CD流水线如何工作？"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"CI\/CD 的步骤"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"构建文件的第一部分包括我们将在部署中需要执行的阶段："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nstages:\n - validate\n - plan_before_apply\n - apply\n - destroy"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Before Script"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"before script为这次部署的成功奠定了基础，在这个过程中会创建两个文件："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、backend.ft：这个文件将负责存放ftstate的s3 bucket。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\n - |\n cat < backend.tf\n terraform {\n backend \"s3\" {\n bucket = \"$BUCKET_NAME\"\n key = \"$BUCKET_KEY\"\n region = \"us-east-1\"\n encrypt = true\n }\n }\n EOF"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2、variables.tf：这个文件将保存证书、VPC、K8S版本等。这些参数是从GitLab dashboard的settings部分传递过来的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\n cat < variables.tf\n variable \"aws_access_keys\" {\n type = map(string)\n description = \"AWS Access Keys for terraform deployment\"\n\n default = {\n access_key = \"$AWS_ACCESS_KEY_ID\"\n secret_key = \"$AWS_SECRET_ACCESS_KEY\"\n region = \"us-east-1\"\n }\n }\n variable \"number_of_nodes\" {"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"构建阶段"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1、 "},{"type":"text","marks":[{"type":"strong"}],"text":"Validate："},{"type":"text","text":"将验证工作目录下的配置文件"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\n validate:\n stage: validate\n script:\n - terraform validate"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2、 plan_before_apply：将运行terraform plan并创建一个执行计划（execution plan）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"plan_before_apply:\nstage: plan_before_apply\nscript:\n - terraform plan\ndependencies:\n - validate"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3、 Apply：将运行terraform apply并执行该计划。这是一个手动步骤。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"apply:\nstage: apply\nscript:\n - apk update && apk add curl git\n - curl -LO https:\/\/storage.googleapis.com\/kubernetes-release\/release\/`curl -s https:\/\/storage.googleapis.com\/kubernetes-release\/release\/stable.txt`\/bin\/linux\/amd64\/kubectl\n - chmod u+x kubectl && mv kubectl \/bin\/kubectl\n - mkdir -p ~\/.kube\n - echo '' > ~\/.kube\/config\n - apk add --update --no-cache curl ca-certificates\n - curl -L https:\/\/get.helm.sh\/helm-v3.1.2-linux-amd64.tar.gz |tar xvz\n - mv linux-amd64\/helm \/usr\/bin\/helm\n - chmod +x \/usr\/bin\/helm\n - terraform apply --auto-approve"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4、 Destroy：将销毁在apply步骤中创建的所有资源。这个步骤也是一个手动的步骤。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"destroy:\nstage: destroy\nscript:\n - mkdir -p ~\/.kube\n - echo '' > ~\/.kube\/config\n - terraform state rm \"helm_release.cert_manager\"\n - terraform state rm \"helm_release.rancher\"\n - terraform destroy --auto-approve\ndependencies:\n - apply\nwhen: manual\n"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Apply"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要执行terraform apply，需要导航到项目的CI\/CD部门。点击New Pipleline并运行新的流水线。一旦完成验证和计划步骤，点击apply步骤并运行。你应该可以了解提交到repo的情况。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Destroy"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要销毁部署，请点击CI\/CD控制台中的destroy步骤并运行。Terraform将销毁流水线之前创建的所有基础设施。唯一会留下的是包含 terraform.tfstate 的 s3 bucket。如果你需要执行销毁步骤，Terraform状态是至关重要的。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"变量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要设置集群的节点数、Kubernetes版本、Rancher版本等，请导航至项目的“Settings”页面，然后在CI \/ CD下设置变量。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"环境变量的建议值"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/56\/564642b183ac57ec4e710def63444302.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"AWS云提供程序"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们在此部署中使用了 AWS 云提供程序。有关提供程序及其工作方式的更多信息，请参考 AWS 文档：https:\/\/www.terraform.io\/docs\/providers\/aws\/index.html"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"provider.tf文件提供了一个如何使用提供程序的好例子。这个文件将允许Terraform代码与AWS交互，并部署资源（EC2、安全组、负载均衡器等）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nprovider \"aws\" {\nregion = \"us-east-1\"\nprofile = \"default\"\naccess_key = lookup(var.aws_access_keys, \"access_key\")\nsecret_key = lookup(var.aws_access_keys, \"secret_key\")\n\n}"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Rancher2 提供程序"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Rancher2提供程序是一个Terraform组件，需要作为插件导入才能工作。rancher-ha.tf文件提供了一个很好的例子来说明如何使用提供程序。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\nresource \"rancher2_bootstrap\" \"admin\" {\nprovider = rancher2.bootstrap\ndepends_on = [null_resource.wait_for_rancher]\npassword = var.ui_password\n}"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们使用Rancher2提供程序来创建Rancher UI管理账户。了解更多关于Rancher2提供程序的信息，欢迎查阅以下文档："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"underline"}],"text":"https:\/\/registry.terraform.io\/providers\/rancher\/rancher2\/latest\/docs"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"GitLab Runner"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你没有配置runner节点，你可以使用这个repo来设置runner的正确配置（"},{"type":"text","marks":[{"type":"underline"}],"text":"https:\/\/gitlab.com\/iby.autometa\/gitlab-runner-aws"},{"type":"text","text":"）。或者按照GitLab文档中的说明来设置一个新的runner（"},{"type":"text","marks":[{"type":"underline"}],"text":"https:\/\/docs.gitlab.com\/runner\/install\/"},{"type":"text","text":"）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你已经有一个正在运行的runner，你可以简单地添加这个配置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"\n#Register the runner\nsudo gitlab-runner register \\\n--non-interactive \\\n--url \"https:\/\/gitlab.com\/\" \\\n--registration-token \"\" \\\n--executor \"docker\" \\\n--docker-image hashicorp\/terraform \\\n--description \"docker-runner\" \\\n--tag-list \"\" \\\n--run-untagged=\"true\" \\\n--locked=\"false\" \\\n--access-level=\"not_protected\""}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"结论"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这篇文章给予了我们几点启示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，我们需要以自动化第一的思维方式来思考我们的日常工作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其次，我们可以利用CI\/CD的几个好处：使用CI\/CD工具可以降低手动管理基础设施的成本；CI\/CD工具使我们能够更有效地协作；而且CI\/CD工具可以让我们深入了解构建步骤和runner节点的CLI输出。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"总的来说，使用CI\/CD有助于我们在代码集成、代码构建和代码部署阶段遵循最佳实践。随着基础设施即代码工具（如Terraform）和AWS、Azure和GCP的云提供商，CI\/CD工具可以让你轻松地将代码与基础设施一起部署。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文转载自：RancherLabs（ID：RancherLabs）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文链接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/6sU1XpSd6J-w8qwKNqm6fQ","title":"xxx","type":null},"content":[{"type":"text","text":"硬核干货 | 使用GitLab CI部署Rancher集群"}]}]}]}