Kubernetes 原生 CI/CD 構建框架 Argo 詳解！

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：FogDong（","attrs":{}},{"type":"link","attrs":{"href":"https://www.volcengine.cn?utm_source=infoQ&utm_medium=Media&utm_term=free&utm_campaign=20210204&utm_content=argo","title":""},"content":[{"type":"text","text":"字節跳動火山引擎","attrs":{}}]},{"type":"text","text":"）","attrs":{}}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"什麼是流水線？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在計算機中，流水線是把一個重複的過程分解爲若干個子過程，使每個子過程可以與其他子過程並行進行的技術，也叫 Pipeline。由於這種工作方式與工廠中的生產流水線十分相似， 因此也被稱爲流水線技術。從本質上講，流水線技術是一種時間並行技術。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以我們最熟悉的“構建鏡像”過程爲例：如下圖，在每一次構建鏡像中，我們都需要首先拉下代碼倉庫中的代碼，進行代碼構建，接着打出鏡像，推往鏡像倉庫。每一次代碼更改過後，這一過程都是不變的。使用流水線工具可以極大的提升這一過程的效率，只需要進行簡單的配置便可以輕鬆的完成重複性的工作。這樣的過程也被稱之爲 CI。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3f/3f07c068781c7feb39ad4f3749c6f7e7.webp","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上圖流程中使用的是 Jenkins。Jenkins 作爲老牌流水線框架被大家所熟知。在雲原生時代，Jenkins 推出了 Jenkins X 作爲基於 Kubernetes 的新一代流水線，另外雲原生時代還誕生了兩大流水線框架—— Argo 和 Tekton。本文就詳細介紹了 Argo 的相關內容。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Argo","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Argo Workflows 是一個開源的容器原生的工作流引擎，用於在 Kubernetes 上編排並行作業。Argo Workflows 實現爲 Kubernetes CRD。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Quick Start","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"kubectl create ns argo\nkubectl apply -n argo -f https://raw.githubusercontent.com/argoproj/argo/stable/manifests/quick-start-postgres.yaml","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Argo 基於 Kubernetes，可以直接使用 kubectl 安裝，安裝的組件主要包括了一些 CRD 以及對應的 controller 和一個 server。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注意，上述安裝只會執行同 namespace 內的 Workflow，cluster install 詳見 文檔","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"三級定義","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要了解 Argo 定義的 CRD，先從其中的三級定義入手。概念上的從大到小分別爲 WorkflowTemplate，Workflow，template。這些資源的命名有些相似，所以會稍微有些迷惑性。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Template","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從最簡單的 template 說起，一個 template 有多種類型，分別爲 container，script，dag，steps，resource 以及 suspend。對於 template，我們可以簡單的將其理解爲一個 Pod ——container/script/resource 類型的 template 都會去實際控制一個 Pod。而 dag/steps 類型的 template 則是由多個基礎類型的 template （container/script/resource）組成的。","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"container","attrs":{}},{"type":"text","text":"：最常見的模板類型，與 Kubernetes container spec 保持一致。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"script","attrs":{}},{"type":"text","text":"：該類型基於 Container，支持用戶在 template 定義一段腳本，另有一個 Source 字段來表示腳本的運行環境。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"resource","attrs":{}},{"type":"text","text":"：該類型支持我們在 template 中對 kubernetes 的資源進行操作，有一個 action 字段可以指定操作類型，如 create, apply, delete 等，並且支持設定相關的成功與失敗條件用於判斷該 template 的成功與失敗。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"suspend","attrs":{}},{"type":"text","text":"：Suspend template 將在一段時間內或在手動恢復執行之前暫停執行。可以從 CLI （使用 argo resume）、API 或 UI 恢復執行。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"steps","attrs":{}},{"type":"text","text":"：Steps Template 允許用戶以一系列步驟定義任務。在 Steps 中，[--] 代表順序執行，[-] 代表並行執行。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"dag","attrs":{}},{"type":"text","text":"：DAG template 允許用戶將任務定義爲帶依賴的有向無環圖。在 DAG 中，通過 dependencies設置在特定任務開始之前必須完成的其他任務。沒有任何依賴項的任務將立即運行。有關 DAG 的詳細邏輯可見","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/argoproj/argo/blob/master/workflow/controller/dag.go#L204","title":""},"content":[{"type":"text","text":"源碼","attrs":{}}]},{"type":"text","text":"。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Workflow","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在一個 Workflow 中，其 spec 中有一個名爲 templates 的字段，在其中至少需要一個 template 作爲其組成的任務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個最簡單的 hello world 例子如下：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"apiVersion: argoproj.io/v1alpha1\nkind: Workflow\nmetadata:\n generateName: hello-world-\n labels:\n workflows.argoproj.io/archive-strategy: \"false\"\nspec:\n entrypoint: whalesay\n templates:\n - name: whalesay\n container:\n image: docker/whalesay:latest\n command: [cowsay]\n args: [\"hello world\"]","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這個例子中，該 Workflow 的 templates 字段中指定了一個類型爲 container 的 template，使用了 whalesay 鏡像。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接着，來看一個稍微複雜一點的 workflow：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"apiVersion: argoproj.io/v1alpha1\nkind: Workflow\nmetadata:\n generateName: steps-\nspec:\n entrypoint: hello-hello-hello\n\n # 在 templates 中有兩個 template，一個爲 hello-hello-hello，一個爲 whalesay\n templates:\n - name: hello-hello-hello\n # Instead of just running a container\n # This template has a sequence of steps\n steps: # 該 template 的類型是 steps \n - - name: hello1 # 在 steps 類型中，[--] 代表順序執行，[-] 代表並行執行 \n template: whalesay # 這裏引用了下面的 template\n arguments:\n parameters:\n - name: message\n value: \"hello1\"\n - - name: hello2a # 兩個短槓 [--] => 順序執行\n template: whalesay\n arguments:\n parameters:\n - name: message\n value: \"hello2a\"\n - name: hello2b # 一個短槓 [-] => 並行執行\n template: whalesay\n arguments:\n parameters:\n - name: message\n value: \"hello2b\"\n### 第二個 template\n - name: whalesay\n inputs:\n parameters:\n - name: message\n container:\n image: docker/whalesay\n command: [cowsay]\n args: [\"{{inputs.parameters.message}}\"]","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"WorkflowTemplate","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"WorkflowTemplate 相當於是 Workflow 的模板庫，和 Workflow 一樣，也由 template 組成。用戶在創建完 WorkflowTemplate 後，可以通過直接提交它們來執行 Workflow。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"apiVersion: argoproj.io/v1alpha1\nkind: WorkflowTemplate\nmetadata:\n name: workflow-template-submittablespec:\n entrypoint: whalesay-template \n arguments:\n parameters:\n - name: message\n value: hello world\n templates:\n - name: whalesay-template\n inputs:\n parameters:\n - name: message\n container:\n image: docker/whalesay\n command: [cowsay]\n args: [\"{{inputs.parameters.message}}\"]","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Workflow Overview","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/62/62d5659cf5dfba26a96ca9b140a41edd.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在瞭解了 Argo 的三級定義後，我們首先來深入一下 Argo 中最爲關鍵的定義，Workflow。Workflow 是Argo 中最重要的資源，有兩個重要的功能：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"它定義了要執行的工作流。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"它存儲了工作流的狀態。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於這些雙重職責，Workflow 應該被視爲一個 Active 的對象。它不僅是一個靜態定義，也是是上述定義的一個“實例”。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以看到 Workflow Template 的定義與 Workflow 幾乎一致，除了類型不同。正因爲 Workflow 既可以是一個定義也可以是一個實例，所以才需要 WorkflowTemplate 作爲 Workflow 的模板，WorkflowTemplate 在定義後可以通過提交（Submit）來創建一個 Workflow。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而 Workflow 由一個 entrypoint 及一系列 template 組成，entrypoint 定義了這個 workflow 執行的入口，而 template 會實際去執行一個 Pod，其中，用戶定義的內容會在 Pod 中以 Main Container 體現。此外，還有兩個 Sidecar 來輔助運行。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Sidecar","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Argo 中，這些 Sidecar 的鏡像都是 argoexec。Argo 通過這個 executor 來完成一些流程控制。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Init","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當用戶的 template 中需要使用到 inputs 中的 artifact 或者是 script 類型時（script 類型需要注入腳本），Argo 都會爲這個 pod 加上一個 Init Container —— 其鏡像爲 argoexec，而命令是 argoexec init。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這個 Init Container 中，主要做的工作便是加載 artifact:","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"func loadArtifacts() error {\n wfExecutor := initExecutor()\n defer wfExecutor.HandleError()\n defer stats.LogStats()\n\n // Download input artifacts\n err := wfExecutor.StageFiles()\n if err != nil {\n wfExecutor.AddError(err)\n return err\n }\n err = wfExecutor.LoadArtifacts()\n if err != nil {\n wfExecutor.AddError(err)\n return err\n }\n return nil\n}","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Wait","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了 Resource 類型外的 template，Argo 都會注入一個 Wait Container，用於等待 Main Container 的完成並結束所有 Sidecar。這個 Wait Container 的鏡像同樣爲 argoexec，而命令是 argoexec wait。（Resource 類型的不需要是因爲 Resource 類型的 template 直接使用 argoexec 作爲 Main Container 運行）","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"func waitContainer() error {\n wfExecutor := initExecutor()\n defer wfExecutor.HandleError()\n defer stats.LogStats()\n stats.StartStatsTicker(5 * time.Minute)\n\n defer func() {\n // Killing sidecar containers\n err := wfExecutor.KillSidecars()\n if err != nil {\n log.Errorf(\"Failed to kill sidecars: %s\", err.Error())\n }\n }()\n\n // Wait for main container to complete\n waitErr := wfExecutor.Wait()\n if waitErr != nil {\n wfExecutor.AddError(waitErr)\n // do not return here so we can still try to kill sidecars & save outputs\n }\n\n // Capture output script result\n err := wfExecutor.CaptureScriptResult()\n if err != nil {\n wfExecutor.AddError(err)\n return err\n }\n // Capture output script exit code\n err = wfExecutor.CaptureScriptExitCode()\n if err != nil {\n wfExecutor.AddError(err)\n return err\n }\n // Saving logs\n logArt, err := wfExecutor.SaveLogs()\n if err != nil {\n wfExecutor.AddError(err)\n return err\n }\n // Saving output parameters\n err = wfExecutor.SaveParameters()\n if err != nil {\n wfExecutor.AddError(err)\n return err\n }\n // Saving output artifacts\n err = wfExecutor.SaveArtifacts()\n if err != nil {\n wfExecutor.AddError(err)\n return err\n }\n err = wfExecutor.AnnotateOutputs(logArt)\n if err != nil {\n wfExecutor.AddError(err)\n return err\n }\n\n // To prevent the workflow step from completing successfully, return the error occurred during wait.\n if waitErr != nil {\n return waitErr\n }\n\n return nil\n}","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"Inputs and Outputs","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在運行 Workflow 時，一個非常常見的場景是輸出產物的傳遞。通常，一個 Step 的輸出產物可以用作後續步驟的輸入產物。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Argo 中，產物可以通過 Artifact 或是 Parameter 傳遞。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Artifact","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要使用 Argo 的 Artifact，首先必須配置和使用 Artifact 存儲倉庫。具體的配置方式可以通過修改存有 Artifact Repository 信息的默認 Config Map 或者在 Workflow 中顯示指定，詳見 配置文檔，在此不做贅述。 下表爲 Argo 支持的倉庫類型。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/46/4687b7ba687bfc5c74d5e830377eac6c.webp","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個簡單的使用了 Artifact 的例子如下：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"apiVersion: argoproj.io/v1alpha1\nkind: Workflow\nmetadata:\n generateName: artifact-passing-spec:\n entrypoint: artifact-example\n templates:\n - name: artifact-example\n steps:\n - - name: generate-artifact\n template: whalesay\n - - name: consume-artifact\n template: print-message\n arguments:\n artifacts:\n # bind message to the hello-art artifact\n # generated by the generate-artifact step\n - name: message\n from: \"{{steps.generate-artifact.outputs.artifacts.hello-art}}\"\n\n - name: whalesay\n container:\n image: docker/whalesay:latest\n command: [sh, -c]\n args: [\"cowsay hello world | tee /tmp/hello_world.txt\"]\n outputs:\n artifacts:\n # generate hello-art artifact from /tmp/hello_world.txt\n # artifacts can be directories as well as files\n - name: hello-art\n path: /tmp/hello_world.txt\n\n - name: print-message\n inputs:\n artifacts:\n # unpack the message input artifact\n # and put it at /tmp/message\n - name: message\n path: /tmp/message\n container:\n image: alpine:latest\n command: [sh, -c]\n args: [\"cat /tmp/message\"]","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"默認情況下，Artifact 被打包爲 tar 包和 gzip 包。也可以使用 archive 字段指定存檔策略。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在上面的例子裏，名爲 whalesay 的 template 使用 cowsay 命令生成一個名爲 /tmp/hello-world.txt 的文件。然後將該文件作爲一個名爲 hello-art 的 Artifact 輸出。名爲 print-message 的 template 接受一個名爲 message 的輸入 Artifact，在 /tmp/message 的路徑上解包它，然後使用 cat 命令打印 /tmp/message 的內容。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在前面的 Sidecar 介紹中提到過，Init Container 主要用於拉取 Artifact 產物。這些 Sidecar 正是產物傳遞的關鍵。接着，我們通過介紹另一種產物傳遞的方式來體驗 Argo 中傳遞產物的關鍵。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Scripts","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"先來看一個簡單的例子：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"apiVersion: argoproj.io/v1alpha1\nkind: Workflow\nmetadata:\n generateName: scripts-bash-spec:\n entrypoint: bash-script-example\n templates:\n - name: bash-script-example\n steps:\n - - name: generate\n template: gen-random-int-bash\n - - name: print\n template: print-message\n arguments:\n parameters:\n - name: message\n value: \"{{steps.generate.outputs.result}}\" # The result of the here-script\n\n - name: gen-random-int-bash\n script:\n image: debian:9.4\n command: [bash]\n source: | # Contents of the here-script\n cat /dev/urandom | od -N2 -An -i | awk -v f=1 -v r=100 '{printf \"%i\\n\", f + r * $1 / 65536}'\n\n - name: gen-random-int-python\n script:\n image: python:alpine3.6\n command: [python]\n source: |\n import random\n i = random.randint(1, 100)\n print(i)\n\n - name: print-message\n inputs:\n parameters:\n - name: message\n container:\n image: alpine:latest\n command: [sh, -c]\n args: [\"echo result was: {{inputs.parameters.message}}\"]","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在上面的例子中，有兩個類型爲 script 的 template，script 允許使用 source 規範腳本主體。這將創建一個包含腳本主體的臨時文件，然後將臨時文件的名稱作爲最後一個參數傳遞給 command（執行腳本主體的解釋器），這樣便可以方便的執行不同類型的腳本（bash、python、js etc）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Script template 會將腳本的標準輸出分配給一個名爲 result 的特殊輸出參數從而被其他 template 調用。在這裏，通過 {{steps.generate.outputs.result}} 即可獲取到名爲 generate 的 template 的腳本輸出。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"{{xxx}} 是 Argo 固定的變量替換格式：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關於變量的格式詳見 ","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/argoproj/argo/blob/master/docs/variables.md","title":""},"content":[{"type":"text","text":"文檔","attrs":{}}]},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關於變量替換的邏輯詳見 ","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/argoproj/argo/blob/master/workflow/common/util.go#L305","title":""},"content":[{"type":"text","text":"源碼","attrs":{}}]},{"type":"text","text":"。","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼，容器內部應該如何獲取這個腳本輸出呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"還是回到 Sidecar，在 Wait Container 中，有這樣一段邏輯：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"// CaptureScriptResult will add the stdout of a script template as output result\nfunc (we *WorkflowExecutor) CaptureScriptResult() error {\n \n ...\n\n log.Infof(\"Capturing script output\")\n mainContainerID, err := we.GetMainContainerID()\n if err != nil {\n return err\n }\n reader, err := we.RuntimeExecutor.GetOutputStream(mainContainerID, false)\n if err != nil {\n return err\n }\n defer func() { _ = reader.Close() }()\n bytes, err := ioutil.ReadAll(reader)\n if err != nil {\n return errors.InternalWrapError(err)\n }\n out := string(bytes)\n // Trims off a single newline for user convenience\n outputLen := len(out)\n if outputLen > 0 && out[outputLen-1] == '\\n' {\n out = out[0 : outputLen-1]\n }\n\n const maxAnnotationSize int = 256 * (1 << 10) // 256 kB\n // A character in a string is a byte\n if len(out) > maxAnnotationSize {\n log.Warnf(\"Output is larger than the maximum allowed size of 256 kB, only the last 256 kB were saved\")\n out = out[len(out)-maxAnnotationSize:]\n }\n\n we.Template.Outputs.Result = &out\n return nil\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再來看看這個 Wait Container 的 Volume Mount 情況：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"volumeMounts:\n - mountPath: /argo/podmetadata\n name: podmetadata\n - mountPath: /var/run/docker.sock\n name: docker-sock\n readOnly: true\n - mountPath: /argo/secret/my-minio-cred\n name: my-minio-cred\n readOnly: true\n - mountPath: /var/run/secrets/kubernetes.io/serviceaccount\n name: default-token-b5grl\n readOnly: true","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現在就十分明確了，Wait Container 通過掛載 docker.sock 以及 service account，獲取到 Main Container 中的輸出結果，並保存到 Workflow 中。當然，也因爲 Workflow 中保存了大量的信息，當一個 Workflow 的 Step 過多時，整個 Workflow 的結構會過於龐大。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Parameter","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Parameter 提供了一種通用機制，可以將步驟的結果用作參數。Parameter 的工作原理與腳本結果類似，除了輸出參數的值會被設置爲生成文件的內容，而不是 stdout 的內容。如：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":" - name: whalesay\n container:\n image: docker/whalesay:latest\n command: [sh, -c]\n args: [\"echo -n hello world > /tmp/hello_world.txt\"] # generate the content of hello_world.txt\n outputs:\n parameters:\n - name: hello-param # name of output parameter\n valueFrom:\n path: /tmp/hello_world.txt # set the value of hello-param to the contents of this hello-world.txt","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Volume","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這並不是 Argo 處理產物傳遞的一種標準方式，但是通過共享存儲，顯然我們也能達到共通產物的結果。當然，若使用 Volume，我們則無需藉助 Inputs 和 Outputs。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在 Workflow 的 Spec 中，我們可以定義一個 Volume 模板：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"apiVersion: argoproj.io/v1alpha1\nkind: Workflow\nmetadata:\n generateName: volumes-pvc-spec:\n entrypoint: volumes-pvc-example\n volumeClaimTemplates: # define volume, same syntax as k8s Pod spec\n - metadata:\n name: workdir # name of volume claim\n spec:\n accessModes: [ \"ReadWriteOnce\" ]\n resources:\n requests:\n storage: 1Gi # Gi => 1024 * 1024 * 1024","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"並在其他的 template 中 mount 該 volume：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":" - name: whalesay\n container:\n image: docker/whalesay:latest\n command: [sh, -c]\n args: [\"echo generating message in volume; cowsay hello world | tee /mnt/vol/hello_world.txt\"]\n # Mount workdir volume at /mnt/vol before invoking docker/whalesay\n volumeMounts: # same syntax as k8s Pod spec\n - name: workdir\n mountPath: /mnt/vol","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"其他流程控制功能","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"循環","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在編寫 Workflow 時，能夠循環迭代一組輸入通常是非常有用的，如下例所示:","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":" templates:\n - name: loop-example\n steps:\n - - name: print-message\n template: whalesay\n arguments:\n parameters:\n - name: message\n value: \"{{item}}\"\n withItems: # invoke whalesay once for each item in parallel\n - hello world # item 1\n - goodbye world # item 2","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在源碼實現中，將會去判斷 withItems，如果存在，則對其中的每個元素進行一次 step 的擴展。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"// expandStepGroup looks at each step in a collection of parallel steps, and expands all steps using withItems/withParam\nfunc (woc *wfOperationCtx) expandStepGroup(sgNodeName string, stepGroup []wfv1.WorkflowStep, stepsCtx *stepsContext) ([]wfv1.WorkflowStep, error) {\n newStepGroup := make([]wfv1.WorkflowStep, 0)\n for _, step := range stepGroup {\n if !step.ShouldExpand() {\n newStepGroup = append(newStepGroup, step)\n continue\n }\n expandedStep, err := woc.expandStep(step)\n if err != nil {\n return nil, err\n }\n if len(expandedStep) == 0 {\n // Empty list\n childNodeName := fmt.Sprintf(\"%s.%s\", sgNodeName, step.Name)\n if woc.wf.GetNodeByName(childNodeName) == nil {\n stepTemplateScope := stepsCtx.tmplCtx.GetTemplateScope()\n skipReason := \"Skipped, empty params\"\n woc.log.Infof(\"Skipping %s: %s\", childNodeName, skipReason)\n woc.initializeNode(childNodeName, wfv1.NodeTypeSkipped, stepTemplateScope, &step, stepsCtx.boundaryID, wfv1.NodeSkipped, skipReason)\n woc.addChildNode(sgNodeName, childNodeName)\n }\n }\n newStepGroup = append(newStepGroup, expandedStep...)\n }\n return newStepGroup, nil\n}","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"條件判斷","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過 when 關鍵字指定：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":" templates:\n - name: coinflip\n steps:\n # flip a coin\n - - name: flip-coin\n template: flip-coin\n # evaluate the result in parallel\n - - name: heads\n template: heads # call heads template if \"heads\"\n when: \"{{steps.flip-coin.outputs.result}} == heads\"\n - name: tails\n template: tails # call tails template if \"tails\"\n when: \"{{steps.flip-coin.outputs.result}} == tails\"","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"錯誤重嘗","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":" templates:\n - name: retry-backoff\n retryStrategy:\n limit: 10\n retryPolicy: \"Always\"\n backoff:\n duration: \"1\" # Must be a string. Default unit is seconds. Could also be a Duration, e.g.: \"2m\", \"6h\", \"1d\"\n factor: 2\n maxDuration: \"1m\" # Must be a string. Default unit is seconds. Could also be a Duration, e.g.: \"2m\", \"6h\", \"1d\"","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"遞歸","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Template 可以遞歸地相互調用，這是一個非常實用的功能。例如在機器學習場景中：可以設定準確率必需滿足一個值，否則就持續進行訓練。在下面這個拋硬幣例子中，我們可以持續拋硬幣，直到出現正面才結束整個工作流。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"apiVersion: argoproj.io/v1alpha1kind: Workflowmetadata:\n generateName: coinflip-recursive-spec:\n entrypoint: coinflip\n templates:\n - name: coinflip\n steps:\n # flip a coin\n - - name: flip-coin\n template: flip-coin\n # evaluate the result in parallel\n - - name: heads\n template: heads # call heads template if \"heads\"\n when: \"{{steps.flip-coin.outputs.result}} == heads\"\n - name: tails # keep flipping coins if \"tails\"\n template: coinflip\n when: \"{{steps.flip-coin.outputs.result}} == tails\"\n\n - name: flip-coin\n script:\n image: python:alpine3.6\n command: [python]\n source: |\n import random\n result = \"heads\" if random.randint(0,1) == 0 else \"tails\"\n print(result)\n\n - name: heads\n container:\n image: alpine:3.6\n command: [sh, -c]\n args: [\"echo \\\"it was heads\\\"\"]","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以下是兩次執行的結果，第一次執行直接拋到正面，結束流程；第二次重複三次後才拋到正面，結束流程。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"argo get coinflip-recursive-tzcb5\n\nSTEP PODNAME MESSAGE\n ✔ coinflip-recursive-vhph5\n ├───✔ flip-coin coinflip-recursive-vhph5-2123890397\n └─┬─✔ heads coinflip-recursive-vhph5-128690560\n └─○ tails\n\nSTEP PODNAME MESSAGE\n ✔ coinflip-recursive-tzcb5\n ├───✔ flip-coin coinflip-recursive-tzcb5-322836820\n └─┬─○ heads\n └─✔ tails\n ├───✔ flip-coin coinflip-recursive-tzcb5-1863890320\n └─┬─○ heads\n └─✔ tails\n ├───✔ flip-coin coinflip-recursive-tzcb5-1768147140\n └─┬─○ heads\n └─✔ tails\n ├───✔ flip-coin coinflip-recursive-tzcb5-4080411136\n └─┬─✔ heads coinflip-recursive-tzcb5-4080323273\n └─○ tails","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"退出處理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"退出處理是一個指定在 workflow 結束時執行的 template，無論成功或失敗。","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"spec:\n entrypoint: intentional-fail\n onExit: exit-handler # invoke exit-handler template at end of the workflow\n templates:\n ...","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"對比 Tekton","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"相較於 Tekton 而言，Argo 的流程控制功能更加豐富。擁有着循環、遞歸等功能，這對於一些機器學習的場景都是十分適用的。而 Argo 社區對自己的定位也是 MLOps、AIOps、Data/Batch Processing，這也正是 Kubeflow Pipeline 底層基於 Argo 的原因（儘管 KFP 也在做 Tekton 的 backend）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是在權限控制方面，Argo 做的就不如 Tekton；並且我個人認爲，Tekton 的結構定義更爲清晰。二者各有優劣，可以根據自己的需求進行選擇。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"參考文檔","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"Argo Roadmap：","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"underline","attrs":{}}],"text":"https://github.com/argoproj/argo/blob/master/docs/roadmap.md","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"Argo Examples：","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"underline","attrs":{}}],"text":"https://argoproj.github.io/argo/examples/#welcome","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"Argo Source Code：","attrs":{}},{"type":"text","marks":[{"type":"italic","attrs":{}},{"type":"underline","attrs":{}}],"text":"https://github.com/argoproj/argo","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}