StackStorm介紹
StackStorm是一個強大的自動化平臺,結合DevOps和ChatOps,提供可擴展、靈活和健壯的工具鏈用於應用、服務和工作流的自動化能力。
Mistral
Mistral是Mirantis公司爲Openstack開發的工作流組件,提供Workflow as aService。Stackstorm集成了Mistral,提供更加靈活和強大的Workflow支持。
Mistral的一些概念,需要和StackStrom的概念區分開來:
- Workbook:工作簿。可以理解爲工作流任務的文檔,紀錄了工作流,任務以及任務的執行順序,依賴關係,以及每個任務的輸入輸出等。從開發者的角度看,可以認爲它是一種工作流任務的文檔組織形式,或者命名空間。
- Workflow:工作流。比Workbook小的單元。描述了任務,動作的執行順序,依賴關係。一個Workbook可以有多條Workflow,也可以只有一條Workflow。
- Task:描述Workflow中包含的工作步驟。
- Action:Mistral中最小的單位,描述了某一個具體的動作,比如執行一個http請求,或者發一個e-mail等等。
YAQL & Jinja
Mistral中支持的模板解析語法有YAQL和Jinja,這提供了強大的流程控制和語法支持。下面對比下YAQL和Jinja:
YAQL:
version: '2.0'
examples.mistral-branching: description: > A sample workflow that demonstrates how to use conditions to determine which path in the workflow to take. type: direct input: - which tasks: t1: action: core.local input: cmd: "printf <% $.which %>" publish: path: <% task(t1).result.stdout %> on-success: - a: <% $.path = 'a' %> - b: <% $.path = 'b' %> - c: <% not $.path in list(a, b) %> a: action: core.local input: cmd: "echo 'Took path A.'" publish: stdout: <% task(a).result.stdout %> b: action: core.local input: cmd: "echo 'Took path B.'" publish: stdout: <% task(b).result.stdout %> c: action: core.local input: cmd: "echo 'Took path C.'" publish: stdout: <% task(c).result.stdout %> |
Jinja
version: '2.0'
examples.mistral-jinja-branching: description: > A sample workflow that demonstrates how to use conditions to determine which path in the workflow to take. type: direct input: - which tasks: t1: action: core.local input: cmd: "echo {{ _.which }}" publish: path: "{{ task('t1').result.stdout }}" on-success: - a: "{{ _.path == 'a' }}" - b: "{{ _.path == 'b' }}" - c: "{{ not _.path in ['a', 'b'] }}" a: action: core.local input: cmd: "echo 'Took path A.'" b: action: core.local input: cmd: "echo 'Took path B.'" c: action: core.local input: cmd: "echo 'Took path C.'" |
Mistral示例
StackStorm的examples下面提供了關於Mistral的示例,我們接下來通過一系列的示例來說明Mistral。
examples.mistral-basic
同樣的MistralWorkflow跟ActionChainWorkflow一樣都需要提供元數據文件和規格文件,
Pack examples的元數據文件是在/opt/stackstorm/packs/examples/actions目錄下,規格文件是在/opt/stackstorm/packs/examples/actions/workflows目錄下.
/opt/stackstorm/packs/examples/actions/mistral-basic.yaml
--- description: Run a local linux command enabled: true runner_type: mistral-v2 entry_point: workflows/mistral-basic.yaml name: mistral-basic pack: examples parameters: cmd: required: true type: string timeout: type: integer default: 60 |
規格文件/opt/stackstorm/packs/examples/actions/workflows/mistral-basic.yaml
version: '2.0'
examples.mistral-basic: description: A basic workflow that runs an arbitrary linux command. type: direct input: - cmd - timeout output: stdout: <% $.stdout %> tasks: task1: action: core.local cmd=<% $.cmd %> timeout=<% $.timeout %> publish: stdout: <% task(task1).result.stdout %> stderr: <% task(task1).result.stderr %> |
根據定義examples.mistral-basic需要提供參數cmd(必選)和timeout(可選),然後在task1中調用core.local執行,並且打印執行結果。執行如下:
$ st2 run examples.mistral-basic cmd="date" . id: 5afba6ed2b2556015687cdda action.ref: examples.mistral-basic parameters: cmd: date status: succeeded result_task: task1 result: failed: false return_code: 0 stderr: '' stdout: Wed May 16 03:35:10 UTC 2018 succeeded: true start_timestamp: Wed, 16 May 2018 03:35:09 UTC end_timestamp: Wed, 16 May 2018 03:35:11 UTC +--------------------------+------------------------+-------+------------+-------------------------------+ | id | status | task | action | start_timestamp | +--------------------------+------------------------+-------+------------+-------------------------------+ | 5afba6ee2b2556015687cddd | succeeded (0s elapsed) | task1 | core.local | Wed, 16 May 2018 03:35:10 UTC | +--------------------------+------------------------+-------+------------+-------------------------------+ |
examples. mistral-workbook-basic
Mistral中有個WorkBook的概念,是Workflow的組合,也就是說可以同時定義多個Workflow
/opt/stackstorm/packs/examples/actions/workflows/mistral-workbook-multiple-subflows.yaml
version: "2.0" name: "examples.mistral-workbook-multiple-subflows"
workflows:
main: type: direct output: report: <% $.report %> state: <% $.state %> tasks: call_internal_workflow: workflow: wf1 input: v1: ok publish: report: <% task(call_internal_workflow).result.report %> on-success: - call_internal_workflow_multiple_tasks call_internal_workflow_multiple_tasks: workflow: wf2 input: v1: good v2: excellent on-success: - call_external_workflow call_external_workflow: action: examples.mistral-basic input: cmd: "echo 'external workflow ok'" publish: state: <% task(call_external_workflow).result.stdout %> on-success: - call_action_chain call_action_chain: action: examples.echochain
wf1: type: direct input: - v1 output: report: <% $.report %> tasks: t1: action: core.local input: cmd: "echo 'task1 <% $.v1 %>'; sleep 1" publish: report: <% task(t1).result.stdout %>
wf2: type: direct input: - v1 - v2 tasks: t1: action: core.local input: cmd: "echo 'task2 <% $.v1 %>'; sleep 1" t2: action: core.local input: cmd: "echo 'task3 <% $.v2 %>'; sleep 2" |
examples.mistral-handle-error
Mistral Workflow通過可以設置子任務成功失敗後的執行順序,
/opt/stackstorm/packs/examples/actions/workflows/mistral-handle-error.yaml:
version: '2.0'
examples.mistral-handle-error: description: > A workflow example that illustrates error handling. By default when any task fails, the notify_on_error task will be executed and the workflow will transition to the failed state. type: direct input: - cmd vars: error_handled: False tasks: task1: action: core.local cmd="echo 1 && exit 0" publish: stdout: <% task(task1).result.stdout %> on-success: - task2 on-error: - handle_error task2: action: core.local cmd="echo 2 && exit 1" publish: stdout: <% task(task1).result.stdout %> on-error: - handle_error handle_error: action: core.local input: cmd: "printf '<% task(task1).result.stderr %>'" publish: error_handled: True on-complete: - fail |
examples.mistral-handle-error定義了task1和task2,task1成功後執行task2,task2失敗後執行handle_error。
$ st2 run examples.mistral-handle-error .. id: 5afbca362b2556015687cddf action.ref: examples.mistral-handle-error parameters: None status: failed start_timestamp: Wed, 16 May 2018 06:05:42 UTC end_timestamp: Wed, 16 May 2018 06:05:45 UTC +--------------------------+------------------------+--------------+------------+-------------------------------+ | id | status | task | action | start_timestamp | +--------------------------+------------------------+--------------+------------+-------------------------------+ | 5afbca372b2556015687cde2 | succeeded (0s elapsed) | task1 | core.local | Wed, 16 May 2018 06:05:43 UTC | | 5afbca372b2556015687cde4 | failed (1s elapsed) | task2 | core.local | Wed, 16 May 2018 06:05:43 UTC | | 5afbca382b2556015687cde6 | succeeded (0s elapsed) | handle_error | core.local | Wed, 16 May 2018 06:05:44 UTC | |
Mistral Workflow可以針對失敗的子任務進行重跑(StackStorm2.7版本的ActionChain不支持從失敗的子任務繼續重跑):
$ st2 execution re-run 5afbca362b2556015687cddf --tasks task2 . id: 5afbcadc2b2556015687cde8 action.ref: examples.mistral-handle-error parameters: None status: failed result_task: task2 result: failed: true return_code: 1 stderr: '' stdout: 2 succeeded: false start_timestamp: Wed, 16 May 2018 06:08:28 UTC end_timestamp: Wed, 16 May 2018 06:08:30 UTC +--------------------------+------------------------+--------------+------------+-------------------------------+ | id | status | task | action | start_timestamp | +--------------------------+------------------------+--------------+------------+-------------------------------+ | 5afbcadc2b2556015687cdea | failed (1s elapsed) | task2 | core.local | Wed, 16 May 2018 06:08:28 UTC | | 5afbcadd2b2556015687cdec | succeeded (0s elapsed) | handle_error | core.local | Wed, 16 May 2018 06:08:29 UTC | +--------------------------+------------------------+--------------+------------+-------------------------------+ |
examples.mistral-repeat
Mistral Workflow支持循環語法:
/opt/stackstorm/packs/examples/actions/workflows/mistral-repeat.yaml
version: '2.0'
examples.mistral-repeat: description: > A sample workflow that demonstrates how to repeat a task x number of times with the same inputs. type: direct input: - cmd - count tasks: repeat: with-items: i in <% list(range(0, $.count)) %> action: core.local cmd=<% $.cmd %> publish: result: <% task(repeat).result.select($.stdout) %> |
循環5次執行命令:
st2 run examples.mistral-repeat cmd=date count=5 .. id: 5afbdab82b2556015687ce19 action.ref: examples.mistral-repeat parameters: cmd: date count: 5 status: succeeded result_task: repeat result: - failed: false return_code: 0 stderr: '' stdout: Wed May 16 07:16:09 UTC 2018 succeeded: true - failed: false return_code: 0 stderr: '' stdout: Wed May 16 07:16:09 UTC 2018 succeeded: true - failed: false return_code: 0 stderr: '' stdout: Wed May 16 07:16:09 UTC 2018 succeeded: true - failed: false return_code: 0 stderr: '' stdout: Wed May 16 07:16:09 UTC 2018 succeeded: true - failed: false return_code: 0 stderr: '' stdout: Wed May 16 07:16:09 UTC 2018 succeeded: true start_timestamp: Wed, 16 May 2018 07:16:08 UTC end_timestamp: Wed, 16 May 2018 07:16:11 UTC +--------------------------+------------------------+--------+------------+-------------------------------+ | id | status | task | action | start_timestamp | +--------------------------+------------------------+--------+------------+-------------------------------+ | 5afbdab82b2556015687ce20 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC | | 5afbdab82b2556015687ce22 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC | | 5afbdab82b2556015687ce23 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC | | 5afbdab82b2556015687ce21 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC | | 5afbdab82b2556015687ce24 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC | +--------------------------+------------------------+--------+------------+-------------------------------+ |
參考
- https://yaql.readthedocs.io/en/latest/
- http://jinja.pocoo.org/
- http://jinja.pocoo.org/
作者簡介
吳龍輝,現任網宿科技雲計算架構師,致力於雲計算PaaS的研究和實踐,《Kubernetes實戰》作者,活躍於CloudFoundry,Docker,Kubernetes等開源社區,貢獻代碼和撰寫技術文檔。郵箱:[email protected]