StackStorm分析(四)Mistral說明

StackStorm介紹

       StackStorm是一個強大的自動化平臺,結合DevOpsChatOps,提供可擴展、靈活和健壯的工具鏈用於應用、服務和工作流的自動化能力。

 



Mistral

       Mistral是Mirantis公司爲Openstack開發的工作流組件,提供Workflow as aService。Stackstorm集成了Mistral,提供更加靈活和強大的Workflow支持。

Mistral的一些概念,需要和StackStrom的概念區分開來:

  • Workbook:工作簿。可以理解爲工作流任務的文檔,紀錄了工作流,任務以及任務的執行順序,依賴關係,以及每個任務的輸入輸出等。從開發者的角度看,可以認爲它是一種工作流任務的文檔組織形式,或者命名空間。
  • Workflow:工作流。比Workbook小的單元。描述了任務,動作的執行順序,依賴關係。一個Workbook可以有多條Workflow,也可以只有一條Workflow。
  • Task:描述Workflow中包含的工作步驟。
  • Action:Mistral中最小的單位,描述了某一個具體的動作,比如執行一個http請求,或者發一個e-mail等等。

 

 

YAQL & Jinja

Mistral中支持的模板解析語法有YAQLJinja,這提供了強大的流程控制和語法支持。下面對比下YAQLJinja:

 

YAQL:

version: '2.0'

 

examples.mistral-branching:

    description: >

        A sample workflow that demonstrates how to use conditions

        to determine which path in the workflow to take.

    type: direct

    input:

        - which

    tasks:

        t1:

            action: core.local

            input:

                cmd: "printf <% $.which %>"

            publish:

                path: <% task(t1).result.stdout %>

            on-success:

                - a: <% $.path = 'a' %>

                - b: <% $.path = 'b' %>

                - c: <% not $.path in list(a, b) %>

        a:

            action: core.local

            input:

                cmd: "echo 'Took path A.'"

            publish:

                stdout: <% task(a).result.stdout %>

        b:

            action: core.local

            input:

                cmd: "echo 'Took path B.'"

            publish:

                stdout: <% task(b).result.stdout %>

        c:

            action: core.local

            input:

                cmd: "echo 'Took path C.'"

            publish:

                stdout: <% task(c).result.stdout %>

 

Jinja

version: '2.0'

 

examples.mistral-jinja-branching:

    description: >

        A sample workflow that demonstrates how to use conditions

        to determine which path in the workflow to take.

    type: direct

    input:

        - which

    tasks:

        t1:

            action: core.local

            input:

                cmd: "echo {{ _.which }}"

            publish:

                path: "{{ task('t1').result.stdout }}"

            on-success:

                - a: "{{ _.path == 'a' }}"

                - b: "{{ _.path == 'b' }}"

                - c: "{{ not _.path in ['a', 'b'] }}"

        a:

            action: core.local

            input:

                cmd: "echo 'Took path A.'"

        b:

            action: core.local

            input:

                cmd: "echo 'Took path B.'"

        c:

            action: core.local

            input:

                cmd: "echo 'Took path C.'"


 

Mistral示例

       StackStorm的examples下面提供了關於Mistral的示例,我們接下來通過一系列的示例來說明Mistral。

examples.mistral-basic

同樣的MistralWorkflow跟ActionChainWorkflow一樣都需要提供元數據文件和規格文件,

Pack examples的元數據文件是在/opt/stackstorm/packs/examples/actions目錄下,規格文件是在/opt/stackstorm/packs/examples/actions/workflows目錄下.

/opt/stackstorm/packs/examples/actions/mistral-basic.yaml

---

description: Run a local linux command

enabled: true

runner_type: mistral-v2

entry_point: workflows/mistral-basic.yaml

name: mistral-basic

pack: examples

parameters:

  cmd:

    required: true

    type: string

  timeout:

    type: integer

    default: 60

 

規格文件/opt/stackstorm/packs/examples/actions/workflows/mistral-basic.yaml

version: '2.0'

 

examples.mistral-basic:

    description: A basic workflow that runs an arbitrary linux command.

    type: direct

    input:

        - cmd

        - timeout

    output:

        stdout: <% $.stdout %>

    tasks:

        task1:

            action: core.local cmd=<% $.cmd %> timeout=<% $.timeout %>

            publish:

                stdout: <% task(task1).result.stdout %>

                stderr: <% task(task1).result.stderr %>

 

         根據定義examples.mistral-basic需要提供參數cmd(必選)和timeout(可選),然後在task1中調用core.local執行,並且打印執行結果。執行如下:

$ st2 run examples.mistral-basic cmd="date"

.

id: 5afba6ed2b2556015687cdda

action.ref: examples.mistral-basic

parameters:

  cmd: date

status: succeeded

result_task: task1

result:

  failed: false

  return_code: 0

  stderr: ''

  stdout: Wed May 16 03:35:10 UTC 2018

  succeeded: true

start_timestamp: Wed, 16 May 2018 03:35:09 UTC

end_timestamp: Wed, 16 May 2018 03:35:11 UTC

+--------------------------+------------------------+-------+------------+-------------------------------+

| id                       | status                 | task  | action     | start_timestamp               |

+--------------------------+------------------------+-------+------------+-------------------------------+

| 5afba6ee2b2556015687cddd | succeeded (0s elapsed) | task1 | core.local | Wed, 16 May 2018 03:35:10 UTC |

+--------------------------+------------------------+-------+------------+-------------------------------+

 

examples. mistral-workbook-basic

Mistral中有個WorkBook的概念,是Workflow的組合,也就是說可以同時定義多個Workflow

 

/opt/stackstorm/packs/examples/actions/workflows/mistral-workbook-multiple-subflows.yaml

version: "2.0"

name: "examples.mistral-workbook-multiple-subflows"

 

workflows:

 

    main:

        type: direct

        output:

            report: <% $.report %>

            state: <% $.state %>

        tasks:

            call_internal_workflow:

                workflow: wf1

                input:

                    v1: ok

                publish:

                    report: <% task(call_internal_workflow).result.report %>

                on-success:

                    - call_internal_workflow_multiple_tasks

            call_internal_workflow_multiple_tasks:

                workflow: wf2

                input:

                    v1: good

                    v2: excellent

                on-success:

                    - call_external_workflow

            call_external_workflow:

                action: examples.mistral-basic

                input:

                    cmd: "echo 'external workflow ok'"

                publish:

                    state: <% task(call_external_workflow).result.stdout %>

                on-success:

                    - call_action_chain

            call_action_chain:

                action: examples.echochain

 

    wf1:

        type: direct

        input:

            - v1

        output:

            report: <% $.report %>

        tasks:

            t1:

                action: core.local

                input:

                    cmd: "echo 'task1 <% $.v1 %>'; sleep 1"

                publish:

                    report: <% task(t1).result.stdout %>

 

    wf2:

        type: direct

        input:

            - v1

            - v2

        tasks:

            t1:

                action: core.local

                input:

                    cmd: "echo 'task2 <% $.v1 %>'; sleep 1"

            t2:

                action: core.local

                input:

                    cmd: "echo 'task3 <% $.v2 %>'; sleep 2"

 

examples.mistral-handle-error

Mistral Workflow通過可以設置子任務成功失敗後的執行順序,

/opt/stackstorm/packs/examples/actions/workflows/mistral-handle-error.yaml:

version: '2.0'

 

examples.mistral-handle-error:

    description: >

        A workflow example that illustrates error handling. By default when any task fails,

        the notify_on_error task will be executed and the workflow will transition to the

        failed state.

    type: direct

    input:

        - cmd

    vars:

        error_handled: False

    tasks:

        task1:

            action: core.local cmd="echo 1 && exit 0"

            publish:

                stdout: <% task(task1).result.stdout %>

            on-success:

                - task2

            on-error:

                - handle_error

        task2:

            action: core.local cmd="echo 2 && exit 1"

            publish:

                stdout: <% task(task1).result.stdout %>

            on-error:

                - handle_error

        handle_error:

            action: core.local

            input:

                cmd: "printf '<% task(task1).result.stderr %>'"

            publish:

                error_handled: True

            on-complete:

                - fail

 

examples.mistral-handle-error定義了task1和task2,task1成功後執行task2,task2失敗後執行handle_error。

$ st2 run examples.mistral-handle-error

..

id: 5afbca362b2556015687cddf

action.ref: examples.mistral-handle-error

parameters: None

status: failed

start_timestamp: Wed, 16 May 2018 06:05:42 UTC

end_timestamp: Wed, 16 May 2018 06:05:45 UTC

+--------------------------+------------------------+--------------+------------+-------------------------------+

| id                       | status                 | task         | action     | start_timestamp               |

+--------------------------+------------------------+--------------+------------+-------------------------------+

| 5afbca372b2556015687cde2 | succeeded (0s elapsed) | task1        | core.local | Wed, 16 May 2018 06:05:43 UTC |

| 5afbca372b2556015687cde4 | failed (1s elapsed)    | task2        | core.local | Wed, 16 May 2018 06:05:43 UTC |

| 5afbca382b2556015687cde6 | succeeded (0s elapsed) | handle_error | core.local | Wed, 16 May 2018 06:05:44 UTC |

 

 

Mistral Workflow可以針對失敗的子任務進行重跑(StackStorm2.7版本的ActionChain不支持從失敗的子任務繼續重跑):

$ st2 execution re-run 5afbca362b2556015687cddf --tasks task2

.

id: 5afbcadc2b2556015687cde8

action.ref: examples.mistral-handle-error

parameters: None

status: failed

result_task: task2

result:

  failed: true

  return_code: 1

  stderr: ''

  stdout: 2

  succeeded: false

start_timestamp: Wed, 16 May 2018 06:08:28 UTC

end_timestamp: Wed, 16 May 2018 06:08:30 UTC

+--------------------------+------------------------+--------------+------------+-------------------------------+

| id                       | status                 | task         | action     | start_timestamp               |

+--------------------------+------------------------+--------------+------------+-------------------------------+

| 5afbcadc2b2556015687cdea | failed (1s elapsed)    | task2        | core.local | Wed, 16 May 2018 06:08:28 UTC |

| 5afbcadd2b2556015687cdec | succeeded (0s elapsed) | handle_error | core.local | Wed, 16 May 2018 06:08:29 UTC |

+--------------------------+------------------------+--------------+------------+-------------------------------+

 

examples.mistral-repeat

Mistral Workflow支持循環語法:

/opt/stackstorm/packs/examples/actions/workflows/mistral-repeat.yaml

version: '2.0'

 

examples.mistral-repeat:

    description: >

        A sample workflow that demonstrates how to repeat a task

        x number of times with the same inputs.

    type: direct

    input:

        - cmd

        - count

    tasks:

        repeat:

            with-items: i in <% list(range(0, $.count)) %>

            action: core.local cmd=<% $.cmd %>

            publish:

                result: <% task(repeat).result.select($.stdout) %>

 

循環5次執行命令:

st2 run examples.mistral-repeat cmd=date count=5

..

id: 5afbdab82b2556015687ce19

action.ref: examples.mistral-repeat

parameters:

  cmd: date

  count: 5

status: succeeded

result_task: repeat

result: - failed: false

  return_code: 0

  stderr: ''

  stdout: Wed May 16 07:16:09 UTC 2018

  succeeded: true

- failed: false

  return_code: 0

  stderr: ''

  stdout: Wed May 16 07:16:09 UTC 2018

  succeeded: true

- failed: false

  return_code: 0

  stderr: ''

  stdout: Wed May 16 07:16:09 UTC 2018

  succeeded: true

- failed: false

  return_code: 0

  stderr: ''

  stdout: Wed May 16 07:16:09 UTC 2018

  succeeded: true

- failed: false

  return_code: 0

  stderr: ''

  stdout: Wed May 16 07:16:09 UTC 2018

  succeeded: true

start_timestamp: Wed, 16 May 2018 07:16:08 UTC

end_timestamp: Wed, 16 May 2018 07:16:11 UTC

+--------------------------+------------------------+--------+------------+-------------------------------+

| id                       | status                 | task   | action     | start_timestamp               |

+--------------------------+------------------------+--------+------------+-------------------------------+

| 5afbdab82b2556015687ce20 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC |

| 5afbdab82b2556015687ce22 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC |

| 5afbdab82b2556015687ce23 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC |

| 5afbdab82b2556015687ce21 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC |

| 5afbdab82b2556015687ce24 | succeeded (1s elapsed) | repeat | core.local | Wed, 16 May 2018 07:16:08 UTC |

+--------------------------+------------------------+--------+------------+-------------------------------+

 

 

 

 

參考

  • https://yaql.readthedocs.io/en/latest/
  • http://jinja.pocoo.org/
  • http://jinja.pocoo.org/



作者簡介

吳龍輝,現任網宿科技雲計算架構師,致力於雲計算PaaS的研究和實踐,《Kubernetes實戰》作者,活躍於CloudFoundry,Docker,Kubernetes等開源社區,貢獻代碼和撰寫技術文檔。 
郵箱:[email protected]
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章