簡單上手版使用oozie提交spark程序

1. 做這個實驗之前你的服務器上最好裝了cdh集羣，以及添加必要的組件，如hadoop，oozie，spark，yarn等。必須都啓動成功。
2. 需要準備一個關於spark的demo架包、job.propertie、workflow.xml。創建一個文件夾將三個放入其中，然後將文件夾put到HDFS指定的路徑下面。(*注意每次在linux下修改完後，記得put到HDFS中 *)
3. 準備一個job.propertie和workflow.xml，內容如下
   3.1job.propertie

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#      http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# See the License for the specific language governing permissions and
# limitations under the License.
#

#HDFS文件系統的端口
nameNode=hdfs://emg101.emapgo.com:8020 

#resourceManager的端口,也有人寫jobTracker
resourceManager=emg104.emapgo.com:8032 

#需要有，是否使用系統lib路徑
oozie.use.system.libpath=true 

#需要有，workflow.xml在hdfs上的路徑   
oozie.wf.application.path=${nameNode}/user/${user.name}/examples/apps/spark/workflow.xml

#workflow.xml中使用
queueName=default

#下面是spark的屬性參數，具體參照自己的workflow.xml
jobmaster=yarn-cluster
jobmode=cluster
jobname=oozie_missing
jarclass=com.emg.etp.analysis.missing.MissingRoad
#jar包所在HDFS的絕對路徑
jarpath=${nameNode}/user/root/examples/apps/spark/missing_road.jar
#運行spark任務的參數，有的可能需要assembly相關的jar包，我的不需要
sparkopts=--driver-memory 1G --executor-memory 2G --executor-cores 2 --num-executors 20

3.2workflow.xml

<!--
  Licensed to the Apache Software Foundation (ASF) under one
  or more contributor license agreements.  See the NOTICE file
  distributed with this work for additional information
  regarding copyright ownership.  The ASF licenses this file
  to you under the Apache License, Version 2.0 (the
  "License"); you may not use this file except in compliance
  with the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
-->
<workflow-app xmlns="uri:oozie:workflow:1.0" name="spark-wf">
    <start to="spark-node"/>
    <action name="spark-node">
        <spark xmlns="uri:oozie:spark-action:1.0">
            <resource-manager>${resourceManager}</resource-manager>
            <name-node>${nameNode}</name-node>
            <configuration>
                <property>
                    <name>queueName</name>
                    <value>${queueName}</value>
                </property>
            </configuration>
            <master>${jobmaster}</master>
            <mode>${jobmode}</mode>
            <jar>${jarpath}</jar>
            <spark-opts>${sparkopts}</spark-opts>
            <arg>${input1}</arg>
        </spark>
        <!--<ok to="check-output"/>-->
        <ok to="end"/>
        <error to="fail"/>
    </action>

    <kill name="fail">
        <message>Shell action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
    </kill>
    <kill name="fail-output">
        <message>Incorrect output, expected [Hello Oozie] but was [${wf:actionData('shell-node')['my_output']}]</message>
    </kill>
    <end name="end"/>
</workflow-app>

啓動命令(注意 這裏的job.properties是linux下的路徑)
bin/oozie job -oozie http://xxxx:11000/oozie -config oozie-apps/job.properties -run

kill命令，jobid可以到oozie自帶的web界面查看
bin/oozie job -oozie http://xxxx:11000/oozie -kill 0000000-170724202624644-oozie-oozi-C

參考：https://blog.csdn.net/Abysscarry/article/details/81784179
https://blog.csdn.net/pt707621764/article/details/52437567