設置oozie工作流調度
workflow
Coordinator
bundle
workflow.xml
job.properties內容:
nameNode=hdfs://hadoop01:9000 --客戶端連接hdfs集羣
jobTracker=hadoop01:8032 --客戶端連接yarn集羣
queueName=default --調度隊列
filePath=/gp1819/oozie --oozie的根目錄
oozie.use.system.libpath=true –
oozie.libpath=${nameNode}/gp1819/oozielib --第三方依賴路徑
oozie.wf.application.path={nameNode}nameNode{filePath}/sqoop/ --工作流應用程序的目錄
Coordinator.xml
hdfs dfs -mkdir /gp1919
hdfs dfs -mkdir -p /gp1919/oozie /gp1919/oozielib
hdfs dfs -put $HIVE_HOME/lib/mysql-connector-java-5.1.32.jar /gp1919/oozielib/
1、創建sqoop的job
vi gp1919_sqoop_desc.sh
2、檢查oozie任務的配置是否正確
啓動oozie服務
oozied.sh start
啓動歷史服務器
mr-jobhistory-daemon.sh start historyserver
檢查配置
oozie validate sqoop/workflow.xml
報錯:
java.lang.IllegalArgumentException: Oozie URL is not available neither in command option or in the environment
解決:
export OOZIE_URL=http://hadoop01:11000/oozie
提交oozie的任務:
oozie job --oozie http://hadoop01:11000/oozie -config sqoop/job.properties -run
2、執行sqoop的job
oozie job --oozie http://hadoop01:11000/oozie -config sqoop-exec/job.properties -run
3、工作流調度
oozie job --oozie http://hadoop01:11000/oozie -config coor/job.properties -run