Oozie安裝

安裝Oozie的過程真的是一把血淚史，太TM反人類了，過程無比艱辛，感覺和Tez的安裝一樣，過程繁雜且問題頻出。

環境：

CentOS6.5 * 3

ZooKeeper 3.4.5-cdh5.7.1

Hadoop 2.6.0-cdh5.7.1

Oozie oozie-4.1.0-cdh5.7.1

1.下載oozie-4.1.0-cdh5.7.1.tar.gz

這個包非常大，有1.87G。大的原因主要是一些sharelib的jar包

2.配置oozie-site.xml

 <property>
        <name>oozie.service.JPAService.jdbc.driver</name>
        <value>com.mysql.jdbc.Driver</value>
    </property>
    <property>
         <name>oozie.service.JPAService.jdbc.url</name>
         <value>jdbc:mysql://{mysql的ip}:3306/oozie?createDatabaseIfNotExist=true</value>
    </property>
    <property>
        <name>oozie.service.JPAService.jdbc.username</name>
        <value>{mysql用戶名}</value>
    </property>
    <property>
         <name>oozie.service.JPAService.jdbc.password</name>
         <value>{mysql密碼}</value>
    </property>
    <property>
         <name>oozie.service.HadoopAccessorService.hadoop.configurations</name>
        <value>*=/usr/local/hadoop-2.6.0-cdh5.7.1/etc/hadoop/</value>
    </property>
    <property>
        <name>oozie.action.mapreduce.uber.jar.enable</name>
        <value>true</value>
     </property>
     <property>
       <name>oozie.service.HadoopAccessorService.supported.filesystems</name>
       <value>hdfs,viewfs</value>
     </property>

3.解壓oozie-hadooplibs-4.1.0-cdh5.7.1.tar.gz

4.創建$OOZIE_HOME/libext目錄，將第三步解壓的jar包放入

5.下載ext-2.2.js放入$OOZIE_HOME/oozie-server/webapps/ooziet目錄

6.將htrace-core4-4.0.1-incubating.jar和mysql驅動jar包放入$OOZIE_HOME/oozie-server/webapps/oozie/WEB-INF/lib

7.修改hadoop的core-site.xml配置

加上

 <property>
      <name>hadoop.proxyuser.{用戶名}.hosts</name>
      <value>*</value>
   </property>
   <property>
       <name>hadoop.proxyuser.{用戶名}.groups</name>
       <value>*</value>
   </property>

8.上傳sharelib到HDFS

 ./bin/oozie-setup.sh sharelib create -fs hdfs://hadoop-cluster -locallib /usr/local/oozie-4.1.0-cdh5.7.1/oozie-sharelib-4.1.0-cdh5.7.1-yarn.tar.gz

9.添加環境變量

export OOZIE_HOME=/usr/local/oozie-4.1.0-cdh5.7.1
export OOZIE_URL=http://node1:11000/oozie

10.生成腳本

$OOZIE_HOME/bin/ooziedb.sh create -sqlfile oozie.sql -run

11.啓動jobhistoryserver和timelineserver

$HADOOP_HOME/bin/mr-jobhistory-daemon.sh start historyserver
$HADOOP_HOME/bin/yarn-daemon.sh start timelineserver

12.啓動Oozie

$OOZIE_HOME/bin/oozie-start.sh

13.修改example的job.properties

修改

NameNode和JobTracker

15.上傳examples到HDFS

sh hdfs dfs -copyFromLocal /usr/local/oozie-4.1.0-cdh5.7.1/examples examples

16.執行example的mapreduce程序

./bin/oozie job -oozie http://node1:11000/oozie -config examples/apps/map-reduce/job.properties -run          

17.打開瀏覽器

http://node1:11000/oozie/