Ranger-Yarn插件安裝

Ranger-Yarn插件ranger-1.2.0-yarn-plugin安裝到Yarn的所有ResourceManager節點，
其他的NodeManager節點不需要安裝。

1，安裝yarn-plugin

登陸hdfs安裝的用戶，garrison/zdh1234(用戶組hadoop),獲取安裝包解壓安裝
scp /home/backup/ranger/ranger-0.6.0-yarn-plugin.tar.gz .
tar –zxvf ranger-0.6.0-yarn-plugin.tar.gz
vi install.properties
修改的參數如下：

POLICY_MGR_URL=http://10.43.159.245:6080
SQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
REPOSITORY_NAME=yarndev
CUSTOM_USER=garrison
CUSTOM_GROUP=hadoop

安裝Ranger Yarn Plugin，注意: ./enable-yarn-plugin.sh script should be run as root.：
./enable-yarn-plugin.sh
創建完成後，需要重新啓動yarn.

將zdh-245的包考到zdh-240的garrison裏面
scp -r garrison@zdh-245:/home/garrison/ranger-0.6.0-yarn-plugin .
用root執行安裝腳本，並且重新啓動yarn.

提示：

插件安裝過程腳本做得幾件事情：
1，修改/etc/hadoop/yarn-site.xml,添加內容如下
<property>
        <name>yarn.acl.enable</name>
        <value>true</value>
    </property>
    <property>
      <name>yarn.authorization-provider</name>
      <value>org.apache.ranger.authorization.yarn.authorizer.RangerYarnAuthorizer</value> </property>
 
2，在/home/lzz/app/hadoop-2.7.1/etc/hadoop/中增加
-rwxr--r--. 1 lzz hadoop  9530 8月  26 17:38 ranger-yarn-audit.xml
-rwxr--r--. 1 lzz hadoop  2674 8月  26 17:38 ranger-yarn-security.xml
 
3，將對應jar包軟連接到/home/lzz/app/hadoop-2.7.1/share/hadoop/hdfs/lib/
 
lrwxrwxrwx. 1 root root        66 8月  26 17:38 ranger-yarn-plugin-impl -> /home/lzz/app/ranger-1.2.0-yarn-plugin/lib/ranger-yarn-plugin-impl
lrwxrwxrwx. 1 root root        76 8月  26 17:38 ranger-yarn-plugin-shim-1.2.0.jar -> /home/lzz/app/ranger-1.2.0-yarn-plugin/lib/ranger-yarn-plugin-shim-1.2.0.jar

2,註冊yarn對應服務

Ranger-Admin裏註冊yarn plugin的服務
YARN新建Service，修改如下

Service Name = yarnpdev
UserName = garrison
Password = zdh1234
YARN REST URL = http://10.43.159.240:8188

然後點擊TestConnection，成功即可保存。

關閉all-queue策略，
新建root.default策略,給mysql用戶提交隊列的權限。

3，提交作業
使用mysql用戶執行mapreduce任務,給mysql訪問hdfs相應目錄的權限：

export JAVA_HOME=/usr/share/java/jdk1.7.0_80
/home/garrison/hadoop-2.7.1/bin/hadoop jar /home/garrison/hadoop-2.7.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar wordcount -Dmapreduce.job.queuename=default hdfs://gagcluster/usr/wordcout.txt /usr/wordresult_002
/home/garrison/hadoop-2.7.1/bin/hadoop fs -text /usr/wordresult_002/part-r-00000

沒有權限提交作業會報錯：
User usersync cannot submit applications to queue root.default

注意：1，要先用戶訪問相應目錄權限

           2，測試（圓周率pi）時，使用lzz1用戶一直有提交權限，配置ranger對於deny-condition，禁止lzz1提交作業。

 

4，Yarn隊列權限支持通過capacity schedule queues實現
yarn-site.xml配置文件中加入配置項：

<property>
    <name>yarn.resourcemanager.scheduler.class</name>    
    <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
</property>

capacity-scheduler.xml配置如下，只允許garrison用戶提交作業：

<property>
  <name>yarn.scheduler.capacity.root.acl_submit_applications</name>
   <value>garrison</value>
   <description>
     The ACL of who can submit jobs to the root queue.
   </description>
 </property>
 <property>
  <name>yarn.scheduler.capacity.root.acl_administer_queue</name>
  <value>garrison</value>
  <description>
    The ACL of who can administer jobs on the default queue.
  </description>
</property>
<property>
  <name>yarn.scheduler.capacity.root.default.acl_submit_applications</name>
  <value>garrison</value>
  <description>
    The ACL of who can submit jobs to the default queue.
  </description>
</property>
<property>
  <name>yarn.scheduler.capacity.root.default.acl_administer_queue</name>
  <value>garrison</value>
  <description>
    The ACL of who can administer jobs on the default queue.
  </description>
</property>