hadoop-ha僞分佈式平臺基於yarn

基於大數據僞分佈式平臺-HA搭建，看着有斷層可以看前面的平臺搭建過程。

nodemanage和datanode都是從節點角色，和datanode是1:1的關係，所以用了etc/slaves文件

• 修改mapred-site.xml

複製模板

cp mapred-site.xml.template mapred-site.xml

修改mapred-site.xml

<configuration> 
<property> 
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
 </property> 
</configuration>

• 修改yarn-site.xml

<configuration> 
    <property> 
        <name>yarn.nodemanager.aux-services</name> 
        <value>mapreduce_shuffle</value> 
    </property> 
    <property> 
        <name>yarn.resourcemanager.ha.enabled</name> 
        <value>true</value> 
    </property> 
    <property> 
        <name>yarn.resourcemanager.cluster-id</name> 
        <value>cluster1</value> 
    </property> 
    <property> 
        <name>yarn.resourcemanager.ha.rm-ids</name> 
        <value>rm1,rm2</value> 
    </property> 
    <property> 
        <name>yarn.resourcemanager.hostname.rm1</name> 
        <value>node3</value> 
    </property> 
    <property> 
        <name>yarn.resourcemanager.hostname.rm2</name> 
        <value>node4</value> 
    <property> 
    <property> 
        <name>yarn.resourcemanager.zk-address</name> 
        <value>node2:2181,node3:2181,node4:2181</value> 
    </property> 
</configuration>

• 分發給node2、node3、node4

scp yarn-site.xml mapred-site.xml root@node4:/opt/home/hadoop-2.6.5/etc/hadoop/

• 開啓yarn

start-yarn.sh(1)#因爲namenode 啓動的是自己節點的rm，但配置的是node3/4所以不會啓動 
yarn-daemon.sh start resourcemanager(3,4)#在start-dfs.sh之後開啓

• 修改一鍵啓動腳本

啓動

#!/bin/bash echo "start all zookeeper.." 
for i in {2..4}; 
do 
    ssh node$i "/opt/home/zookeeper-3.4.6/bin/zkServer.sh start"; 
done 
start-all.sh 
for i in {3..4};#node34 rm啓動 
do 
    ssh node$i "/opt/home/hadoop-2.6.5/sbin/yarn-daemon.sh start resourcemanager"; 
done 

關閉

#!/bin/bash 
echo "start all zookeeper.." 
for i in {2..4}; 
do 
    ssh node$i "/opt/home/zookeeper-3.4.6/bin/zkServer.sh start"; 
done 
start-all.sh 
for i in {3..4};#node34 rm關閉 
do 
    ssh node$i "/opt/home/hadoop-2.6.5/sbin/yarn-daemon.sh start resourcemanager"; 
done