由於hadoop2.2.0是32位的,需要編譯成64位的,現以做好編譯成功的rar文件
http://download.csdn.net/detail/u011053107/7242945
http://download.csdn.net/detail/u011053107/7242935
另外提供32位hadoop下載地址wget http://mirrors.hust.edu.cn/apache/hadoop/common/stable2/hadoop-2.2.0.tar.gz
=====正式開始=====
tar -xzvfhadoop-2.2.0.tar.gz
cdhadoop-2.2.0
vi etc/hadoop/hadoop-env.sh
修改:
export JAVA_HOME=/usr/local/java/java-7-oracle/bin/java
vi etc/hadoop/slaves
修改localhost爲host001
vi etc/hadoop/core-site.xml
增加:
<property>
<name>fs.defaultFS</name>
<value>hdfs://host001:9000</value>
</property>
cp etc/hadoop/mapred-site.xml.templateetc/hadoop/mapred-site.xml
vi etc/hadoop/mapred-site.xml
增加:
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapreduce.cluster.local.dir</name>
<value>/home/ysc/mapreduce/local</value>
</property>
vi etc/hadoop/yarn-site.xml
增加:
<property>
<name>yarn.resourcemanager.hostname</name>
<value>host001</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
vi etc/hadoop/hdfs-site.xml
增加:
<property>
<name>dfs.name.dir</name>
<value>/home/ysc/dfs/filesystem/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/ysc/dfs/filesystem/data</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
sudo vi /etc/proclusterIDfile
增加:
export HADOOP_PREFIX=/root/openstruct/hadoop-2.2.0
export HADOOP_COMMON_HOME=/root/openstruct/hadoop-2.2.0
export HADOOP_MAPRED_HOME=/root/openstruct/hadoop-2.2.0
export HADOOP_CONF_DIR=/root/openstruct/hadoop-2.2.0/etc/hadoop
export HADOOP_HDFS_HOME=/root/openstruct/hadoop-2.2.0
export HADOOP_YARN_HOME=/root/openstruct/hadoop-2.2.0
vi ./dfs/filesystem/data/current/VERSION
修改data節點的VERSION與name節點的clusterID值一致
clusterID=CID-889138e7-2a9c-4492-ac8c-63b339437563
source /etc/profile
格式化:
bin/hdfs namenode -format
啓動dfs:
sbin/start-dfs.sh
啓動yarn:
sbin/start-yarn.sh
啓動historyserver:
sbin/mr-jobhistory-daemon.sh start historyserver
運行wordcount:
echo "APDPlat is a java open source project, Application Product Development Platform." > text1.txt
echo "Yang Shangchuan is the founder of APDPlat whichwon the "2013 Outstanding Open Source Project" award." > text2.txt
bin/hadoop fs -mkdir /input
bin/hadoop fs -put text1.txt /input
bin/hadoop fs -put text2.txt /input
bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jarwordcount input output
訪問管理頁面:
ResourceManager:http://host001:8088
NodeManager:http://host001:8042
NameNode:http://host001:50070
NameNode:http://host001:50070
停止dfs:
sbin/stop-dfs.sh
停止yarn:
sbin/stop-yarn.sh
停止historyserver:
sbin/mr-jobhistory-daemon.sh stop historyserver
注意:要把這裏的主機host001和用戶ysc替換爲你自己的,在/etc/hosts文件中要把127.0.0.1 localhost改爲實際IP地址 host001
如果有需要在windows訪問本發佈管理平臺,需要將windows下C:\Windows\System32\drivers\etc的hosts文件新增內容爲192.168.6.141 host001