環境說明：
系統都爲CentOS 6.0
192.168.255.128 server01
192.168.255.130 server02
192.168.255.131 server03
1. server01設置hosts文件
- vi /etc/hosts
- # 添加以下主機和IP的對應關係
- # hadoop
- 192.168.255.128 server01
- 192.168.255.130 server02
- 192.168.255.131 server03
2. server01安裝java
- yum groupinstall "Additional Development"
- # 或者
- yum install java-1.6.0-openjdk-devel
在"Additional Development"組總包含了openjdk的環境。
3. server01設置java環境
- vi /etc/profile.d/java.sh
- # hadoop
- export JAVA_HOME="/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64"
- export PATH=$PATH:$JAVA_HOME/bin
- export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar
4. server01驗證JDK是否成功
- java -version
- java version "1.6.0_22"
- OpenJDK Runtime Environment (IcedTea6 1.10.4) (rhel-1.42.1.10.4.el6_2-x86_64)
- OpenJDK 64-Bit Server VM (build 20.0-b11, mixed mode)
5. server01添加hadoop用戶，並設置密碼
- useradd hadoop
- passwd hadoop
6. 創建目錄
- mkdir -p /data/hadoop/{tmp,hdfs/{name,data}}
- chown -R hadoop:hadoop /data/hadoo
目錄的所有這爲hadoop。
7. server01創建.ssh目錄和密鑰並添加環境變量
- su - hadoop
- ssh-keygen -t rsa
hadoop用戶添加環境變量
- vi .bashrc
- # hadoop
- export HADOOP_HOME=/usr/local/hadoop
- export PATH=$PATH:$HADOOP_HOME/bin
8. 在server02和server03上重複此操作
可以使用Xshell的"to all sessions"的功能同時執行以上命令。
9. 配置server01密鑰訪問server01，server02和server03
a. 配置server01訪問server01
- su - hadoop
- cd .ssh
- # 複製server01公鑰的內容
- vi id_rsa.pub
- ...
- # 創建authorized_keys2
- vi authorized_keys2
- # 粘貼server01的id_rsa.pub文件的內容
- chmod 0400 authorized_keys2
登錄測試
- whoami
- hadoop
- ssh hadoop@server01
b. 配置server01訪問server02
- su - hadoop
- cd .ssh
- vi authorized_keys2
- # 粘貼server01的id_rsa.pub文件的內容
- chmod 0400 authorized_keys2
測試從server01訪問server02
- whoami
- hadoop
- ssh hadoo@server02
c. 配置server01訪問server03
重複b步驟
10. server01下載hadoop包（注意，這些步驟只要在server01上執行，之後通過scp複製到server02和server03上即可）
- wget http://mirror.bjtu.edu.cn/apache/hadoop/common/hadoop-0.20.205.0/hadoop-0.20.205.0.tar.gz -P /usr/local/src
解壓，安裝
- cd /usr/local/src
- tar zxvf hadoop-0.20.205.0.tar.gz
- mv hadoop-0.20.205.0 /usr/local/hadoop
- chown -R hadoop:hadoop /usr/local/hadoop
11. server01配置hadoop配置文件
- su - hadoop
- whoami
- # 確認是hadoop用戶
- hadoop
a. 設置hadoop環境變量
- vi conf/hadoop-env.sh
- export JAVA_HOME=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64
配置文件分爲core-site.xml，hdfs-site.xml和mapred-site.xml
b. 配置core-site.xml
- <configuration>
- <property>
- <name>fs.default.name</name>
- <value>hdfs://server01:9000</value>
- </property>
- <property>
- <name>hadoop.tmp.dir</name>
- <value>/data/hadoop/tmp</value>
- </property>
- </configuration>
c. 配置hdfs-site.xml
- vi conf/hdfs-site.xml
- <configuration>
- <property>
- <name>dfs.replication</name>
- <value>3</value>
- </property>
- <property>
- <name>dfs.name.dir</name>
- <value>/data/hadoop/hdfs/name</value>
- </property>
- <property>
- <name>dfs.data.dir</name>
- <value>/data/hadoop/hdfs/data</value>
- </property>
- </configuration>
d. 設置mapred-site.xml
- vi conf/mapred-site.xml
- <configuration>
- <property>
- <name>mapred.job.tracker</name>
- <value>server01:9001</value>
- </property>
- </configuration>
e. 設置master和slave文件
- vi conf/master
- server01
- vi conf/slave
- server02
- server03
注意以上步驟只要在server01上執行即可。
11. server02和server03安裝hadoop包
- cd /usr/local/
- tar czvf ~/hadoop.tar.gz hadoop/
- # 複製包到server02和server03的主目錄
- scp ~/hadoop.tar.gz hadoop@server02:~/
- scp ~/hadoop.tar.gz hadoop@server03:~/
解壓安裝
- ssh hadoop@server02（or server03）
- tar zxvf hadoop.tar.gz
- su -
- mv /home/hadoop/hadoop /usr/local
- chown -R hadoop:hadoop /usr/local/hadoop
12. server02上在hadoop用戶下啓動hadoop
- start-all.sh
會顯示信息。
namenode初始化
- hadoop namenode -format
重新啓動hadoop
- stop-all.sh
- start-all.sh
13. 測試
- hadoop dfs -mkdir test
- hadoop dfs -copyFromLocal $HADOOP_HOME/conf test
14. Web訪問
設置端口隧道
訪問
Hadoop Map/Reduce Administration
在瀏覽集羣文件系統的時候報錯，請在自己的windows的hosts文件添加
- 192.168.255.128 server01
- 192.168.255.130 server02
- 192.168.255.131 server03