hadoop 2.2 搭建
http://blog.csdn.net/pelick/article/details/12065147
1
但是上面這個配置裏面有點小問題問題
- <property>
- <name>yarn.nodemanager.aux-services</name>
- <value>mapreduce.shuffle</value>
- </property>
改爲
- <property>
- <name>yarn.nodemanager.aux-services</name>
- <value>mapreduce_shuffle</value>
- </property>
java.lang.IllegalArgumentException: The ServiceName: mapreduce.shuffle set in yarn.nodemanager.aux-services is invalid.The
valid service name should only contain a-zA-Z0-9_
錯誤日誌在
hadoop-2.2.0/logs/yarn-hadoop-nodemanager-cs212.log
2
添加
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_PREFIX}/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib"
不然會出現下面的錯誤。
Hadoop 2.2.0 - warning: You have loaded library /home/hadoop/2.2.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard.
解決:
ou add a variable on .bash_profile
$ nano ~/.bash_profile
export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_PREFIX}/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_PREFIX/lib"
Apply it.
$ source ~/.bash_profile
And.. starting!!
$ start-all.sh
3.
centos 默認開啓了 ipv6 hadoop 8088 那些服務完全打不開。
centos6 linux禁用IPv6,解決大量 TIME_WAIT ::ffff:127.0.0.1:59299
http://www.qbage.com/post/48-centos6-disable-ipv6.html
eclipse hadoop fatjar 安
http://mclspace.com/2015/05/10/Fatjar-install/