教你安裝單機版Hadoop，初學必備

hadoop單機安裝

關注公衆號【愛做夢的錘子】回覆【java知識點】可下載我整理的完整版思維導圖

操作系統:centos7.4
cdh的hadoop發行版:hadoop-2.6-chd5.15.1

ssh免密登錄配置

• ssh安裝(已安裝跳過):yum install -y openssh-server openssh-clients
• 生成密鑰:ssh-keygen -t rsa
• 拷貝密鑰:ssh-copy-id -i ~/.ssh/id_rsa.pub [ip/hosts]

注意事項:

1. .ssh目錄的權限必須是700
2. .ssh/authorized_keys文件權限必須是600

hadoop環境變量配置

• hadoop-env.sh配置:${HADOOP_HOME}/etc/hadoop/hadoop-env.sh

export JAVA_HOME=/bigdata/jdk/jdk1.8.0_231
export HADOOP_HOME=/bigdata/hadoop/hadoop-2.6.0-cdh5.15.1
export HADOOP_CONF_DIR=/bigdata/hadoop/hadoop-2.6.0-cdh5.15.1/etc/hadoop

HDFS啓動

• core-site.xml配置:${HADOOP_HOME}/etc/hadoop/core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://主機名:9000</value>
    </property>
</configuration>

• hdfs-site.xml配置:${HADOOOP_HOME}/etc/hadoop/hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
    </property>
</configuration>

• slave文件配置:${HADOOP_HOME}/etc/hadoop/slaves

#根據自己的主機名填寫
${主機名}

• HDFS文件系統格式化:${HADOOP_HOME}/bin/hdfs namenode -format

• 啓動HDFS:${HADOOP_HOME}/sbin/start-dfs.sh

YARN啓動

• mapred-site.xml配置:${HADOOP_HOME}/etc/hadoop/mapred-site.xml

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>
</configuration>

• yarn-site.xml配置：${HADOOP_HOME}/etc/hadoop/yarn-site.xml

<configuration>
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

• 啓動yarn：${HADOOP_HOME}/sbin/start-yarn.sh

文章歡迎轉載，轉載請註明出處，個人公衆號【愛做夢的錘子】，全網同id，個站 http://te-amo.site，歡迎關注，裏面會分享更多有用知識，還有我的私密照片