目錄
系列：
大數據入門（一）環境搭建，VMware15+CentOS8.1 配置
https://blog.csdn.net/qq_34391511/article/details/104874044
大數據入門（二）Centos8,JDK 配置
https://blog.csdn.net/qq_34391511/article/details/104893587
大數據入門（三）CentOS 網絡配置
https://blog.csdn.net/qq_34391511/article/details/104895498
大數據入門（四）Hadoop 集羣搭建
https://blog.csdn.net/qq_34391511/article/details/104885278
大數據入門（五）windows 上搭建單機版 Hadoop2.8（踩坑記錄）
https://blog.csdn.net/qq_34391511/article/details/104948319
大數據入門（六）win10 對 Hadoop hdfs 的基本操作
https://blog.csdn.net/qq_34391511/article/details/105070955
大數據入門（七）win10 上 eclipse 使用 Hadoop 的配置
https://blog.csdn.net/qq_34391511/article/details/105066667
大數據入門（八）win10 下的 wordcount
https://blog.csdn.net/qq_34391511/article/details/105073076
大數據入門（九）基於 win10 的 Hadoop，java 代碼進行 hdfs 操作
https://blog.csdn.net/qq_34391511/article/details/105145380
liunx單機安裝參考：https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html#Standalone_Operation
安裝jdk
1、下載jdk的包並安裝
友情提示：jdk安裝路徑不要帶空格！！！！
我選擇的是jdk1.8；
官網上下載：
按提示安裝，配置好環境變量
2、配置環境變量
新建變量：JAVA_HOME
對應的值是其安裝路徑
如果路徑帶空格，參考下面的填坑記錄
新建變量：CLASSPATH
值：.;%JAVA_HOME%\lib;%JAVA_HOME%\lib\tools.jar
雙擊“path”新建添加%JAVA_HOME%\bin，完成，確定即可，注意這裏不要置頂，因爲帶“%”的放到了環境變量path的最前面，導致“環境變量path”變成了“系統/用戶變量path”。如果爲了讓優先級高一點，就放到前面一點就可以了。如果出現問題，解決方法可參考：win10解決環境變量變成系統/用戶變量的方法
3、驗證
如果失敗，參考：https://jingyan.baidu.com/album/36d6ed1f602f8c1bcf4883ed.html?picindex=4
Hadoop安裝和配置
1、下載Hadoop包及安裝
我選擇的是2.8版本的Hadoop，解壓縮即可。下載：https://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/
另外還需要下載winutils：https://github.com/steveloughran/winutils
找到相應版本，用裏面的bin去替代Hadoop文件裏面的bin，原因：Hadoop不支持win，需要改一下
2、環境變量配置
系統環境變量裏面新建“HADOOP_HOME”；值爲其路徑，我的是：D:\Learning\Hadoop\hadoop-2.8.5
“系統環境變量”的“path”（位置參考jdk的環境配置）裏添加：%HADOOP_HOME%\bin 、 %HADOOP_HOME%\sbin
#打開cmd：
#測試
hadoop
hadoop -version
Hadoop的文件修改
文件所在路徑：D:\Learning\Hadoop\hadoop-2.8.5\etc\hadoop（注意是：etc下的hadoop）,把以下內容替代掉對應的文件全部內容就行，另外需要注意core-site.xml、hdfs-site.xml這兩個在修改的時候有對應路徑的修改，需要提前建好文件夾，路徑自己設定，如果是別的版本，不替代的話，只需要在相應的<configuration><\configuration>裏面按照下面的進行內容填充即可。
1、core-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<!--注意！！！！提前建好文件夾，路徑自設 -->
<value>/D:/Learning/Hadoop/hadoop-2.8.5/workspace/tmp</value>
</property>
<property>
<name>dfs.name.dir</name>
<!--注意！！！！提前建好文件夾，路徑自設 -->
<value>/D:/Learning/Hadoop/hadoop-2.8.5/workspace/name</value>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
2、mapred-site.xml（hadoop2.8.5將mapred-site.xml.template重命名爲mapred-site.xml）
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
<property>
<name>mapred.job.tracker</name>
<value>hdfs://localhost:9001</value>
</property>
</configuration>
3、hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<!-- Put site-specific property overrides in this file. -->
<configuration>
<!-- 這個參數設置爲1，因爲是單機版hadoop -->
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.data.dir</name>
<!--注意！！！！提前建好文件夾，路徑自設 -->
<value>/D:/Learning/Hadoop/hadoop-2.8.5/workspace/data</value>
</property>
</configuration>
4、yarn-site.xml
<?xml version="1.0"?>
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
5、hadoop-env.cmd
修改set JAVA_HOME=這裏填本機的jdk路徑，如果jdk路徑有空格，參考下面解決方法
啓動Hadoop
#打開cmd：
#格式化hdfs
hdfs namenode -format
#到\hadoop-2.8.5\sbin目錄下，如果把sbin放到環境變量裏面，就不需要到sbin目錄下了：
start-all.cmd
啓動成功：
jps查看，每個都需要，纔是成功：
下面的圖，都是處於運行狀態：
若有掛閃或啓動失敗，則可能的原因及其解決辦法：
| 原因 | 解決方法 |
|---|---|
| 使用winutils的替換bin版本不匹配 | 找到合適的版本代替 |
| 前面五個文件配置錯誤 | 對照，自行重新修改 |
| 9000端口被佔用 | 關閉9000端口運行的程序或者重啓 |
| 其他 | 看掛閃文件的提示進行相應的修改 |
| 比較偏激的解決方法 | 刪除格式化HDFS後在前面新建的文件夾下的內容，然後重新格式化HDFS，再重新啓動即可 |
查看狀態的網址
查看集羣狀態：http://localhost:8088
查看Hadoop狀態：http://localhost:50070
stop-all.cmd關閉Hadoop
JDK路徑帶空格問題的解決方法
1、
java -version沒有用，直接在環境變量path裏面新建，添加jdk的路徑：
2、
按照提示，修改hadoop-env.cmd文件：
PROGRA~1是 Program Files 目錄的dos文件名模式下的縮寫 ，必須寫成這樣的形式
參考
windows上部署hadoop(單機版)
Windows環境下執行hadoop命令出現Error: JAVA_HOME is incorrectly set Please update D:\SoftWare\hadoop-2.6.0\conf\hadoop-env.cmd錯誤的解決辦法（圖文詳解）
Windows下安裝Hadoop2.8.5