hadoop在win10下的安裝

博客的質量有高有低，只能當做參考。在參考着博客和官方文檔進行探索之後，終於把hadoop僞集羣的生產環境安裝好了；大家可以做參考，我的系統是win10，jdk爲1.8；

本文只做參考，不提供下載地址，(#^.^#)；

1、首先我下載了hadoop-2.9，還有對應的windows.util，用來適配win10，雖然官方說hadoop2.x以後的版本可以在win上裸奔，但我爲了小心起見，就下載下來了，hadoop的win適配文件在github上有開源的哦，大家可以找一下順便學點東西；

2、下載好之後，把windowsutil.exe和hadoop.dll放在C：/windows/system32 文件下，並且配置好bin和sbin的環境變量；

3、配置好環境變量之後，配置xml文件，我把我的xml文件發出來供大家參考一下，xml作爲程序與人的交流處，還是應該好好理解一下，推薦大家認真閱讀一下xml文件的配置；

core-site.xml:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
	<property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost:9633</value>
        <description>服務器IP地址，其實也可以使用主機名</description>
    </property>
	<property>
        <name>hadoop.tmp.dir</name>
        <value>file:///H:/hadoop-2.9.2/tmp</value>
         <description>Abase for other temporary directories.</description>
    </property>
    <property>
        <name>io.file.buffer.size</name>
        <value>131072</value>
        <description>該屬性值單位爲KB，131072KB即爲默認的64M</description>
    </property>
</configuration>

hdfs-site.xml:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
        <description>分片數量，僞分佈式將其配置成1即可</description>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:///H:/hadoop-2.9.2/tmp/data/shr/dfs</value>
        <description>命名空間和事務在本地文件系統永久存儲的路徑</description>
    </property>
	<property>
        <name>dfs.datanode.data.dir</name>
        <value>file:///H:/hadoop-2.9.2/tmp/data/shr/DataNode</value>
    <description>DataNode在本地文件系統中存放塊的路徑</description>
    </property>
    <!--
	<property>
        <name>dfs.namenode.hosts</name>
        <value>localhost</value>
        <description>對應DataNode所在服務器主機名,可以用逗號分隔</description>
    </property>
    <property>
        <name>dfs.blocksize</name>
        <value>268435456</value>
        <description>大文件系統HDFS塊大小爲256M，默認值爲64M</description>
    </property>
    <property>
        <name>dfs.namenode.handler.count</name>
        <value>100</value>
        <description>更多的NameNode服務器線程處理來自DataNodes的RPCS</description>
    </property>
	-->
</configuration>

maperd-site.xml.template:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
    <property>
        <name>dfs.replication</name>
        <value>1</value>
        <description>分片數量，僞分佈式將其配置成1即可</description>
    </property>
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:///H:/hadoop-2.9.2/tmp/data/shr/dfs</value>
        <description>命名空間和事務在本地文件系統永久存儲的路徑</description>
    </property>
	<property>
        <name>dfs.datanode.data.dir</name>
        <value>file:///H:/hadoop-2.9.2/tmp/data/shr/DataNode</value>
    <description>DataNode在本地文件系統中存放塊的路徑</description>
    </property>
    <!--
	<property>
        <name>dfs.namenode.hosts</name>
        <value>localhost</value>
        <description>對應DataNode所在服務器主機名,可以用逗號分隔</description>
    </property>
    <property>
        <name>dfs.blocksize</name>
        <value>268435456</value>
        <description>大文件系統HDFS塊大小爲256M，默認值爲64M</description>
    </property>
    <property>
        <name>dfs.namenode.handler.count</name>
        <value>100</value>
        <description>更多的NameNode服務器線程處理來自DataNodes的RPCS</description>
    </property>
	-->
</configuration>

yarn-site.xml:

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>

<!-- Site specific YARN configuration properties -->
	<property>
        <name>yarn.resourcemanager.address</name>
        <value>localhost</value>
        <description>IP地址localhost也可替換爲ip地址，指定resourcemanager地址</description>
    </property>
	
    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
		<description>nomenodeManager獲取數據的方式是shuffle</description>
    </property>
	<property>  
		<name>yarn.nodemanager.resource.cpu-vcores</name>  
		<value>4</value>  
	</property>  
	<property>  
		<name>yarn.nodemanager.resource.memory-mb</name>  
		<value>8192</value>  
	</property>
	<property>    
        <name>yarn.log-aggregation-enable</name> 
        <value>true</value>    
		<description>yarn打印工作日誌</description>
    </property>
	<!--
    <property>
        <name>yarn.resourcemanager.scheduler.address</name>
        <value>localhost:18082</value>
        <description>IP地址localhost也可替換爲ip地址</description>
    </property>
    <property>
        <name>yarn.resourcemanager.resource-tracker.address</name>
        <value>localhost:18083</value>
        <description>IP地址localhost也可替換爲ip地址</description>
    </property>
    <property>
        <name>yarn.resourcemanager.admin.address</name>
        <value>localhost:18084</value>
        <description>IP地址localhost也可替換爲ip地址</description>
    </property>
    <property>
        <name>yarn.resourcemanager.webapp.address</name>
        <value>localhost:18085</value>
        <description>IP地址localhost也可替換爲ip地址</description>
    </property>
    <property>
        <name>yarn.resourcemanager.scheduler.class</name>
        <value>FairScheduler</value>
        <description>常用類：CapacityScheduler、FairScheduler、orFifoScheduler</description>
    </property>
    <property>
        <name>yarn.scheduler.minimum</name>
        <value>100</value>
        <description>單位：MB</description>
    </property>
    <property>
        <name>yarn.scheduler.maximum</name>
        <value>256</value>
        <description>單位：MB</description>
    </property>
    <property>
        <name>yarn.resourcemanager.nodes.include-path</name>
        <value>localhost</value>
        <description>對應服務器ip地址,可以用逗號分隔</description>
    </property>
	-->
	<!-- 配置DataManager
	<property>
        <name>yarn.nodemanager.resource.memory-mb</name>
        <value>256</value>
        <description>單位爲MB</description>
    </property>
    <property>
        <name>yarn.nodemanager.vmem-pmem-ratio</name>
        <value>90</value>
        <description>百分比</description>
    </property>
    <property>
        <name>yarn.nodemanager.local-dirs</name>
        <value>file:///H:/hadoop-2.9.2/tmp/nodemanager</value>
        <description>列表用逗號分隔</description>
    </property>
    <property>
        <name>yarn.nodemanager.log-dirs</name>
        <value>file:///H:/hadoop-2.9.2/tmp/nodemanager/logs</value>
        <description>列表用逗號分隔</description>
    </property>
    <property>
        <name>yarn.nodemanager.log.retain-seconds</name>
        <value>10800</value>
        <description>單位爲S</description>
    </property>
    -->
</configuration>

基本上xml中的各項配置都做了註解，大家可以根據自己的目錄進行修正；

4、配置好環境的可以直接在cmd中輸入 hdfs namenode -format，先將節點格式化，格式化完成後，start-all.cmd --> jps

(如果cmd命令窗口運行不成功的話，則在bin目錄中打開cmd，輸入 hadoop namenode -format，再進入sbin目錄下，start-all.cmd --> jps) ,出現

6832 DataNode
14804 NameNode
4484 Jps
13512 ResourceManager
4652 NodeManager

並且保證啓動成功；

5、輸入127.0.0.1:8088 和 127.0.0.1::50070 查看hadoop相關狀態

hadoop在win10下的安裝

DAPPER 事務 TRANSACTION

畢設 - 用戶登錄用cookie來實現會話跟蹤

畢設 - 網絡商城之環境搭建（SSM）

畢設 - 網絡商城開篇

面試過程中遇到的代碼問題 --- 查詢一個目錄下的所有文件和文件夾

面試過程中遇到的代碼問題 --- 兩個線程交替輸出

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結