hadoop重啓虛機後hdfs中數據丟失，需要重新格式化問題

原創

2020-02-21 11:44

1，虛機每次重啓後，如果不格式化namenode，就錯誤，日誌如下：

INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemStateMBean and NameNodeMXBean INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /tmp/hadoop-sylar/dfs/name does not exist. ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed. org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-sylar/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.

2，解決方法，重新格式化：

~]$bin/hadoop namenode -format

3, 如果想徹底消除錯誤，並且每次重啓機器不格式化，需要修改core-site.xml 和 hdfs-site.xml，如下：

vm1:~/hadoop/conf$ cat core-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>



<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://192.168.1.105:9000</value>
</property>

<property>
    <name>hadoop.tmp.dir</name>
    <value>/home/hadoop/filesystem/tmp</value>
    <description> temporary directories.</description>
</property>

<property>
    <name>dfs.name.dir</name>
    <value>/home/hadoop/filesystem/name</value>
    <description>where on the local filesystem the DFS name node should store the name table</description>
</property>

<property>
    <name>dfs.data.dir</name>
    <value>/home/hadoop/filesystem/data</value>
    <description>where on the local filesystem an DFS data node should store its blocks.</description>
</property>

</configuration>

vm1:~/hadoop/conf$

~/hadoop/conf$ cat hdfs-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>



<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
<name>dfs.permissions</name>
<value>false</value>
</property>
</configuration>