關於hadoop 1.2.1版本不支持“dfs.support.append”參數的解決辦法

最近在測試hadoop+fluentd方案，但是fluentd日誌收集系統，需要append功能選擇，寫日誌到HDFS中，官方給出的解決方案是：

修改hdfs-site.xml 文件,增加如下行：

<property>
  <name>dfs.webhdfs.enabled</name>
  <value>true</value>
</property>

<property>
  <name>dfs.support.append</name>
  <value>true</value>
</property>

<property>
  <name>dfs.support.broken.append</name>
  <value>true</value>
</property>

但是format namenode節點的時候會報錯：

[hadoop@node1 ~]$ hadoop namenode -format

14/12/25 10:35:25 INFO namenode.NameNode: STARTUP_MSG: 

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = node1.test.com/172.16.41.151

STARTUP_MSG:   args = [-format]

STARTUP_MSG:   version = 1.2.1

STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013

STARTUP_MSG:   java = 1.7.0_67

************************************************************/

14/12/25 10:35:25 INFO util.GSet: Computing capacity for map BlocksMap

14/12/25 10:35:25 INFO util.GSet: VM type       = 64-bit

14/12/25 10:35:25 INFO util.GSet: 2.0% max memory = 932184064

14/12/25 10:35:25 INFO util.GSet: capacity      = 2^21 = 2097152 entries

14/12/25 10:35:25 INFO util.GSet: recommended=2097152, actual=2097152

14/12/25 10:35:26 INFO namenode.FSNamesystem: fsOwner=hadoop

14/12/25 10:35:26 INFO namenode.FSNamesystem: supergroup=supergroup

14/12/25 10:35:26 INFO namenode.FSNamesystem: isPermissionEnabled=true

14/12/25 10:35:26 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100

14/12/25 10:35:26 WARN namenode.FSNamesystem: The dfs.support.append option is in your configuration, however append is not supported. This configuration option is no longer required to enable sync

14/12/25 10:35:26 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)

14/12/25 10:35:26 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0

14/12/25 10:35:26 INFO namenode.NameNode: Caching file names occuring more than 10 times 

14/12/25 10:35:26 INFO common.Storage: Image file /usr/local/hadoop_tmp/dfs/name/current/fsp_w_picpath of size 112 bytes saved in 0 seconds.

14/12/25 10:35:26 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/usr/local/hadoop_tmp/dfs/name/current/edits

14/12/25 10:35:26 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/usr/local/hadoop_tmp/dfs/name/current/edits

14/12/25 10:35:26 INFO common.Storage: Storage directory /usr/local/hadoop_tmp/dfs/name has been successfully formatted.

網上搜索半天，最好解決辦法如下：

用一下語句：

<property>  
         <name>dfs.support.broken.append</name>  
         <value>true</value>  
 </property>

然後再次格式化namenode，就不報錯了！

[hadoop@node1 conf]$ hadoop namenode -format

14/12/25 10:47:18 INFO namenode.NameNode: STARTUP_MSG: 

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG:   host = node1.test.com/172.16.41.151

STARTUP_MSG:   args = [-format]

STARTUP_MSG:   version = 1.2.1

STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013

STARTUP_MSG:   java = 1.7.0_67

************************************************************/

Re-format filesystem in /usr/local/hadoop_tmp/dfs/name ? (Y or N) Y

14/12/25 10:47:21 INFO util.GSet: Computing capacity for map BlocksMap

14/12/25 10:47:21 INFO util.GSet: VM type       = 64-bit

14/12/25 10:47:21 INFO util.GSet: 2.0% max memory = 932184064

14/12/25 10:47:21 INFO util.GSet: capacity      = 2^21 = 2097152 entries

14/12/25 10:47:21 INFO util.GSet: recommended=2097152, actual=2097152

14/12/25 10:47:21 INFO namenode.FSNamesystem: fsOwner=hadoop

14/12/25 10:47:21 INFO namenode.FSNamesystem: supergroup=supergroup

14/12/25 10:47:21 INFO namenode.FSNamesystem: isPermissionEnabled=true

14/12/25 10:47:21 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100

14/12/25 10:47:21 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)

14/12/25 10:47:21 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0

14/12/25 10:47:21 INFO namenode.NameNode: Caching file names occuring more than 10 times 

14/12/25 10:47:21 INFO common.Storage: Image file /usr/local/hadoop_tmp/dfs/name/current/fsp_w_picpath of size 112 bytes saved in 0 seconds.

14/12/25 10:47:21 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/usr/local/hadoop_tmp/dfs/name/current/edits

14/12/25 10:47:21 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/usr/local/hadoop_tmp/dfs/name/current/edits

14/12/25 10:47:21 INFO common.Storage: Storage directory /usr/local/hadoop_tmp/dfs/name has been successfully formatted.

14/12/25 10:47:21 INFO namenode.NameNode: SHUTDOWN_MSG: 

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at node1.test.com/172.16.41.151

************************************************************/