Requested data length 88431035 is longer than maximum configured RPC length 67108864.

HDFS DataNode RPC 異常

Requested data length 88431035 is longer than maximum configured RPC length 67108864.

異常信息

2020-04-05 08:29:19,993 WARN org.apache.hadoop.ipc.Server: Requested data length 88431035 is longer than maximum configured RPC length 67108864.  RPC came from  xxx.xxx.xxxx.xxx
2020-04-05 08:29:19,993 INFO org.apache.hadoop.ipc.Server: Socket Reader #1 for port 8022: readAndProcess from client  xxx.xxx.xxxx.xxx threw exception [java.io.IOException: Requested data length 88431035 is longer than maximum configured RPC length 67108864.  RPC came from  xxx.xxx.xxxx.xxx]
java.io.IOException: Requested data length 88431035 is longer than maximum configured RPC length 67108864.  RPC came from  xxx.xxx.xxxx.xxx
	at org.apache.hadoop.ipc.Server$Connection.checkDataLength(Server.java:1665)
	at org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1727)
	at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:935)
	at org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:791)
	at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:762)

解決方式

修改NameNode的hdfs-site.xml配置文件，添加以下配置：

<property>
    <name>ipc.maximum.data.length</name>
    <value>134217728</value>
</property>

原因

因爲集羣規模比較小，但是小文件數量特別多，導致每個datanode中存在大量的數據塊信息，當datanode向namenode彙報塊信息時RPC最大設置爲64M，當集羣報錯時RPC的信息量爲80M左右。從而導致RPC異常，DataNode沒能把block信息上報給NameNode，NameNode就認爲block丟失了。

  public static final String IPC_MAXIMUM_DATA_LENGTH ="ipc.maximum.data.length";
  
  public static final int IPC_MAXIMUM_DATA_LENGTH_DEFAULT = 64 * 1024 * 1024;