http://www.360doc.com/content/14/0724/15/597197_396743445.shtml
用官方的Hadoop 2.1.0-beta安裝後,每次hadoop命令進去都會拋出這樣一個Warning
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
設置logger級別,看下具體原因
- 13/08/29 13:59:38 DEBUG util.NativeCodeLoader: Trying to load the custom-built native-hadoop library...
- 13/08/29 13:59:38 DEBUG util.NativeCodeLoader: Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: /usr/local/hadoop/hadoop-2.1.0-beta/lib/native/libhadoop.so.1.0.0: /usr/local/hadoop/hadoop-2.1.0-beta/lib/native/libhadoop.so.1.0.0: wrong ELF class: ELFCLASS32 (Possible cause: architecture word width mismatch)
- 13/08/29 13:59:38 DEBUG util.NativeCodeLoader: java.library.path=/usr/local/hadoop/hadoop-2.1.0-beta/lib/native/Linux-amd64-64:/usr/local/hadoop/hadoop-2.1.0-beta/lib/native
- 13/08/29 13:59:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
- file *
- libhadoop.a: current ar archive
- libhadooppipes.a: current ar archive
- libhadoop.so: symbolic link to `libhadoop.so.1.0.0'
- libhadoop.so.1.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, not stripped
- libhadooputils.a: current ar archive
- libhdfs.a: current ar archive
- libhdfs.so: symbolic link to `libhdfs.so.0.0.0'
- libhdfs.so.0.0.0: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, not stripped
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform...
using builtin-java classes where applicable
搜了好多文章,都說是跟系統位數有關係,我使用的是Centos 6.5 64位操作系統。前兩天在做Docker鏡像的時候發現了一個步驟可以解決這個問題,親試了一下,果然不再提示了。
首先下載hadoop-native-64-2.4.0.tar:
http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64-2.4.0.tar
如果你是hadoop2.6的可以下載下面這個:
http://dl.bintray.com/sequenceiq/sequenceiq-bin/hadoop-native-64-2.6.0.tar
tar -x hadoop-native-64-2.4.0.tar -C hadoop/lib/native/
在基於hadoop 2.0的hbase上執行命令,經常會看到如下提示:
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
1. 首先要檢查hdfs有沒有加載native lib庫,檢查hdfs的日誌,如果發現其中有:
ERROR org.apache.hadoop.io.nativeio.NativeIO: Unable to initialize NativeIO libraries
說明hdfs的native的lib沒有加載,檢查hadoop目錄下的lib/native下是否存在libhadoop.so,並且用file命令檢查該so是由於操作系統的平臺對應,默認下載的bin包中的so是32位的,如果用在64位的操作系統上,需要重新編譯。可以參見http://blog.csdn.net/lalaguozhe/article/details/10580727
2. 如果以上問題解決,hdfs的日誌中,沒有發現有關native的錯誤信息,則要檢查hbase的native庫配置,在hbase中默認的native庫路徑是hbase目錄下的lib/native/Linux-*(跟平臺類型相關),可以通過ps aux|grep `jps|grep HRegion|awk '{print $1}'`命令獲得region server的啓動項,檢查其中-Djava.library.path=的配置,我的服務器上是lib/native/Linux-amd64-64,於是在hbase目錄下創建lib/native/Linux-amd64-64這個目錄,並把hadoop目錄下的lib/natvie下的so拷貝過來,在集羣的所有結點上進行相同的操作,之後重啓hbase集羣,問題解決