Hive源碼編譯支持UDF

一、準備源碼

1. 下載

[hadoop@bigdata source]$ wget http://archive.cloudera.com/cdh5/cdh/5/hive-1.1.0-cdh5.15.1-src.tar.gz

2. 解壓

[hadoop@bigdata source]$ tar -zxvf hive-1.1.0-cdh5.15.1-src.tar.gz

3. 上傳UDF到hive-1.1.0-cdh5.15.1/ql/src/java/org/apache/hadoop/hive/ql/udf/目錄下
   注意修改UDF的package爲

package org.apache.hadoop.hive.ql.udf;

4. 註冊UDF

• 切換目錄到hive-1.1.0-cdh5.15.1/ql/src/java/org/apache/hadoop/hive/ql/exec/

[hadoop@bigdata exec]$ vi FunctionRegistry.java 

# 根據自己的UDF繼承的父類選擇對應的註冊方式：
system.registerGenericUDF("***", xxxx.class);

system.registerUDF("add_prefix", AddPrefixUDF.class, false);

注意將自己的UDF，import進來

import org.apache.hadoop.hive.ql.udf.AddPrefixUDF;

二、編譯及測試

1. 編譯

[hadoop@bigdata hive-1.1.0-cdh5.15.1]$ mvn clean package -DskipTests -Phadoop-2 -Pdist
# 出現如下字樣即編譯成功
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

# 編譯成功之後
[hadoop@bigdata hive-1.1.0-cdh5.15.1]$cd /packaging/target 

[hadoop@bigdata target]$ ll
total 139884
drwxrwxr-x 2 hadoop hadoop        27 Sep 23 14:52 antrun
drwxrwxr-x 3 hadoop hadoop        44 Sep 23 14:53 apache-hive-1.1.0-cdh5.15.1-bin
-rw-rw-r-- 1 hadoop hadoop 117647722 Sep 23 14:54 apache-hive-1.1.0-cdh5.15.1-bin.tar.gz
-rw-rw-r-- 1 hadoop hadoop  13339891 Sep 23 14:54 apache-hive-1.1.0-cdh5.15.1-jdbc.jar
-rw-rw-r-- 1 hadoop hadoop  12247215 Sep 23 14:54 apache-hive-1.1.0-cdh5.15.1-src.tar.gz
drwxrwxr-x 2 hadoop hadoop         6 Sep 23 14:53 archive-tmp
drwxrwxr-x 3 hadoop hadoop        21 Sep 23 14:52 maven-shared-archive-resources
drwxrwxr-x 3 hadoop hadoop        17 Sep 23 14:52 tmp
drwxrwxr-x 2 hadoop hadoop         6 Sep 23 14:52 warehouse

2. 測試

• apache-hive-1.1.0-cdh5.15.1-bin.tar.gz即爲我們需要的包
• 安裝部署參考Hive基於MySQL安裝
• 測試

[hadoop@bigdata apache-hive-1.1.0-cdh5.15.1-bin]$ bin/hive
hive (default)> show functions;
能夠看到一個名叫`add_prefix`的函數，

hive (default)> select * from test;
OK
test.name
zhang
chen
liu
Time taken: 0.386 seconds, Fetched: 3 row(s)
hive (default)> select add_prefix(name) from test;
OK
_c0
8_zhang
7_chen
6_liu
Time taken: 0.132 seconds, Fetched: 3 row(s)

完成！！