版本匹配最重要
hive-hbase
配置：
把hbase對應jar包替換掉hive/lib下的包，
cp protobuf-java-2.4.0a.jar /opt/hadoop/hive/lib
啓動hive
hive --auxpath /opt/hadoop/hive/lib/hbase-0.94.10-security.jar,/opt/hadoop/hive/lib/hive-hbase-handler-0.10.0.jar
創建hbase的hive 外部表
hive> CREATE EXTERNAL TABLE f2 (name string, dd double)
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,percent:user_mer_dist")
> TBLPROPERTIES("hbase.table.name" = "f2_test_out");
OK
Time taken: 1.279 seconds
hive> select * from f2;
OK
13005400398_104440354113025_14007572 0.5
13005400398_898440354119665_96654013 0.5
13005400686_104440354113025_14007572 1.0
13005404751_104440354113025_14007572 1.0
13025423121_104440354113025_14007572 1.0
13025425645_104440354113025_14007572 1.0
18998919503_000000440301609_58403918 1.0
Time taken: 0.122 seconds
hive創建hbase表
CREATE TABLE hbase_hive_table(key int, value1 string)
STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
WITH SERDEPROPERTIES (
"hbase.columns.mapping" = ":key,a:b"
);
hive> desc hbase_hive_table;
OK
key int from deserializer
value1 string from deserializer
Time taken: 0.082 seconds
1.9.3p385 :011 > put 'hbase_hive_table','1','a:b','abc'
0 row(s) in 2.0550 seconds
1.9.3p385 :012 > scan 'hbase_hive_table'
ROW COLUMN+CELL
1 column=a:b, timestamp=1377262561618, value=abc
1 row(s) in 0.9260 seconds
hive> select * from hbase_hive_table;
OK
1 abc
Time taken: 1.625 seconds
--------------------------------------------------------------------------
運行job需要輔助jar包
hive --auxpath /opt/hadoop/hive/lib/hbase-0.94.10-security.jar,
/opt/hadoop/hive/lib/hive-hbase-handler-0.10.0.jar,
/opt/hadoop/hive/lib/zookeeper-3.4.5.jar,
/opt/hadoop/hive/lib/protobuf-java-2.4.0a.jar
hive> select count(*) from f2;
Total MapReduce CPU Time Spent: 2 seconds 610 msec
OK
7
