StreamSet之數據流從JDBC到Hbase

環境準備：

5.5.56-MariaDB

HDP-2.6.3.0-Hbase-1.1.2

zookeeper-3.4.10

 

1. 點擊創建管道流
2. 隨便寫一個有意義的title和描述

 

 

1. 3.擊創建好的管道流的title進入管理頁面，然後選擇All Stages，在下面輸入jdbc

 

1. 4.首先配置JDBC

 

 

注意：sql一定要加上as，as “hbase列族:hbase列名” ,${OFFSET}和Initial Offset的值一樣，

下邊用戶認證和增量模式打上勾，sql quer 要遵循以下格式：

SELECT * FROM table WHERE id > ${OFFSET} ORDER BY id，要不然會不成功。

 

數據庫用戶名和密碼

1. 配置Hbase

下面3個都勾選

Ignore Missing Field：忽略空字段

Implicit field mapping：自動匹配字段映射，勾上這個我們就不用配置字段映射了(fields)

Ignore Invalid Column：忽略空行

1. 啓動

注意：jdbc語句要和Hbase字段對應:
SELECT id as ID,
id as "f1:id",
name as "f1:name",
idcard as "f1:idcard",
sex as "f1:sex",
created_date as "f1:created_date",
last_modified_date as "f1:last_modified_date",
password as "f1:password",
username as "f1:username",
created_by_id as "f1:created_by_id",
last_modified_by_id as "f1:last_modified_by_id"
FROM user where id > ${OFFSET} ORDER BY id;