StreamSet之数据流从JDBC到Hbase

环境准备：

5.5.56-MariaDB

HDP-2.6.3.0-Hbase-1.1.2

zookeeper-3.4.10

 

1. 点击创建管道流
2. 随便写一个有意义的title和描述

 

 

1. 3.击创建好的管道流的title进入管理页面，然后选择All Stages，在下面输入jdbc

 

1. 4.首先配置JDBC

 

 

注意：sql一定要加上as，as “hbase列族:hbase列名” ,${OFFSET}和Initial Offset的值一样，

下边用户认证和增量模式打上勾，sql quer 要遵循以下格式：

SELECT * FROM table WHERE id > ${OFFSET} ORDER BY id，要不然会不成功。

 

数据库用户名和密码

1. 配置Hbase

下面3个都勾选

Ignore Missing Field：忽略空字段

Implicit field mapping：自动匹配字段映射，勾上这个我们就不用配置字段映射了(fields)

Ignore Invalid Column：忽略空行

1. 启动

注意：jdbc语句要和Hbase字段对应:
SELECT id as ID,
id as "f1:id",
name as "f1:name",
idcard as "f1:idcard",
sex as "f1:sex",
created_date as "f1:created_date",
last_modified_date as "f1:last_modified_date",
password as "f1:password",
username as "f1:username",
created_by_id as "f1:created_by_id",
last_modified_by_id as "f1:last_modified_by_id"
FROM user where id > ${OFFSET} ORDER BY id;