新項目使用ogg抽數據按照年月日+小時的形式保存到hdfs，由於ogg數據是保留修改前數據和修改後數據的，所以採用json格式保存文本
{"table":"TEST.TT_SALES_RECORDS","op_type":"U","op_ts":"2020-05-19 02:05:03.000701","current_ts":"2020-05-19T10:05:10.427000","before":{"ID":178733,"PS_ORDER_NO":"PV2003110002","PS_ORDER_STATUS":35121004,"PS_ORDER_TYPE":null,"PS_ORDER_DATE":"2020-03-11 17:52:20","SALES_CONSULTANT":null,"BUYER_NAME":"豬哈哈"},"after":{"ID":178733,"PS_ORDER_NO":"PV2003110002","PS_ORDER_STATUS":35121004,"PS_ORDER_TYPE":null,"PS_ORDER_DATE":"2020-03-12 17:52:20","SALES_CONSULTANT":null,"BUYER_NAME":"豬堅強"}}
問題一：由於數據沒有主鍵，必須配置row_id
具體配置略
問題二：json格式數據中間沒有換行符
gg.handler.name.format=json
gg.handler.name.format.jsonDelimiter=CDATA[\N]
最終文本格式：
{"table":"TEST.TT_SALES_RECORDS","op_type":"U","op_ts":"2020-05-19 02:05:03.000701","current_ts":"2020-05-19T10:05:10.427000","tokens":{"DBROWID":"AABIl2AAiAABwGZAAB"},"before":{"ID":178733,"PS_ORDER_NO":"PV2003110002","PS_ORDER_STATUS":35121004,"PS_ORDER_TYPE":null,"PS_ORDER_DATE":"2020-03-11 17:52:20","SALES_CONSULTANT":null,"BUYER_NAME":"豬哈哈"},"after":{"ID":178733,"PS_ORDER_NO":"PV2003110002","PS_ORDER_STATUS":35121004,"PS_ORDER_TYPE":null,"PS_ORDER_DATE":"2020-03-12 17:52:20","SALES_CONSULTANT":null,"BUYER_NAME":"豬堅強"}}
然後使用sqoop把全量數據表按照rowid格式抽取到hdfs：
sqoop import --connect jdbc:oracle:thin:@//IP:1521/DB --username u --password p --query "SELECT rowidtochar(t.ROWID) as ROW_ID,t.* FROM SBPOPT.TT_TEST t where \$CONDITIONS " --delete-target-dir --target-dir /user/asmp/hive/ogg/tt_test --split-by bill_no --as-parquetfile -m 16
之後就可以用spark讀數據，解析修改數據&合併到全量數據~
注意事項：
一般源端只有主鍵，唯一鍵，外鍵開了附加日誌；
沒有主鍵的表開的是全列附件日誌（因爲開啓全列附加日誌會導致日誌量大很多，會影響源端數據庫的提交效率）
Tips:
鑑於上述情況需要解析數據的時候需要用代碼判斷是否只有主鍵！