規避Debezium master purged GTID問題

MySQL開啓了gtid，debezium同步的時候報了錯：

[root@ali-37 ~]# curl localhost:18083/connectors/debezium-mysql-source-3310/status
{"name":"debezium-mysql-source-3310","connector":{"state":"RUNNING","worker_id":"172.16.6.37:18083"},"tasks":[{"state":"FAILED","trace":"org.apache.kafka.connect.errors.ConnectException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires. Error code: 1236; SQLSTATE: HY000.
	at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:200)
	at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:167)
	at io.debezium.connector.mysql.BinlogReader$ReaderThreadLifecycleListener.onCommunicationFailure(BinlogReader.java:955)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:921)
	at com.github.shyiko.mysql.binlog.BinaryLogClient.connect(BinaryLogClient.java:559)
	at com.github.shyiko.mysql.binlog.BinaryLogClient$7.run(BinaryLogClient.java:793)
	at java.lang.Thread.run(Thread.java:748)
Caused by: com.github.shyiko.mysql.binlog.network.ServerException: The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.
	at com.github.shyiko.mysql.binlog.BinaryLogClient.listenForEventPackets(BinaryLogClient.java:882)
	... 3 more
","id":0,"worker_id":"172.16.6.37:18083"}],"type":"source"} 

原因是因爲debezium僞裝成MySQL的slave去拉取master的binlog文件，但是master把binlog文件purge了（阿里雲數據庫DRDS每6個小時purge一次，也就是把以前的binlog刪除，防止文件無限增大）。

那麼GTID是必須的麼？
從Debezium官網（https://debezium.io/docs/connectors/mysql/#enabling-gtids-optional） 上看到：

Enabling GTIDs (optional)
The MySQL server can be configured to use GTID-based replication. Global transaction identifiers, or GTIDs, were introduced in MySQL 5.6.5, and they uniquely identify a transaction that occurred on a particular server within a cluster. Using GTIDs greatly simplifies replication and makes it possible to easily confirm whether masters and slaves are consistent. Note that if you’re using an earlier version of MySQL, you will not be able to enable GTIDs.

也就是說只是可選的。但是MySQL又開啓了，只好讓Debezium忽略GTID了。
通過在MySQL中執行如下命令查看GTID的"UUID:TID":

MySQL [(none)]> show variables like '%gtid%';
+----------------------------------+-----------------------------------------------+
| Variable_name                    | Value                                         |
+----------------------------------+-----------------------------------------------+
| binlog_gtid_simple_recovery      | ON                                            |
| enforce_gtid_consistency         | ON                                            |
| gtid_executed_compression_period | 1000                                          |
| gtid_mode                        | ON                                            |
| gtid_next                        | AUTOMATIC                                     |
| gtid_owned                       |                                               |
| gtid_purged                      | 9e7744a9-0e19-11e7-b734-00163e133a1d:1-340167 |
| session_track_gtids              | OFF                                           |
+----------------------------------+-----------------------------------------------+
8 rows in set (0.00 sec)

可以看到服務器的UUID是：9e7744a9-0e19-11e7-b734-00163e133a1d，於是在Debezium MySQL Connector的配置文件中增加了gtid.source.excludes配置：

{
 "name":"debezium-mysql-source-3310",
 "config":{
    "connector.class": "io.debezium.connector.mysql.MySqlConnector",
    "database.hostname": "rm-bp1**q.mysql.rds.aliyuncs.com",
    "database.port": "3306",
    "database.user": "**",
    "database.password": "**",
    "database.server.id": "184002",
    "database.server.name": "prod",
    "table.whitelist":"ann_release.*\\.(announcement_member|announcement_follower)",
    "database.history.kafka.bootstrap.servers": "ali-18:9092,ali-36:9092,ali-37:9092",
    "database.history.kafka.topic": "schema-changes-3310.prod" ,
    "include.schema.changes": "true" ,
    "mode":"incrementing",
    "incrementing.column.name":"id",
    "snapshot.mode":"when_needed",
    "gtid.source.excludes":"9e7744a9-0e19-11e7-b734-00163e133a1d.*",
    "database.history.skip.unparseable.ddl":"true"
  }
}

讓滿足"9e7744a9-0e19-11e7-b734-00163e133a1d.*"正則的GTID都被過濾掉。
這樣再啓動Debezium MySQL Connector就不報錯了。

這種解決方案只是規避，並沒有很好的解決問題，還需要再好好研究debeizum和mysql的底層細節。