sql_log_bin在GTID複製下的一個現象

背景

現網環境是M-M GTID+haproxy+組成的高可用需要做一個基於時間點的回檔

之前的方法:

1 新建出另外一組M-M GTID高可用集羣。我們將新的集羣成爲B,舊的集羣稱爲A

2  分別導入A的歷史備份到B集羣的兩個實例上,分開導入,導入期間雙主同步不關,但會關閉sql_log_bin

3 分別導入A的增量binlog到B集羣的兩個實例上,分開導入,導入期間雙主同步不關,但會關閉sql_log_bin


結果發現集羣B並沒有回檔到預期的效果,有時B上的主節點丟失增量數據,有時B上的備用節點丟失增量數據


原因分析

通過分析,發現是gtid複製引起的原因,即在B集羣的主節點上執行A的增量binlog時,即使這時我們將sql_log_bin關閉了,但是依舊會產生一些binlog,發現這些binlog都是一些帶A集羣節點uuid的空事務,而這些空事務也會產生一個個的gtid傳到B集羣的備用節點;當B集羣的備用節點執行增量binlog時,發現之前自己"似乎已經"執行過了,於是就跳過了這些增量操作,導致B幾羣的備用節點丟失這些增量數據

場景模擬

mysql> reset master;
Query OK, 0 rows affected (0.00 sec)


mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      151 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)


mysql> select * from xx;
Empty set (0.00 sec)


mysql> set sql_log_bin=0;
Query OK, 0 rows affected (0.00 sec)


mysql> insert into xx values(123);
Query OK, 1 row affected (0.00 sec)


mysql> select * from xx;
+------+
| id   |
+------+
|  123 |
+------+
1 row in set (0.00 sec)


mysql> show master status;
+------------------+----------+--------------+------------------+-------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 |      151 |              |                  |                   |
+------------------+----------+--------------+------------------+-------------------+
1 row in set (0.00 sec)

//可以發現當sql_log_bin關閉後,再執行簡單的insert是不會產生binlog的


mysql> SET @@SESSION.GTID_NEXT='8356b68c-5cfb-11e7-b683-6c92bf2352b8:1';
Query OK, 0 rows affected (0.00 sec)


mysql> begin;
Query OK, 0 rows affected (0.00 sec)


mysql> insert into xx values(234);
Query OK, 1 row affected (0.00 sec)


mysql> commit;
Query OK, 0 rows affected (10.00 sec)


mysql> SET @@SESSION.GTID_NEXT='AUTOMATIC';
Query OK, 0 rows affected (0.00 sec)


mysql> show master status;
+------------------+----------+--------------+------------------+----------------------------------------+
| File             | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set                      |
+------------------+----------+--------------+------------------+----------------------------------------+
| mysql-bin.000001 |      346 |              |                  | 8356b68c-5cfb-11e7-b683-6c92bf2352b8:1 |
+------------------+----------+--------------+------------------+----------------------------------------+
1 row in set (0.00 sec)

//可以發現當指定了另外一個uuid的gtid事務後,再insert記錄時,產生了binlog

SET @@SESSION.GTID_NEXT= '8356b68c-5cfb-11e7-b683-6c92bf2352b8:1'/*!*/;
# at 199
#170704 17:22:29 server id 168410320  end_log_pos 272 CRC32 0x232648e0 Query thread_id=1524016 exec_time=0 error_code=0
SET TIMESTAMP=1499160149/*!*/;
SET @@session.pseudo_thread_id=1524016/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1073741824/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=33/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
BEGIN
/*!*/;
# at 272
#170704 17:22:29 server id 168410320  end_log_pos 346 CRC32 0x5661cdc7 Query thread_id=1524016 exec_time=0 error_code=0
SET TIMESTAMP=1499160149/*!*/;
COMMIT
/*!*/;
DELIMITER ;
# End of log file
ROLLBACK /* added by mysqlbinlog */;
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;

//解析了產生的binlog後發現,這些binlog實際都是空事務,也就是說把insert into xx values(234);這條sql過濾了,但是gtid信息卻是不會過濾,記錄了下來,而這些信息傳到備庫,肯定會影響上述恢復的邏輯,備庫誤以爲這些gtid已經執行過了,遂不會再執行備庫的增量binlog


參數理解

sql_log_bin只是讓SQL內容不在binlog裏顯示,但是gtid的信息並無法屏蔽,查看官網對該參數解釋有點含糊不清

Setting this variable to 0 prevents GTIDs from being assigned to transactions in the binary log. If you
are using GTIDs for replication, this means that, even when binary logging is later enabled once again,
the GTIDs written into the log from this point do not account for any transactions that occurred in the
meantime—in effect, those transactions are lost.


解決方法

知道了原因,解決方法就非常多樣了,最簡單的方法,也就是上述代碼量更改最小的,就是mysqlbinlog在解析增量部分的binlog時,加上

 --skip-gtids=true,這樣解析出來的binlog就不會存在gtid信息,自然就不會有上述現象了




發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章