20_Mirroring

備註：文章編寫時間201904-201905期間，後續官方在github的更新沒有被寫入

~
~
鏡像[Mirroring]
注意：
1）它包含的規則可以隨時改變；
2）它不支持預備(prepare)語句；

一、mysql_query_rules的擴展[Extensions to mysql_query_rules]

修改了 mysql_query_rules 表，添加了2個列：
1） mirror_flagOUT
2） mirror_hostgroup

因此，mysql_query_rules 表的新的定義變爲：

Admin> show create table mysql_query_rules\G
*************************** 1. row ***************************
       table: mysql_query_rules
Create Table: CREATE TABLE mysql_query_rules (
    rule_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 0,
    username VARCHAR,
    schemaname VARCHAR,
    flagIN INT CHECK (flagIN >= 0) NOT NULL DEFAULT 0,
    client_addr VARCHAR,
    proxy_addr VARCHAR,
    proxy_port INT CHECK (proxy_port >= 0 AND proxy_port <= 65535), digest VARCHAR,
    match_digest VARCHAR,
    match_pattern VARCHAR,
    negate_match_pattern INT CHECK (negate_match_pattern IN (0,1)) NOT NULL DEFAULT 0,
    re_modifiers VARCHAR DEFAULT 'CASELESS',
    flagOUT INT CHECK (flagOUT >= 0), 
    replace_pattern VARCHAR CHECK(CASE WHEN replace_pattern IS NULL THEN 1 WHEN replace_pattern IS NOT NULL AND match_pattern IS NOT NULL THEN 1 ELSE 0 END),
    destination_hostgroup INT DEFAULT NULL,
    cache_ttl INT CHECK(cache_ttl > 0),
    cache_empty_result INT CHECK (cache_empty_result IN (0,1)) DEFAULT NULL,
    cache_timeout INT CHECK(cache_timeout >= 0),
    reconnect INT CHECK (reconnect IN (0,1)) DEFAULT NULL,
    timeout INT UNSIGNED CHECK (timeout >= 0),
    retries INT CHECK (retries>=0 AND retries <=1000),
    delay INT UNSIGNED CHECK (delay >=0),
    next_query_flagIN INT UNSIGNED,
    mirror_flagOUT INT UNSIGNED,
    mirror_hostgroup INT UNSIGNED,
    error_msg VARCHAR,
    OK_msg VARCHAR,
    sticky_conn INT CHECK (sticky_conn IN (0,1)),
    multiplex INT CHECK (multiplex IN (0,1,2)),
    gtid_from_hostgroup INT UNSIGNED,
    log INT CHECK (log IN (0,1)),
    apply INT CHECK(apply IN (0,1)) NOT NULL DEFAULT 0,
    comment VARCHAR)
1 row in set (0.00 sec)

二、功能實現概述[Implementation overview]

當爲匹配查詢的規則設置了 mirror_flagOUT 或 mirror_hostgroup 時，將自動啓用鏡像實時查詢功能。

請注意：
如果在規則中設置了 mirror_flagOUT 或 mirror_hostgroup 時又設置了replace_pattern：也就是如果原始查詢的文本內容(SQL語句)已被重寫(即查詢規則設置了replace_pattern對原始SQL進行重寫/替換)，則會爲最終執行的查詢(被修改後的語句)啓用鏡像查詢功能，而會爲原始SQL啓用鏡像查詢：即如果查詢(SQL語句)在根據digest、match_digest或match_pattern匹配到了查詢規則，卻發現該查詢規則設置了replace_pattern對匹配到的查詢SQL進行重寫，那麼，這時鏡像邏輯將應用於被重寫後的查詢(SQL)。雖然被鏡像的查詢(即原始SQL語句)可以被重新編寫或修改，但只要設置了mirror_flagOUT 或 mirror_hostgroup就會啓用鏡像查詢功能。詳情在後面。

如果源查詢(SQL)與多個查詢規則匹配，則可能會多次更改 mirror_flagOUT 或 mirror_hostgroup 。

鏡像邏輯如下：
1）如果在處理源查詢時設置了 mirror_flagOUT 或 mirror_hostgroup (查詢規則設置的)，則會創建一個新的mysql會話。
2）新的mysql會話將獲得原始mysql會話的所有相同屬性：相同的憑據，庫名，默認主機組等（注意：charset當前未被複制）
3）如果在原始會話中設置了 mirror_hostgroup ，則新會話將其默認主機組更改爲 mirror_hostgroup 。
4）如果未設置 mirror_flagOUT ，則新會話將針對定義的 mirror_hostgroup 執行原始查詢。
5）如果在原始會話中設置了 mirror_flagOUT ，則新的mysql會話將嘗試根據原始會話的mirror_flagOUT的值在 mysql_query_rules 中查找FlagIN值與之相等的查詢規則(即查找FlagIN=mirror_flagOUT的規則)；然後將鏡像查詢請求發送到這個規則中進行處理：這樣就可以修改查詢，如重寫查詢，或再次更改 hostgroup 。
(參考後面的<七、高級示例：使用鏡像測試查詢重寫>內容)

三、案例一：原始查詢與鏡像查詢選擇同一主機組[Mirror selects to same hostgroup]

在這個非常簡單的示例中，我們將把所有SELECT語句發送到hostgroup10，包括原始語句和鏡像語句。

1、設置查詢規則

1）設置查詢規則

將原始查詢與鏡像查詢指向同一主機組：

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+------------------+-------+
| 5       | 1      | NULL          | NULL                  | NULL             | 1     |
+---------+--------+---------------+-----------------------+------------------+-------+
1 row in set (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply) VALUES (6,1,'^SELECT',10,10,1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+------------------+-------+
| 5       | 1      | NULL          | NULL                  | NULL             | 1     |
| 6       | 1      | ^SELECT       | 10                    | 10               | 1     |
+---------+--------+---------------+-----------------------+------------------+-------+
2 rows in set (0.00 sec)

2）加載配置到RUNTIME層

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM runtime_mysql_query_rules ;
+---------+--------+---------------+-----------------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+------------------+-------+
| 5       | 1      | NULL          | NULL                  | NULL             | 1     |
| 6       | 1      | ^SELECT       | 10                    | 10               | 1     |
+---------+--------+---------------+-----------------------+------------------+-------+
2 rows in set (0.00 sec)

2、連接ProxySQL執行查詢

從mysql會話中我們將運行一些查詢：

1）使用業務賬號連入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空並查看當前以執行SQL的查詢概要統計：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 10        | 1          | sbtest     | show databases                 |
| 10        | 1          | sbtest     | show tables                    |
+-----------+------------+------------+--------------------------------+
2 rows in set (0.00 sec)

3）業務賬號執行查詢

mysql> SELECT id FROM sbtest1 LIMIT 3;
+------+
| id   |
+------+
| 1891 |
| 1511 |
| 8032 |
+------+
3 rows in set (0.00 sec)

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 10        | 2          | sbtest     | SELECT id FROM sbtest1 LIMIT ? |
+-----------+------------+------------+--------------------------------+
1 row in set (0.00 sec)

我們可以看到 SELECT 語句被執行了兩次！！

5）作爲附加測試，我們重新運行相同的查詢：

mysql> SELECT id FROM sbtest1 LIMIT 3;
+------+
| id   |
+------+
| 1891 |
| 1511 |
| 8032 |
+------+
3 rows in set (0.00 sec

6）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 10        | 4          | sbtest     | SELECT id FROM sbtest1 LIMIT ? |
+-----------+------------+------------+--------------------------------+
1 row in set (0.01 sec)

count_star是我們執行查詢次數的兩倍，因爲它是鏡像的。值得注意的是，ProxySQL會收集原始查詢和鏡像查詢的指標。

四、案例二：原始查詢與鏡像查詢選擇到不同的主機組[Mirror SELECT to different hostgroup]

在此示例中，我們將重新配置proxysql以將所有SELECT語句發送到hostgroup10，但要在hostgroup20上執行鏡像查詢：

1、設置查詢規則

1）設置查詢規則

Admin> DELETE FROM mysql_query_rules;
Query OK, 2 rows affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply) VALUES (5,1,'^SELECT',10,20,1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+------------------+-------+
| 5       | 1      | ^SELECT       | 10                    | 20               | 1     |
+---------+--------+---------------+-----------------------+------------------+-------+
1 row in set (0.00 sec)

2）加載配置到RUNTIME層

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

2、連接ProxySQL執行查詢

1）使用業務賬號連入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sbtest             |
+--------------------+
2 rows in set (0.00 sec)

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+----------------+
| hostgroup | count_star | schemaname | digest_text    |
+-----------+------------+------------+----------------+
| 10        | 1          | sbtest     | show databases |
| 10        | 1          | sbtest     | show tabels    |
| 10        | 2          | sbtest     | show tables    |
+-----------+------------+------------+----------------+
3 rows in set (0.00 sec)

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
Empty set (0.00 sec)

3）業務賬號執行查詢

從mysql客戶端我們現在可以運行一些查詢（爲簡單起見，我們運行相同）：

mysql> SELECT id FROM sbtest1 LIMIT 3;
+------+
| id   |
+------+
| 1891 |
| 1511 |
| 8032 |
+------+
3 rows in set (0.00 sec)

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 20        | 1          | sbtest     | SELECT id FROM sbtest1 LIMIT ? |
| 10        | 1          | sbtest     | SELECT id FROM sbtest1 LIMIT ? |
+-----------+------------+------------+--------------------------------+
2 rows in set (0.00 sec)

可以看到ProxySQL向hostgroup10和hostgroup20發送了相同的相同查詢！

五、案例三：重寫源查詢和鏡像[rewrite both source query and mirror]

在這個例子中，我們將重寫原始查詢，然後鏡像它：爲簡單起見，我們將對錶 sbtest[0-9]+ 的操作重寫爲對錶 sbtest3 的操作：

1、設置查詢規則

1）設置查詢規則

Admin> DELETE FROM mysql_query_rules;
Query OK, 1 row affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,replace_pattern,mirror_hostgroup,apply) VALUES (5,1,'sbtest[0-9]+',10,'sbtest3',20,1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,replace_pattern,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+-----------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | replace_pattern | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+-----------------+------------------+-------+
| 5       | 1      | sbtest[0-9]+  | 10                    | sbtest3         | 20               | 1     |
+---------+--------+---------------+-----------------------+-----------------+------------------+-------+
1 row in set (0.00 sec)

2）加載配置到RUNTIME層

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

2、連接ProxySQL執行查詢

1）使用業務賬號連入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sbtest             |
+--------------------+
2 rows in set (0.00 sec)

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest_reset WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+----------------+
| hostgroup | count_star | schemaname | digest_text    |
+-----------+------------+------------+----------------+
| 10        | 1          | sbtest     | show databases |
| 10        | 2          | sbtest     | show tables    |
+-----------+------------+------------+----------------+
2 rows in set (0.00 sec)

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
Empty set (0.01 sec)

3）業務賬號執行查詢

從mysql客戶端我們可以運行通常的查詢：

mysql> SELECT id FROM sbtest1 LIMIT 3;
+------+
| id   |
+------+
| 1149 |
| 9825 |
| 5704 |
+------+
3 rows in set (0.00 sec)

正如預期的那樣，這次查詢輸出的結果與前一個不同，因爲現在重寫了原始查詢。表名是是查詢了sbtest1其實被改寫到了sbtest3;通過下面內容可以驗證。

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

Admin> SELECT hostgroup,count_star,schemaname,digest_text FROM stats_mysql_query_digest WHERE schemaname='sbtest' ORDER BY digest;
+-----------+------------+------------+--------------------------------+
| hostgroup | count_star | schemaname | digest_text                    |
+-----------+------------+------------+--------------------------------+
| 20        | 1          | sbtest     | SELECT id FROM sbtest3 LIMIT ? |
| 10        | 1          | sbtest     | SELECT id FROM sbtest3 LIMIT ? |
+-----------+------------+------------+--------------------------------+
2 rows in set (0.01 sec)

正如所料，修改後的查詢在兩個主機組上執行，而且原始SQL中的表名sbtest1被重寫爲了sbtest3。

六、僅重寫鏡像查詢[rewrite mirror query only]

在此示例中，我們將只重寫鏡像查詢。
這非常有用，例如，我們想要了解重寫查詢的性能，或者新索引是否會提高性能。
在此示例中，我們將以使用和不使用索引來比較相同查詢的性能。當然，我們也會將查詢發送到相同的主機組。

1、設置查詢規則

1）設置查詢規則

創建以下規則（rule_id = 5）：
1）匹配 FROM sbtest1 ;
2）設置destination_hostgroup = 20 ;
3）設置mirror_flagOUT=100 ;
4）不設置mirror_hostgroup ;

Admin> DELETE FROM mysql_query_rules;
Query OK, 1 row affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,apply) VALUES (5,1,'FROM sbtest1 ',20,100,1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+---------------+-----------------------+----------------+------------------+-------+
| rule_id | active | match_pattern | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply |
+---------+--------+---------------+-----------------------+----------------+------------------+-------+
| 5       | 1      | FROM sbtest1  | 20                    | 100            | NULL             | 1     |
+---------+--------+---------------+-----------------------+----------------+------------------+-------+
1 row in set (0.00 sec)

由於設置了 mirror_flagOUT ，因此將創建一個新會話來運行相同的查詢。但是，因爲未設置 mirror_hostgroup ，所以會根據當前用戶在 mysql_users 中設置的默認主機組，將查詢將被髮送到其默認主機組上。相反的，如果我們希望將鏡像查詢發送到與原始主機組相同的主機組。我們可以在rule_id = 5的規則中設置mirror_hostgroup，或者創建
一個新規則。這裏，選擇後者，我們將創建一個新規則來匹配重寫查詢：

Admin> INSERT INTO mysql_query_rules (rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,apply) VALUES (10,1,100,'FROM sbtest1 ',20,'FROM sbtest1 IGNORE INDEX(k_1) ',1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id,active,flagIN,match_pattern,replace_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
| rule_id | active | flagIN | match_pattern | replace_pattern                 | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply |
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
| 5       | 1      | 0      | FROM sbtest1  | NULL                            | 20                    | 100            | NULL             | 1     |
| 10      | 1      | 100    | FROM sbtest1  | FROM sbtest1 IGNORE INDEX(k_1)  | 20                    | NULL           | NULL             | 1     |
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
2 rows in set (0.00 sec)

或查看規則的全部信息：

Admin> SELECT * FROM mysql_query_rules \G 
*************************** 1. row ***************************
              rule_id: 5
               active: 1
             username: NULL
           schemaname: NULL
               flagIN: 0
          client_addr: NULL
           proxy_addr: NULL
           proxy_port: NULL
               digest: NULL
         match_digest: NULL
        match_pattern: FROM sbtest1 
 negate_match_pattern: 0
         re_modifiers: CASELESS
              flagOUT: NULL
      replace_pattern: NULL
destination_hostgroup: 20
            cache_ttl: NULL
   cache_empty_result: NULL
        cache_timeout: NULL
            reconnect: NULL
              timeout: NULL
              retries: NULL
                delay: NULL
    next_query_flagIN: NULL
       mirror_flagOUT: 100
     mirror_hostgroup: NULL
            error_msg: NULL
               OK_msg: NULL
          sticky_conn: NULL
            multiplex: NULL
  gtid_from_hostgroup: NULL
                  log: NULL
                apply: 1
              comment: NULL
*************************** 2. row ***************************
              rule_id: 10
               active: 1
             username: NULL
           schemaname: NULL
               flagIN: 100
          client_addr: NULL
           proxy_addr: NULL
           proxy_port: NULL
               digest: NULL
         match_digest: NULL
        match_pattern: FROM sbtest1 
 negate_match_pattern: 0
         re_modifiers: CASELESS
              flagOUT: NULL
      replace_pattern: FROM sbtest1 IGNORE INDEX(k_1) 
destination_hostgroup: 20
            cache_ttl: NULL
   cache_empty_result: NULL
        cache_timeout: NULL
            reconnect: NULL
              timeout: NULL
              retries: NULL
                delay: NULL
    next_query_flagIN: NULL
       mirror_flagOUT: NULL
     mirror_hostgroup: NULL
            error_msg: NULL
               OK_msg: NULL
          sticky_conn: NULL
            multiplex: NULL
  gtid_from_hostgroup: NULL
                  log: NULL
                apply: 1
              comment: NULL
2 rows in set (0.00 sec)

需要注意的是，在rule_id = 10的規則中，鏡像查詢將匹配該規則[因爲mirror_flagOUT(5號規則)=flagIN(100=10號規則)]，此時我們需要設置 destination_hostgroup 而不是 mirror_hostgroup ：只應該爲原始查詢設置mirror_hostgroup ，以便立即可以將鏡像查詢請求發送到指定位置，而無需 mysql_query_rules 中的其他額外規則介入。

2）加載配置到RUNTIME層

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,active,flagIN,match_pattern,replace_pattern,destination_hostgroup,mirror_flagOUT,mirror_hostgroup,apply FROM runtime_mysql_query_rules ;
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
| rule_id | active | flagIN | match_pattern | replace_pattern                 | destination_hostgroup | mirror_flagOUT | mirror_hostgroup | apply |
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
| 5       | 1      | 0      | FROM sbtest1  | NULL                            | 20                    | 100            | NULL             | 1     |
| 10      | 1      | 100    | FROM sbtest1  | FROM sbtest1 IGNORE INDEX(k_1)  | 20                    | NULL           | NULL             | 1     |
+---------+--------+--------+---------------+---------------------------------+-----------------------+----------------+------------------+-------+
2 rows in set (0.00 sec)

配置已生效！

2、業務賬號執行查詢

1）使用業務賬號連入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sbtest             |
+--------------------+
2 rows in set (0.00 sec)

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset;
+----------+
| COUNT(*) |
+----------+
| 0        |
+----------+
1 row in set (0.00 sec)

3）業務賬號執行查詢

從mysql客戶端我們可以運行通常的查詢：

mysql> SELECT id FROM sbtest1 ORDER BY k DESC LIMIT 3;
+------+
| id   |
+------+
| 5076 |
| 5121 |
| 8573 |
+------+
3 rows in set (0.00 sec)

mysql> SELECT id,k FROM sbtest1 ORDER BY k DESC LIMIT 3;
+------+------+
| id   | k    |
+------+------+
| 5076 | 7672 |
| 5121 | 7655 |
| 8573 | 7565 |
+------+------+
3 rows in set (0.01 sec)

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

Admin> SELECT hostgroup,count_star,sum_time,digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC;
+-----------+------------+----------+--------------------------------------------------------------------+
| hostgroup | count_star | sum_time | digest_text                                                        |
+-----------+------------+----------+--------------------------------------------------------------------+
| 20        | 1          | 3997     | SELECT id,k FROM sbtest1 IGNORE INDEX(k_1) ORDER BY k DESC LIMIT ? |
| 20        | 1          | 3997     | SELECT id FROM sbtest1 IGNORE INDEX(k_1) ORDER BY k DESC LIMIT ?   |
| 20        | 1          | 615      | SELECT id FROM sbtest1 ORDER BY k DESC LIMIT ?                     |
| 20        | 1          | 342      | SELECT id,k FROM sbtest1 ORDER BY k DESC LIMIT ?                   |
+-----------+------------+----------+--------------------------------------------------------------------+
4 rows in set (0.01 sec)

表stats_mysql_query_digest 的結果表明：
1）原始查詢被鏡像了；
2）原始查詢未被重寫(而是鏡像)；
3）鏡像查詢被重寫了(帶了規則裏的IGNORE語句)；
4）鏡像查詢速度要慢得多，因爲忽略了索引；

七、高級示例：使用鏡像測試查詢重寫[Advanced example: use mirroring to test query rewrite]

在處理鏡像時，我被問到一個完全不同的問題，與查詢重寫有關：如何知道給定的正則表達式是否與給定的查詢匹配，並驗證重寫模式是否正確？
更具體地說，問題是要了解重寫是否正確而不影響實時流量。雖然鏡像最初並不是爲此設計的，但它可以回答這個問題。

在這個例子中，我們將編寫一個規則來匹配所有SELECT，"鏡像"它們，並嘗試重寫它們。

1、設置查詢規則

1）設置查詢規則

Admin> DELETE FROM mysql_query_rules;
Query OK, 2 rows affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,match_pattern,destination_hostgroup,mirror_flagOUT,apply) VALUES (5,1,'^SELECT ',20,100,1);
Query OK, 1 row affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,apply) VALUES 
(10,1,100,'^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$',20,'SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c',1);
Query OK, 1 row affected (0.00 sec)

查看設置結果：

Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM mysql_query_rules \G
*************************** 1. row ***************************
              rule_id: 5
               active: 1
               flagIN: 0
        match_pattern: ^SELECT 
destination_hostgroup: 20
      replace_pattern: NULL
       mirror_flagOUT: 100
                apply: 1
*************************** 2. row ***************************
              rule_id: 10
               active: 1
               flagIN: 100
        match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
      replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
       mirror_flagOUT: NULL
                apply: 1
2 rows in set (0.00 sec)

2）加載配置到RUNTIME層

Admin> LOAD MYSQL QUERY RULES TO RUNTIME ;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM runtime_mysql_query_rules \G
*************************** 1. row ***************************
              rule_id: 5
               active: 1
               flagIN: 0
        match_pattern: ^SELECT 
destination_hostgroup: 20
      replace_pattern: NULL
       mirror_flagOUT: 100
                apply: 1
*************************** 2. row ***************************
              rule_id: 10
               active: 1
               flagIN: 100
        match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
      replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c
       mirror_flagOUT: NULL
                apply: 1
2 rows in set (0.00 sec)

上面的正則表達式非常複雜，這就是爲什麼鏡像查詢功能有用之處，它不是直接重寫實時流量。

2、業務賬號執行查詢

1）使用業務賬號連入ProxySQL

# mysql -h 188.188.0.71 -P 6033 -umsandbox -p

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| sbtest             |
+--------------------+
2 rows in set (0.00 sec)

mysql> use sbtest;

mysql> show tables;
+------------------+
| Tables_in_sbtest |
+------------------+
| sbtest1          |
| sbtest2          |
| sbtest3          |
| sbtest4          |
| sbtest5          |
+------------------+
5 rows in set (0.00 sec)

2）ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset;
+----------+
| COUNT(*) |
+----------+
| 10       |
+----------+
1 row in set (0.00 sec)

3）業務賬號執行查詢

從mysql客戶端我們可以運行通常的查詢：

mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c;
+-------------------------------------------------------------------------------------------------------------------------+
| c                                                                                                                       |
+-------------------------------------------------------------------------------------------------------------------------+
| 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 |
| 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 |
| 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 |
+-------------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)

查詢已成功運行。如上所述，我們沒有修改原始流量。
那 stats_mysql_query_digest 中是又怎麼樣？

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text;
+-----------+------------+----------+----------------------------------------------------------------------+
| hostgroup | count_star | sum_time | digest_text                                                          |
+-----------+------------+----------+----------------------------------------------------------------------+
| 20        | 2          | 1493     | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c |
+-----------+------------+----------+----------------------------------------------------------------------+
1 row in set (0.00 sec)

可以看到，原始查詢執行了兩次，因此某些內容無法正常運行。我們可以注意到兩個查詢都被髮送到hostgroup20：我們應該相信rule_id = 10是匹配的，但是沒有重寫查詢。
讓我們驗證它是匹配的：

Admin> SELECT * from stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 5       | 2    |  -->其中有一次爲show tables觸發的。
| 10      | 1    |
+---------+------+
2 rows in set (0.00 sec)

根據 stats_mysql_query_rules 中的規則命中信息可以看到，rule_id = 10的規則是匹配的。
那爲什麼不重寫查詢？

5）問題修復

在重新審查一次rule_id=10的replace_pattern內容：

SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 \+ \4 ORDER BY c

發現，+號前面不該有轉義的。更新查詢規則10：

Admin> UPDATE mysql_query_rules SET replace_pattern='SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c' WHERE rule_id=10;
Query OK, 1 row affected (0.00 sec)

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply FROM runtime_mysql_query_rules WHERE rule_id=10 \G
*************************** 1. row ***************************
              rule_id: 10
               active: 1
               flagIN: 100
        match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
      replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c
       mirror_flagOUT: NULL
                apply: 1
1 row in set (0.00 sec)

修改已經生效！！

6）驗證結果

ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset;
+----------+
| COUNT(*) |
+----------+
| 5        |
+----------+
1 row in set (0.00 sec)

在客戶端再次執行原SQL：

mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c;
+-------------------------------------------------------------------------------------------------------------------------+
| c                                                                                                                       |
+-------------------------------------------------------------------------------------------------------------------------+
| 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 |
| 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 |
| 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 |
+-------------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)

現在讓我們驗證查詢是否被正確重寫：

Admin>  select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text;
+-----------+------------+----------+----------------------------------------------------------------------+
| hostgroup | count_star | sum_time | digest_text                                                          |
+-----------+------------+----------+----------------------------------------------------------------------+
| 20        | 1          | 414      | SELECT DISTINCT c FROM sbtest1 WHERE id = ? + ? ORDER BY c           |
| 20        | 1          | 661      | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c |
+-----------+------------+----------+----------------------------------------------------------------------+
2 rows in set (0.00 sec)

到此可以看到，查詢被正確重寫，並且執行了！

八、高級示例：使用鏡像和防火牆來測試查詢重寫[Advanced example: use mirroring and firewall to test query rewrite]

前面的示例/練習有點超前：那麼，是否可以在不執行查詢的情況下重寫查詢？答案是：可以的！
爲此，我們將爲鏡像查詢設置 error_msg ：這樣ProxySQL將處理鏡像查詢，但會過濾它而不將其發送到任何mysql服務器。
如最初所講的那樣，可以修改鏡像查詢，並且防火牆是修改鏡像查詢的示例。

例如：
接着前面的案例，繼續操作。

1）修改查詢規則

爲規則10添加error_msg信息。

Admin> UPDATE mysql_query_rules SET error_msg="random error, blah blah" WHERE rule_id=10;
Query OK, 1 row affected (0.00 sec)

2）加載配置到RUNTIME層

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.01 sec)

查看修改結果：

Admin> SELECT rule_id,active,flagIN,match_pattern,destination_hostgroup,replace_pattern,mirror_flagOUT,apply,error_msg FROM runtime_mysql_query_rules WHERE rule_id=10 \G
*************************** 1. row ***************************
              rule_id: 10
               active: 1
               flagIN: 100
        match_pattern: ^SELECT DISTINCT c FROM sbtest([0-9]{1,2}) WHERE id BETWEEN ([0-9]+) AND ([0-9]+)\+([0-9]+) ORDER BY c$
destination_hostgroup: 20
      replace_pattern: SELECT DISTINCT c FROM sbtest\1 WHERE id = \3 + \4 ORDER BY c
       mirror_flagOUT: NULL
                apply: 1
            error_msg: random error, blah blah
1 row in set (0.00 sec)

可以看到，修改已生效！！

3）ProxySQL管理端清空並查看當前以執行SQL的查詢概要統計：

Admin> SELECT COUNT(*) FROM stats_mysql_query_digest_reset;
+----------+
| COUNT(*) |
+----------+
| 0        |
+----------+
1 row in set (0.00 sec)

4）業務賬號執行查詢

在mysql客戶端重新運行查詢：

mysql> SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN 10 AND 10+2 ORDER BY c;
+-------------------------------------------------------------------------------------------------------------------------+
| c                                                                                                                       |
+-------------------------------------------------------------------------------------------------------------------------+
| 06208928544-69213163800-95083408911-83949560459-26629535077-58798231143-58688386449-59141897529-07315042085-86003451120 |
| 68305043604-07392484646-78480928447-88155597080-08908465928-35357008626-44894171482-13904841657-13998032237-49278517178 |
| 84225420767-95119807827-48689909948-04145663437-29723649568-88238910120-61256632514-12324871715-71270848294-09484980067 |
+-------------------------------------------------------------------------------------------------------------------------+
3 rows in set (0.00 sec)

5）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

Admin> select hostgroup,count_star,sum_time,digest_text from stats_mysql_query_digest ORDER BY digest_text;
+-----------+------------+----------+----------------------------------------------------------------------+
| hostgroup | count_star | sum_time | digest_text                                                          |
+-----------+------------+----------+----------------------------------------------------------------------+
| 10        | 1          | 0        | SELECT DISTINCT c FROM sbtest1 WHERE id = ? + ? ORDER BY c           |
| 20        | 1          | 730      | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c |
+-----------+------------+----------+----------------------------------------------------------------------+
2 rows in set (0.00 sec)

Admin> SELECT * from stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 5       | 1    |
| 10      | 1    |
+---------+------+
2 rows in set (0.01 sec)

Great!!我們已看到查詢已被重寫，但實際上並未在任何地方發送：
1）sum_time = 0 ，因爲響應是立即的;
2）hostgroup=10 ，在規則10中，設置了destination_hostgroup=20，但這裏卻顯示了業務賬號默認的主機組10；如果業務賬號未設置默認主機組，則顯示爲0；這種顯示和設置不一致也表面沒有發送到任何地方(不信可以打開默認主機組MySQL的general_log去驗證).

完畢！

ProxySQL官檔翻譯__20_Mirroring

20_Mirroring

一、mysql_query_rules的擴展[Extensions to mysql_query_rules]

二、功能實現概述[Implementation overview]

三、案例一：原始查詢與鏡像查詢選擇同一主機組[Mirror selects to same hostgroup]

1、設置查詢規則

1）設置查詢規則

2）加載配置到RUNTIME層

2、連接ProxySQL執行查詢

1）使用業務賬號連入ProxySQL

2）ProxySQL管理端清空並查看當前以執行SQL的查詢概要統計：

3）業務賬號執行查詢

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

5）作爲附加測試，我們重新運行相同的查詢：

6）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

四、案例二：原始查詢與鏡像查詢選擇到不同的主機組[Mirror SELECT to different hostgroup]

1、設置查詢規則

1）設置查詢規則

2）加載配置到RUNTIME層

2、連接ProxySQL執行查詢

2）ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

3）業務賬號執行查詢

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

五、案例三：重寫源查詢和鏡像[rewrite both source query and mirror]

1、設置查詢規則

1）設置查詢規則

2）加載配置到RUNTIME層

2、連接ProxySQL執行查詢

1）使用業務賬號連入ProxySQL

2）ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

3）業務賬號執行查詢

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

六、僅重寫鏡像查詢[rewrite mirror query only]

1、設置查詢規則

1）設置查詢規則

2）加載配置到RUNTIME層

2、業務賬號執行查詢

1）使用業務賬號連入ProxySQL

2）ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

3）業務賬號執行查詢

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

七、高級示例：使用鏡像測試查詢重寫[Advanced example: use mirroring to test query rewrite]

1、設置查詢規則

1）設置查詢規則

2）加載配置到RUNTIME層

2、業務賬號執行查詢

1）使用業務賬號連入ProxySQL

2）ProxySQL管理端清空stats_mysql_query_digest以獲得新的統計信息：

3）業務賬號執行查詢

4）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：

5）問題修復

6）驗證結果

八、高級示例：使用鏡像和防火牆來測試查詢重寫[Advanced example: use mirroring and firewall to test query rewrite]

1）修改查詢規則

2）加載配置到RUNTIME層

3）ProxySQL管理端清空並查看當前以執行SQL的查詢概要統計：

4）業務賬號執行查詢

5）ProxySQL管理端，查看當前以執行SQL的查詢概要統計：