ProxySQL官档翻译__14_ProxySQL_Configuration

14_ProxySQL_Configuration

备注:文章编写时间201904-201905期间,后续官方在github的更新没有被写入

~
~
如何最简配置ProxySQL[Mini HOW TO on ProxySQL Configuration]

这个简单的HOWTO描述了如何一步步的配置ProxySQL的一些组件。但注意,这不是一个完整的指南。

假设您已经了解ProxySQL体系结构,并且本HOWTO假定使用标准SQL管理界面来重新配置ProxySQL,默认情况下使用简单(可更改)凭据连接到端口6032:

$ mysql -u admin -padmin -h 127.0.0.1 -P6032

首先,查验以下表中没有配置任何内容:mysql_servers、mysql_replication_hostgroups、mysql_query_rules。

mysql> \R Admin>
PROMPT set to 'Admin> '
Admin>
Admin> SELECT * FROM mysql_servers;
Empty set (0.00 sec)

Admin> SELECT * from mysql_replication_hostgroups;
Empty set (0.00 sec)

Admin> SELECT * from mysql_query_rules;
Empty set (0.00 sec)

一、添加后端数据库服务器[Add backends]

在这个演示中,我使用mysql_sandbox(MySQL沙箱工具https://launchpad.net/mysql-sandbox)在本地启动了3个MySQL服务器。让我们把它们添加到ProxySQL中。

Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'127.0.0.1',21891);
Query OK, 1 row affected (0.01 sec)

Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'127.0.0.1',21892);
Query OK, 1 row affected (0.01 sec)

Admin> INSERT INTO mysql_servers(hostgroup_id,hostname,port) VALUES (1,'127.0.0.1',21893);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT * FROM mysql_servers;
+--------------+-----------+-------+--------+--------+-------------+-----------------+---------------------+
| hostgroup_id | hostname  | port  | status | weight | compression | max_connections | max_replication_lag |
+--------------+-----------+-------+--------+--------+-------------+-----------------+---------------------+
| 1            | 127.0.0.1 | 21891 | ONLINE | 1      | 0           | 1000            | 0                   |
| 1            | 127.0.0.1 | 21892 | ONLINE | 1      | 0           | 1000            | 0                   |
| 1            | 127.0.0.1 | 21893 | ONLINE | 1      | 0           | 1000            | 0                   |
+--------------+-----------+-------+--------+--------+-------------+-----------------+---------------------+
3 rows in set (0.00 sec)

注意:默认情况下,mysql_sandbox将在slaves上设置read_only = 0。需要手动在slaves上设置set global read_only = 1。

二、配置监控[Configure monitoring]

ProxySQL会经常的监视它为它配置的后端服务器。为此,配置一些变量很重要。

1、添加监视后端所需用户的凭据(需要在MySQL Server中已创建该用户):

Admin> SELECT * FROM global_variables WHERE variable_name='mysql-monitor_username';
+------------------------+----------------+
| variable_name          | variable_value |
+------------------------+----------------+
| mysql-monitor_username | monitor        |   <==默认值
+------------------------+----------------+
1 row in set (0.00 sec)

Admin> SELECT * FROM global_variables WHERE variable_name='mysql-monitor_password';
+------------------------+----------------+
| variable_name          | variable_value |
+------------------------+----------------+
| mysql-monitor_password | monitor        |   <==默认值
+------------------------+----------------+
1 row in set (0.00 sec)

这里使用默认值介绍使用方式:

Admin> UPDATE global_variables SET variable_value='monitor' WHERE variable_name='mysql-monitor_username';
Query OK, 1 row affected (0.00 sec)

Admin> UPDATE global_variables SET variable_value='monitor' WHERE variable_name='mysql-monitor_password';
Query OK, 1 row affected (0.00 sec)

然后我们配置各种监控间隔:

Admin> SELECT * FROM global_variables WHERE variable_name IN ('mysql-monitor_connect_interval','mysql-monitor_ping_interval','mysql-monitor_read_only_interval');
+----------------------------------+----------------+
| variable_name                    | variable_value |
+----------------------------------+----------------+
| mysql-monitor_connect_interval   | 60000          |
| mysql-monitor_ping_interval      | 10000          |
| mysql-monitor_read_only_interval | 1500           |
+----------------------------------+----------------+
3 rows in set (0.00 sec)

Admin> UPDATE global_variables SET variable_value='2000' WHERE variable_name IN ('mysql-monitor_connect_interval','mysql-monitor_ping_interval','mysql-monitor_read_only_interval');
Query OK, 3 rows affected (0.00 sec)

Admin> SELECT * FROM global_variables WHERE variable_name LIKE 'mysql-monitor_%';
+----------------------------------------+---------------------------------------------------+
| variable_name                          | variable_value                                    |
+----------------------------------------+---------------------------------------------------+
| mysql-monitor_history                  | 600000                                            |
| mysql-monitor_connect_interval         | 2000                                              |
| mysql-monitor_connect_timeout          | 200                                               |
| mysql-monitor_ping_interval            | 2000                                              |
| mysql-monitor_ping_timeout             | 100                                               |
| mysql-monitor_read_only_interval       | 2000                                              |
| mysql-monitor_read_only_timeout        | 100                                               |
| mysql-monitor_replication_lag_interval | 10000                                             |
| mysql-monitor_replication_lag_timeout  | 1000                                              |
| mysql-monitor_username                 | monitor                                           |
| mysql-monitor_password                 | monitor                                           |
| mysql-monitor_query_variables          | SELECT * FROM INFORMATION_SCHEMA.GLOBAL_VARIABLES |
| mysql-monitor_query_status             | SELECT * FROM INFORMATION_SCHEMA.GLOBAL_STATUS    |
| mysql-monitor_query_interval           | 60000                                             |
| mysql-monitor_query_timeout            | 100                                               |
| mysql-monitor_timer_cached             | true                                              |
| mysql-monitor_writer_is_also_reader    | true                                              |
+----------------------------------------+---------------------------------------------------+
17 rows in set (0.00 sec)

在ProxySQL中有很多变量,有些变量尚未使用或与此无关。当前只考虑上边列出的那些。
表global_variables中与MySQL Monitor相关的更改仅在运行 LOAD MYSQL VARIABLES TO RUNTIME 命令后才会生效,
并且在运行 SAVE MYSQL VARIABLES TO DISK 后它们才将永久存储到磁盘。

Admin> LOAD MYSQL VARIABLES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SAVE MYSQL VARIABLES TO DISK;
Query OK, 110 rows affected (0.02 sec)

三、后端MySQL服务健康检测[Backend's health check] -- '

现在,让我们看看ProxySQL是否能够与这些主机通信。ProxySQL有几个存储监视信息的表。

Admin> show databases;
+-----+---------------+-------------------------------------+
| seq | name          | file                                |
+-----+---------------+-------------------------------------+
| 0   | main          |                                     |
| 2   | disk          | /var/lib/proxysql/proxysql.db       |
| 3   | stats         |                                     |
| 4   | monitor       |                                     |
| 5   | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.00 sec)

Admin> SHOW TABLES FROM monitor;
+------------------------------------+
| tables                             |
+------------------------------------+
| mysql_server_connect_log           |
| mysql_server_galera_log            |
| mysql_server_group_replication_log |
| mysql_server_ping_log              |
| mysql_server_read_only_log         |
| mysql_server_replication_lag_log   |
+------------------------------------+
6 rows in set (0.00 sec)

并非当前监视器中的所有表都会被使用。现在我们可以使用以下查询检查相关表:

Admin> SELECT * FROM monitor.mysql_server_connect_log ORDER BY time_start_us DESC LIMIT 10;
+-----------+-------+------------------+----------------------+---------------+
| hostname  | port  | time_start_us       | connect_success_time | connect_error |
+-----------+-------+------------------+----------------------+---------------+
| 127.0.0.1 | 21891 | 1456968814253432 | 562                  | NULL          |
| 127.0.0.1 | 21892 | 1456968814253432 | 309                  | NULL          |
| 127.0.0.1 | 21893 | 1456968814253432 | 154                  | NULL          |
| 127.0.0.1 | 21891 | 1456968812252146 | 689                  | NULL          |
| 127.0.0.1 | 21892 | 1456968812252146 | 424                  | NULL          |
| 127.0.0.1 | 21893 | 1456968812252146 | 174                  | NULL          |
| 127.0.0.1 | 21891 | 1456968810251585 | 569                  | NULL          |
| 127.0.0.1 | 21892 | 1456968810251585 | 316                  | NULL          |
| 127.0.0.1 | 21893 | 1456968810251585 | 155                  | NULL          |
| 127.0.0.1 | 21891 | 1456968808250762 | 570                  | NULL          |
+-----------+-------+------------------+----------------------+---------------+
10 rows in set (0.00 sec)

Admin> SELECT * FROM monitor.mysql_server_ping_log ORDER BY time_start_us DESC LIMIT 10;
+-----------+-------+------------------+-------------------+------------+
| hostname  | port  | time_start_us       | ping_success_time | ping_error |
+-----------+-------+------------------+-------------------+------------+
| 127.0.0.1 | 21891 | 1456968828686787 | 124               | NULL       |
| 127.0.0.1 | 21892 | 1456968828686787 | 62                | NULL       |
| 127.0.0.1 | 21893 | 1456968828686787 | 57                | NULL       |
| 127.0.0.1 | 21891 | 1456968826686385 | 99                | NULL       |
| 127.0.0.1 | 21892 | 1456968826686385 | 46                | NULL       |
| 127.0.0.1 | 21893 | 1456968826686385 | 42                | NULL       |
| 127.0.0.1 | 21891 | 1456968824685162 | 135               | NULL       |
| 127.0.0.1 | 21892 | 1456968824685162 | 61                | NULL       |
| 127.0.0.1 | 21893 | 1456968824685162 | 57                | NULL       |
| 127.0.0.1 | 21891 | 1456968822684689 | 215               | NULL       |
+-----------+-------+------------------+-------------------+------------+
10 rows in set (0.01 sec)

根据上面的结果,我们可以得出结论,所有配置的服务器都是健康的。
这里需要特别注意是:即使在将表加载到RUNTIME之前,也会根据表mysql_servers的内容执行对connect和ping的监视。
这样做是有意的:通过这种方式,可以在生产中添加节点之前执行基本运行状况检查。

确认后端服务正常后,就可以加载到RUNTIME层进行启用:

Admin> SELECT * FROM runtime_mysql_servers;
Empty set (0.00 sec)

Admin> LOAD MYSQL SERVERS TO RUNTIME;
Query OK, 0 rows affected (0.01 sec)

Admin> SELECT * FROM runtime_mysql_servers;
+--------------+-----------+---—---+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname  | port  | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+-----------+----—--+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 1            | 127.0.0.1 | 21891 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 1            | 127.0.0.1 | 21892 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 1            | 127.0.0.1 | 21893 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
+--------------+-----------+----—--+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
3 rows in set (0.00 sec)

四、配置MySQL复制架构主机组[MySQL replication hostgroups]

检查一下监视器模式中的另一个表,monitor.mysql_server_read_only_log:

Admin> SELECT * FROM monitor.mysql_server_read_only_log ORDER BY time_start_us DESC LIMIT 10;
Empty set (0.00 sec)

此表目前为空。原因是ProxySQL仅检查在 mysql_replication_hostgroups 中配置的主机组中设定了read_only值的服务器。

查看 mysql_replication_hostgroups ,发现它也是空的:

Admin> SELECT * FROM mysql_replication_hostgroups;
Empty set (0.00 sec)

但是这个表的功能是什么?使用此表,可以将列出的主机组配置为成对的读写、只读主机组。ProxySQL将监视指定主机组中所有服务器的read_only值,并根据read_only的值将服务器分配给读写或只读主机组。

例如:
查看该表的表结构:

Admin> SHOW CREATE TABLE mysql_replication_hostgroups \G
*************************** 1. row ***************************
       table: mysql_replication_hostgroups
Create Table: CREATE TABLE mysql_replication_hostgroups (
    writer_hostgroup INT CHECK (writer_hostgroup>=0) NOT NULL PRIMARY KEY,
    reader_hostgroup INT NOT NULL CHECK (reader_hostgroup<>writer_hostgroup AND reader_hostgroup>=0),
    check_type VARCHAR CHECK (LOWER(check_type) IN ('read_only','innodb_read_only','super_read_only')) NOT NULL DEFAULT 'read_only',
    comment VARCHAR NOT NULL DEFAULT '', UNIQUE (reader_hostgroup))
1 row in set (0.00 sec)

插入读写、只读组配置信息:

Admin> INSERT INTO mysql_replication_hostgroups(writer_hostgroup,reader_hostgroup,check_type,comment) VALUES (1,2,'read_only','cluster1');
Query OK, 1 row affected (0.00 sec)

读写组编号为1,只读组编号为2;

Admin> select * from mysql_replication_hostgroups;
+------------------+------------------+------------+----------+
| writer_hostgroup | reader_hostgroup | check_type | comment  |
+------------------+------------------+------------+----------+
| 1                | 2                | read_only  | cluster1 |
+------------------+------------------+------------+----------+
1 row in set (0.00 sec)

设置完成后,在主机组1或2中配置的所有服务器都将自动移动到正确的主机组(起初我们配置的服务器都在组1中):
1)如果它们的read_only = 0,将被自动移动到主机组1 ;
2)如果它们的read_only = 1,将被自动移动到主机组2 ;

但此时此算法仍未运行,因为新的配置并未在加载到RUNTIME层:

Admin> select * from mysql_servers;
+--------------+-----------+---—---+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname  | port  | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+-----------+----—--+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 1            | 127.0.0.1 | 21891 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 1            | 127.0.0.1 | 21892 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 1            | 127.0.0.1 | 21893 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
+--------------+-----------+----—--+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
3 rows in set (0.00 sec)

使用与MYSQL SERVERS相同的LOAD命令加载mysql_replication_hostgroups到RUNTIME层:
事实上,LOAD MYSQL SERVERS TO RUNTIME 会同时处理mysql_servers和mysql_replication_hostgroups表。

Admin> LOAD MYSQL SERVERS TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

查看MySQL服务分组:

Admin> select * from mysql_servers;
+--------------+-----------+---—---+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| hostgroup_id | hostname  | port  | gtid_port | status | weight | compression | max_connections | max_replication_lag | use_ssl | max_latency_ms | comment |
+--------------+-----------+----—--+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
| 1            | 127.0.0.1 | 21891 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 2            | 127.0.0.1 | 21892 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
| 2            | 127.0.0.1 | 21893 | 0         | ONLINE | 1      | 0           | 1000            | 0                   | 0       | 0              |         |
+--------------+-----------+----—--+-----------+--------+--------+-------------+-----------------+---------------------+---------+----------------+---------+
3 rows in set (0.00 sec)

等几秒钟,再次检查状态:

Admin> SELECT * FROM monitor.mysql_server_read_only_log ORDER BY time_start_us DESC LIMIT 10;
+-----------+-------+------------------+--------------+-----------+-------+
| hostname  | port  | time_start_us       | success_time | read_only | error |
+-----------+-------+------------------+--------------+-----------+-------+
| 127.0.0.1 | 21891 | 1456969634783579 | 762          | 0         | NULL  |
| 127.0.0.1 | 21892 | 1456969634783579 | 378          | 1         | NULL  |
| 127.0.0.1 | 21893 | 1456969634783579 | 317          | 1         | NULL  |
| 127.0.0.1 | 21891 | 1456969632783364 | 675          | 0         | NULL  |
| 127.0.0.1 | 21892 | 1456969632783364 | 539          | 1         | NULL  |
| 127.0.0.1 | 21893 | 1456969632783364 | 550          | 1         | NULL  |
| 127.0.0.1 | 21891 | 1456969630783159 | 493          | 0         | NULL  |
| 127.0.0.1 | 21892 | 1456969630783159 | 626          | 1         | NULL  |
| 127.0.0.1 | 21893 | 1456969630783159 | 572          | 1         | NULL  |
| 127.0.0.1 | 21891 | 1456969628782328 | 433          | 0         | NULL  |
+-----------+-------+------------------+--------------+-----------+-------+
10 rows in set (0.01 sec)

ProxySQL正在监视服务器的read_only值。并且还创建了hostgroup2,它从hostgroup1移动了read_only = 1(reader)的服务器。

一切看起来都不错是时候将配置保存到磁盘:

Admin> SAVE MYSQL SERVERS TO DISK;
Query OK, 0 rows affected (0.03 sec)

Admin> SAVE MYSQL VARIABLES TO DISK;
Query OK, 110 rows affected (0.00 sec)

五、设置MySQL Users(连接后端使用)

在mysql_servers中配置服务器之后,还需要配置mysql的用户。这是使用 mysql_users 表实现的:

Admin> SELECT * FROM mysql_users;
Empty set (0.00 sec)

Admin> SHOW CREATE TABLE mysql_users \G
*************************** 1. row ***************************
       table: mysql_users
Create Table: CREATE TABLE mysql_users (
    username VARCHAR NOT NULL,
    password VARCHAR,
    active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 1,
    use_ssl INT CHECK (use_ssl IN (0,1)) NOT NULL DEFAULT 0,
    default_hostgroup INT NOT NULL DEFAULT 0,
    default_schema VARCHAR,
    schema_locked INT CHECK (schema_locked IN (0,1)) NOT NULL DEFAULT 0,
    transaction_persistent INT CHECK (transaction_persistent IN (0,1)) NOT NULL DEFAULT 1,
    fast_forward INT CHECK (fast_forward IN (0,1)) NOT NULL DEFAULT 0,
    backend INT CHECK (backend IN (0,1)) NOT NULL DEFAULT 1,
    frontend INT CHECK (frontend IN (0,1)) NOT NULL DEFAULT 1,
    max_connections INT CHECK (max_connections >=0) NOT NULL DEFAULT 10000,
    comment VARCHAR NOT NULL DEFAULT '',
    PRIMARY KEY (username, backend),
    UNIQUE (username, frontend))
1 row in set (0.00 sec)

该表初始时为空的。

1)添加后端MySQL上的业务账号到 mysql_users (确保该账号在MySQL上已存在):

-- MYSQL上建立用户

mysql> CREATE DATABASE sbtest ;
mysql> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, INDEX, DROP ON sbtest.* TO 'msandbox'@'188.188.0.%' IDENTIFIED BY '123456'; 
Query OK, 0 rows affected (0.01 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)

Admin> INSERT INTO mysql_users(username,password,default_hostgroup) VALUES ('root','123456',1);
Query OK, 1 row affected (0.00 sec)

Admin> INSERT INTO mysql_users(username,password,default_hostgroup) VALUES ('msandbox','123456',1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT * FROM mysql_users;
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| username | password | active | use_ssl | default_hostgroup | default_schema | schema_locked | transaction_persistent | fast_forward | backend | frontend | max_connections | comment |
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| root     | 123456   | 1      | 0       | 1                 | NULL           | 0             | 1                      | 0            | 1       | 1        | 10000           |         |
| msandbox | 123456   | 1      | 0       | 1                 | NULL           | 0             | 1                      | 0            | 1       | 1        | 10000           |         |
+----------+----------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
2 rows in set (0.00 sec)

对于该表,将大多数字段保留为默认值。这里配置的最重要的字段是:username、password、default_hostgroup
用户名和密码的含义应该非常清楚。default_hostgroup是主机组,如果特定查询没有匹配到查询规则,则将该特定用户生成的流量都发往该主机组(稍后将详细介绍)。

2)再次,将配置加载到RUNTIME层以使其生效,并将其保存到磁盘以使其在重新启动时保持不变。

Admin> SELECT * FROM runtime_mysql_users;
Empty set (0.00 sec)

Admin> SELECT * FROM runtime_mysql_users;
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| username | password                                  | active | use_ssl | default_hostgroup | default_schema | schema_locked | transaction_persistent | fast_forward | backend | frontend | max_connections | comment |
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| root     | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 0       | 1        | 10000           |         |
| msandbox | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 0       | 1        | 10000           |         |
| root     | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 1       | 0        | 10000           |         |
| msandbox | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 1       | 0        | 10000           |         |
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
4 rows in set (0.00 sec)

被添加的每个用户出现了成对的记录,这是因为每个用户在前端和后端都有一条用户记录(ProxySQL使用了后端MySQL的用户做连接认证);

从RUNTIME层获取用户加密码密码,更新MEMORY层:

Admin> SAVE MYSQL USERS FROM RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT * FROM runtime_mysql_users;
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| username | password                                  | active | use_ssl | default_hostgroup | default_schema | schema_locked | transaction_persistent | fast_forward | backend | frontend | max_connections | comment |
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| root     | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 0       | 1        | 10000           |         |
| msandbox | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 0       | 1        | 10000           |         |
| root     | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 1       | 0        | 10000           |         |
| msandbox | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 1       | 0        | 10000           |         |
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
4 rows in set (0.00 sec)

持久化用户信息到DISK层:

Admin> SELECT * FROM disk.mysql_users;
Empty set (0.00 sec)

Admin> SAVE MYSQL USERS TO DISK;
Query OK, 0 rows affected (0.01 sec)

Admin> SELECT * FROM disk.mysql_users;
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| username | password                                  | active | use_ssl | default_hostgroup | default_schema | schema_locked | transaction_persistent | fast_forward | backend | frontend | max_connections | comment |
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
| root     | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 1       | 1        | 10000           |         |
| msandbox | *6BB4837EB74329105EE4568DDA7DC67ED2CA2AD9 | 1      | 0       | 1                 |                | 0             | 1                      | 0            | 1       | 1        | 10000           |         |
+----------+-------------------------------------------+--------+---------+-------------------+----------------+---------------+------------------------+--------------+---------+----------+-----------------+---------+
2 rows in set (0.00 sec)

3)我们现在可以尝试从不同的终端连接(连接端口默认是6033):

[root@localhost ~]# mysql -u msandbox -p123456 -h 188.188.0.71 -P6033 -e "SELECT 1"
mysql: [Warning] Using a password on the command line interface can be insecure.
+---+
| 1 |
+---+
| 1 |
+---+

注意:连接端口默认是6033,不是6032(ProxySQL管理端口)也不是3306(MySQL端口)。

[root@localhost ~]# mysql -u msandbox -p123456 -h 188.188.0.71 -P6033 -e "SELECT @@port"
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------+
| @@port |
+--------+
|  21891 |
+--------+

这里查询出了后端MySQL服务的端口。

它似乎有效,并且毫不奇怪,查询被发送到侦听端口21891上的服务器(Master),因为用户msandbox的默认组为hostgroup1,并且当前也没有针对该用户的查询规则可用。

六、功能测试[Functional tests]

现在可以尝试一些"benchmark"测试,来验证ProxySQL是否正常运行。

如果已经创建了sysbench表,则可以使用以下命令运行负载测试:
这里使用 sysbench-1.0.14 进行测试。

-- 测试数据初始
[root@localhost ~]# sysbench oltp_common --mysql_storage_engine=innodb --db-driver=mysql --mysql-host=188.188.0.71 --mysql-port=6033 \
--mysql-user='msandbox' --mysql-password='123456' --mysql-db=sbtest --threads=4 --table_size=10000 --tables=5  prepare

-- 执行读写测试
sysbench oltp_read_write --mysql_storage_engine=innodb --db-driver=mysql --mysql-host=188.188.0.71 --mysql-port=6033 \
--mysql-user='msandbox' --mysql-password='123456' --mysql-db=sbtest --threads=4 --table_size=10000 --tables=5 \
--time=200 --histogram --report-interval=10 --db-ps-mode=disable run

备注: 如果设置了--db-ps-mode=disable,则效果为每个线程始终在一个Session中执行所有SQL;如果不设置,则为每次执行都新建连接。

在测试过程中你会发现,所有的请求都集中在了Master上,而Slave上却没有任何请求到达。

七、ProxySQL信息收集[ProxySQL Statistics]

ProxySQL在stats库中收集了大量实时统计信息:

Admin> SHOW SCHEMAS;
+-----+---------------+-------------------------------------+
| seq | name          | file                                |
+-----+---------------+-------------------------------------+
| 0   | main          |                                     |
| 2   | disk          | /var/lib/proxysql/proxysql.db       |
| 3   | stats         |                                     |
| 4   | monitor       |                                     |
| 5   | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.00 sec)

Admin> SHOW TABLES FROM stats;
+--------------------------------------+
| tables                               |
+--------------------------------------+
| global_variables                     |
| stats_memory_metrics                 |
| stats_mysql_commands_counters        |
| stats_mysql_connection_pool          |
| stats_mysql_connection_pool_reset    |
| stats_mysql_errors                   |
| stats_mysql_errors_reset             |
| stats_mysql_global                   |
| stats_mysql_gtid_executed            |
| stats_mysql_prepared_statements_info |
| stats_mysql_processlist              |
| stats_mysql_query_digest             |
| stats_mysql_query_digest_reset       |
| stats_mysql_query_rules              |
| stats_mysql_users                    |
| stats_proxysql_servers_checksums     |
| stats_proxysql_servers_metrics       |
| stats_proxysql_servers_status        |
+--------------------------------------+
18 rows in set (0.00 sec)

stats架构中有很多表。我们将对它们进行分析。

1、表 stats.stats_mysql_connection_pool

Admin> SELECT * FROM stats.stats_mysql_connection_pool;
+-----------+-----------+----------+--------------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
| hostgroup | srv_host  | srv_port | status       | ConnUsed | ConnFree | ConnOK | ConnERR | MaxConnUsed | Queries | Queries_GTID_sync | Bytes_data_sent | Bytes_data_recv | Latency_us |
+-----------+-----------+----------+--------------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
| 1         | 127.0.0.1 | 21891    | ONLINE       | 0        | 4        | 11     | 66      | 5           | 3262254 | 0                 | 156040355       | 6371749875      | 99         |
| 1         | 127.0.0.1 | 21892    | OFFLINE_HARD | 0        | 0        | 0      | 0       | 0           | 0       | 0                 | 0               | 0               | 97         |
| 2         | 127.0.0.1 | 21893    | ONLINE       | 0        | 0        | 0      | 0       | 0           | 0       | 0                 | 0               | 0               | 96         |
| 2         | 127.0.0.1 | 21892    | ONLINE       | 0        | 0        | 0      | 0       | 0           | 0       | 0                 | 0               | 0               | 97         |
+-----------+-----------+----------+--------------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
4 rows in set (0.00 sec)

说明:当前,当服务器被删除(完全删除或从主机组中移出)时,它在内部被标记为"OFFLINE_HARD",而不是真正删除。这就是为什么它将端口21892上的服务器显示为主机组1的"OFFLINE_HARD"。
此表返回有关发送到每个服务器的流量的相关信息。正如预期的那样,所有流量都被发送到主服务器端口21891上的服务器。

2、表stats_mysql_commands_counters

Admin> SELECT * FROM stats_mysql_commands_counters WHERE Total_cnt;
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
| Command      | Total_Time_us | Total_cnt | cnt_100us | cnt_500us | cnt_1ms | cnt_5ms | cnt_10ms | cnt_50ms | cnt_100ms | cnt_500ms | cnt_1s | cnt_5s | cnt_10s | cnt_INFs |
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
| BEGIN        | 60686611      | 163092    | 2550      | 126541    | 24450   | 9549    | 2        | 0        | 0         | 0         | 0      | 0      | 0       | 0        |
| COMMIT       | 651883996     | 163019    | 13        | 3         | 0       | 134329  | 26377    | 2294     | 3         | 0         | 0      | 0      | 0       | 0        |
| CREATE_INDEX | 799325        | 14        | 0         | 4         | 0       | 0       | 0        | 2        | 5         | 3         | 0      | 0      | 0       | 0        |
| CREATE_TABLE | 668337        | 30        | 0         | 9         | 4       | 2       | 1        | 10       | 4         | 0         | 0      | 0      | 0       | 0        |
| DELETE       | 66235607      | 163109    | 66        | 134225    | 21610   | 7137    | 70       | 1        | 0         | 0         | 0      | 0      | 0       | 0        |
| DROP_TABLE   | 130117        | 32        | 0         | 5         | 0       | 18      | 3        | 6        | 0         | 0         | 0      | 0      | 0       | 0        |
| INSERT       | 71401062      | 163144    | 65        | 135608    | 21000   | 6413    | 2        | 2        | 25        | 29        | 0      | 0      | 0       | 0        |
| SELECT       | 973021665     | 2283433   | 2272      | 1695366   | 421243  | 164544  | 7        | 1        | 0         | 0         | 0      | 0      | 0       | 0        |
| UPDATE       | 142844582     | 326280    | 134       | 255645    | 54795   | 15573   | 125      | 8        | 0         | 0         | 0      | 0      | 0       | 0        |
| SHOW         | 8005          | 9         | 0         | 5         | 1       | 3       | 0        | 0        | 0         | 0         | 0      | 0      | 0       | 0        |
+--------------+---------------+-----------+-----------+-----------+---------+---------+----------+----------+-----------+-----------+--------+--------+---------+----------+
10 rows in set (0.00 sec)

表stats_mysql_commands_counters返回有关执行的语句类型和执行时间分布的详细信息!

3、表stats_mysql_query_digest

表 stats_mysql_commands_counters 提供了非常有用的信息。我们可以获得有关已执行查询的更多详细信息吗?表 stats_mysql_query_digest 有助于:

Admin> SELECT * FROM stats_mysql_query_digest ORDER BY sum_time DESC limit 10;
+-----------+------------+----------+----------------+--------------------+--------------------------------------------------------------------+------------+------------+------------+-----------+----------+----------+
| hostgroup | schemaname | username | client_address | digest             | digest_text                                                        | count_star | first_seen | last_seen  | sum_time  | min_time | max_time |
+-----------+------------+----------+----------------+--------------------+--------------------------------------------------------------------+------------+------------+------------+-----------+----------+----------+
| 1         | sbtest     | msandbox |                | 0x695FBF255DBEB0DD | COMMIT                                                             | 163019     | 1556684776 | 1556685389 | 651883996 | 96       | 57674    |
| 1         | sbtest     | msandbox |                | 0xBF001A0C13781C1D | SELECT c FROM sbtest1 WHERE id=?                                   | 328166     | 1556684776 | 1556685389 | 112340379 | 83       | 4543     |
| 1         | sbtest     | msandbox |                | 0x0250CB4007721D69 | SELECT c FROM sbtest3 WHERE id=?                                   | 327446     | 1556684776 | 1556685389 | 112260374 | 82       | 4517     |
| 1         | sbtest     | msandbox |                | 0x03744DC190BC72C7 | SELECT c FROM sbtest5 WHERE id=?                                   | 327416     | 1556684776 | 1556685389 | 111997649 | 83       | 11095    |
| 1         | sbtest     | msandbox |                | 0x9AF59B998A3688ED | SELECT c FROM sbtest2 WHERE id=?                                   | 324866     | 1556684776 | 1556685389 | 111375893 | 86       | 9565     |
| 1         | sbtest     | msandbox |                | 0x9D058B6F3BC2F754 | SELECT c FROM sbtest4 WHERE id=?                                   | 322922     | 1556684776 | 1556685389 | 110558550 | 80       | 9678     |
| 1         | sbtest     | msandbox |                | 0xFAD1519E4760CBDE | BEGIN                                                              | 163092     | 1556684776 | 1556685389 | 60686611  | 61       | 5466     |
| 1         | sbtest     | msandbox |                | 0x44BCB144058686EB | SELECT DISTINCT c FROM sbtest3 WHERE id BETWEEN ? AND ? ORDER BY c | 32783      | 1556684776 | 1556685389 | 35102860  | 112      | 2626     |
| 1         | sbtest     | msandbox |                | 0x283AA9863F85EFC8 | SELECT DISTINCT c FROM sbtest4 WHERE id BETWEEN ? AND ? ORDER BY c | 32563      | 1556684776 | 1556685389 | 35056979  | 86       | 6810     |
| 1         | sbtest     | msandbox |                | 0x847CD40BA8EA5175 | SELECT DISTINCT c FROM sbtest5 WHERE id BETWEEN ? AND ? ORDER BY c | 32695      | 1556684776 | 1556685389 | 34881271  | 87       | 3293     |
+-----------+------------+----------+----------------+--------------------+--------------------------------------------------------------------+------------+------------+------------+-----------+----------+----------+
10 rows in set (0.01 sec)

在这里太多的信息使得格式化现实很困难。让我们只获得重要的几个指标:

Admin> SELECT hostgroup hg, sum_time, count_star, digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC limit 10;
+----+-----------+------------+--------------------------------------------------------------------+
| hg | sum_time  | count_star | digest_text                                                        |
+----+-----------+------------+--------------------------------------------------------------------+
| 1  | 651883996 | 163019     | COMMIT                                                             |
| 1  | 112340379 | 328166     | SELECT c FROM sbtest1 WHERE id=?                                   |
| 1  | 112260374 | 327446     | SELECT c FROM sbtest3 WHERE id=?                                   |
| 1  | 111997649 | 327416     | SELECT c FROM sbtest5 WHERE id=?                                   |
| 1  | 111375893 | 324866     | SELECT c FROM sbtest2 WHERE id=?                                   |
| 1  | 110558550 | 322922     | SELECT c FROM sbtest4 WHERE id=?                                   |
| 1  | 60686611  | 163092     | BEGIN                                                              |
| 1  | 35102860  | 32783      | SELECT DISTINCT c FROM sbtest3 WHERE id BETWEEN ? AND ? ORDER BY c |
| 1  | 35056979  | 32563      | SELECT DISTINCT c FROM sbtest4 WHERE id BETWEEN ? AND ? ORDER BY c |
| 1  | 34881271  | 32695      | SELECT DISTINCT c FROM sbtest5 WHERE id BETWEEN ? AND ? ORDER BY c |
+----+-----------+------------+--------------------------------------------------------------------+
10 rows in set (0.01 sec)

所有流量都将发送到hostgroup1。现在那如何将特定的查询发送到Slave上呢?继续下面的查询规则部分......

八、MySQL查询规则[MySQL Query Rules]

1、添加查询规则

表 mysql_query_rules 有很多字段,它是控制通过ProxySQL的流量的一个非常强大的工具。
其表定义如下:

Admin> SHOW CREATE TABLE mysql_query_rules \G
*************************** 1. row ***************************
       table: mysql_query_rules
Create Table: CREATE TABLE mysql_query_rules (
    rule_id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    active INT CHECK (active IN (0,1)) NOT NULL DEFAULT 0,
    username VARCHAR,
    schemaname VARCHAR,
    flagIN INT CHECK (flagIN >= 0) NOT NULL DEFAULT 0,
    client_addr VARCHAR,
    proxy_addr VARCHAR,
    proxy_port INT CHECK (proxy_port >= 0 AND proxy_port <= 65535), digest VARCHAR,
    match_digest VARCHAR,
    match_pattern VARCHAR,
    negate_match_pattern INT CHECK (negate_match_pattern IN (0,1)) NOT NULL DEFAULT 0,
    re_modifiers VARCHAR DEFAULT 'CASELESS',
    flagOUT INT CHECK (flagOUT >= 0), replace_pattern VARCHAR CHECK(CASE WHEN replace_pattern IS NULL THEN 1 WHEN replace_pattern IS NOT NULL AND match_pattern IS NOT NULL THEN 1 ELSE 0 END),
    destination_hostgroup INT DEFAULT NULL,
    cache_ttl INT CHECK(cache_ttl > 0),
    cache_empty_result INT CHECK (cache_empty_result IN (0,1)) DEFAULT NULL,
    cache_timeout INT CHECK(cache_timeout >= 0),
    reconnect INT CHECK (reconnect IN (0,1)) DEFAULT NULL,
    timeout INT UNSIGNED CHECK (timeout >= 0),
    retries INT CHECK (retries>=0 AND retries <=1000),
    delay INT UNSIGNED CHECK (delay >=0),
    next_query_flagIN INT UNSIGNED,
    mirror_flagOUT INT UNSIGNED,
    mirror_hostgroup INT UNSIGNED,
    error_msg VARCHAR,
    OK_msg VARCHAR,
    sticky_conn INT CHECK (sticky_conn IN (0,1)),
    multiplex INT CHECK (multiplex IN (0,1,2)),
    gtid_from_hostgroup INT UNSIGNED,
    log INT CHECK (log IN (0,1)),
    apply INT CHECK(apply IN (0,1)) NOT NULL DEFAULT 0,
    comment VARCHAR)
1 row in set (0.00 sec)

现在可以配置ProxySQL将前2个查询发送给从Slave服务器,将其他所有查询发送给主服务器。

查看当前设置的查询规则:

Admin> SELECT * FROM mysql_query_rules ;
Empty set (0.00 sec)

插入查询规则:

-- 将对sbtest1表的以下2种查询发送到Slave。
Admin> INSERT INTO mysql_query_rules (rule_id,active,username,match_digest,destination_hostgroup,apply) VALUES (10,1,'msandbox','^SELECT c FROM sbtest1 WHERE id=\?$',2,1);
Query OK, 1 row affected (0.00 sec)

Admin> INSERT INTO mysql_query_rules (rule_id,active,username,match_digest,destination_hostgroup,apply) VALUES (20,1,'msandbox','DISTINCT c FROM sbtest1',2,1);
Query OK, 1 row affected (0.00 sec)

查看插入结果:

Admin> SELECT * FROM mysql_query_rules ;
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
| rule_id | active | username | schemaname | flagIN | client_addr | proxy_addr | proxy_port | digest | match_digest                        | match_pattern | negate_match_pattern | re_modifiers | flagOUT | replace_pattern | destination_hostgroup | cache_ttl | cache_empty_result | cache_timeout | reconnect | timeout | retries | delay | next_query_flagIN | mirror_flagOUT | mirror_hostgroup | error_msg | OK_msg | sticky_conn | multiplex | gtid_from_hostgroup | log | apply | comment |
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
| 10      | 1      | msandbox | NULL       | 0      | NULL        | NULL       | NULL       | NULL   | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL          | 0                    | CASELESS     | NULL    | NULL            | 2                     | NULL      | NULL               | NULL          | NULL      | NULL    | NULL    | NULL  | NULL              | NULL           | NULL             | NULL      | NULL   | NULL        | NULL      | NULL                | NULL | 1     | NULL    |
| 20      | 1      | msandbox | NULL       | 0      | NULL        | NULL       | NULL       | NULL   | DISTINCT c FROM sbtest1             | NULL          | 0                    | CASELESS     | NULL    | NULL            | 2                     | NULL      | NULL               | NULL          | NULL      | NULL    | NULL    | NULL  | NULL              | NULL           | NULL             | NULL      | NULL   | NULL        | NULL      | NULL                | NULL | 1     | NULL    |
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
2 rows in set (0.00 sec)

Admin> SELECT rule_id,active,username,match_digest,destination_hostgroup,apply FROM mysql_query_rules ;
+---------+--------+----------+-------------------------------------+-----------------------+-------+
| rule_id | active | username | match_digest                        | destination_hostgroup | apply |
+---------+--------+----------+-------------------------------------+-----------------------+-------+
| 10      | 1      | msandbox | ^SELECT c FROM sbtest1 WHERE id=\?$ | 2                     | 1     |
| 20      | 1      | msandbox | DISTINCT c FROM sbtest1             | 2                     | 1     |
+---------+--------+----------+-------------------------------------+-----------------------+-------+
2 rows in set (0.00 sec)

几点说明:
1)查询规则按rule_id顺序进行处理;
2)仅处理具有active = 1的规则,即开启状态的。
因为查询规则是一个非常强大的工具,如果配置错误会导致排错比较困难(我们都喜欢用正则表达式,不是吗?),默认情况下,active为0(active = 0)。 在启用它们之前,应该仔细检查规则的正则表达式!
3)第一个规则示例使用插入符号(^)和美元符号($):
这些是标记模式开头和结尾的特殊正则表达式字符。在这种情况下,这意味着match_digest或match_pattern应该与查询完全匹配。
4)与第一个规则示例相比,第二个规则示例不使用插入符号或美元:匹配可以在查询中的任何位置。
5)要特别注意正则表达式,以避免某些规则不应该与它匹配!
6)您可能会注意到问号(?)已转义。它在正则表达式中具有特殊含义;如上所述,需要对正则表达式语法特别重视!
7)apply = 1表示如果匹配则不会检查其他规则。

通过上面说明,表 mysql_query_rules 的查询可以简化为如下这样:

Admin> SELECT match_digest,destination_hostgroup FROM mysql_query_rules WHERE active=1 AND username='msandbox' ORDER BY rule_id;
+-------------------------------------+-----------------------+
| match_digest                        | destination_hostgroup |
+-------------------------------------+-----------------------+
| ^SELECT c FROM sbtest1 WHERE id=\?$ | 2                     |
| DISTINCT c FROM sbtest1             | 2                     |
+-------------------------------------+-----------------------+
2 rows in set (0.00 sec)

对于这两个特定规则,查询将被发送到Slave。如果查询没有匹配到规则,则发送到default_hostgroup(对于用户msandbox,该值为1)。

2、清空旧的查询统计记录

接下来,让我们重置表stats_mysql_query_digest的内容。为了实现这一点,我们可以简单地对stats_mysql_query_digest_reset运行任何查询即可,例如:

Admin> SELECT * FROM stats_mysql_query_digest_reset LIMIT 1;

查询 stats_mysql_query_digest_reset 将会以原子方式获取 stats_mysql_query_digest 表的内容,并截断它!

3、加载规则到RUNTIME层

现在我们可以在加载查询规则到RUNTIME层,使其生效:

Admin> SELECT * FROM runtime_mysql_query_rules;
Empty set (0.00 sec)

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT * FROM runtime_mysql_query_rules;
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
| rule_id | active | username | schemaname | flagIN | client_addr | proxy_addr | proxy_port | digest | match_digest                        | match_pattern | negate_match_pattern | re_modifiers | flagOUT | replace_pattern | destination_hostgroup | cache_ttl | cache_empty_result | cache_timeout | reconnect | timeout | retries | delay | next_query_flagIN | mirror_flagOUT | mirror_hostgroup | error_msg | OK_msg | sticky_conn | multiplex | gtid_from_hostgroup | log | apply | comment |
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
| 10      | 1      | msandbox | NULL       | 0      | NULL        | NULL       | NULL       | NULL   | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL          | 0                    | CASELESS     | NULL    | NULL            | 2                     | NULL      | NULL               | NULL          | NULL      | NULL    | NULL    | NULL  | NULL              | NULL           | NULL             | NULL      | NULL   | NULL        | NULL      | NULL                | NULL | 1     | NULL    |
| 20      | 1      | msandbox | NULL       | 0      | NULL        | NULL       | NULL       | NULL   | DISTINCT c FROM sbtest1             | NULL          | 0                    | CASELESS     | NULL    | NULL            | 2                     | NULL      | NULL               | NULL          | NULL      | NULL    | NULL    | NULL  | NULL              | NULL           | NULL             | NULL      | NULL   | NULL        | NULL      | NULL                | NULL | 1     | NULL    |
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
2 rows in set (0.00 sec)

4、将查询规则持久化到DISK层

Admin> SELECT * FROM disk.mysql_query_rules;
Empty set (0.00 sec)

Admin> SAVE MYSQL QUERY RULES TO DISK;
Query OK, 0 rows affected (0.01 sec)

Admin> SELECT * FROM disk.mysql_query_rules;
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
| rule_id | active | username | schemaname | flagIN | client_addr | proxy_addr | proxy_port | digest | match_digest                        | match_pattern | negate_match_pattern | re_modifiers | flagOUT | replace_pattern | destination_hostgroup | cache_ttl | cache_empty_result | cache_timeout | reconnect | timeout | retries | delay | next_query_flagIN | mirror_flagOUT | mirror_hostgroup | error_msg | OK_msg | sticky_conn | multiplex | gtid_from_hostgroup | log | apply | comment |
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
| 10      | 1      | msandbox | NULL       | 0      | NULL        | NULL       | NULL       | NULL   | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL          | 0                    | CASELESS     | NULL    | NULL            | 2                     | NULL      | NULL               | NULL          | NULL      | NULL    | NULL    | NULL  | NULL              | NULL           | NULL             | NULL      | NULL   | NULL        | NULL      | NULL                | NULL | 1     | NULL    |
| 20      | 1      | msandbox | NULL       | 0      | NULL        | NULL       | NULL       | NULL   | DISTINCT c FROM sbtest1             | NULL          | 0                    | CASELESS     | NULL    | NULL            | 2                     | NULL      | NULL               | NULL          | NULL      | NULL    | NULL    | NULL  | NULL              | NULL           | NULL             | NULL      | NULL   | NULL        | NULL      | NULL                | NULL | 1     | NULL    |
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
2 rows in set (0.00 sec)

5、重新执行测试

最后我们重新执行sysbench加载:

-- 执行读写测试
sysbench oltp_read_write --mysql_storage_engine=innodb --db-driver=mysql --mysql-host=188.188.0.71 --mysql-port=6033 \
--mysql-user='msandbox' --mysql-password='123456' --mysql-db=sbtest --threads=4 --table_size=10000 --tables=5 \
--time=200 --histogram --report-interval=10 --db-ps-mode=disable run

6、再次查看查询统计信息

1)表 stats.stats_mysql_connection_pool

Admin> SELECT * FROM stats.stats_mysql_connection_pool;
+-----------+-----------+----------+--------------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
| hostgroup | srv_host  | srv_port | status       | ConnUsed | ConnFree | ConnOK | ConnERR | MaxConnUsed | Queries | Queries_GTID_sync | Bytes_data_sent | Bytes_data_recv | Latency_us |
+-----------+-----------+----------+--------------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
| 1         | 127.0.0.1 | 21891    | ONLINE       | 0        | 4        | 12     | 66      | 5           | 5548772 | 0                 | 250629298       | 11033778165     | 93         |
| 1         | 127.0.0.1 | 21892    | OFFLINE_HARD | 0        | 0        | 0      | 0       | 0           | 0       | 0                 | 0               | 0               | 94         |
| 2         | 127.0.0.1 | 21893    | ONLINE       | 0        | 3        | 3      | 0       | 3           | 140634  | 0                 | 3477406         | 176787576       | 91         |
| 2         | 127.0.0.1 | 21892    | ONLINE       | 0        | 3        | 3      | 0       | 3           | 140419  | 0                 | 3472382         | 176972544       | 94         |
+-----------+-----------+----------+--------------+----------+----------+--------+---------+-------------+---------+-------------------+-----------------+-----------------+------------+
4 rows in set (0.00 sec)

这次组2中的2个MySQL也分配到了查询。

2)表stats_mysql_query_digest

Admin> SELECT hostgroup hg, sum_time, count_star, digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC limit 15;
+----+-----------+------------+--------------------------------------------------------------------+
| hg | sum_time  | count_star | digest_text                                                        |
+----+-----------+------------+--------------------------------------------------------------------+
| 1  | 528005107 | 128310     | COMMIT                                                             |
| 1  | 85388244  | 258648     | SELECT c FROM sbtest3 WHERE id=?                                   |
| 1  | 84775526  | 257258     | SELECT c FROM sbtest5 WHERE id=?                                   |
| 1  | 84527137  | 257098     | SELECT c FROM sbtest4 WHERE id=?                                   |
| 1  | 84271753  | 255208     | SELECT c FROM sbtest2 WHERE id=?                                   |
| 2  | 46785931  | 255528     | SELECT c FROM sbtest1 WHERE id=?                                   |
| 1  | 43321724  | 128379     | BEGIN                                                              |
| 1  | 27591062  | 25737      | SELECT DISTINCT c FROM sbtest2 WHERE id BETWEEN ? AND ? ORDER BY c |
| 1  | 27147997  | 25698      | SELECT DISTINCT c FROM sbtest3 WHERE id BETWEEN ? AND ? ORDER BY c |
| 1  | 27139402  | 25743      | SELECT DISTINCT c FROM sbtest5 WHERE id BETWEEN ? AND ? ORDER BY c |
| 1  | 27046866  | 25702      | SELECT DISTINCT c FROM sbtest4 WHERE id BETWEEN ? AND ? ORDER BY c |
| 2  | 26084914  | 25529      | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c |
| 1  | 15539338  | 25714      | SELECT c FROM sbtest2 WHERE id BETWEEN ? AND ? ORDER BY c          |
| 1  | 15535214  | 25786      | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c          |
| 1  | 15499890  | 25664      | SELECT c FROM sbtest3 WHERE id BETWEEN ? AND ? ORDER BY c          |
+----+-----------+------------+--------------------------------------------------------------------+
15 rows in set (0.00 sec)

正如所料,2个类型的查询被发送到hostgroup2(Slave上)。表stats_mysql_query_digest允许聚合结果,例如:

Admin> SELECT hostgroup hg, SUM(sum_time), SUM(count_star) FROM stats_mysql_query_digest GROUP BY hostgroup;
+----+---------------+-----------------+
| hg | SUM(sum_time) | SUM(count_star) |
+----+---------------+-----------------+
| 1  | 1421564103    | 2286517         |
| 2  | 72870845      | 281057          |
+----+---------------+-----------------+
2 rows in set (0.00 sec)

九、查询缓存[Query Caching]

ProxySQL的一个流行用途是充当查询缓存。默认情况下,查询不会被缓存,但可以在mysql_query_rules中设置cache_ttl(以毫秒为单位)。
假设我们还要将发送给Slave的所有查询缓存5秒。

1、修改查询规则

Admin> UPDATE mysql_query_rules SET cache_ttl=5000 WHERE active=1 AND destination_hostgroup=2;
Query OK, 2 rows affected (0.00 sec)

2、将修改加载到RUNTIME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT * FROM runtime_mysql_query_rules;
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
| rule_id | active | username | schemaname | flagIN | client_addr | proxy_addr | proxy_port | digest | match_digest                        | match_pattern | negate_match_pattern | re_modifiers | flagOUT | replace_pattern | destination_hostgroup | cache_ttl | cache_empty_result | cache_timeout | reconnect | timeout | retries | delay | next_query_flagIN | mirror_flagOUT | mirror_hostgroup | error_msg | OK_msg | sticky_conn | multiplex | gtid_from_hostgroup | log | apply | comment |
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
| 10      | 1      | msandbox | NULL       | 0      | NULL        | NULL       | NULL       | NULL   | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL          | 0                    | CASELESS     | NULL    | NULL            | 2                     | 5000      | NULL               | NULL          | NULL      | NULL    | NULL    | NULL  | NULL              | NULL           | NULL             | NULL      | NULL   | NULL        | NULL      | NULL                | NULL | 1     | NULL    |
| 20      | 1      | msandbox | NULL       | 0      | NULL        | NULL       | NULL       | NULL   | DISTINCT c FROM sbtest1             | NULL          | 0                    | CASELESS     | NULL    | NULL            | 2                     | 5000      | NULL               | NULL          | NULL      | NULL    | NULL    | NULL  | NULL              | NULL           | NULL             | NULL      | NULL   | NULL        | NULL      | NULL                | NULL | 1     | NULL    |
+---------+--------+----------+------------+--------+-------------+------------+------------+--------+-------------------------------------+---------------+----------------------+--------------+---------+-----------------+-----------------------+-----------+--------------------+---------------+-----------+---------+---------+-------+-------------------+----------------+------------------+-----------+--------+-------------+-----------+---------------------+-----+-------+---------+
2 rows in set (0.00 sec)

3、清空旧的查询统计记录

Admin> SELECT 1 FROM stats_mysql_query_digest_reset LIMIT 1;
Empty set (0.01 sec)

4、再次执行测试

-- 执行读写测试
sysbench oltp_read_write --mysql_storage_engine=innodb --db-driver=mysql --mysql-host=188.188.0.71 --mysql-port=6033 \
--mysql-user='msandbox' --mysql-password='123456' --mysql-db=sbtest --threads=4 --table_size=10000 --tables=5 \
--time=200 --histogram --report-interval=10 --db-ps-mode=disable run

5、查看stats_mysql_query_digest

Admin> SELECT hostgroup hg, sum_time, count_star, digest_text FROM stats_mysql_query_digest where digest_text like 'SELECT%sbtest1%' ORDER BY sum_time DESC ;
+----+----------+------------+--------------------------------------------------------------------+
| hg | sum_time | count_star | digest_text                                                        |
+----+----------+------------+--------------------------------------------------------------------+
| 2  | 23806132 | 128797     | SELECT c FROM sbtest1 WHERE id=?                                   |
| 2  | 13463802 | 12767      | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c |
| 1  | 7612531  | 12616      | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c          |
| 1  | 5591611  | 12755      | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ?                     |
| 1  | 5260845  | 12806      | SELECT SUM(k) FROM sbtest1 WHERE id BETWEEN ? AND ?                |
| 2  | 409      | 1          | SELECT DISTINCT c FROM sbtest1 WHERE id=?                          |
| 1  | 216      | 1          | SELECT DISTINCAT c FROM sbtest1 WHERE id=?                         |
| -1 | 0        | 7          | SELECT DISTINCT c FROM sbtest1 WHERE id=?                          |
| -1 | 0        | 24         | SELECT c FROM sbtest1 WHERE id=?                                   |
+----+----------+------------+--------------------------------------------------------------------+
9 rows in set (0.00 sec)

可以发现:
1)它们仍然被发送到hostgroup2;
2)如果它们出现在查询缓存中,则不会将它们发送到任何主机组并标记为特殊的主机组-1;

通过以上实验,我们可以通过 mysql_query_rules 中的 cache_ttl 来控制内存占用量和结果集的生命周期:选择 cache_ttl 是明智地,他直到可以获得对查询缓存的更多控制。

特别注意:
目前无法定义查询缓存使用的最大内存量,也无法强制选择性地或完全刷新查询缓存。

十、查询的改写[Query Rewrite]

ProxySQL支持多种方式来匹配查询,如flagIN, username, schemaname。

匹配查询的最常用方法是编写与查询本身的文本相匹配的正则表达式。为了匹配查询的文本,ProxySQL提供了2种机制,使用了2个不同的字段:

  • match_digest :针对查询语句的命令概要(被用?替换掉变量值的SQL语句)的正则表达式;针对的查询语句文本可以在 stats_mysql_query_digest.query_digest中查看。
  • match_pattern :针对原始SQL文本的正则表达式.

为什么会有这些不同的机制?
查询的摘要可能比查询本身小得多(例如,具有几MB数据的INSERT语句),因此对较小的字符串运行正则表达式肯定会更快。因此,如果不需要尝试匹配查询中的特定文字,
建议使用match_digest(它更快)。尽管如此,但如果要重写查询,则必须与原始查询匹配(使用match_pattern),因为它是需要重写的是原始查询。

-- 》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》

1、改写配置举例

将同时包含DISTINCT和ORDER BY c的语句改写成DISTINCT 1的语句:

1)添加查询规则

Admin> INSERT INTO mysql_query_rules (rule_id,active,username,match_pattern,replace_pattern,apply) VALUES (30,1,'msandbox','DISTINCT(.*)ORDER BY c','DISTINCT\1',1);
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id, match_digest, match_pattern, replace_pattern, cache_ttl, apply FROM mysql_query_rules ORDER BY rule_id;
+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
| rule_id | match_digest                        | match_pattern          | replace_pattern | cache_ttl | apply |
+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
| 10      | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL                   | NULL            | 10000     | 1     |
| 20      | DISTINCT c FROM sbtest1             | NULL                   | NULL            | 10000     | 1     |
| 30      | NULL                                | DISTINCT(.*)ORDER BY c | DISTINCT\1      | NULL      | 1     |
+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
3 rows in set (0.00 sec)

2)加载到RUNTME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

3)清空旧的查询统计记录

Admin> SELECT 1 FROM stats_mysql_query_digest_reset LIMIT 1;

4)执行测试

-- 执行读写测试
sysbench oltp_read_write --mysql_storage_engine=innodb --db-driver=mysql --mysql-host=188.188.0.71 --mysql-port=6033 \
--mysql-user='msandbox' --mysql-password='123456' --mysql-db=sbtest --threads=4 --table_size=10000 --tables=5 \
--time=200 --histogram --report-interval=10 --db-ps-mode=disable run

5)查看stats_mysql_query_digest

Admin> SELECT hostgroup hg, sum_time, count_star, digest_text FROM stats_mysql_query_digest ORDER BY sum_time DESC limit 10;
+----+-----------+------------+--------------------------------------------------------------------+
| hg | sum_time  | count_star | digest_text                                                        |
+----+-----------+------------+--------------------------------------------------------------------+
| 1  | 262096703 | 65249      | COMMIT                                                             |
| 1  | 42813558  | 133794     | SELECT c FROM sbtest2 WHERE id=?                                   |
| 1  | 41657095  | 130914     | SELECT c FROM sbtest4 WHERE id=?                                   |
| 1  | 41641341  | 130064     | SELECT c FROM sbtest3 WHERE id=?                                   |
| 1  | 41610083  | 129484     | SELECT c FROM sbtest5 WHERE id=?                                   |
| 2  | 23794597  | 128544     | SELECT c FROM sbtest1 WHERE id=?                                   |
| 1  | 21205569  | 65282      | BEGIN                                                              |
| 2  | 13508337  | 13160      | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ? ORDER BY c |
| 1  | 12705099  | 13233      | SELECT DISTINCT c FROM sbtest2 WHERE id BETWEEN ? AND ?            |
| 1  | 12153358  | 13101      | SELECT DISTINCT c FROM sbtest4 WHERE id BETWEEN ? AND ?            |
+----+-----------+------------+--------------------------------------------------------------------+
10 rows in set (0.00 sec)

看起来有些不对劲,因为似乎没有重写过。这是故意的,所以我们现在可以进行故障排除。

6)规则诊断

对于故障排除有一个非常有用的表就是 stats.stats_mysql_query_rules:

Admin> SELECT hits, mysql_query_rules.rule_id, match_digest, match_pattern, replace_pattern, cache_ttl, apply 
    -> FROM mysql_query_rules NATURAL JOIN stats.stats_mysql_query_rules ORDER BY mysql_query_rules.rule_id;
+------+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
| hits | rule_id | match_digest                        | match_pattern          | replace_pattern | cache_ttl | apply |
+------+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
| 4    | 10      | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL                   | NULL            | 10000     | 1     |
| 4    | 20      | DISTINCT c FROM sbtest1             | NULL                   | NULL            | 10000     | 1     |
| 16   | 30      | NULL                                | DISTINCT(.*)ORDER BY c | DISTINCT\1      | NULL      | 1     |
+------+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
3 rows in set (0.01 sec)

似乎有些不对:带有rule_id = 30的规则有0次命中!

问题是带有rule_id = 20的规则也匹配在rule_id = 30中匹配的查询,而未匹配到规则20的查询,由于20中设置的apply = 1,这将阻止它们达到规则30。
(因为apply = 1时,未匹配的查询将被发送到默认组处理)

7)修该规则

Admin> UPDATE mysql_query_rules SET apply=0 WHERE rule_id=20;
Query OK, 1 row affected (0.00 sec)

Admin> SELECT rule_id, match_digest, match_pattern, replace_pattern, cache_ttl, apply FROM mysql_query_rules ORDER BY rule_id;
+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
| rule_id | match_digest                        | match_pattern          | replace_pattern | cache_ttl | apply |
+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
| 10      | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL                   | NULL            | 10000     | 1     |
| 20      | DISTINCT c FROM sbtest1             | NULL                   | NULL            | 10000     | 0     |
| 30      | NULL                                | DISTINCT(.*)ORDER BY c | DISTINCT\1      | NULL      | 1     |
+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
3 rows in set (0.00 sec)

8)将修改后的配置加载到RUNTME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id,apply FROM runtime_mysql_query_rules;
+---------+-------+
| rule_id | apply |
+---------+-------+
| 10      | 1     |
| 20      | 0     |
| 30      | 1     |
+---------+-------+
3 rows in set (0.00 sec)

注意:运行LOAD MYSQL QUERY RULES TO RUNTIME时,不仅会重置内部查询处理结构,还会重置stats.stats_mysql_query_rules中的计数器。

Admin> SELECT * FROM stats.stats_mysql_query_rules;
+---------+------+
| rule_id | hits |
+---------+------+
| 10      | 0    |
| 20      | 0    |
| 30      | 0    |
+---------+------+
3 rows in set (0.00 sec)

9)清空旧的查询统计记录

Admin> SELECT 1 FROM stats_mysql_query_digest_reset LIMIT 1;

10)再次执行测试

-- 执行读写测试
sysbench oltp_read_write --mysql_storage_engine=innodb --db-driver=mysql --mysql-host=188.188.0.71 --mysql-port=6033 \
--mysql-user='msandbox' --mysql-password='123456' --mysql-db=sbtest --threads=4 --table_size=10000 --tables=5 \
--time=200 --histogram --report-interval=10 --db-ps-mode=disable run

11)再次查看规则命中情况

Admin> SELECT hits, mysql_query_rules.rule_id, match_digest, match_pattern, replace_pattern, cache_ttl, apply
    -> FROM mysql_query_rules NATURAL JOIN stats.stats_mysql_query_rules ORDER BY mysql_query_rules.rule_id;
+------+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
| hits | rule_id | match_digest                        | match_pattern          | replace_pattern | cache_ttl | apply |
+------+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
| 4    | 10      | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL                   | NULL            | 10000     | 1     |
| 4    | 20      | DISTINCT c FROM sbtest1             | NULL                   | NULL            | 10000     | 0     |
| 20   | 30      | NULL                                | DISTINCT(.*)ORDER BY c | DISTINCT\1      | NULL      | 1     |
+------+---------+-------------------------------------+------------------------+-----------------+-----------+-------+
3 rows in set (0.00 sec)

这次规则30被命中了20次,有重写了,看起来不错:-)
那查询执行情况怎么样?

12)查看 stats_mysql_query_digest

Admin> SELECT hostgroup hg, sum_time, count_star, digest_text FROM stats_mysql_query_digest WHERE digest_text like '%sbtest1%' ORDER BY sum_time DESC;
+----+----------+------------+-------------------------------------------------------------+
| hg | sum_time | count_star | digest_text                                                 |
+----+----------+------------+-------------------------------------------------------------+
| 1  | 6127168  | 4856       | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ?+? ORDER BY c |
| 2  | 4264263  | 7359       | SELECT c FROM sbtest1 WHERE id=?                            |
| 1  | 4081063  | 4856       | UPDATE sbtest1 SET k=k+? WHERE id=?                         |
| 1  | 3497644  | 4856       | SELECT c FROM sbtest1 WHERE id BETWEEN ? AND ?+?            |
| 1  | 3270527  | 4856       | DELETE FROM sbtest1 WHERE id=?                              |
| 1  | 3193123  | 4856       | SELECT SUM(K) FROM sbtest1 WHERE id BETWEEN ? AND ?+?       |
| 1  | 3124698  | 4856       | UPDATE sbtest1 SET c=? WHERE id=?                           |
| 1  | 2866474  | 4856       | INSERT INTO sbtest1 (id, k, c, pad) VALUES (?, ?, ?, ?)     |
| 2  | 1889996  | 1633       | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+?   |
| -1 | 0        | 3223       | SELECT DISTINCT c FROM sbtest1 WHERE id BETWEEN ? AND ?+?   |
| -1 | 0        | 41201      | SELECT c FROM sbtest1 WHERE id=?                            |
+----+----------+------------+-------------------------------------------------------------+
11 rows in set (0.00 sec)

注意:
rule_id = 20和rule_id = 30的规则可以合并为一个规则。这里把它们被分开是为了描述应用字段的重要性,并且不仅多个规则可以匹配相同的查询,而且多个规则可以将设置转换并应用于同一查询。

-- 》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》

2、查询重写的几个例子。

我们想要重写如下查询:
将查询
SELECT c FROM sbtest1 WHERE id=?
转换为入:
SELECT c FROM sbtest2 WHERE id=?

但对于ID的值,只知道1000到3999之间的...;这是没有任何意义的,它只是展示一些潜力,包括一些复杂的分片能力!!
看来得使用正则表达式匹配? :-)

1)添加对应得规则

Admin> INSERT INTO mysql_query_rules (rule_id,active,username,match_pattern,replace_pattern,apply) VALUES (5,1,'msandbox','^SELECT (c) FROM sbtest(1) WHERE id=(1|2|3)(...)$','SELECT c FROM sbtest2 WHERE id=\3\4',1);
Query OK, 1 row affected (0.00 sec)

说明:选择"c"和"1"(在sbtest1中)只是为了显示语法。

2)查看规则添加结果

Admin> SELECT rule_id, match_digest, match_pattern, replace_pattern, cache_ttl, apply FROM mysql_query_rules ORDER BY rule_id;
+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
| rule_id | match_digest                        | match_pattern                                     | replace_pattern                     | cache_ttl | apply |
+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
| 5       | NULL                                | ^SELECT (c) FROM sbtest(1) WHERE id=(1|2|3)(...)$ | SELECT c FROM sbtest2 WHERE id=\3\4 | NULL      | 1     |
| 10      | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL                                              | NULL                                | 10000     | 1     |
| 20      | DISTINCT c FROM sbtest1             | NULL                                              | NULL                                | 10000     | 0     |
| 30      | NULL                                | DISTINCT(.*)ORDER BY c                            | DISTINCT\1                          | NULL      | 1     |
+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
4 rows in set (0.00 sec)

3)将修改后的配置加载到RUNTME层

Admin> LOAD MYSQL QUERY RULES TO RUNTIME;
Query OK, 0 rows affected (0.00 sec)

Admin> SELECT rule_id, match_digest, match_pattern, replace_pattern, cache_ttl, apply FROM runtime_mysql_query_rules ORDER BY rule_id;
+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
| rule_id | match_digest                        | match_pattern                                     | replace_pattern                     | cache_ttl | apply |
+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
| 5       | NULL                                | ^SELECT (c) FROM sbtest(1) WHERE id=(1|2|3)(...)$ | SELECT c FROM sbtest2 WHERE id=\3\4 | NULL      | 1     |
| 10      | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL                                              | NULL                                | 10000     | 1     |
| 20      | DISTINCT c FROM sbtest1             | NULL                                              | NULL                                | 10000     | 0     |
| 30      | NULL                                | DISTINCT(.*)ORDER BY c                            | DISTINCT\1                          | NULL      | 1     |
+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
4 rows in set (0.00 sec)

4)清空旧的查询统计记录

Admin> SELECT 1 FROM stats_mysql_query_digest_reset LIMIT 1;

5)再次执行测试

执行由123组成得id:

mysql> SELECT c FROM sbtest1 WHERE id=1123;

它有用吗?显然是的:)

6)再次查看规则命中情况

Admin> SELECT hits, mysql_query_rules.rule_id, match_digest, match_pattern, replace_pattern, cache_ttl, apply FROM mysql_query_rules NATURAL JOIN stats.stats_mysql_query_rules;
+------+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
| hits | rule_id | match_digest                        | match_pattern                                     | replace_pattern                     | cache_ttl | apply |
+------+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
| 3    | 5       | NULL                                | ^SELECT (c) FROM sbtest(1) WHERE id=(1|2|3)(...)$ | SELECT c FROM sbtest2 WHERE id=\3\4 | NULL      | 1     |
| 7    | 10      | ^SELECT c FROM sbtest1 WHERE id=\?$ | NULL                                              | NULL                                | 10000     | 1     |
| 4    | 20      | DISTINCT c FROM sbtest1             | NULL                                              | NULL                                | 10000     | 0     |
| 20   | 30      | NULL                                | DISTINCT(.*)ORDER BY c                            | DISTINCT\1                          | NULL      | 1     |
+------+---------+-------------------------------------+---------------------------------------------------+-------------------------------------+-----------+-------+
4 rows in set (0.00 sec)

每执行一次测试得查询语句,规则5得命中就增加1。

十一、将配置持久化到DISK层

将实验中涉及到的配置,持久化到DISK层。

Admin> SAVE MYSQL SERVERS TO DISK;
Query OK, 0 rows affected (0.02 sec)

Admin> SAVE MYSQL USERS TO DISK;
Query OK, 0 rows affected (0.01 sec)

Admin> SAVE MYSQL QUERY RULES TO DISK;
Query OK, 0 rows affected (0.02 sec)

~
~
完毕!

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章