MySQL中間件之ProxySQL總結
1、ProxySQL簡介
ProxySQL爲MySQL的中間件，其有兩個版本，官方版和percona版，percona版是基於官方版基礎上修改而來。ProxySQL是用C++語言開發的，雖然也是一個輕量級產品，但性能很好(據測試，能處理千億級的數據)，功能也足夠，能滿足中間件所需的絕大多數功能，包括：
1)最基本的讀/寫分離，且方式有多種。
2)可定製基於用戶、基於schema、基於語句的規則對SQL語句進行路由。換句話說，規則很靈活。基於schema和與語句級的規則，可以實現簡單的sharding。
3)可緩存查詢結果。雖然ProxySQL的緩存策略比較簡陋，但實現了基本的緩存功能，絕大多數時候也夠用了。此外，作者已經打算實現更豐富的緩存策略。
4)監控後端節點。ProxySQL可以監控後端節點的多個指標，包括：ProxySQL和後端的心跳信息，後端節點的read-only/read-write，slave和master的數據同步延遲性(replication lag)。
2、proxysql的配置系統體系結構
整個配置系統分爲三層，如下所示:
proxysql的每一個配置項在三層中都存在，但是這三層是獨立的，也就是說，proxysql可以同時擁有三份配置，每層都是獨立的，可能三份都不一樣，可能三份都一樣。
RUNTIME這個頂級層，就是proxysql運行過程中實際採用的那一份配置，這一份配置就是要影響實際生產的，所以將你的配置加進RUNTIME層時需要三思而行。
MEMORY這個中間層，上面接着生產配置項RUNTIME，下面接着持久化層DISK,CONFIG FILE。MEMORY也是我們修改proxysql的唯一正常入口。一般的，我們修改一個配置，先改Memory，確認無誤後再接入生產(RUNTIME)，和持久化到磁盤(DISK).也就是說memeory裏面的配置隨便改，不影響生產，也不影響磁盤中保存的數據。
DISK和CONFIG FILE這一層是持久化層，我們做的任何配置更改，如果不持久化下來，重啓後，配置都將丟失。持久化層主要將數據存儲在sqlite(`$(DATADIR)/proxysql.db`.)數據庫中和文件中(/etc/proxysql.cnf)。
當proxysql啓動時，首先讀取配置文件CONFIG FILE(/etc/proxysql.cnf)，然後從該配置文件中獲取datadir，datadir中存儲的是sqlite的數據目錄。如果該目錄存在，且sqlite數據文件存在，那麼正常啓動，將sqlite中的配置項讀進內存，並且加載進RUNTIME，用於初始化proxysql的運行。如果datadir目錄下沒有找到sqlite的數據文件，proxysql就會使用config file中的配置來初始化proxysql，並且將這些配置保存至數據庫。
3、安裝部署
3.1、安裝包下載
安裝包有兩個地方可以下載，
· percona站點:
https://www.percona.com/downloads/proxysql/
· github：
https://github.com/sysown/ProxySQL
我們選擇proxysql-2.0.1-1-centos7.x86_64.rpm,rpm包直接裝。
3.2、安裝
直接使用rpm安裝即可:
[root@proxysql ~]# rpm -ivh proxysql-2.0.1-1-centos7.x86_64.rpm
3.3、啓動ProxySQL
[root@proxysql ~]#service ProxySQL start
3.4、ProxySQL相關文件
[root@proxysql]# pwd
/var/lib/proxysql
[root@proxysql]# ll
total 300
-rw-rw-r--. 1 proxysql proxysql 1050 Dec 12 13:34 proxysql-ca.pem
-rw-rw-r--. 1 proxysql proxysql 1058 Dec 12 13:34 proxysql-cert.pem
-rw-------. 1 proxysql proxysql 151552 Dec 12 13:34 proxysql.db
-rw-rw-r--. 1 proxysql proxysql 1679 Dec 12 13:34 proxysql-key.pem
-rw-------. 1 proxysql proxysql 3732 Dec 12 13:34 proxysql.log
-rw-r--r--. 1 proxysql proxysql 6 Dec 12 13:34 proxysql.pid
-rw-------. 1 proxysql proxysql 135168 Dec 12 15:00 proxysql_stats.db
[root@proxysql]# ls -al /etc/proxysql.cnf
-rw-r-----. 1 root proxysql 6282 Jan 25 2019 /etc/proxysql.cnf
ProxySQL.db是SQLITE的數據文件，Proxysql很多東西是存儲在這個數據庫裏面的。ProxySQL.log是日誌文件，排查問題好地方。ProxySQL.pid這個pid文件不多說了。ProxySQL.cnf是ProxySQL的一些靜態配置項，比如一些啓動選項，sqlite的數據目錄等等。
3.5、ProxySQL的相關進程
[root@proxysql]# ps -ef| grep proxysql
proxysql 23958 1 0 13:34 ? 00:00:00 proxysql -c /etc/proxysql.cnf -D /var/lib/proxysql
proxysql 23959 23958 0 13:34 ? 00:00:05 proxysql -c /etc/proxysql.cnf -D /var/lib/proxysql
和MySQL的很相似，我們啓動一個進程，然後fork出一個子進程，父進程負責監控子進程運行狀況如果掛了則拉起來，子進程負責執行真正的任務。
3.6、登錄ProxySQL
[root@proxysql]# mysql -u admin -padmin -h 127.0.0.1 -P6032
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 3
Server version: 5.5.30 (ProxySQL Admin Module)
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [(none)]> show dtabases;
ERROR 1045 (28000): near "show": syntax error
MySQL [(none)]> show databases;
+-----+---------------+-------------------------------------+
| seq | name | file |
+-----+---------------+-------------------------------------+
| 0 | main | |
| 2 | disk | /var/lib/proxysql/proxysql.db |
| 3 | stats | |
| 4 | monitor | |
| 5 | stats_history | /var/lib/proxysql/proxysql_stats.db |
+-----+---------------+-------------------------------------+
5 rows in set (0.00 sec)
MySQL [(none)]> show tables;
+--------------------------------------------+
| tables |
+--------------------------------------------+
| global_variables |
| mysql_collations |
| mysql_galera_hostgroups |
| mysql_group_replication_hostgroups |
| mysql_query_rules |
| mysql_query_rules_fast_routing |
| mysql_replication_hostgroups |
| mysql_servers |
| mysql_users |
| proxysql_servers |
| runtime_checksums_values |
| runtime_global_variables |
| runtime_mysql_galera_hostgroups |
| runtime_mysql_group_replication_hostgroups |
| runtime_mysql_query_rules |
| runtime_mysql_query_rules_fast_routing |
| runtime_mysql_replication_hostgroups |
| runtime_mysql_servers |
| runtime_mysql_users |
| runtime_proxysql_servers |
| runtime_scheduler |
| scheduler |
+--------------------------------------------+
22 rows in set (0.00 sec)
從show tables可以看出，主要分mysql 開頭的和runtime開頭的表。
Mysql開頭的主要是memory層的配置，通過sql修改配置時，主要就是改變這一層的數據。Runtime是實時生效的配置信息。
3.7、如何修改配置
通過sql 語句修改mysql開頭的表。
| global_variables |
| mysql_collations |
| mysql_galera_hostgroups |
| mysql_group_replication_hostgroups |
| mysql_query_rules |
| mysql_query_rules_fast_routing |
| mysql_replication_hostgroups |
| mysql_servers |
| mysql_users
將配置加載到runtime和保存到disk上。
LOAD MYSQL SERVERS TO RUNTIME;
SAVE MYSQL SERVERS RULES TO DISK;
LOAD MYSQL USERS TO RUNTIME;
SAVE MYSQL USERS RULES TO DISK;
LOAD MYSQL VARIABLES TO RUNTIME;
SAVE MYSQL VARIABLES RULES TO DISK;
LOAD MYSQL QUERY RULES TO RUNTIME;
SAVE MYSQL QUERY RULES TO DISK;
4、ProxySQL 主從讀寫分離配置
4.1、環境說明：
maste (10.100.251.221:3306) , mysql master
slave01 (10.100.251.222:3306) , mysql slave
slave02+ proxysql (10.100.251.223:3306) , mysql slave
vip : 10.100.251.238
4.2、配置
4.2.1、添加數據庫信息
把一主兩從的mysql數據庫信息添加到proxysql中。我們將主庫master也就是做寫入的節點放到group 10中，salve節點做讀放到group20。
[root@slave02 ~]# mysql -uadmin -padmin -h127.0.0.1 -P6032
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.5.30 (ProxySQL Admin Module)
Copyright (c) 2000, 2017, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MySQL [(none)]> INSERT INTO mysql_servers(hostgroup_id, hostname, port) VALUES (10,'10.100.251.221',3306);
Query OK, 1 row affected (0.00 sec)
MySQL [(none)]> INSERT INTO mysql_servers(hostgroup_id, hostname, port) VALUES (20,'10.100.251.222',3306);
Query OK, 1 row affected (0.00 sec)
MySQL [(none)]> INSERT INTO mysql_servers(hostgroup_id, hostname, port) VALUES (20,'10.100.251.223',3306);
Query OK, 1 row affected (0.00 sec)
MySQL [(none)]> select hostgroup_id,hostname,port,status from mysql_servers;
+--------------+----------------+------+--------+
| hostgroup_id | hostname | port | status |
+--------------+----------------+------+--------+
| 10 | 10.100.251.221 | 3306 | ONLINE |
| 20 | 10.100.251.222 | 3306 | ONLINE |
| 20 | 10.100.251.223 | 3306 | ONLINE |
+--------------+----------------+------+--------+
LOAD MYSQL SERVERS TO RUNTIME;
/*將配置應用於proxysql運行環境*/
SAVE MYSQL SERVERS TO DISK;
/*將配置存儲到sqlite數據庫中*/
4.2.2、添加數據庫監控賬號和業務賬號
在數據庫master中需要配置監控賬號(ProxySQL)和應用賬號(pstest)。以下SQL在master節點執行。
CREATE USER 'ProxySQL'@'%' IDENTIFIED BY 'ProxySQL';
GRANT USAGE ON *.* TO 'ProxySQL'@'%';
CREATE USER 'pstest'@'%' IDENTIFIED BY 'pstest';
GRANT ALL ON * . * TO 'pstest'@'%';
FLUSH PRIVILEGES;
確認賬號創建成功：
mysql [(none)]> select host,user from mysql.user;
+-----------------------+----------+
| host | user |
+-----------------------+----------+
| % | ProxySQL |
| % | proxysql |
| % | pstest |
4.2.3、在proxysql中添加賬號
INSERT INTO mysql_users(username,password,default_hostgroup) VALUES ('pstest','pstest',10);
UPDATE global_variables SET variable_value='ProxySQL' WHERE variable_name='mysql-monitor_username';
UPDATE global_variables SET variable_value='ProxySQL' WHERE variable_name='mysql-monitor_password';
設置讀寫分組(配置MHA纔用到，能自動檢測主從切換，如果發現組從切換，底下表會自動更改。)
INSERT INTO mysql_replication_hostgroups VALUES(10,20,'read_only','test');
SELECT * FROM mysql_replication_hostgroups;
MySQL [(none)]> select * from runtime_mysql_replication_hostgroups;
+------------------+------------------+------------+---------+
| writer_hostgroup | reader_hostgroup | check_type | comment |
+------------------+------------------+------------+---------+
| 10 | 20 | read_only | test |
+------------------+------------------+------------+---------+
LOAD MYSQL VARIABLES TO RUNTIME;
LOAD MYSQL USERS TO RUNTIME;
/*將配置應用於proxysql運行環境*/
SAVE MYSQL VARIABLES TO DISK;
SAVE MYSQL USERS TO DISK;
/*將配置存儲到sqlite數據庫中*/
查看proxysql監控數據庫的一些指標
MySQL [(none)]> SELECT * FROM monitor.mysql_server_connect_log ORDER BY time_start_us DESC LIMIT 10;
+----------------+------+------------------+-------------------------+---------------+
| hostname | port | time_start_us | connect_success_time_us | connect_error |
+----------------+------+------------------+-------------------------+---------------+
| 10.100.251.223 | 3306 | 1576137235801049 | 781 | NULL |
| 10.100.251.221 | 3306 | 1576137235142148 | 1261 | NULL |
| 10.100.251.222 | 3306 | 1576137234483025 | 1167 | NULL |
| 10.100.251.222 | 3306 | 1576137175889798 | 1482 | NULL |
| 10.100.251.221 | 3306 | 1576137175186269 | 1506 | NULL |
| 10.100.251.223 | 3306 | 1576137174482801 | 809 | NULL |
| 10.100.251.222 | 3306 | 1576137115351565 | 1375 | NULL |
| 10.100.251.223 | 3306 | 1576137114917005 | 705 | NULL |
| 10.100.251.221 | 3306 | 1576137114482624 | 1518 | NULL |
| 10.100.251.223 | 3306 | 1576137082353092 | 859 | NULL |
+----------------+------+------------------+-------------------------+---------------+
MySQL [(none)]> SELECT * FROM monitor.mysql_server_ping_log ORDER BY time_start_us DESC LIMIT 10;
+----------------+------+------------------+----------------------+------------+
| hostname | port | time_start_us | ping_success_time_us | ping_error |
+----------------+------+------------------+----------------------+------------+
| 10.100.251.223 | 3306 | 1576137324343870 | 200 | NULL |
| 10.100.251.221 | 3306 | 1576137324257100 | 575 | NULL |
| 10.100.251.222 | 3306 | 1576137324170319 | 531 | NULL |
| 10.100.251.223 | 3306 | 1576137314311303 | 167 | NULL |
| 10.100.251.221 | 3306 | 1576137314240587 | 373 | NULL |
| 10.100.251.222 | 3306 | 1576137314170297 | 519 | NULL |
| 10.100.251.222 | 3306 | 1576137304408942 | 403 | NULL |
| 10.100.251.221 | 3306 | 1576137304289561 | 312 | NULL |
| 10.100.251.223 | 3306 | 1576137304170287 | 235 | NULL |
| 10.100.251.223 | 3306 | 1576137294348098 | 174 | NULL |
+----------------+------+------------------+----------------------+------------+
4.2.4、測試分發情況
使用業務用戶通過proxysql登錄mysql,默認沒有配置規則所有都只會往master節點轉發。
[root@mysql5 ~]# mysql -upstest -ppstest -h10.100.251.223 -P6033 -e "select @@server_id"
+-------------+
| @@server_id |
+-------------+
| 1 |
+-------------+
查看統計統計信息，所有的分發都到group 10。
MySQL [(none)]> select * from stats_mysql_query_digest order by sum_time desc;
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
| hostgroup | schemaname | username | client_address | digest | digest_text | count_star | first_seen | last_seen | sum_time | min_time | max_time |
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
| 10 | information_schema | pstest | | 0xDA65260DF35B8D13 | select @@server_id | 42 | 1576137416 | 1576137785 | 17728 | 303 | 822 |
| 10 | information_schema | pstest | | 0xAE77E0F7B80AFC0B | select @@version | 1 | 1576137405 | 1576137405 | 1828 | 1828 | 1828 |
| 10 | information_schema | pstest | | 0xE6E1D6C08ABD4EC8 | select @@server-id | 1 | 1576137412 | 1576137412 | 567 | 567 | 567 |
| 10 | information_schema | pstest | | 0x226CD90D52A2BA0B | select @@version_comment limit ? | 44 | 1576137405 | 1576137785 | 0 | 0 | 0 |
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
查看命令執行情況
select * from stats_mysql_commands_counters;
4.2.5、讀寫分離配置
登錄proxysql配置路由項。
INSERT INTO mysql_query_rules(active,match_pattern,destination_hostgroup,apply) VALUES(1,'^SELECT.*FOR UPDATE$',10,1);
INSERT INTO mysql_query_rules(active,match_pattern,destination_hostgroup,apply) VALUES(1,'^SELECT',20,1);
LOAD MYSQL QUERY RULES TO RUNTIME;
SAVE MYSQL QUERY RULES TO DISK;
active表示是否啓用這個sql路由項，
match_pattern就是我們正則匹配項，
destination_hostgroup表示我們要將該類sql轉發到哪些mysql上面去，這裏我們將select轉發到group 1，也就是兩個slave上。
apply爲1表示該正則匹配後，將不再接受其他匹配，直接轉發。
添加了sql路由，我們來看看是否實現了讀寫分離。
首先記得清空proxysql的query統計
MySQL [(none)]> select * from stats_mysql_query_digest order by sum_time desc;
Empty set (0.01 sec)
4.2.6、測試讀寫分離
1）執行select測試腳本
for ((i=0;i<30;i++)); do mysql -upstest -ppstest -h10.100.251.223 -P6033 -e "select @@server_id"; sleep 0.1; done
MySQL [(none)]> select * from stats_mysql_query_digest order by sum_time desc;
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
| hostgroup | schemaname | username | client_address | digest | digest_text | count_star | first_seen | last_seen | sum_time | min_time | max_time |
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
| 20 | information_schema | pstest | | 0xDA65260DF35B8D13 | select @@server_id | 30 | 1576138545 | 1576138549 | 12956 | 254 | 696 |
| 10 | information_schema | pstest | | 0x226CD90D52A2BA0B | select @@version_comment limit ? | 30 | 1576138545 | 1576138549 | 0 | 0 | 0 |
+-----------+--------------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
2 rows in set (0.01 sec)
可以看到查詢分發到了group 20上
2）執行insert測試腳本
for ((i=0;i<30;i++)); do mysql -upstest -ppstest -h10.100.251.223 -P6033 test -e "insert into t1 values(1)"; sleep 0.1; done
MySQL [(none)]> select * from stats_mysql_query_digest order by sum_time desc ;
+-----------+------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
| hostgroup | schemaname | username | client_address | digest | digest_text | count_star | first_seen | last_seen | sum_time | min_time | max_time |
+-----------+------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
| 10 | test | pstest | | 0x3C44D988579DAFFA | insert into t1 values(?) | 30 | 1576138894 | 1576138897 | 95755 | 2285 | 12862 |
| 10 | test | pstest | | 0x226CD90D52A2BA0B | select @@version_comment limit ? | 30 | 1576138894 | 1576138897 | 0 | 0 | 0 |
+-----------+------------+----------+----------------+--------------------+----------------------------------+------------+------------+------------+----------+----------+----------+
可以看到所有insert分發到了group 10上。