前面給大家介紹過：MySQL 高可用方案選型解析，今天我將詳細的爲大家介紹Centos 7.5 基於 MySQL 5.7的 InnoDB Cluster 多節點高可用集羣環境部署的相關知識，希望大家能夠從中收穫多多！如有幫助，請點在看、轉發支持一波！！！
MySQL InnoDB Cluster 介紹
MySQL的高可用架構無論是社區還是官方，一直在技術上進行探索，這麼多年提出了多種解決方案，比如 MMM, MHA, NDB Cluster, Galera Cluster, InnoDB Cluster, 騰訊的PhxSQL, MySQL Fabric., aliSQL。MySQL官方在2017年4月推出了一套完整的、高可用的Mysql解決方案 - MySQL InnoDB Cluster, 即一組MySQL服務器可以配置爲一個MySQL集羣。在默認的單主節點模式下，集羣服務器具有一個讀寫主節點和多個只讀輔節點。輔助服務器是主服務器的副本。客戶端應用程序通過MySQL Router連接到主服務程序。如果主服務連接失敗，則次要的節點自動提升爲主節點，MySQL Router請求到新的主節點。InnoDB Cluster不提供NDB Cluster支持。更多關於MySQL學習的文章，請參閱：死磕數據庫系列之 MySQL ，本系列持續更新中。
分佈式MySQL之InnoDB和NDB
分佈式MySQL主要有InnoDB和NDB模式, NDB是基於集羣的引擎-數據被自動切分並複製到數個機器上（數據節點）， 適合於那些需要極高查詢性能和高可用性的應用， 原來是爲愛立信的電信應用設計的。 NDB提供了高達99.999%的可靠性，在讀操作多的應用中表現優異。 對於有很多併發寫操作的應用， 還是推薦用InnoDB。
========== NDB和InnoDB存儲引擎之間的特性差異 ==========
InnoDB(MySQL 5.7)特性:
- InnoDB版本：InnoDB 5.7.20；
- NDB Cluster版本：不支持；
- 最大存儲長度：64TB；
- 事物：所有標準事物類型；
- 多版本併發控制：支持；
- 數據壓縮：支持；
- 大行支：VARBINARY、VARCHAR、BLOB；
- 同步支持：半同步、異步；
- 塊讀取：支持；
- 塊寫入：需要使用水平分區；
- 高可用性：高；
NDB 7.5/7.6特性：
- InnoDB版本：InnoDB 5.7.20；
- NDB Cluster版本： NDB 7.5.8/7.6.4；
- 最大存儲長度：128TB；
- 事物：讀提交；
- 多版本併發控制：不支持；
- 數據壓縮支持：不支持；
- 大行支持：BLOB、 TEXT；
- 同步支持半：自動同步；
- 塊讀取：支持；
- 塊寫入：支持；
- 高可用性：非常高；
Mysql InnoDB Cluster 工作原理和流程
MySQL InnoDB集羣提供了一個集成的，本地的，HA解決方案。Mysq Innodb Cluster是利用組複製的 pxos 協議，保障數據一致性，組複製支持單主模式和多主模式。
MySQL InnoDB 集羣由以下幾部分組成：
MySQL Servers with Group Replication：向集羣的所有成員複製數據，同時提供容錯、自動故障轉移和彈性。MySQL Server 5.7.17或更高的版本。MySQL Router：確保客戶端請求是負載平衡的，並在任何數據庫故障時路由到正確的服務器。MySQL Router 2.1.3或更高的版本。MySQL Shell：通過內置的管理API創建及管理Innodb集羣。MySQL Shell 1.0.9或更高的版本。
各個組件的關係和工作流程如下:
MySQL InnoDB Cluster 集羣特性
集成易用
MySQL InnoDB集羣緊密集成了MySQL Servers with Group Replication，MySQL Router，和MySQL Shell，所以不必依賴於外部工具，腳本或其他部件。 另外它利用了現有的MySQL特性，如：InnoDB, GTIDs, binary logs, multi-threaded slave execution, multi-source replication and Performance Schema。可以在五分鐘內利用MySQL Shell中的腳本化的管理API來創建及管理MySQL集羣。
使用組複製的 mysql server HA
組複製提供了內置的組成員管理、數據一致性保證、衝突檢測和處理、節點故障檢測和數據庫故障轉移相關操作的本地高可用性，無需人工干預或自定義工具。組複製同時實現了帶自動選主的單主模式及任意更新的多主模式。通過使用一個強大的新的組通信系統，它提供了流行的Paxos算法的內部實現，來自動協調數據複製、一致性、membership。這提供了使MySQL數據庫高度可用所需的所有內置機制。
彈性
通過組複製，一組服務器協調在一起形成一個組。組成員是動態的，服務器可以自願或強制的地離開及隨時加入。組將根據需要自動重新配置自己，並確保任何加入成員與組同步。這樣就可以方便地在需要時快速地調整數據庫的總容量。
故障檢測
組複製實現了一個分佈式故障檢測器來查找並報告failed或不再參與組的服務器，組中剩餘成員將重新配置。
容錯
組複製基於流行的Paxos分佈式算法來提供服務器之間的分佈式協調。爲了使一個小組繼續發揮作用，它要求大多數成員在線，並就每一個變化達成協議。這允許MySQL數據庫在發生故障時安全地繼續操作，而無需人工干預，不存在數據丟失或數據損壞的風險。
自愈
如果一個服務器加入該組，它將自動將其狀態與現有成員同步。如果服務器離開該組，例如它被取下來進行維護，剩下的服務器將看到它已離開，並將自動重新配置組。當服務器後重新加入組，它會自動重新與組同步。
監測
MySQL Enterprise Monitor 3.4及以後的版本全面支持組複製；監控每個節點的配置，健康，和性能。並且提供最佳實踐建議和提醒，以及易於理解的可視化工具，允許您輕鬆地監控和管理您的組複製和InnoDB集羣。
通過MySQL Router爲mysql客戶機應用程序實現HA
MySQL的路由器允許您輕鬆遷移您的獨立的MySQL實例到本地分佈式高可用集羣而不影響現有的應用程序。新metadata_cache插件爲Innodb 集羣提供了透明的客戶端連接路由、負載平衡和故障轉移的能力。
簡單易用的 MySQL shell
MySQL Shell爲所有MySQL相關的任務提供了一個直觀、靈活、功能強大的接口。 新的adminapi使得它很容易用一種自我描述的自然語言來創建，監控和管理包括MySQL Router在內的MySQL InnoDB集羣，而不需要了解低層次的概念，配置選項，或其他複雜的方面。
基於 MySQL5.7 的 InnoDB Cluster 高可用部署
下面部署採用InnoDB Cluster, 每臺服務器實例都運行MySQL Group Replication (即冗餘複製機制，內置failover), MGR有兩種模式，一種是Single-Primary，一種是Multi-Primary，即單主或者多主。
需求注意：模式Multi-Primary中，所有的節點都是主節點，都可以同時被讀寫，看上去這似乎更好，但是因爲多主的複雜性，在功能上如果設置了多主模式，則會有一些使用的限制，比如不支持Foreign Keys with Cascading Constraints。更多關於MySQL學習的文章，請參閱：死磕數據庫系列之 MySQL ，本系列持續更新中。
環境準備
這裏準備了4臺centos7版本的服務器用來部署innodb cluster多節點集羣環境 (至少也要需要3臺服務器), 其中:
- db-node01、db-node02、db-node03 作爲 cluster 節點服務器, 三個節點都要安裝 mysql8.0.x 與 mysql-shell
- db-route01 作爲管理節點服務器，用來負責創建 cluster，並作爲 cluster 的路由, 該節點需要安裝mysql-shell、mysql-router
- 所有節點的python版本要在2.7以上
ip地址 主機名 角色 安裝軟件
172.16.60.211 db-node01 cluster節點1 Mysql5.7, mysql-shell
172.16.60.212 db-node02 cluster節點2 Mysql5.7, mysql-shell
172.16.60.213 db-node03 cluster節點3 Mysql5.7, mysql-shell
172.16.60.214 db-route01 管理節點1 mysql-shell, mysql-route
[root@db-node01 ~]# cat /etc/redhat-release
CentOS Linux release 7.5.1804 (Core)
[root@db-node01 ~]# python -V
Python 2.7.5
爲了方便實驗，關閉所有節點的防火牆
[root@MGR-node1 ~]# systemctl stop firewalld
[root@MGR-node1 ~]# firewall-cmd --state
not running
關閉每個節點的selinux
[root@MGR-node1 ~]# cat /etc/sysconfig/selinux |grep "SELINUX=disabled"
SELINUX=disabled
[root@MGR-node1 ~]# setenforce 0
setenforce: SELinux is disabled
[root@MGR-node1 ~]# getenforce
Disabled
配置每個節點的/etc/hosts主機映射, 方便通過節點的hostname進行連接。這一步很重要，否則可能會出現無法同步的情況，因爲數據庫需要根據member_host同步，如果不配置，默認就是localhost，這樣時無法通信的!!!
[root@db-node01 ~]# vim /etc/hosts
...........
172.16.60.211 db-node01
172.16.60.212 db-node02
172.16.60.213 db-node03
172.16.60.214 db-route01
所有節點進行如下的相關優化配置
[root@db-node01 ~]# cat>>/etc/sysctl.conf <<EOF
fs.aio-max-nr = 1048576
fs.file-max = 681574400
kernel.shmmax = 137438953472
kernel.shmmni = 4096
kernel.sem = 250 32000 100 200
net.ipv4.ip_local_port_range = 9000 65000
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048586
EOF
[root@db-node01 ~]# sysctl -p
[root@db-node01 ~]# cat>>/etc/security/limits.conf <<EOF
mysql soft nproc 65536
mysql hard nproc 65536
mysql soft nofile 65536
mysql hard nofile 65536
EOF
[root@db-node01 ~]# cat>>/etc/pam.d/login <<EOF
session required /lib/security/pam_limits.so
session required pam_limits.so
EOF
[root@db-node01 ~]# cat>>/etc/profile<<EOF
if [ $USER = "mysql" ]; then
ulimit -u 16384 -n 65536
fi
EOF
[root@db-node01 ~]# source /etc/profile
mysql-shell下載地址:https://downloads.mysql.com/archives/get/p/43/file/mysql-shell-1.0.9-linux-glibc2.12-x86-64bit.tar.gz
mysql-route下載地址: https://downloads.mysql.com/archives/get/p/41/file/mysql-router-2.1.4-linux-glibc2.12-x86-64bit.tar.gz
更多關於MySQL學習的文章，請參閱：死磕數據庫系列之 MySQL ，本系列持續更新中。
在管理節點安裝mysql shell 和 mysql-route
[root@db-route01 ~]# cd /usr/local/src/
[root@db-route01 src]# ll
total 21648
-rw-rw-r-- 1 root root 15526654 Mar 8 16:08 mysql-router-2.1.4-linux-glibc2.12-x86-64bit.tar.gz
-rw-rw-r-- 1 root root 6635831 Mar 22 2017 mysql-shell-1.0.9-linux-glibc2.12-x86-64bit.tar.gz
[root@db-route01 src]# tar -zvxf mysql-shell-1.0.9-linux-glibc2.12-x86-64bit.tar.gz
[root@db-route01 src]# tar -zvxf mysql-router-2.1.4-linux-glibc2.12-x86-64bit.tar.gz
[root@db-route01 src]# mv mysql-router-2.1.4-linux-glibc2.12-x86-64bit mysql-route
[root@db-route01 src]# mv mysql-shell-1.0.9-linux-glibc2.12-x86-64bit mysql-shell
[root@db-route01 src]# mv mysql-route /usr/local/
[root@db-route01 src]# mv mysql-shell /usr/local/
[root@db-route01 src]# vim /etc/profile
..............
export PATH=$PATH:/usr/local/mysql-shell/bin/:/usr/local/mysql-route/bin/
[root@db-route01 src]# source /etc/profile
[root@db-route01 ~]# mysqlprovision --version
mysqlprovision version 2.0.0
[root@db-route01 ~]# mysqlsh --version
MySQL Shell Version 1.0.9
[root@db-route01 ~]# mysqlrouter --version
MySQL Router v2.1.4 on Linux (64-bit) (GPL community edition)
在三個cluster節點安裝和部署Mysql5.7及 mysql-shell
安裝mysql-shell (三個節點同樣操作)
[root@db-node01 ~]# cd /usr/local/src/
[root@db-node01 src]# ll mysql-shell-1.0.9-linux-glibc2.12-x86-64bit.tar.gz
-rw-r--r-- 1 root root 6635831 Mar 22 2017 mysql-shell-1.0.9-linux-glibc2.12-x86-64bit.tar.gz
[root@db-node01 src]# tar -zvxf mysql-shell-1.0.9-linux-glibc2.12-x86-64bit.tar.gz
[root@db-node01 src]# mv mysql-shell-1.0.9-linux-glibc2.12-x86-64bit mysql-shell
[root@db-node01 src]# mv mysql-shell /usr/local/
[root@db-node01 src]# echo "export PATH=$PATH:/usr/local/mysql-shell/bin/" >> /etc/profile
[root@db-node01 src]# source /etc/profile
[root@db-node01 ~]# mysqlprovision --version
mysqlprovision version 2.0.0
[root@db-node01 ~]# mysqlsh --version
MySQL Shell Version 1.0.9
安裝mysql5.7 (三個節點同樣操作)
安裝MySQL yum資源庫
[root@db-node01 ~]# yum localinstall https://dev.mysql.com/get/mysql57-community-release-el7-8.noarch.rpm
安裝MySQL 5.7
[root@db-node01 ~]# yum install -y mysql-community-server
啓動MySQL服務器和MySQL的自動啓動
[root@db-node01 ~]# systemctl start mysqld.service
[root@db-node01 ~]# systemctl enable mysqld.service
設置登錄密碼
由於MySQL從5.7開始不允許首次安裝後使用空密碼進行登錄！爲了加強安全性，系統會隨機生成一個密碼以供管理員首次登錄使用，這個密碼記錄在/var/log/mysqld.log文件中，使用下面的命令可以查看此密碼：
[root@db-node01 ~]# cat /var/log/mysqld.log|grep 'A temporary password'
2019-01-11T05:53:17.824073Z 1 [Note] A temporary password is generated for root@localhost: TaN.k:*Qw2xs
使用上面查看的密碼TaN.k:*Qw2xs 登錄mysql，並重置密碼爲123456
[root@db-node01 ~]# mysql -p #輸入默認的密碼：TaN.k:*Qw2xs
.............
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)
mysql> set global validate_password_length=1;
Query OK, 0 rows affected (0.00 sec)
mysql> set password=password("123456");
Query OK, 0 rows affected, 1 warning (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
查看mysql版本
[root@db-node01 ~]# mysql -p123456
........
mysql> select version();
+-----------+
| version() |
+-----------+
| 5.7.24 |
+-----------+
1 row in set (0.00 sec)
溫馨提示：mysql5.7通過上面默認安裝後，執行語句可能會報錯：
ERROR 1819 (HY000): Your password does not satisfy the current policy requirements
這個報錯與Mysql 密碼安全策略validate_password_policy的值有關，validate_password_policy可以取0、1、2三個值： 解決辦法：
set global validate_password_policy=0;
set global validate_password_length=1;
配置my.cnf
先配置db-node01節點的my.cnf
[root@db-node01 ~]# cp /etc/my.cnf /etc/my.cnf.bak
[root@db-node01 ~]# >/etc/my.cnf
[root@db-node01 ~]# vim /etc/my.cnf
[mysqld]
datadir = /var/lib/mysql
socket = /var/lib/mysql/mysql.sock
symbolic-links = 0
log-error = /var/log/mysqld.log
pid-file = /var/run/mysqld/mysqld.pid
#複製框架
server_id=1
gtid_mode=ON
enforce_gtid_consistency=ON
binlog_checksum=NONE
log_bin=binlog
log_slave_updates=ON
binlog_format=ROW
master_info_repository=TABLE
relay_log_info_repository=TABLE
#組複製設置
#server必須爲每個事務收集寫集合，並使用XXHASH64哈希算法將其編碼爲散列
transaction_write_set_extraction=XXHASH64
#告知插件加入或創建組命名，UUID
loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
#server啓動時不自啓組複製,爲了避免每次啓動自動引導具有相同名稱的第二個組,所以設置爲OFF。
loose-group_replication_start_on_boot=off
#告訴插件使用IP地址，端口24901用於接收組中其他成員轉入連接
loose-group_replication_local_address="172.16.60.211:24901"
#啓動組server，種子server，加入組應該連接這些的ip和端口；其他server要加入組得由組成員同意
loose-group_replication_group_seeds="172.16.60.211:24901,172.16.60.212:24901,172.16.60.213:24901"
loose-group_replication_bootstrap_group=off
# 使用MGR的單主模式
loose-group_replication_single_primary_mode = on
disabled_storage_engines = MyISAM,BLACKHOLE,FEDERATED,CSV,ARCHIVE
report_port=3306
如上配置完成後, 將db-node01節點的/etc/my.cnf文件拷貝到其他兩個節點
[root@db-node01 ~]# rsync -e "ssh -p22" -avpgolr /etc/my.cnf [email protected]:/etc/
[root@db-node01 ~]# rsync -e "ssh -p22" -avpgolr /etc/my.cnf [email protected]:/etc/
3個cluster節點除了server_id、loose-group_replication_local_address 兩個參數不一樣外，其他保持一致。
所以待拷貝完成後, 分別修改db-node02和db-node03節點/etc/my.cnf文件的server_id、loose-group_replication_local_address兩個參數
配置完成後, 要依次重啓三個節點的數據庫，安裝MGR插件，設置複製賬號（所有MGR節點都要執行）
[root@db-node01 ~]# systemctl restart mysqld
創建Innodb Cluster集羣
- 在 db-node01 上創建集羣，通過 db-node01 上的 shell 連接db-node01 的 mysql
[root@db-node01 ~]# mysqlsh
...................
# 執行配置命令，也需要密碼
# 然後需要輸入MySQL配置文件路徑，本示例中的路徑是 /usr/local/data/s1/s1.cnf
# 接下來需要創建供其他主機訪問的用戶，這裏選擇第1項，爲root用戶授權
mysql-js> shell.connect('root@localhost:3306');
Please provide the password for 'root@localhost:3306': #輸入密碼123456
Creating a Session to 'root@localhost:3306'
Classic Session successfully established. No default schema selected.
mysql-js> dba.configureLocalInstance();
Please provide the password for 'root@localhost:3306': #輸入密碼123456
Detecting the configuration file...
Found configuration file at standard location: /etc/my.cnf
Do you want to modify this file? [Y|n]: #直接回車, 使用默認的/etc/my.cnf配置文件
MySQL user 'root' cannot be verified to have access to other hosts in the network.
1) Create root@% with necessary grants
2) Create account with different name
3) Continue without creating account
4) Cancel
Please select an option [1]: 1 #選擇第1項，爲root用戶授權, 創建供其他主機訪問的用戶
Password for new account: #輸入供其他主機訪問的用戶root用戶授權的密碼. 這裏依然設置123456
Confirm password:
Validating instance...
Dba.configureLocalInstance: Your password does not satisfy the current policy requirements (MySQL Error 1819)
出現上面報錯的解決辦法:打開另一個終端窗口, 登錄db-node01節點的mysql,執行下面命令:
[root@db-node01 ~]# mysql -p123456
.................
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)
mysql> set global validate_password_length=1;
Query OK, 0 rows affected (0.00 sec)
然後接着上面db-node01終端窗口的mysql-shell繼續執行:
mysql-js> shell.connect('root@localhost:3306');
Please provide the password for 'root@localhost:3306': #輸入密碼123456
Creating a Session to 'root@localhost:3306'
Classic Session successfully established. No default schema selected.
mysql-js> dba.configureLocalInstance();
Please provide the password for 'root@localhost:3306': #輸入密碼123456
Detecting the configuration file...
Found configuration file at standard location: /etc/my.cnf
Do you want to modify this file? [Y|n]: #直接回車, 使用默認的/etc/my.cnf配置文件
MySQL user 'root' cannot be verified to have access to other hosts in the network.
1) Create root@% with necessary grants
2) Create account with different name
3) Continue without creating account
4) Cancel
Please select an option [1]: 1 # 選擇第1項，爲root用戶授權, 即供其他主機訪問的用戶root用戶授權的密碼, 否則其他機器使用root用戶連接不上不本機的mysql
Password for new account: #輸入. 這裏依然設置123456. 這裏授權之後, 登錄db-node01節點的mysql, 執行"select host, user from mysql.user" 命令就能發現
Confirm password:
Validating instance...
The instance 'localhost:3306' is valid for Cluster usage
You can now use it in an InnoDB Cluster.
{
"status": "ok"
}
mysql-js>
由上面的信息看出, status 爲 ok 說明配置沒問題了，可以用來創建cluster。
- 通過 db-route01 的 mysql-shell 連接 node01 創建 cluster
[root@db-route01 ~]# mysqlsh
................
# 連接db-node01
mysql-js> shell.connect('root@db-node01:3306');
Please provide the password for 'root@db-node01:3306': #輸入密碼123456
Creating a Session to 'root@db-node01:3306'
Classic Session successfully established. No default schema selected.
# 創建一個 cluster，命名爲 'myCluster'
mysql-js> var cluster = dba.createCluster('myCluster');
A new InnoDB cluster will be created on instance 'root@db-node01:3306'.
Creating InnoDB cluster 'myCluster' on 'root@db-node01:3306'...
Adding Seed Instance...
Cluster successfully created. Use Cluster.addInstance() to add MySQL instances.
At least 3 instances are needed for the cluster to be able to withstand up to
one server failure.
如上的信息, 如果創建成功, 則會輸出的信息中會有類似“Cluster successfully created.”的語句。
#創建成功後，查看cluster狀態
mysql-js> cluster.status();
{
"clusterName": "myCluster",
"defaultReplicaSet": {
"name": "default",
"primary": "db-node01:3306",
"status": "OK_NO_TOLERANCE",
"statusText": "Cluster is NOT tolerant to any failures.",
"topology": {
"db-node01:3306": {
"address": "db-node01:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
}
}
}
mysql-js> dba.getCluster();
<Cluster:myCluster>
注意上面這個db-route01的mysql-shell終端窗口就不要關閉了, 一直保持連接中,也就是一直在當前集羣狀態中, 後面添加其他節點到cluster集羣中會用到! (後面說到常用命令時會解釋)。更多關於MySQL學習的文章，請參閱：死磕數據庫系列之 MySQL ，本系列持續更新中。
- 添加節點 db-node02到上面創建的"myCluster"集羣中
通過db-node02本機 mysql-shell 對 mysql 進行配置
[root@db-node02 ~]# mysqlsh
................
MySQL JS > shell.connect('root@localhost:3306');
Creating a session to 'root@localhost:3306'
Please provide the password for 'root@localhost:3306': ****** #輸入密碼123456
Save password for 'root@localhost:3306'? [Y]es/[N]o/Ne[v]er (default No): Y
Fetching schema names for autocompletion... Press ^C to stop.
Your MySQL connection id is 251
Server version: 5.7.25-log MySQL Community Server (GPL)
No default schema selected; type \use <schema> to set one.
<ClassicSession:root@localhost:3306>
MySQL localhost:3306 ssl JS > dba.configureLocalInstance();
Configuring local MySQL instance listening at port 3306 for use in an InnoDB cluster...
This instance reports its own address as 172.16.60.212
WARNING: User 'root' can only connect from localhost.
If you need to manage this instance while connected from other hosts, new account(s) with the proper source address specification must be created.
1) Create remotely usable account for 'root' with same grants and password
2) Create a new admin account for InnoDB cluster with minimal required grants
3) Ignore and continue
4) Cancel
Please select an option [1]: 1
Please provide a source address filter for the account (e.g: 192.168.% or % etc) or leave empty and press Enter to cancel.
Account Host: % #選擇%, 表示允許任何機器遠程使用root用戶連接本機的mysql
The instance 'localhost:3306' is valid for InnoDB cluster usage.
Your password does not satisfy the current policy requirements (MySQL Error 1819)
解決辦法:登錄db-node02節點的mysql
[root@db-node02 ~]# mysql -p123456
...............
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)
mysql> set global validate_password_length=1;
Query OK, 0 rows affected (0.00 sec)
接着繼續登錄db-node02本機的mysql-shell 進行配置
[root@db-node02 ~]# mysqlsh
.............
MySQL JS > shell.connect('root@localhost:3306');
Creating a session to 'root@localhost:3306'
Fetching schema names for autocompletion... Press ^C to stop.
Your MySQL connection id is 180
Server version: 5.7.25-log MySQL Community Server (GPL)
No default schema selected; type \use <schema> to set one.
<ClassicSession:root@localhost:3306>
MySQL localhost:3306 ssl JS > dba.configureLocalInstance();
Configuring local MySQL instance listening at port 3306 for use in an InnoDB cluster...
This instance reports its own address as db-node02
Clients and other cluster members will communicate with it through this address by default. If this is not correct, the report_host MySQL system variable should be changed.
WARNING: User 'root' can only connect from localhost.
If you need to manage this instance while connected from other hosts, new account(s) with the proper source address specification must be created.
1) Create remotely usable account for 'root' with same grants and password
2) Create a new admin account for InnoDB cluster with minimal required grants
3) Ignore and continue
4) Cancel
Please select an option [1]: 1
Please provide a source address filter for the account (e.g: 192.168.% or % etc) or leave empty and press Enter to cancel.
Account Host: %
The instance 'localhost:3306' is valid for InnoDB cluster usage.
Cluster admin user 'root'@'%' created.
MySQL localhost:3306 ssl JS >
然後登錄db-node02節點的mysql, 發現上面使用root用戶遠程連接的授權已經有了
[root@db-node02 ~]# mysql -p123456
..............
mysql> select host,user from mysql.user;
+-----------+---------------+
| host | user |
+-----------+---------------+
| % | root |
| localhost | mysql.session |
| localhost | mysql.sys |
| localhost | root |
+-----------+---------------+
4 rows in set (0.00 sec)
接着修改 my.cnf，添加配置項:
[root@db-node02 ~]# vim /etc/my.cnf
............
loose-group_replication_allow_local_disjoint_gtids_join=on
重啓mysql服務
[root@db-node02 ~]# systemctl restart mysqld
然後通過 db-route01節點 的 mysql-shell 添加 node02 到 "myCluster"集羣中 接着上面的db-route01的mysql-shell終端窗口 (注意這個終端窗口是上面執行後, 沒有關閉一直開着的)
mysql-js> cluster.addInstance('root@db-node02:3306');
A new instance will be added to the InnoDB cluster. Depending on the amount of
data on the cluster this might take from a few seconds to several hours.
Please provide the password for 'root@db-node02:3306':
Adding instance to the cluster ...
The instance 'root@db-node02:3306' was successfully added to the cluster.
上面信息表示db-node02節點已經成功添加到"myCluster"集羣中了. 如下查看集羣狀態
mysql-js> cluster.status();
{
"clusterName": "myCluster",
"defaultReplicaSet": {
"name": "default",
"primary": "db-node01:3306",
"status": "OK_NO_TOLERANCE",
"statusText": "Cluster is NOT tolerant to any failures.",
"topology": {
"db-node01:3306": {
"address": "db-node01:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"db-node02:3306": {
"address": "db-node02:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
}
}
}
同樣, 上面操作後, 這個db-route01節點的mysql-shell當前終端窗口不要關閉,繼續保持在集羣狀態中, 下面添加db-node03節點到集羣中會用到這裏.(後面常用命令中會提到)
- 添加節點 db-node03到上面創建的"myCluster"集羣中
首先登錄db-node03節點的mysql
[root@db-node03 ~]# mysql -p123456
............
mysql> set global validate_password_length=1;
Query OK, 0 rows affected (0.00 sec)
mysql> set global validate_password_length=1;
Query OK, 0 rows affected (0.00 sec)
登錄db-node03節點的mysql-shell, 進行配置
[root@db-node03 ~]# mysqlsh
.............
MySQL JS > shell.connect('root@localhost:3306');
Creating a session to 'root@localhost:3306'
Please provide the password for 'root@localhost:3306': ******
Save password for 'root@localhost:3306'? [Y]es/[N]o/Ne[v]er (default No): Y
Fetching schema names for autocompletion... Press ^C to stop.
Your MySQL connection id is 393
Server version: 5.7.25-log MySQL Community Server (GPL)
No default schema selected; type \use <schema> to set one.
<ClassicSession:root@localhost:3306>
MySQL localhost:3306 ssl JS > dba.configureLocalInstance();
Configuring local MySQL instance listening at port 3306 for use in an InnoDB cluster...
This instance reports its own address as db-node03
Clients and other cluster members will communicate with it through this address by default. If this is not correct, the report_host MySQL system variable should be changed.
WARNING: User 'root' can only connect from localhost.
If you need to manage this instance while connected from other hosts, new account(s) with the proper source address specification must be created.
1) Create remotely usable account for 'root' with same grants and password
2) Create a new admin account for InnoDB cluster with minimal required grants
3) Ignore and continue
4) Cancel
Please select an option [1]: 1
Please provide a source address filter for the account (e.g: 192.168.% or % etc) or leave empty and press Enter to cancel.
Account Host: %
The instance 'localhost:3306' is valid for InnoDB cluster usage.
Cluster admin user 'root'@'%' created.
接着修改 my.cnf，添加配置項:
[root@db-node03 ~]# vim /etc/my.cnf
............
loose-group_replication_allow_local_disjoint_gtids_join=on
重啓mysql服務
[root@db-node03 ~]# systemctl restart mysqld
然後通過 db-route01節點 的 mysql-shell 添加 node03 到 "myCluster"集羣中 接着上面的db-route01的mysql-shell終端窗口 (注意這個終端窗口是上面執行後, 沒有關閉一直開着的)
mysql-js> cluster.addInstance('root@db-node03:3306');
A new instance will be added to the InnoDB cluster. Depending on the amount of
data on the cluster this might take from a few seconds to several hours.
Please provide the password for 'root@db-node03:3306':
Adding instance to the cluster ...
The instance 'root@db-node03:3306' was successfully added to the cluster.
上面信息表示db-node02節點已經成功添加到"myCluster"集羣中了. 如下查看集羣狀態
mysql-js> cluster.status();
{
"clusterName": "myCluster",
"defaultReplicaSet": {
"name": "default",
"primary": "db-node01:3306",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
"db-node01:3306": {
"address": "db-node01:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"db-node02:3306": {
"address": "db-node02:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"db-node03:3306": {
"address": "db-node03:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
}
}
}
通過上面cluster集羣信息可知, db-node01節點是主節點, 具有R/W讀寫權限, 其他兩個節點是從節點, 具有R/O 只讀權限
啓動管理節點的route
進入 db-route01管理節點中mysql-router 安裝目錄，配置並啓動 router
[root@db-route01 ~]# /usr/local/mysql-route/bin/mysqlrouter --bootstrap root@db-node01:3306 -d myrouter --user=root
Please enter MySQL password for root:
Bootstrapping MySQL Router instance at /root/myrouter...
MySQL Router has now been configured for the InnoDB cluster 'myCluster'.
The following connection information can be used to connect to the cluster.
Classic MySQL protocol connections to cluster 'myCluster':
- Read/Write Connections: localhost:6446 #讀寫端口
- Read/Only Connections: localhost:6447 #只讀端口
X protocol connections to cluster 'myCluster':
- Read/Write Connections: localhost:64460
- Read/Only Connections: localhost:64470
這裏會在當前目錄下產生mysql-router 目錄, 並生成router配置文件，接着把配置文件修改一下：
[root@db-route01 ~]# ls /root/myrouter/
data log mysqlrouter.conf mysqlrouter.key run start.sh stop.sh
[root@db-route01 ~]# cat /root/myrouter/mysqlrouter.conf #可以修改配置文件, 也可以默認不修改
默認通過route連接mysql後, 6446端口連接後可以進行讀寫操作. 6447端口連接後只能進行只讀操作.
然後啓動mysqlroute
[root@db-route01 ~]# /root/myrouter/start.sh
PID 16484 written to /root/myrouter/mysqlrouter.pid
[root@db-route01 ~]# ps -ef|grep myroute
root 18473 1 0 22:26 pts/1 00:00:00 sudo ROUTER_PID=/root/myrouter/mysqlrouter.pid /usr/local/mysql-route/bin/mysqlrouter -c /root/myrouter/mysqlrouter.conf --user=root
root 18486 18473 0 22:26 pts/1 00:00:00 /usr/local/mysql-route/bin/mysqlrouter -c /root/myrouter/mysqlrouter.conf --user=root
root 18612 5091 0 22:26 pts/1 00:00:00 grep --color=auto myroute
[root@db-route01 ~]# netstat -tunlp|grep 18486
tcp 0 0 0.0.0.0:64460 0.0.0.0:* LISTEN 18486/mysqlrouter
tcp 0 0 0.0.0.0:6446 0.0.0.0:* LISTEN 18486/mysqlrouter
tcp 0 0 0.0.0.0:6447 0.0.0.0:* LISTEN 18486/mysqlrouter
tcp 0 0 0.0.0.0:64470 0.0.0.0:* LISTEN 18486/mysqlrouter
這樣就可以使用MySQL客戶端連接router了. 下面驗證下連接router:
a) 管理節點本機mysql-shell連接:
[root@db-route01 ~]# mysqlsh --uri root@localhost:6446
b) 管理節點本機mysql連接:
[root@db-route01 ~]# mysql -u root -h 127.0.0.1 -P 6446 -p
c) 遠程客戶機通過route連接mysql
[root@db-node01 ~]# mysql -u root -h 172.16.60.214 -P 6446 -p
測試cluster節點數據同步. 這裏選擇db-node03節點作爲遠程客戶端連接router
[root@db-node03 ~]# mysql -u root -h 172.16.60.214 -P 6446 -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1054
Server version: 5.7.25-log MySQL Community Server (GPL)
Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+-------------------------------+
| Database |
+-------------------------------+
| information_schema |
| mysql |
| mysql_innodb_cluster_metadata |
| performance_schema |
| sys |
+-------------------------------+
5 rows in set (0.00 sec)
測試測試庫kevin
mysql> CREATE DATABASE kevin CHARACTER SET utf8 COLLATE utf8_general_ci;
ERROR 1044 (42000): Access denied for user 'root'@'%' to database 'kevin'
這是因爲'root@%'沒有創建庫的權限
mysql> select host,user from mysql.user;
+-----------+----------------------------------+
| host | user |
+-----------+----------------------------------+
| % | mysql_innodb_cluster_rp496261783 |
| % | mysql_innodb_cluster_rp496457975 |
| % | mysql_innodb_cluster_rp496569258 |
| % | mysql_innodb_cluster_rp496629685 |
| % | mysql_router1_olzau3ltjqzx |
| % | root |
| localhost | mysql.session |
| localhost | mysql.sys |
| localhost | root |
+-----------+----------------------------------+
9 rows in set (0.00 sec)
mysql> show grants for root@'%';
+-----------------------------------------------------------------------------------------------------------------------------------------+
| Grants for root@% |
+-----------------------------------------------------------------------------------------------------------------------------------------+
| GRANT RELOAD, SHUTDOWN, PROCESS, FILE, SUPER, REPLICATION SLAVE, REPLICATION CLIENT, CREATE USER ON *.* TO 'root'@'%' WITH GRANT OPTION |
| GRANT SELECT, INSERT, UPDATE, DELETE ON `mysql`.* TO 'root'@'%' WITH GRANT OPTION |
| GRANT SELECT ON `performance_schema`.* TO 'root'@'%' WITH GRANT OPTION |
| GRANT ALL PRIVILEGES ON `mysql_innodb_cluster_metadata`.* TO 'root'@'%' WITH GRANT OPTION |
+-----------------------------------------------------------------------------------------------------------------------------------------+
4 rows in set (0.00 sec)
登錄主庫, 創建一個具有管理權權限的用戶
[root@db-node01 ~]# mysql -p123456
.............
mysql> set global validate_password_policy=0;
Query OK, 0 rows affected (0.00 sec)
mysql> set global validate_password_length=1;
Query OK, 0 rows affected (0.00 sec)
mysql> grant all on *.* to bobo@'%' identified by "bo@123" with grant option;
Query OK, 0 rows affected, 1 warning (0.05 sec)
接着遠程使用上面創建的新賬號登錄router操作
[root@db-node03 ~]# mysql -u bobo -h 172.16.60.214 -P 6446 -p
........
mysql> show grants for bobo@'%';
+-------------------------------------------------------------+
| Grants for bobo@% |
+-------------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'bobo'@'%' WITH GRANT OPTION |
+-------------------------------------------------------------+
1 row in set (0.00 sec)
測試測試庫kevin
mysql> CREATE DATABASE kevin CHARACTER SET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.06 sec)
mysql> use kevin;
Database changed
mysql> create table if not exists haha (id int(10) PRIMARY KEY AUTO_INCREMENT,name varchar(50) NOT NULL);
Query OK, 0 rows affected (0.22 sec)
mysql> insert into kevin.haha values(1,"wangshibo"),(2,"guohuihui"),(3,"yangyang"),(4,"shikui");
Query OK, 4 rows affected (0.13 sec)
Records: 4 Duplicates: 0 Warnings: 0
mysql> select * from kevin.haha;
+----+-----------+
| id | name |
+----+-----------+
| 1 | wangshibo |
| 2 | guohuihui |
| 3 | yangyang |
| 4 | shikui |
+----+-----------+
4 rows in set (0.00 sec)
分別登錄三個cluster節點的mysql, 發現測試庫kevin已經完成同步了, 其中:寫操作的數據會先寫到db-node01節點, 然後同步到db-node02和db-node03只讀節點上.
注意: 上面使用6446端口連接的route, 可以進行讀寫操作. 但是使用6447端口連接後, 就只能進行只讀操作了. 登錄後可以執行" select @@hostname" 查看登錄到哪個節點上.
[root@db-node03 ~]# mysql -u bobo -h 172.16.60.214 -P 6447 -p
.............
mysql> select * from kevin.haha;
+----+-----------+
| id | name |
+----+-----------+
| 1 | wangshibo |
| 2 | guohuihui |
| 3 | yangyang |
| 4 | shikui |
+----+-----------+
4 rows in set (0.00 sec)
mysql> delete from kevin.haha where id>2;
ERROR 1290 (HY000): The MySQL server is running with the --super-read-only option so it cannot execute this statement
此外, 還可以利用keepalived實現InnoDB Cluster的高可用, 即兩臺db-route管理節點, 通過VIP資源實現故障無感知切換. 這樣需要準備5臺節點, 其中3個cluster節點(安裝mysql, mysql-shell), 2個route管理節點(安裝keepalived, mysql-shell, mysql-route, mysql-client)
InnoDB Cluster 集羣日常維護命令
在各節點配置之後, 創建cluster集羣之前, 可以依次檢查下cluster各個節點是否可用
[root@db-node01 ~]# mysqls
.................
mysql-js> dba.checkInstanceConfiguration("root@localhost:3306")
Please provide the password for 'root@localhost:3306':
Validating instance...
The instance 'localhost:3306' is valid for Cluster usage
{
"status": "ok"
}
mysql-js> dba.checkInstanceConfiguration("root@db-node02:3306")
Please provide the password for 'root@db-node02:3306':
Validating instance...
The instance 'db-node02:3306' is valid for Cluster usage
{
"status": "ok"
}
mysql-js> dba.checkInstanceConfiguration("root@db-node03:3306")
Please provide the password for 'root@db-node03:3306':
Validating instance...
The instance 'db-node03:3306' is valid for Cluster usage
{
"status": "ok"
}
比如在上面創建Innodb cluster集羣過程中, 再次登錄mysqlsh (從客戶端遠程登錄, 或任意節點本地登錄), 怎麼獲得並查看集羣狀態
[root@db-node01 ~]# mysqlsh
.................
mysql-js> shell.connect("root@db-node01:3306");
Please provide the password for 'root@db-node01:3306':
Creating a Session to 'root@db-node01:3306'
Classic Session successfully established. No default schema selected.
查看集羣狀態
mysql-js> cluster.status();
ReferenceError: cluster is not defined
上面方式查看, 會報錯說集羣沒有定義, 這時需要先執行下面這條語句之後,纔看查看到集羣狀態!!!!!
mysql-js> cluster.status();
ReferenceError: cluster is not defined
然後就可以查看集羣狀態了
mysql-js> cluster=dba.getCluster();
<Cluster:myCluster>
mysql-js> cluster.status();
{
"clusterName": "myCluster",
"defaultReplicaSet": {
"name": "default",
"primary": "db-node01:3306",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
"db-node01:3306": {
"address": "db-node01:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"db-node02:3306": {
"address": "db-node02:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"db-node03:3306": {
"address": "db-node03:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
}
}
}
從遠程客戶端登錄
[root@db-node03 ~]# mysqlsh --uri [email protected]:6446
......................
MySQL 172.16.60.214:6446 ssl JS > cluster=dba.getCluster();
<Cluster:myCluster>
MySQL 172.16.60.214:6446 ssl JS > cluster.status();
{
"clusterName": "myCluster",
"defaultReplicaSet": {
"name": "default",
"primary": "db-node01:3306",
"ssl": "REQUIRED",
"status": "OK",
"statusText": "Cluster is ONLINE and can tolerate up to ONE failure.",
"topology": {
"db-node01:3306": {
"address": "db-node01:3306",
"mode": "R/W",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"db-node02:3306": {
"address": "db-node02:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
},
"db-node03:3306": {
"address": "db-node03:3306",
"mode": "R/O",
"readReplicas": {},
"role": "HA",
"status": "ONLINE"
}
}
},
"groupInformationSourceMember": "mysql://[email protected]:6446"
}
查看已創建的集羣名稱
MySQL 172.16.60.214:6446 ssl JS > dba.getCluster();
<Cluster:myCluster>
總結
dba.getCluster(); #查看創建的集羣
cluster=dba.getCluster(); #獲取當前集羣
cluster.status(); #查看集羣狀態
InnoDB Cluster集羣維護的命令幫助
mysql-js> dba.help();
The global variable 'dba' is used to access the MySQL AdminAPI functionality
and perform DBA operations. It is used for managing MySQL InnoDB clusters.
The following properties are currently supported.
- verbose Enables verbose mode on the Dba operations.
The following functions are currently supported.
- checkInstanceConfiguration Validates an instance for usage in Group
Replication.
- configureLocalInstance Validates and configures an instance for
cluster usage.
- createCluster Creates a MySQL InnoDB cluster.
- deleteSandboxInstance Deletes an existing MySQL Server instance on
localhost.
- deploySandboxInstance Creates a new MySQL Server instance on
localhost.
- dropMetadataSchema Drops the Metadata Schema.
- getCluster Retrieves a cluster from the Metadata Store.
- help Provides help about this class and it's
members
- killSandboxInstance Kills a running MySQL Server instance on
localhost.
- rebootClusterFromCompleteOutage Brings a cluster back ONLINE when all
members are OFFLINE.
- resetSession Sets the session object to be used on the
Dba operations.
- startSandboxInstance Starts an existing MySQL Server instance on
localhost.
- stopSandboxInstance Stops a running MySQL Server instance on
localhost.
For more help on a specific function use: dba.help('<functionName>')
e.g. dba.help('deploySandboxInstance')
比如獲取當前集羣名稱
mysql-js> dba.getCluster();
<Cluster:myCluster>
日常使用的幾個重要命令 (mysqlsh的JS語法)
dba.checkInstanceConfiguration("root@hostname:3306") #檢查節點配置實例，用於加入cluster之前
dba.rebootClusterFromCompleteOutage('myCluster'); #重啓
dba.dropMetadataSchema(); #刪除schema
var cluster = dba.getCluster('myCluster') #獲取當前集羣
cluster.checkInstanceState("root@hostname:3306") #檢查cluster裏節點狀態
cluster.rejoinInstance("root@hostname:3306") #重新加入節點，我本地測試的時候發現rejoin一直無效，每次是delete後
addcluster.dissolve({force：true}) #刪除集羣
cluster.addInstance("root@hostname:3306") #增加節點
cluster.removeInstance("root@hostname:3306") #刪除節點
cluster.removeInstance('root@host:3306',{force:true}) #強制刪除節點
cluster.dissolve({force:true}) #解散集羣
cluster.describe(); #集羣描述
集羣節點狀態
- ONLINE: The instance is online and participating in the cluster.
- OFFLINE: The instance has lost connection to the other instances.
- RECOVERING: The instance is attempting to synchronize with the cluster by retrieving transactions it needs before it can become an ONLINE member.
- UNREACHABLE: The instance has lost communication with the cluster.
- ERROR: The instance has encountered an error during the recovery phase or while applying a transaction
InnoDB Cluster 集羣部署中的注意事項
請保證所有的集羣機器在一個子網內，網絡必須要通, 不然會失敗；考慮到可以用橋接的方式實現不同網絡之間集羣的搭建, 這個並沒有親測；
統一使用hostname進行配置；請更改每臺機器的hosts文件；
報錯1
ERROR:
Group Replication join failed.
ERROR: Error joining instance to cluster: 'host-192-168-1-101:3306' - Query failed. MySQL Error (3092): The server is not configureroperly to be an active member of the group. Please see more details on error log.. Query: START group_replication
通過如下方式進行處理
mysql> install plugin group_replication soname 'group_replication.so'; ##安裝插件
mysql> set global group_replication_allow_local_disjoint_gtids_join=ON;
mysql> START GROUP_REPLICATION;
mysql> select * from performance_schema.replication_group_members;
報錯2
Dba.getCluster: This function is not available through a session to a standalone instance (RuntimeError)
說明集羣中的主節點已經不在該機器上，查詢後更改機器重試一下即可；
報錯3
Dba.getCluster: Dba.getCluster: Unable to get cluster. The instance 'host-192-168-1-101:3306'
may belong to a different ReplicaSet as the one registered in the Metadata since the value of 'group_replication_group_name'
does not match the one registered in the ReplicaSet's Metadata: possible split-brain scenario. Please connect to another member of the ReplicaSet to get the Cluster. (RuntimeError)
最致命的錯誤，master/slave的數據不一致所致，沒辦法，只能重新來
mysql-js>dba.dropMetadataSchema();
請保證集羣中的數據庫表都存在主鍵，不然會掛掉；
安裝集羣監控，保證集羣中機器掛掉的時候及時啓動，不然所有節點宕機的時候就是災難到來之時!!! 到時哭都來不及；
如何重置Innodb cluster集羣環境？
主節點：
mysql-js>dba.dropMetadataSchema(); 登錄mysql-shell清空集羣
mysql> stop group_replication;
mysql> reset master; （清空日誌，確保和從庫的表沒有衝突奧，）
mysql> reset slave;
其他節點(主要清理和主庫的主從信息， 確保主庫和從庫的表沒有衝突奧)
mysql> stop group_replication;
mysql> reset master;
mysql> reset slave
主機名和 /etc/hosts中名字不一致，出現報錯:
[Repl] Slave I/O for channel 'group_replication_recovery': error connecting to master 'mysql_innodb_cluster_r0430970923@mysql3:3306' - retry-time: 60 retries: 1, Error_code: MY-002005
[ERROR] [MY-011582] [Repl] Plugin group_replication reported: 'There was an error when connecting to the donor server. Please check that group_replication_recovery channel credentials and all MEMBER_HOST column values of performance_schema.replication_group_members table are correct and DNS resolvable.'
[ERROR] [MY-011583] [Repl] Plugin group_replication reported: 'For details please check performance_schema.replication_connection_status table and error log messages of Slave I/O for channel group_replication_recovery.'
主庫的日誌應用卡在某個位置無法應用到從庫，出現報錯:
[ERROR] [MY-010586] [Repl] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'binlog.000007' position 151
[ERROR] [MY-010584] [Repl] Slave SQL for channel 'group_replication_applier': Error executing row event: 'Unknown database 'mysql_innodb_cluster_metadata'', Error_code: MY-001049
重建master：
mysql> stop group_replication;
mysql> reset master;
報錯 4
[ERROR] Slave SQL for channel 'group_replication_recovery': Could not execute Write_rows event on table mysql_innodb_cluster_metadata.instances;
Cannot add or update a child row: a foreign key constraint fails (mysql_innodb_cluster_metadata.instances, CONSTRAINT instances_ibfk_1 FOREIGN KEY (host_id) REFERENCES hosts (host_id)),
Error_code: 1452; handler error HA_ERR_NO_REFERENCED_ROW; the event's master log binlog.000001, end_log_pos 3059, Error_code: 1452
解決方式：清空表mysql_innodb_cluster_metadata.hosts; 重新建立集羣
報錯5
This member has more executed transactions than those present in the group
解決方式：
mysql> stop group_replication;
mysql> reset master;
用戶操作系統資源的限制
[Warning] Buffered warning: Changed limits: max_open_files: 1024 (requested 5000)
[Warning] Buffered warning: Changed limits: table_open_cache: 431 (requested 2000)
解決方式：
# vim /etc/security/limits.conf # 添加下面內容
mysql soft nproc 2047
mysql hard nproc 16384
mysql soft nofile 1024
mysql hard nofile 65535
報錯 6
dba.rebootClusterFromCompleteOutage: The active session instance isn't the most updated in comparison with the ONLINE instances of the
Cluster's metadata.
在集羣沒有起來時某些機器的數據表發生變動，導致數據不一致；
解決方式：所有MySQL機器通過reset master命令清空binlogs
mysql> reset master;
mysql> show master logs;
然後再運行Dba.rebootClusterFromCompleteOutage重啓集羣。
service mysql restart 無法重啓mysql，mysql stuck，並一直輸出日誌'[Note] Plugin group_replication reported: '[GCS] cli_err 2''
解決方式：唯一停止MySQL的命令爲：
#pkill -9 mysqld
如何將Multi-Primary改爲Single-Primary？
- a) 解散原來的集羣：mysql-js> cluster.dissolve({force: true})
- b) 每臺主機MySQL修改如下配置：
mysql> set global group_replication_enforce_update_everywhere_checks=OFF;
mysql> set global group_replication_single_primary_mode=ON;
- c) 重新創建集羣：
mysql-js> var cluster = dba.createCluster('mysqlCluster');
mysql-js> cluster.addInstance('chianyu@svr2:3306');
mysql-js> cluster.addInstance('chianyu@svr3:3306');
組複製的限制
- 事物鎖缺失問題：
- 組複製建議，事物隔離級別，read commit
- 序列化隔離級別：多主模式不支持
- 併發DDL和DML: 多主模式下，不支持 一邊對一個表進行DDL,另一邊進行更新，這樣對於DDL在其他實例上操作有未檢出的風險
- 外鍵級聯約束：多主模式下，多級外鍵依賴對引起多級操作， 因此可能導致未知衝突，建議打開 group_replication_enforce_update_everywhere_checks=ON
- 大事物，超過5秒未提交，會導致組通信失敗，
- 多主模式下：select * for update 會導致 死鎖。因爲這個鎖並非全組共享。
- 部分複製不支持：組複製下，設置部分複製，會過濾事物，導致組事物不一致。
- Mysql 8.0.11 group_replication_enforce
轉自
Centos 7.5基於MySQL 5.7的 InnoDB Cluster 多節點高可用集羣環境部署記錄 - 散盡浮華 - 博客園
https://www.cnblogs.com/kevingrace/p/10466530.html