Centos7搭建MySQL集羣

軟件包下載地址:http://galeracluster.com/downloads/

安裝前需卸載原有mysql,刪除相關文件夾;卸載Postfix,這個可能跟MySQL配置有衝突: yum remove postfix –y
先檢測一下有沒有:yum list installed | grep mysql 卸載: yum -y remove mysql-libs.x86_64

卸載系統自帶的mariadb

[root@node0 ~]# rpm -qa| grep mariadb
mariadb-libs-5.5.41-2.el7_0.x86_64
[root@node0 ~]# rpm -e --nodeps mariadb-libs-5.5.41-2.el7_0.x86_64

建議先使用阿里雲 yum 源安裝基礎依賴

yum -y install gcc gcc-c++ openssl openssl-devel lsof socat perl boost-devel rsync jemalloc libaio libaio-devel

1、所需安裝包如下:

[root@node0 ~]# ll
-rw-r--r--  1 root root  12867764 12月 21 10:41 galera-3-25.3.25-2.el7.x86_64.rpm
-rw-r--r--  1 root root     44852 12月 21 10:41 mysql-wsrep-5.7-5.7.24-25.16.el7.x86_64.rpm
-rw-r--r--  1 root root  25405632 12月 21 10:41 mysql-wsrep-client-5.7-5.7.24-25.16.el7.x86_64.rpm
-rw-r--r--  1 root root    283836 12月 21 10:41 mysql-wsrep-common-5.7-5.7.24-25.16.el7.x86_64.rpm
-rw-r--r--  1 root root   3908032 12月 21 10:41 mysql-wsrep-devel-5.7-5.7.24-25.16.el7.x86_64.rpm
-rw-r--r--  1 root root   2278988 12月 21 10:41 mysql-wsrep-libs-5.7-5.7.24-25.16.el7.x86_64.rpm
-rw-r--r--  1 root root   2148744 12月 21 10:41 mysql-wsrep-libs-compat-5.7-5.7.24-25.16.el7.x86_64.rpm
-rw-r--r--  1 root root 143030144 12月 21 10:41 mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64.rpm
-rw-r--r--  1 root root  26199024 12月 21 10:41 mysql-wsrep-test-5.7-5.7.24-25.16.el7.x86_64.rpm

開始安裝(安裝過程中,若出現提示需要安裝的依賴,就先把所需依賴安裝了,然後繼續安裝軟件)

[root@node0 ~]# rpm -ivh mysql-wsrep-common-5.7-5.7.24-25.16.el7.x86_64.rpm 

[root@node0 ~]# rpm -ivh mysql-wsrep-libs-5.7-5.7.24-25.16.el7.x86_64.rpm 

[root@node0 ~]# rpm -ivh mysql-wsrep-client-5.7-5.7.24-25.16.el7.x86_64.rpm 

安裝到mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64.rpm時發需要依賴,用yum -y install lsof && yum -y install socat && yum -y install perl && yum -y install net-tools安裝下依賴,然後繼續安裝軟件包

[root@node0 ~]# rpm -ivh mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64.rpm 
警告:mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64.rpm: 頭V4 RSA/SHA512 Signature, 密鑰 ID bc19ddba: NOKEY
錯誤:依賴檢測失敗:
	lsof 被 mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64 需要
	net-tools 被 mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64 需要
	socat 被 mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64 需要

[root@node0 ~]# yum -y install lsof && yum -y install socat && yum -y install perl && yum -y install net-tools

 

依賴被安裝以後,安裝mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64.rpm成功了;繼續安裝···

[root@node0 ~]# rpm -ivh mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64.rpm 
警告:mysql-wsrep-server-5.7-5.7.24-25.16.el7.x86_64.rpm: 頭V4 RSA/SHA512 Signature, 密鑰 ID bc19ddba: NOKEY
準備中...                          ################################# [100%]
正在升級/安裝...
   1:mysql-wsrep-server-5.7-5.7.24-25.################################# [100%]
[root@node0 ~]# rpm -ivh mysql-wsrep-libs-compat-5.7-5.7.24-25.16.el7.x86_64.rpm
[root@node0 ~]# rpm -ivh galera-3-25.3.25-2.el7.x86_64.rpm 

好,安裝完畢!接下來我們就開始編寫配置文件了:vi /etc/my.cnf 將裏面的內容全部註釋掉,然後輸入: !includedir /etc/my.cnf.d/ 。接下來創建wsrep.cnf文件: vi /etc/my.cnf.d/wsrep.cnf 然後輸入根據自己的需求輸入配置信息:

# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html
# *** DO NOT EDIT THIS FILE. It's a template which will be copied to the
# *** default location during install, and will be replaced if you
# *** upgrade to a newer version of MySQL.
[mysqld]
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
# These are commonly set, remove the # and set as required.
# basedir = .....
# datadir = .....
# port = .....
# server_id = .....
# socket = .....
log_timestamps=SYSTEM    #這個是我們自己加的,防止日誌時間和系統時間不一樣
port=3306
server_id=11         #MySQL服務器的ID,必須是唯一的,集羣各個節點也不同
explicit_defaults_for_timestamp=true
basedir=/usr/share/mysql/   
datadir=/usr/local/mysql/data                     
socket=/usr/local/mysql/data/mysql.sock
pid_file=/run/mysqld/mysqld.pid
log_error=/var/log/mysql.error
wsrep_cluster_name='cs_cluster'     #galera集羣的名字,必須是統一的
wsrep-provider=/usr/lib64/galera-3/libgalera_smm.so   #wsrep提供者,必須配置(.so文件的路徑在哪,就配置成哪,一般安裝好後都是在這個目錄下)
wsrep_node_name = node1   #wsrep節點的ID,必須是唯一的,集羣各個節點也不同
wsrep_cluster_address=gcomm:// #192.168.190.132,192.168.190.133,192.168.190.134(第一次啓動前把地址都註釋掉,之後再啓動節點時再次添加)   #集羣中的其他節點地址,可以使用主機名或IP
wsrep_node_address='192.168.190.132'   #本機節點地址,可以使用主機名或IP
wsrep_provider_options ="gmcast.listen_addr=tcp://192.168.190.132:4567"   #指定wsrep啓動端口號,4567爲默認值
wsrep_sst_donor='node1,node2,node3'   #一個逗號分割的節點串作爲狀態轉移源,比如wsrep_sst_donor=node1,node2,node3 如果node1可用,用node2,如果node2不可用,用node3,最後的逗號表明讓提供商自己選擇一個最優的。
wsrep_sst_method=rsync      #集羣同步方式,我的系統沒有可以用yum安裝一下這個遠程連接 yum -y install  rsync
wsrep_sst_auth=test:123456       #集羣同步的用戶名密碼
slow_query_log=on
[client]
default-character-set=utf8
socket=/usr/local/mysql/data/mysql.sock
[mysql]
default-character-set=utf8
socket=/usr/local/mysql/data/mysql.sock
[mysqldump]
max_allowed_packet = 512M
[mysqld_safe]
malloc-lib=/usr/lib64/libjemalloc.so.1  #這個我的系統裏也沒有可以用yum安裝一下  yum -y install jemalloc  如果獲取不到的話,下載一個數據源 rpm -ivh 

編輯好配置文件就可以初始化數據庫了:

[root@node0 ~]# mysqld --initialize  --user=mysql
mysqld: Can't create directory '/usr/local/mysql/data/' (Errcode: 2 - No such file or directory)
2018-12-21T14:11:27.076527+08:00 0 [ERROR] Can't find error-message file '/usr/local/mysql/share/mysql/errmsg.sys'. Check error-message file location and 'lc-messages-dir' configuration directive.
2018-12-21T14:11:27.082187+08:00 0 [ERROR] Aborting

出現了兩個錯誤,第一個是無法創建我們制定的文件目錄是權限問題mkdir /usr/local/mysql && chown mysql /usr/local/mysql && chgrp mysql /usr/local/mysql 這樣就好了,創建一個目錄更改一下權限。第二個是在/usr/local/mysql/share/mysql/errmsg.sys目錄下找不到文件,這和我們之前配置文件中basedir=/usr/local/mysql有關係,因爲basedir指定是數據庫的bin,lib之類的文件,我們rpm安裝,這些文件默認的路徑是在/usr/share/mysql下面,所以basedir這個參數可以不指定也可以,系統會自己找,或者直接指定到/usr/share/mysql/這個目錄下面。所以可以將basedir註釋掉或者basedir=/usr/ 再次初始化:
 

[root@node0 ~]# mysqld --initialize  --user=mysql
2018-12-21T14:15:16.922554+08:00 0 [ERROR] Could not open file '/var/log/mysql.error' for error logging: Permission denied
2018-12-21T14:15:16.922696+08:00 0 [ERROR] Aborting

還是權限問題,創建一個更改下屬組,然後就可以了touch /var/log/mysql.error && chown mysql /var/log/mysql.error && chgrp mysql /var/log/mysql.error。初始化後會有一個隨機密碼在錯誤日誌中我們找一下:

[root@node0 ~]# touch /var/log/mysql.error && chown mysql /var/log/mysql.error && chgrp mysql /var/log/mysql.error
[root@node0 ~]# mysqld --initialize  --user=mysql
[root@node0 ~]# grep 'temporary password' /var/log/mysql.error
2018-12-21T14:17:34.557895+08:00 1 [Note] A temporary password is generated for root@localhost: W#_jrIo<U9tG

然後就啓動數據庫了,我們先啓動第一個節點,並查看數據庫啓動狀態,如果數據庫啓動失敗,看是否防火牆沒有關閉等問題,可以事先執行setenforce 0 && systemctl stop firewalld關閉防火牆,再重新輸入命令啓動一下。

[root@node0 ~]# /usr/sbin/mysqld --wsrep-new-cluster --user=mysql &
[1] 6660
[root@node0 ~]# systemctl status mysqld
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
[1]+  退出 1                /usr/sbin/mysqld --wsrep-new-cluster --user=mysql

數據庫沒有啓動成功,將vi /etc/my.cnf.d/wsrep.cnf 第一個啓動節點的配置文件中的wsrep_cluster_address=gcomm:// 設置爲這樣。並且使用數據庫啓動命令:service mysqld start 再次查看數據庫啓動狀態:

[root@node0 sbin]# systemctl status mysqld
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: activating (start-pre) since 五 2018-12-21 15:14:03 CST; 20s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Control: 7474 (mysqld_pre_syst)
   CGroup: /system.slice/mysqld.service
           └─control
             ├─7474 /bin/bash /usr/bin/mysqld_pre_systemd --pre
             └─7491 /usr/sbin/mysqld --datadir=/usr/local/mysql/data --user=mysql --wsrep_recover

12月 21 15:14:03 node0 systemd[1]: mysqld.service holdoff time over, scheduling restart.
12月 21 15:14:03 node0 systemd[1]: Starting MySQL Server...

數據庫會出現如下提示,根據失敗提示查看數據庫狀態

[root@localhost ~]# service mysqld start
Redirecting to /bin/systemctl start mysqld.service
Job for mysqld.service failed because the control process exited with error code. See "systemctl status mysqld.service" and "journalctl -xe" for details.

[root@localhost ~]# systemctl status mysqld.service
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: activating (start-pre) since 二 2018-12-25 10:32:22 CST; 13s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Control: 3808 (mysqld_pre_syst)
   CGroup: /system.slice/mysqld.service
           └─control
             ├─3808 /bin/bash /usr/bin/mysqld_pre_systemd --pre
             └─3825 /usr/sbin/mysqld --datadir=/usr/local/mysql/data --user=mysql --wsrep_recover

12月 25 10:32:22 localhost.localdomain systemd[1]: mysqld.service holdoff time over, scheduling rest
12月 25 10:32:22 localhost.localdomain systemd[1]: Starting MySQL Server...
[root@localhost ~]# journalctl -xe
# --------------以下是 journalctl -xe 最後幾行內容---------------------------------
12月 25 10:32:21 localhost.localdomain systemd[1]: mysqld.service: control process exited, code=exit
12月 25 10:32:21 localhost.localdomain systemd[1]: Failed to start MySQL Server.
-- Subject: Unit mysqld.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit mysqld.service has failed.
-- 
-- The result is failed.
12月 25 10:32:21 localhost.localdomain systemd[1]: Unit mysqld.service entered failed state.
12月 25 10:32:21 localhost.localdomain systemd[1]: mysqld.service failed.
12月 25 10:32:22 localhost.localdomain systemd[1]: mysqld.service holdoff time over, scheduling rest
12月 25 10:32:22 localhost.localdomain systemd[1]: Starting MySQL Server...
-- Subject: Unit mysqld.service has begun start-up
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit mysqld.service has begun starting up.

最後會發現在journalctl -xe記錄記錄中,數據庫實際是啓動成功了。然後咱們再次運行systemctl status mysqld,啓動成功

[root@localhost ~]# systemctl status mysqld
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: activating (start-pre) since 二 2018-12-25 10:34:04 CST; 1min 1s ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Control: 3848 (mysqld_pre_syst)
   CGroup: /system.slice/mysqld.service
           └─control
             ├─3848 /bin/bash /usr/bin/mysqld_pre_systemd --pre
             └─3865 /usr/sbin/mysqld --datadir=/usr/local/mysql/data --user=mysql --wsrep_recover...

12月 25 10:34:04 localhost.localdomain systemd[1]: mysqld.service holdoff time over, schedulin...t.
12月 25 10:34:04 localhost.localdomain systemd[1]: Starting MySQL Server...
Hint: Some lines were ellipsized, use -l to show in full.

進入數據庫,根據日誌密碼進入數據庫後修改密碼,添加同步數據的賬戶,也就是配置文件中我們設置的,更改密碼後,我的節點二和三也變了,好像是同步過去的,不過咱們還需要在創建一下,要不然節點一掛掉後,其他用戶無法使用了。再看一下節點狀態:

[root@localhost ~]# mysql -uroot -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.7.24-log

Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> ALTER USER 'root'@'localhost' IDENTIFIED BY '666666';
Query OK, 0 rows affected (0.00 sec)

mysql> GRANT ALL PRIVILEGES ON *.* TO 'test'@'%' IDENTIFIED BY '123456' WITH GRANT OPTION;FLUSH PRIVILEGES;
Query OK, 0 rows affected, 1 warning (0.00 sec)

Query OK, 0 rows affected (0.01 sec)

mysql> show status like "wsrep%";
+------------------------------+--------------------------------------+
| Variable_name                | Value                                |
+------------------------------+--------------------------------------+
| wsrep_local_state_uuid       | af4b7ecb-07ec-11e9-9d71-cfb0574baac9 |
| wsrep_protocol_version       | 9                                    |
| wsrep_last_committed         | 3                                    |
| wsrep_replicated             | 3                                    |
| wsrep_replicated_bytes       | 712                                  |
| wsrep_repl_keys              | 3                                    |
| wsrep_repl_keys_bytes        | 96                                   |
| wsrep_repl_data_bytes        | 415                                  |
| wsrep_repl_other_bytes       | 0                                    |
| wsrep_received               | 2                                    |
| wsrep_received_bytes         | 144                                  |
| wsrep_local_commits          | 0                                    |
| wsrep_local_cert_failures    | 0                                    |
| wsrep_local_replays          | 0                                    |
| wsrep_local_send_queue       | 0                                    |
| wsrep_local_send_queue_max   | 1                                    |
| wsrep_local_send_queue_min   | 0                                    |
| wsrep_local_send_queue_avg   | 0.000000                             |
| wsrep_local_recv_queue       | 0                                    |
| wsrep_local_recv_queue_max   | 2                                    |
| wsrep_local_recv_queue_min   | 0                                    |
| wsrep_local_recv_queue_avg   | 0.500000                             |
| wsrep_local_cached_downto    | 1                                    |
| wsrep_flow_control_paused_ns | 0                                    |
| wsrep_flow_control_paused    | 0.000000                             |
| wsrep_flow_control_sent      | 0                                    |
| wsrep_flow_control_recv      | 0                                    |
| wsrep_cert_deps_distance     | 1.000000                             |
| wsrep_apply_oooe             | 0.000000                             |
| wsrep_apply_oool             | 0.000000                             |
| wsrep_apply_window           | 1.000000                             |
| wsrep_commit_oooe            | 0.000000                             |
| wsrep_commit_oool            | 0.000000                             |
| wsrep_commit_window          | 1.000000                             |
| wsrep_local_state            | 4                                    |
| wsrep_local_state_comment    | Synced                               |
| wsrep_cert_index_size        | 1                                    |
| wsrep_causal_reads           | 0                                    |
| wsrep_cert_interval          | 0.000000                             |
| wsrep_open_transactions      | 0                                    |
| wsrep_open_connections       | 0                                    |
| wsrep_incoming_addresses     | 192.168.190.132:3306                 |
| wsrep_cluster_weight         | 1                                    |
| wsrep_desync_count           | 0                                    |
| wsrep_evs_delayed            |                                      |
| wsrep_evs_evict_list         |                                      |
| wsrep_evs_repl_latency       | 0/0/0/0/0                            |
| wsrep_evs_state              | OPERATIONAL                          |
| wsrep_gcomm_uuid             | af4a65e4-07ec-11e9-9aac-76a760227320 |
| wsrep_cluster_conf_id        | 1                                    |
| wsrep_cluster_size           | 1                                    |
| wsrep_cluster_state_uuid     | af4b7ecb-07ec-11e9-9d71-cfb0574baac9 |
| wsrep_cluster_status         | Primary                              |
| wsrep_connected              | ON                                   |
| wsrep_local_bf_aborts        | 0                                    |
| wsrep_local_index            | 0                                    |
| wsrep_provider_name          | Galera                               |
| wsrep_provider_vendor        | Codership Oy <[email protected]>    |
| wsrep_provider_version       | 3.25(rddf9876)                       |
| wsrep_ready                  | ON                                   |
+------------------------------+--------------------------------------+
60 rows in set (0.03 sec)

mysql> 

最後不要忘了將配置文件/etc/my.cnf.d/wsrep.cnf中節點的設置修改回來(設置成各節點的IP地址)wsrep_cluster_address=gcomm://192.168.190.132,192.168.190.133,192.168.190.134 

【剩下兩臺機羣如此配置即可,注意幾點:】

1、初次啓動不需要使用啓動腳本:/usr/bin/mysqld_bootstrap,因爲這個腳本中帶有一個參數:--wsrep-new-cluster,代表新集羣。
2、配置文件 wsrep.cnf 中的參數 wsrep_cluster_address,一開始就要配置好了,全部配置成指向主節點的IP地址,如:wsrep_cluster_address=gcomm://192.168.190.132:4567  (192.168.190.132爲主節點IP地址 )不能爲空。

3、配置文件 wsrep.cnf 中的參數 wsrep_node_address主節點爲wsrep_node_address=node1,其他的個節點依次爲:wsrep_node_address=node2  , wsrep_node_address=node3 ,wsrep_node_address=node...

4、配置文件 wsrep.cnf 中的參數server_id必須是唯一的,在配置各節點時,避免出現重複值!
5、類似配置文件 wsrep.cnf 中的 wsrep_node_address 這種參數要配置成自己當前及其所在的IP地址信息。
6、正式環境情況下,我們不要把防火牆關閉掉,把需要使用的端口開放給指定 IP 即可,更加安全。

都搭建成功後,可以看到,在任何一個節點上改動數據庫(如創建數據庫,改動數據表等操作),另外的節點都會有同步變化!

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章