概述
複製通常用來創建主節點的副本,通過添加冗餘節點來保證高可用性,當然複製也可以用於其他 用途,例如在從節點上進行數據讀、分析等等。在橫向擴展的業務中,複製很容易實施,主要表現在在利用主節點進行寫操作,多個從節點進行讀操作,在mysql5.5中默認爲異步複製。
mysql 複製的異步性是指:事物首先在主節點上提交,然後複製給從節點並在從節點上應用,這樣意味着在同一個時間點主從上的數據可能不一致,異步複製的好處在於它 比同步複製要快,如果對數據的一致性要求很高,還是採用同步複製較好。
mysql-5.5 開始支持semi-synchronous的複製,也叫半同步複製,目的在於事務環境下保持主從一致
mysql-5.6 開始支持延時複製。
mysql複製的原理現階段都是一樣的,master將操作記錄到bin-log中,slave的一個線程去master讀取bin-log,並將他們保存到relay-log中,slave的另外一個線程去重放relay-log中的操作來實現和master數據同步。
該過程的第一部分就是master記錄二進制日誌。在每個事務更新數據完成之前,master在二日誌記錄這些改變。MySQL將事務串行的寫入二進制日誌,即使事務中的語句都是交叉執行的。在事件寫入二進制日誌完成後,master通知存儲引擎提交事務。
下一步就是slave將master的binary
log拷貝到它自己的中繼日誌。首先,slave開始一個工作線程——I/O線程。I/O線程在master上打開一個普通的連接,然後開始binlog
dump process。Binlog dump
process從master的二進制日誌中讀取事件,如果已經跟上master,它會睡眠並等待master產生新的事件。I/O線程將這些事件寫入中
繼日誌。
SQL slave
thread(SQL從線程)處理該過程的最後一步。SQL線程從中繼日誌讀取事件,並重放其中的事件而更新slave的數據,使其與master中的數
據一致。只要該線程與I/O線程保持一致,中繼日誌通常會位於OS的緩存中,所以中繼日誌的開銷很小。
此外,在master中也有一個工作線程:和其它MySQL的連接一樣,slave在master中打開一個連接也會使得master開始一個線程。複製
過程有一個很重要的限制——複製在slave上是串行化的,也就是說master上的並行更新操作不能在slave上並行操作。
異步主從複製配置
準備:
OS:rhel5.8_i386
SoftWare: mysql-5.5.28-linux2.6-i686.tar.gz
1、主從安裝mysql
tar xf mysql-5.5.28-linux2.6-i686.tar.gz -C /usr/local/ # cd /usr/local/ # ln -s mysql-5.5.28-linux2.6-i686/ mysql # groupadd -r mysql # useradd -r -g mysql -s /sbin/nologin mysql # mkdir /mydata/data -p # chown -R mysql.mysql /mydata/data/ # chown -R root.mysql /usr/local/mysql/* # cp support-files/my-large.cnf /etc/my.cnf # cp support-files/mysql.server /etc/init.d/mysqld [mysqld] innodb_file_per_table = 1 datadir = /mydata/data #由於是二進制安裝的mysql所以必須指定數據庫目錄位置 # vim /etc/profile.d/mysqld.sh export PAHT=$PATH:/usr/local/mysql/bin # . /etc/profile.d/mysqld.sh
2、主服務器配置
# vim /etc/my.cnf [mysqld] log-bin = master-bin log-bin-index = master-bin.index server-id = 1 # scripts/mysql_install_db --user=mysql --datadir=/mydata/data/ # service mysqld start # mysql mysql> grant replication slave on *.* to repl@'192.168.100.12' identified by 'asdasd'; mysql> flush privileges; mysql> flush logs; mysql> show master logs; +-------------------+-----------+ | Log_name | File_size | +-------------------+-----------+ | master-bin.000001 | 27326 | | master-bin.000002 | 1038693 | | master-bin.000003 | 379 | | master-bin.000004 | 107 | +-------------------+-----------+ mysql> purge binary logs to 'master-bin.000004';
3、從服務器配置
# vim /etc/my.cnf [mysqld] relay-log = relay-log relay-log-index = relay-log.index read-only = 1 #innodb_file_per_table = 1 #binlog_format=mixed server-id = 10 # scripts/mysql_install_db --user=mysql --datadir=/mydata/data/ # service mysqld start mysql> change master to master_host='192.168.100.11',master_user='repl',master_password='asdasd',master_log_file='master-bin.000004',master_log_pos=107; mysql> show slave status\G *************************** 1. row *************************** Slave_IO_State: Master_Host: 192.168.100.11 Master_User: repl Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000004 Read_Master_Log_Pos: 107 Relay_Log_File: relay-log.000001 Relay_Log_Pos: 4 Relay_Master_Log_File: master-bin.000004 Slave_IO_Running: No Slave_SQL_Running: No Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 107 Relay_Log_Space: 107 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: No Master_SSL_CA_File: Master_SSL_CA_Path: Master_SSL_Cert: Master_SSL_Cipher: Master_SSL_Key: Seconds_Behind_Master: NULL Master_SSL_Verify_Server_Cert: No Last_IO_Errno: 0 Last_IO_Error: Last_SQL_Errno: 0 Last_SQL_Error: Replicate_Ignore_Server_Ids: Master_Server_Id: 0 1 row in set (0.00 sec) mysql> start slave; mysql> show slave status\G Slave_IO_Running: Yes Slave_SQL_Running: Yes ..............
至此主從異步複製就完成了
說明:
slave_id 必須是唯一的
slave沒有必要開啓二進制日誌,但在有些情況下必須設置,如mysql級聯. slave爲其它slave的master,所以要設置bin_log,默認爲hostname,但如果hostname改變則會出問題。
有些人可能開啓了slave二進制日誌,卻沒有設置log_slave_updates,然後查看slave的數據是否改變,這是錯誤的配置。所以儘量使用read_only = 1 ,防止改變數據(除了sql_thread進程)。
start slave :啓動從服務器IO_Thread和SQL_Thread線程,這裏也可以單獨對它們進行啓動
在主服務器上需設置sync-binlog = 1 ,用於事務安全
重置change master參數:
mysql> slave stop; mysql> reset slave; mysql> change master to master_host='192.168.100.11',master_user='repl',master_password='asdasd',master_log_file='master-bin.000005',master_log_pos=107;
由於slave都會自動連接上master,當我們有時需要手動調整時可以在啓動前移動slave數據目錄下的master.ino和relay.info文件,或者查看variables中有無“skip-slave-start”變量,有就設置爲ON
半同步複製
/usr/local/mysql/lib/plugin/semisync_master.so
/usr/local/mysql/lib/plugin/semisync_slave.so
1、主服務器配置
mysql> install plugin rpl_semi_sync_master soname 'semisync_master.so'; mysql> show variables like '%semi%'; +------------------------------------+-------+ | Variable_name | Value | +------------------------------------+-------+ | rpl_semi_sync_master_enabled | OFF | | rpl_semi_sync_master_timeout | 10000 | | rpl_semi_sync_master_trace_level | 32 | | rpl_semi_sync_master_wait_no_slave | ON | +------------------------------------+-------+ mysql> set global rpl_semi_sync_master_enabled=1; mysql> set global rpl_semi_sync_master_timeout=1000;
2、Slave上配置
mysql> install plugin rpl_semi_sync_slave soname 'semisync_slave.so'; mysql> show variables like '%semi%'; +---------------------------------+-------+ | Variable_name | Value | +---------------------------------+-------+ | rpl_semi_sync_slave_enabled | OFF | | rpl_semi_sync_slave_trace_level | 32 | +---------------------------------+-------+ mysql> set global rpl_semi_sync_slave_enabled=1;
如果需要永久生效,請將上面幾個變量分別寫入master與slave中mysqld字段中。
MySQL複製過濾
MySQL複製過濾可以在Master,也可以在Slave
由於基於Master的過濾操作爲影響到二進制日誌的完整性,對於我們以後做及時點還原會有影響,所以我們一般不建議在Maser上做複製過濾。
1、基於數據庫
binlog-do-db //binlog-do-db表示和哪個數據庫相關的寫入類、修改類指令會被寫入 binlog-ignore-db //binlog-ignore-db表示忽略(黑名單)
2、基於表
replicate-do-table=
replicate-ignore-table=
3、對於表,還可以用通配符配置過濾
replicate-wild-do-table=
replicate-wild-ignore-table=
SSL複製
要求主從服務器各自都要有證書和私鑰;默認情況下主從服務器的SSL功能是沒有啓用的,需要先啓用。
mysql> show variables like '%ssl%'; +---------------+----------+ | Variable_name | Value | +---------------+----------+ | have_openssl | DISABLED | | have_ssl | DISABLED | | ssl_ca | | | ssl_capath | | | ssl_cert | | | ssl_cipher | | | ssl_key | | +---------------+----------+
1、配置Master爲CA證書服務器
# vim /etc/pki/tls/openssl.cnf # cd /etc/pki/CA/ # (umask 077; openssl genrsa 1024 >private/cakey.pem) # openssl req -new -x509 -key private/cakey.pem -out cacert.pem Country Name (2 letter code) [GB]:CN State or Province Name (full name) [Berkshire]:GD Locality Name (eg, city) [Newbury]:ZS Organization Name (eg, company) [My Company Ltd]:NEO Organizational Unit Name (eg, section) []:tech Common Name (eg, your name or your server's hostname) []:station01.neo.com # mkdir newcerts certs crl # touch index.txt # echo 01 >serial
2、爲Master上的MySQL準備私鑰以及頒發證書
# mkdir /usr/local/mysql/ssl # cd /usr/local/mysql/ssl/ #(umask 077; openssl genrsa 1024 > mysql.key) # openssl req -new -key mysql.key -out mysql.csr -days 3650 Country Name (2 letter code) [GB]:CN State or Province Name (full name) [Berkshire]:GD Locality Name (eg, city) [Newbury]:ZS Organization Name (eg, company) [My Company Ltd]:NEO Organizational Unit Name (eg, section) []:tech Common Name (eg, your name or your server's hostname) []:station01.neo.com Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: # openssl ca -in mysql.csr -out mysql.crt Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok Certificate Details: Serial Number: 1 (0x1) Validity Not Before: May 28 02:26:17 2014 GMT Not After : May 28 02:26:17 2015 GMT Subject: countryName = CN stateOrProvinceName = GD organizationName = NEO organizationalUnitName = tech commonName = station01.neo.com X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: A4:B7:A6:98:9F:60:08:BE:86:87:65:5F:B6:13:BC:4A:5B:D4:44:3A X509v3 Authority Key Identifier: keyid:4F:D8:57:42:D9:39:17:7D:39:44:91:01:A4:01:DE:32:92:D6:F9:DF Certificate is to be certified until May 28 02:26:17 2015 GMT (365 days) Sign the certificate? [y/n]:y 1 out of 1 certificate requests certified, commit? [y/n]y Write out database with 1 new entries Data Base Updated # chown mysql.mysql *
3、Slave上申請證書
# mkdir /usr/local/mysql/ssl # (umask 077; openssl genrsa 1024 >mysql.key) # openssl req -new -key mysql.key -out mysql.csr -days 3650 Country Name (2 letter code) [GB]:CN State or Province Name (full name) [Berkshire]:GD Locality Name (eg, city) [Newbury]:ZS Organization Name (eg, company) [My Company Ltd]:NEO Organizational Unit Name (eg, section) []:tech Common Name (eg, your name or your server's hostname) []:station02.neo.com Email Address []: Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []: # scp mysql.csr 192.168.100.11:/root/
4、Master上爲Slave簽發證書
# openssl ca -in mysql.csr -out mysql.crt Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok Certificate Details: Serial Number: 2 (0x2) Validity Not Before: May 28 02:36:24 2014 GMT Not After : May 28 02:36:24 2015 GMT Subject: countryName = CN stateOrProvinceName = GD organizationName = NEO organizationalUnitName = tech commonName = station02.neo.com X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: 81:9F:5B:E7:06:D0:64:B7:E6:81:3F:98:95:71:D4:DF:C6:B8:CE:3D X509v3 Authority Key Identifier: keyid:4F:D8:57:42:D9:39:17:7D:39:44:91:01:A4:01:DE:32:92:D6:F9:DF Certificate is to be certified until May 28 02:36:24 2015 GMT (365 days) Sign the certificate? [y/n]:yes 1 out of 1 certificate requests certified, commit? [y/n]yes Write out database with 1 new entries Data Base Updated # scp mysql.crt 192.168.100.12:/usr/local/mysql/ssl/ # scp /etc/pki/CA/cacert.pem 192.168.100.12:/usr/local/mysql/ssl/
5、Master上編緝/etc/my.cnf啓用ssl,並設置主從
# vim /etc/my.cnf [mysqld] log-bin=mysql-bin sync_binlog = 1 ##二進制日誌 server-id = 1 ##此id必須全局唯一 innodb_flush_log_at_trx_commit=1 ##每秒將事務日誌立刻刷寫到磁盤 ssl ##啓用ssl默認是不開啓的,mysql中show variables like '%ssl%'查看 ssl_ca =/usr/local/mysql/ssl/cacert.pem ##ca文件的位置 ssl_cert= /usr/local/mysql/ssl/mysql.crt ##證書文件的位置 ssl_key = /usr/local/mysql/ssl/mysql.key ##私鑰文件的位置
6、啓動mysql,並查看ssl信息
# service mysqld start # mysql mysql> show variables like '%ssl%'; +---------------+---------------------------------+ | Variable_name | Value | +---------------+---------------------------------+ | have_openssl | YES | | have_ssl | YES | | ssl_ca | /usr/local/mysql/ssl/cacert.pem | | ssl_capath | | | ssl_cert | /usr/local/mysql/ssl/mysql.crt | | ssl_cipher | | | ssl_key | /usr/local/mysql/ssl/mysql.key | +---------------+---------------------------------+
7、爲同步建立一最小權限賬戶,並要求ssl
mysql> create user 'backup_ssl'@'192.168.100.12' identified by 'redhat'; mysql> revoke all privileges,grant option from 'backup_ssl'@'192.168.100.12'; mysql> grant replication slave,replication client on *.* to 'backup_ssl'@'192.168.100.12' require ssl; mysql> flush privileges; mysql> flush logs;
8、Slave上編緝/etc/my.cnf,啓用ssl,並設置主從
# vim /etc/my.cnf [mysqld] server-id = 2 ##此id必須全局唯一 ##log-bin = mysql-bin ##註釋掉,從服務器不需要二進制日誌 relay-log = mysql-relay ##中繼日誌 relay-log-index = mysql-ralay.index ##中繼目錄 read-only = 1 ##從服務器只讀 ssl ##啓用ssl默認是不開啓的,mysql中show variables like '%ssl%'查看 ssl_ca =/usr/local/mysql/ssl/cacert.pem ##ca文件的位置 ssl_cert= /usr/local/mysql/ssl/mysql.crt ##證書文件的位置 ssl_key = /usr/local/mysql/ssl/mysql.key ##私鑰文件的位置
9、啓用mysqld並查看ssl相關信息
# servie mysqld start mysql> show variables like '%ssl%'; +---------------+---------------------------------+ | Variable_name | Value | +---------------+---------------------------------+ | have_openssl | YES | | have_ssl | YES | | ssl_ca | /usr/local/mysql/ssl/cacert.pem | | ssl_capath | | | ssl_cert | /usr/local/mysql/ssl/mysql.crt | | ssl_cipher | | | ssl_key | /usr/local/mysql/ssl/mysql.key | +---------------+---------------------------------+
10、啓動slave同步進程,連接主服務器
mysql> change master to -> master_host='192.168.100.11', -> master_user='backup_ssl', -> master_password='redhat', -> master_log_file='mysql-bin.000004', -> master_ssl=1, -> master_ssl_ca='/usr/local/mysql/ssl/cacert.pem', -> master_ssl_cert='/usr/local/mysql/ssl/mysql.crt', -> master_ssl_key='/usr/local/mysql/ssl/mysql.key'; mysql> start slave mysql> show slave status\G; ##查看slave狀態
11、關注以下參數:
Slave_IO_Running: Yes ##IOthread是否運行,如果爲No代表slave運行不正常 Slave_SQL_Running: Yes ##SQLthread是否運行,如果爲No代表slave運行不正常 Master_SSL_CA_File: /usr/local/mysql/ssl/cacert.pem ##是否啓用了ssl Master_SSL_Cert: /usr/local/mysql/ssl/mysql.crt Master_SSL_Key: /usr/local/mysql/ssl/mysql.key Master_Log_File: mysql-bin.00005 ##最後接收的主服務器的二進制 Exec_Master_Log_Pos: 338 ##最後執行的位置,查看master中是不是該位置 Last_IO_Errno: 0 ##最後一次IOthread有沒有報錯