Docker部署MySQL主從同步解決方案

MySQL之間數據複製的基礎是二進制日誌文件（binary log file）。一臺MySQL數據庫一旦啓用二進制日誌後，其作爲master，它的數據庫中所有操作都會以“事件”的方式記錄在二進制日誌中，其他數據庫作爲slave通過一個I/O線程與主服務器保持通信，並監控master的二進制日誌文件的變化，如果發現master二進制日誌文件發生變化，則會把變化複製到自己的中繼日誌中，然後slave的一個SQL線程會把相關的“事件”執行到自己的數據庫中，以此實現從數據庫和主數據庫的一致性，也就實現了主從複製。

實現MySQL主從複製需要進行的配置：

主服務器：
- 開啓二進制日誌
- 配置唯一的server-id
- 獲得master二進制日誌文件名及位置
- 創建一個用於slave和master通信的用戶賬號
從服務器:
- 配置唯一的server-id
- 使用master分配的用戶賬號讀取master二進制日誌
- 啓用slave服務

一、封裝Docker鏡像

1、創建MySQL Dockerfile

操作系統鏡像採用 ubuntu:trusty
MySQL採用最新的8.0版本, https://dev.mysql.com/doc/refman/8.0/en/ ，端口採用3306

FROM ubuntu:trusty

# Add MySQL configuration
COPY my.cnf /etc/mysql/conf.d/my.cnf
COPY mysqld_charset.cnf /etc/mysql/conf.d/mysqld_charset.cnf
COPY mysql-apt-config_0.8.12-1_all.deb /mysql-apt-config_0.8.12-1_all.deb

RUN apt-get update && apt-get install -y wget && apt-get install -y vim
RUN dpkg -i /mysql-apt-config_0.8.12-1_all.deb && \
    apt-get -yq install mysql-server pwgen && \
    rm -rf /var/lib/apt/lists/* && \
    rm /etc/mysql/conf.d/mysqld_safe_syslog.cnf && \
    if [ ! -f /usr/share/mysql/my-default.cnf ] ; then cp /etc/mysql/my.cnf /usr/share/mysql/my-default.cnf; fi && \
    mysql_install_db > /dev/null 2>&1 && \
    touch /var/lib/mysql/.EMPTY_DB

# Add MySQL scripts
COPY import_sql.sh /import_sql.sh
COPY run.sh /run.sh

ENV MYSQL_USER=admin \
    MYSQL_PASS=**Random** \
    ON_CREATE_DB=**False** \
    REPLICATION_MASTER=**False** \
    REPLICATION_SLAVE=**False** \
    REPLICATION_USER=replica \
    REPLICATION_PASS=replica

# Add VOLUMEs to allow backup of config and databases
VOLUME  ["/etc/mysql", "/var/lib/mysql"]

EXPOSE 3306
CMD ["/run.sh"]

2、創建MySQL配置文件

my.cnf

[mysqld]
bind-address=0.0.0.0
# http://www.percona.com/blog/2008/05/31/dns-achilles-heel-mysql-installation/
skip_name_resolve
#server-id
#log-bin

mysqld_charset.cnf

[mysqld]
character_set_server=utf8
character_set_filesystem=utf8
collation-server=utf8_general_ci
init-connect='SET NAMES utf8'
init_connect='SET collation_connection = utf8_general_ci'
skip-character-set-client-handshake

3、創建MySQL腳本文件

import_sql.sh

#!/bin/bash

if [[ $# -ne 3 ]]; then
    echo "Usage: $0 <username> <password> </path/to/sql_file.sql>"
    exit 1
fi

echo "=> Starting MySQL Server"
/usr/bin/mysqld_safe > /dev/null 2>&1 &
PID=$!

RET=1
while [[ $RET -ne 0 ]]; do
    echo "=> Waiting for confirmation of MySQL service startup"
    sleep 5
    mysql -u"$1" -p"$2" -e "status" > /dev/null 2>&1
    RET=$?
done

echo "   Started with PID ${PID}"

echo "=> Importing SQL file"
mysql -u"$1" -p"$2" < "$3"

echo "=> Stopping MySQL Server"
mysqladmin -u"$1" -p"$2" shutdown

echo "=> Done!"

run.sh

#!/bin/bash

set -m
set -e

VOLUME_HOME="/var/lib/mysql"
CONF_FILE="/etc/mysql/conf.d/my.cnf"
LOG="/var/log/mysql/error.log"

# Set permission of config file
chmod 644 ${CONF_FILE}
chmod 644 /etc/mysql/conf.d/mysqld_charset.cnf

StartMySQL ()
{
    /usr/bin/mysqld_safe ${EXTRA_OPTS} > /dev/null 2>&1 &
    # Time out in 1 minute
    LOOP_LIMIT=60
    for (( i=0 ; ; i++ )); do
        if [ ${i} -eq ${LOOP_LIMIT} ]; then
            echo "Time out. Error log is shown as below:"
            tail -n 100 ${LOG}
            exit 1
        fi
        echo "=> Waiting for confirmation of MySQL service startup, trying ${i}/${LOOP_LIMIT} ..."
        sleep 1
        mysql -uroot -e "status" > /dev/null 2>&1 && break
    done
}

CreateMySQLUser()
{
    if [ "$MYSQL_PASS" = "**Random**" ]; then
        unset MYSQL_PASS
    fi

    PASS=${MYSQL_PASS:-$(pwgen -s 12 1)}
    _word=$( [ ${MYSQL_PASS} ] && echo "preset" || echo "random" )
    echo "=> Creating MySQL user ${MYSQL_USER} with ${_word} password"

    mysql -uroot -e "CREATE USER '${MYSQL_USER}'@'%' IDENTIFIED BY '$PASS'"
    mysql -uroot -e "GRANT ALL PRIVILEGES ON *.* TO '${MYSQL_USER}'@'%' WITH GRANT OPTION"
    echo "=> Done!"
    echo "========================================================================"
    echo "You can now connect to this MySQL Server using:"
    echo ""
    echo "    mysql -u$MYSQL_USER -p$PASS -h<host> -P<port>"
    echo ""
    echo "Please remember to change the above password as soon as possible!"
    echo "MySQL user 'root' has no password but only allows local connections"
    echo "========================================================================"
}

OnCreateDB()
{
    if [ "$ON_CREATE_DB" = "**False**" ]; then
        unset ON_CREATE_DB
    else
        echo "Creating MySQL database ${ON_CREATE_DB}"
        mysql -uroot -e "CREATE DATABASE IF NOT EXISTS ${ON_CREATE_DB};"
        echo "Database created!"
    fi
}

ImportSql()
{
    for FILE in ${STARTUP_SQL}; do
        echo "=> Importing SQL file ${FILE}"
        if [ "$ON_CREATE_DB" ]; then
            mysql -uroot "$ON_CREATE_DB" < "${FILE}"
        else
            mysql -uroot < "${FILE}"
        fi
    done
}

# Main
if [ ${REPLICATION_MASTER} == "**False**" ]; then
    unset REPLICATION_MASTER
fi

if [ ${REPLICATION_SLAVE} == "**False**" ]; then
    unset REPLICATION_SLAVE
fi

# Initialize empty data volume and create MySQL user
if [[ ! -d $VOLUME_HOME/mysql ]]; then
    echo "=> An empty or uninitialized MySQL volume is detected in $VOLUME_HOME"
    echo "=> Installing MySQL ..."
    if [ ! -f /usr/share/mysql/my-default.cnf ] ; then
        cp /etc/mysql/my.cnf /usr/share/mysql/my-default.cnf
    fi
    mysql_install_db || exit 1
    touch /var/lib/mysql/.EMPTY_DB
    echo "=> Done!"
else
    echo "=> Using an existing volume of MySQL"
fi

# Set MySQL REPLICATION - MASTER
if [ -n "${REPLICATION_MASTER}" ]; then
    echo "=> Configuring MySQL replication as master (1/2) ..."
    if [ ! -f /replication_set.1 ]; then
        RAND="$(date +%s | rev | cut -c 1-2)$(echo ${RANDOM})"
        echo "=> Writting configuration file '${CONF_FILE}' with server-id=${RAND}"
        sed -i "s/^#server-id.*/server-id = ${RAND}/" ${CONF_FILE}
        sed -i "s/^#log-bin.*/log-bin = mysql-bin/" ${CONF_FILE}
        touch /replication_set.1
    else
        echo "=> MySQL replication master already configured, skip"
    fi
fi

# Set MySQL REPLICATION - SLAVE
if [ -n "${REPLICATION_SLAVE}" ]; then
    echo "=> Configuring MySQL replication as slave (1/2) ..."
    if [ -n "${MYSQL_PORT_3306_TCP_ADDR}" ] && [ -n "${MYSQL_PORT_3306_TCP_PORT}" ]; then
        if [ ! -f /replication_set.1 ]; then
            RAND="$(date +%s | rev | cut -c 1-2)$(echo ${RANDOM})"
            echo "=> Writting configuration file '${CONF_FILE}' with server-id=${RAND}"
            sed -i "s/^#server-id.*/server-id = ${RAND}/" ${CONF_FILE}
            sed -i "s/^#log-bin.*/log-bin = mysql-bin/" ${CONF_FILE}
            touch /replication_set.1
        else
            echo "=> MySQL replication slave already configured, skip"
        fi
    else
        echo "=> Cannot configure slave, please link it to another MySQL container with alias as 'mysql'"
        exit 1
    fi
fi


echo "=> Starting MySQL ..."
StartMySQL
tail -F $LOG &

# Create admin user and pre create database
if [ -f /var/lib/mysql/.EMPTY_DB ]; then
    echo "=> Creating admin user ..."
    CreateMySQLUser
    OnCreateDB
    rm /var/lib/mysql/.EMPTY_DB
fi


# Import Startup SQL
if [ -n "${STARTUP_SQL}" ]; then
    if [ ! -f /sql_imported ]; then
        echo "=> Initializing DB with ${STARTUP_SQL}"
        ImportSql
        touch /sql_imported
    fi
fi

# Set MySQL REPLICATION - MASTER
if [ -n "${REPLICATION_MASTER}" ]; then
    echo "=> Configuring MySQL replication as master (2/2) ..."
    if [ ! -f /replication_set.2 ]; then
        echo "=> Creating a log user ${REPLICATION_USER}:${REPLICATION_PASS}"
        mysql -uroot -e "CREATE USER '${REPLICATION_USER}'@'%' IDENTIFIED BY '${REPLICATION_PASS}'"
        mysql -uroot -e "GRANT REPLICATION SLAVE ON *.* TO '${REPLICATION_USER}'@'%'"
        mysql -uroot -e "reset master"
        echo "=> Done!"
        touch /replication_set.2
    else
        echo "=> MySQL replication master already configured, skip"
    fi
fi

# Set MySQL REPLICATION - SLAVE
if [ -n "${REPLICATION_SLAVE}" ]; then
    echo "=> Configuring MySQL replication as slave (2/2) ..."
    if [ -n "${MYSQL_PORT_3306_TCP_ADDR}" ] && [ -n "${MYSQL_PORT_3306_TCP_PORT}" ]; then
        if [ ! -f /replication_set.2 ]; then
            echo "=> Setting master connection info on slave"
            mysql -uroot -e "CHANGE MASTER TO MASTER_HOST='${MYSQL_PORT_3306_TCP_ADDR}',MASTER_USER='${MYSQL_ENV_REPLICATION_USER}',MASTER_PASSWORD='${MYSQL_ENV_REPLICATION_PASS}',MASTER_PORT=${MYSQL_PORT_3306_TCP_PORT}, MASTER_CONNECT_RETRY=30"
            mysql -uroot -e "start slave"
            echo "=> Done!"
            touch /replication_set.2
        else
            echo "=> MySQL replication slave already configured, skip"
        fi
    else
        echo "=> Cannot configure slave, please link it to another MySQL container with alias as 'mysql'"
        exit 1
    fi
fi

fg

4、創建MySQL鏡像

按照自定義的tag創建MySQL鏡像

docker build -t wxsc/mysql:stable .

查看鏡像信息

二、啓動Master容器

docker run -d --privileged=true -e REPLICATION_MASTER=true -e REPLICATION_PASS=admin -e MYSQL_PASS=vdi_system -e MYSQL_USER=vdi_system -p 3306:3306 -v /home/workspace/mysql/sock/master:/var/run/mysqld -v /home/workspace/mysql/data/master:/var/lib/mysql --name mysql wxsc/mysql:stable

-p，表示暴露的端口，即本機端口3306與容器端口3306映射
-v /home/workspace/mysql/sock/master:/var/run/mysqld掛載sock文件（前面爲本地文件夾，後面爲容器中的文件夾）
-v /home/workspace/mysql/data/master:/var/lib/mysql掛載數據
-e，表示傳入的環境變量，這裏修改了mysql的用戶名和密碼，即MYSQL_USER = vdi_system，MYSQL_PASS = vdi_system

Permission denied錯誤

錯誤1: changing ownership of ‘/var/lib/mysql’: Permission denied

chown: changing ownership of '/var/lib/mysql': Permission denied
Cannot change ownership of the database directories to the 'mysql'
user.  Check that you have the necessary permissions and try again.

錯誤2: Can’t read dir of ‘/etc/mysql/conf.d/’ (Errcode: 13 - Permission denied)

mysqld: Can't read dir of '/etc/mysql/conf.d/' (Errcode: 13 - Permission denied) mysqld: [ERROR] Fatal error in defaults handling. Program aborted!

原因

因爲Centos7安全Selinux禁止了一些安全權限，導致mysql和mariadb在進行掛載/var/lib/mysql的時候會提示如下信息

解決方法

在docker run中加入–privileged=true 給容器加上特定權限

錯誤3: Can’t start server : Bind on unix socket: Permission denied

chown: changing ownership of '/var/lib/mysql': Permission denied
Cannot change ownership of the database directories to the 'mysql'
user.  Check that you have the necessary permissions and try again.

解決方法

找到.sock文件設置的路徑，給此文件所在文件夾更改爲mysql的用戶所有，並且增加所有用戶的讀寫權限

chmod -R 777 /home/workspace/mysql
docker start CONTAINER ID

三、啓動Slave容器

docker run -d --privileged=true -e REPLICATION_SLAVE=true -e MYSQL_PASS=vdi_system -e MYSQL_USER=vdi_system -p 3307:3306 -v /home/workspace/mysql/sock/slave:/var/run/mysqld -v /home/workspace/mysql/data/slave:/var/lib/mysql --name mysql-slave --link mysql:mysql wxsc/mysql:stable

-p，表示暴露的端口，即本機端口3307與容器端口3306映射
–link，該參數的格式爲–link :alias，其中name和id是源容器的name和id，alias是源容器在link下的別名。主從模式alias必須叫mysql，否則會提示 `Cannot configure slave, please link it to another MySQL container with alias as ‘mysql’！

查看容器信息

四、啓動MySQL主從同步

1、登錄數據庫

登錄主數據庫

mysql -h 192.168.212.213 -uvdi_system -p

登錄從數據庫

mysql -h 192.168.212.213 -P3307 -uvdi_system -p

2、配置主數據庫

查看binlog日誌的狀態

mysql > show variables like '%log_bin%';

更改配置文件/etc/mysql/my.cnf

server-id		= 1
log_bin			= /var/lib/mysql/mysql-bin

重啓鏡像

主數據庫加鎖，阻止對數據庫進行任何的寫操作

mysql > flush tables with read lock;

獲取二進制日誌的信息

mysql > show master status;

File：當前使用的二進制日誌的文件名
Position：日誌裏面的位置信息

導出主數據庫數據

mysqldump -uvdi_system -p'vdi_system' -P 3306 -S /home/workspace/mysql/sock/master/mysqld.sock --all-databases > /home/workspace/mysql/backup/mysql_bak.$(date +%F).sql

數據量大可以採用壓縮方式導出

mysqldump -uvdi_system -p'vdi_system' -P 3306 -S /home/workspace/mysql/sock/master/mysqld.sock --all-databases | gzip > /home/workspace/mysql/backup/mysql_bak.$(date +%F).sql.gz

主數據庫解鎖

mysql > unlock tables;

3、配置從數據庫

更改配置文件/etc/mysql/my.cnf

server-id		= 2
log_bin			= /var/lib/mysql/mysql-bin

如果有多個從服務器，那麼每個服務器上配置的server-id都必須不一致。從服務器上無需配置log-bin。如果需要在從服務器上進行數據備份和災難恢復，或者讓從服務器變成一個主服務器，也可以配置log-bin選項。

導入主數據庫數據

mysql -uvdi_system -p'vdi_system' -P 3307 -S /home/workspace/mysql/sock/slave/mysqld.sock < /home/workspace/mysql/backup/mysql_bak.2019-04-03.sql

配置同步參數（先檢查狀態)

mysql> CHANGE MASTER TO
-> MASTER_HOST='master_host_name',
-> MASTER_PORT='master_port',
-> MASTER_USER='replication_user_name',
-> MASTER_PASSWORD='replication_password',
-> MASTER_LOG_FILE='recorded_log_file_name',
-> MASTER_LOG_POS='recorded_log_pos',

MASTER_HOST：主數據庫服務器IP地址
MASTER_PORT：主數據庫服務器端口
MASTER_USER，MASTER_PASSWORD：主從同步的用戶名密碼
MASTER_LOG_FILE，MASTER_LOG_POS：主數據庫服務器狀態

啓動主從同步進程

mysql > start slave;

檢查狀態

mysql > show slave status \G

Slave_IO_Running：負責從庫去主庫讀取二進制日誌，並寫入到從庫的中繼日誌
Slave_SQL_Running：負責將中繼日誌轉換成SQL語句後執行

Docker部署MySQL主從同步解決方案

一、封裝Docker鏡像

1、創建MySQL Dockerfile

2、創建MySQL配置文件

3、創建MySQL腳本文件

4、創建MySQL鏡像

二、啓動Master容器

Permission denied錯誤

錯誤1: changing ownership of ‘/var/lib/mysql’: Permission denied

錯誤2: Can’t read dir of ‘/etc/mysql/conf.d/’ (Errcode: 13 - Permission denied)

錯誤3: Can’t start server : Bind on unix socket: Permission denied

三、啓動Slave容器

四、啓動MySQL主從同步

1、登錄數據庫

2、配置主數據庫

3、配置從數據庫

10分鐘搞定Mysql主從部署配置

如何使用 JS 判斷用戶是否處於活躍狀態

「Pygors跨平臺GUI」2：安裝MinGW-w64、MSYS2還是WSL2

[轉帖]

python列出centos7內存使用前50的進程信息

「Pygors跨平臺GUI」1：Pygors跨平臺GUI應用研究

一鍵自動化博客發佈工具,用過的人都說好(掘金篇)

lightdb數據庫超時相關控制參數

lightdb秒級增加列和刪除列（not null帶默認值）

Java ThreadPoolShutdown

在Visual Studio Code中利用python3 matplotlib繪製等值線圖

使用uglifyjs進行javascript文件的壓縮混淆

Docker MySQL從.frm和.ibd文件恢復數據

OpenLDAP源碼安裝及配置管理

OpenLDAP在線安裝及配置管理

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結