数据目录迁移那些事

现象:一次磁盘告警,提示剩余空间超阈值,df -h查看根目录的Avail已剩不到2G。

原因:MySQL的数据目录挂载在根目录的data目录下,撑爆了

解决方法:迁移数据目录。

实际处理过程:set global innodb_fast_shutdown = 0; 后关闭数据库,mv走数据目录,修改配置文件的datadir,重启数据库。

然后。。。

报错了:

Starting MySQL….. ERROR! The server quit without updating PID file

 

以下为故障重现过程:

数据库原datadir=/data/mysql3306、log-bin = /data/mysql3306/mybinlog,准备迁移到/home路径下:

mkdir /home/mysqldata

mv /data/mysql3306 /home/mysqldata/

那么,新的datadir=/home/mysqldata/mysql3306,新的log-bin = /home/mysqldata/mysql3306/mybinlog

在my.cnf中按需修改相关路径后,另一一个session执行 # tail -f error.log 监控错误日志的输出,然后重启数据库。

(1)使用support-files下的mysql.server 启动一般报错如下:
Starting MySQL...... ERROR! The server quit without updating PID file (/home/mysqldata/mysql3306/mysqldb.pid).

(2)使用mysqld启动,一般直接退出,如下:
[1] 14789

[1]+  Exit 1                  /usr/local/mysql3306/bin/mysqld --defaults-file=/usr/local/mysql3306/my.cnf

(3)新数据目录的权限未调整,或者采用rpm包方式安装的MySQL受到SELinux的影响,同样会导致 Starting MySQL...... ERROR! 的问题,但错误日志与(1)(2)不同,本文不讨论。

回到头两种情况,由于mysql.server是调用的mysqld_safe,再调用mysqld进行启动,故最终错误日志是一样的,截取部分输出如下:

2019-12-10T10:10:03.319664+08:00 0 [Note] --secure-file-priv is set to NULL. Operations related to importing and exporting data are disabled
2019-12-10T10:10:03.319837+08:00 0 [Note] /usr/local/mysql3306/bin/mysqld (mysqld 5.7.28-log) starting as process 11147 ...
2019-12-10T10:10:03.330161+08:00 0 [Note] InnoDB: PUNCH HOLE support available
2019-12-10T10:10:03.330222+08:00 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
2019-12-10T10:10:03.330242+08:00 0 [Note] InnoDB: Uses event mutexes
2019-12-10T10:10:03.330258+08:00 0 [Note] InnoDB: GCC builtin __sync_synchronize() is used for memory barrier
2019-12-10T10:10:03.330274+08:00 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2019-12-10T10:10:03.330289+08:00 0 [Note] InnoDB: Using Linux native AIO
2019-12-10T10:10:03.330858+08:00 0 [Note] InnoDB: Number of pools: 1
2019-12-10T10:10:03.331056+08:00 0 [Note] InnoDB: Using CPU crc32 instructions
2019-12-10T10:10:03.334548+08:00 0 [Note] InnoDB: Initializing buffer pool, total size = 6G, instances = 4, chunk size = 128M
2019-12-10T10:10:04.421360+08:00 0 [Note] InnoDB: Completed initialization of buffer pool
2019-12-10T10:10:04.510523+08:00 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2019-12-10T10:10:04.535726+08:00 0 [Note] InnoDB: Opened 95 undo tablespaces
2019-12-10T10:10:04.535771+08:00 0 [Note] InnoDB: 95 undo tablespaces made active
2019-12-10T10:10:04.536161+08:00 0 [Note] InnoDB: Highest supported file format is Barracuda.
2019-12-10T10:10:04.764728+08:00 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2019-12-10T10:10:04.764878+08:00 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2019-12-10T10:10:04.788994+08:00 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2019-12-10T10:10:04.790995+08:00 0 [Note] InnoDB: 96 redo rollback segment(s) found. 96 redo rollback segment(s) are active.
2019-12-10T10:10:04.791030+08:00 0 [Note] InnoDB: 32 non-redo rollback segment(s) are active.
2019-12-10T10:10:04.791550+08:00 0 [Note] InnoDB: Waiting for purge to start
2019-12-10T10:10:04.841937+08:00 0 [Note] InnoDB: 5.7.28 started; log sequence number 2711531
2019-12-10T10:10:04.842111+08:00 0 [Note] InnoDB: Loading buffer pool(s) from /home/mysqldata/mysql3306/ib_buffer_pool
2019-12-10T10:10:04.842291+08:00 0 [Note] Plugin 'FEDERATED' is disabled.
mysqld: File '/data/mysql3306/mybinlog.000002' not found (Errcode: 2 - No such file or directory)2019-12-10T10:10:04.847771+08:00 0 [Note] InnoDB: Buffer pool(s) load completed at 191210 10:10:04

2019-12-10T10:10:04.847822+08:00 0 [ERROR] Failed to open log (file '/data/mysql3306/mybinlog.000002', errno 2)
2019-12-10T10:10:04.847848+08:00 0 [ERROR] Could not open log file
2019-12-10T10:10:04.847867+08:00 0 [ERROR] Can't init tc log
2019-12-10T10:10:04.847890+08:00 0 [ERROR] Aborting

2019-12-10T10:10:04.847915+08:00 0 [Note] Binlog end

显然,是由于未找到binlog导致启动失败,mysqld依然从老的数据目录下去寻找。但之前已在配置文件中指定log-bin的路径,那么是什么原因导致的呢?查阅官方文档:

The default location for binary log files is the data directory. You can use the --log-bin option to specify an alternative location, by adding a leading absolute path name to the base name to specify a different directory. When the server reads an entry from the binary log index file, which tracks the binary log files that have been used, it checks whether the entry contains a relative path. If it does, the relative part of the path is replaced with the absolute path set using the --log-bin option. An absolute path recorded in the binary log index file remains unchanged; in such a case, the index file must be edited manually to enable a new path or paths to be used. (In older versions of MySQL, manual intervention was required whenever relocating the binary log or relay log files.) (Bug #11745230, Bug #12133)

可知是通过index文件来追踪binlog文件,并且如果 --log-bin 选项中含有绝对路径时,将会在index文件中以绝对路径表示binlog文件,而不再是在默认的datadir下。

将如下index文件中的路径做修改

实测已能正常修改

To keep track of which binary log files have been used, mysqld also creates a binary log index file that contains the names of the binary log files. By default, this has the same base name as the binary log file, with the extension '.index'. You can change the name of the binary log index file with the --log-bin-index[=file_name] option. You should not manually edit this file while mysqld is running; doing so would confuse mysqld.

若配置文件中不指定--log-basename 或者 log-bin = /data/mysql/mybinlog,显然会避免上述问题,但会带来另外一个问题:一旦主库hostname有修改并重启后,会导致主从复制断连。

The name for the binary log index file, which contains the names of the binary log files. By default, it has the same location and base name as the value specified for the binary log files using the --log-bin option, plus the extension .index. If you do not specify --log-bin, the default binary log index file name is binlog.index. If you omit the file name and do not specify one with --log-bin, the default binary log index file name is host_name-bin.index, using the name of the host machine.

Log file names are based on the server host name if you do not specify a file name with the startup option. To retain the same log file names if you change your host name to something else, you must explicitly use options such as --log-bin=old_host_name-bin. See Section 5.1.6, “Server Command Options”. Alternatively, rename the old files to reflect your host name change. If these are binary logs, you must edit the binary log index file and fix the binary log file names there as well. (The same is true for the relay logs on a slave server.)

 

参考文档:

16.1.6.4 Binary Logging Options and Variables:log-bin

5.4.4 The Binary Log

16.1.6.4 Binary Logging Options and Variables:log-bin-index

B.4.7 Known Issues in MySQL

腾讯工程师带你深入解析 MySQL binlog

发布了47 篇原创文章 · 获赞 6 · 访问量 6万+
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章