提高MySQL 導入導出測試

MySQL導出的SQL語句在導入時有可能會非常非常慢,在處理百萬級數據的時候,可能導入要花幾小時。在導出時合理使用幾個參數,可以大大加快導 入的速度。

-e 使用包括幾個VALUES列表的多行INSERT語法;
--max_allowed_packet=XXX 客戶端/服務器之間通信的緩存區的最大大小;
--net_buffer_length=XXX TCP/IP和套接字通信緩衝區大小,創建長度達net_buffer_length的行。

注意:max_allowed_packet 和 net_buffer_length 不能比目標數據庫的設定數值 大,否則可能出錯。

首先確定目標數據庫的參數值

mysql> show variables like 'max_allowed_packet';
mysql> show variables like 'net_buffer_length';
根據參數值書寫 mysqldump 命令,如:

# mysqldump -uroot -p dbhost -e --max_allowed_packet=16777216 --net_buffer_length=16384 > dbhost.sql
OK,現在速度就很快了,主要注意的是導入和導出端的 max_allowed_packet 和 net_buffer_length 這2個參數值設定,弄大點就OK了

其實,最快的方法,是直接COPY數據庫目錄,不過記得先停止 MySQL 服務。


測試導出
bblaishu數據庫大小740M
mysql> show variables like 'net_buffer_length';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| net_buffer_length | 16384 |
+-------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'max_allowed_packet';
+--------------------+---------+
| Variable_name      | Value   |
+--------------------+---------+
| max_allowed_packet | 1047552 |
+--------------------+---------+
1 row in set (0.00 sec)


time mysqldump -uroot -p`cat /data/save/mysql_root` -T/data/test/test bblaishu -e --max_allowed_packet=1047552 --net_buffer_length=16384

real    0m14.183s
user    0m0.005s
sys     0m0.003s


mysql> show variables like 'net_buffer_length';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| net_buffer_length | 65536 |
+-------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'max_allowed_packet';
+--------------------+---------+
| Variable_name      | Value   |
+--------------------+---------+
| max_allowed_packet | 2096128 |
+--------------------+---------+
1 row in set (0.00 sec)

time mysqldump -uroot -p`cat /data/save/mysql_root` -T/data/test/test bblaishu -e --max_allowed_packet=2096128 --net_buffer_length=65536

real    1m8.397s
user    0m0.007s
sys     0m0.008s


mysql> show variables like 'max_allowed_packet';
+--------------------+---------+
| Variable_name      | Value   |
+--------------------+---------+
| max_allowed_packet | 2096128 |
+--------------------+---------+
1 row in set (0.01 sec)

mysql> show variables like 'net_buffer_length';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| net_buffer_length | 16384 |
+-------------------+-------+
1 row in set (0.00 sec)
time mysqldump -uroot -p`cat /data/save/mysql_root` -T/data/test/test bblaishu -e --max_allowed_packet=2096128 --net_buffer_length=16384

real    0m12.997s
user    0m0.004s
sys     0m0.007s

結論:在將數據庫導出的時候net_buffer_length默認就好,max_allowed_packet建議設置大一點,測試設置2M比較合適


測試導入

mysql> show variables like 'net_buffer_length';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| net_buffer_length | 16384 |
+-------------------+-------+
1 row in set (0.00 sec)

mysql> show variables like 'max_allowed_packet';
+--------------------+---------+
| Variable_name      | Value   |
+--------------------+---------+
| max_allowed_packet | 1047552 |
+--------------------+---------+
1 row in set (0.00 sec)

time mysqlimport --local -l -uroot -p`cat /data/save/mysql_root` bblaishu `find ./ -name "*.txt"`

real    1m31.587s
user    0m0.115s
sys     0m0.409s

 

mysql> show variables like 'max_allowed_packet';
+--------------------+---------+
| Variable_name      | Value   |
+--------------------+---------+
| max_allowed_packet | 2096128 |
+--------------------+---------+
1 row in set (0.00 sec)

mysql> show variables like 'net_buffer_length';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| net_buffer_length | 65536 |
+-------------------+-------+
1 row in set (0.00 sec)
time mysqlimport --local -l -uroot -p`cat /data/save/mysql_root` bblaishu `find ./ -name "*.txt"`

real    1m31.524s
user    0m0.122s
sys     0m0.355s


mysql> show variables like 'max_allowed_packet';
+--------------------+---------+
| Variable_name      | Value   |
+--------------------+---------+
| max_allowed_packet | 2096128 |
+--------------------+---------+
1 row in set (0.00 sec)

mysql> show variables like 'net_buffer_length';
+-------------------+-------+
| Variable_name     | Value |
+-------------------+-------+
| net_buffer_length | 16384 |
+-------------------+-------+
1 row in set (0.00 sec)

time mysqlimport --local -l -uroot -p`cat /data/save/mysql_root` bblaishu `find ./ -name "*.txt"`
real    1m32.959s
user    0m0.123s
sys     0m0.429s

結論:在導入的時候同樣net_buffer_length默認就好,max_allowed_packet建議設置爲2M

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章