pt-query-digest

支持四種文件分析：

1: genlog

2: slowlog

3: binlog

4: tcpdump

注意：tcpdump 如果是unix socket，則pt-query-digest分析時可能會有錯誤

終端A：

tcpdump -s 65535 -x -nn -q -tttt -i eth0 -c 10000 port 3306 > mysql.tcp.txt

終端B：

sysbench --test=oltp --num-threads=100 --db-driver=mysql --mysql-table-engine=innodb --mysql-host=172.17.0.2 --mysql-port=3306 --mysql-db=test --oltp-table-size=500000 --mysql-user=root --mysql-password=123456 run

通過sysbench 做壓測，發現對tcpdump分析會出錯，官方文檔說是unix socket 會有問題，不知道是不是這個原因

先看錯誤的

tcpdump -s 65535 -x -nn -q -tttt -i eth0 -c 10000 port 3306 > mysql.tcp.txt

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

10000 packets captured

271384 packets received by filter

261331 packets dropped by kernel 拋棄了好多包

但是分析時還是會出錯

[root@756bf8df1092 opt]# pt-query-digest --type=tcpdump mysql.tcp.txt

TCP session 172.17.0.1:47174 had errors, will save them in /tmp/pt-query-digest-errors.f0uZ2o8

Pipeline process 4 (MySQLProtocolParser) caused an error: Can't use an undefined value as an ARRAY reference at /usr/bin/pt-query-digest line 4802, <> line 3628.

Will retry pipeline process 3 (MySQLProtocolParser) 100 more times.

Pipeline process 4 (MySQLProtocolParser) caused an error: Can't use an undefined value as an ARRAY reference at /usr/bin/pt-query-digest line 4802, <> line 3693.

Will retry pipeline process 3 (MySQLProtocolParser) 99 more times.

Pipeline process 4 (MySQLProtocolParser) caused an error: Can't use an undefined value as an ARRAY reference at /usr/bin/pt-query-digest line 4802, <> line 3713.

如果沒有錯誤的話，分析如下

root@python-develpment:/data/3307/data# pt-query-digest --type tcpdump mysqltcp.txt

# 3.5s user time, 100ms system time, 35.55M rss, 99.80M vsz

# Current date: Wed Nov 1 16:07:01 2017

# Hostname: python-develpment

# Files: mysqltcp.txt

# Overall: 5.50k total, 6 unique, 420.62 QPS, 0.80x concurrency __________

# Time range: 2017-11-01 15:08:37.337813 to 15:08:50.418468

# Attribute total min max avg 95% stddev median

# ============ ======= ======= ======= ======= ======= ======= =======

# Exec time 10s 0 132ms 2ms 10ms 4ms 301us

# Rows affecte 917 0 1 0.17 0.99 0.37 0

# Query size 149.25k 6 43 27.78 42.48 12.97 28.75

# Warning coun 0 0 0 0 0 0 0

# Profile

# Rank Query ID Response time Calls R/Call V/M Item

# ==== ================== ============= ===== ====== ===== =============

# 1 0x813031B8BBC3B329 7.9226 75.6% 916 0.0086 0.00 COMMIT

# 2 0x5D51E5F01B88B79E 1.0746 10.3% 918 0.0012 0.00 ADMIN CONNECT

# 3 0xA55840651A445715 0.8524 8.1% 917 0.0009 0.01 INSERT test

# 4 0xB0710D0F90B2ABBA 0.4745 4.5% 917 0.0005 0.00 SELECT test

# MISC 0xMISC 0.1538 1.5% 1834 0.0001 0.0 <2 ITEMS>

# Query 1: 70.25 QPS, 0.61x concurrency, ID 0x813031B8BBC3B329 at byte 6746408

# This item is included in the report because it matches --limit.

# Scores: V/M = 0.00

# Time range: 2017-11-01 15:08:37.355930 to 15:08:50.395871

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 16 916

# Exec time 75 8s 2ms 132ms 9ms 16ms 6ms 7ms

# Rows affecte 0 0 0 0 0 0 0 0

# Query size 3 5.37k 6 6 6 6 0 6

# Warning coun 0 0 0 0 0 0 0 0

# String:

# Databases d

# Hosts 172.17.0.1

# Users root

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms ################################################################

# 10ms #########################

# 100ms #

# 1s

# 10s+

COMMIT\G

再看：general log的

pt-query-digest --type genlog python-develpment.log

Will retry pipeline process 2 (GeneralLogParser) 92 more times.

Pipeline process 3 (GeneralLogParser) caused an error: Use of uninitialized value $user in split at /usr/bin/pt-query-digest line 9949, <$fh> line 3082.

Will retry pipeline process 2 (GeneralLogParser) 91 more times.

# 850ms user time, 40ms system time, 35.79M rss, 100.38M vsz

# Current date: Wed Nov 1 15:21:59 2017

# Hostname: python-develpment

# Files: python-develpment.log

# Overall: 4.72k total, 119 unique, 0 QPS, 0x concurrency ________________

# Time range: 2017-10-13T07:48:54.430658Z to 2017-10-13T10:01:22.545480Z

# Attribute total min max avg 95% stddev median

# ============ ======= ======= ======= ======= ======= ======= =======

# Exec time 0 0 0 0 0 0 0

# Query size 288.15k 6 732 62.50 212.52 100.47 28.75

# Profile

# Rank Query ID Response time Calls R/Call V/M Item

# ==== ================== ============= ===== ====== ===== ===============

# 1 0x7DD5F6760F2D2EBB 0.0000 0.0% 783 0.0000 0.00 SHOW VARIABLES

# 2 0x6C099B0B73EA7633 0.0000 0.0% 258 0.0000 0.00

# 3 0xD3F15165B8904806 0.0000 0.0% 230 0.0000 0.00 SELECT

# 4 0x77B20FAC5E701D4E 0.0000 0.0% 192 0.0000 0.00 SELECT

# 5 0x5D51E5F01B88B79E 0.0000 0.0% 189 0.0000 0.00 ADMIN CONNECT

# 6 0xAA353644DE4C4CB4 0.0000 0.0% 169 0.0000 0.00 ADMIN QUIT

# 7 0xDA69864C6215FEDB 0.0000 0.0% 168 0.0000 0.00 SET

# 8 0xAD312D56C6B3D880 0.0000 0.0% 168 0.0000 0.00 SET

# 9 0x2EAFE6610FDB4B07 0.0000 0.0% 141 0.0000 0.00 SELECT

# 10 0x6C2D4BD68F056F6B 0.0000 0.0% 125 0.0000 0.00 SET

# 11 0x992F97D6C4D52DF6 0.0000 0.0% 117 0.0000 0.00 SHOW GLOBAL STATUS

# 12 0x50CE925F596A92CD 0.0000 0.0% 116 0.0000 0.00 SELECT

# 13 0xA3C958A64E1E64AB 0.0000 0.0% 109 0.0000 0.00 SHOW ENGINES

# 14 0xC727AFF0F56E1A27 0.0000 0.0% 105 0.0000 0.00 SELECT

# 15 0x6B4EFE5C463C2798 0.0000 0.0% 105 0.0000 0.00 SET

# 16 0x9F5AF5493DFF0F21 0.0000 0.0% 87 0.0000 0.00

# 17 0x329C82672D99E7C8 0.0000 0.0% 78 0.0000 0.00

# 18 0x7F563888280A36F3 0.0000 0.0% 78 0.0000 0.00

# 19 0x1AD8D6CCA3B56799 0.0000 0.0% 77 0.0000 0.00 SELECT

# 20 0x5CBA2034458B5BC9 0.0000 0.0% 73 0.0000 0.00 SHOW DATABASES

# MISC 0xMISC 0.0000 0.0% 1353 0.0000 0.0 <99 ITEMS>

# Query 1: 0 QPS, 0x concurrency, ID 0x7DD5F6760F2D2EBB at byte 481767 ___

# This item is included in the report because it matches --limit.

# Scores: V/M = 0.00

# Time range: 2017-10-13T07:48:54.478994Z to 2017-10-13T10:01:21.263089Z

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 16 783

# Exec time 0 0 0 0 0 0 0 0

# Query size 9 26.10k 26 47 34.13 46.83 6.07 28.75

# String:

# Databases test (766/97%), pcik_cente... (14/1%)

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms

# 100ms

# 1s

# 10s+

SHOW VARIABLES LIKE 'innodb_version'\G

# Query 2: 0 QPS, 0x concurrency, ID 0x6C099B0B73EA7633 at byte 483175 ___

# This item is included in the report because it matches --limit.

# Scores: V/M = 0.00

# Time range: 2017-10-13T07:48:54.769631Z to 2017-10-13T10:01:22.455701Z

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 5 258

# Exec time 0 0 0 0 0 0 0 0

# Query size 0 2.54k 8 15 10.07 9.83 0.78 9.83

# String:

# Databases test

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms

# 100ms

# 1s

# 10s+

USE `sync`\G

# Query 3: 0 QPS, 0x concurrency, ID 0xD3F15165B8904806 at byte 481196 ___

# This item is included in the report because it matches --limit.

# Scores: V/M = 0.00

# Time range: 2017-10-13T07:48:54.527457Z to 2017-10-13T10:01:21.259803Z

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 4 230

# Exec time 0 0 0 0 0 0 0 0

# Query size 1 3.82k 17 17 17 17 0 17

# String:

# Databases test (220/95%), pcik_cente... (8/3%)

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms

# 100ms

# 1s

# 10s+

# EXPLAIN /*!50100 PARTITIONS*/

SELECT @@SQL_MODE\G

# Query 4: 0 QPS, 0x concurrency, ID 0x77B20FAC5E701D4E at byte 483124 ___

# This item is included in the report because it matches --limit.

# Scores: V/M = 0.00

# Time range: 2017-10-13T07:48:54.824398Z to 2017-10-13T10:01:22.455285Z

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 4 192

# Exec time 0 0 0 0 0 0 0 0

# Query size 1 3.81k 17 27 20.33 26.08 4.37 16.81

# String:

# Databases test

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms

# 100ms

# 1s

# 10s+

# EXPLAIN /*!50100 PARTITIONS*/

SELECT CRC32('a')\G

# Query 5: 0 QPS, 0x concurrency, ID 0x5D51E5F01B88B79E at byte 480934 ___

# This item is included in the report because it matches --limit.

# Scores: V/M = 0.00

# Time range: 2017-10-13T07:48:54.430658Z to 2017-10-13T10:01:21.214034Z

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 4 189

# Exec time 0 0 0 0 0 0 0 0

# Query size 1 5.54k 30 30 30 30 0 30

# String:

# Databases test (180/95%), pcik_cente... (8/4%)

# Hosts 127.0.0.1 (147/77%), 192.168.134.187 (33/17%)... 2 more

# Users root (187/98%), r1oot (2/1%)

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms

# 100ms

# 1s

# 10s+

administrator command: Connect\G

分析一般日誌，有的會有host，user，database，有的沒有，應該是沒有記錄，或者分析不是很準

比如：

2017-10-13T07:48:54.532376Z 220 Connect [email protected] on test using TCP/IP

2017-10-13T07:48:54.532922Z 220 Query SHOW VARIABLES LIKE 'innodb\_lock_wait_timeout'

2017-10-13T07:48:54.557778Z 220 Query SET SESSION innodb_lock_wait_timeout=1

2017-10-13T07:48:54.558185Z 220 Query SHOW VARIABLES LIKE 'wait\_timeout'

2017-10-13T07:48:54.560046Z 220 Query SET SESSION wait_timeout=10000

2017-10-13T07:48:54.560274Z 220 Query SELECT @@SQL_MODE

2017-10-13T07:48:54.560419Z 220 Query SET @@SQL_QUOTE_SHOW_CREATE = 1/*!40101, @@SQL_MODE='NO_AUTO_VALUE_ON_ZERO,ONLY_FULL_GROUP_BY,STRICT_TRANS_TA

BLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'*/

2017-10-13T07:48:54.560585Z 220 Query SELECT @@server_id /*!50038 , @@hostname*/

2017-10-13T07:48:54.560768Z 220 Query SHOW VARIABLES LIKE 'wsrep_on'

2017-10-13T07:48:54.576935Z 220 Query SELECT dsn FROM `test`.`dsns` ORDER BY id

2017-10-13T07:48:54.752198Z 220 Quit

2017-10-13T07:48:54.752375Z 219 Query SHOW VARIABLES LIKE 'wsrep_on'

2017-10-13T07:48:54.754398Z 219 Query SELECT @@SERVER_ID

2017-10-13T07:48:54.757833Z 219 Query SHOW VARIABLES LIKE 'wsrep_on'

2017-10-13T07:48:54.759769Z 219 Query SELECT @@SERVER_ID

2017-10-13T07:48:54.765776Z 219 Query SHOW DATABASES LIKE 'test'

2017-10-13T07:48:54.766635Z 219 Query CREATE DATABASE IF NOT EXISTS `test` /* pt-table-checksum */

2017-10-13T07:48:54.769631Z 219 Query USE `test`

2017-10-13T07:48:54.769974Z 219 Query SHOW TABLES FROM `test` LIKE 'checksum'

2017-10-13T07:48:54.774802Z 219 Query CREATE TABLE IF NOT EXISTS `test`.`checksum` (

db CHAR(64) NOT NULL,

tbl CHAR(64) NOT NULL,

chunk INT NOT NULL,

chunk_time FLOAT NULL,

chunk_index VARCHAR(200) NULL,

lower_boundary TEXT NULL,

upper_boundary TEXT NULL,

this_crc CHAR(40) NOT NULL,

this_cnt INT NOT NULL,

master_crc CHAR(40) NULL,

master_cnt INT NULL,

ts TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,

PRIMARY KEY (db, tbl, chunk),

INDEX ts_db_tbl (ts, db, tbl)

) ENGINE=InnoDB DEFAULT CHARSET=utf8

2017-10-13T07:48:54.816241Z 219 Query SHOW GLOBAL STATUS LIKE 'Threads_running'

2017-10-13T07:48:54.819643Z 219 Query SELECT CONCAT(@@hostname, @@port)

2017-10-13T07:48:54.824398Z 219 Query SELECT CRC32('test-string')

3、再看slowlog

pt-query-digest --type slowlog python-develpment-slow.log-1504154700

順便說一下，slowlog切換：

1 set global slow_query_log=off;

2 mv /data/3307/data/python-develpment-slow.log /data/3307/data/python-develpment-slow.log-115115143

3 set global slow_query_log=on;

slowlog自己會生成

root@python-develpment:/data/3307/data# pt-query-digest --type slowlog python-develpment-slow.log

# 380ms user time, 20ms system time, 34.24M rss, 98.70M vsz

# Current date: Wed Nov 1 15:53:46 2017

# Hostname: python-develpment

# Files: python-develpment-slow.log

# Overall: 595 total, 2 unique, 3.61 QPS, 0.16x concurrency ______________

# Time range: 2017-11-01T07:44:02 to 2017-11-01T07:46:47

# Attribute total min max avg 95% stddev median

# ============ ======= ======= ======= ======= ======= ======= =======

# Exec time 26s 10ms 2s 45ms 128ms 140ms 14ms

# Lock time 183us 0 110us 0 0 5us 0

# Rows sent 0 0 0 0 0 0 0

# Rows examine 0 0 0 0 0 0 0

# Query size 3.56k 6 46 6.13 5.75 2.19 5.75

# Profile

# Rank Query ID Response time Calls R/Call V/M Item

# ==== ================== ============= ===== ====== ===== =========

# 1 0x813031B8BBC3B329 26.4473 99.9% 593 0.0446 0.44 COMMIT

# MISC 0xMISC 0.0359 0.1% 2 0.0179 0.0 <1 ITEMS>

# Query 1: 3.59 QPS, 0.16x concurrency, ID 0x813031B8BBC3B329 at byte 32559

# This item is included in the report because it matches --limit.

# Scores: V/M = 0.44

# Time range: 2017-11-01T07:44:02 to 2017-11-01T07:46:47

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 99 593

# Exec time 99 26s 10ms 2s 45ms 128ms 140ms 14ms

# Lock time 0 0 0 0 0 0 0 0

# Rows sent 0 0 0 0 0 0 0 0

# Rows examine 0 0 0 0 0 0 0 0

# Query size 97 3.47k 6 6 6 6 0 6

# String:

# Databases d

# Hosts 192.168.134.130

# Users root

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms ################################################################

# 100ms ###

# 1s #

# 10s+

COMMIT\G

4、binlog 分析

如果binlog_format = row(分析這個沒啥用了就)

statement 模式示例：

pt-query-digest --type=binlog test.sql

# 1s user time, 160ms system time, 29.43M rss, 119.05M vsz

# Current date: Wed Nov 1 16:01:59 2017

# Hostname: 756bf8df1092

# Files: test.sql

# Overall: 2.64k total, 6 unique, 10.90 QPS, 88.74Mx concurrency _________

# Time range: 2017-11-01 15:57:31 to 16:01:33

# Attribute total min max avg 95% stddev median

# ============ ======= ======= ======= ======= ======= ======= =======

# Exec time 21474836480s 0 4294967295s 8140575s 0 179865991s 0

# Query size 70.46k 1 161 18.23 44.60 17.85 5.75

# @@session.au 1 1 1 1 1 0 1

# @@session.ch 33 33 33 33 33 0 33

# @@session.co 33 33 33 33 33 0 33

# @@session.co 8 8 8 8 8 0 8

# @@session.fo 1 1 1 1 1 0 1

# @@session.lc 0 0 0 0 0 0 0

# @@session.ps 3 3 3 3 3 0 3

# @@session.sq 0 0 0 0 0 0 0

# @@session.un 1 1 1 1 1 0 1

# error code 0 0 0 0 0 0 0

# Profile

# Rank Query ID Response time Calls R/Call V/M

# ==== ================== ====================== ===== ============= =====

# 1 0xA55840651A445715 17179869180.0000 80.0% 1319 13024919.7726 39... INSERT test

# 2 0x85FFF5AA78E5FF6A 4294967300.0000 20.0% 1319 3256229.9469 39... BEGIN

# Query 1: 41.22 QPS, 536.87Mx concurrency, ID 0xA55840651A445715 at byte 539040

# This item is included in the report because it matches --limit.

# Scores: V/M = 3969610770.02

# Time range: 2017-11-01 16:01:01 to 16:01:33

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 50 1319

# Exec time 79 17179869180s 0 4294967295s 13024920s 0 227384832s 0

# Query size 79 56.01k 37 46 43.49 44.60 1.52 42.48

# error code 0 0 0 0 0 0 0 0

# String:

# Databases d

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms

# 100ms

# 1s

# 10s+ ################################################################

# Tables

# SHOW TABLE STATUS FROM `d` LIKE 'test'\G

# SHOW CREATE TABLE `d`.`test`\G

insert into test VALUES (1270,1271,'test1271')\G

# Query 2: 41.22 QPS, 134.22Mx concurrency, ID 0x85FFF5AA78E5FF6A at byte 91563

# This item is included in the report because it matches --limit.

# Scores: V/M = 3978666911.63

# Time range: 2017-11-01 16:01:01 to 16:01:33

# Attribute pct total min max avg 95% stddev median

# ============ === ======= ======= ======= ======= ======= ======= =======

# Count 50 1319

# Exec time 20 4294967300s 0 4294967295s 3256230s 0 113822029s 0

# Query size 9 6.44k 5 5 5 5 0 5

# @@session.au 100 1 1 1 1 1 0 1

# @@session.ch 100 33 33 33 33 33 0 33

# @@session.co 100 33 33 33 33 33 0 33

# @@session.co 100 8 8 8 8 8 0 8

# @@session.fo 100 1 1 1 1 1 0 1

# @@session.lc 0 0 0 0 0 0 0 0

# @@session.ps 100 3 3 3 3 3 0 3

# @@session.sq 0 0 0 0 0 0 0 0

# @@session.un 100 1 1 1 1 1 0 1

# error code 0 0 0 0 0 0 0 0

# String:

# @@session.co default

# Databases d

# Query_time distribution

# 1us

# 10us

# 100us

# 1ms

# 10ms

# 100ms

# 1s ################################################################

# 10s+ ############

BEGIN\G

總結：

1、tcpdump 分析還有欠缺，bug不斷，多嘗試

2、除了binlog，都記錄host，user，database，只是全不全的問題

3、binlog_format 必須是statement 才能分析完整，線上數據庫一般都是row ，full p_w_picpath，基本上沒撒用了

一鍵自動化博客發佈工具,用過的人都說好(頭條篇)

MHA官方文檔

centos7 systemctl 管理 mysql

mha 故障

我的友情鏈接

django 打包發佈及安裝

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結