Percona-Toolkit系列(13) --- pt-index-usage

pt-index-usage

pt-index-usage：官方地址

說明

用途：從日誌中(慢查詢日誌，查詢日誌)讀取查詢語句並分析其如何使用索引。

語法：

pt-index-usage [OPTIONS] [FILES]

主要是通過EXPLAIN來分析MySQL是如何使用索引，分析最後會給出沒有使用到的索引並給出刪除索引的語句，當然只針對當前的查詢日誌的分析結果不代表該索引就沒有用，所以要人工覈實後再進行相關刪除操作。

具體例子

分析慢查詢日誌中的查詢語句

[root@BigData ~]# pt-index-usage /var/lib/mysql/mysql.slow --host localhost --password 123

# A software update is available:
DBD::mysql::db do failed: Unknown database 'monitor' [for Statement "USE `monitor`"] at /bin/pt-index-usage line 6683, <> line 12550.
/var/lib/mysql/mysql.slow:  37% 00:50 remain
DBD::mysql::db selectall_arrayref failed: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'SELECT * FROM (SELECT * FROM MONITOR_IO WHERE HOST_ID=3 ORDER BY TIME DESC) A' at line 3 [for Statement "EXPLAIN SELECT * FROM MONITOR_IO WHERE HOST_ID=3 ORDER BY TIME DESC

SELECT * FROM (SELECT * FROM MONITOR_IO WHERE HOST_ID=3 ORDER BY TIME DESC) A"] at /bin/pt-index-usage line 4598, <> line 129297.
Use of uninitialized value $query in pattern match (m//) at /bin/pt-index-usage line 4591, <> line 130130.
Use of uninitialized value $query in pattern match (m//) at /bin/pt-index-usage line 4591, <> line 132062.
/var/lib/mysql/mysql.slow:  79% 00:15 remain

ALTER TABLE `dbmonitor`.`FLASH_INFO` DROP KEY `IP_IND`, DROP KEY `Time_IND`; -- type:non-unique
ALTER TABLE `dbmonitor`.`MARIADB_MONITOR_CONNECT` DROP KEY `IP_IND`, DROP KEY `TIME_IND`; -- type:non-unique
ALTER TABLE `dbmonitor`.`MARIADB_MONITOR_SINGLE` DROP KEY `IP_IND`, DROP KEY `TIME_IND`; -- type:non-unique
ALTER TABLE `dbmonitor`.`MARIADB_MS_SINGLE` DROP KEY `IP_IND`; -- type:non-unique
ALTER TABLE `dbmonitor`.`MARIADB_TRAFFIC_STATISTICS` DROP KEY `IP_IND`; -- type:non-unique
ALTER TABLE `dbmonitor`.`MEMORY` DROP KEY `ip_IND`; -- type:non-unique
ALTER TABLE `dbmonitor`.`MONITORIO` DROP KEY `IP_IND`; -- type:non-unique
ALTER TABLE `dbmonitor`.`MONITOR_DBLINK` DROP KEY `IP`; -- type:non-unique
ALTER TABLE `dbmonitor`.`SYSTEM_LOAD` DROP KEY `IP_IND`; -- type:non-unique

也可以將分析結果保存到數據庫中,以後分析

pt-index-usage slow.log --no-report --save-results-database percona

會自動創建索引、查詢、表和存儲使用情況的相關表，表結構如下：

CREATE TABLE IF NOT EXISTS indexes (
  db           VARCHAR(64) NOT NULL,
  tbl          VARCHAR(64) NOT NULL,
  idx          VARCHAR(64) NOT NULL,
  cnt          BIGINT UNSIGNED NOT NULL DEFAULT 0,
  PRIMARY KEY  (db, tbl, idx)
)

CREATE TABLE IF NOT EXISTS queries (
  query_id     BIGINT UNSIGNED NOT NULL,
  fingerprint  TEXT NOT NULL,
  sample       TEXT NOT NULL,
  PRIMARY KEY  (query_id)
)

CREATE TABLE IF NOT EXISTS tables (
  db           VARCHAR(64) NOT NULL,
  tbl          VARCHAR(64) NOT NULL,
  cnt          BIGINT UNSIGNED NOT NULL DEFAULT 0,
  PRIMARY KEY  (db, tbl)
)

CREATE TABLE IF NOT EXISTS index_usage (
  query_id      BIGINT UNSIGNED NOT NULL,
  db            VARCHAR(64) NOT NULL,
  tbl           VARCHAR(64) NOT NULL,
  idx           VARCHAR(64) NOT NULL,
  cnt           BIGINT UNSIGNED NOT NULL DEFAULT 1,
  UNIQUE INDEX  (query_id, db, tbl, idx)
)

CREATE TABLE IF NOT EXISTS index_alternatives (
  query_id      BIGINT UNSIGNED NOT NULL, -- This query used
  db            VARCHAR(64) NOT NULL,     -- this index, but...
  tbl           VARCHAR(64) NOT NULL,     --
  idx           VARCHAR(64) NOT NULL,     --
  alt_idx       VARCHAR(64) NOT NULL,     -- was an alternative
  cnt           BIGINT UNSIGNED NOT NULL DEFAULT 1,
  UNIQUE INDEX  (query_id, db, tbl, idx, alt_idx),
  INDEX         (db, tbl, idx),
  INDEX         (db, tbl, alt_idx)
)

常用的查詢語句：

1.哪些查詢有時使用不同的索引，以及每個索引選擇的時間比例是多少?

SELECT iu.query_id, CONCAT_WS('.', iu.db, iu.tbl, iu.idx) AS idx,
   variations, iu.cnt, iu.cnt / total_cnt * 100 AS pct
FROM index_usage AS iu
   INNER JOIN (
      SELECT query_id, db, tbl, SUM(cnt) AS total_cnt,
        COUNT(*) AS variations
      FROM index_usage
      GROUP BY query_id, db, tbl
      HAVING COUNT(*) > 1
   ) AS qv USING(query_id, db, tbl);

2.哪些索引有很多備選項，即被選擇而不是其他索引，以及用於哪些查詢?

SELECT CONCAT_WS('.', db, tbl, idx) AS idx_chosen,
   GROUP_CONCAT(DISTINCT alt_idx) AS alternatives,
   GROUP_CONCAT(DISTINCT query_id) AS queries, SUM(cnt) AS cnt
FROM index_alternatives
GROUP BY db, tbl, idx
HAVING COUNT(*) > 1;

3.哪些索引被認爲是其他索引的替代，以及哪些查詢的替代?

SELECT CONCAT_WS('.', db, tbl, alt_idx) AS idx_considered,
   GROUP_CONCAT(DISTINCT idx) AS alternative_to,
   GROUP_CONCAT(DISTINCT query_id) AS queries, SUM(cnt) AS cnt
FROM index_alternatives
GROUP BY db, tbl, alt_idx
HAVING COUNT(*) > 1;

4.哪些索引時從來沒有使用過的?

SELECT CONCAT_WS('.', i.db, i.tbl, i.idx) AS idx,
   alt.alternative_to, alt.queries, alt.cnt
FROM indexes AS i
   INNER JOIN (
      SELECT db, tbl, alt_idx, GROUP_CONCAT(DISTINCT idx) AS alternative_to,
         GROUP_CONCAT(DISTINCT query_id) AS queries, SUM(cnt) AS cnt
      FROM index_alternatives
      GROUP BY db, tbl, alt_idx
      HAVING COUNT(*) > 1
   ) AS alt ON i.db = alt.db AND i.tbl = alt.tbl
     AND i.idx = alt.alt_idx
WHERE i.cnt = 0;

5.給定一個表，有多少查詢使用了哪些索引，有多少不同的指紋?有替代品嗎?哪些索引沒有使用?

SELECT i.idx, iu.usage_cnt, iu.usage_total,
   ia.alt_cnt, ia.alt_total
FROM indexes AS i
   LEFT OUTER JOIN (
      SELECT db, tbl, idx, COUNT(*) AS usage_cnt,
         SUM(cnt) AS usage_total, GROUP_CONCAT(query_id) AS used_by
      FROM index_usage
      GROUP BY db, tbl, idx
   ) AS iu ON i.db=iu.db AND i.tbl=iu.tbl AND i.idx = iu.idx
   LEFT OUTER JOIN (
      SELECT db, tbl, idx, COUNT(*) AS alt_cnt,
         SUM(cnt) AS alt_total,
         GROUP_CONCAT(query_id) AS alt_queries
      FROM index_alternatives
      GROUP BY db, tbl, idx
   ) AS ia ON i.db=ia.db AND i.tbl=ia.tbl AND i.idx = ia.idx;

6.給定表上的哪些索引對於至少一個查詢是至關重要的(沒有其他選擇)?

SELECT i.db, i.tbl, i.idx, no_alt.queries
FROM indexes AS i
   INNER JOIN (
      SELECT iu.db, iu.tbl, iu.idx,
         GROUP_CONCAT(iu.query_id) AS queries
      FROM index_usage AS iu
         LEFT OUTER JOIN index_alternatives AS ia
            USING(db, tbl, idx)
      WHERE ia.db IS NULL
      GROUP BY iu.db, iu.tbl, iu.idx
   ) AS no_alt ON no_alt.db = i.db AND no_alt.tbl = i.tbl
      AND no_alt.idx = i.idx
ORDER BY i.db, i.tbl, i.idx, no_alt.queries;