使用Explain完成SQL性能優化從小白 → 大白

/ 前言 /

對於服務端開發人員來說性能非常重要, 接口的響應時間也是關係到用戶體驗的大事, 所以SQL的執行時長就至關重要了, 我們通過SQL語句結構優化和添加索引的方式來壓縮SQL的執行時長, 但是優化一條SQL我們首先需要知道這條SQL的問題在哪, 你需要一個好的工具Explain

/ 1 / 索引信息

我們來看下索引所在的表結構

CREATE TABLE "user" (
  "id" int(11) NOT NULL AUTO_INCREMENT,
  "name" varchar(20) NOT NULL,
  "age" int(3) NOT NULL,
  "sex" int(1) NOT NULL COMMENT '0:女, 1:男',
  "marital_status" int(1) DEFAULT NULL COMMENT '0:未婚, 1:已婚, 2:離婚, 3:再婚',
  "create_date" datetime DEFAULT NULL,
  PRIMARY KEY ("id"),
  KEY "marital_status_index" ("marital_status")
) ENGINE=InnoDB AUTO_INCREMENT=149 DEFAULT CHARSET=utf8;

我們可以看到這個表中只有一個額外的索引marital_status, 但是爲何我不給age和sex加索引呢?

和年齡相關的業務呢通常情況下都是範圍查詢, 範圍查詢會導致索引失效, 所以也就沒有必要爲age建立索引, 而sex則是因爲其只有倆個屬性, 分別是0和1, 建立索引也沒有意義

/ 2 / Explain函數

在MySQL中想要排查索引是否失效我們需要一個特殊的函數explain, 我們來看下它的作用

mysql> explain select age from user where marital_status = 1;
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys     | key               | key_len | ref   | rows | filtered | Extra       |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
|  1 | SIMPLE      | user  | NULL       | ref  | marital_age_index | marital_age_index | 4       | const |   29 |   100.00 | Using index |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
1 row in set (0.00 sec)
mysql>

我們來解釋下每個列所代表的的含義

2 . 1 select_type ( 當前索引語句類型 )

SIMPLE : 最簡單的select, 不包括子查詢和使用union函數
PRIMARY : 如果使用了子查詢或者union, 那麼這個就代表當前使用到的索引是子查詢的外層或者union的外層
UNION : union內層的查詢, 但是倆個select中沒有依賴關係
DEPENDENT UNION : union內層的查詢, 倆個select中有依賴關係
UNIOIN RESULT : union查詢的結果, 沒有id值
SUBQUERY : select語句中的子查詢, 不依賴外部select
DEPENDENT SUBQUERY : select語句中的子查詢, 依賴外部select
DERIVED : 子查詢位於FROM中的查詢語句

2 . 2 table ( 當前索引所在表名 )

2 . 3 partitions( 使用到的分區 )

2 . 4 `type` ( 查詢的類型/級別 )

級別順序

system > const > eq_ref > ref > range > index > all

system : 表中只有一條數據, 類似於系統表

const : 一次命中, 基本發生在where後面是主鍵的條件語句中

mysql> explain select * from user where id = 35;
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type  | possible_keys | key     | key_len | ref   | rows | filtered | Extra |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | user  | NULL       | const | PRIMARY       | PRIMARY | 4       | const |    1 |   100.00 | NULL  |
+----+-------------+-------+------------+-------+---------------+---------+---------+-------+------+----------+-------+
1 row in set (0.00 sec)

eq_ref : 唯一索引纔可能達到這個級別, 當在多表連接時使用唯一索引(primary key|unique key)作爲連接條件時纔會觸發
ref : 最常見的級別之一,

range : 最常見的級別之一, 檢索指定範圍的行, 比如你在where中使用了between或者in函數

mysql> explain select * from user where age between 20 and 21;
+----+-------------+-------+------------+-------+---------------+-----------+---------+------+------+----------+-----------------------+
| id | select_type | table | partitions | type  | possible_keys | key       | key_len | ref  | rows | filtered | Extra                 |
+----+-------------+-------+------------+-------+---------------+-----------+---------+------+------+----------+-----------------------+
|  1 | SIMPLE      | user  | NULL       | range | age_index     | age_index | 4       | NULL |    8 |   100.00 | Using index condition |
+----+-------------+-------+------------+-------+---------------+-----------+---------+------+------+----------+-----------------------+
1 row in set (0.00 sec)

但是要注意, 如果你在已經有索引的字段上使用between或者in會導致索引失效觸發全表掃描

mysql> explain select * from user where marital_status between 0 and 1;
+----+-------------+-------+------------+------+----------------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys        | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+----------------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | user  | NULL       | ALL  | marital_status_index | NULL | NULL    | NULL |  137 |    45.99 | Using where |
+----+-------------+-------+------------+------+----------------------+------+---------+------+------+----------+-------------+
1 row in set (0.00 sec)

index : 全表掃描, 但掃描的是文件, 一般存在於沒有條件語句的查詢中, 且只查詢一列

mysql> explain select age from user;
+----+-------------+-------+------------+-------+---------------+-----------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type  | possible_keys | key       | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+-------+---------------+-----------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | user  | NULL       | index | NULL          | age_index | 4       | NULL |  137 |   100.00 | Using index |
+----+-------------+-------+------------+-------+---------------+-----------+---------+------+------+----------+-------------+
1 row in set (0.00 sec)

ALL : 全表掃描, 最差的級別,

2 . 5 possible_keys( 可能使用到的索引列表 )

例如我們表中除了marital_age_index一個索引外還存在另一個marital_status_index複合索引, 在符合條件的情況下possible_keys中會列出marital_status_index,marital_age_index

mysql> explain select * from user where marital_status =1 ;
+----+-------------+-------+------------+------+----------------------------------------+----------------------+---------+-------+------+----------+-------+
| id | select_type | table | partitions | type | possible_keys                          | key                  | key_len | ref   | rows | filtered | Extra |
+----+-------------+-------+------------+------+----------------------------------------+----------------------+---------+-------+------+----------+-------+
|  1 | SIMPLE      | user  | NULL       | ref  | marital_status_index,marital_age_index | marital_status_index | 5       | const |   29 |   100.00 | NULL  |
+----+-------------+-------+------------+------+----------------------------------------+----------------------+---------+-------+------+----------+-------+
1 row in set (0.00 sec)

上面說的符合條件指的是符合索引的最左原則, 意思就是隻有我用到複合索引中最左邊的列或者全部列該索引纔會生效, 詳細的解讀在下面

2 . 6 `key`( 查詢中實際使用到的索引 )

2 . 7 key_len( 索引的字節數 )

我們知道int類型是4bytes, 如果在數據庫中設置的是允許爲null的話, 那麼就會佔用5bytes, 這也就是爲什麼我上面代碼中條件語句是id的話`key_len = 4`, 是marital_status的話就是`key_len = 5`

2 . 8 ref( 索引使用的列或常數 )

2 . 9 `rows` ( 查詢的行數,越小越好 )

2 . 10 filtered( 結果行數佔rows行數的百分比 )

2 . 11 `Extra`

Using filesort

當MySQL無法使用索引完成排序時就會使用filesort來完成排序

mysql> explain select age from user where sex = 1 order by age;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-----------------------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra                       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-----------------------------+
|  1 | SIMPLE      | user  | NULL       | ALL  | NULL          | NULL | NULL    | NULL |  137 |    10.00 | Using where; Using filesort |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-----------------------------+
1 row in set (0.00 sec)

感興趣的朋友可以看一下MySQL官方對於filesort使用的介紹, 這裏就不多敘述了
MySQL優化排序官方文檔

Using index

當使用覆蓋索引會觸發, 意思就是當查詢中只有索引列時, 此時不必讀取整個數據行, 直接根據索引獲取所需數據即可

mysql> explain select age from user where marital_status = 1;
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys     | key               | key_len | ref   | rows | filtered | Extra       |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
|  1 | SIMPLE      | user  | NULL       | ref  | marital_age_index | marital_age_index | 4       | const |   29 |   100.00 | Using index |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
1 row in set (0.00 sec)

Using index; Using where同時出現的情況會在下面詳細講解↓

Using index condition()

Index Condition Pushdown(ICP), MySQL5.6中新加入的特性, 直接在存儲引擎中使用索引對數據進行過濾, 可以減少存儲引擎必須訪問基表的次數以及MySQL服務器必須訪問存儲引擎的次數
感興趣的朋友可以看下MySQL官方介紹索引條件下推優化

mysql> explain select marital_status from user where marital_status = 1 order by sex;
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+---------------------------------------+
| id | select_type | table | partitions | type | possible_keys     | key               | key_len | ref   | rows | filtered | Extra                                 |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+---------------------------------------+
|  1 | SIMPLE      | user  | NULL       | ref  | marital_age_index | marital_age_index | 4       | const |   29 |   100.00 | Using index condition; Using filesort |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+---------------------------------------+
1 row in set (0.00 sec)

Using where

當查詢語句中使用了where進行過濾時會觸發Using where, 和索引無關

  mysql> explain select sex from user where sex = 1;
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
  | id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
  |  1 | SIMPLE      | user  | NULL       | ALL  | NULL          | NULL | NULL    | NULL |  137 |    10.00 | Using where |
  +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
  1 row in set (0.00 sec)

Using join buffer

表連接頻繁的情況下將早期連表中的部分行數據存入緩存期, 然後使用緩衝區中的行來執行與當前表的聯接
Using MRR

當表較大且未存儲在存儲引擎的高速緩存中時，在輔助索引上使用範圍掃描來讀取行會導致對錶的許多隨機磁盤訪問。通過磁盤掃描多範圍讀取（MRR）優化，MySQL嘗試通過首先僅掃描索引並收集相關行的鍵來減少用於範圍掃描的隨機磁盤訪問次數。然後對鍵進行排序，最後使用主鍵的順序從基表中檢索行。磁盤掃描MRR的動機是減少隨機磁盤訪問的次數，而是對基表數據進行更順序的掃描
官方介紹 : 多範圍讀取優化

Using temporary

當查詢中包含了group by和order by時會觸發Using temporary, 觸發後MySQL會生成一個臨時表來存儲此次查詢的結果, 然後在臨時表中進行排序或分組

mysql> explain select marital_status from user where marital_status between 1 and 2 group by age;
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------------------------------------------+
| id | select_type | table | partitions | type  | possible_keys     | key               | key_len | ref  | rows | filtered | Extra                                                     |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------------------------------------------+
|  1 | SIMPLE      | user  | NULL       | range | marital_age_index | marital_age_index | 4       | NULL |   63 |   100.00 | Using where; Using index; Using temporary; Using filesort |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------------------------------------------+
1 row in set (0.00 sec)

類型太多我們就不一一列舉了, 感興趣的朋友可以看一下官方文檔EXPLAIN輸出格式

/ 3 / 最左原則與Using index; Using where

其實最左原則本不應該寫到Explain的博文中的, 但是在Explain的解析下你會發現不一樣的最左原則
我們之前創建過一個複合索引KEY "marital_age_index" ("marital_status","age"), 還記得我上面提到過的最左原則吧, 我們來看下什麼時候會出現Using index, 什麼時候出現Using where, 什麼時候又會出現Using where;Using index

我們對照着來看, 先看第一組

# 1. 沒有order by
mysql> explain select age from user where marital_status = 1;
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys     | key               | key_len | ref   | rows | filtered | Extra       |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
|  1 | SIMPLE      | user  | NULL       | ref  | marital_age_index | marital_age_index | 4       | const |   29 |   100.00 | Using index |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
1 row in set (0.00 sec)

# 2. 有order by
mysql> explain select age from user where marital_status = 1 order by age;
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+--------------------------+
| id | select_type | table | partitions | type | possible_keys     | key               | key_len | ref   | rows | filtered | Extra                    |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+--------------------------+
|  1 | SIMPLE      | user  | NULL       | ref  | marital_age_index | marital_age_index | 4       | const |   29 |   100.00 | Using where; Using index |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+--------------------------+
1 row in set (0.00 sec)

我們看到此時的Extra分別是是Using index和Using where; Using index

在第二句SQL中我們加了一個order by age結果就變成了Using where; Using index, 這是因爲order by也會用到索引, 而我們order by的對象是複合索引中的另一列, 當我們在order by或者group by中加入了索引的另一列, 此時仍然會觸發複合索引, 然後使用索引讀取數據

我們再來看第二組SQL的執行結果

# 1. select marital_status
mysql> explain select marital_status from user where age = 1;
+----+-------------+-------+------------+-------+---------------+-------------------+---------+------+------+----------+--------------------------+
| id | select_type | table | partitions | type  | possible_keys | key               | key_len | ref  | rows | filtered | Extra                    |
+----+-------------+-------+------------+-------+---------------+-------------------+---------+------+------+----------+--------------------------+
|  1 | SIMPLE      | user  | NULL       | index | NULL          | marital_age_index | 8       | NULL |  137 |    10.00 | Using where; Using index |
+----+-------------+-------+------------+-------+---------------+-------------------+---------+------+------+----------+--------------------------+
1 row in set (0.00 sec)

# 2. select *
mysql> explain select * from user where age = 1;
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key  | key_len | ref  | rows | filtered | Extra       |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
|  1 | SIMPLE      | user  | NULL       | ALL  | NULL          | NULL | NULL    | NULL |  137 |    10.00 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+-------------+
1 row in set (0.00 sec)

Extra的值分別是Using where; Using index和Using where

這就是最左原則中的例外情況, 雖然條件語句沒有符合最左原則的規範, 但是我們在select後面添加了索引中的另一個列, 此時MySQL檢測到了之後它就會先去根據這倆個列去找到對應的索引鍵值, 然後使用索引讀取數據

我們來看第三組SQL的執行結果

# 1. where age
mysql> explain select age from user where age = 1;
+----+-------------+-------+------------+-------+---------------+-------------------+---------+------+------+----------+--------------------------+
| id | select_type | table | partitions | type  | possible_keys | key               | key_len | ref  | rows | filtered | Extra                    |
+----+-------------+-------+------------+-------+---------------+-------------------+---------+------+------+----------+--------------------------+
|  1 | SIMPLE      | user  | NULL       | index | NULL          | marital_age_index | 8       | NULL |  137 |    10.00 | Using where; Using index |
+----+-------------+-------+------------+-------+---------------+-------------------+---------+------+------+----------+--------------------------+
1 row in set (0.00 sec)

# 2. where marital_status
mysql> explain select age from user where marital_status = 1 ;
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys     | key               | key_len | ref   | rows | filtered | Extra       |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
|  1 | SIMPLE      | user  | NULL       | ref  | marital_age_index | marital_age_index | 4       | const |   29 |   100.00 | Using index |
+----+-------------+-------+------------+------+-------------------+-------------------+---------+-------+------+----------+-------------+
1 row in set (0.00 sec)

這組中第一句我們在select中和where中同時使用到了複合索引中的age列, 我們任然可以使用該複合索引, 只要我們滿足select中及where中必須都存在複合索引的另一列且必須是複合索引中的列

我們得到的結論是當我們的SQL語句不符合最左原則時我們任然可以使用該複合索引, 但是要符合以下幾點

必須在select後指定索引中符合最左原則的另一列
select中及where中必須都存在複合索引的另一列且必須是複合索引中的列

他們三個性能的排序關係是

Using index > Using index; Using where > Using where

參考文檔

EXPLAIN官方文檔

使用Explain完成SQL性能優化從小白 → 大白

/ 前言 /

/ 1 / 索引信息

/ 2 / Explain函數

2 . 1 select_type ( 當前索引語句類型 )

2 . 2 table ( 當前索引所在表名 )

2 . 3 partitions( 使用到的分區 )

2 . 4 `type` ( 查詢的類型/級別 )

2 . 5 possible_keys( 可能使用到的索引列表 )

2 . 6 `key`( 查詢中實際使用到的索引 )

2 . 7 key_len( 索引的字節數 )

2 . 8 ref( 索引使用的列或常數 )

2 . 9 `rows` ( 查詢的行數,越小越好 )

2 . 10 filtered( 結果行數佔rows行數的百分比 )

2 . 11 `Extra`

Using filesort

Using index

Using index condition()

Using where

Using join buffer

Using MRR

Using temporary

/ 3 / 最左原則與Using index; Using where

認知提升的方法

螞蟻面試：Springcloud核心組件的底層原理，你知道多少？

C#開源的兩款功能強大的錄屏神器

使用MarkDown來寫一份漂亮簡約的簡歷

Typora中的emoji圖標標籤

解決Go無法下載依賴問題 i/o timeout

解決go run: cannot run non-main package異常

解決Helm Chart異常Error: a release named tomcat already exists

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結

使用Explain完成SQL性能優化 從小白 → 大白

/ 前言 /

/ 1 / 索引信息

/ 2 / Explain函數

2 . 1 select_type ( 當前索引語句類型 )

2 . 2 table ( 當前索引所在表名 )

2 . 3 partitions( 使用到的分區 )

2 . 4 type ( 查詢的類型/級別 )

2 . 5 possible_keys( 可能使用到的索引列表 )

2 . 6 key( 查詢中實際使用到的索引 )

2 . 7 key_len( 索引的字節數 )

2 . 8 ref( 索引使用的列或常數 )

2 . 9 rows ( 查詢的行數,越小越好 )

2 . 10 filtered( 結果行數佔rows行數的百分比 )

2 . 11 Extra

Using filesort

Using index

Using index condition()

Using where

Using join buffer

Using MRR

Using temporary

/ 3 / 最左原則與Using index; Using where

使用Explain完成SQL性能優化從小白 → 大白

2 . 4 `type` ( 查詢的類型/級別 )

2 . 6 `key`( 查詢中實際使用到的索引 )

2 . 9 `rows` ( 查詢的行數,越小越好 )

2 . 11 `Extra`