MySQL中怎樣快速找出超長索引

大家好，我是知數堂SQL 優化班老師網名：騎龜的兔子

需求：

想要查找哪些索引太長了，這個SQL在5.7下跑的特別慢，8.0則挺快的，幫看下有啥優化方案沒

具體SQL 和執行計劃如下：

SELECT c.TABLE_SCHEMA AS DB,  c .TABLE_NAME AS TBL, c.COLUMN_NAME AS COL, c.CHARACTER_OCTET_LENGTH AS COL_LEN_BYTES,  s.INDEX_NAME,  s.SUB_PART * CHARACTER_OCTET_LENGTH/CHARACTER_MAXIMUM_LENGTH AS SUB_PART_LENFROM information_schema.COLUMNS c INNER JOIN information_schema.STATISTICS s USING(TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME)INNER JOIN information_schema.TABLES t USING(TABLE_SCHEMA, TABLE_NAME)  WHERE c.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test')AND c.DATA_TYPE IN ("varchar", "char", "text", "blob") AND ((CHARACTER_OCTET_LENGTH > 50 and SUB_PART is null) orSUB_PART * CHARACTER_OCTET_LENGTH/CHARACTER_MAXIMUM_LENGTH > 50)AND t.TABLE_ROWS > 10000ORDER BY COL_LEN_BYTES DESC;
執行計劃
*************************** 1. row ***************************           id: 1  select_type: SIMPLE        table: c   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_frm_only; Scanned all databases; Using temporary; Using filesort*************************** 2. row ***************************           id: 1  select_type: SIMPLE        table: s   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_frm_only; Scanned all databases; Using join buffer (Block Nested Loop)*************************** 3. row ***************************           id: 1  select_type: SIMPLE        table: t   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_full_table; Scanned all databases; Using join buffer (Block Nested Loop)3 rows in set, 1 warning (0.01 sec)

select count(*) from information_schema.tables;+----------+| count(*) |+----------+|    33600 |+----------+
select count(*) from information_schema.COLUMNS;+----------+| count(*) |+----------+|   342967 |+----------+select count(*) from information_schema.STATISTICS;+----------+| count(*) |+----------+|   135167 |+----------+

上面的SQL 運行450+ s 也運行不出來，最後kill掉了。

我們初步分析一下，從執行計劃中可以看出三個表都是ALL 所以很慢

那添加索引不就行了嗎，因爲是系統表，所以不能隨便添加！

那該怎麼辦？想到了AUTOKEY 就是臨時索引，那思路就是改寫SQL

達到生成臨時索引，最終達到優化效果

改寫的SQL 如下

SELECT c.TABLE_SCHEMA AS DB, c.TABLE_NAME AS TBL, c.COLUMN_NAME AS COL, c.CHARACTER_OCTET_LENGTH AS COL_LEN_BYTES, s.INDEX_NAME,s.SUB_PART * c.CHARACTER_OCTET_LENGTH/c.CHARACTER_MAXIMUM_LENGTH AS SUB_PART_LENFROM ( select c.TABLE_SCHEMA, c.TABLE_NAME, c.COLUMN_NAME ,c.CHARACTER_OCTET_LENGTH ,c.CHARACTER_MAXIMUM_LENGTH , c.DATA_TYPEfrom information_schema.COLUMNS c WHERE c.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test') AND c.DATA_TYPE IN ("varchar", "char", "text", "blob") limit 10000000000) c INNER JOIN (select s.TABLE_SCHEMA, s.TABLE_NAME, s.COLUMN_NAME ,s.SUB_PART,s.INDEX_NAMEfrom information_schema.STATISTICS s WHERE s.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test')limit 10000000000 
)s USING(TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME)INNER JOIN information_schema.TABLES t USING(TABLE_SCHEMA, TABLE_NAME) WHERE c.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test') AND c.DATA_TYPE IN ("varchar", "char", "text", "blob") AND ((c.CHARACTER_OCTET_LENGTH > 50 and s.SUB_PART is null) ors.SUB_PART * c.CHARACTER_OCTET_LENGTH/c.CHARACTER_MAXIMUM_LENGTH > 50)AND t.TABLE_ROWS > 10000ORDER BY COL_LEN_BYTES DESC;

*************************** 1. row ***************************           id: 1  select_type: PRIMARY        table: t   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_full_table; Scanned all databases; Using temporary; Using filesort*************************** 2. row ***************************           id: 1  select_type: PRIMARY        table: <derived2>   partitions: NULL         type: refpossible_keys: <auto_key0>          key: <auto_key0>      key_len: 388          ref: information_schema.t.TABLE_SCHEMA,information_schema.t.TABLE_NAME         rows: 2     filtered: 50.00        Extra: Using where*************************** 3. row ***************************           id: 1  select_type: PRIMARY        table: <derived3>   partitions: NULL         type: refpossible_keys: <auto_key0>          key: <auto_key0>      key_len: 582          ref: information_schema.t.TABLE_SCHEMA,information_schema.t.TABLE_NAME,c.COLUMN_NAME         rows: 2     filtered: 100.00        Extra: Using where*************************** 4. row ***************************           id: 3  select_type: DERIVED        table: s   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_frm_only; Scanned all databases*************************** 5. row ***************************           id: 2  select_type: DERIVED        table: c   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_frm_only; Scanned all databases5 rows in set, 1 warning (0.01 sec)

結果來了 2463 rows in set, 417 warnings (23.39 sec)

但是經過幾次運行之後有時候是40多秒有時候甚至達到了166s 非常不穩定！

那分析下上面這個SQL的問題在哪裏？

問題就是生成的AUTO KEY的量相對來說非常大！因爲沒有進行任何過濾

那現在的思路就是對生成的AUTOKEY的量進行減少

我們通過相對小的表TABLES 表生成autokey 之後 STATISTICS ,COLUMNS

表分別跟 TABLES 表進行JOIN 然後減少數據量達到減少生成AUOKEY 的量減少達到優化目的，具體的方法如下


select count(1) from (select s.TABLE_SCHEMA, s.TABLE_NAME, s.COLUMN_NAME ,s.SUB_PART,s.INDEX_NAMEfrom information_schema.STATISTICS s WHERE s.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test') )s straight_join (select t.TABLE_SCHEMA, t.TABLE_NAMEfrom  information_schema.TABLES t  WHERE t.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test')  AND t.TABLE_ROWS > 10000limit 10000000000) t on s.TABLE_SCHEMA=t.TABLE_SCHEMA and  s.TABLE_NAME =t.TABLE_NAME
*************************** 1. row ***************************           id: 1  select_type: PRIMARY        table: s   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_frm_only; Scanned all databases*************************** 2. row ***************************           id: 1  select_type: PRIMARY        table: <derived3>   partitions: NULL         type: refpossible_keys: <auto_key0>          key: <auto_key0>      key_len: 388          ref: information_schema.s.TABLE_SCHEMA,information_schema.s.TABLE_NAME         rows: 2     filtered: 100.00        Extra: Using index*************************** 3. row ***************************           id: 3  select_type: DERIVED        table: t   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_full_table; Scanned all databases3 rows in set, 1 warning (0.00 sec)

+----------+| count(1) |+----------+|     7478 |+----------+1 row in set, 40 warnings (7.52 sec)

select count(1) from ( select c.TABLE_SCHEMA, c.TABLE_NAME, c.COLUMN_NAME ,c.CHARACTER_OCTET_LENGTH ,c.CHARACTER_MAXIMUM_LENGTH , c.DATA_TYPEfrom information_schema.COLUMNS c WHERE c.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test') AND c.DATA_TYPE IN ("varchar", "char", "text", "blob")  ) c straight_join (select t.TABLE_SCHEMA, t.TABLE_NAMEfrom  information_schema.TABLES t  WHERE t.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test')  AND t.TABLE_ROWS > 10000limit 10000000000) t on c.TABLE_SCHEMA=t.TABLE_SCHEMA and  c.TABLE_NAME =t.TABLE_NAME
*************************** 1. row ***************************           id: 1  select_type: PRIMARY        table: c   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_frm_only; Scanned all databases*************************** 2. row ***************************           id: 1  select_type: PRIMARY        table: <derived3>   partitions: NULL         type: refpossible_keys: <auto_key0>          key: <auto_key0>      key_len: 388          ref: information_schema.c.TABLE_SCHEMA,information_schema.c.TABLE_NAME         rows: 2     filtered: 100.00        Extra: Using index*************************** 3. row ***************************           id: 3  select_type: DERIVED        table: t   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_full_table; Scanned all databases3 rows in set, 1 warning (0.00 sec)
+----------+| count(1) |+----------+|     8106 |+----------+1 row in set, 417 warnings (8.62 sec)

最終SQL 如下


SELECT c.TABLE_SCHEMA AS DB, c.TABLE_NAME AS TBL, c.COLUMN_NAME AS COL, c.CHARACTER_OCTET_LENGTH AS COL_LEN_BYTES, s.INDEX_NAME,s.SUB_PART * c.CHARACTER_OCTET_LENGTH/c.CHARACTER_MAXIMUM_LENGTH AS SUB_PART_LENfrom ( select c.TABLE_SCHEMA, c.TABLE_NAME, c.COLUMN_NAME ,c.CHARACTER_OCTET_LENGTH ,c.CHARACTER_MAXIMUM_LENGTH , c.DATA_TYPEfrom information_schema.COLUMNS c WHERE c.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test') AND c.DATA_TYPE IN ("varchar", "char", "text", "blob")  ) c straight_join (select t.TABLE_SCHEMA, t.TABLE_NAMEfrom  information_schema.TABLES t  WHERE t.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test')  AND t.TABLE_ROWS > 10000limit 10000000000) t on c.TABLE_SCHEMA=t.TABLE_SCHEMA and  c.TABLE_NAME =t.TABLE_NAME
straight_join(select s.* from (select s.TABLE_SCHEMA, s.TABLE_NAME, s.COLUMN_NAME ,s.SUB_PART,s.INDEX_NAMEfrom information_schema.STATISTICS s WHERE s.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test') )s straight_join (select t.TABLE_SCHEMA, t.TABLE_NAMEfrom  information_schema.TABLES t  WHERE t.TABLE_SCHEMA not in ('information_schema','performance_schema','mysql','sys', 'test')  AND t.TABLE_ROWS > 10000limit 10000000000) t on s.TABLE_SCHEMA=t.TABLE_SCHEMA and  s.TABLE_NAME =t.TABLE_NAMElimit 10000000000) s on c.TABLE_SCHEMA=s.TABLE_SCHEMA and  c.TABLE_NAME=s.TABLE_NAME and c.COLUMN_NAME =s.COLUMN_NAMEwhere   ((c.CHARACTER_OCTET_LENGTH > 50 and s.SUB_PART is null) ors.SUB_PART * c.CHARACTER_OCTET_LENGTH/c.CHARACTER_MAXIMUM_LENGTH > 50)

*************************** 1. row ***************************           id: 1  select_type: PRIMARY        table: c   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_frm_only; Scanned all databases*************************** 2. row ***************************           id: 1  select_type: PRIMARY        table: <derived3>   partitions: NULL         type: refpossible_keys: <auto_key0>          key: <auto_key0>      key_len: 388          ref: information_schema.c.TABLE_SCHEMA,information_schema.c.TABLE_NAME         rows: 2     filtered: 100.00        Extra: Using index*************************** 3. row ***************************           id: 1  select_type: PRIMARY        table: <derived4>   partitions: NULL         type: refpossible_keys: <auto_key0>          key: <auto_key0>      key_len: 582          ref: information_schema.c.TABLE_SCHEMA,information_schema.c.TABLE_NAME,information_schema.c.COLUMN_NAME         rows: 2     filtered: 100.00        Extra: Using where*************************** 4. row ***************************           id: 4  select_type: DERIVED        table: s   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_frm_only; Scanned all databases*************************** 5. row ***************************           id: 4  select_type: DERIVED        table: <derived6>   partitions: NULL         type: refpossible_keys: <auto_key0>          key: <auto_key0>      key_len: 388          ref: information_schema.s.TABLE_SCHEMA,information_schema.s.TABLE_NAME         rows: 2     filtered: 100.00        Extra: Using index*************************** 6. row ***************************           id: 6  select_type: DERIVED        table: t   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_full_table; Scanned all databases*************************** 7. row ***************************           id: 3  select_type: DERIVED        table: t   partitions: NULL         type: ALLpossible_keys: NULL          key: NULL      key_len: NULL          ref: NULL         rows: NULL     filtered: NULL        Extra: Using where; Open_full_table; Scanned all databases7 rows in set, 1 warning (0.00 sec)

看起來穩定了，跑了幾次，都沒超過15秒

我的新一輪的SQL 優化課即將在春節後開課

我是知數堂SQL 優化班老師~ ^^

如有關於SQL優化方面疑問和一起交流的請加並且 @兔子@知數堂SQL優化

高性能MySQL,SQL優化羣有葉金榮，吳炳錫兩位大神坐鎮：579036588

歡迎加入知數堂大家庭。

我的微信公衆號：SQL開發與優化(sqlturning)

掃碼直達寶藏課程

本文分享自微信公衆號 - 老葉茶館（iMySQL_WX）。
如有侵權，請聯繫 [email protected] 刪除。
本文參與“OSC源創計劃”，歡迎正在閱讀的你也加入，一起分享。

MySQL中怎樣快速找出超長索引

高效率使用windows

面向金融級應用的GreatSQL正式開源

《葉問》36期，MySQL最多隻能用到128個邏輯CPU，是真的嗎

GreatSQL重磅特性，InnoDB並行並行查詢優化測試

《葉問》35期，binlog解析出來的日誌爲何無法恢復

一週碎碎念，2021.7.25，ansible文件傳輸有問題？

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結