mysql大表統計查詢優化

大表統計查詢優化

1. 統計

1.1 使用count

select count(*) as num from table_name ; 

準確率：運行查詢100%準確度。
效率：不適合大表查詢。
MyISAM ：表查詢非常快，但是使用where後,依然要進行全表掃描
InnoDB ：取決於表的大小，因爲引擎必須掃描整個表或整個索引以獲得準確的計數。表越大，越慢。

eg：

mysql> select count(*) from user_test;
+----------+
| count(*) |
+----------+
|  5456864 |
+----------+
1 row in set (6.94 sec)

1.2 使用SQL_CALC_FOUND_ROWS和FOUND_ROWS

select sql_calc_found_rows * from table_name limit 0 ; 
select found_rows() as num ;

準確率和效率和 1.1 完全一樣，但是這裏做列表分頁查詢時，第一次查完列表數據，第二次統計會更快，不需要再次掃描

eg.

mysql> select sql_calc_found_rows * from user_test limit 0;
Empty set (4.19 sec)

mysql> select found_rows() as num;
+---------+
| num     |
+---------+
| 5456864 |
+---------+
1 row in set (0.00 sec)

1.3 使用information_schema

SELECT `TABLE_ROWS` FROM `INFORMATION_SCHEMA`.`TABLES` WHERE `TABLE_SCHEMA`='dbname' AND `TABLE_NAME`='table_name';

準確率: 結果是近似值。如果表是頻繁插入和刪除的表，則結果可能與實際計數不同。這可以通過ANALYZE TABLE更頻繁地運行來改善。改善的結果仍可能不是準確的結果。
效率：非常好，它根本不觸及具體表

eg.

mysql> SELECT `TABLE_ROWS` FROM `INFORMATION_SCHEMA`.`TABLES` WHERE `TABLE_SCHEMA`='thd' AND `TABLE_NAME`='user_test';
+------------+
| TABLE_ROWS |
+------------+
|    5294801 |
+------------+
1 row in set (0.00 sec)

1.4 開啓mysql查詢緩存(8.0已廢棄)

更改my.ini

query_cache_type = 1
query_cache_size = 10M

準確率 ：100%
效率 ：第一次查詢比較慢，以後的每次查詢(數據變更不頻繁的情況下)直接返回緩存結果，超級快
但是數據變更比較頻繁的話，不建議使用

eg.

mysql> select count(*) from user_test;
+----------+
| count(*) |
+----------+
|  5456864 |
+----------+
1 row in set (7.01 sec)

mysql> select count(*) from user_test;
+----------+
| count(*) |
+----------+
|  5456864 |
+----------+
1 row in set (0.00 sec)

1.5 增加統計表

我們在原有表的基礎上增加一個維護數量的統計表

select num from table_statics where table_name = `table_name`

準確率 ：100%
效率 ：非常快
缺點：增加數據庫操作開銷

表名： table_statics

字段名	說明
id	記錄id
table_name	表名
table_count	行數

使用觸發器

DELIMITER $$
 
CREATE
    TRIGGER `triggerA` AFTER INSERT 
    ON `user_test`
    FOR EACH ROW BEGIN
 
    IF NOT EXISTS(SELECT id FROM table_statics WHERE table_name = 'user_test') THEN  
      INSERT INTO table_statics(table_name,table_count) VALUES('user_test',1);
    ELSE 
      UPDATE table_statics SET table_count = table_count + 1 WHERE table_name  = 'user_test';
    END IF;     
    END$$
 
DELIMITER ;


DELIMITER $$
 
CREATE
    TRIGGER `triggerB` AFTER DELETE  
    ON `user_test`
    FOR EACH ROW BEGIN
 
    IF NOT EXISTS(SELECT id FROM table_statics WHERE table_name = 'user_test') THEN
      INSERT INTO table_statics(table_name,table_count) VALUES('user_test',1);
    ELSE 
      UPDATE table_statics SET table_count = table_count - 1 WHERE table_name  = 'user_test';
    END IF;     
    END$$
 
DELIMITER ;

原始數據：

mysql> select * from table_statics where table_name = 'user_test';
+----+------------+-------------+---------------------+
| id | table_name | table_count | create_time         |
+----+------------+-------------+---------------------+
|  1 | user_test  |     5456860 | 2019-03-21 17:24:06 |
+----+------------+-------------+---------------------+
1 row in set (0.00 sec)

刪除一條後數據：

mysql> delete from user_test limit 1;
Query OK, 1 row affected (0.09 sec)

mysql> select * from table_statics where table_name = 'user_test';
+----+------------+-------------+---------------------+
| id | table_name | table_count | create_time         |
+----+------------+-------------+---------------------+
|  1 | user_test  |     5456859 | 2019-03-21 17:34:14 |
+----+------------+-------------+---------------------+

增加一條數據：

mysql> insert into user_test(username,`desc`,email,create_time) VALUES('張三30899','這是一段描述30899','[email protected]','2019-03-21 11:49:53');
Query OK, 1 row affected (0.10 sec)

mysql> select * from table_statics where table_name = 'user_test';
+----+------------+-------------+---------------------+
| id | table_name | table_count | create_time         |
+----+------------+-------------+---------------------+
|  1 | user_test  |     5456860 | 2019-03-21 17:35:08 |
+----+------------+-------------+---------------------+
1 row in set (0.00 sec)

1.6 使用應用程序緩存

$key = 'table_count:user_test';
$count = $redis->get($key);
if(empty($count)) {
    $count = 'from db count';
	$redis->set($key,$count,10*60);
}

準確率 ：根據數據的更新頻率近似 100%
效率 ：非常快
缺點：更新非常頻繁的，會有數據統計誤差

參考思路來自 ：https://dba.stackexchange.com/questions/151769/mysql-difference-between-using-count-and-information-schema-tables-for-coun

覺得好看，打賞一下小弟吧：