收到一個mysql服務器負載告警,上去一看,load average都飙到280多了,用top一看,CPU跑到了336%,不過IO和內存的負載並不高,根據經驗,應該又是一起索引引起的慘案了。
看下processlist以及slow query情況,發現有一個SQL經常出現,執行計劃中的掃描記錄數看著還可以,單次執行耗時為0.07s,還不算太大。乍一看,可能不是它引發的,但出現頻率實在太高,而且執行計劃看起來也不夠完美:
mysql> explain SELECT count(1) FROM a , b WHERE a.id = b.video_id and b.state = 1 AND b.column_id = '81'\G
*************************** 1. row *************************** id: 1 select_type: SIMPLE table: b type: index_merge possible_keys: columnid_videoid,column_id,state,video_time_stamp,idx_videoid key: column_id,state key_len: 4,4 ref: NULL rows: 100 Extra: Using intersect(column_id,state); Using where *************************** 2. row *************************** id: 1 select_type: SIMPLE table: a type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 4 ref: b.video_id rows: 1 Extra: Using where; Using index
再看下該表的索引情況:
mysql> show index from b\G
*************************** 1. row *************************** Table: b Non_unique: 0 Key_name: PRIMARY Seq_in_index: 1 Column_name: id Collation: A Cardinality: 167483 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: Index_comment: *************************** 2. row *************************** Table: b Non_unique: 1 Key_name: column_id Seq_in_index: 1 Column_name: column_id Collation: A Cardinality: 8374 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: Index_comment: *************************** 3. row *************************** Table: b Non_unique: 1 Key_name: state Seq_in_index: 2 Column_name: state Collation: A Cardinality: 5 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: Index_comment:
可以看到執行計劃中,使用的是index merge,效率自然沒有用聯合索引(也有的叫做覆蓋索引)來的好了,而且 state 字段的基數(唯一性)太差,索引效果很差。刪掉兩個獨立索引,修改成聯合看看效果如何:
mysql> show index from b;
*************************** 1. row *************************** Table: b Non_unique: 0 Key_name: PRIMARY Seq_in_index: 1 Column_name: id Collation: A Cardinality: 128151 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: Index_comment: *************************** 2. row *************************** Table: b Non_unique: 1 Key_name: idx_columnid_state Seq_in_index: 1 Column_name: column_id Collation: A Cardinality: 3203 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: Index_comment: *************************** 3. row *************************** Table: b Non_unique: 1 Key_name: idx_columnid_state Seq_in_index: 2 Column_name: state Collation: A Cardinality: 3463 Sub_part: NULL Packed: NULL Null: Index_type: BTREE Comment: Index_comment: mysql> explain SELECT count(1) FROM a , b WHERE a.id = b.video_id and b.state = 1 AND b.column_id = '81' \G *************************** 1. row *************************** id: 1 select_type: SIMPLE table: b type: ref possible_keys: columnid_videoid,idx_videoid,idx_columnid_state key: columnid_videoid key_len: 4 ref: const rows: 199 Extra: Using where *************************** 2. row *************************** id: 1 select_type: SIMPLE table: a type: eq_ref possible_keys: PRIMARY key: PRIMARY key_len: 4 ref: b.video_id rows: 1 Extra: Using where; Using index
可以看到執行計劃變成了只用到了 idx_columnid_state 索引,而且 ref 類型也變成了 const,SQL執行耗時也從0.07s變成了0.00s,相應的CPU負載也從336%突降到了12%不到。
總結下,從多次歷史經驗來看,如果CPU負載持續很高,但內存和IO都還好的話,這種情況下,首先想到的一定是索引問題,十有八九錯不了。