Why does MySQL use the wrong index?

Question

Hello! I have another question regarding optimizing mysql indices for our prioritizing jBPM. The relevant indices look like this:

    | JBPM_TIMER |          1 | JBPM_TIMER_REVERSEPRIORITY__DUEDATE_ |            1 | REVERSEPRIORITY_ | A         |          17 |     NULL | NULL   | YES  | BTREE      |         | 
    | JBPM_TIMER |          1 | JBPM_TIMER_REVERSEPRIORITY__DUEDATE_ |            2 | DUEDATE_         | A         |      971894 |     NULL | NULL   | YES  | BTREE      |         | 
    | JBPM_TIMER |          1 | JBPM_TIMER_DUEDATE_                  |            1 | DUEDATE_         | A         |      971894 |     NULL | NULL   | YES  | BTREE      |         |

JBPM asks two questions when retrieving timers. The first one is dependent on the combined index (reversed priority and duedate) and the second one on the solo duedate index. However, when adding the solo index it takes precedence over the correct one when running this query:

mysql> explain select timer0_.ID_ as col_0_0_ from JBPM_TIMER timer0_ where timer0_.ISSUSPENDED_<>1 and  timer0_.DUEDATE_<='2009-08-17 14:51:06' order  by timer0_.REVERSEPRIORITY_ asc, timer0_.DUEDATE_ asc limit 160;
+----+-------------+---------+-------+---------------------------+---------------------------+---------+------+-------+-----------------------------+
| id | select_type | table   | type  | possible_keys             | key                       | key_len | ref  | rows  | Extra                       |
+----+-------------+---------+-------+---------------------------+---------------------------+---------+------+-------+-----------------------------+
|  1 | SIMPLE      | timer0_ | range | JBPM_TIMER_DUEDATE_       | JBPM_TIMER_DUEDATE_ERIK_T | 9       | NULL | 971894| Using where; Using filesort | 
+----+-------------+---------+-------+---------------------------+---------------------------+---------+------+-------+-----------------------------+
1 row in set (0.00 sec)

This index is needed for another query:

mysql> explain select timer0_.ID_ as col_0_0_ from JBPM_TIMER timer0_ where (timer0_.EXCEPTION_ is null) and timer0_.ISSUSPENDED_<>1 order by timer0_.DUEDATE_ asc limit 160;
+----+-------------+---------+-------+---------------+---------------------+---------+------+-------+-------------+
| id | select_type | table   | type  | possible_keys | key                 | key_len | ref  | rows  | Extra       |
+----+-------------+---------+-------+---------------+---------------------+---------+------+-------+-------------+
|  1 | SIMPLE      | timer0_ | index | NULL          | JBPM_TIMER_DUEDATE_ | 9       | NULL | 24249 | Using where | 
+----+-------------+---------+-------+---------------+---------------------+---------+------+-------+-------------+
1 row in set (0.00 sec)

When removing the solo index, query number 1 executes correctly and query 2 needs a filesort. Adding the solo index query number 2 executes correctly and query 1 needs a filesort.

This unwanted behavior can be overridden by adding an index hint to the first query:

explain select timer0_.ID_ as col_0_0_ 
from JBPM_TIMER timer0_ USE INDEX (JBPM_TIMER_REVERSEPRIORITY__DUEDATE_) 
where timer0_.ISSUSPENDED_<>1 and  
    timer0_.DUEDATE_<='2009-08-17 14:51:06' 
order  by timer0_.REVERSEPRIORITY_ asc, timer0_.DUEDATE_ asc 
limit 160;

Is the hint the only way to go to make MySQL optimize both queries correctly? Or are we doing something wrong?

Answer 1

try adding an index to (DUEDATE_, REVERSEPRIORITY_) in that order. it will still use filesort (i think) but on much less rows.

also try to OPTIMIZE your table and CHECK your table (so mysql will re-calculate index values)

this is all just an educated guess.

Answer 2

You are correct, we can even remove the other indices and only have (duedate,reversepriority).

It seems that this will make MySQL filesort the rows selected by the where clause in the case of the query sorting both columns.

The query that only sorts duedate will not have to be sorted.

However, we discovered that modifying the query:

mysql> explain select timer0_.ID_ as col_0_0_ from JBPM_TIMER timer0_ where timer0_.ISSUSPENDED_<>1 and  timer0_.DUEDATE_<='2009-08-17 14:51:06' order  by timer0_.REVERSEPRIORITY_ asc, timer0_.DUEDATE_ asc limit 160;

and adding the REVERSEPRIORITY_ column to the where clause, as such:

mysql> explain select timer0_.ID_ as col_0_0_ from JBPM_TIMER timer0_ where timer0_.ISSUSPENDED_<>1 and  timer0_.DUEDATE_<='2009-08-17 14:51:06' and timer0_.REVERSEPRIORITY < 0 order  by timer0_.REVERSEPRIORITY_ asc, timer0_.DUEDATE_ asc limit 160;

makes MySQL use the correct (combined) index.