MySQL Query: Long time spent in 'init' state

Question

I'm performing an update on a MySQL table (myisam engine) that, according to the profiler, is spending an inordinate amount of time in the 'init' state:

mysql> show profile for query 2;
+----------------------+-----------+
| Status               | Duration  |
+----------------------+-----------+
| starting             |  0.000057 |
| checking permissions |  0.000006 |
| Opening tables       |  0.000020 |
| System lock          |  0.000007 |
| Table lock           |  0.000005 |
| init                 | 21.911657 |
| Updating             |  0.002363 |
| end                  |  0.000009 |
| query end            |  0.000004 |
| freeing items        |  0.000051 |
| logging slow query   |  0.000003 |
| logging slow query   |  0.000002 |
| cleaning up          |  0.000005 |
+----------------------+-----------+

The query is as follows:

mysql> update my_table
    -> set rank = 
    ->    greatest(
    ->       @rank := if(@score = score, @rank, @rank + 1),
    ->       least(0, @score := score)
    ->    )
    -> where game=7 and zone=11 and ladder=2
    -> order by score
    -> limit 100;

Query OK, 100 rows affected (21.92 sec)
Rows matched: 100  Changed: 100  Warnings: 0

I have a compound index on all columns listed in the 'where' and 'order by' clauses (see index named 'zone_lad_score' below):

mysql> show indexes from my_table;
+--------------------+------------+-----------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| Table              | Non_unique | Key_name        | Seq_in_index | Column_name  | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------------+------------+-----------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+
| my_table           |          1 | indx_e          |            1 | col_e        | A         |     2937401 |     NULL | NULL   |      | BTREE      |         |
| my_table           |          1 | zone_score      |            1 | zone         | A         |         217 |     NULL | NULL   |      | BTREE      |         |
| my_table           |          1 | zone_score      |            2 | score        | A         |    23499213 |     NULL | NULL   | YES  | BTREE      |         |
| my_table           |          1 | zone_d_score    |            1 | zone         | A         |         217 |     NULL | NULL   |      | BTREE      |         |
| my_table           |          1 | zone_d_score    |            2 | col_d        | A         |      123355 |     NULL | NULL   | YES  | BTREE      |         |
| my_table           |          1 | zone_d_score    |            3 | score        | A         |    46998427 |     NULL | NULL   | YES  | BTREE      |         |
| my_table           |          1 | zone_lad_score  |            1 | zone         | A         |         217 |     NULL | NULL   |      | BTREE      |         |
| my_table           |          1 | zone_lad_score  |            2 | ladder       | A         |         868 |     NULL | NULL   | YES  | BTREE      |         |
| my_table           |          1 | zone_lad_score  |            3 | score        | A         |    23499213 |     NULL | NULL   | YES  | BTREE      |         |
+--------------------+------------+-----------------+--------------+--------------+-----------+-------------+----------+--------+------+------------+---------+

I also have the table partitioned on 'game', with 10 partitions total. There are approximately 47 million total records in the table. Table definition is as follows:

my_table | CREATE TABLE `my_table` (
  `col_e` bigint(20) NOT NULL,
  `zone` bigint(20) NOT NULL,
  `score` int(11) DEFAULT NULL,
  `game` tinyint(4) DEFAULT NULL,
  `ladder` tinyint(4) DEFAULT NULL,
  `col_d` int(11) DEFAULT NULL,
  `rank` int(11) DEFAULT NULL,
  KEY `indx_e` (`col_e`),
  KEY `zone_score` (`zone`,`score`),
  KEY `zone_d_score` (`zone`,`col_d`,`score`),
  KEY `zone_lad_score` (`zone`,`ladder`,`score`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1
/*!50100 PARTITION BY LIST (game)
(PARTITION p1 VALUES IN (1) ENGINE = MyISAM,
 PARTITION p2 VALUES IN (2) ENGINE = MyISAM,
 PARTITION p3 VALUES IN (3) ENGINE = MyISAM,
 PARTITION p4 VALUES IN (4) ENGINE = MyISAM,
 PARTITION p5 VALUES IN (5) ENGINE = MyISAM,
 PARTITION p6 VALUES IN (6) ENGINE = MyISAM,
 PARTITION p7 VALUES IN (7) ENGINE = MyISAM,
 PARTITION p8 VALUES IN (8) ENGINE = MyISAM,
 PARTITION p9 VALUES IN (9) ENGINE = MyISAM,
 PARTITION p10 VALUES IN (10) ENGINE = MyISAM) */

Now, according to the MySQL docs (http://dev.mysql.com/doc/refman/5.0/en/general-thread-states.html), actions in the 'init' state include "flushing the binary log, the InnoDB log, and some query cache cleanup operations." Okay... so since I'm not using InnoDB, doesn't sound like anything that should take very much time.

I guess I'm wondering why this update, which should be utilizing an index, and only affecting 100 records would be taking so long? What specifically would be holding it up in the 'init' state for so long? If I perform a select on the targeting records (select * from my_table where game=7 and zone=11 and ladder=2 order by score limit 100), it returns almost instantly. Performing similar updates on that table (using zone_d_score index) take less than a second. What could be slowing down this particular update?

Edit: Added table definition, full listing of all indexes on the table in question, and renamed columns to make things a little easier to follow.

Edit 2: Here's an 'explain' on the query closest to the update:

mysql> explain select * from my_table where game=7 and zone=11 and ladder=2 order by score limit 100;
+----+-------------+--------------------+------+------------------------------------------------+-----------------+---------+-------------+-------+-------------+
| id | select_type | table              | type | possible_keys                                  | key             | key_len | ref         | rows  | Extra       |
+----+-------------+--------------------+------+------------------------------------------------+-----------------+---------+-------------+-------+-------------+
|  1 | SIMPLE      | my_table           | ref  | zone_score,zone_d_score,zone_lad_score         | zone_lad_score  | 10      | const,const | 53952 | Using where |
+----+-------------+--------------------+------+------------------------------------------------+-----------------+---------+-------------+-------+-------------+
1 row in set (0.00 sec)

Answer 1

After some more experimenting, I added an index on the table that also includes the column on which I partitioned the table:

CREATE INDEX game_zone_ladder_score ON my_table(game,zone,ladder,score)

and suddenly the UPDATE is performing much better (sub-second). I would've expected the UPDATE to take advantage of partitions just as the SELECTs do, but apparently not.

Would still love to know what exactly MySQL is doing during the 'init' state during UPDATEs, and/or why UPDATEs don't honor partitions.

Answer 2

There's a requirement that if you use partitioning, the partition column(s) must appear in your primary key. (Bullet point near the bottom)

http://dev.mysql.com/tech-resources/articles/mysql_5.1_partitions.html