views:

55

answers:

1

I have three columns in my table.

+-----------+-----------------------+------+-----+---------+-------+
| Field     | Type                  | Null | Key | Default | Extra |
+-----------+-----------------------+------+-----+---------+-------+
| hash      | mediumint(8) unsigned | NO   | PRI | 0       |       | 
| nums      | int(10) unsigned      | NO   | PRI | 0       |       | 
| acc       | smallint(5) unsigned  | NO   | PRI | 0       |       | 
+-----------+-----------------------+------+-----+---------+-------+

I am expecting duplicates in my data so I went ahead and added a unique constraint:

ALTER TABLE nt_accs ADD UNIQUE(hash,nums,acc);

I have about 500 million rows to insert into this table and this table has been paritioned using a RANGE on nums into about 20 partitions.

  1. Does the unique constraint slow down inserts? How does this differ in just making both a Primary Key instead of imposing a unique constraint?
  2. I have a lot of GROUP BY type queries using both the hash and nums columns. Do I go ahead and add a convering index on and or do I just add individual indexes?

EDIT:

Explain plan after partitioning and inserting some test data

1. mysql> explain partitions select * from nt_accs;
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-------------+
| id | select_type | table     | partitions                                                                | type  | possible_keys | key      | key_len | ref  | rows | Extra       |
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-------------+
|  1 | SIMPLE      | nt_accs   | p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20 | index | NULL          | hash     | 7       | NULL |   10 | Using index | 
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-------------+
1 row in set (0.00 sec)



2. mysql> explain partitions select * from nt_accs WHERE nums=1504887570;
+----+-------------+-----------+------------+-------+---------------+----------+---------+------+------+--------------------------+
| id | select_type | table     | partitions | type  | possible_keys | key      | key_len | ref  | rows | Extra                    |
+----+-------------+-----------+------------+-------+---------------+----------+---------+------+------+--------------------------+
|  1 | SIMPLE      | nt_accs   | p7         | index | NULL          | hash     | 7       | NULL |   10 | Using where; Using index | 
+----+-------------+-----------+------------+-------+---------------+----------+---------+------+------+--------------------------+
1 row in set (0.00 sec)

3. mysql> explain partitions select * from nt_accs WHERE hash=2347200;
+----+-------------+-----------+---------------------------------------------------------------------------+------+---------------+----------+---------+-------+------+-------------+
| id | select_type | table     | partitions                                                                | type | possible_keys | key      | key_len | ref   | rows | Extra       |
+----+-------------+-----------+---------------------------------------------------------------------------+------+---------------+----------+---------+-------+------+-------------+
|  1 | SIMPLE      | nt_accs  | p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20 | ref  | hash          | hash     | 3       | const |   27 | Using index | 
+----+-------------+-----------+---------------------------------------------------------------------------+------+---------------+----------+---------+-------+------+-------------+
1 row in set (0.00 sec)

4. mysql> EXPLAIN PARTITIONS SELECT hash, count(distinct nums) FROM nt_accs GROUP BY hash;
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-------------+
| id | select_type | table     | partitions                                                                | type  | possible_keys | key      | key_len | ref  | rows | Extra       |
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-------------+
|  1 | SIMPLE      | nt_accs   | p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20 | index | NULL          | hash     | 7       | NULL |   10 | Using index | 
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-------------+
1 row in set (0.00 sec)

5. mysql> EXPLAIN PARTITIONS SELECT nums, count(distinct hash) FROM nt_accs GROUP BY nums;
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-----------------------------+
| id | select_type | table     | partitions                                                                | type  | possible_keys | key      | key_len | ref  | rows | Extra                       |
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-----------------------------+
|  1 | SIMPLE      | nt_accs   | p0,p1,p2,p3,p4,p5,p6,p7,p8,p9,p10,p11,p12,p13,p14,p15,p16,p17,p18,p19,p20 | index | NULL          | hash     | 7       | NULL |   10 | Using index; Using filesort | 
+----+-------------+-----------+---------------------------------------------------------------------------+-------+---------------+----------+---------+------+------+-----------------------------+
1 row in set (0.00 sec)

I am perfectly fine with the first and second queries but I'm not sure about the performance of the 3rd, 4th and 5th. Is there anything else I can do at this point to optimize this?

+3  A: 

Does the unique constraint slow down inserts? How does this differ in just making both a Primary Key instead of imposing a unique constraint?

Yes, an index (MySQL implements a unique constraint as an index) will slow down inserts.
The same goes a primary key, which is why tables expecting high insertion loads (IE: for logging) do not have a primary key defined--to make insertions faster.

I have a lot of GROUP BY type queries using both the hash and nums columns. Do I go ahead and add a convering index on and or do I just add individual indexes?

The only way to definitely know is to test & check the EXPLAIN plan.

UPDATE

In light of the provided explain plans, I don't see the concern for 3rd & 4th versions. MySQL can only use one index per select_type. The fifth version might benefit from a covering index.

Addendum

Just want to make sure that you are aware that:

ALTER TABLE nt_accs ADD UNIQUE(hash, nums, acc);

...means the combination of the three column values will be unique. IE: These are valid, the unique constraint will allow:

hash  nums  acc
----------------
1     1     1
1     1     2
1     2     1
2     1     1
OMG Ponies
@OMG Ponies: Thank you for the explanation. I'm aware of the unique constraint rule.
Legend
@OMG Ponies: Sorry about unaccepting the answer. I forgot I had one more question. I added extra details into the question.
Legend
@OMG Ponies: So the covering index should be on (nums, hash) I presume. Is that right?
Legend
@Legend: Try it, but the order of the covering index should be enough--the filesort could be due to the aggregate function use.
OMG Ponies
@OMG Ponies: Great. Thank you very much.
Legend