views:

53

answers:

2

I have a 10M-row table product with fields like color (int), price (float), weight (float), unitprice (int), etc ... Now users from Web dynamically generate queries to lookup data from this table with random conditions (color is a must have here) and order-by such as

select * from product where color=1 and price >5 and price <220 and .... order by unitprice limit 75, 25;

select count(*) from product where color=3 and weight <500 and price <30 ... ;

How to index a table (InnoDB or NDB) with about 10 possible filtering fields (with range, sorting ...) like this in MySQL?


EDIT: In my understanding MySQL most likely will pick only one index for a query, and only the left hand part of a composite index will work. Obviously indexing all possible combinations is not a feasible option, such as (color, price, weight, create_date, unitprice, ....), (color, weight, price, create_date, unitprice, ....), (color, unitprice, weight, ....) .... Not all conditions are necessarily present in all queries.

What would you do to index this table?

+1  A: 

If you want quick lookups/filters/sorts on any field, you must put indexes on all of them.

If color ist a must have (i.e. used in every query), it's best to make composite indexes on (color, field) for each field you have.

Putting the clustered index over (color, product_id) might also be worth a try if color really is a part of every common query.

Tomalak
Thanks for the tip. I have like 10 such fields in this table. Are you suggesting composite indexes of only 2 key parts? or all 10? Yes color will be the first part of all indexes.
QWJ QWJ
@QWJ My suggestion was two key parts for each index, 10 indexes for your table. This way MySQL can already satisfy the most significant part of the query from looking at a single index (color + x). Here is [some more reading on the topic](http://stackoverflow.com/questions/1277865/mysql-indexes-how-many-are-enough).
Tomalak
You could, perhaps, gather statistics about the queries made (by writing the queries to a log, for instance) and remove indexes that are hardly ever used, if the large number of indexes starts to prove troublesome.
Brian Hooper
@QWJ Indexes have two down sides: a) They increase the size of the table and b) they slow down any write operation (insert, update, delete) on the table, because not only the table but also every affected index needs a change. Throwing out unnecessary indexes usually improves write performance, creating necessary indexes usually improves read performance. Finding the right balance is subject to the use case.
Tomalak
Thanks all. I'll try to create 10 indexes with 2 parts. But in the future when the table grows, I'll probably need more parts in the composite indexes - still a headache. Regarding inserts, I'll look into cluster or replication etc.
QWJ QWJ
A: 

As Tomalak already answered, you probably should add indexes for all fields (and composite indexes, depending on your queries). But of course this can slow writes down.

If you're not sure how the indexes will be used, you can use the explain command.

Waleed Eissa