Should I denormalize properties to reduce the number of indexes required by App Engine?

Question

One of my queries can take a lot of different filters and sort orders depending on user input. This generates a huge index.yaml file of 50+ indexes.

I'm thinking of denormalizing many of my boolean and multi-choice (string) properties into a single string list property. This way, I will reduce the number of query combinations because most queries will simply add a filter to the string list property, and my index count should decrease dramatically.

It will surely increase my storage size, but this isn't really an issue as I won't have that much data.

Does this sound like a good idea or are there any other drawbacks with this approach?

Answer 1

Denormalizing your data to cut back on the number of indices sounds like it a good tradeoff. Reducing the number of indices you need will not only save space, but it will speed up writes (fewer indices to update). Of course your entities themselves will be a little larger which could make reads a little longer (but they shouldn't be significantly impacted unless you add a lot of denormalized data).

This is complicated a little bit since the index on the list will contain one entry for each element in the list on each entity (rather than just one entry per entity). This will certainly impact space, and query performance. Also, if you have any other indexed list properties on your model already, then you could run into a problem with exploding indices.

Try experimenting and see how it works in practice for you (use AppStats!).