views:

39

answers:

1

I have a App Engine/Python/Django application which has grown and been modified over the past year and currently has 175 indexes. The problem is that I have not been thourough in cleaning up/removing indexes that are no longer needed. Now, I am not sure which indexes are active and which are essentially dead, but I am guessing that about 20% of the idexes are useless.

I am curious if there are any App Engine tools available for tracking/counting number of accesses to indexes?

If no tools are available, then one possible idea is to overload the fetch method to track this information everytime an index is accessed, but I am not sure if this is a good idea (could slow things down) and I don't know what the best way to implement this might be.

If anybody has already gone through the experience of cleaning up (searching for) dead indexes, I would be interested in hearing about your experiences.

+4  A: 

The App Engine SDK tracks this for its automatic index creation. Delete your index.yaml, then give your app a good workout. As long as you hit every distinct query in your testing, the SDK will generate a new index.yaml that contains only the indexes you need.

Nick Johnson
Hi Nick, thanks for the answer -- but I believe that you are referring to the development server only, correct? ... I am curious in seeing statistics from the production/live system. Trying to execute all combinations of indexes on the testing server is a lot of work and could be prone to missing a particular combination which would then show up when I go live...
Alexander
Yes, this is only on the SDK. If you can't exercise your whole app, though, I suspect your pre-release testing needs some improvement. It'd certainly be possible to instrument the production environment to record index usage, but not trivial. See my article about hooks for how to get started: http://blog.notdot.net/2009/11/API-call-hooks-for-fun-and-profit
Nick Johnson
Excellent, that looks like something that could do what I need.
Alexander