views:

102

answers:

2

We want to collect usage statistics with our app. Therefore, we want to track user actions somewhere on the server side.

Which option is more suitable, performance-wise:

  1. Track user actions in the App Engine Request Log. i.e. write a log entry for each user action.
  2. Track user actions in a table in the Datastore. Write an entry for each user action.

I guess that request log writes are much faster than Datastore writes. Is this correct? What are your experiences and suggestions?

Thanks: Henning

+4  A: 

I can confirm (through my own empirical observation, only: I do not see this addressed in the official docs) that writing a log entry is faster than writing a storage entry. Thinking about it, that makes sense: log entries are always appended, no indices, no particular transactional issues (beyond atomicity of each log-entry writing), etc etc; being so strictly limited it makes sense that they can be more optimized.

However I once (very very early on in the pre-beta days over a year ago) observed one case where I thought there should be a log entry I had written but just couldn't find it; I was unable to reproduce the observation, it may have been a glitch in my code or a since-fixed bug in GAE, but it makes me wonder a tiny bit about data integrity -- how guaranteed is the logs' integrity? Until and unless I see something in the docs explicitly about it, I'm not going to feel I can count on the logs for 100% integrity -- if accidentally losing one record in a million would be an absolute tragedy, maybe going for the store's higher guarantees is worth the performance impact.

Alex Martelli
I agree. It is not well-established that the logs are reliable enough for major bookkeeping. But for basic stats I would use them to avoid datastore quota and possibly an index. To me the major problem is getting the logs *off* app engine. There is a lot of potential for losing data since the logs are on a ring buffer.
jhs
+2  A: 

Request logs are very much faster - there's no need for a round-trip on each logging statement; logs can be accumulated and pushed after the end of the user request. However, there's no programmatic access to request logs, as far as I'm aware - so if you need that, you need to use the datastore.

If you want the best of both worlds, keep an eye out for the task queue API, being released very soon. :)

Nick Johnson
Thanks. Actually there is a way to access the request logs, you can download them with the appcfg.py script. As old logs get deleted, I will have to download them regularly however.
henning77