Each of my users is polling the server every few seconds. I need to keep a list of the users that have polled in the last 30 seconds handy for a task I have queued to run every few seconds.
The obvious way I see to do it is to update a datastore entry every time the user polls, and query the entries that have a timestamp within the last N seconds within my task queue. I can't imagine this scaling well.
Any recommendations?
Thanks.
I've benchmarked writes for small entities at around 1/20 of a second each. If you base the key of your poll entities on a unique value associated with a user, then you can access the poll entities by key when you update them which is a very fast hash instead of a search.
The writes should be fine as long as you don't need to write to entities simultaneously (which it sounds like you won't be doing since you have one for each user). Just make sure that you don't put the poll entities in the same entity group, if you do that, writing to one will freeze all of the others during the write.
You can query for all poll entities that were updated in the last 30 seconds, this is a read and should be fast.
Having one datastore entity per user, and updating that with the current time each time they poll should work just fine - though it will, of course, add a little latency to every user request.