views:

81

answers:

3

I want to load info from another site (this part is done), but i am doing this every time the page is loaded and that wont do. So i was thinking of having a variable in a table of settings like 'last checked bbc site' and when the page loads it would check if its been long enough since last check to check again. Is there anything silly about doing it that way?

Also do i absolutely have to use tables to store 1 off variables like this setting?

+1  A: 

You could also use Scheduled Tasks.

Also, you don't absolutely need to use the Datastore for configuration parameters: you could have this in a script / config file.

jldupont
When i declare variables in a script they will disappear when the script is finished i thought. Am i wrong?
tm1rbrt
more precisely: when the request finishes. But there is nothing preventing from loading the script again and again etc. If you use a Scheduled Task, then this configuration can be only located in cron.yaml (of course you can still load stuff from the task script).
jldupont
I think the poster was talking about storing the "last fetched time", not the configuration parameters.
Peter Recore
@Peter: oh yes, of course... there is the memcache API although this is "best efforts" in nature.
jldupont
A: 

If you want some handler on your GAE app (including one for a scheduled task, reception of messages, web page visits, etc) to store some new information in such a way that some handler in the future can recover that information, then GAE's storage is the only good general way (memcache could expire from under you, for example). Not sure what you mean by "tables" (?!), but guessing that you actually mean GAE's storage the answer is "yes". (Under very specific circumstances you might want to put that data to some different place on the network, such as your visitor's browser e.g. via cookies, or an Amazon storage instance, etc, but it does not appear to me that those specific circumstances are appliable to your use case).

Alex Martelli
+2  A: 

I think there are 2 options that would work for you, besides creating a entity in the datastore to keep track of "last visited time".

One way is to just check the external page periodically, using the cron api as described by jldupont.

The second way is to store the last visited time in memcache. Although memcache is not permanent, it doesn't have to be if you are only storing last refresh times. If your entry in memcache were to disappear for some reason, the worst that would happen would be that you would fetch the page again, and update memcache with the current date/time.

The first way would be best if you want to check the external page at regular intervals. The second way might be better if you want to check the external page only when a user clicks on your page, and you haven't fetched that page yourself in the recent past. With this method, you aren't wasting resources fetching the external page unless someone is actually looking for data related to it.

Peter Recore
"With this method, you aren't wasting resources fetching the external page unless someone is actually looking for data related to it." I was thinking this, but then i thought i would i hope i get more than 2 hits a day :P
tm1rbrt
yeah, it depends on how many different pages you are checking, and how often you need to check them. If you had 1000 different pages you *might* need to check, then you probably would save resources. But if you only have one page, then you might as well just check that once every hour or once every day or whatever.
Peter Recore