views:

401

answers:

2

I'm using HttpWebRequest to log on to a website and get the session cookie. My application scrapes certain pages, and eventually becomes idle, resulting in the session dying.

What would be a good way to prevent this and -just being curious- how do web browsers do it?

+1  A: 

You can't prevent it as such, unless you keep the session active by requesting a particular resource of the website in question at certain fixed intervals, say every 10-15 minutes.

Wim Hollebrandse
Agreed, just constantly poll the site.
Chris Marisic
+3  A: 

Browsers by themselves don't do this at all. They only do so if the Web App itself builds in support for it. For example there might be some javascript which makes an AJAX request every 30 seconds or so to send a "keep-session-alive" type request to the server, which would then go and update its state information to reflect that the "most recent activity" was now, and hence prevent a session timeout.

In the example above however, the "client" i.e. html/javascript in the browser, and the "server" would have been written together and thus this type of functionality built in.

If you aren't responsible for the server-side app, and thus can't build the "keep-alive" functionality into it yourself, your best bet would be to either physically go to the web-page, look through all the javascript to try to find possible "keep-alive" code (look for SetTimeout, SetInterval or related methods in any js frameworks they may be using). Of course there might not be any such functionality, in which case your best bet is to simply "refresh" the page you are on (i.e. send the same request again)

Graza