views:

43

answers:

4

Some of my programs send direct queries to Google and then parse the HTML results - for instance http://www.google.com/search?q=foobar&hl=en&num=20.

Unfortunately, it seems that since very recently, when sending such queries to Google, the "num" parameter is ignored because of Instant Search. No matter what, only 10 results are shown in the page. If you disable Instant Search, then it works again. Problem is that settings is stored in a cookie or something and it's very impractical, if at all possible, to pre-set from the program side.

Is there a way to add an extra parameter to the query to bypass Instant Search and get "num" working again? I'm sure I'm not only one parsing Google HTML results...

A: 

One workaround is to use the "start" parameter which still works and send multiple queries, but it's not very clean.

http://www.google.com/search?q=foobar&hl=en&num=20&start=0
http://www.google.com/search?q=foobar&hl=en&num=20&start=10
...
Pol
A: 

It looks like when you set it (using your mouse), you're just clicking a link. Here's mine (snipped):

http://www.google.com/setprefs?sig=0_4baC3UslsXJ-SNcQMDzaagFeq5k=&suggon=2

The important part is the suggon parameter. suggon=1 turns it on, suggon=2 turns it off. You need the sig parameter to switch it using just that request, else you get redirected to your Google Preferences page, without the change being made.

...Anyway, for my Google account, I can turn Instant Search off by going to this URL (the prev parameter is optional):

http://www.google.com/setprefs?sig=0_4baC3UslsXJ-SNcQMDzaagFeq5k=&suggon=2&prev=http://www.google.com/search%3Fq%3DFoo


If you have the source HTML of a page that contains the links, look for <a> elements whose href attribute starts with /setprefs?sig= - so you could write it as a jQuery selector like this: $('a[href^=/setprefs?sig=]').

Matt Ball
do you know what the sig parameter is for? I'm being redirected to the preferences page.
Alexander Sagen
@Alexander - I can only guess that it's something like a user identifier. I don't actually know what it is though (see my edit, if it's any help).
Matt Ball
A: 

More investigation shows that this "bug" seems to only happen for web browsers, not when using HTTP clients that don't provide a user agent.

So things are still good after all!

Pol
A: 

Just add "&as_qdr=all"

http://www.google.com/search?q=foobar&amp;hl=en&amp;num=2&amp;start=0&amp;as_qdr=all

maclema
Awesome, thanks!
Pol