tags:

views:

83

answers:

2

I want to make a simple stupid twitter app using Twitter API.

If I request this page from my browser it does work:

http://search.twitter.com/search.atom?q=hello&rpp=10&page=1

but if I request this page from python using urllib or urllib2 most of the times it doesn't work:

response = urllib2.urlopen("http://search.twitter.com/search.atom?q=hello&rpp=10&page=1")

and I get this error:

Traceback (most recent call last):
  File "twitter.py", line 24, in <module>
    response = urllib2.urlopen("http://search.twitter.com/search.atom?q=hello&amp;rpp=10&amp;page=1")
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 391, in open
    response = self._open(req, data)
  File "/usr/lib/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 1161, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.6/urllib2.py", line 1136, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 110] Connection timed out>

Why ??

+1  A: 

did you change the default socket timeout somewhere in your script? your example code works reliably for me.

it could be your internet connection, or you might try

import socket
socket.setdefaulttimeout(30)

assuming urllib/2 don't override the socket timeout.

lunixbochs
no I haven't change anything. Some times it work for me as well but most of times NOT. Try more times please :D
xRobot
If it works sometimes and not others, then the problem is likely what the error says - your connection is timing out.
Tim
I ran it a hundred times in a loop and none of them timed out.
lunixbochs
you can try increasing the socket timeout as described, maybe change the number from 30 to something higher. again, I don't know how urllib2 handles timeouts.
lunixbochs
mmm very strange :(. I have increased the socket timeout as you described ( 30 50 and 100 ) but it still doesn't work :(
xRobot
+2  A: 

The code seems alright.

The following worked.

>>> import urllib
>>> import urllib2
>>> user_agent = 'curl/7.21.1 (x86_64-apple-darwin10.4.0) libcurl/7.21.1'
>>> url='http://search.twitter.com/search.atom?q=hello&amp;rpp=10&amp;page=1'
>>> headers = { 'User-Agent' : user_agent }
>>> req = urllib2.Request(url, None, headers)
>>> response = urllib2.urlopen(req)
>>> the_page = response.read()
>>> print the_page

The other is twitter actually could not respond. This happens once too often with Twitter.

pyfunc