views:

225

answers:

5

Let's say I wanted to make a python script interface with a site like Twitter.

What would I use to do that? I'm used to using curl/wget from bash, but Python seems to be much nicer to use. What's the equivalent?

(This isn't Python run from a webserver, but run locally via the command line)

+2  A: 

Python has a very nice httplib module as well as a url module which together will probably accomplish most of what you need (at least with regards to wget functionality).

Joe Corkery
+3  A: 

I wholeheartedly recommend mechanize for python. It's exactly a programmable web browser that you can use from python, which handles forms and cookies as well! It makes any kind of site crawling a breeze.

Take a look at the examples on that link to see what it can do.

Claudiu
Mechanize is ideal for situations where you DON'T have an API, but the OP specifies a site like Twitter, which has an API, so urllib2 is usually the way to go.
bouvard
ah, i didn't exactly get what you meant by "has an API" - but you mean it has an extensive way of interfacing with it just through URLs. in that case, yeah, urllib2 should be sufficient.
Claudiu
+5  A: 

Python has urllib2, which is extensible library for opening URLs

Full-featured easy to use library.

http://www.python.org/doc/2.5.2/lib/module-urllib2.html

+6  A: 

For something like Twitter, you'll save yourself a ton of time by not reinventing the wheel. Try a library like python-twitter. This way, you can write your script, or even a full fledged application, that interfaces with Twitter, and you don't have to care about the implementation details.

If you want to roll your own interface library, you're going to have to get familiar with urllib and depending on what format they provide results, either lxml (or some other xml parser) or simplejson.

Jeremy Cantrell
+1 Simplejson is a vital component here.
Ali A
urllib, lxml and simplejson sound like the tools I need. Thanks!
Rich Bradshaw
@Rich: if you found this helpful, then you know what you must do... wink wink nudge nudge
Jeremy Cantrell
A: 

If you're used to dealing with cURL, consider PycURL.

John McCollum