How to access the web with Python?

Question

I want to access websites without using their API. Would i do this by using something like Mechanize?

Answer 1

You can access websites by HTTP protocol client: httplib

Though maybe you'd like urllib2, in specific the urllib2.urlopen

Here's little example about using urllib2:

import urllib2
page = urllib2.urlopen("http://example.com/").read()
print page

Answer 2

If you simply ask google, you will get at least two answers.

http://docs.python.org/library/httplib.html

http://docs.python.org/library/urllib.html

Good introduction is also a chapter from Dive into Python Chapter 11. HTTP Web Services

Answer 3

Use urllib.request in python 3.x, and urllib2 in python 2.x