Add params to given URL in Python

Question

Suppose I was given by some URL.
It might already have GET parameters (e.g. http://example.com/search?q=question) or not (e.g. http://example.com/).

And now I need to add some parameters to it like {'lang':'en','tag':'python'}. In first case I'm going to have http://example.com/search?q=question&lang=en&tag=python and in second — http://example.com/search?lang=en&tag=python.

Is there any standard way to do this?

Answer 1

Yes: use urllib.

From the examples in the documentation:

>>> import urllib
>>> params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
>>> f = urllib.urlopen("http://www.musi-cal.com/cgi-bin/query?%s" % params)
>>> print f.geturl() # Prints the final URL with parameters.
>>> print f.read() # Prints the contents

Answer 2

Use the various urlparse functions to tear apart the existing URL, urllib.urlencode() on the combined dictionary, then urlparse.urlunparse() to put it all back together again.

Or just take the result of urllib.urlencode() and concatenate it to the URL appropriately.

Answer 3

You want to use URL encoding if the strings can have arbitrary data (for example, characters such as ampersands, slashes, etc. will need to be encoded).

Check out urllib.urlencode:

>>> import urllib
>>> urllib.urlencode({'lang':'en','tag':'python'})
'lang=en&tag=python'

Answer 4

There are couple quirks with urllib and urlparse modules. Here's working example:

import urllib
import urlparse

url = "http://stackoverflow.com/search?q=question"
params = {'lang':'en','tag':'python'}

url_parts = list(urlparse.urlparse(url))
query = dict(urlparse.parse_qsl(url_parts[4]))
query.update(params)

url_parts[4] = urllib.urlencode(query)

print urlparse.urlunparse(url_parts)