How do you do string interpolation with a URL that contains formating characters?

Question

I'm trying to use URLLIB2 to open a URL and read back the contents into an array. The issue seems to be that you cannot use string interpolation in a URL that has formating characters such as %20 for space, %3C for '<'. The URL in question has spaces and a bit of xml in it.

My code is pretty simple, looks something like this:

#Python script to fetch NS Client Policies using GUID

import sys
import urllib2

def GetPolicies(ns, guid):
    ns = sys.argv[1]
    guid = sys.argv[2]
    fetch = urllib2.urlopen('http://%s/Altiris/NS/Agent/GetClientPolicies.aspx?xml=%3Crequest%20configVersion=%222%22%20guid=%22{%s}%22') % (ns, guid)

I've shortened the URL for brevity but you get the general idea, you get a 'Not enough arguments for format string' error since it assumes you're wanting to use the %3, %20, and other things as string interpolations. How do you get around this?

Edit: Solution requires Python 2.6+, 2.5 or prior has no support for the string.format() method

Answer 1

Use the .format method on the string instead. From its documentation:

str.format(*args, **kwargs) 
Perform a string formatting operation. The string on which this method is called can contain literal text or replacement fields delimited by braces {}. Each replacement field contains either the numeric index of a positional argument, or the name of a keyword argument. Returns a copy of the string where each replacement field is replaced with the string value of the corresponding argument.

>>> "The sum of 1 + 2 is {0}".format(1+2)
'The sum of 1 + 2 is 3'

While we all sin by sticking to % as we're used to from C, the format method is really a more robust method of interpolating values into strings.

Answer 2

You can double the % signs

url = 'http://%s/Altiris/NS/Agent/GetClientPolicies.aspx?xml=%%3Crequest%%20configVersion=%%222%%22%%20guid=%%22{%s}%%22' % (ns, guid)

or you can use .format() method

url = 'http://{hostname}/Altiris/NS/Agent/GetClientPolicies.aspx?xml=%3Crequest%20configVersion=%222%22%20guid=%22{id}%%2''.format(hostname=ns, id=guid)

Answer 3

Build your string up in steps, doing each encoding layer separately. Much more manageable than trying to cope with the multiple levels of escaping in one go.

xml= '<request configVersion="2" guid="{%s}"/>' % cgi.escape(guid, True)
query= 'xml=%s' % urllib2.quote(xml)
url= 'http://%s/Altiris/NS/Agent/GetClientPolicies.aspx?%s' % (ns, query)
fetch= urllib2.urlopen(url)

Answer 4

If you're trying to build the url yourself use urllib.urlencode. It will deal with a lot of the quoting issues for you. Just pass it a dict of the info you want:

from urllib import urlencode

args = urlencode({'xml': '<',
           'request configVersion': 'bar',
               'guid': 'zomg'})

As for replacing hostname in the base of your url string, just do what everyone else said and use the %s formatting. The final string could be somthing like:

print 'http://%s/Altiris/NS/Agent/GetClientPolicies.aspx?%s' % ('foobar.com', args)