Convert a curl POST request to Python only using standard libary

Question

I would like to convert this curl command to something that I can use in Python for an existing script.

curl -u 7898678:X -H 'Content-Type: application/json' \
-d '{"message":{"body":"TEXT"}}' http://sample.com/36576/speak.json

TEXT is what i would like to replace with a message generated by the rest of the script.(Which is already working reasonable, although I don't think it is following best practices or particularity reliable. - need to find out how to properly learn to program (ie not use google for assembling stuff))

I would like this to work with the standard library if possible.

Thanks

Answer 1

Take a look at pycurl http://pycurl.sourceforge.net/

Answer 2

I would like this to work with the standard library if possible.

The standard library provides urllib and httplib for working with URLs:

>>> import httplib, urllib
>>> params = urllib.urlencode({'apple': 1, 'banana': 2, 'coconut': 'yummy'})
>>> headers = {"Content-type": "application/x-www-form-urlencoded",
...            "Accept": "text/plain"}
>>> conn = httplib.HTTPConnection("example.com:80")
>>> conn.request("POST", "/some/path/to/site", params, headers)
>>> response = conn.getresponse()
>>> print response.status, response.reason
200 OK

If you want to execute curl itself, though, you can just invoke os.system():

import os
TEXT = ...
cmd = """curl -u 7898678:X -H 'Content-Type: application/json'""" \
  """-d '{"message":{"body":"%{t}"}}' http://sample.com/36576/speak.json""" % \
  {'t': TEXT}

If you're willing to relax the standard-library-only restriction, you can use PycURL. Beware that it isn't very Pythonic (it's pretty much just a thin veneer over libcurl), and I'm not sure how compatible it is with Python 3.

Answer 3

With your example John, how exactly do I get it to work the same way as curl (this is to run on a machine without curl).

In other words where does the password go this service doesn't use a password rather just a string for user name and a dummy password?

Where do I put my message?

What is the port 80 stuff for?

Thanks for your help

Answer 4

While there are ways to handle authentication in urllib2, if you're doing Basic Authorization (which means effectively sending the username and password in clear text) then you can do all of what you want with a urllib2.Request and urllib2.urlopen:

import urllib2

def basic_authorization(user, password):
    s = user + ":" + password
    return "Basic " + s.encode("base64").rstrip()

req = urllib2.Request("http://localhost:8000/36576/speak.json",
                      headers = {
        "Authorization": basic_authorization("7898678", "X"),
        "Content-Type": "application/json",

        # Some extra headers for fun
        "Accept": "*/*",   # curl does this
        "User-Agent": "my-python-app/1", # otherwise it uses "Python-urllib/..."
        },
                      data = '{"message":{"body":"TEXT"}}')

f = urllib2.urlopen(req)

I tested this with netcat so I could see that the data sent was, excepting sort order, identical in both cases. Here the first one was done with curl and the second with urllib2

% nc -l 8000
POST /36576/speak.json HTTP/1.1
Authorization: Basic Nzg5ODY3ODpY
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4 OpenSSL/0.9.8k zlib/1.2.3
Host: localhost:8000
Accept: */*
Content-Type: application/json
Content-Length: 27

{"message":{"body":"TEXT"}} ^C

% nc -l 8000
POST /36576/speak.json HTTP/1.1
Accept-Encoding: identity
Content-Length: 27
Connection: close
Accept: */*
User-Agent: my-python-app/1
Host: localhost:8000
Content-Type: application/json
Authorization: Nzg5ODY3ODpY

{"message":{"body":"TEXT"}}^C

(This is slightly tweaked from the output. My test case didn't use the same url path you used.)

There's no need to use the underlying httplib, which doesn't support things that urllib2 gives you like proxy support. On the other hand, I do find urllib2 to be complicated outside of this simple sort of request and if you want better support for which headers are sent and the order they are sent then use httplib.

Answer 5

Thank-you dalke

Your solution is working although the terminal is printing out a lot of errors also how to I make it so that I can do this

speak('message')

and then the command runs replacing TEXT with message

Answer 6

Thanks every

this works

import urllib2

def speak(status):

    def basic_authorization(user, password):
        s = user + ":" + password
        return "Basic " + s.encode("base64").rstrip()

    req = urllib2.Request("http://example.com/60/speak.json",
                  headers = {
       "Authorization": basic_authorization("2345670", "X"),
     "Content-Type": "application/json",


        "Accept": "*/*",   
        "User-Agent": "my-python-app/1", 
        },
                      data = '{"message":{"body":'+ status +'}}')

f = urllib2.urlopen(req)


speak('Yay')