In Python, how I do use subprocess instead of os.system?

Question

I have a Python script that calls an executable program with various arguments (in this example, it is 'sqlpubwiz.exe' which is the "Microsoft SQL Server Database Publishing Wizard"):

import os

sqlpubwiz = r'"C:\Program Files\Microsoft SQL Server\90\Tools\Publishing\sqlpubwiz.exe"'
server = 'myLocalServer'
database = 'myLocalDatabase'
connection_values = ['server=' + server, 'database=' + database, 'trusted_connection=true']
connection_string = ';'.join(connection_values)
dbms_version = '2000'
sqlscript_filename = 'CreateSchema.sql'

args = [
        sqlpubwiz,
        'script',
        '-C ' + connection_string,
        sqlscript_filename,
        '-schemaonly',
        '-targetserver ' + dbms_version,
        '-f',
]

cmd = ' '.join(args)
os.system(cmd)

This code runs properly but I have would like to get into the habit of using subprocess since it is intended to replace os.system. However, after a few failed attempts, I can not seem to get it work properly.

How would the above code look like if it was converted to use subprocess in place of os.system?

Answer 1

import subprocess
p=subprocess.Popen(args, stdout=subprocess.PIPE)
print p.communicate()[0]

It would look pretty much the same. But the path should not be r'"whatever the path is"'. Because that gives me an error. You want "the path with escaped backslashes" or r'the path without escaping'.

Also args should be of the form ['-arg', 'args'] instead of ['arg argsval'].

Answer 2

Remove quotes from the name of the executable. On the first line of your example, instead of

sqlpubwiz = r'"C:\Program Files\Microsoft SQL Server\90\Tools\Publishing\sqlpubwiz.exe"'

use:

sqlpubwiz = r'C:\Program Files\Microsoft SQL Server\90\Tools\Publishing\sqlpubwiz.exe'

That's because you don't have to escape anything since a shell won't be involved.

Then just use subprocess.call(args) (don't join the args, pass them as a list)

If you want to capture the output (os.system can't do it) just follow subprocess documentation:

result = subprocess.Popen(args, stdout=subprocess.PIPE).communicate()[0]
print result

Answer 3

Please remember that os.system uses the shell, and so you must really pass

shell=True

to the Popen constructor/call to emulate it properly. You may not actually need a shell, of course, but there it is.

Answer 4

This isnt an answer directly to your question but I thought it might be helpful.

In case you ever want more granular control over what is returned for exception handling etc, you can also check out pexpect. I have used it in situations where the process I was calling didn't necessarily exit with normal status signals, or I wanted to interact with it more. It's a pretty handy function.

Answer 5

Below is my revised code based on Carlos Rendon (and nosklo) help and suggestions:

# import os
import subprocess    

sqlpubwiz = r'C:\Program Files\Microsoft SQL Server\90\Tools\Publishing\sqlpubwiz.exe'
server = 'myLocalServer'
database = 'myLocalDatabase'
connection_values = ['server=' + server, 'database=' + database, 'trusted_connection=true']
connection_string = ';'.join(connection_values)
dbms_version = '2000'
sqlscript_filename = 'CreateSchema.sql'       

args = [
            sqlpubwiz,
            'script',
            '-C',
            connection_string,
            sqlscript_filename,
            '-schemaonly',
            '-targetserver',
            dbms_version,
            '-f',
    ]   

# cmd = ' '.join(args)
# os.system(cmd)

subprocess.call(args)

(Note: The original argument values that contained spaces needed to be converted into separate list items.)

Answer 6

FYI, subprocess has a list2cmdline() function that will let you see the string that Popen will be using.

Your version gives:

'"C:\\Program Files\\Microsoft SQL Server\\90\\Tools\\Publishing\\sqlpubwiz.exe" script "-C server=myLocalServer;database=myLocalDatabase;trusted_connection=true" CreateSchema.sql -schemaonly "-targetserver 2000" -f'

with extra quotes around "-C server=myLocalServer;database=myLocalDatabase;trusted_connection=true" and "-targetserver 2000".

Properly formatted:

args = [
        sqlpubwiz,
        'script',
        '-C', connection_string,
        sqlscript_filename,
        '-schemaonly',
        '-targetserver', dbms_version,
        '-f',
]

gives:

'"C:\\Program Files\\Microsoft SQL Server\\90\\Tools\\Publishing\\sqlpubwiz.exe" script -C server=myLocalServer;database=myLocalDatabase;trusted_connection=true CreateSchema.sql -schemaonly -targetserver 2000 -f'

Also, minor point, but it's a good habit to make sequences such as args that don't need to be mutable into tuples instead of lists.

Answer 7

Windows commands will accept forward slashes '/' in place of backslashes in pathnames, so you can use the former to avoid escaping backslashes in your command strings. Not exactly an answer to your question, but perhaps useful to know.