Capturing system command output as a string

Question

Perl and PHP do this with backticks. For example:

$output = `ls`;

This code returns a directory listing into the variable $output. A similar function, system("ls"), returns the operating system return code for the given command. I'm talking about a variant that returns whatever the command prints to stdout. (There are better ways to get the list of files in a directory; the example code is an example of this concept.)

How do other languages do this? Is there a canonical name for this function? (I'm going with "backtick"; though maybe I could coin "syslurp".)

Answer 1

Perl:

$output = `foo`;

ADDED: This is really a multi-way tie. The above is also valid PHP, and Ruby, for example, uses the same backtick notation as well.

Answer 2

Mathematica:

output = Import["!foo", "Text"];

Answer 3

Python:

import os
output = os.popen("foo").read()

Answer 4

In shell

OUTPUT=`ls`

or alternatively

OUTPUT=$(ls)

This second method is better because it allows nesting, but isn't supported by all shells, unlike the first method.

Answer 5

Ruby: either backticks or the '%x' builtin syntax.

puts `ls`;
puts %x{ls};

Answer 6

In PHP

$output = `ls`;

or

$output = shell_exec('ls');

Answer 7

An alternative method in perl

$output = qx/ls/;

This had the advantage that you can choose your delimiters, making it possible to use ` in the command (though IMHO you should reconsider your design if you really need to do that). Another important advantage is that if you use single quotes as delimiter, variables will not be interpolated (a very useful)

Answer 8

Erlang:

os:cmd("ls")

Answer 9

Yet another way to do it in Perl (TIMTOWTDI)

$output = <<`END`;
ls
END

This is specially useful when embedding a relatively large shell script in a Perl program

Answer 10

[At the request of Alexman and dreeves -- see comments --, you will find at this DZones Java Snippet page a full version Os-independent for making, in this instance, a 'ls'. This is a direct answer to their code-challenge.
What follows below is just the core: Runtime.exec, plus 2 thread to listen to stdout and stderr. ]

Java "Simple!":

E:\classes\com\javaworld\jpitfalls\article2>java GoodWindowsExec "dir *.java"
Executing cmd.exe /C dir *.java
...

Or in java code

String output = GoodWindowsExec.execute("dir");

But to do that, you need to code...
... this is embarrassing.

import java.util.*;
import java.io.*;
class StreamGobbler extends Thread
{
    InputStream is;
    String type;
    StringBuffer output = new StringBuffer();

    StreamGobbler(InputStream is, String type)
    {
        this.is = is;
        this.type = type;
    }

    public void run()
    {
        try
        {
            InputStreamReader isr = new InputStreamReader(is);
            BufferedReader br = new BufferedReader(isr);
            String line=null;
            while ( (line = br.readLine()) != null)
                System.out.println(type + ">" + line);
                output.append(line+"\r\n")
            } catch (IOException ioe)
              {
                ioe.printStackTrace();  
              }
    }
    public String getOutput()
    {
        return this.output.toString();
    }
}
public class GoodWindowsExec
{
    public static void main(String args[])
    {
        if (args.length < 1)
        {
            System.out.println("USAGE: java GoodWindowsExec <cmd>");
            System.exit(1);
        }
    }
    public static String execute(String aCommand)
    {
        String output = "";
        try
        {            
            String osName = System.getProperty("os.name" );
            String[] cmd = new String[3];
            if( osName.equals( "Windows 95" ) )
            {
                cmd[0] = "command.com" ;
                cmd[1] = "/C" ;
                cmd[2] = aCommand;
            }
            else if( osName.startsWith( "Windows" ) )
            {
                cmd[0] = "cmd.exe" ;
                cmd[1] = "/C" ;
                cmd[2] = aCommand;
            }

            Runtime rt = Runtime.getRuntime();
            System.out.println("Executing " + cmd[0] + " " + cmd[1] 
                               + " " + cmd[2]);
            Process proc = rt.exec(cmd);
            // any error message?
            StreamGobbler errorGobbler = new 
                StreamGobbler(proc.getErrorStream(), "ERROR");            

            // any output?
            StreamGobbler outputGobbler = new 
                StreamGobbler(proc.getInputStream(), "OUTPUT");

            // kick them off
            errorGobbler.start();
            outputGobbler.start();

            // any error???
            int exitVal = proc.waitFor();
            System.out.println("ExitValue: " + exitVal);   

            output = outputGobbler.getOutput();
            System.out.println("Final output: " + output);   

        } catch (Throwable t)
          {
            t.printStackTrace();
          }
        return output;
    }
}

Answer 11

Python

from subprocess import check_output as qx

output = qx(['ls', '-lt'])

Python <2.7 or <3.1

Extract subprocess.check_output() from subprocess.py or adapt something similar to:

import subprocess

def cmd_output(args, **kwds):
  kwds.setdefault("stdout", subprocess.PIPE)
  kwds.setdefault("stderr", subprocess.STDOUT)
  p = subprocess.Popen(args, **kwds)
  return p.communicate()[0]

print cmd_output("ls -lt".split())

The subprocess module has been in the stdlib since 2.4.

Answer 12

Yet another way (or 2!) in Perl....

open my $pipe, 'ps |';
my @output = < $pipe >;
say @output;

open can also be written like so...

open my $pipe, '-|', 'ps'

/I3az/

Answer 13

C (with glibc extension):

#define _GNU_SOURCE
#include <stdio.h>
int main() {
    char *s = NULL;
    FILE *p = popen("ls", "r");
    getdelim(&s, NULL, '\0', p);
    pclose(p);
    printf("%s", s);
    return 0;
}

Okay, not really concise or clean. That's life in C...

Answer 14

J:

output=:2!:0'ls'

Answer 15

Haskell:

import Control.Exception
import System.IO
import System.Process
main = bracket (runInteractiveCommand "ls") close $ \(_, hOut, _, _) -> do
    output <- hGetContents hOut
    putStr output
  where close (hIn, hOut, hErr, pid) =
          mapM_ hClose [hIn, hOut, hErr] >> waitForProcess pid

With MissingH installed:

import System.Cmd.Utils
main = do
    (pid, output) <- pipeFrom "ls" []
    putStr output
    forceSuccess pid

This is an easy operation in "glue" languages like Perl and Ruby, but Haskell isn't.

Answer 16

Years ago I wrote a plugin for jEdit that interfaced to a native application. This is what I used to get the streams off the running executable. Only thing left to do is while((String s = stdout.readLine())!=null){...}:

/* File:    IOControl.java
 *
 * created: 10 July 2003
 * author:  dsm
 */
package org.jpop.io;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.PrintStream;

/**
 *  Controls the I/O for a process. When using the std[in|out|err] streams, they must all be put on
 *  different threads to avoid blocking!
 *
 * @author     dsm
 * @version    1.5
 */
public class IOControl extends Object {
    private Process process;
    private BufferedReader stdout;
    private BufferedReader stderr;
    private PrintStream stdin;

    /**
     *  Constructor for the IOControl object
     *
     * @param  process  The process to control I/O for
     */
    public IOControl(Process process) {
        this.process = process;
        this.stdin = new PrintStream(process.getOutputStream());
        this.stdout = new BufferedReader(new InputStreamReader(process.getInputStream()));
        this.stderr = new BufferedReader(new InputStreamReader(process.getErrorStream()));
    }

    /**
     *  Gets the stdin attribute of the IOControl object
     *
     * @return    The stdin value
     */
    public PrintStream getStdin() {
        return this.stdin;
    }

    /**
     *  Gets the stdout attribute of the IOControl object
     *
     * @return    The stdout value
     */
    public BufferedReader getStdout() {
        return this.stdout;
    }

    /**
     *  Gets the stderr attribute of the IOControl object
     *
     * @return    The stderr value
     */
    public BufferedReader getStderr() {
        return this.stderr;
    }

    /**
     *  Gets the process attribute of the IOControl object. To monitor the process (as opposed to
     *  just letting it run by itself) its necessary to create a thread like this: <pre>
     *. IOControl ioc;
     *.
     *. new Thread(){
     *.     public void run(){
     *.         while(true){    // only necessary if you want the process to respawn
     *.             try{
     *.                 ioc = new IOControl(Runtime.getRuntime().exec("procname"));
     *.                 // add some code to handle the IO streams
     *.                 ioc.getProcess().waitFor();
     *.             }catch(InterruptedException ie){
     *.                 // deal with exception
     *.             }catch(IOException ioe){
     *.                 // deal with exception
     *.             }
     *.
     *.             // a break condition can be included here to terminate the loop
     *.         }               // only necessary if you want the process to respawn
     *.     }
     *. }.start();
     *  </pre>
     *
     * @return    The process value
     */
    public Process getProcess() {
        return this.process;
    }
}

Answer 17

Don't forget Tcl:

set result [exec ls]

Answer 18

Perl, another way:

use IPC::Run3

my ($stdout, $stderr);
run3 ['ls'], undef, \$stdout, \$stderr
    or die "ls failed";

Useful because you can feed the command input, and get back both stderr and stdout separately. Nowhere near as neat/scary/slow/disturbing as IPC::Run, which can set up pipes to subroutines.

Answer 19

In C on Posix conformant systems:

#include <stdio.h> 

FILE* stream = popen("/path/to/program", "rw");
fprintf(stream, "foo\n"); /* Use like you would a file stream. */
fclose(stream);

Answer 20

Granted, it is not the smaller ( from all the languages available ) but it shouldn't be that verbose.

This version is dirty. Exceptions should be handled, reading may be improved. This is just to show how a java version could start.

Process p = Runtime.getRuntime().exec( "cmd /c " + command );
InputStream i = p.getInputStream();
StringBuilder sb = new StringBuilder();
for(  int c = 0 ; ( c =  i.read() ) > -1  ; ) {
    sb.append( ( char ) c );
}

Complete program below.

import java.io.*;

public class Test { 
    public static void main ( String [] args ) throws IOException { 
        String result = execute( args[0] );
        System.out.println( result );
    }
    private static String execute( String command ) throws IOException  { 
        Process p = Runtime.getRuntime().exec( "cmd /c " + command );
        InputStream i = p.getInputStream();
        StringBuilder sb = new StringBuilder();
        for(  int c = 0 ; ( c =  i.read() ) > -1  ; ) {
            sb.append( ( char ) c );
        }
        i.close();
        return sb.toString();
    }
}

Sample ouput ( using the type command )

C:\oreyes\samples\java\readinput>java Test "type hello.txt"
This is a sample file
with some
lines

Sample output ( dir )

 C:\oreyes\samples\java\readinput>java Test "dir"
 El volumen de la unidad C no tiene etiqueta.
 El número de serie del volumen es:

 Directorio de C:\oreyes\samples\java\readinput

12/16/2008  05:51 PM    <DIR>          .
12/16/2008  05:51 PM    <DIR>          ..
12/16/2008  05:50 PM                42 hello.txt
12/16/2008  05:38 PM             1,209 Test.class
12/16/2008  05:47 PM               682 Test.java
               3 archivos          1,933 bytes
               2 dirs            840 bytes libres

Try any

java Test netstat
java Test tasklist
java Test "taskkill /pid 416"

EDIT

I must admit I'm not 100% sure this is the "best" way to do it. Feel free to post references and/or code to show how can it be improved or what's wrong with this.

Answer 21

Why there is still no c# guy here :)

This is how to do it in C#. The built-in way.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;

namespace TestConsole
{
    class Program
    {
        static void Main(string[] args)
        {
            Process p = new Process();

            p.StartInfo.UseShellExecute = false;
            p.StartInfo.CreateNoWindow = true;
            p.StartInfo.RedirectStandardOutput = true;
            p.StartInfo.RedirectStandardError = true;
            p.StartInfo.FileName = "cmd";
            p.StartInfo.Arguments = "/c dir";
            p.Start();

            string res = p.StandardOutput.ReadToEnd();
            Console.WriteLine(res);
        }

    }
}

Answer 22

Icon/Unicon:

stream := open("ls", "p")
while line := read(stream) do { 
    # stuff
}

The docs call this a pipe. One of the good things is that it makes the output look like you're just reading a file. It also means you can write to the app's stdin, if you must.

Answer 23

C# 3.0, less verbose than this one:

using System;
using System.Diagnostics;

class Program
{
    static void Main()
    {
        var info = new ProcessStartInfo("cmd", "/c dir") { UseShellExecute = false, RedirectStandardOutput = true };
        Console.WriteLine(Process.Start(info).StandardOutput.ReadToEnd());
    }
}

Caveat: Production code should properly dispose the Process object...

Answer 24

Here's another Lisp way:

(defun execute (program parameters &optional (buffer-size 1000))
  (let ((proc (sb-ext:run-program program parameters :search t :output :stream))
        (output (make-array buffer-size :adjustable t :fill-pointer t 
                            :element-type 'character)))
    (with-open-stream (stream (sb-ext:process-output proc))
      (setf (fill-pointer output) (read-sequence output stream)))
    output))

Then, to get your string:

(execute "cat" '("/etc/hosts"))

If you want to run a command that creates prints a great deal of info to STDOUT, you can run it like this:

(execute "big-writer" '("some" "parameters") 1000000)

The last parameter preallocates a large amount of space for the output from big-writer. I'm guessing this function could be faster than reading the output stream one line at a time.

Answer 25

Lua:

    foo = io.popen("ls"):read("*a")

Answer 26

Clozure Common Lisp:

(with-output-to-string (stream)
   (run-program "ls" '("-l") :output stream))

LispWorks

(with-output-to-string (*standard-output*)
  (sys:call-system-showing-output "ls -l" :prefix "" :show-cmd nil))