What is the best way of running shell commands from a web based interface?

Question

Imagine a web application that allows a logged in user to run a shell command on the web server at the press of a button. This is relatively simple in most languages via some standard library os tools.

But if that command is long running you don't want your UI to hang. Again this is relatively easy to deal with using some sort of background process or putting the command to be executed onto a message queue (and maybe saving the output and status somewhere for later consumption). Just return quickly saving we'll run that and get back to you.

What I'd like to do is show the output of said web ui triggered shell command as it happens. So vertically scrolling text like when running in a terminal.

I have a vague idea of how I might approach this, streaming the output to a websocket perhaps and simply printing the output to screen.

What I'd like to ask is:

Are their any plugins, libraries or applications that already do this. Something I can either use or read the source of. Ideally an open source python/django or ruby/rails tool, but other stacks would be interesting too.

Answer 1

I'm not sure if it's what you want, but there are some web based ssh clients out there. If you care about security and really just want dynamic feedback, you could look into comet or just have a frame with its own http session that doesn't end until it's done printing.

Answer 2

web-based ssh client would work, into the host (there are java ssh clients out there).

Ruby has a web-based terminal: http://tryruby.org (link to the source is at the bottom of the page). You could also embed Ruby via jruby: http://tim.lossen.de/2007/03/jruby/applet.html http://github.com/jruby/jruby/blob/master/samples/irb-applet.html

Answer 3

I haven't heard of any libraries that do this, but you'll need to setup the system command and call out to the system. You will then need to "pump" the sysout and syserr standard inputs and pipe that data back out to your web client.

As an example for this style of problem, look into code snippits of how people use ruby/python/etc to transcode a video, i.e. http://kpumuk.info/ruby-on-rails/encoding-media-files-in-ruby-using-ffmpeg-mencoder-with-progress-tracking/ - my example was taken from this blog post.

class MediaFormatException < StandardError
end

def execute_mencoder(command)
  progress = nil
  IO.popen(command) do |pipe|
    pipe.each("r") do |line|
      if line =~ /Pos:[^(]*(s*(d+)%)/
        p = $1.to_i
        p = 100 if p > 100
        if progress != p
          progress = p
          print "PROGRESS: #{progress}n"
          $defout.flush
        end
      end
    end
  end
  raise MediaFormatException if $?.exitstatus != 0
end

I don't know if this example is pulling data from both sysout and syserr, but you will definitely need to be pulling data from both of those interfaces, typically if the buffer fills up, the executing command might hang or fail (I have experienced this with Python). This method will also look different if the only thing you do is return line to the web client - in a terminal, the progress indicator of ffmpeg/mencoder remains stationary on the bottom line, but this method will give you a long list of progress indicator updates. Pipe line out to your terminal and you'll see what I'm referring to.

Answer 4

Certainly not the best way to run shell commands, but likely the easiest:

#!/bin/sh

echo Content-Type: text/plain
echo

/usr/bin/uptime

http://www.sente.cc/scripts/uptime.cgi

Answer 5

So, I've tried to answer my own question with code as I couldn't find anything to quite fit the bill. Hopefully it's useful to anyone coming across the same problem.

Redbeard 0X0A pointed me in the general direction, I was able to get a stand along ruby script doing what I wanted using popen. Extending this to using EventMachine (as it provided a convenient way of writing a websocket server) and using it's inbuilt popen method solved my problem.

More details here http://morethanseven.net/2010/09/09/Script-running-web-interface-with-websockets.html and the code at http://github.com/garethr/bolt/