Running applictions via ruby and multi-core support? (OSX)

Question

Hi All,

I'm looking for some tutorials/resources/tips that will show me how to run applications via a ruby script. I have several small tools that we use in our day to day operations that I want to manage their tasks in one ruby script.

Basically what I'm trying to do is:

1. run app via ruby script.
2. (wait for result) get result code (success, or error msg)
3. if ok, start the app on its next task.

Also each of the tasks are independent so I'd like to take advantage of the 8 cores on my MacPro and run 8 instances at a time.

Any resources you could point me towards would be greatly appreciated!

Answer 1

I just found this yesterday, it might help you - http://github.com/rightscale/right_popen, which relies on EventMachine: http://rubyeventmachine.com/. Writing your script in an evented or "reactor" style will save you some headaches. Think of it as writing javascript with some callbacks.

Something like this (I'm not familiar with EM, so pseudo-code):

do_something do |event|
  event.success{ next_script }
  event.failure{ report_failure }
end

Answer 2

First: Use MacRuby. The latest MacRuby has Grand Central dispatching built in -- that's the Snow Leopard spiffiness that automatically manages multicore threading, etc. You may not need all that, but if you know it's only going to run on a Mac, why not?

Second: Get the 1.9 edition of the Pickaxe book (that's Programming Ruby 1.9) and read the entire chapter on threads, fibers, and processes.

Third: Adapt what you learn to your own needs, but I'd probably approach the problem by spinning off a new Thread for each task and then running a system or `backtick` call for each utility within its own thread. Ruby 1.9 has real threads (and MacRuby has better ones), so these really would all run at once -- and interpreter locking doesn't matter at all if you're mostly waiting on external processes.

(If you didn't need to pay attention to the results of the calls and didn't need to loop, the Pickaxe's example of exec('foo') if fork.nil? would be even simpler. I've used it many times for things like backing up multiple directories at once, etc.)

Colin's suggestion of EventMachine would also work and would be as fast as you could possibly get if you implemented it right (and if your workflow made sense for evented async I/O), but would be far more complex and fiddly. Since you aren't trying to scale out to the entire universe, you just want to get some local utility tasks done, I think threads will do you fine and will make for simpler, more readable and maintainable code.