views:

34

answers:

1

I just saw this

http://www.scribd.com/doc/13054288/Tap-Workflows

What I need is a replacement of make to perform computations that are dependent on each other but whose dependency is not only on something like the filename. I already posted a question about it a while ago, but I did not get satisfying answers, although I accepted one which solved my immediate problem.

Now, my question is: I looked everywhere, tried the most expected searches of "pipeline engine" and "workflow engine" but I still cannot find something that is, in my opinion, a trivial tool: something like make, but not focused on compiling files. I have one or more files, one or more database columns, one or more remote data provider, and I need a commandline tool that allows me to generate pictures, perform computations, create movies, insert new rows into a database. I am amazed by the fact that something so vital for any website with a computational backend is basically nonexistent, to the best of my knowledge. How is it possible? Do I really need to start coding my own python workflow engine, or it is out there and I just can't find it?

I am not alone in this quest

http://stackoverflow.com/questions/704834/does-anyone-know-about-workflow-frameworks-libraries-in-python

Fact is that there's confusion between human workflow and process workflow. I don't need to coordinate a group of people for a task. I need to write build all_stuff file.input and get all my stuff done.

A: 

Could you explain what the tools in the question you linked to don't do for you?

If you want to (ab)use a Python-based software build tool, you could try Paver or Buildout.

On the heavyweight side there's SCons, but I've not heard particularly good things about it.

Nicholas Riley
That's exactly the point. To achieve what I need I need to __ab__use a build system which is focused on software distribution and packaging.
Stefano Borini
goflow is a django tool to administer human workflows, not computer process workflows.
Stefano Borini
Yeah, I realized that about 10 seconds after posting the answer. Ah well.
Nicholas Riley
Anyway, if what you're doing doesn't vary that much in principle from building software, you can probably just use an existing tool and ignore the standard "recipes". Or write your own. :-)
Nicholas Riley
I can't believe there is nothing. We go from nothing to Accelrys Pipeline Pilot ( http://accelrys.com/flash/pp/index.html ) which makes coffee. That's plain impossible there's nothing in between.
Stefano Borini