What problems have you solved using a Workflow Engine?

Question

I'd like to know about specific problems you - the SO reader - have solved using Workflow Engines and what libraries/frameworks you used if you didn't roll your own. I'd also like to know when a Workflow Engine wasn't the best choice and if/how you chose something simpler, like a TaskList/WorkList/Task-Management type application using state machines.

Questions:

• What problems have you used workflow engines to solve?
• What libraries/frameworks did you use?
• When did a simpler State Machine/Task Management like system suffice?
• Bonus: How did/do you make the distinction between Task Management and Workflow Engine?

I'm looking for first-hand experiences.

Some of the resources I've checked out:

• Ruote and State Machine != Workflow Engine
• StonePath and Docs
• Creating and Managing Worklist Task Plans with Oracle
• Design and Implementation of a Workflow Engine - Thesis
• What to use Windows Workflow Foundation For
• JBoss jBPM Docs

Answer 1

I rolled my own workflow engine to support phased processing of documents - cataloging, sending for image processing (we work with redaction sw), if needed sending to validation, then release and finally shipping back to the client. In our case we have a truckload of documents to process so sometimes we need to run each service separately to control delivery and resources usage. Simple in concept but high performance and distributed processing needed, and we could't find any off the shelf product that fit the bill for us.

Answer 2

I'm biased, I'm one of the authors of ruote.

variant 1) state machine attached to a resource (document, order, invoice, book, piece of furniture).

variant 2) state machine attached to a virtual resource named a task

variant 3) workflow engine interpreting workflow definitions

Now your question is tagged "BPM" we can be expanded into "Business Process management". How does that kind of management occur in each of the variant ?

In variant 1, the business process (or workflow) is scattered in the application. The state machine attached to the resource enforces some of the aspects of the workflow, but only those related to the resource. There may be other resources with their own state machine following the same business process.

In variant 2, the workflow can be concentrated around the task resource and represented by the state machine around that resource.

In variant 3, the workflow is enacted by interpreting a resource called a workflow definition (or business process definition).

What happens when the business process changes ? Is it worth having a workflow engine where business processes are manageable resources ?

Most of the state machine libraries have 1 set states + transitions. Workflow engines are, most of them, workflow definition interpreters and they allow multiple different workflows to run together.

What will be the cost of changing the workflow ?

The variants are not mutually exclusive. I have seen many examples where a workflow engine changes the state of multiple resources some of them guarded by state machines.

I also use variant 3 + 2 a lot, for human tasks : the workflow engine, at some points when running a process instance, hands a task (workitem) to a human participant (resource task is created and placed in state 'ready').

You can go a long way with variant 2 alone (the task manager variant).

We could also mention variant 0), where there is no state machine, no workflow engine, and the business process(es) are scattered and/or hardcoded in the application.

You can ask many questions, but if you don't take the time to read the answers and don't take the time to try out and experiment, you won't go very far, and will never acquire any flair for when to use this or that tool.

Answer 3

On a previous project I was working on i added some Workflow type rules to a set of Government Forms in the Healhcare industry.

Forms needed to be filled out by the end user , and depending on some answers other Forms were scheduled to be filled out at a later date. There were also external events that would cancel scheduled Forms or schedule new ones.

Sample Flow :

Patient Admitted -> Schedule Initial Assessment FOrm -> Schedule Quarterly Review Form -> Patient Died -> Cancel Review -> Schedule Discharge Assessment Form

Many other rules were based on things such as Patient age, where they were being admitted etc.

This was an ASP.NET app, the rules were basically a table in the database. I added scripting, so a script would run on Form completion to determine what to do next. This was a horrid design, and would have been perfect for a proper Workflow engine.

Answer 4

I'm biased as well, as I am the main author of StonePath.

I have developed workflow applications for the U.S. State Department, the Geneva Centre for Humanitarian Demining, several fortune 500 clients, and most recently the Washington DC Public School System. Every time I have seen a 'workflow engine' that tried to be the one master reference for business processes, I have seen an organization fighting itself to work around the tool. This may be due to fact that these solutions have always been vendor/product driven, and then end up with a tactical team of 'consultants' constantly feeding the app... but because of this, I tend to react negatively when I hear the benefits of process-based tools that promise to 'centralize the workflow definitions in one place and make them repeatable'.

That said, I very much like Ruote - I have been following that project for some time and should I need that kind of solution, it will be the next tool I'll be willing to try. StonePath has a very different purpose than ruote - where Ruote is useful to Ruby in general, StonePath is aimed at Rails, the web framework written in Ruby. Where Ruote is about long-lived business processes and their associated definitions, StonePath is about managing State-based workflow and tasking. Frankly, I think the distinction from the outside looking in might be subtle - many times the same kinds of business processes can be represented either way - the state-and-task-based model tends to map to my mental model though.

Let me describe the highlights of a state-based workflow. In short, imagine a workflow revolving around the processing of something like a mortgage loan or a passport renewal. As the document moves 'around the office', it travels from state to state. Imagine if you are responsible for the document, and your boss asked you every few hours for a status update, and wanted a brief answer... you'd say things like "It is in data entry"... "We are checking the applicant's credentials now"... "we are awaiting quality review"... "We are done"... and so on. These are the states in a state-based workflow. We move from state to state via transitions - like "approve", "apply", kickback", "deny", and so on. these tend to be action verbs. Things like this are modeled all the time in software as a state machine.

The next part of a state/task-based workflow is the creation of tasks. A Task is a unit of work, typically with a due date and handling instructions, that connects a work item (the loan application or passport renewal, for instance), to a users "in box". Tasks can happen in parallel with each other or sequentialy, and we can create tasks automatically when we enter states, create tasks manually as people realize work needs to get done, and require tasks be complete before we can move onto a new state. All of this kind of behavior is optional, and part of the workflow definition.

The rabbit hole can go a lot deeper than this, and I wrote an article about it for Issue #4 of PragPub, the Pragmatic Programmer's Magazine. Check out the reo link above for an updated PDF of that article.

In working with StonePath the last few months, I have found that the state based model maps really well to restful web architectures - in particular, the tasks and state transitions map nicely as nested resources. Expect to see future writing from me on this subject.