Creating an application to move files around ... am I looking for a service bus?

Question

We do a lot of document scanning, and I find myself now with a bunch of simple scripts that watch incoming folders for files, and then route the files to their final destination (i.e. they might end up in a Sharepoint library, or be encrypted and stored in an archive folder, etc).

I'm considering developing a rather simple application to consolidate this effort. Essentially it could be configured with inputs (i.e. folders to watch, mailboxes to check, etc), and then one or more actions that could be performed per input (i.e. copy to webdav folder, send to web service, send via email, etc).

Now this isn't a terribly difficult application to whip out, but I'm curious... I heard a podcast or something about "Mule", I believe, which is an ESB. And I don't know much about service buses, but it sounds to me like what I'm considering doing is at least similar. Is it?

Is there a lightweight framework / library / application that I can use to accomplish my task without a gigantic learning curve? Preferably Java-based, but C#/.Net isn't out of the question.

(P.S. I considered this for ServerFault, but it seems appropriate here...)

Answer 1

Spring Integration is a little more lightweight and has less of a learning curve. However, it is quite a bit less flexible. I'd say that Mule is the most powerful and flexible of the open source options right now, but it comes with a learning curve.

This article is a good intro to Spring Integration and the design patterns of EAI (Enterprise Application Integration) in general.

I have used Mule in similar situations in the past. What I will say about deciding whether to use it is as follows:

• it could definitely be overkill, depending on how many scripts you are maintaining and how many people need to help maintain them over time
• be prepared to experience a conceptual learning curve - try lots of experiments!
• check out the book Enterprise Integration Patterns for a deeper understanding of how to use any ESB most intelligently
• once you've mastered the Mule, you will have a seriously flexible toolkit for handling almost any integration problem quickly and robustly

So far, my experience with Mule has definitely been a net win. But it's always a tradeoff between:

• building your own, which you know will work but which may eventually grow in proportion, and which you will be the only one who can document and teach people about
• learning a new toolset, which may take time but which every future programmer on the project can do without having to rely on you

I tend to enjoy the learning curve, and be good at wrestling through issues when the framework isn't "cooperating" (meaning I don't understand it completely yet). I'm also usually working on a project that should outlive my involvement on it. So getting Mule involved has really opened doors and made our code much more collectively maintainable.

Other coders I know eschew frameworks altogether, insisting that it's more pragmatic not to use them. If you're in the latter camp you should definitely NOT get into Mule. Probably not Spring Integration, either!

Hope this helps :-)

Answer 2

I'm going to conjecture that the scripting required to get the tool to work is at least as complex as writing the application.