Has a system that incorporated a rule engine ever been TRULY successful?

Question

Our system (exotic commodity derivative trade capture and risk management) is being redeveloped shortly. One proposal that I have heard is that a rule engine will be incorporated to make it easier for the end-users (commodities traders, so fairly sophisticated) to make certain changes to the business logic.

I am a little skeptical of rules engines. The agilist in me wonders if they are just a technical solution to a process problem... ie. it takes too long for our developers to respond to the business's need for change. The solution to that problem should be a more collaborative approach to development, better test coverage, more agile practices all around.

Hearing about situations where a rule engine was truly a boon (especially in a trading environment) would be certainly helpful.

Answer 1

My experience is limited to (i)not much and (ii)prolog; but I can safely say that a rule engine can help you express propositional concepts much cleaner than procedural code.

Answer 2

I don't know if I'd say they're ever truly a boon, but I think they can certainly be valuable. I worked on a system for a few years in the insurance industry, where a rules engine was employed quite successfully to allow the business users to create rules that determined what policies were legal, depending on the state.

For instance, if you had to have a copay in certain states, or certain combinations of deductible and copay were not allowed, either because of product considerations, or because it was simply illegal due to state law.

The number of states that the company operated in, along with the constant change in rules (quarterly) would make this a dizzying coding practice. More importantly, it's not in the expertise of a programmer. It adds extra pointless communication where the end user is describing the rule to be put in effect to a programmer who is not an insurance industry expert like they are.

Designed correctly, a rules engine can still enable a workflow system that allows for good testing. In this case, the rules were stored in a database, and there were QA and PROD databases. So the BA's could test their rules in QA, and then promote them to PROD.

As with anything, its usually about the implementation, and not the actual technique.

Answer 3

For a while I worked for the PEATE distributed computing project which was developing a system for calculation of large scale, high volume atmospheric data. The system had three parts to it: the data manager, the scheduler, and the algorithm execution component. There could be any number of any of these components, all done through web services, but what it allowed for was for different researchers to execute arbitrary jobs against arbitrary data, and also allowed for different scheduling mechanisms to be plugged in as requirements changed.

I left the project before it got too far off the ground, but this seems like it could potentially fit the scenario, and serve as another example for some kind of rule engine. That being said, however, if the original developers are still going to be the one's making the algorithms to run, I can't see too much benefit in having a rule engine unless it handled a substantial overhead that each rule or algorithm would incur on it's own.

This sounds like a bit more involved than a simple rule engine, but such an architecture could feasibly apply to a rule engine as well.

Answer 4

Yes, Microsoft has a Business Rule Engine (BRE) in BizTalk that has been used successfully for years. I've heard that they've had clients buy BizTalk (very expensive) just for the BRE.

In my experience, the practicality of having a business user update the rules is slim to none. It usually takes a technical person to work the business rules editor.

Answer 5

The problem with many of these rule engines are the lack of speed and the fact that replacing or augmenting rules can break existing working rules in subtle ways. So you still have to re-test the system thoroughly after each rule change. So you're basically just exchanging one computer language for another one - one with a much smaller base of users. As another poster mentioned, I've yet to see a business analyst successfully use a rule engine. You need a programmer anyway.

Answer 6

I certainly have, but can't publicly talk about them, but its likely you have interacted with one several times this year ;)

I see it in 2 camps: the logic programmers and the business users. Different tools target different sets, some both. The successful cases of business users have only worked when it was a subset of the logic, and they also had a way to define test cases and run them themselves (and they are prepared to think logically). Logic programmers are rarer, but can often be found coming from non imperative programming backgrounds (they are also the sort of people that find functional programming intuitive).

Keep in mind at the end of the day even with visual tools, if you are telling a computer to do something it is still programming.

Answer 7

I've seen two applications that used the Blaze Rete engine from Fair Issac.

One application slammed thousands of rules into a single knowledge base, had terrible memory problems, has become a black box that few understand. I would not call that a success, but it is running in production.

Another application used decision trees to represent on the order of hundreds of questions on a medical form to disposition clients. It was done so elegantly that business people can update the rules as needed, without having to involve a developer. (Still has to be deployed by one, though.) I'd call that a great success.

So it depends on how well focused the problem is, the size of the rule set, the knowledge of the developers. My prejudice is that simply making a rules engine a single point of failure and dumping rules into it probably isn't a good approach. I'd start with a data-driven or table-driven approach and grow that until a rules engine was needed. I'd also strive to encapsulate the rules engine as part of the behavior of an object. I'd hide the rules engine from users and try to partition the rules space into the domain model.

%

Answer 8

A rule engine is little more than something that executes declarative statements. They come with two primary advantages (that I see):

1. Your business logic is maintained from a single place instead of being sprinkled throughout application code. Technically, a well-designed application should already do this with the architecture, regardless of a rule engine being present or not.
2. You need to worry [less] about dependencies between declarative statements. The rule engine should be smart enough to decide the order to run rules based on dependencies. You may find that some rule engines support a sequential ordering of rules within a ruleset or calling rulesets (groups of rules) in a particular order, but this isn't really in the spirit of declarative programming. Many rule engines use Rete (an algorithm) to decide when to schedule the execution of declarative statements.

I suspect most, if not all, rule engines add more overhead than if you were to write the best possible program that doesn't use a rule engine. This is similar to how writing code in assembly is generally faster than a compiler (but you usually don't write assembly because it's more convenient and productive to use higher-level abstractions).

If you were to stop here, then you would probably use programmers to maintain rules and use a rule engine as a convenient way to build a business logic tier in your application. Some rule engines offer something called templates that let you define templates for rules. The advantage here is that non-technical users are supposed to be able to write their own rules and modify existing rules.

A rule engine is one more tool in your tool chest that, when used properly, can be valuable.

Answer 9

Rules engines are routinely used in the insurance business. I've worked on systems with hundreds (600ish) rules that were implemented in a rules engine. It worked very well.

Answer 10

Do you have a credit rating? A FICO score, perhaps? That's Fair Isaac COrporation, the developers of the Blaze rules engine.

Answer 11

I work with lots of vendors in the space and one of the great things about this is that I get to talk to lots of their customers. So, yes, hundreds of companies have got exactly the benefits they were promised - increased agility, better business/IT collaboration, easier regulatory compliance, better consistency of decision making, lower maintenance costs, faster times to market etc.

Over and over again, across all the major vendors and the open source players, I see that used correctly - to automate and improve high-volume operational decisions with many rules, rules that change a lot, rules that interact in complex ways or rules with a high business domain content - business rules management systems work.

Really.