Rules Engine - pros and cons

Question

I'm auditing a project that uses what is called a Rules Engine. In short, it's a way to externalize business logic from application code.

This concept is entirely new to me and I'm pretty skeptical about it. After hearing people talk about Anemic Domain Models for the past few years, I'm questioning the Rules Engine Approach. To me they seem like a great way to WEAKEN a domain model. For example say I'm doing a java webapp interacting with a Rules Engine. Then I decide I want to have an Android app based on the same domain. Unless I want the Android app to interact with the Rules Engine as well, I'm going to have to miss out on whatever business logic was already written.

As I don't have any experience with them yet, just curiosity, I was interested to hear about the pros and cons are in using a Rules Engine? The only pro that I can think of is that you don't need to rebuild your entire Application just to change some business rule (but really, how many apps really have that many changes?). But using a Rules Engine to solve that problem kind of sounds to me like putting a band-aid over a shotgun wound.

UPDATE - since writing this, the god himself, Martin Fowler, has blogged about using a Rules engine.

Opsis

Are you looking into any third party products or are you going to roll your own? – Enterogastrone

That is a great article from Martin Fowler, thanks! – Fulltime

you are right - they are very anti-OO. They come from a time when OO wasn't common (that partly explains it) but yes they very much want to work with records/value objects which are "anemic" as you say. This isn't necessarily a bad thing, but it is what it is. If you don't like that - you won't like rule engines at all. – Perish

I would also examine whether one really wants the (forward chaining) rule engine or one actually wants the semantically clear (in the sense of: at least one knows what they compute in the first place) logic programs of Prolog or very much simpler Datalog (or one can go into more rarified territories and take a look at Transaction Logic). If one wants reactivity ("whenever something changes in the database, do something, anything!"), a look a a Tuplespace/Lindaspace implementation linked to good old OO programming in $PREFERRED_LANGUAGE might be worth it. – Krongold

Answer 1

Most rule engines that I have seen are viewed as a black box by system code. If I were to build a domain model, I would probably want certain business rules to be intrinsic to the domain model, e.g. business rules that tell me when an object has invalid values. This allows multiple systems to share the domain model without duplicating business logic. I could have each system use the same rule service to validate my domain model, but this appears to weaken my domain model (as was pointed out in the question). Why? Because instead of consistently enforcing my business rules across all systems at all times, I am relying on system programmers to determine when the business rules should be enforced (by calling the rule service). This may not be a problem if the domain model comes to you completely populated, but can be problematic if you're dealing with a user interface or system that changes values in the domain model over its lifetime.

There is another class of business rules: decision making. For example, an insurance company may need to classify the risk of underwriting an applicant and arrive at a premium. You could place these types of business rules in your domain model, but a centralized decision for scenarios like this are usually desirable and, actually, fit quite well into a service-oriented architecture. This does beg the question of why a rule engine and not system code. The place where a rule engine may be a better choice is where business rules responsible for the decision change over time (as some other answers have pointed out).

Rule engines usually allow you to change rules without restarting your system or deploying new executable code (regardless of what promises you receive from a vendor, do make sure you test your changes in a non-production environment because, even if the rule engine is flawless, humans are still changing the rules). If you're thinking, "I can do that by using a database to store values that change", you're right. A rule engine is not a magical box that does something new . It is intended to be a tool that provides a higher level of abstraction so you can focus less on reinventing the wheel. Many vendors take this a step further by letting you create templates so that business users can fill in the blanks instead of learning a rule language.

One parting caution about templates: templates can never take less time than writing a rule without a template because the template must, at the bare minimum, describe the rule. Plan for a higher initial cost (the same as if you were to build a system that used a database to store values that change vs. writing the rules in directly in system code) - the ROI is because you save on future maintenance of system code.

Cohbath

I'm coming to this discussion 2 years late, so if you're still around Aaron, can you describe how the above benefits (not having to restart the system for changes; abstraction, decoupling and reuse of business rules) are distinct to Rules Engines and not simply a benefit of SOA architectures in general? In other words, aren't rules engines a subset of SOA, and if so, what are their distinct benefits, vs. simply deploying an SOA service written in imperative style code? – Sanjay

A SOA architecture is a good ecosystem for a rules engine for decision making. The engine can route calls to different services depending on the system's needs. Without one, an admin would manually configure the system to follow another path. – Compossible

Many of the benefits I listed do come with a SOA architecture. A couple benefits offered by rule engines that don't necessarily come from SOA architecture are declarative code and a way for business owners to change rules, e.g. decrease some dollar amount that triggers a manual review. It isn't that these aren't achievable with non-rule engine solutions, but more that rule engines try to offer these features out of the box. – Cohbath

An even more recent blog post by Martin Fowler goes in depth on how to refactor code into data + rules engine: Refactoring to an Adaptive Model – Jillian

Answer 2

I think your concerns about anemic domain models are valid.

I've seen two applications of a well-known commercial Rete rules engine running in production where I work. I'd consider one a success and the other a failure.

The successful application is a decision tree app, consisting of ~10 trees of ~30 branch points each. The rules engine has a UI that does allow business folks to maintain the rules.

The less successful application has ~3000 rules slammed into a rules database. No one has any idea if there are conflicting rules when a new one is added. There is little understanding of the Rete algorithm, and the expertise with the product has left the firm, so it's become a black box that's untouchable and unrefactorable. The deployment cycle is still affected by rules changes - a complete regression test must be done when rules are changed. Memory was an issue, too.

I'd tread lightly. When a rule set is modest in size it's easy to understand changes, like the simplistic e-mail sample given above. Once the number of rules climbs into the hundreds I think you might have a problem.

I'd also worry about a rules engine becoming a singleton bottleneck in your application.

I see nothing wrong with using objects as a way to partition that rules engine space. Embedding behavior in objects that defer to a private rules engine seems okay to me. Problems will hit you when the rules engine requires state that isn't part of its object to fire properly. But that's just another example of design being difficult.

Fretwork

Answer 3

Rule Engines can offer a lot of value, in certain instances.

First, many rule engines work in a more declarative way. A very crude example would be AWK, where you can assign regexes to blocks of code. When the regex is seen by the file scanner, the block of code is executed.

You can see that in this case, if you had, say, a large AWK file and you wanted to add Yet Another "rule", you can readily go to the bottom of the file, add you regex and logic, and be done with it. Specifically, for many applications, you're not particularly concerned with what the other rules are doing, and the rules don't really interoperate with each other.

Thus the AWK file becomes more like a "rule soup". This "rule soup" nature lets folks focus very tightly on their domain with little concern for all of the other rules that may be in the system.

For example, Frank is interested in orders with a total of more than $1000, so he puts in to the rule system that he's interested. "IF order.total > 1000 THEN email Frank".

Meanwhile, Sally wants all orders from the west coast: "IF order.source == 'WEST_COAST' THEN email Sally".

So, you can see in this trivial, contrived case, that an order can satisfy both rules, yet both rules are independent of each other. A $1200 order from the West Coast notifies both Frank and Sally. When Frank is no longer concerned, he'll simply yank his rule out from the soup.

For many situations, this flexibility can be very powerful. It can also, like this case, be exposed to end users for simple rules. Using high level expressions and perhaps lightweight scripting.

Now, clearly, in a complicated system there are all sorts of interrelationships that can happen, an this is why the entire system is not "Done with rules". Someone, somewhere is going to be in charge of the rules not getting out of hand. But that doesn't necessarily lessen the value such a system can provide.

Mind this doesn't even go in to things like expert systems, where rules fire on data that rules can create, but a simpler rules system.

Anyway, I hope this example shows how a rules system can help augment a larger application.

Bevash

For the Frank & Sally example, I would argue that the same is easily done in Java as well, it doesn't need to be in a rule engine. Unless you want Frank to be able to modify the rule himself instead of having a programmer does it – Regressive

Answer 4

The biggest pro that I've seen for rules engines is that it lets the Business Rule owners implement the business rules, instead of putting the onus on programmers. Even if you have an agile process where you are constantly getting feedback from stakeholders and going through rapid iterations, it's still not going to achieve the level of efficiency that can be achieved by having the people making the business rules implement them as well.

Also, you can't under-emphasize the value in removing the recompile-retest-redeploy cycle that can result from a simple rule change, if the rules are embedded in code. There are often several teams that are involved in putting the blessing on a build, and using a Rules Engine can make much of that unnecessary.

Simmers

This is calling for trouble. Letting bis rule owners write production rules into a reactive system (with very unclear semantics and nondeterministic behaviour), with no testing and coordination with the rest of the team is a Bad Idea. Better to calmly talk with devops about adding a method or function where appropriate. If needed, give biz rule owners a generic or a foot-shootery-avoiding domain-specific language (these are rather easy to develop these days) to code. They can then hook this into the overall system - after they have tested it. A biz rule is generally NOT a production rule! – Krongold

"recompile-retest-redeploy cycle" is there for a reason, and it should be fully automated, so you just click one button and it works. – Evora

as soon as your rules involve chaining - where the action part of one rule changes the state of the system in such a way that it alters the value of the condition part of other rules Then you need to watch out for infinite loop, calling update() on the correct variable. There is no way non programmer can understand it – Regressive

Answer 5

I've written a rules engine for a client. The biggest win was including all the stakeholders. The engine could run (or replay) a query and explain what was happening in text. The business people could look at the text description and quickly point out nuances in rules, exceptions, and other special cases. Once the business side was involved, the validation got much better because it was easy to get their input. Additionally, the rules engine can live separately from other parts of an application code base so you can use it across applications.

The con is that some programmers don't like to learn too much. Rule engines and the rules you put into them, along with the stuff that implements them, can be a bit hairy. Although a good system can easily handle sick and twisted webs of logic (or illogic often ;), it's not as simple as coding a bunch of if statements (no matter what some of the simple-minded rule engines do). The rules engine gives you the tools to handle rule relationships, but you still have to be able to imagine all of that in your mind. Sometimes it's like living in the movie Brazil. :)

Hypogynous

Answer 6

It (as everything else) depends on your application. For some applications (usually the ones that never change or the rules are best on real life constants, i.e. won't change noticeably in eons, for instance physical properties and formulae) it doesn't make sense to use a rule engine, it just introduces additional complexity and requires the developer to have a larger skill set.

For other applications it's really a good idea. Take for instance order processing (orders being anything from invoicing to processing currency transactions), every now and then there's a minute change to some relevant law or code (in the judicial sense) that requires you to fulfil a new requirement (for instance sales tax, a classic). Rather than trying to force your old application into this new situation where all the sudden you have to think about sales tax, where as before you didn't, it is easier to adapt your rule set rather than having to meddle about in potentially a large set of your code.

Then the next amendment from your local government requires reporting of all sales within a certain criteria, rather than you have to go in and add that, too. In the end you'll end up with very complex code that will prove pretty difficult to manage when you turn around and want to revert effect of one of the rules, without influencing all the others...

Lavonia

Answer 7

Everybody thus far has been very positive about rules engines, but I advise the reader to be wary. When a problem becomes a little bit more complicated, you may suddenly find that an entire rules engine has been rendered unsuitable, or much more complicated than in a more powerful language. Also, for many problems, rules engines will not be able to easily detect properties that greatly reduce the runtime and memory footprint of evaluating the condition. There are relatively few situations in which I would prefer a rule engine to a dependency injection framework or a more dynamic programming language.

Bouchier

Answer 8

"but really, how many apps really have that many changes?"

Honestly, every app I have worked on has gone through serious workflow and/or logic changes from concept until well after deployment. It's the number one reason for "maintenance" programming...

The reality is that you can't think of everything up front, hence the reason for Agile processes. Further, the BA's always seem to miss something vital until it's found in testing.

Rule Engines force you to truly separate business logic from presentation and storage. Further, if using the right engine, your BA's can add and remove logic as necessary. As Chris Marasti-Georg said, it puts the onus on the BA. But more than that, it allows the BA to get exactly what they are asking for.

Thissa

@Jasper: I didn't specifically call out Drools. There are other products. One I used at another location was called Haley Rules Engine (since has been purchased by Oracle and taken off the market). That particular engine had an interface for writing rules which our BAs were trained on. It was very slick and FAST. Due to its ease of use and speed, we used it to drive data entry requirements on web pages on a per client basis. The requirements were entered by the BAs directly. An example rule was "If the APR is greater than 3.0 then require addendum ABC123" – Thissa

Answer 9

A rules engine is a win on a configurable application where you don't want to have to do custom builds if it can be avoided. They are also good at centralising large bases of rules and algorithms like Rete are efficient for quickly matching against large rule sets.

Explain

Answer 10

Lots of good answers already but wanted to add a couple of things:

1. in automating a decision of any complexity the critical thing rapidly becomes your ability to manage rather than execute the logic involved. A rules engine will not help with this - you need to think about the rule management capabilities that a business rules management system has. Most commercial and open source rules engines have evolved into rule management systems with repositories, reporting on rule usage, versioning etc. A repository of rules, structured into coherent rule sets that can be orchestrated to make business decisions is a lot easier to manage than either thousands of lines of code or a rule soup.
2. There are many ways to use a declarative, rules-based approach. Using rules to manage a UI or as part of defining a process can be very effective. The most valuable use of a rules approach, however, is to automate business decisions and to deliver this as loosely coupled decision services that take input, execute rules and return an answer - a decision. Thing of these as services that answer questions for other services like "is this customer a good credit risk" or "what discount should I give this customer for this order or "what's the best cross sell for this customer at this time. These decision services can be very effectively built using a rules management system and allow for easy integration of analytics over time, something many decisions benefit from.

Unpin

Answer 11

The biggest complexity from my experience in Rule Engines is that:

1. from OOP POV it's a real pain to refactor and test rules written in a declarative language while you are refactoring code that affects them.
2. Often we should always think about the execution order of rules which turns into a mess when there are lots of them.
3. Some minor changes may trigger incorrect behaviour of rules leading to production bugs. In practice it's not always possible to cover all cases with tests up front.
4. Rules mutating objects used in other ones also increase complexity causing developers to break them into stages.

Trestlework

Answer 12

I see rule, process and data engines (a.k.a. databases) as essentially being similar. But, for some reason, we never say that blackboxing the persistence subsystem is bad.

Secondly, from my POV, an anemic model is not one that is light in implementation of behaviors, it is one that is light in behaviors itself. The actual method that describes an available behavior in a domain model object does not have to be done by the object itself.

Grater