Expert/Rule Engine that updates facts atomically?

Question

Atomically might not be the right word. When modelling cellular automata or neural networks, usually you have two copies of the system state. One is the current state, and one is the state of the next step that you are updating. This ensures consistency that the state of the system as a whole remains unchanged while running all of the rules to determine the next step. For example, if you run the rules for one cell/neuron to determine the state of it for the next step, you then run the rules for the next cell, it's neighbor, you want to use as the input for those rules the current state of the neighbor cell, not its updated state.

This may seem inefficient due to the fact that each step requires you copy all of the current step states to the next step states before updating them, however it is important to do this to accurately simulate the system as if all cells/neurons were actually being processed simultaneously, and thus all inputs for rules/firing functions were the current states.

Something that has bothered me when designing rules for expert systems is how one rule can run, update some facts that should trigger other rules to run, and you might have 100 rules queued up to run in response, but the salience is used as a fragile way to ensure the really important ones run first. As these rules run, the system changes more. The state of the facts are consistently changing, so by the time you get to processing the 100th rule, the state of the system has changed significantly since the time it was added to the queue when it was really responding to the first fact change. It might have changed so drastically that the rule doesn't have a chance to react to the original state of the system when it really should have. Usually as a workaround you carefully adjust its salience, but then that moves other rules down the list and you run into a chicken or egg problem. Other workarounds involve adding "processing flag" facts that serve as a locking mechanism to suppress certain rules until other rules process. These all feel like hacks and cause rules to include criteria beyond just the core domain model.

If you build a really sophisticated system that modeled a problem accurately, you would really want the changes to the facts to be staged to a separate "updates" queue that doesn't affect the current facts until the rules queue is empty. So lets say you make a fact change that fills the queue of rules to run with 100 rules. All of these rules would run, but none of them would update facts in the current fact list, any change they make gets queued to a change list, and that ensures no other rules get activated while the current batch is processing. Once all rules are processed, then the fact changes get applied to the current fact list, all at once, and then that triggers more rules to be activated. Rinse repeat. So it becomes much like how neural networks or cellular automata are processed. Run all rules against an unchanging current state, queue changes, after running all rules apply the changes to current state.

Is this mode of operation a concept that exist in the academic world of expert systems? I'm wondering if there is a term for it.

Does Drools have the capability to run in a way that allows all rules to run without affecting the current facts, and queue fact changes separately until all rules have run? If so, how? I don't expect you to write the code for me, but just some keywords of what it's called or keywords in the API, some starting point to help me search.

Do any other expert/rule engines have this capability?

Note that in such a case, the order rules run in no longer matters, because all of the rules queued to run will all be seeing only the current state. Thus as the queue of rules is run and cleared, none of the rules see any of the changes the other rules are making, because they are all being run against the current set of facts. Thus the order becomes irrelevant and the complexities of managing rule execution order go away. All fact changes are pending and not applied to the current state until all rules have been cleared from the queue. Then all of those changes are applied at once, and thus cause relevant rules to queue again. So my goal is not to have more control over the order that rules run in, but to avoid the issue of rule execution order entirely by using an engine that simulates simultaneous rule execution.

Inaptitude

What you describe sounds to me more like Complex Event Processing. Otherwise, if you are looking for stronger mechanisms for enforcing the order in which rules are able to run, then Drools offers tools such as agenda groups, but I prefer to design the rules themselves to enforce when they can activate. For a typical pattern for that, take a look at "Marker Facts" in here: engage.redhat.com/forms/rule-design-patterns – Joellyn

In theory, it would be great to have a core domain model which doesn't know Drools. Still, in practice, drools can become so complex, that it can make sense to just say that your domain is also drools (in fact: it is driven by rules, so it is up to you to decide that rules can be considered in some way your domain). I have many facts to ensure my rules are processed in the right order, it is part of my business logic. Personally, I prefer that compromise instead of heavily relying on salience (with 'magic' numbers) or agenda-group (with hidden meaning if they are triggered from outside DRL). – Foxed

Thanks for the feedback. Trying to avoid workarounds that involve managing rule execution order, see update. I did consider using marker facts to emulate simultaneous execution, by marking all new facts as "next" and all rules would exclude those facts. Then one final rule with lowest precedence would remove the "next" marker. Thus the "next" facts would only get "applied" once all rules have been executed. I would also have to handle changes and deleted facts in similar fashion, not actually deleting them, but delaying the delete until all other rules have run. – Inaptitude

Answer 1

If I understand what you describe:

• You have one fact that is managed by many rules
• Each rule should apply on the initial value of your fact and has no right to modify the fact value (to not modify other rules'executions)
• You then batch all the updates made by the rules on your fact
• Other rules apply on this new fact value in a similar manner 'simutanously'

It seems to me that it is a Unit of Work design pattern just like Hibernate implements it (and many ORM in fact): http://www.codeproject.com/Articles/581487/Unit-of-Work-Design-Pattern

Basically you store in-memory all the changes (in a 'technical' fact for instance) and then execute a 'transaction' when all the rules based on the initial value have been fired that updates the fact value, and so on. Hibernate does that with its session (you modify your attached object, then when required it executes the update query on the database, not all modifications on the java object produce queries on your database).

Still you will have troubles if updates conflict (same fact field value modified, which value to choose? Same as a source version control conflict), you will have to define a determinist way to order updates, but it will be defined only once and available for all rules and for other changes it will work seamlessly.

Foxed

Answer 2

This workaournd may/may not work based on your rather vague description. If you really are concerned about rules triggering further activations, why not queue the intermediate state yourself. And once the current evaluation is complete, insert those new facts into the working memory.

You would have to invoke fireAllRules() after inserting each fact though, this could be quite expensive. And then in the rules, rather than inserting the facts directly, push these into a queue. Once the above call returns, walk through the queue doing the same (or after inserting the original facts completely...)

I would imagine that this will be quite slow, to speed up, you could have multiple parallel working memories with the same rules, and evaluate multiple facts in one go into several queues etc. But things get pretty hairy..

Anyway, just an idea that's too long for the comments...

Johannajohannah