How to Lose Business Rules

I was recently heard of a tragic case of a company that lost their business logic in a certain domain. This sounds incredible, but it can happen. How could you avoid the same happening to you?

Introduction

This is the sad story of two companies that buried their business rules and had to resort to expensive rule mining to recover them. In one case the company concerned squandered an expensive BRMS license on ‘rules’ which are later lost by the business. How should you prevent the same happening to you?

Two Tales of Woe

In the first case a company elected to use the C programming language as a vehicle for expressing the simple, but large number of rules governing their financial processing. This decision was motivated by performance. The rules were captured by successive teams of SMEs over a period of a year and loosely documented. The repository had a flat structure: all the rules being at one of two levels. Over time the dependencies between rules grew ever more complex and 1000’s ‘reusable’ rule fragments were created. The project went live and performed well for ten months. However, after many of the SMEs left the project (and some left the company), much of the context of the rule’s creation was lost, it was discovered that these rules were hard to understand and change. Eventually the business were in crisis: a minor business change was required, but it was very difficult for rule owners to determine which of the C rules were responsible for the functionality affected, or how they worked. A survey of the rules was done and the company discovered over two-thirds of them were not understood by current SMEs because of their technical format and maze of interdependencies. The company had effectively lost their rules and the ability to quickly and safely change them. The rule mining effort than followed took three months and cost nearly $100,000.

In another case, a household name company invested in excess of $200,000 in a BRMS license to manage the rules of their financial transaction processing. Again fearing the adverse performance effect of using literate rules (i.e. natural language rules, written in plain business English), they elected to use a technical rule language instead. The view that the BRMS constituted sufficient documentation prompted the business not to invest in external documentation effort. Over 25,000 rules were authored over seven months. Performance was excellent and the system went live. However less than six months later a version update of the BRMS changed the behaviour of the ruleset. The company did an audit and found that because the technical rules had been authored by a team with minimal training rudimentary mistakes had been made in the formulation of rules which were hidden by a minor feature of the BRMS. Furthermore, since many of the original rule authors had left, none of their personnel understood more than 40% of the ruleset. A mining exercise was requested, which is still being costed.

Lessons Learned

Where did these companies go wrong?

1. Avoid implementing business rules in a technical language unless you have business SMEs in the team that will stay there for the life of the system. Even then consider how much easier it is to change a business rule when it is expressed in English based on a well formed business vocabulary.
2. Ensure you always get full training for any toolset you are planning to use. Hire an expert to supplement and mentor any inexperienced rule authoring team.
3. Never make a decision based solely on performance without doing full research and testing prototypes. In fact well engineered rules expressing in plain English have the sameperformance as technical rules (because they are compiled). The performance advantage of raw C over modern rule engines is slight and can in some cases be worse.
4. Ensure your business rules are fully documented, provided full traceability to business documentation and business drivers where possible. Phrase and document your reset with the assumption that all SMEs on the project will leave within six months.
5. Ensure your business rules are atomic (i.e. have a single condition, with AND clauses, but avoid ORs and a single action) and that any dependencies between them is reflected in the rule repositories. Many BRMS can detect rule inter-dependencies. This will make it less likely that a change in policy will require widespread rule change, or that a rule change will yield multiple side effects, some of which are undesirable.
6. Do not implement rule fragments, they undermine the dependency analysis of BRMS tools. Ensure that every rule has a condition and an action.
7. Build your ruleset to facilitate quick location of rules (according to their meaning) and easy change. Use the ruleset hierarchy to support this semantic categorisation.
8. If your rules are not undergoing constant change, periodically review them to ensure they are representative of current business practices and clear in their function.
9. Ensure the business review any major rule installation that is under-way both 30% into development and prior to go-live.
10. Use automated regression testing to ensure that after infrastructure updates and rule changes, that there are no unanticipated consequences.
11. If you only require technical rules, because your domain categorically does not need business experts to understand or manipulate the rules and the rules do not change a great deal, then think twice before buying a commercial BRMS. The strengths and value proposition of such a tool do not apply in these situations. A technical rule management system like Drools may be better suited to your needs.

Conclusion

Be clear about who will need to understand and edit your business rules going forward and phrase the rules appropriately (in business language for business SMEs and technical languages for IT practitioners). Develop the system with the assumption that the staff will turn over every six months

Useful? Please share: