While James Taylor and I were collaborating on our Decision Modelling book and discussing our experiences of using DMN with clients, the question “what additional features should be adopted in the next major release?” was a frequent subject of conversation. We found that our respective wish lists had a lot in common, reflecting our views on decision modeling best practice. We decided to describe these proposed new features in a series of posts and encourage readers to give their own opinions.

DMN (The Decision Model and Notation) is a way of representing  Decision Models using diagrams and text; it does not address issues such as method or approach. Therefore wishes must be constrained to new or amended notation. Given this, do you agree with our items? What features would you like to see included in the decision logic level of the standard?

Decision Logic Level Wish List

The Decision Logic Level of DMN defines decision logic using notations mandated by the standard: Decision Tables, Boxed Expressions and the expression language, FEEL, with which they are constructed. We think this could be usefully extended in several ways.

Support Expressions in Decision Table Input Entries

Currently a Decision Table condition cell (formally a Input Entry in DMN, see diagram on the right) is defined in the FEEL grammar as unary tests. This supports input entries which test an input expression for any of the following:

• equality (or inequality) with a…
  • literal numeric, string, Boolean or date time value (e.g., ‘>100’);
  • qualified name (a variable, e.g., ‘<=Credit Limit’)
• inclusion within an interval (e.g., ‘[0..100]’)
• having an unknown value (i.e., ‘null’)
• having any value (i.e., ‘–’).
• satisfying any of the conditions above using a comma separated list of conditions which is satisfied if any one of its members is (e.g., ‘<=Credit Limit, <=Auto Loan Threshold’).

As a result condition cells cannot perform tests using fully featured arithmetic expressions (e.g., ‘<=Salary * 4.0’). In practice this restriction is rather irksome and means that many decision tables have to resort to contexts or other FEEL-heavy means to test an inequality with a calculated amount. This is something we see a lot of demand for by business users of DMN ‘in the field’. It would be valuable to extend input entries to support simple expression or even a new type combining unary tests with arithmetic expression, simple value and function invocation to allow conditions like:

• ‘>=(Expiry Date + Grace Period)’ (simple expression)
• ‘>max(Applicant Credit Limit, Guarantor Credit Limit)’ (function invocation)

Better still, allow Decision Table Input Entries to support all FEEL functions directly. Take the case of a Decision Table with a condition (input) that is a string. Remember FEEL defines a function for testing if a string contains a substring (e.g., contains(“shire”,”ir”)). In this case, this FEEL function could be reused in an input entry by:

• Using a reserved word to represent the Input Expression being tested (e.g., ‘contains(value,”ir”)’) or
• Using a simplified version of the function (e.g., ‘contains(”ir”)’) where the first argument is always the input value (condition) by convention

This would allow Decision Tables like the one on the right.

This may strike some as over complicating Decision Tables and it would require discipline to use effectively. Undisciplined use of complex conditions runs the risk of making the Decision Table incomprehensible and hard to validate—but the alternatives (FEEL contexts) are far more ‘heavyweight’ in our experience and can alienate business users.

Decision Table Output Entries

By a similar argument to the above, the content of conclusion cells (output entries) which are already simple expressions, should be extended to support FEEL function calls. This would allow Decision Tables to represent a selection of outcome calculations which currently cannot be performed without resorting to a context or boxed expression.

Decision Tables Supporting Collection Input Expressions

Currently DMN Decision Table Input Entries (see above) cannot directly handle collections only single values. Therefore, although an Input Expression (or input variable) may be a collection, the only way to process a collection is to use FEEL’s ‘for .. in .. return’ construct to allow iteration through the collection or use an input expression that aggregates the collection to a single value that can therefore be tested. Collections may be tested for equality or inequality, but input entries cannot have any ‘set’ based operators.

Simple conditions, for example checking an input collection for the existence therein of a specific value, requires a FEEL context. This should not be necessary. Input Expressions should be extended to support conditions on collection, perhaps based on the simplified functions idea, above. This would support condition tests like:

• Checking collection membership (‘list contains(USD)’)
• Checking collection size (‘count()>5’)
• Or checking aggregate properties (‘min()<4.5’).

If TDM can manage this, so can DMN.

Rule Level Annotations in Decision Tables

DMN Decision Requirement Diagrams benefit from text comments called annotations to clarify their meaning (e.g., to express assumptions, denote use of patterns, etc.). So, as some vendors have rightly suggested, why not have a rule level annotation as well? A rule level annotation is an additional conclusion in a Decision Table that acts like a comment and post-execute explanation (much like the Message construct of TDM) which:

• Documents the intent, rationale and idiosyncrasies of each rule (as required)
• Acts as a means of supporting after-the-fact behavioural explanations of why specific rules were selected, including the ability to evaluate expressions to assist with diagnostics (see examples below)
• Optionally links a rule with a specific section of a Knowledge Source that it satisfies

Here is our informal proposal for what this would look like:

Note:

• Annotations are confined to the right-most conclusion columns (or bottom rows in a ‘rules-as-columns’ table).
• Annotations contain references to specific sections of named Knowledge Sources (traceability to requirements). In this example, IASIC is a nickname for a Knowledge Source and 5-19-4b is a section of the document referenced by this source.
• Annotations are string expressions which can embed diagnostic information (see the annotation for rules 3 and 4)
• Annotations are an effective means of embedding business justifications, making your Decision Tables easier to understand.

Completeness

Earlier versions of DMN supported the notion of completeness: a flag, denoted alongside the hit policy, to indicate that a Decision Table has a rule to cover every possible case—every combination of logical scenarios. Many of our clients found this very useful and were disappointed when the concept of completeness was discontinued in DMN 1.1 because of implementation concerns.

In our view, the concept of completeness should be reintroduced with a subtly different meaning. DMN should include a completeness marker to indicate the decision modeler’s intent that a Decision Table has complete logical coverage in practice. In other words, a Decision Table is complete if:

• For every possible combination of input values that can occur in practice at least one rule is satisfied

The counter-argument has been made that this is not required because Decision Tables should always be explicitly complete. That is: each impossible combination of inputs should be supported by a rule with a null conclusion to deliberately document ‘impossible cases’. We have some sympathy with this argument but, in practice, there are many ‘real-world’ Decision Tables with enumerated inputs that have many nonsensical combinations. Listing them all would ‘bloat’ the table and obscure its business intent. Business analysts need a means of saying: “this set of rules covers all business valid combinations, all others either cannot happen or are not addressed here.”

Other Logical Representations

We would like to see the introduction of a DMN standard Decision Tree and a more business like interface to Analytic Models (business users find PMML rather opaque).

Hit Policies

The hit policies ‘R’ (Rule Order) and ‘O’ (Output Order) are virtually useless and lead to some spectacularly obscure logic—please, get rid of them!

Missing FEEL Functions

FEEL incorporates some useful built-in functions for manipulating and aggregating collections of values. This is excellent as far as it goes, but many of the included functions lack mathematically associated functions: union() is supported but not intersection(), is superset of() or is subset of(); sum() is supported but not product(); there is flatten() but no zip() and mean() is supported but not median(), mode() or stddev().

Naturally there is an interface to add new functions as needed and the standard should not be encumbered with too many functions—not everybody’s favorite functions can or should be be supported. However we would like to see a more inclusive and mathematically balanced set of built-in functions.

Glossary

Some have said DMN already defines all the metadata needed to interact with a glossary – anything else is tool specific. They assert that it’s not the job of a decision modeling standard to dictate the structure of a business glossary.

This is true, but we feel the DMN standard could go further in defining its interface to a glossary and what it requires of a glossary. As a result the importance a business glossary has been lost on many new users of DMN and DMN handles business terms in a naïve way, standing aloof from the data it manipulates.

DMN should not define what constitutes a glossary, but it should define what the glossary provides to a decision model, specifically:

Robust handling of enumerations

Enumerations should be a first class data type and not “strings with benefits”. Decision models should be able to manage externally defined enumerations and import them fully. In our experience of finance, more than two thirds of data types manipulated by decision models are enumerated.

Value lists should refer to named enumerated types in glossary

In practice a value list (a way of constraining the enumerated values of conditions and conclusions in Decision Tables) is not very

scalable. It breaks down for the Decision Tables using enumerated types that: have tens of permitted values or more, have constantly changing permitted values and are used in many Decision Tables. The maintenance of such large and ubiquitous enumerations, when represented as a value list, is very user-intensive and error prone—leading to inconsistencies. A much better scheme is to use a glossary to define named enumerated types and allow Decision Tables to refer to these named types in the value list. Consider the example shown on the right.

In this Decision Table, the value list of Issuer Type does not explicitly repeat all the allowed values in each table, but instead refers to a named enumerated type in angled braces (Issuer Type). This allows us to define the list of permitted values just once in the glossary.

Robust handling of context names

Contexts in DMN can be used to rename information requirements to make a Decision Table more readable. As with enumerations, the same renaming may be used in multiple Decision Tables and this should be managed once not repeated each time. We propose a means of introducing contexts that can be easily shared between many tables.

Conclusion

We hope you found these interesting and we will post again soon with our wishes for additional features in the Decision Requirements level of the DMN. Meanwhile, we would be interested to hear your views: what additional features would you like to see in DMN?