My last two blog posts have generated a lot of interest and discussion on the xbrl-dev mailing list. I thought it might be interesting to go back to the article that triggered the discussion of rules languages in the first place.

Charlie Hoffman posted some interesting observations on his blog about common patterns in the formulae that he was creating, noting that he could distill most of what he was doing into 10 patterns. Charlie then took the next step which was to parameterise the patterns, and create code to generate the XBRL Formulas from a simpler XML format, which he refers to as an info set.

For example, one of the patterns is a “roll forward” where a closing balance should be equal to an opening balance, plus the sum of a set of changes for the period. Charlie uses the following example with US GAAP concepts:

CashAndCashEquivalentsAtCarryingValue (end of period) = CashAndCashEquivalentsAtCarryingValue (start of period) + CashAndCashEquivalentsPeriodIncreaseDecrease (during period)

You can view the XBRL Formula for this example. As you can see, it’s quite involved. Charlie’s solution is to create a much simpler XML format that contains only the parameters for this constraint:

<BusinessRule number='11'>
  <Network href='abc-20101231.xsd#StatementOfCashFlows'></Network>
    <ChangeConcept operator='+'>us-gaap:CashAndCashEquivalentsPeriodIncreaseDecrease</ChangeConcept>

Charlie has code that takes the above XML and converts it to the necessary XBRL Formula, allowing users to work with a much simpler format. To quote Charlie:

Go look at the complexity of the XBRL Formula file. Then go look at the complexity of the business rules info set file. Calculate in your head how much effort it would take to teach someone to create that XBRL Formulas file. Then, think about how long it might take to explain how to create that business rules info set file.

I certainly agree with this, and Charlie’s approach certainly goes a long way to making these problems easier to solve, although it does suffer from the same problem as all other code generation approaches, specifically, that you can’t round-trip the resulting XBRL Formulas back to the simpler format. If you want to do something that isn’t covered by one of the patterns, then you’re left editing XBRL Formula, and you have to make sure that your edits don’t get overwritten if you regenerate from the input file.

The underlying problem is that XBRL Formula doesn’t make the simple things simple enough for a business user to work with. It won’t surprise you to learn that I think that Sphinx can do a better job here.

Firstly, I think the Sphinx implementation of this problem is much more accessible in the first place:

raise StatementOfCashFlowsRollForwardCheck
  d = foreach set(values us-gaap:CashAndCashEquivalentsPeriodIncreaseDecrease[]::period) 
  bop = $d::start-date
  eop = $d::end-date 
  us-gaap:CashAndCashEquivalentsAtCarryingValue[period = $eop] != 
  us-gaap:CashAndCashEquivalentsAtCarryingValue[period = $bop] + 
  us-gaap:CashAndCashEquivalentsPeriodIncreaseDecrease[period = $d]

See my previous post for an explanation of how this works. I think this sample is pretty readable as it is, but Charlie is quite right in observing that there’s a pattern here. If I were writing rules for a taxonomy like this, I’d be doing a lot of copying and pasting of code. Fortunately, Sphinx has the ability to define custom functions, so I can create a function for this pattern. Here’s what it would look like:

function roll-forward(balance, change) 
  d = foreach set(values [primary = $change]::period) 
  bop = $d::start-date
  eop = $d::end-date 
  [primary = $balance; period = $eop] != [primary = $balance; period = $bop] + [primary = $change; period = $d]

Having done this, writing the rule itself is reduced to a single function call, providing the two parameters to the pattern – the balance concept, and the change concept:

raise StatementOfCashFlowsRollForwardCheck
  roll-forward(us-gaap:CashAndCashEquivalentsAtCarryingValue, us-gaap:CashAndCashEquivalentsPeriodIncreaseDecrease)

I can create functions for other patterns in Charlie’s article, allowing authors to write rules by providing little more than the names of the concepts involved. If you find the definition of the function intimidating, that’s fine. It only has to be written once, and can then be squirreled away in a separate file in your Sphinx rulebase, so that your business users need only be concerned with the rules themselves. (And just in case you were worried, it’s trivial to extend the function to cope with multiple “change” concepts, rather than just one)

Aside from the rules being syntactically more concise than even the simplified XML, this has a few advantages over the code generation approach:

  1. The rules I write are still Sphinx expressions, meaning that I don’t need to create an additional piece of software to edit them with. I can still benefit from SpiderMonkey’s rule editing environment which provides auto-completion of QNames, and on-the-fly syntax validation.
  2. If I need to write something that’s not covered by a pattern, I don’t have to switch to some other environment, or some lower level language.

In other words, Sphinx allows us to model these patterns in a similar way to the proposed infoset, making simple things even simpler, but whilst still making the more difficult problems solvable.