Conceptual clarity and test-driven development

Yesterday I did something that made me think about test-driven development and refactoring. In the system I’m working on, the overall concept is about entering periodic information, that is processed in batches at a regular interval, at which time a set of “documents” are generated, some of which are intended to be read by humans, while some are to be sent to other systems. I was modifying the generation of a particular type of document, representing a list of payments to be made after each batch has run.

Prior to the change, this list of payments was unidirectional – meaning that payments were always made by one party to a set of other parties. The change introduced the possibility that payment recipients might pay money back on certain occasions.

Since this was the case, the payment list (in the form of an XML document) contained a list of “agencies”, specifying for each the amount to be paid and to which bank account. The document also contained the bank account to transfer the money from, but this was treated as fairly static – indeed changeable, but since that would seldom happen, it is only changeable via SQL.

As I was briefed about the needed change, I realized for the first time that the list should actually be a list of money transfers between accounts – instead of a specification of which amounts a set of agencies should be paid.

Now, on to my point with this post. My solution to this implied, I think, a degradation in conceptual clarity. What I did was to add the possibility to “mark” an agency that it should be “flipped” in case its amount was negative. All processors that generate documents operate in a sequence of prepare, process, and conclude. So what I did was to add to the conclude phase that all negative amounts that are marked as “flippable” are turned into repayments.

In Martin Fowler’s outstanding book Refactoring, a set of “code smells” are listed, such as “Feature Envy,” “Long Method,” etc. – with its prescribed refactorings. In this case, it’s hard to see how navigating using code smells would reveal what is bad about the code. I’m definitely not trying to put down the concept of code smells, which I think is brilliant – I’m just exploring cases where they might be insufficient. In my case, the flaw is conceptual rather than apparent in the design.

If you follow the simplicity guidelines of Extreme Programming, the rule “Expresses every idea that we need to express” could be said to capture conceptual flaws – but perhaps it’s important to emphasize the importance of conceptual clarity as well? My code would be much more easily understandable if it dealt with a list of money transfers, but since this is something that would involve changing the XML schema, and a substantial part of the code, while also providing backwards compatibility with the previous XML schema, I might have to defer this for later (as is too often the case in projects).