Rules

In software development for many years, there was a stream of new processes, or methodologies – each representing a slightly different approach and trying to improve on the existing ones. Some became popular and people abandoned their processes for them. There were process wars. Then things settled down. The Rational Unified Process (RUP) was formed as process heavyweights Grady Booch, Ivar Jacobson and James Rumbaugh joined forces. There are still process wars, and perhaps fiercer, since RUP’s opponents have joined in turn to form the Agile Movement, with Extreme Programming (XP) as its celebrity backer.

The RUP is often referred to, by followers of the agile movement, as a “heavyweight process”, meaning that it involves way too much paperwork and bureaucracy. Is this perhaps a result of humans wanting to follow rules when facing challenges? When things get tough, we tend to revert to being very disciplined, attending to one small thing after another, in a very step-by-step fashion. I guess we also would want clearly defined rules, so that we don’t have to constantly assess the situation, but just go on to the next rule on the list after having dealt with the first one. See for example the visual guide for when you are trapped in debris, at the ready.gov site.

Today, software projects are seen as risky challenges. At the beginning of almost every new project, the stakeholders and the developers worry about whether the project will deliver the planned features on time, and within budget. I think that, given this situation, software professionals – and especially those on the managerial level – want rules. Rules are safe: the illusion is that rules come with predictability, and should things go wrong, you can always blame the rules – after all, everybody did what the rules said, right?

One of the problems with XP is that people see the twelve practices as rules. The thing is, that XP is an adaptive process. The way it is described in the literature is nothing more than a suggested starting point. There’s no optimal set of practices that cover every situation. I remember reading on the XP mailing list a question regarding what to do when someone in the team is sick, resulting in someone being left without a pair programming partner. If you want to cover all such situations, you would need shelves of process documentation – and it still wouldn’t be complete.

With too strict rules, there is no room for creativity and for adapting to reality. In rough times, we prefer strictness and discipline, to give us the illusion that things are under perfect control. My feeling is that this situation is rather unique to software development, but I guess that wherever you have big money at stake, and poor track records, there are heavyweight processes to be found. I know from reading Mary Poppendieck’s Lean Development (see the January archives for all my posts about this) that the US car industry was in the same situation, but had to adapt as the Japanese gained a large share of the automobile market.

The agile movement represents a reaction towards heavyweight processes, but in history there are numerous examples of agile businesses. Also, other creative businesses – such as filmmakers, ad agencies, etc – are very agile, meaning that they don’t worry that much about process, but with what works. When facing new and unique challenges, they don’t make the mistake of following rules that were made without considering such special cases. Software development has repeatedly compared itself to engineering and construction, but I’d say that any architect (whether he designs skyscrapers or bridges) is more agile than the average software project. Why is that?