Mary & Tom Poppendieck: Lean Development (part 2)
In Lean Development, Mary and Tom Poppendieck compare development to creating a recipe, while production is like following a recipe. A chef creating a recipe needs to try many variations to discover the best. Software, similarly, is to a large extent about trial and error. They cite a study by Raymonde Guindon, which “found that cycling between high level design and detailed solution was typical of good designers when dealing with ill-structured problems, that is, problems which do not have a single right answer or a best way to arrive at a solution. Guindon theorized that this unstructured approach is necessary to understand and give structure to such problems.”
The Poppendiecks also write that the preferred method for solving complex problems is to use the scientific method. You observe, create a hypothesis, then devise an experiment to test the hypothesis. The interesting thing, they write, is that to maximize learning, you have to have a 50 percent probability of failure when testing your hypotheses. In other words, if your hypotheses are always correct, you don’t learn as much; it is by failing that you learn. This speaks loudly in favor of experimentation in software development, either by creating prototypes or (which is even better) by writing real code.
One thing I didn’t know that they mention in the book: you often hear the familiar Frederick Brooks quote: “>Plan to throw one away, you will anyway.” What he referred to was that, in order to learn enough to develop a system, you need to create it twice – first you create an experimental prototype, then you create the actual system. However, he retracted this twenty years later, saying that it implied a sequential process, a waterfall process – which he said was wrong: to succeed, you need an incremental process, with progressive refinement. The Poppendiecks also quote Ed Yourdon as saying that any piece of code needs to be rewritten three or four times to be consistuted as an elegant, professional piece of work. For prose, we take this for granted, he said, why not for code?
Today, iterative, adaptive processes are widely regarded as the most suitable for software development. But the Poppendiecks write that upon trouble, we are inclined to turn to a more disciplined process, which in fact is sequential: we first want the requirements to be signed off by the customer, then we consider them frozen, and start implementing, resisting any changes the customer wants, and so on. This, they continue, contradicts control theory, since what you are doing is lengthening the feedback loop. What you must do is to shorten it – do shorter iterations, increasing the frequency of feedback from the customer; accept new requirements only at the start of these short iterations, et cetera.
The Poppendiecks write about firefighters as intuitive decision makers, who operate in very complex situations where rational approaches are deemed to fail, as you can’t possibly take everything into account, in an analytical fashion. Something that was new to me was that a firefighter, at times when his experience is inadequate, resorts to rational decision making. This must be the same mechanism that lies behind the tendency to seek a more disciplined process when meeting challenges in a project. (See also my post Intuition at Tesugen.com.)
They continue by citing a book called Corps Business by David H. Freeman. Marines are trained to deal with immense complexity, with paradoxical and contradictory circumstances. Their training aims to expose them to more challenging situations than they will ever experience in reality, using situational training to build intuitive decision making skills. For them, mistakes are considered necessary. First they make plans to understand the essence of a situation. Then the organizational structure is collapsed, leaving the decisions to the men at the front line, which are less likely to make mistakes than the remote officers. The principle with the training is to expose them to situations where they are given a goal, but also the freedom to decide the details in fulfilling that goal. The focus is on small teams at the lowest levels, rather than way up the hierarchy.
To me, the essence of this is about communicating a clear statement of intent to the team, giving them a common vision, or goal – then giving them the freedom to find the best way to fulfill that goal, which happens within a framework of simple rules or guidelines (such as the seven principles listed in my previous post).
Later in the book, the Poppendiecks write about an automobile plant that was a real disaster: alcohol problems both on and off site; strikes; high sick leave rates. It was closed down, only to re-open under mixed management between General Motors (GM) and Toyota. The bulk of the previous workers return, and the plant turns into a huge success: productivity and quality is higher than any other GM plant; the problems are essentially gone and 90 percent of the workers say they are happy. The decisive factor, they write, seems to have been the fact that the workers were reorganized into small teams with the power to self-organize. Previously, there were trackers measuring their effectivity; after the reopening they were trained to measure themselves and given the freedom to discover how to improve. (See also my post titled Slime molds.)
They quote someone (I don’t remember who, and I don’t have access to the draft any longer) as saying that the greatest demotivators are policy, supervision, and administration, and that the greatest motivators are achievement, recognition, and responsibility. I often think about the importance of individual freedom in software projects. These demotivators and motivators are definitely about freedom, or the lack of freedom. The Poppendiecks also mention the importance that individuals feel that they are allowed to make mistakes – or, in fact, that they should make mistakes. Making mistakes is a necessary element of achieving success, and they write that companies that are very successful (and hence innovative) often have a high tolerance for mistakes.