Lean development #8
Last night I finished reading Mary Poppendieck’s draft copy of Lean Development. It will be an great book when it is released, I’m sure. Myself, I haven’t read much books on the agile process – only a bunch of books on XP. Although many of the XP books are great, too, I’m very familiar with the lingo by now; there are few surprises besides in the details – small new discoveries here and there in the more recent books and articles. But this book looks at these things from new angles, relying to a great extent on comparisons with other fields. And this is something I believe we must do to “sell” agile: we must show examples of other businesses and industries that we would call agile. There are reasons that people seek ever more sequential and rational processes in software, given that many feel that software largely has failed (blown budgets, gotten lost in the woods), and they must be reassured that there are better ways to do things, and that these ways might seem at the first glance to be chaotic and irrational.
Poppendieck mainly relies on looking at the car industry in this book, something that I find to be very relevant: making cars are very costly and riskful endeavors; there’s a long time to market and no-one knows really what cars people will want when a new one is ready for release; there are many people involved; there’s a pressure to invent new things with each car. And it might come as a surprise to many that the most effective approaches to making automobiles are far from what is sought in software. They don’t strive to have a perfect design, where every tiny little aspect is covered and guaranteed not to be changed, before they start producing the vehicles. Instead, what software people want to do in different phases, with clear milestones in between, is done concurrently in the automobile industry. In fact, change is seen as a given, and mistakes are seen as a natural ingredient in discovering the design for a new car.
I think that many people are fascinated by these stories of “odd” but highly effective approaches, and that the surprise effect can be used to convey the message of the agile movement. The stories seem odd because the practices we have come to use are so unnatural. As Poppendieck shows in her book, we tend to seek discipline and rationality in chaotic times; we focus on details and ignore the whole; we grasp what we feel to be in control of. Rational, sequential processes seems to comforting, but in fact they are to blame for many of the failures in software. For example, the much cited cost-escalation curve: that a change in analysis and design is cheap, but a change late in development costs a hundred, if not a thousand times more. This is indeed true for some things, but far from all. Most of the changes aren’t that more expensive to make at a later time. And many late changes become expensive because of us trying to nail the design at an early stage.
Again, by looking at the automobile industry, the approach invented by Toyota in the 1980’s involves delaying decisions to the “last responsible moment”. By keeping your options open, you are free to change. And since your entire process is centered around change, with each short iteration being about producing something, and finding out if we need to improve the result. In a change-driven process, changes become less expensive. The most powerful example from the auto industry is the one about “die cutting” – the process of creating the die to shape the individual parts of the car’s body. Poppendieck writes (emphasis mine):
When sheet metal is formed into a car body, a massive machine called a stamping machine presses the metal into shape. The stamping machine has a huge metal tool call [sic!] a die which makes contact with the sheet metal and presses it into the shape of a fender or a door or a hood. Designing and cutting these dies to the proper shape accounts for half of the capital investment of a new car development program, and drives the critical path. If a mistake is made and a die is ruined, the entire development program suffers a huge set-back. So if there is one thing that automakers want to do right, it is the die design and cutting.
Before the US car industry had to change to catch up with the Japanese, the cost of changes to the die design was 30–50 percent of the total die cost; in Japan it was 10–20 percent. And the Japanese starts the die cutting process at the same time as the designers start working on a new car design – something that is profoundly contrary to our intuition: if failures are so expensive, how can you dare beginning before you have a solid feel for the design?
(You’ll have to read the book yourself to learn how it is done.) This is just one example of how it is possible to start without a detailed architectural plan or design; of how you can’t do things effectively unless you use an iterative, incremental approach. We need more examples like these to explain what agile is about. Poppendieck sets a great example of how this can be achieved. Her seven guidelines borrowed from lean thinking, and adapted to software, are very effective:
- Eliminate waste.
- Amplify learning.
- Decide as late as possible.
- Deliver as fast as possible.
- Empower the team.
- Build integrity in.
- Avoid sub-optimization.
As I began reading the book, I understood the essence of all these, except for the last two. I think I felt that they had more to do with the actual programming, rather than with the process. True, they can, and should, all be applied to software as well. In fact, you can’t really separate the process from the product, but somehow the last two felt odd in company of the others. I’ll run through them quickly backwards from “Avoid sub-optimization.”
Avoiding sub-optimization means that overall success is what’s important – not success of individual tasks. Poppendieck uses Tour de France as an analogy, and lists how many stages Lance Armstrong, winner of the last four races, won at each time: in 1999, he won four; in 2000 he won one; in 2001 he won four; and last year he won one. There are twenty one stages in all, and the point is that you can win even if you don’t win all the stages. In fact, aiming to win each stage seems to be considered a lousy strategy by the racers themselves.
So, what does this mean? Well, Poppendieck talks about measurement, and about how you can’t measure everything, and that measuring can lead to focusing on the wrong things. She uses an example of a manufacturing company that has invested in a very expensive machine, and where they spread the cost of the machine across all the parts the machine produces. The idea is that the more parts produced, the less part of the unit price will be the machine cost. This measurement resulted in maximizing the utilization of the machine, which led to a large inventory of parts – which looked good in the books, since inventory is filed as an asset, but it didn’t lead to profitability; rather, the inventory clogged “the arteries of the plant and [made] it very inefficient”. The measurements showed success, though, with low unit prices and high machine utilization.
I’m unsure about whether this can be applied to iterations and the success of individual iterations. It seems intuitive that when each iteration succeeds in delivering the features that were planned, the entire project will succeed. But then again, if you measure completed features, you might, I suppose, forget what really matters: producing value for the customer. I can easily imagine a project where all features are in place, but the customer is unhappy about the result. The guideline is, I guess, to know what you are measuring, and to don’t worry about measuring at the micro level.
Next, about building integrity in. At Oops, we are often located at the customers’ sites, but we try to communicate as much as possible with each other. We have an internal chat server, for example, and every now and then the tone gets quite playful (especially in Friday afternoons, which we call “Funday”). The other day, we had a discussion about the definition of gadgets. I said that the size of a gadget matters: if it is too big, it isn’t a gadget however cool it is. Urban (I think) asked whether some four-wheel drive Volvo could be seen as a gadget, but I said that it couldn’t – unless it had an autopilot. Then I thought that Segway definitely is a gadget, although that it is big compared to something hand-held. I said that the whole is important: manuals, the packaging, etc – a potential gadget can be ruined if it is poorly packaged or documented. This led me to think that the Babybjörn definitely is a gadget: it is a brilliant product; very well designed and packaged.
Perhaps the discussion got quite out of hand. I’m not sure whether the Babybjörn actually qualifies as a gadget. Perhaps gadgets must be electronic devices. But what we were talking about was a quality that Poppendieck calls “perceived integrity” – based on what Kim Clark calls “external integrity” (he has written several books about this, but I’m on the train now so I can’t link to them at Amazon). Perceived integrity is what we were actually talking about, perhaps constrained a little by what constitutes a gadget. Poppendieck writes, for example, about Google and how it seems to be designed with a profound knowledge of what you want; it’s as if they “were inside my head,” she writes.
She also talks about “conceptual integrity”, which is something vital to achieve perceived integrity: with poor conceptual integrity, it’s hard, if not impossible, to create a user interface that just feels right. In this context, she talks about refactoring as the system evolves. The users’ reality evolves, so what was once a system with perceived integrity, might in a few years lose that quality; so when the system evolves with its users, the problem is that conceptual integrity must be maintained, to which the solution is constantly refactoring.
Ever since I read Refactoring by Martin Fowler the first time, I’ve repeated the refactoring mantra to myself – but I seem to always do too little of it (although I’m definitely getting better). An experience last week reminded me again that each time I think that something needs to be refactored, I should do it immediately instead of putting it off to a later time.
I’ll continue this post later.