Learning and adaption in software development

I’m going to quote a long passage from Mitchell Waldrop’s Complexity:

“It’s as if you’ve parachuted into some negotiating session in Japan,” says [Brian] Arthur. “You’ve never been in Japan before, you don’t know how the Japanese think, or act, or work. You can’t quite understand what is going on. So most of the things you do are completely out of cultural context. And yet over time, you notice that some of the things you do are successful. So slowly, you and your company somehow learn to adapt and behave.” [...] Think of the situation as a competition like chess, he says. Players have fragmentary information about their opponents’ intentions and abilities. And to fill in the gaps, they do indeed use logical, deductive reasoning. But they can only use it to look a few moves ahead, at most. Much more often the players operate in a world of induction. They try to fill in the gaps on the fly by forming hypotheses, by making analogies, by drawing from past experience, by using heuristic rules of thumb. Whatever works, works – even if they don’t understand why. And for that very reason, induction cannot depend upon precise, deductive logic.

At the time, Arthur admits, even he found this troubling. “Until I went to Santa Fe,” he says, “I thought that an economic problem had to be well defined before you could even talk about it. And if it wasn’t well defined, what the hell could you do with it? You certainly couldn’t apply logic.”

“But then John Holland taught us that this isn’t so. When we talked to John and read his papers we began to realize that he was talking about cases where the problem context isn’t well defined, and the environment isn’t stationary over time. We said to him, “John, how can you even learn in that environment?”“

Holland’s answer was essentially that you learn in that environment because you have to: “Evolution doesn’t care whether problems are well defined or not.” Adaptive agents are just responding to a reward, he pointed out. They don’t have to make assumptions about where the reward is coming from.

Waldrop goes on to write that this was the whole point of John Holland’s “classifier systems.” A classifier system, as defined by Holland, was a kind of hybrid of a rule-based system (as in expert systems) and genetic algorithms. Holland invented a very powerful metaphor for such systems, where “classifiers” are seen as goods on a free market, bought and sold by agents in auctions, and with learning being constituted by some getting richer than others. The point in this context is that a classifier system “could operate in an environment that was not well defined at all. Since the classifier rules were only hypotheses about the world, not “facts,” they could be mutually contradictory. Moreover, because the system was always testing those hypotheses to find out which ones were useful and led to rewards, it could continue to learn even in the face of crummy, incomplete information – and even while the environment was changing in unexpected ways.”

This passage, to me, is about software development. The problems are seldom well defined, still you are expected to work with them and produce good solutions. Being a developer is like playing chess with the customer, in the sense that the developer has to understand the intentions of the customer – but, unfortunately, also in the sense that customers and developers often act as if they are competing, when they should be collaborating. This is especially true in cases where the customer has hired someone to do some analysis and put together a requirements specification, which is then handed to a team of developers – who, as they stumble upon things that are unclear in the spec, don’t get much help from either the customer, who obviously didn’t believe he or she was capable of deciding what was needed, or the consultant that wrote up the spec, who most likely is working with other projects.

It’s interesting, because in the computer models of evolutionary processes that are described in Waldrop’s book, the scientists had found that competition among agents in fact resulted in collaboration. I guess this is explained by the fact that all agents in the computer models were trying to pursue the same goal. Does that tell us that the goals of the customers and the developers are different? Then perhaps we should agree upon the goals at the start of each new project?