Pandemonium

I liked the passage in Emergence about how Oliver Selfridge of MIT came up with his “Pandemonium” idea. He had been experimenting in the 1950s with teaching computers to learn, and as he re-read John Miltons Paradise Lost the image of shrieking demons enlightened him.

This gave birth to a new paradigm for pattern recognition which Selfridge dubbed “Pandemonium” (Greek for “all the demons”), where the task is split between a number of demons voting (or “shrieking” in his metaphor) to higher levels of demons, with a super-demon collecting the votes.

In the book, the example is a hand-writing recognition system. Steven Johnson writes, “imagine a system with twenty-six individual demons, each trained to recognize a letter of the alphabet. The pool of demons is shown a series of words, and each demon ‘votes’ as to whether each letter displayed represents its chosen letter.” The votes then determine which character it is most likely to be.

Then extend this system by a set of lower-level demons trained to recognize shapes, such as straight lines and circles. These report to the demons that are trained to recognize characters, and the system learns by strenghtening or weakening the connections between these lower and higer level demons. Provided that it’s the same person’s hand writing being analyzed, Johnson writes, “the system learns to associate specific assembles of shape-recognizers with specific letters and soon enough is capable of translating entire sentences with remarkable accuracy.”

The system Selfridge described – with its bottom-up learning, and its evaluating feedback loops – belongs in the history books as the first practical description of an emergent software program. The world now swarms with millions of his demons.

(I wonder whether Unix daemons got their name from Selfridge. Not that they are emergent or anything.)

The book is about all kinds of emergence, whether in ant colonies, cities or software. But I don’t think it mentions emergent design of software. It does mention the genetic algorithm, though, where you define fitness rules and feed it a piece of code that it mutates and cross breeds for a few thousand generations, using the fitness rules for the natural selection – all in order to produce a more effective piece of code. When I read about this in Out of Control, Kevin Kelly wrote that the resulting organically grown software was essentially undecipherable.

Emergent software design is where the programmer acts as incubator, evolving the code in the direction it wants to go. The equivalent of the fitness rules are generic design principles as well as functional (or non-functional) requirements. But there aren’t one single programmer, but many distributed over space and time, shrieking like Selfridge’s demons through code – or if you prefer another analogy: leaving pheromone trails like ants or slime molds through code.

XP argues that for emergent design to work, you need to keep the code as simple as possible – no unnecessary complexity – and to refactor as you learn. It is also important, XP says, to program in pairs and to frequently switch who sits with whom, so that everyone on the team has spent time with each part of the system. Then everyone must be present in all meetings and work in an office space that encourages communication.

Then there’s the initially agreed-upon system metaphor (which indeed can evolve as the system evolves) that serves as guide for the programmer ants/demons. It’s their “culture” or their “genetic configuration”, that determines what they do upon hitting a pheromone trail; that makes sure that they put stuff where it belongs – like the harvester ants put their garbage in a place as far away from the anthill, but also as far away from the ant cemetary as possible, the programmers put code where it makes sense to put it, paying attention to complexity (cohesion and coupling) and refactoring to maintain clarity. The system metaphor and the communication – oral and through code – are critical for emergent design to work, to produce good designs.

See also my previous post.