I’ve just read Clay Shirky’s new article, The Collapse of Complex Business Models, which is in turn based on Joseph Tainter’s 1990 book The Collapse of Complex Societies [amazon.com, amazon.co.uk]. Shirky summarises Tainter’s analysis of why ancient cultures such as the Romans and Maya collapsed so catastrophically:
A group of people, through a combination of social organization and environmental luck, finds itself with a surplus of resources. Managing this surplus makes society more complex — agriculture rewards mathematical skill, granaries require new forms of construction, and so on. Early on, the marginal value of this complexity is positive — each additional bit of complexity more than pays for itself in improved output — but over time, the law of diminishing returns reduces the marginal value, until it disappears completely. At this point, any additional complexity is pure cost.
Tainter’s thesis is that when society’s elite members add one layer of bureaucracy or demand one tribute too many, they end up extracting all the value from their environment. […] Complex societies collapse because, when some stress comes, those societies have become too inflexible to respond. […] In such systems, there is no way to make things a little bit simpler — the whole edifice becomes a huge, interlocking system not readily amenable to change.
When the value of complexity turns negative, a society plagued by an inability to react remains as complex as ever, right up to the moment where it becomes suddenly and dramatically simpler, which is to say right up to the moment of collapse. Collapse is simply the last remaining method of simplification.
Stop me if this seems too obvious to be worth saying, but isn’t this exactly what happens to big programs? “When society’s elite members add one layer of bureaucracy or demand one tribute too many” sounds disturbingly like “When the framework introduces a notion of a connection factory manager builder”.
This is what happened to Multics (and UNIX was born from its ashes). It’s what happened to Netscape (and Mozilla was born from its ashes, and died in turn of the same disease; and Firefox was born from its ashes). And — dare I say it? — it’s the natural death that comes to over-Fowlered programs.
Please note, by the way, Fowler fans, that I am talking about over-Fowlered programs. And, dear reader, it hardly need be said that I am not talking about the programs that you write. Heaven forfend. Your programs are tastefully designed, superbly executed, and engineered for a bright, shiny, future! It’s your colleagues I’m worried about. You’d better tell them about this important lesson!