I just came back from the annual F+W Media innovation summit, where as a company we were discussing the challenges of the publishing business—a mature business—that is entering a period of creative destruction.
We had a guest speaker, William Welter (go check out the book here, releases this June), who offered up a fascinating story about the "curse of knowledge."
A.C. Clarke, a well-known science-fiction novelist, was quite accurate in predicting the evolution of spaceflight. But in one particular novel, when a journalist was in space writing an article, what technology was depicted? A typewriter and regular old paper (plus carbon paper).
Why couldn't Clarke foresee how writing and publishing would evolve? Presumably because he was too close to it (or maybe he just didn't care about predicting THAT future).
A variation on this phenomenon is described in a New York Times article, "Innovative Minds Don't Think Alike." It says, "When it’s time to accomplish a task — open a store, build a house, buy new cash registers, sell insurance — those in the know get it done the way it has always been done, stifling innovation as they barrel along the well-worn path."
This connects with another article I read on the future of scientific publishing: It argues how the people to revolutionize an industry always come from the outside of it, and aren't usually big businesses. So, the piece asks, how is it that large, powerful organizations, with access to vast sums of money, and many talented, hardworking people, can't do it themselves—and, as a result, disappear?
The first possible answer is that the people in charge are just stupid. Sometimes that might be the case. But more often, it's the complexity of the structure that sinks it:
Each industry has (or had) a standard organizational architecture. That organizational architecture is close to optimal, in the sense that small changes mostly make things worse, not better. Everyone in the industry uses some close variant of that architecture. Then a new technology emerges and creates the possibility for a radically different organizational architecture, using an entirely different combination of skills and relationships. The only way to get from one organizational architecture to the other is to make drastic, painful changes. The money and power that come from commitment to an existing organizational architecture actually place incumbents at a disadvantage, locking them in. It’s easier and more effective to start over, from scratch.
Clay Shirky recently wrote on this phenomenon as well. It's well worth a read: "The Collapse of Complex Business Models."
It makes one wonder if, even with the brightest minds in publishing attempting to reinvent the business, will we fail because we are cursed with being too knowledgeable about it?