The Curse of Knowledge & Complexity of Publishing

I just came back from the annual F+W Media innovation summit, where as a company we were discussing the challenges of the publishing business—a mature business—that is entering a period of creative destruction.

We had a guest speaker, William Welter (go check out the book here, releases this June), who offered up a fascinating story about the “curse of knowledge.”

A.C. Clarke, a well-known science-fiction novelist, was quite accurate in predicting the evolution of spaceflight. But in one particular novel, when a journalist was in space writing an article, what technology was depicted? A typewriter and regular old paper (plus carbon paper).

Why couldn’t Clarke foresee how writing and publishing would evolve? Presumably because he was too close to it (or maybe he just didn’t care about predicting THAT future).

A variation on this phenomenon is described in a New York Times article, “Innovative Minds Don’t Think Alike.” It says, “When it’s time to accomplish a task — open a store, build a house, buy new cash registers, sell insurance — those in the know get it done the way it has always been done, stifling innovation as they barrel along the well-worn path.”

This connects with another article I read on the future of scientific publishing: It argues how the people to revolutionize an industry always come from the outside of it, and aren’t usually big businesses. So, the piece asks, how is it that large, powerful organizations, with access to vast sums of money, and many talented, hardworking people, can’t do it themselves—and, as a result, disappear?

The first possible answer is that the people in charge are just stupid. Sometimes that might be the case. But more often, it’s the complexity of the structure that sinks it:

Each industry has (or had) a standard organizational architecture. That organizational architecture is close to optimal, in the sense that small changes mostly make things worse, not better. Everyone in the industry uses some close variant of that architecture. Then a new technology emerges and creates the possibility for a radically different organizational architecture, using an entirely different combination of skills and relationships. The only way to get from one organizational architecture to the other is to make drastic, painful changes. The money and power that come from commitment to an existing organizational architecture actually place incumbents at a disadvantage, locking them in. It’s easier and more effective to start over, from scratch.

Clay Shirky recently wrote on this phenomenon as well. It’s well worth a read: “The Collapse of Complex Business Models.”

It makes one wonder if, even with the brightest minds in publishing attempting to reinvent the business, will we fail because we are cursed with being too knowledgeable about it?

You might also like:

  • No Related Posts

9 thoughts on “The Curse of Knowledge & Complexity of Publishing

  1. Thad McIlroy

    You reference Michael Nielsen’s fine essay "Is Scientific Publishing about to be Disrupted." Nielsen in turn ends with: "My account of how industries fail was influenced by and complements Clayton Christensen’s book ‘The Innovator’s Dilemma’." ( That book is also my # 1 guide for advising publishing organizations on how to cope with disruptive change as they tackle the challenge of adapting to the future of publishing.

    You ask, "It makes one wonder if, even with the brightest minds in publishing attempting to reinvent the business, will we fail because we are cursed with being too knowledgeable about it?"

    What makes Christensen’s advice even more valuable is that he proffers a model that allows a role for the "brightest minds" to add their knowledge and experience, while blocking their compulsion to stifle innovation "as they barrel along the well-worn path."

  2. Deborah Bouziden

    I agree with Lynnda. The changes paper, the printing press, and then computers brought to our world were mind boggling. Just 15 years ago we weren’t as computer savvy as we are now. I predict in ten years from now we will be discussing this same issue except on a different level because we will have advanced through this one.

  3. Steve

    Well, one thing going on here is what a mathematician would call "local optimization". Imagine you are in the mountains and want to reach the highest elevation. "Local optimization" would mean to start at your present location and trend steadily upward. You will eventually reach a peak, but not (in general)the highest peak. For that, you would have to travel downward, and go to the foothills of the highest available peak, which you would have discovered by other methods.

    How is all this relevant to the inertia of large established organizztions? The evolution of a complex system involves many episodes of local optimization. I see it in my work creating and modifying custom database software for small businesses, but I feel confident that a similar process goes on in the design and updating of procedures within business organizations.

    Here’s how the process works as I have experienced it.

    (1) The decision maker or programmer is given a set of reauirements – a task the program or organization is to perform. A solution is generated that globally (not locally) optimizes the design to perform the task.

    (2) The requirements of the task change or increase. A design change is needed. Generally, it is uneconomic from the immediate viewpoint to tear down the existing organization or program and do a new global optimization for the amended task requirements. Instead, a "local optimization" is performed. What is the quickest and cheapest way to modify the program (or organizational structrue/process) to met the new requirements. In the world of software this is sometimes refered to as a "patch" because it is an addition or modification to the main body of the software – not a full rewrite.

    Fast-forward a number of years into this process. You now have a program (or an organization) that is perhaps more patches than not. It works – but a competitior starting from scratch has no incentive not to perform a global optimization and design a program (organizationn) that efficiently performs the new task as defined in the present. If the economic environment permits the necessary investment in the new structures, they will inevitably be superior to the older "patchwork" structure – always assuming the new design is competent.

    That’s how I think it happens,


  4. Lynnda Ell

    Jane, sometimes we can learn a lot by looking at the past. Publishing had a complete paradigm shift when paper was invented and another one when the printing press took the place of the quill pen. (People were martyred when printing presses were breaking into the industry.)

    Every paradigm shift includes upheaval and trauma. In my opinion, we’re seeing only the little snowball of change at the top of a snow-covered hill. The changes are small and slow, now, but momentum is increasing. In the not-too-distant future, I expect to see even greater change.

    As everyone tells me, publishing is a business. Work, processes and products that make a profit, today, may not meet that goal toorrow. To make a profit tomorrow, we need to keep our heads (pun intended) and be ready to move in any direction. After all, if most electronic goods companies can recreate themselves every three to five years, we know it can be done. We just need to learn how to do that for ourselves and keep our audience ready to read.

  5. TimBarrus

    I want to be a teenieweenie voice in the destruction of publishing as we know it. Even "nice" folks in the digital world become very disgruntled with that voice. Ornery. But one just marches on…

    Sometimes with the French flag clasped hard in one’s bloody hand as one steps over the rolled heads and dead bodies. They only are. The flag only is. Metaphor is metaphor. It’s still powerful.

    What puts a burr up my butt are the "digital" thinkers who employ the old paradigms of marginalizing people with ideas they do not agree with, as they build brick walls of exclusivity around the platform. This is old media disguised as new media but is wearing a new schmata. Pretty dress. She’s the same old whore.

    The "idea" of comments embraces inclusion. Not brick walls (oh, we do not want to hear the ideas of the little people belongs to the era of the Knopf who cannot be bothered). Finding publishers who embrace that idea ARE FEW AND FAR BETWEEN.

    It’s the IDEA that publishing NEEDS to change that is radical.

    I don’t have TIME to write queries to editors anymore. Get off the pot. Publish me or not. I’ll take my ideas to comments because you don’t pay that much anyway.

    They don’t like that. Miss Pringles gets all huffy. She’ll come after you in comments, too.

    Like anyone could trash me in comments that haven’t been published on the Internet a ZILLION times. La-te-da.

    I deal with BOYS WITH AIDS (lots of skateboards in the hall and shoes) and they have GREAT IDEAS! Here’s one. Spray painted all over Paris: SKATE OR GO HOME.



    And, often, a push. As in get out of the way you old fat thing. Like it or not, we’re coming through.

  6. Tom Bentley

    Jane, the "Curse of Knowledge" theme reminds me of the saying of a famed teacher of Zen:

    "In the beginner’s mind there are many possibilities, in the expert’s there are few."
    – Shunryu Suzuki-roshi

    There’s always a chance that deep expertise in something leaves the mental joints unflexible, the window to new possibility with it shade drawn, blinders to seeing spring flowers if it’s been a long winter.

    Another Zennish tale (abridged): a famous professor comes to take tea with a Zen master. The professor is very knowledgeable about many subjects and holds forth on many of them while the master quietly prepares the tea. The master starts pouring the tea in the professor’s cup and keeps pouring and pouring, spilling tea everywhere, until the professor shouts, "The teacup is full! Stop!"

    The Zen master then said "You are like this tea cup, so full that nothing more can be added. Come back to me when the cup is empty. Come back to me with an empty mind.”

    Maybe the publishing industry needs a little Zen?

  7. Theresa Milstein

    You’re right. That’s probably why the biggest changes are being foisted upon the industry from the outside. Some writers self-publish, there are iPhone apps, there are serial blogs on the Internet. We now have kindles. The reaction is to resist it all instead of figuring out how best to incorporate, complete, or come up with something new.

  8. Colette

    Jane, great question. It doesn’t mean it will be ‘wrong’, but it will be different – unless folks from outside the business are brought in to think it through. I like to say, "everyone sees things differently depending on where they sit."

  9. Natalie Cutsforth

    Thank you, these are questions that need to be taken seriously. Without a mirror, how do we see ourselves? Through the eyes of others. Outsiders with different perspectives and experiences are most likely to come up with an innovative approach to bridge a new business model in publishing. I recently suggested to a favorite magazine of mine that they pull together a group of volunteers from their readership with technology and social media background to brainstorm the next step for their business.