Why Innovation Is So Hard: Techniques for Accelerating Innovation Series Part 1

Published on INFORMS Analytics Magazine (Joseph Byrum)

Author’s note: This series Techniques for Accelerating Innovation will explore a new approach to innovation grounded in the Adaptive Response Framework—observe, orient, decide, act—which helps organizations navigate complexity with agility.

Traditional methods for innovation are slow, linear and hampered by a number of hurdles. This is the first in a series of blogs that will explore a new approach to innovation, but this installment will begin by examining the limits of traditional methods to innovating.

Not so long ago, the introduction of a new cellphone meant the availability of great new features and speedier operation. When Steve Jobs introduced the first iPhone in 2007, it changed the way people thought about communicating. Subsequent releases of the device were so eagerly anticipated that shortages proved common [1]. Fast forward to the present and an increasing number of users see little reason to upgrade. They’re holding onto their phones longer [2] because the iPhone has followed a well-established pattern – a pattern that illustrates the limits of innovation.

Author, scientist and thinker Vaclav Smil observed there have only been a handful of truly breakthrough inventions: the wheel, the generator, the transformer and the internal combustion engine [3]. “And we haven’t made any fundamental difference to this in 100 years,” he explains [4]. Every invention that has followed has been a gradual refinement of those concepts, resulting in what can be described as an “S curve” of innovation.

The basic idea behind an S-curve is that growth will start slowly, accelerate and then slow. There is a birth at the beginning, growth in the middle, and then death or obsolescence at the end. When you look at the end of the curve, you can see the diminishing returns over time following the initial burst in capabilities – the point where product launches start to become a bit repetitive and uninspiring. New products will then be introduced, starting again from the beginning of a new S-curve.

This is a pessimistic view that suggests truly new innovations become rarer with time.

Blocks to Creation (Pre-Market)

An inventor faces many difficulties simply in recognizing the need for a new invention:

Lack of usable data

Information can be what data scientists call noisy – they don’t give valid signals. Noisy datasets feature missing information, redundant information or outliers that distort patterns [5]. Not being able to separate the signal from the noise can cause an inventor to misperceive a need and create an invention that’s not actually useful.

Groupthink

It is widely recognized that groupthink kills innovation. A study by Girotra et al. found that the quality of ideas generated by a group was inferior to ideas generated first by individuals then shared in a group (hybrid) [6]. One pundit has even called it an innovation “antibody” [7]. In the new virtual environment, groupthink is on the rise, with one author coining the phrase “Zoomthink” [8].

Novelty-vs.-utility paradox

Ella Miron-Spektor of the Technion-Israel Institute of Technology has written extensively about how companies often choose to create a less innovative product because people resist change (hence innovation) even when it will improve their circumstances [9]. She says, “product developers will assess whether to develop a product that is slightly improved over a current version that may be more useful to customers or to develop a sophisticated product that is more original but requires a great investment (on the user’s part) in learning how to use it” [10]. This tendency helps to explain the mediocrity of some inventions [11].

Modularity

Modularization has become an accepted means to advance innovation in complex organizations. It can be defined as “a general set of design principles for managing the complexity of such large-scale inter-dependent systems” [12]. An example would be grouping company employees by product lines, functions or regions. This form of labeling tends to reduce innovation because the labels frequently become silos – working groups that do not communicate with each other. Ethiraj and Leventhal have warned against the “destabilizing effects of overly refined modularization” [13], by which they mean breaking a process up into teams that are too small.

Blocks to Diffusion (In-Market)

The second kind of block to innovation is a block to diffusion. Here are some of the challenges faced by individuals and organizations investing in innovations that may never succeed in the marketplace:

Multiple discoveries

An individual or company may create something new but it may still fail if, for example, another inventor moves more quickly to bring the same idea to the market first. This is a phenomenon known as “multiple discoveries,” “simultaneous inventions” or “idea twins” [14] – innovations that happen simultaneously or in rapid succession, cancelling out competitive advantage. While extraordinary individuals like Thomas Edison are seen as the sole inventors of key products, even Edison’s ideas had twins, as the history of the storied lightbulb shows [15, 16]. Information scientist Eugene Garfield quotes anthropologist A. L. Kroeber as saying, in effect, that “if a particular inventor had died as an infant, there is a good chance that the invention would still have been conceived by someone else.”

Path dependence 

Even when innovations are truly new and have the potential to be useful, they may never get used or adopted due to a phenomenon known as path dependence, which is the idea that “current and future states, actions or decisions depend on the sequence of states, actions or decisions that preceded them” [17]. This dependence has been blamed for the adoption of inferior technologies. The most famous example of this is the 19th century choice of the QWERTY keyboard layout that remains the standard today. The story, as first relayed by Davids in 1985 and still commonly retold, is that “the standard ‘QWERTY’ keyboard arrangement, introduced in the 1870s, is dramatically inferior to an arrangement offered by August Dvorak in the 1930s. We are, however, regrettably locked into the inferior arrangement by a coordination failure: no one trains on the Dvorak keyboard because Dvorak machines are hard to find, and Dvorak machines are hard to find because no one trains on Dvorak keyboards. The process is said to be path dependent in that the timing of the adoption of QWERTY, and not its efficiency, explains its survival” [18].

Rembrandts in the attic

Another problem affecting innovation involves the afterlife of inventions after they are created. Even when the creators or owners obtain intellectual property protection, they can become, in the words of Rivette and Klein, “Rembrandts in the attic,” instead of being put to use [19]. And all too often companies fail to defend patents, let them expire or fail to leverage them – a problem identified by Julie Davis and Suzanne Harris in their book, “Edison in the Boardroom” [20]. Davis and Harris recommend an IP Pyramid moving upward from mere defense to strategic shaping (defend, manage, capture, synthesize, shape) [21].

Counterfeit innovation 

Some inventions hailed as innovation are merely tweaks on existing inventions. Ideally, such non-innovative inventions will be screened out at the time of patenting. Under U.S. law, for example, an invention must be novel, useful, non-obvious and “statutory” – meaning patentable under the law. Ideas cannot be patented, but processes, machines, manufactured items and material compositions can be so protected [22]. Since the bar for utility under different national standards can be very low, these faux inventions rarely get screened out.

Patent ambush 

A patent ambush occurs when the holder of a patent protecting an invention required for compliance with a technical standard (a so-called standard-essential patent) deliberately hides the fact of its intellectual property rights and then asserts these rights only after the standard is in force [23]. When this happens, the patent holder can prevent wide use of an essential invention. Antitrust laws attempt to curb this behavior, but it still happens [24].

The next part of this series will lay the groundwork for the understanding needed to better achieve innovation.

References and Notes

1. Marin Perez, 2009, “Apple sees iPhone 3GS Shortages,” InformationWeek, June 29, https://www.informationweek.com/government/apple-sees-iphone-3gs-shortages.
2. Abigail Ng, 2019, “Smartphone Users are Waiting Longer before Upgrading – Here’s why,” CNBC, May 17, https://www.cnbc.com/2019/05/17/smartphone-users-are-waiting-longer-before-upgrading-heres-why.html.
3. Vaclav Smil, 2021, “Grand Transitions: How the Modern World Was Made,” Oxford University Press.
4. Radio Spectrum interview with Vàclav Smil, “A Theory of (Almost) Everything,” IEEE Spectrum.
5. “Data Quality and Preprocessing,” http://jcsites.juniata.edu/faculty/rhodes/ml/datapreprocessing.htm.
6. https://knowledge.wharton.upenn.edu/article/how-group-dynamics-may-be-killing-innovation/
7. Phil McKinney, 2021, “Groupthink Kills Innovation,” Killer Innovations podcast, https://killerinnovations.com/groupthink-kills-innovation/.
8. Erica Dhawan, 2020, “How to Fight the Groupthink that Happens When we Work Virtually,” Fast Company, Nov. 11, https://www.fastcompany.com/90572111/how-to-fight-the-groupthink-that-happens-when-we-work-virtually.
9. Ella Miron-Spektor and Miriam Erez, 2016, “Chapter on Paradox and Creativity Final,” https://www.researchgate.net/figure/Creativity-Paradoxes_tbl1_306259882.
10. https://www.researchgate.net/publication/305719267_Looking_at_Creativity_through_a_Paradox_Lens_Deeper_Understanding_and_New_Insights.
11. This tradeoff operates differently in the private vs. public sector. As Orazem and van Bilon have noted, in the private sector innovations advance more quickly than in the government domain, but they are less novel. The public sector advances more slowly but can take on more risk. Furthermore, the goal of innovation is different. In the private sector, the goal is to increase returns to investors, whereas in the public sector, the goal is to “create/incubate a new sector or industry for the country,” “What Makes Public Sector Innovation Different?” fedscout.com.
12. Sendil K. Ethiraj and Daniel Levinthal, 2004, “Modularity and Innovation in Complex Systems, Management Science, Vol. 50, No. 2, pp. 159-173.
13. Ibid.
14. Michaël Bikard, 2020, “Innovation Plucked from the Zeitgeist,” INSEAD Knowledge, https://knowledge.insead.edu/strategy/innovation-plucked-from-the-zeitgeist-15356?vid=362.
15. For an artistic rendering of innovation as a lightbulb, see the illustration from “Paradigm Shifts in Science: Insights from the Arts,” Lasker Foundation, https://laskerfoundation.org/paradigm-shifts-in-science-insights-from-the-arts/.
16. Edison’s lightbulb was anticipated by Allessandro Volta, Humphrey Davy, Waren de la Rue, Joseph Swan, Henry Edward, Matthew Evans, William Sawyer and Albon Man, https://www.livescience.com/43424-who-invented-the-light-bulb.html.
17. Complexity Explorer glossary, Santa Fe Institute, https://www.complexityexplorer.org/explore/glossary/350-path-dependence.
18. https://ecsocman.hse.ru/data/018/784/1216/0770book.pdf
19. Kevin G. Rivette and David Kline, 1999, “Rembrandts in the Attic: Unlocking the Hidden Value of Patents,” Brighton, Mass.: Harvard Business Review Press.
20. Julie L. Davis and Suzanne S. Harrison, 2002, “Edison in the Boardroom: How Leading Companies Realize Value from Their Intellectual Assets,” New York: Wiley.
21. Julie L. Davis, Suzanne S. Harrison and Patrick H. Sullivan, “Edison in the Boardroom Revisited: How Leading Companies Realize Value from Their Intellectual Property,” New York: Wiley.
22. Patentability Requirements Under U.S. Patent Law, Justia.
23. https://thelawreviews.co.uk/title/the-intellectual-property-and-antitrust-review/european-union 
24. Ibid.