Upload
christian-tillegreen
View
538
Download
0
Embed Size (px)
DESCRIPTION
The major finding of this thesis is thus that desirable capabilities and behaviors of crowd and screen team are highly distinctive but complementary; they are interdependent and indispensable to enable accelerated innovation in the context of Novozymes. It follows that sustainable use of crowd-sourcing as an innovation accelerator in Novozymes must pay special attention to ensure that innovator crowds stay creative and engaged but also to muster critical, yet open-minded screen teams. The value proposition to these expert teams must be that facing the creative, ‘noisy’ chaos of the crowd can have significant paybacks in the form of new maps of insights which could be much more intriguing than ‘just’ a couple of new good ideas.
Citation preview
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
1 | P a g e
Creative crowds and wise screen teams are needed to accelerate
customer-centric innovation
A case study of an internal online ideation at Novozymes
Copenhagen Business School
MSc. Management of Innovation & Business Development
January - 2014
Christian Brix Tillegreen (021187-2179)
-
Supervisor: Jörg Claussen
Dept. of Innovation and Organizational Economics
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
2 | P a g e
Std. Pages: 68 (STUs: 129.430)
Executive Summary
Development of innovative solutions which meet market needs timely and accurately is becoming more
and more critical for companies in order to stay competitive. With the advent of the digital
collaboration age crowdsourcing offers a novel approach to conceive and co-develop innovative
solution concepts online. The implicit attractiveness of crowdsourcing lies in the expectation of an
acceleration effect: while traditional communication of market needs and ensuing solution development
involves lengthy, error-prone communication processes, crowdsourcing opens the possibility to engage
all critical business functions simultaneously and in a real-time fashion. Thus, market needs could be
shared instantaneously across the organization and turned into solutions concepts faster and more
accurately through cross-functional concept development.
The present thesis investigates certain success factors which need to be in place in order to further
strengthen the role of crowdsourcing as an innovation accelerator in the Danish biotech company
Novozymes. Earlier research on an internal crowdsourcing campaign in Novozymes had already
indicated three key success factors: (1.) diversity of crowd composition, (2.) absorptive capacity and
capability of the idea-receiving organization and (3.) a culture permissive to new ideas and innovation
(Lauto et al., 2013).
The present thesis now focusses on a more recent dataset, recorded from a subsequent internal digital
campaign, called ‘New Claims for Detergents’. Here the focus of attention lied with particular
capabilities of the crowd and the screen team, respectively. ‘Crowd wisdom’ was investigated as the
crowd’s capability to create a sufficiently large number of quality ideas, to co-develop these through
cross-functional discussion as well as their collective receptiveness to novelty and to market needs as
articulated in the presented ideas. The ‘screen team’s wisdom’, on the other hand, related to the
professional scrutiny of this expert group and their cognitive capability of gaining new insights from
the presented ideas and their discussion by the crowd.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
3 | P a g e
With regards the crowd’s wisdom it was found that the present crowd of 105 employees was highly
engaged in sharing and actively discussing 76 diverse ideas, many of which featured novelty and
market needs. Moreover, idea novelty, articulation of market needs and cross-functional discussions
where significant predictors of high dot-vote scores as awarded by the crowd. Interestingly, the screen
team’s perception of the same ideas and discussions was distinctively different: novelty, market needs
and cross-functional discussions were not correlated with the screen team scores. This was interpreted
as the result of the team’s critical scrutiny for idea consistency and in particular for supporting
evidence. Furthermore, this 5-person team took the ad hoc decision to go beyond their originally stated
objective of identifying a couple of trophy ideas (‘idea hunting’) because they became intrigued by
emerging idea-connections during their meticulous work with the idea-universe generated by the
crowd. They conceived a systematic approach to cluster and organize all ideas which lead to a
‘strategic’ idea map. This was an unintended product of their collective cognitive process and enabled
new angles of discussion in their management circles.
While it is too early to conclude whether this serendipitously arising cognition process can form the
foundation of a new repeatable ‘crowd-strategizing’ protocol, it seems very unlikely that the condensed
insights of the screen team would have been possible without the spontaneous, collaborative mass-
creativity of the crowd.
It is discussed whether it is desirable to push innovation burden from the screen team onto the crowd
(e.g. by demanding better idea consistency and evidence), because one has to consider the inherent risk
of negatively impacting the very crowd-creativity without which the screen team cannot unfold its
potential of insight creation and ‘wisdom’.
The major finding of this thesis is thus that desirable capabilities and behaviors of crowd and screen
team are highly distinctive but complementary; they are interdependent and indispensable to enable
accelerated innovation in the context of Novozymes.
It follows that sustainable use of crowd-sourcing as an innovation accelerator in Novozymes must pay
special attention to ensure that innovator crowds stay creative and engaged but also to muster critical,
yet open-minded screen teams. The value proposition to these expert teams must be that facing the
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
4 | P a g e
creative, ‘noisy’ chaos of the crowd can have significant paybacks in the form of new maps of insights
which could be much more intriguing than ‘just’ a couple of new good ideas.
Table of Content
1. Introduction...................................................................................................................................... 7
1.1 Problem statement ..........................................................................................................................................9
1.2 Research hypotheses .....................................................................................................................................11
1.3 Research relevance .......................................................................................................................................15
1.4 Disposition ....................................................................................................................................................16
2. Literature review ........................................................................................................................... 17
2.1 Innovation – in the science-based corporation .............................................................................................18
2.2 Crowdsourcing .............................................................................................................................................19
2.2.1 Assembling the “wise crowd” ................................................................................................ 20
2.2.2 Crowdsourcing in the corporation ......................................................................................... 21
2.3 Collaboration modes .....................................................................................................................................25
3. Company and case description ..................................................................................................... 28
3.1 Description of Novozymes ...........................................................................................................................28
3.2 Innovation in Novozymes .............................................................................................................................30
3.3 Innovation processes in Novozymes ............................................................................................................31
3.4 Case: New Claims for Detergent Enzymes ..................................................................................................32
3.4.1 The process of “New Claims for Detergents” ........................................................................ 33
3.4.2 Scope and plan ........................................................................................................................ 33
3.4.3 Mobilization and composition the crowd ............................................................................... 34
3.4.4 The online ideation phase ....................................................................................................... 34
3.4.5 Screening and selection of ideas ............................................................................................. 35
4. Methodology ................................................................................................................................... 37
4.1 Research approach and design ......................................................................................................................37
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
5 | P a g e
4.1.1 Data collection ........................................................................................................................ 38
4.1.2 Qualitative data ...................................................................................................................... 39
4.1.3 Quantitative data .................................................................................................................... 40
4.2 Two-fold analysis of crowd versus screen team ...........................................................................................43
4.3 Limitations ....................................................................................................................................................44
5. Analysis ........................................................................................................................................... 45
5.1 Analysis part 1: Descriptive analysis and distribution of ideas ...................................................................46
5.1.1 Outcome of “New claims for detergent” ................................................................................ 46
5.1.2 Ideas with cross-functional involvement: ............................................................................... 47
5.1.3 Novelty of ideas: ..................................................................................................................... 49
5.1.4 Clustering of the ideas ............................................................................................................ 50
5.1.5 Responsiveness to market needs: ............................................................................................ 51
5.1.6 Screen team evaluation: ......................................................................................................... 54
5.2 Conclusion of analysis part 1: ......................................................................................................................56
5.3 Analysis part 2 ..............................................................................................................................................58
5.3.1 Putting the numbers into relation ........................................................................................... 58
6. Discussion ........................................................................................................................................... 60
6.1. Revisiting the research hypotheses ..............................................................................................................60
6.2 The crowd‘s excitement about the presented ideas was not shared by the screen team ...............................62
6.2 The screen team’s perspective ......................................................................................................................63
6.3 From singular ideas to ‘strategic idea landscapes’ .......................................................................................65
7. Conclusions and Perspectives - Moving from idea-hunting to ‘crowd-strategizing’ .............. 68
7.1 Further Research ...........................................................................................................................................71
Bibliography .......................................................................................................................................... 72
Appendices .................................................................................................. Error! Bookmark not defined.
Appendix 1: ........................................................................................................ Error! Bookmark not defined.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
6 | P a g e
Appendix 2: ........................................................................................................ Error! Bookmark not defined.
Appendix 3: ........................................................................................................ Error! Bookmark not defined.
List of Figures and Tables
Figure 1; Disposition of the thesis .......................................................................................................... 16
Figure 2; the layers of crowds ................................................................................................................. 22
Figure 3; Modes of Collaboration ........................................................................................................... 26
Figure 4; Novozymes organization ......................................................................................................... 29
Figure 5; Process of online ideation ........................................................................................................ 33
Figure 6; Composition of the Crowd ...................................................................................................... 47
Figure 7; Distribution of new ideas ........................................................................................................ 49
Figure 8; Comprehensive idea map ........................................................................................................ 50
Figure 9; Statistical comparison of screen team scores given on commercial and technical criteria ..... 55
Table 1; Table of hypotheses .................................................................................................................. 11
Table 2; Pros and Cons of Crowdsourcing ............................................................................................. 23
Table 3; Total outcome of the dataset: .................................................................................................... 47
Table 4; Distribution of cross functionally discussed ideas .................................................................... 48
Table 5; Origin of the Top 25 ideas ........................................................................................................ 49
Table 7; Distribution of ideas with market articulation .......................................................................... 54
Table 8; Regression model of the screen team score and the crowd score: ............................................ 59
Table 9; Summary of findings and hypotheses acceptance .................................................................... 62
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
7 | P a g e
1. Introduction
“…You can’t ask customers what they want and then try to give that to them. By the time you get it
built, they’ll want something new…” - This quote by Steve Jobs is a great point of departure for this
thesis. The ability to develop new innovations that customers actually want is one of the single hardest
things to do for a modern corporation. Especially in times where technology and markets are changing
by the day, in relation to Moors law (Moore, 1965), the rapid development in production and new
market spaces are forcing companies to innovate faster and to come up with new ideas they can deliver
to the market more rapidly than the competitor.
One of the greatest challenges for innovation in modern corporations is the organization’s ability to
listen to the ‘market needs’ and to develop solutions which meet the customer’s needs accurately,
timely and cost competitively. In essence, this capability equals accelerated innovation, which should
give companies a substantial competitive advantage.
Looking at Danish biotech company Novozymes, the challenge of accelerating innovation is put high
on the agenda by top management. In April 2013 the new CEO, Peder Holk Nielsen, addressed this
challenge in a press release video about the new strategy in Novozymes in which he focusses on
customer-centric innovation as a new ambition for Novozymes:
“…The first focus area of the new leadership-team is on growth, it is going to be, bringing innovation
quicker, faster, from labs, from our research through business development and to our customers. Get
this process to work faster and to get more delivered to our customers, this is focus area number
one…” - Peder Holk Nielsen, CEO of Novozymes, interview from Spark TV 02/04/20131.
This thesis is set out to investigate how Novozymes handles this challenge by using digital tools and
internal crowdsourcing in their goal to deliver innovation faster to their customer, and more accurate to
the market needs.
Listening, understanding and responding to the market needs with competitive innovations is difficult
because of the organizational complexity: even the most skilled sales representatives and technical
1http://www.novozymes.tv/video/7631519/peder-holk-nielsens-view-on.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
8 | P a g e
service experts may not understand what the customer really wants and the reports they forward to the
other parts of the organization may dilute or even distort the essence of the customer’s need. As a
result, R&D more often than not may end up developing the wrong products too late. A partial solution
to this challenge lies in establishing multidisciplinary account teams, made up of sales reps, customer
service, supply chain and R&D experts. Still, many account teams may find themselves challenged in
lacking the necessary diversity of skills and insights – which they naturally will attempt to mitigate by
engaging into a dialogue with their cross-functional networks. Undoubtedly, such dialogue increases a
given account team’s chances to kick start the right innovation activities but the underlying problem of
lacking diversity still lingers because account teams will in most cases try to get their answers and
insights from their ‘usual suspects’ – i.e. the expert circles which the habitually consult and collaborate
with. Consequently, innovation will follow the established patterns of failures and successes. The
advent of collaborative online ideation tools which combine elements of traditional ideation and social
media seem to offer a more fundamental solution to the problem of limited diversity. Idea campaigns
can be designed in such a way that very large and very diverse crowds are composed of members
across all functions of the business system, all markets, segments and hierarchy levels. Naturally,
customers can be part of such ideations as well.
In a sense, online ideations facilitate a ‘level playing field’ discussion because everybody’s’ ‘voice’ is
equal and is being ‘heard’ by everybody else, and everybody’s dot-votes count the same. Thus,
differences in hierarchy are – at least partially – abolished. Hence, the anticipation is that in such an
arena there is a much higher chance that the actual market needs are being heard in an undistorted
fashion and responded to in a meaningful way by a motivated crowd of creative participants.
Novozymes has been using collaborative online ideations since 2011 in a successful manner and
routinely across business divisions. The first online ideation in Novozymes was studied thoroughly by
Lauto et al (2013). The main findings were that online ideations are a potent tool to boost front-end
innovation through digital collaboration.
The research subject for this thesis is to give a descriptive analysis of one particular Novozymes online
ideation campaign called ‘New Claims for Detergent’ and to investigate how the ‘market needs’ are
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
9 | P a g e
being perceived in the articulated ideas and in the crowd discussion. One of the most tantalizing
questions which this thesis tries to answer is whether there is significant evidence that the crowd
responds to market needs with innovative solutions.
“…Innovation has nothing to do with how many R&D dollars you have. When Apple came up with the
Mac, IBM was spending at least 100 times more on R&D. It's not about money. It's about the people
you have, how you're led, and how much you get it…” Steve Jobs quoted in "TIME digital 50"
in TIME digital archive (1999)
The overall objective of this thesis is not to arrive at general conclusions or validations about
crowdsourcing as such rather its aim is a more in-depth investigation of a single case on how internal
crowdsourcing can accelerate the innovation process in a global company. By applying newly gained
knowledge in the most appropriate way, and thereby pose some unambiguous propositions and
managerial recommendations for future acceleration of innovation within Novozymes.
1.1 Problem statement
The internal crowdsourcing exercise ‘New Claims for Detergents’ which is the subject of this thesis
was novel in Novozymes in the sense that it aimed at create new concepts in response to market needs
as articulated in the ideas from Sales and Technical Service participants. This was in contrast to earlier
internal crowdsourcing exercises which were mostly conducted within the R&D community
exclusively, with no or little representation of customer-facing employees. The crowd of ‘New Claims
for Detergents’ thus consisted of employees from Sales, Technical Service, Marketing and R&D,
spanning across the entire value creation chain of Novozymes. Participants were partially known
experts within the field (enzymes for detergent) and partially ‘unusual suspects’ i.e. people without an
expertise in detergent enzymes but with a high potential to contribute with good ideas and energy to the
cause. Typically, newcomers with a rising star reputation working in other business divisions were
chosen to complement the ‘usual suspects’. Diversity was also present in terms of geographical
location and tenure of participants. The dataset used for this thesis represents a digital ideation
campaign which lasted approx. 14 days, leading to 74 ideas and over 200 comments, produced by 105
employees from worldwide locations. The limitation however was that the ideation did not involve
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
10 | P a g e
customers directly but that representatives from Sales and Tech Service functioned as ‘proxies’. This
thesis will investigate the following aspects in order to answer the posted research questions:
The presented thesis studies the abilities of online ideation as a tool for idea generation filtering as well
as deeper insight creation into idea clusters, which extends beyond identification of singular winner
ideas. As argued by a number of scholars and researchers the ability of crowdsourcing to turn ideas into
innovations and new products it is difficult to measure or even predict (Poetz and Schreier, 2012).
The purpose of this thesis is therefore to contribute to the existing literature with empirical evidence
from the case of “New Claims for detergent”. The exploratory nature of the research has led to an
investigation of the effects of inventor diversity on the development of ideas. The question examined is
thus:
Which role does the ‘wisdom’ of the crowd and the screen team play in internal crowdsourcing as a
tool to accelerate a corporation’s response to market needs?
In order to answer this central research question the analysis will go into investigation of nine different
hypotheses as stated below. The examination of these research hypotheses will elucidate the different
conditions which need to be fulfilled in order to conduct online ideations successfully – which, in turn,
is expected to accelerate customer centric innovation processes. Based upon the results and managerial
recommendations will be proposed to aid the company’s ongoing exploration of opportunities within
crowdsourcing. It is thus hoped that this study makes a contribution to enhance and to optimize future
customer-centric innovation processes.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
11 | P a g e
1.2 Research hypotheses
In order to answer the question whether the crowd and the screen team in the present campaign can be
considered as ‘wise’ and thus having the potential to accelerate customer centric innovation, nine
hypotheses are stated which are clustered in four categories (Table 1). In the context of this thesis a
wise crowd is expected to:
(1.) Come up with a large proportion of high-quality ideas in relation to the stated challenge
(hypotheses 1-3)
(2.) Present a substantial proportion of novel ideas which are considered as having a high innovation
potential by crowd and screen team alike (hypotheses 4 and 5)
(3.) Present a substantial proportion of ideas which trigger cross-functional discussions which serve to
develop these ideas in a collaborative fashion. Such ideas are ranked highly by crowd and screen
team, respectively (hypotheses 6 and 7)
(4.) Propose a substantial fraction of ideas which contain a clear articulation of market needs.
Furthermore, such ideas are ranked highly by crowd and screen team alike (hypotheses 8 and 9).
Table 1; Table of hypotheses
Hypothesis
category
Hypo-
thesis
no.
Hypothesis text Criteria for
acceptance
Method of
testing
Idea
quality
1 Ideas of high quality are characterized by a substantial length
of text in order to articulate thoughts of a certain complexity.
Such ideas represent the majority in the present idea
population.
75% of all ideas
contain more
than 420
characters
Descriptive
statistics
2 Ideas of high quality draw multiple comments from the crowd
and constitute the majority in the present idea population.
75% of all ideas
are associated
with at least 2
comments
Descriptive
statistics
3 Ideas of high quality contain supporting references to internal
or external sources, which is the case for a substantial fraction
of present ideas.
25% of all ideas
contain
references
Descriptive
statistics
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
12 | P a g e
Idea
Novelty
4 Novel ideas represent a substantial fraction of the idea
population
25% of all ideas
were judged as
novel by the
screen team
Descriptive
statistics
5 Novelty is a predictor of high idea ranking. This is expressed
by significant correlation of novelty with idea scores, as given
by the crowd and the screen team, respectively
p<0.05 Regression
analysis
Idea co-
creation
6 Ideas with cross-functional comments constitute a substantial
proportion of the entire idea population
25% of all ideas
are associated
with cross-
functional
comments
Descriptive
statistics
7 Co-creation is a predictor of high idea ranking. This is
expressed by significant correlation of cross-functional
discussion with idea scores from crowd and screen team,
respectively.
P<0.05 Regression
analysis
Responsiv
eness to
market
needs
8 Ideas which contain a clear articulation of market needs
constitute a substantial proportion of all ideas
25% of all ideas
contain market
needs
articulation
Descriptive
statistics
9 Ideas with market needs articulation are ranked highly. This is
expressed by significant correlation of ‘market need
articulation’ with idea scores from crowd and screen team,
respectively.
P<0.05 Regression
analysis
The reasoning behind picking the various acceptance criteria for the nine stated hypotheses is explained
in the following: arbitrary criteria were used for all hypotheses which were tested by descriptive
analysis (hypotheses no. 1-4, 6 and 8). Statistical significance (p>0.05) was the acceptance criterion
when testing hypotheses no. 5, 7 and 9.
Hypothesis 1: ‘Substantial length’ is arbitrarily set to mean 420 characters of text based on the
communication principle in the social medium Twitter: in Twitter a post is limited to 140 characters,
but the immense global success of the medium proves empirically and beyond doubt that 140
characters comprise an information ‘package’ or ‘string’ whose length is fully sufficient to express a
virtually unlimited number of reasonably complex thoughts. In the context of this thesis it is further
inferred that a well-articulated idea consists of at least three such Twitter-length statements (3 x 140 =
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
13 | P a g e
420): first statement pertaining to background or context of the presented idea, second statement
embodying the problem at hand and the third statement describing the proposition of its solution.
‘Majority of ideas’ is defined to mean 75% of all ideas posted.
Hypothesis 2: It is assumed that the number of comments which an idea triggers is likely to be an
indicator of content relevance and thereby of idea quality. It is reasoned that ideas with no comments
are most irrelevant since nobody in the crowd was enticed to respond to them. Ideas which drew only
one comment are equally likely to have a no or only very low relevance since singular comments may
just be an expression of spontaneous (dis)approval. Thus, the minimum number of comments which an
idea needs to draw from a crowd to be considered relevant – at least to some extent – is two. Two
comments per idea are viewed as a minimum threshold for relevance: an initial idea is sufficiently
relevant to trigger somebodies comment and somebody else adds another comment which contributes
an additional perspective to the idea or the first comment. One could also say that one idea and two
ensuing comments constitute a set of minimum requirements to justify the use of the term ‘idea
discussion’.
Hypothesis 3: References to internal or external sources are a strong indicator of idea quality since
references support ideas with ‘reason to believe’ originating from others than the idea proposer. The
crowd of the investigated campaign consisted of 50% participants from R&D. It is therefore argued that
since R&D employees (i.e. scientists) are to a high degree used to reference their results and
conclusions, it would be fair to expect that at least half of all ideas coming from R&D should contain
some kind of reference. This would correspond to a reference incidence of 25%, assuming an equal
distribution of idea generation frequency across all business functions involved.
Hypothesis 4: Idea novelty is a key feature of innovation and in the present campaign the screen team
scored all 74 ideas as being novel or not after the campaign was concluded (Appendix 2). The re-
combinatorial nature of ideas, as described by Schumpeter (1939), Nonaka (1994) and others, implies
that one can expect novel ideas to constitute a substantial but still minor percentage of all idea posted.
In the context of an earlier Novozymes crowdsourcing exercise by Lauto et al, (2013) found that only
about 25% of all posted ideas could be considered as novel. Based on this evidence the acceptance
criteria for using the term ‘wise crowd’ is equally being set to 25% for the idea population studied here.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
14 | P a g e
Hypothesis 5: Novelty is also expected to be correlated positively with idea ranking. In the present
dataset two such rankings exist: the idea ranking of the crowd, as expressed by the sum of dot-votes per
idea, and the idea ranking of the screen team, recorded as the summed quantitative scores on six pre-set
screening criteria for each idea (appendix 1). Regression analysis was used to answer the question
whether there was a statistically significant (p< 0.05) correlation between the crowd and screen team
perceptions of idea novelty as expressed.
Hypothesis 6: Co-creation and collaboration are argued to be important – if not critical - conditions of
innovation since different modules of knowledge and insight are recombined into something new
(Keller, 2001). The crowd on ‘New Claims’ was cross-functional in its composition. Participants came
from R&D, Sales, Marketing and Technical Service. It is arbitrarily expected that at least 25% of all
ideas led to discussions between participants from different business functions. The reason why this
threshold was not set higher is the notion that most employees from the implied functions were not
used to have collaborative idea discussions with each other – but were faced with such a ‘challenge’ for
the first time.
Hypothesis 7: Cross-functional discussion is viewed as an indicator of collaboration on a given idea in
the framework of the online process. Different perspectives and information are added to the initial idea
which is expected to increase the innovation potential of the idea. It is therefore hypothesized that such
ideas receive higher scores from crowd and screen team alike. Regression analysis was used to answer
this question and significance level was set to be a p-value of p < 0.05.
Hypothesis 8: In order to accelerate customer centric innovation, crowds need to be responsive to
market needs. In the context of this thesis, market needs are indirectly articulated in the ideas posted by
the various participants. A classification scheme was established to score the ‘intensity’ and ‘clarity’ of
market need articulation as explained in Analysis 5.1.5. Since 50% of all crowd participants came from
customer-facing business functions such as Sales, Technical Service and Marketing it is expected that
at least 25% of all ideas are categorized as containing a market need.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
15 | P a g e
Hypothesis 9: It is hypothesized that a ‘wise crowd’ shows a significant (p<0.05) response to ideas with
market need articulations, as expressed through high dot-vote scores. It is also expected that the screen
team shares this perspective, as expressed through high scores on the pre-set criteria as detailed in
Appendix 1.
1.3 Research relevance
Since the internet has made it possible to connect individuals in larger groups to interact in online
communities, there has been done extensive research in the area of crowdsourcing. In the resent years
special focus has been put on the question on the maximal productive size of crowds, and which role
crowd diversity plays in the innovation performance, (Soukhoroukova, 2012). However, there seems to
be a need for a deeper perspective and analysis of what crowds actually express online and how
constructions of new ideas emerge in such campaigns. The current literature seems elusive on this
point.
Recent work by Poetz and Schreier, (2012) is one of the few articles which c deals with research
related to the present thesis. However, the focus in their study was more on the differences between the
external versus the internal crowd performance. The research is especially narrowed down to a single
company’s internal performance and how the generated ideas are related to the external world. It is also
more focused on the screening process by senior experts versus the crowd’s opinion. The analysis of
the participant’s ideas is relevant both in terms of strategic implications for the company and for the
academic world, since such ‘crowd insights’ or ‘wisdom’ has not been investigated systematically
before.
The relevance of this thesis lies in its findings around how wise an internal crowd actually is and what
unconsciously happens in the aftermath of the screening process. Especially in terms of ‘crowd-
strategizing’ a novel cognitive step process was performed by the screen team which according to
present search efforts has not been described previously in such form.
In the sense of future crowdsourcing exercises in Novozymes, this thesis will be used as a key element
in designing new opportunities with this field. The finding conclusion will also be presented to
Innovation Management for recommendations in driving customer-centric innovation forward.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
16 | P a g e
1.4 Disposition
This figure illustrates the deposition which the thesis is built on and gives the reader a better
perspective of what is done to answer and conducting the research.
Figure 1; Disposition of the thesis
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
17 | P a g e
2. Literature review
This chapter will provide a view of some of the recent and most relevant research literature on
crowdsourcing and idea management in large corporations. The research, done in this field has been
focusing to a large extend on external crowdsourcing (Howe, 2006). In the recent years, quantity has
been the main research object, in terms of maximizing the crowd. However, the latest research
investigates how to measure and manage the ideas coming in from the crowd (Soukhoroukova, 2012).
The object of this part of the thesis is also to identify the how the literature reflects upon theory of
assembling and using a crowd to achieve business objectives. The notion of using internal
crowdsourcing in a strategic context is not something that has been documented that much, in contrast
to a lot of theory regarding large open source and external crowdsourcing (Flynn, et al., 2003).
In order to discuss the research hypotheses, it is necessary to create some clarification on how
crowdsourcing has been presented and used to generate new ideas and solutions for the past decade.
The measurement of a “wise crowd” and “picking the right idea” are two very different things, and
seen as two different ways of executing an ideation. A winning idea can easily be a pure technology
focused solution, however in this thesis, the theory is focused on the way companies built and structure
their crowd in order to use the generated outcome in a strategic matter and identity knowledge and
market gaps.
The provided starting point is to give an understanding for what types of arguments that can be stated
in order to justify online idea campaigns inside companies, and to give a perspective to narrow down
some related and specific research. This enables the thesis to draw on some of the relevant theory to
answer the research questions around the use of internal crowdsourcing in Novozymes.
The following chapter will start with a short explanatory approach of where the need for a collaborative
environment has emerged from in science based companies. Then the chapter goes into the core of the
literature around crowdsourcing and explains the theoretical benefits and barriers and other
implications of crowdsourcing.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
18 | P a g e
2.1 Innovation – in the science-based corporation
In the text “The evolution of Science-based business: innovating on how we innovate” by Gary Pisano
from 2010, it is argued that traditionally, science has always been connected to innovation and thereby
states that it has its home in the R&D part of the organization. However Pisano explains that in the later
20th
century, innovation systems starts to change in the area of the emerging bioscience and
biotechnology industry; especially in the way science and business are connected. A shift emerged in
the way science moves more efficiently from the laboratory to the commercial market (Pisano, 2010).
“…even DuPont, by the 1980s, was asking its research laboratories to focus more on the commercial
needs of the existing businesses (Hounshell and Smith, 1989)…”
Now, this seemed like a good way to get R&D aligned with the rest of the organization and an easy
way to commercialize new innovations fast to the market, but Pisano argues that there is an important
attribute that lies in the iterative nature of R&D, “…time horizons to resolve fundamental uncertainty
can be quite long. Thus, not only might the financial costs of exploration be high, but critical technical
uncertainties may not be easily or quickly resolvable early in the development process.” This is not a
new challenge of the science-based business field, it is more a question of the knowledge base that the
companies now operates in, which now is changing in an extreme pace.
In areas where the underlying science is more mature, knowledge is often modular. That is, with deeper
understanding comes knowledge about fundamental “building blocks” and how those interact (Ibid).
Collaborative software is an example of how R&D can break down problems into module components,
for example in the idea generation phase. In sense there could be different “pieces” but their boundaries
are not clearly defined. How one thing affects the other may not be well understood at all (ibid). Pisano
calls this “the integration problem” and relates it to the argument made more than 50 years ago by
Schumpeter (1939), where he found that breakthrough innovation is the result of recombination and
integration of existing bodies of knowledge (Fleming, 2001). Many empirical studies have confirmed
that Schumpeter was right in his observations (ibid).
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
19 | P a g e
2.2 Crowdsourcing
Crowdsourcing is a recent approach to increase the amount of ideas, solution and turn them into
innovations in companies. The ideology behind crowdsourcing is to use the wisdom of many
individuals instead of relying on only a few experts (Surowiecki, 2004). The literature uses various
terms to describe related phenomena, such as peer production, collaborative systems, community
systems, collective intelligence, crowd wisdom and mass collaboration (Flynn, et al. 2003). Other terms
often used in the literature include consumer co-creation (Simula, et al. 2012), open innovation
(Chesbrough, 2011), user innovation (von Hippel, 2005), collaborative innovation (Soukhoroukova,
2012), customer driven (Schreier, 2011) and used-generated content (Hine and Kapeleris, 2006). In this
thesis the use of the term “crowdsourcing” is set to describe idea and innovation generation.
Crowdsourcing is still on its raising in the corporate world, however the concept of crowdsourcing is
very known in world around, basically it is a way for someone; a person, department, group,
government, company, even countries, to source a question, problem, issues or new ideas and get
feedback from a wide number of people; the crowd. In history we see a lot of different examples.
Boudreau and Lakhani (2011) mention an example from the 15th century where authorities in Florence
presented an open invitation for everyone to participate in designing what would be the world’s widest
and tallest dome for the city’s new cathedral.
One of the largest and most well-known examples of crowdsourcing is Wikipedia.org; this website is
used all over the world and is developed by a global crowd to create free access to knowledge. The size
of knowledge shared on Wikipedia is a great example of how powerful the internet can be in sharing
and generate knowledge (Surowiecki, 2004).
In the literature around crowdsourcing, the notion of “wisdom of the crowd” is very often discussed.
The wisdom of the crowd refers to “the discovery that the aggregate of a set of proposed solutions from
a group of individuals performs better than the majority of individual solutions.” (Yi et al., 2012:452).
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
20 | P a g e
2.2.1 Assembling the “wise crowd”
In crowdsourcing the single most important variable that needs to be in place and function correctly if
of cause the crowd itself. Surowiecki, 2004 suggest a framework whereas it is necessary to break down
the advantages of crowdsourcing into three types of “wisdom of crowds”, which is classified as the
following (Surowiecki, 2004):
Cognition
In order for a crowd to be cognitional, it needs to be highly capable of intellectual thinking and
understand the information processing aspect of the problem proposed. Secondly the crowd needs to
have some understanding of market judgment, which Surowiecki argues can be much quicker and more
reliable, in producing ideas that will succeed in the evaluation process. The crowd is also less subject to
political or managerial powers than the discussions of experts or in knowledge heavy committees
(ibid).
Coordination
In order for a crowd to be classified as a coordinating crowd their behavioral approach needs to be
clearly focusing around optimizing the utilization of the ideas co-development aspects. Surowiecki uses
examples from experimental economics, however it is also takes in the cultural aspect in terms of how
common understanding within a culture allows remarkably accurate judgments about specific reactions
of other members of the culture (ibid)
Cooperation
The cooperative aspect of assembling the “wise crowd” refers to how crowds can form networks and
connections of trust without a central system controlling their behavior or directly enforcing their
compliance.
Furthermore Surowiecki argues that, not every crowd is rational and can create wise decisions, (e.g.
group of investors in a stock market bubble), so he describe these following four criteria you need to
form a wise crowd, in order to create a rational crowd and not end up with an irrational crowd
(Surowiecki, 2004).:
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
21 | P a g e
Table 2; Criteria of a wise crowd
(Source: Surowiecki, 2004)
Surowiecki argues however, that failure of crowd intelligence is still a risk. In some cases the crowd
can come up with bad judgment. He argues that the crowd cognition or cooperation failed because the
participants of the crowd were too conscious of the opinions of others and instead of thinking
differently started to emulate and conform each other’s ideas (ibid). He asserts that when the decision
making environment is not set up to accept the crowd, then the benefits of individual and private
knowledge is lost and not articulated. This can lead to the crowd only being able to be as beneficial as
its smartest member, instead of perform better as a crowd (ibid).
2.2.2 Crowdsourcing in the corporation
The preceding chapter has defined and discovered which factors are important to secure when using
crowdsourcing as a tool. Now it is wanted to give a deeper understanding of crowdsourcing within the
corporation, and look at some of the literature around internal crowdsourcing. In the text of Simula
et.al, 2012, is it explained that the potential of crowdsourcing in relation to creating new ideas and
innovations in a business-to-business environment is starting to get quite popular, because
crowdsourcing is lowering the cost and shortening the product development cycles. One of the key
value propositions of crowdsourcing is that it enables companies to collect ideas from large groups and
manage review them, instead of using time and money on sourcing from few experts (Simula et.al,
2012) .
In order to get an understanding of how crowdsourcing can be handled by companies I will use the
framework provided by Simula et.al, 2012. This framework consists of four layers of participants,
which a company can use when initiating an exercise for crowdsourcing. The framework builds on the
traditional stakeholder theory (Freeman et. al. 2004). It is argued that this model can be applied to a
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
22 | P a g e
number of industries, but in this thesis the research will mainly be used in the context of internal
crowds.
Figure 2; the layers of crowds
(Scource: Simula et. al., 2012).
In the center of the figure (3) are the employees of the company, this is also were most companies
begin when using crowdsourcing in the business-to-business sector, however after some practices,
proof of concept and success cases, the company can start to move into the external layers, consisted of
trusted partner (also known as “value-chain partners”) by Freeman et al., 2004. The next layer consists
of a specific crowd, including people with certain skills, knowledge and expertise or other pre-
qualifications; it can also refer to a community of like-minded individuals (Simula et. al., 2012).
The third and last layer is the general crowd which can consist of everyone, who has an interest in the
scope of the campaign, even competitors can participate here.
However in this case study it is necessary to return back to the first layer, and focus on the internal
crowd, within the boundaries of the company. The other layers are outside the scope of this thesis.
According to (Simula et. al., 2012) Companies usually target internal idea ideation at all employees, in
order to increase serendipity, which is seen as a good thing. Howe, 2008, argues that the best solutions
often emerge from crowds that are the least likely to come up with the right way of solving the raised
problem. Internal crowdsourcing has been practiced by many companies. One example is the
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
23 | P a g e
pharmaceutical giant Eli Lilly, who have demonstrated the value of harnessing the knowledge of
employees to solve problem and idea generate throughout the organization. However Boudreau and
Lakhani 2011, points out that company culture can be a major issue in the pursuit of getting a crowd to
be innovative thinking, they state that one of the most important motivators is to provide the crowd
with the sense that their ideas are valued by management. When designing an online ideation Simula et.
al. 2012 argues that it needs to be done in such a way that they can be unambiguously understood by
the crowd. In order to engage people in crowdsourcing and to make them contribute, people need to
understand the context of the idea and have the right frame of mind (ibid). They argue that feedback
also plays a major role in the perception of how the crowd will react to the requested task; “There is
also the risk that people may want to develop their ideas by themselves and are not ready to share or
that they think their idea is not good enough to post. Another challenge related to engaging users is the
importance of feedback: not receiving feedback may have a negative impact on future participation of
the contributor. Similarly, the informant in Gamma pointed out that if people think that the internal
idea service is meant for R&D people only, there may be less interest in participating.”(Ibid).
Simula et. al. 2012 gathers their findings in a table of advantages and benefits and some of
consequences and barriers in using internal crowds in crowdsourcing. These thoughts of internal
crowdsourcing will be taken into consideration in the discussion part of this thesis.
Table 2; Pros and Cons of Crowdsourcing
(Source: Simula et. al. 2012.)
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
24 | P a g e
To get a bit deeper into the research related more specific to my study of internal crowdsourcing, I will
draw on the recent published article, “The Value of Crowdsourcing: Can Users Really Compete with
Professionals in Generation New Product Ideas?” By Poetz and Schreier (2012). They make an
analysis of a case company that crowd source ideas for new products, and ask a crowd of 100+
participants. 52 were internal employees and the remaining 51 were customers. The participants where
posting ideas through the company website. The ideas were screened and evaluated by the company’s
CEO and the Head of R&D, blinded of whether the source of the ideas originally came from inside the
company or outside. The study is similar to my research in terms of measuring the ideas and where the
ideas come from. Poetz and Schreier set the following research question; how attractive are new
product ideas generated by users through a crowdsourcing process compared with new product ideas
generated by a firm’s professionals? In this study it is stated that the professional engineers out
performed in the creation of feasible ideas, in contrast to the ideas generated outside R&D, in the case
the, which the authors conclude to be an indication that the professional crowd is much more focused
on deliver ideas that is easier to realize in technical terms, whereas the ideas from costumers are
focused on the commercial benefits for the end-user, but lacks of realistic deliverables and timeframes
for actual execution to the market. Whereas the authors draw on the theory from Chesbrough (2011),
that they need to reinvent the formulated question to a specific problem-solver, crowd or group.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
25 | P a g e
2.3 Collaboration modes
In the article by Pisano and Verganti, 2009, called “Which kind of collaboration is the right on for
you?” the authors go through different types of crowdsourcing strategies. They are districting between
open and closed networks, and between a hierarchical and a flat governance approach. This very simple
framework is effective in order to get an overview of where the strategic decisions should be made
when initiating a crowdsourcing exercise. The text especially discusses which types of crowds
companies should invite and focusing on who will carry the “Innovation burden”, meaning screening
and filtering ideas etc.
They argue that the cost of searching and screening ideas increases when a company engages with
larger crowds than the internal capacity can carry. However the advantages of having an open network
could result in attracting a large number of problem solvers and consequently a huge number of ideas
generated. The text argues that in this scenario the company doesn’t have to identify either the best
knowledge domain or the most appropriate experts in those domains (Ibid). The company doesn’t even
have to know the contributor, but the text argues that it can be dangerous as well, if the knowledge
domain is a sensitive business area, where you don’t want to have competitors or others who can use
the information provided to other winnings. The authors note that often interesting innovative solutions
can emerge from contributors the company never has imagined could come up with good ideas.
Open modes, however, have their disadvantages. An interesting notion is that, they are not as effective
as closed approaches in identifying and attracting the best participants. Pisano and Verganti argue this
is because as the number of participants increases, the likelihood that a participant’s solution will be
selected (especially for an ambiguous problem) decreases (ibid).
“…Open modes are effective only under certain conditions. First, it must be possible to evaluate
proposed solutions at a low cost. Sometimes the screening process is extremely cheap and fast…”
“…In other cases, though, the only way to find out whether an idea is worth pursuing is through
expensive and time-consuming experiments, and you’ll want to consider fewer (but better) ideas. The
only way to do that is to invite contributions from the problem solvers that you think will have the best
chance of providing good ideas. That is, to opt for a closed mode…” (ibid)
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
26 | P a g e
The figure (4) shows the four basic modes of collaboration: a closed and hierarchical network (an elite
circle), an open and hierarchical network (an innovation mall), an open and flat network (an innovation
community), and a closed and flat network (a consortium).
(Source: Pisano and Verganti, 2009)
Figure 3; Modes of Collaboration
Elite circle: In this mode one company selects the crowd, the screening process is also made by the
company and they define the problem and choose the solution. Pisano and Verganti argue that this
mode is appropriate when the company know the knowledge domain and can determine from where the
best solution to the problem are likely to emerge from. In this mode the chosen experts play a major
role and the company needs to have the capabilities to pick them internally. The owners of the ideation
do the evaluation and screening process and only they evaluate the proposed ideas.
Innovation Mall: In this scenario there is still one company that posts a problem, but here anyone can
propose the solution, and the company chooses the best idea. This mode of collaboration enables the
company to get ideas from many parties, and the best ideas can come from unexpected sources. Here
Pisano and Verganti suggest that the consequences of missing out on a good idea, as in the elite circle,
are limited in this scenario due to the self-selection of participant. Thus if the problem is small or
narrow it can be broken down into smaller sessions, to resolve it faster. In this mode the evaluation
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
27 | P a g e
process is spread over the different parties involved and thereby enables the company to screen the
ideas fairly cheaply.
Innovation Community: In this mode anybody can propose problems, suggest solution and decide
which solutions are best and put into use. This is the most open approach, this mode is properly most
appropriate when the company needs ideas from many different parties. The company can’t own the
intellectual property underlying the solutions, and needs to take this into consideration these
collaboration modes are therefor often seen in open source software projects e.g. Linux or Apache.
Consortium: The authors suggest that this collaboration mode is like a private club, where participants
jointly select the problem and co-create the chosen solution and decide together how to conduct the
further assessment of the winning ideas. This mode is appropriate when the company, like in the Elite
circle, knows the knowledge domain and knows where the best solutions are most likely to emerge.
Once again the importance of having the right experts is a key factor in this mode and the company
needs to have the capabilities to pick them; however the experts need to have “share power” over the
decision making in the selection of winning ideas. Pisano and Verganti also suggest that all the
expertise of all participants is needed in this mode in order to harness the true innovation. In this mode
there can be different ways of sharing the intellectual property, e.g. a co-owned patent or royalties to
the owners of the winning ideas.
Choosing a collaboration mode involves more than understanding the trade-offs. A company must take
into account its strategy for building and capturing value. And as the strategy evolves, the right mode
of collaboration might change, too. (ibid)
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
28 | P a g e
3. Company and case description
In this chapter I will give a short presentation of the case company; Novozymes and provide an
overview of how the company innovate, more specific how Novozymes handle ideas and how the
Innovation Department manage it. In the last part of this chapter I will provide a deeper understanding
of the chosen idea campaign; “New Claims for Detergent”. That later will be analyzed.
3.1 Description of Novozymes
Novozymes is the world leader in Bioinnovation and Biosolutions. Novozymes' core business is
industrial enzymes, microorganisms, and biopharmaceutical ingredients. Their goal is to help their
customers (such as P&G, Unilever, Nestle, PepsiCo) to achieve more efficient product and process
solutions to save energy, raw materials, and reduce waste. The pursued result is higher quality, lower
costs, and a better environment. The biological solutions are used in the production of numerous
products such as biofuels, detergents, food, and animal feed. (NZ1)
Novozymes has over 6000 employees globally, working in Research and Development (R&D),
Production, Sales, Marketing, Technical Service and general administration. The company has a
portfolio of over 700 products, used in 130 countries. The company is quoted on NASDAQ OMX
Copenhagen A/S (NZ2).
The company is performing well financially. Over the past 10 Years, during which the company has
been operating separately from its sister company Novo Nordisk, Novozymes has achieved an annual
sales CAGR of 8% (NZ1). Due to the rapid development within their fields of technology and the
company’s ability to innovate, as well as strong secular trends such as sustainability, chemical
replacement, and energy security, Novozymes today aims to increase their business by more than 10%
a year. On top of this lies the opportunity within cellulosic biofuels. Turning agricultural waste into
sugars for the production of biofuels and other chemicals is a very interesting opportunity for
Novozymes. However, due to the uncertainties associated with the timing and scope of this
opportunity, it is not yet included in their long-term sales growth ambition (The Novozymes Report
2010). In 2012 Novozymes achieved a turnover of 11 billion DKK, and had a net profit of 1,6 billion
DKK. (NZ1)
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
29 | P a g e
Business areas
Novozymes’ business consists of two segments: Enzyme Business and BioBusiness. Enzyme Business
is engaged in development, production and distribution of enzymes, this currently accounts for more
than 90% of sales, while BioBusiness generates the remaining 10%.
Detergent enzymes
These enzymes are used in laundry and dishwashing detergents. In the process of washing clothes,
certain enzymes break down water-insoluble stains into water-soluble molecules that can be rinsed
away by the wash water.
Technical enzymes
Technical enzymes are used, among other things, in the transformation of starch into different kinds of
sugars. This functionality is used in the starch and fuel industries. By 2014 the company expects that
the enzymes will make it possible to produce advanced biofuel from certain agricultural residues in
large-scale production. Technical enzymes are also used for many other applications, for example
leather and textile treatment and forest product industries.
Food enzymes
Enzymes enhance quality or production efficiency in the production of food products such as bread,
wine, juice, beer, noodles, alcohol, and pasta.
Figure 4; Novozymes organization
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
30 | P a g e
Feed enzymes
Adding enzymes to animal feed increases the nutritional value of the feed and improves phosphorus
absorption in the animals. This leads to faster growth of the animals and improves the environment as
less phosphorus is released via manure. (NZ2)
Microorganisms
Novozymes’ beneficial microorganisms are used in industrial and municipal wastewater treatment, as
well as in the cleaning of surfaces such as carpet, concrete, drain lines, and septic tanks in industrial
and household applications. Beneficial microorganisms are also at work in aquaculture and agricultural
applications.
Biopharmaceutical ingredients
Biopharmaceutical ingredients are proteins and other biological substances used in the pharmaceutical
industry. The proteins replace proteins from humans and animals that have traditionally been used and
have posed the risk of transferring disease. The industrial proteins do not pose this risk and offer further
advantages such as cost savings, process performance, consistency, and compliance. (NZ2)
3.2 Innovation in Novozymes
Innovation plays an important role for Novozymes as a business. In the industry of biotechnology, it is
vital to stay ahead of competition all the time through new innovations, and to protect technologies and
business areas’ extensive use of patents (NZ1). Novozymes currently holds over 6000 patents, and is
filing about 150 new patents per year (NZ1). In comparison, Novozymes’ largest competitor, the
Danish company DANISCO (now acquired by DuPont) holds about 2500 patents and is filing about
50-70 per year. This is why Novozymes can call themselves the world leader of bio-innovation.
Novozymes puts a lot of focus on R&D. In 2010, 14 % of turnover was invested in their research and
development department, which is located at 8 different sites around the world and employs 800
researchers. Within this department a great amount of attention is directed at promoting collaboration
across geographical boarders, and the sharing of knowledge through both face-to-face contact and
databases (NZ1). Innovation by Novozymes typically takes place in close collaboration with their
customers. Partnerships play an important role in Novozymes’ business model. While developing a
new biological solution, Novozymes works closely with their partner in order to optimize their
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
31 | P a g e
technology’s functionality for application in the customers’ products or processes (NZ1). When it
comes to commercialization, Novozymes role is generally to deliver the technology, while their
partners take it to the market. Because of this, Novozymes is extremely dependent on strong
partnerships (NZ1)
3.3 Innovation processes in Novozymes Face-to-face ideation is a common and fairly used tool within the company to brainstorm on new
concepts and to come up with new ideas on specific topic. These workshops a usually administered and
facilitated by the Innovation Office and has been focusing a lot on technical solution and are mainly
exercised in the R&D community. Since R&D are located around the world these face-to-face ideation
are heavily costly to arrange and typically scientist are flown in from different destinations, which
means the budget goes mostly to travelling expenses.
Another tool idea generation tool is the Idea Web (an idea suggestion box on the companies Intranet),
which is also administered by the Innovation Office. This idea box is open to all members of R&D and
Business Development. The Idea Web is open 24/7 all year around. The Innovation Office is
responsible for screening and forwards the posted ideas to an expert within the field. If the idea is good
the idea will be taking into further assessment in R&D. However the screening process is not optimal
and can often lead to very little feedback to the idea submitter. On the positive side it can be argued
that the Idea Web is good place to showcase your idea to senior experts, especially if the submitter is
new in Novozymes or perhaps located far away from Head Quarters in Denmark.
For maturing ideas and concepts, is the initiative called RIC (Radical Innovation Catalyst) which goal
is to mature and initiate good ideas coming out i.e. Ideations and turn them into projects. The RIC
community is built on volunteer allocations, to make sure that employees are choosing the projects they
really believe in. The Innovation Office is responsible to gather the ideas and run a website where
employees can read about the new projects and decide if they would like to get involved.
With a need for more and faster acceleration of innovation processes and a more customer-centric
approach, demanded from Senior Management, the Online Ideation tool is set out to be the right
process to fulfill the wanted goal of new projects delivered faster to customer. The Innovation Office is
thereby on a journey to test out internal crowdsourcing as the right method to achieve this goal. The
R&D Management initiated this strategy in late 2011, and has now slow but steady grown into a
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
32 | P a g e
concept, that today is a well-known tool for idea management in all of Novozymes. Since 2011
Novozymes has performed 17 online ideation campaigns, most of them with a technology and product
innovation focus, thus drawing heavily on R&D crowds. The parameters of success for internal
crowdsourcing were analyzed and described by Lauto et al (2013). Diversity, absorptive capacity and a
culture permissive to innovation were the most important ones.
3.4 Case: New Claims for Detergent Enzymes
The overall goal of the online Ideation; “New Claims for Detergent“ was to identify new ideas that
could boost the growing area and core business of enzymes for detergents and solutions that could lead
to new claims for Novozymes’ customers. In the term of “new claims”, it is meant that customers can
use the provided technology as sustainable “green” claim of e.g. washing powder without harsh
chemicals or claiming ‘whiter than white, or ‘total stain removal at 45C’ and similar Some of the
largest detergent manufactures and marketers are customers at Novozymes (e.g. Procter & Gamble and
Unilever) today. In order to keep sales growing to these customers, Novozymes needed to come up
with more ideas on creating enzymatic solutions that they can be used to sell consumer products and
use “green” claims. By applying an internal online idea generation tool, it became possible to crowd-
source ideas from around the organization, in that way the tool combined the classical ideation process
with an online community, with selected employees from around the world. The online platform was
provided by an external consultancy firm NOSCO2, who has been offering the online platform to
Novozymes for a little over a year at this point. The collaborative online ideation process was designed
together with the Innovation Office at Novozymes and consultants from NOSCO.
2 Website: nos.co
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
33 | P a g e
3.4.1 The process of “New Claims for Detergents”
The process was divided into different phases to make sure that the ideas would make it through the
whole campaign and at the same time to mobilize the crowd in order to get the most out of their
creativity.
Figure 5; Process of online ideation
3.4.2 Scope and plan
The online ideation aimed at identifying problematic stains and possible solutions with focus on
Americas and EMEA markets. The division called Household Care, who is responsible for developing
the detergent business within Novozymes, was the main sponsor of the campaign. Household Care had
just developed a new strategy called “Triple 20”, with the goal of triple the turnover within Household
Care by 2020, one of the way of succeeding this was to identify, develop and launch 5 new laundry
claim enzymes similar to the very successful Mannaway3, which sells for close to 250 million DKK a
year. Basically the challenge was formulated to find “the next big thing” within the detergent business
area. This meant also that this online ideation had a very high priority in Novozymes and participants
were allowed to spent time on the online platform.
3 A big blockbuster Novozymes product within detergent
- Problem statement - Source / Solve? - Admin Team
- Process template, - Communication plan - Timeline - Screen team - Crowd
Execute communication plan: - Purpose - Expectations - Process - Timing - Incentives - Feedback - Support
Posting & discussing by crowd
Screen team 5 ideas /solutions
1 week 1 week 1 week 2 weeks 2 days
NL?
1-3 months
Assessment of winning ideas
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
34 | P a g e
3.4.3 Mobilization and composition the crowd
Assembling and construction of the crowd was steered by the Innovation Office in close collaboration
with the Household Care group, and started to handpick the different scientist who was specialized
within enzymes for detergent, and the unusual scientist how could possibly come with a creative angle
on ideas. The campaign was also the first cross-functional online ideation done in Novozymes. 50% of
the crowd was origin in R&D and the other 50% was coming from departments of Marketing, Sales
and Technical Service.
This was done in order to get a more diverse and a more customer oriented approach to the ideas, and
not only conceive ideas heavily focused on technological propositions.
3.4.4 The online ideation phase
The online phase was planned to run for 2 weeks and the 105 invited participants was able to submit an
unlimited amount of ideas and comments. The online ideation was kicked off with a conference call
where the participants were briefed by Nosco’s consultant and the scope was clarified and questions
from the crowd would be answered.
The participants were given profile with name and picture on the platform delivered from Nosco, the
profile was typical profile as seen on social media. The platforms had a few features, where participants
could follow, share, vote, and comment on ideas. The platform was designed to give a simple overview
of the idea flow and showcase which ideas were most popular at the given moment.
When posting an idea the crowd was kindly asked to write the idea with a short description and attach
documents and other relevant items. The participants was presented by a short summary of what the
screen team was looking for in an “good” idea and what types of criteria an idea needed to contain and
address in order to succeed in the evaluation process. These were the criteria set up by the screen team;
A stain which is problematic for many consumers
Future stains based on market trends within food compositions
An enzyme which there is reason to believe will be able to remove the problematic stain(s)
Completely new idea for an enzyme
An enzyme solution with high technical probability
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
35 | P a g e
In order to get the crowd motivated and get started on posting good ideas, the Innovation office and
screen team announced that they would nominate the most novel idea and the author(s) would then be
awarded with prizes. The winner would also be appraised as an innovator within Novozymes. In order
get ideas flowing fast, was the most active person also awarded, the person with the best arguments for
choice of a given stain (to develop an enzyme to remove) and the best team/department (most active
and best argues) would also be awarded. This should generate internal competition and help the idea
flow, since it is critical to get the first few ideas posted on the platform.
3.4.5 Screening and selection of ideas
The phase after the idea generation had ended the phase of screening and selection of the winning ideas
started. The online ideation had generated 74 ideas, and had over 200 comments from the 105
participants. Furthermore the crowd had been voting on the ideas, they believe the most in. The screen
team was now set out to identify the ideas that had the most potential and which ones should go into
further assessment in R&D.
The screen team was a diverse group with senior managers and directors from R&D, Marketing and
Business Development.
Title Department Tenure with Novozymes Tenure with NZ Detergents
Senior Director Innovation Office 15 years none
Senior Manager Detergent Marketing 12 years 5
Senior Manager Business Development 7 years 7
Dept. Manager Detergent R&D 10 years 10
Senior Manger Technical Service 8 years none
The point is to show the diversity of the screen team, its high level of knowledge as expressed by
tenure but also to illustrate that none of these people had direct decision or resource power in
detergents as such their findings to Management in Detergents were on a pure recommendation for
discussion and decision basis.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
36 | P a g e
The screening process was conducted over 2 days where the team reviewed all of the ideas and gave
them ratings in the different criteria’s chosen for this campaign. Each of the screen team members need
to score the idea within 6 overall criteria from a score of 1-5, where 5 was the highest score. Following
meetings was held where the screen team would present their scores and discuss which ideas they liked
the most.
The first three criteria’s was related to the commercial/market needs in the idea, and the last three
criteria were related to technical probability. The screen team did extended background check on the
idea both with internal and external databases, and could only score the ideas high if there were found
written evidence that the idea could be realized. (For a closer look at the specific criteria please see
Appendix 1)
Once the screen team had previewed all of the ideas and scored them, the top 5 ideas was picked and
presented to the crowd as the winning ideas. The ideas would then go into further assessment and
matured for development in R&D.
The description above of “New claims for detergent” has provided an overview of the process and
designs the idea campaign. The following chapter will present the methodological approach to the
research in this thesis.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
37 | P a g e
4. Methodology
In the pursuit to answer the research questions that have been proposed in this thesis, the research is set
in the perspective of social constructivism in order to address the exploratory and explanatory findings
and thereby put it in an abductive 4 design to seek evidence and conclusions.
Since this thesis is a single case study of an online ideation within Novozymes, the investigation and
research have been focusing around a number of data sources, mainly quantitative data have been
collected and analyzed, but also qualitative data have been collected to reach a point of in-depth and
understanding in the process around the investigated online ideation.
In the following section, it will be explained how the different data sources have been collected and
made it possible to create a foundation for the research approach and develop new knowledge
regarding internal crowdsourcing as a tool for creating new ideas.
4.1 Research approach and design
In line with the structure described above the method applied in this case study will approach the
conduction of the case study method proposed by Yin (1994).
According to Yin (1994) a “case study is an empirical inquiry that
- Investigates a contemporary phenomenon within its real-life context, especially when
- The boundaries between phenomenon and context are not clearly evident” (Yin, 1994; 13).
Further the “case study inquiry:
- Copes with the technically distinctive situation in which there will be many more variables of
interest than data points, and as one result relies on multiple sources of evidence, with data
needing to converge in a triangulating fashion, and as another result.
4 Abduction is a form of logical inference that goes from observation to a hypothesis that accounts for the reliable data
(observation) and seeks to explain relevant evidence.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
38 | P a g e
- Benefits from the prior development of theoretical propositions to guide data collection and
analysis.” (ibid.).
The analysis includes internal sources from Novozymes, in order to come up with a conclusion
followed by managerial recommendations. The case study approach fulfills the characteristics above
and the empirical data does as well since it is chosen to base the analysis on both qualitative and
quantitative data, which is recognized as evidence in this approach (Yin, 1994).
There are certain areas where the case study deviates from the method proposed by Yin (1994). The
data is conducting on an embedded single case study due to the fact that it is analyzed as a subunit that
contributes to Novozymes overall innovation strategy; e.g. the industry trends, competitors, core
competencies and different resources has not been analyzed due to the scope of the thesis.
Analyzing an embedded unit as an online ideation require that the analysis of the embedded unit
contribute to the “major interest of the study”, which the analysis does by going in-depth with internal
factors to get a broader picture of how Novozymes internal resources could be used in a future
approach of crowdsourcing exercises (Yin, 1994; 120).
4.1.1 Data collection
The gathering of information and data has been extensive. As explained above, the exploratory
approach of the research has led to a process where the data has been collected from different sources,
and continually been revisited and made it possible to get new insight. Since the online ideation was
held in October 2012, and the data for this thesis have been collected in April/May 2013, it has been
important to read through all the ideas and do semi-constructed interviews with members of the screen
team and some face-to-face meetings, in order to get a understanding of what the organizational
conditions was at the time for the campaign and to understand how the screen team evaluated and
researched the ideas submitted.
This has enabled the study to retrieve both qualitative and quantitative date, which has led to an in-
depth understanding of the research inquiry. The method for collecting the data, has been through
exported excel spreadsheets from the Nosco platform, secondly the screen team evaluation documents
has been collected and analyzed, questions and clarifying of the documents has been asked to the
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
39 | P a g e
screen team, this has been done through semi-constructed interviews. The last data source has been the
employee database at Novozymes, in order to get the job titles and departments of the participants of
the online ideation and thereby classify and label them to the ideas they were involved in.
The analysis in this thesis are mainly focused around the quantitative data collection; although it is held
that without the qualitative data collection, the development of hypotheses and the subsequent
development of discussions and conclusions could not have been completed. The specific data
collections – qualitative and quantitative – will be presented in the following sections.
4.1.2 Qualitative data
As presented in the Chapter 3 – Company and Case description the qualitative data collection is
primarily conducted in explanatory manner to get an understanding of what happened in the idea
campaign. This made it possible to get a deeper understanding of what the process ideation was and to
figure out the reflections behind the “New claims for detergents” was, here were a number of
qualitative collections applied. This included observations of the full idea campaign process,
thoroughly read-through of all the ideas and exploratory interviews with some of the screen team
members. A transcript of the specific arguments of the ideas from the screen team, has also been
collected, to identify what kind of background research the screen team had conducted when scoring
the ideas from there different parameters.
The deliberations and assumptions in this thesis were built on a method of triangulation drawing on (i)
theory, (ii) qualitative and (iii) quantitative data. “Triangulation refers to the use of different data
collection techniques within one study in Order to ensure that the data are telling you what you think
they are telling you.” (Saunders et al, 2007:139). The qualitative data collections are therefore
important in the methodological considerations although the quantitative data are the dominant in this
thesis’ analysis.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
40 | P a g e
4.1.3 Quantitative data
The quantitative data is the dominating part of the analysis in this thesis. The data has been collected
from multiple sources. The exported data from the idea platform, delivered by Nosco has mainly been
the primarily source of data. The export options in Nosco’ platform was either a PDF file or an Excel
spreadsheet, the PDF file is a full extracts of all the ideas posted in the campaign, whereas the Excel
Spreadsheet contained the following information; (i) ideas ID; (ii) number of comments; (iii) number of
crowd votes and thereby the crowd score; and (iv) submitters name. However, these data points doesn’t
completely fulfill the wanted data to answer the developed hypotheses, it was important to create even
more data point in order to provide the regression models to identify if there were any correlations and
statistical evidence. The following variables were thereby developed; (v) functional involvement in the
idea; (vi) idea depth; (vii) comments depth; (viii) market needs articulated in idea; (ix) cross-functional
involvement.
The functional involvement in an idea (v) includes the functions or department that commented or
submitted a specific idea, divided into Sales, Marketing, Technical Service and R&D. The idea depth
(vii) and comments depth (viii) was measured from calculating the characters written in the ideas, in
order to see if the ideas with many words were rated higher than the ones with short descriptions. The
market needs articulation was measured out from a reading through the ideas, this variable was score
from a scale of 1-3, 3 being the highest in terms of market needs articulation, this meant that the ideas
was classified to see if the crowd was responsive to the market. The cross-functional (ix) variable was
measured out from a binomial setting, if more than one department was involved in the idea, the idea
was scored with the number of 1, and if only one department was involved it was scored with a 0. .
The second source of data was collected from the screen team, who had ranked all the 74 ideas after the
online ideation had ended. It was thereby possible to use the screen team spreadsheets to develop
further variables; (STi5) screen team total idea score; (STii) novelty of idea; (STiii) commercial
probability; and (STiv) technical probability. All of these multiple collections of quantitative data
enabled the analyses in this thesis. The statistical measurements of data involve calculations of means
5 ST= Screen Team
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
41 | P a g e
and standard deviations. Furthermore, it will be presented as contingency tables of various variables to
test for statistical significance using simple regression models tested in the software program called
STATA6. The dependent and independent variables will examined in the analysis, and will be
explained in the following sections.
4.1.3.1 Dependent variables
The regression models provided in the analysis contains two different dependent variables; the
regression models have been calculated with the same independent variables in order to see if there was
any significant evidence and relations between the two dependent variables.
The first dependent variable is the final screen team score; this variable is set as the dependent variable
to identify how the screen team interpreted the ideas. The - screen team score is therefor set up to be
the indicator for how the ideas performed throughout the online ideation.
The second dependent variable was the - crowd score, this variable is set up to be an indicator for how
the crowd interpreted the ideas and makes is possible to identify the “wisdom of the crowd”. The crowd
rated the ideas from 1-5 stars, this shows clearly which ideas the crowd believed the most in.
The indicator in the statistical analyses is therefore the screen team score and the crowd score. The
mission of the indicator, as described by De Solla Price (1978), is to find the simplest pattern in the
data at hand, and then look for more complex patterns, which modify the first. The more complex
patterns in the dataset are thus the correlation between the crowd and the screen team, and will enable
to identify if the “wisdom of the crowd” is present in this ideation. The independent variables will be
explained in the following section.
6 http://www.stata.com/
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
42 | P a g e
4.1.3.2 Independent variables
In the regression models provided, the independent variables are based on the different department’s
performance in the online ideation and on which type of ideas posted, in terms of novelty and market
articulation. In order to answer the developed hypotheses and to investigate if there are any correlations
between the crowd choices and the screen team choices on the quality of ideas. The chosen
independent variables were: (i) New idea to Novozymes; (ii) market articulation; (iii) cross-functional
involvement; and (iv) origin of ideas (Divided of the departments, Marketing, Sales, Technical Service
and R&D). (i) The New idea to Novozymes variable refers to how new ideas posted in the ideation are
correlated to the depended variables, this variable is discovered from analyzing the ideas and is put into
a binary score of 1 or 0, when it was ranked as a new ideas. (ii) The market articulation variable refers
to how strong the market or customer needs are articulated in the ideas. This variable is used to get a
perspective of how the market is interpreted in the online ideation, and will be used as a parameter to
figure out how the clear the market is represented. The variable is created by an read through of the all
the ideas, and is score every time a sign of market is articulated inside the idea or in the discussion
affiliated to the idea. (iii) The cross-functional variable refers to the combinations of participant
involved in the ideas. This is measured by looking at how and from which departments the ideas and its
comments are represented from. The variable is also made as a binary, whereas 1 if there are more than
one department involved in the idea, and 0 if it is only one department posting and discussing the idea.
This variable is used to give a perspective on how collaborative the crowd was, and how the different
departments interacted in developing new concepts and to see if they could understand their different
professional languages. (iv) The origin of ideas variable is created to track where the ideas are coming
from, this variable is developed by tracking from which department the idea submitter is, and thereby
figure out from where the ideas origin from. The four departments are divided into there each own
variable to be able to run regression model on them. This variable is used to figure out how the Crowd
and Screen Team looks at ideas coming from the different eras of Novozymes, and to test if the
strength of quality in the departments in there submission of ideas in the campaign.
To test the hypotheses these variables are put in to the analyses to gain the insights on how the different
conditions had any relevance in developing, evaluation and choosing the winning top-5 ideas.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
43 | P a g e
4.2 Two-fold analysis of crowd versus screen team
Since the analysis is created by qualitative and quantitative data, the following analysis will be two-
folded. By two-folded it is meant that the data received from the screen team is generated in a
descripted way and gives a clear insight on how the ideas were evaluated together with the semi-
constructed interviews. The screen team, spreadsheet has also provided a deep insight to what the 5
members were focusing on, and gives a clear perspective of what the more in-depth knowledge has
provided in the final evaluation.
The quantitative data has been retrieved and modeled in a way that gives insight on how the crowd and
the screen team acted purely based on data during the ideation. The regression models provided in the
analysis are thereby made only to give a perspective of how the different ideas were conceived and
how they were generated and evaluated.
The analysis will therefore be based on the different types of information, and in order to answer the
research question, the final conclusion will be based on some of the assumptions made on top of the
findings from the two-fold investigation of data.
The way the analysis is created is based on the relevant findings from the data and later put into
discussion to give a more accurate answer on how Novozymes can use crowdsourcing in their pursuit
of customer-centric innovation.
The two-folded approach, enables the thesis to get a more realistic outcome, and makes it possible to
generate some more executable managerial recommendations.
However, a single case thesis will also have its limitations; the following section will explain the
different limitations will come into play, and argue why this thesis not can be used as a general
objective case study.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
44 | P a g e
4.3 Limitations
Conducting a single case study requires that it is a critical case, extreme or unique case, or a revelatory
case in order to be able to generalize from (Yin, 1994; 39). The case study does not test directly any
specific theory, nor is it an extreme or unique case and the type of case has not been inaccessible to
prior studies. This means that the analysis cannot be used for any generalization or base precedence for
other biotech companies with the same challenges. The analysis, conclusion and recommendations are
aimed at Novozymes’ specific strategy and the output of this case study is only valid when used by
Novozymes. However in light of the strategic managerial recommendations it is in some sense,
possible to apply to other similar companies if the internal setup is ready and mature for online idea
generation. The single case study has it limitation especially when it comes to objectivity and
generalization conclusions.
A huge limitation, when investigating an internal crowdsourcing exercise, focused on the market needs,
is of cause that there have been no external parties involved at all. It would have been a lot easier to
answer the research question, if a customer or a partner had been involved in the online ideation.
The empirical data consists of both qualitative and quantitative data however the dominant use of
quantitative data in the analyses has its implications. The quantitative data is used to simplify the
answer to the research question. The Analysis is arguably limited by the use of quantitative data in that
it fails to capture the complex nature of the situation. The case for quantitative data analyses is,
however, that it reduces the interpretative and subjective elements in the research inquiry. Through the
use of quantitative data, it is possible to support the found assumptions by testing of hypotheses to a
greater extent than with qualitative data. The information available in the data set has inevitably guided
the research. That is to say, the limitations of data have resulted in the chosen variables; other variables
could have been examined had different data been available. The four chosen independent variables,
however, are consistent with the suggestions from theory presented in the Literature Review and it is
therefore believed that they are important diversity dimensions in relation to the scope of this thesis.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
45 | P a g e
5. Analysis
In the analysis the findings will be presented from the investigation of the “New Claims for Detergent”
online ideation data set. This will be presented in an analysis that is formed into two main parts. The
first part of the analysis will provide an overview of the distribution of ideas and how the participants
are interacting and to what extent they are sharing their insight, regarding their occupation in the
organization. The analysis will then look more closely into the articulation of the ideas, and with the
use of regression analysis try to explore the patterns of the market needs articulated in the ideas
generated. This will lead me to test my hypotheses that are provided in the section Problem statement.
The empirical findings investigated in this analysis will be discussed in the next chapter and will enable
me to answer my research question.
In the first part of the analysis it will be examined how the outcome and distribution of the ideas to
show the patterns of what the crowd produced in the idea campaign. Secondly, the screen team data set
will be examined, to show which criteria and what background the ideas was rated on. By analyzing
the output from the campaign, it will be possible to draw some assumptions on what was going on in
the online ideation, during and after. The data set retrieved from the screen team, and their comments
are used in descripted manner to help get a perspective on how the evaluation process was conducted.
This will lead to the second part of the analysis.
In the second part of the analysis, the data will be put into relation and be examined to find the
implications of market needs articulated in the ideas, and how this relates to the perception of the
crowd and the screen team. There will be provided a regression model of the data analyzed and
afterwards used to illustrate the results to answer the hypotheses on the possibility that internal ideation
can enhance the acceleration of innovation processes in Novozymes, and which role the crowd and the
screen team are taking on. In order to provide a deeper analysis of the implications of an internal online
idea generation, with a cross-functional crowd and a diverse screen team the following discussion will
put the findings in a broader strategic perspective. The following chapter will enable a thorough
discussion of the empirical findings and results, and put these into relation to the theoretical research
provided in the Literature review.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
46 | P a g e
5.1 Analysis part 1: Descriptive analysis and distribution of ideas
In this part of the analysis the quantitative outcome will investigated of the idea campaign “New claims
for detergents”. This will be done in a descriptive matter to get a deeper understanding of the
distribution of ideas, comments, and how the crowd interacts with one another. The provided
descriptive statistics and the given examples of conversation between the different departments will be
used in order to describe the way e.g. R&D communicates with Sales. Secondly the screen team data
set will be provided and analyzed to identify the different parameters of evaluation criteria. The
presented outcome of the idea campaign is the full data from the crowd consisting of 105 participants;
this is created to give an overview of how the ideas were generated. The 74 generated ideas will be
examined based on novelty, functionality, market needs articulated and how the ideas are developed in
a cross-functional manner. The second part of this first analysis will focus on the perception of the
screen team and how they determine “winning ideas”. This will also be a more in-depth analysis on
how the screen team evaluate and rate the novelty of ideas in relation to how the crowd rates them. Due
to the scope of this thesis, the activity, time spend on the campaign, diversity of tenure within
Novozymes, or the age, gender or location will not be investigated.
5.1.1 Outcome of “New claims for detergent”
In order to measure, (i) idea quality; (ii) novelty of ideas; and (iii) cross-functional co-development of
ideas, it is needed to go into the ideas generated and classify the different parameters. This will be
explained further in the following section. The screen team evaluation will be presented afterwards, so
it can be put into the provided regression model in the second part of the analysis.
The table (3) below shows the total outcome of the dataset from the campaign, in sense of number of
descriptive counts of from the ideas generated. The table is set up in order to identify hypothesis 1 and
2, with regards to the number of characters and references in the ideas. The table also shows the total
outcome of ideas and the metadata of the generated outcome in terms of; comments, submitters, votes,
number of novelty, market needs articulated and cross-functional co-development of ideas. The
different data point will be further analyzed in the following sections, in order to answer the hypothesis
in discussion chapter.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
47 | P a g e
Table 3; Total outcome of the dataset:
Totals of the dataset No.
Total no. of participants 105
Total no. of ideas 74
Total no. of idea submitters 53
Total no. of comments 200
Total no. of votes 744
Total no. of ideas w. Cross-functional involvement 38
Total no. of novel ideas to Novozymes 27
Total no. of idea w. market needs articulated 33
Total number of references 14
Total numbers of ideas with more than 420 characters 60
5.1.2 Ideas with cross-functional involvement:
In the Chapter Company and Case, it is presented how the crowd was composed:
Figure 6; Composition of the Crowd
This section goes into detail about the composition and the distribution of the cross-functional ideas
that was co-developed by the crowd. The crowd consisted to 50% of R&D employees and 50% Sales &
Tech Service and Marketing.(Sales and Tech Service reporting to the same executive Vice President):
the cross-functional composition of the crowd was chosen to enhance the idea development in a more
cross functional way. By cross-functional involvement is meant e.g. that if a participant coming from
R&D submits an idea and it is only R&D colleagues that comments on the idea, this is classified as
R&D ideas only. The table provided is an overview of how the idea with cross-functional involvement
was distributed in the campaign. It also shows how the cross-functional ideas performed in the
evaluation of the screen team.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
48 | P a g e
Table 4; Distribution of cross functionally discussed ideas
Distribution of cross-functional ideas Total ideas: Cross-func. Top 25 ideas
R&D 34 9 3
Sales 17 9 1
Marketing 11 9 3
Technical Service 12 11 4
Total: 74 38 11
R&D submitted 34 ideas in all, out of these 34 only 9 ideas had cross-functional involvement in the
comments within the ideas. 3 of the ideas with cross-functional involvement got rated in the top 25
ideas by the screen team.
Sales submitted 17 ideas in all, out of these 17 ideas, 9 ideas had cross-functional involvement. 1 idea
with cross-functional involvement got rated in the top 25 ideas by the screen team.
Marketing submitted 11 ideas in all, out of these 11 ideas, 9 ideas had cross-functional involvement. 3
ideas with cross-functional involvement got rated in the top 25 ideas by the screen team.
Technical Service submitted 12 ideas in all, out of these 12 ideas, 11 ideas had cross-functional
involvement. 4 ideas with cross-functional involvement got rated in the top 25 ideas by the screen
team.
The results of the cross-functional involvement shows that even though R&D posted most ideas there
was “only” 9 ideas with involvement from other departments and 4 ideas in the top 25. Ideas from
Technical Service had 11 out 12 ideas with cross-functional involvement, and 4 ideas in the top 25.
This could suggest that ideas from Technical Service should be the superior in posting ideas that
creates cross-functional involvement. But if we take a closer look at the involvement of the ideas
posted another picture emerges. Out of the 38 cross-functional ideas R&D was involved in 35 of the
ideas In relation to the others where Sales was involved in 13 ideas, Marketing was involved in 21
ideas, Technical Service involved in 14 ideas. So it is clear that R&D is heavily active in the comments
of the ideas, whereas an idea coming from Technical Service was more likely to get discussed or
commented by others.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
49 | P a g e
Table 5; Origin of the Top 25 ideas
Where idea origin from No. Of ideas in Top 25
R&D 15
Sales 4
Marketing 2
Technical Service 4
Total 25
The implications of these findings will be discussed later in the Chapter Discussion.
5.1.3 Novelty of ideas:
One of the criteria the screen team was set up to evaluate was the novelty of an idea. The novelty of an
idea means that, the idea needs to be new to Novozymes. This could be a new technology, or a new
way of solving detergents problems, or just an incremental improvement to the existing process.
Each of the five screen team members was asked to score the novelty of an idea on a scale from 1-5, 5;
being a completely new idea and unexplored to Novozymes, 1; for being an incremental improvement
of a known concept or technology. All five screen team members was not obliged to rate all of the
ideas individually, so some of the ideas was only rated by some of the Screen team members.
If the novelty sum rose over a score of 10 points, this is classified as a new idea to Novozymes, since
this is where the screen team agrees that this is a fairly new idea. Figure 8 displays the distribution of
the novelty sum of all the ideas generated. 27 ideas were rated higher than the 10 point mark. That
correspond to around 37% of the posted ideas was new to Novozymes.
Figure 7; Distribution of new ideas
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
50 | P a g e
5.1.4 Clustering of the ideas
The initiative to cluster and order all ideas generated by the crowd was taken spontaneously by the
screen team who got intrigued by emerging connections between the presented ideas. the original
overview table produced by them is reproduced in Appendix 3. It needs to be stated already here that
this ad hoc clustering exercise performed by the screen team was not according to the originally
planned protocol of the campaign, as detailed further in the discussion chapter.
In connection with the present thesis research the original two-dimensional idea-chart was expanded to
contain a third dimension which was the final screen team score (figure 8). After analyzing the ideas it
was possible to illustrate graphically how the clustering of the ideas shaped a three-dimenional map.
This enabled the analysis to demonstrate what an idea should contain to be rated high in the evaluation:
Figure 8; Comprehensive idea map
The entire idea population depicted as a three-dimensional plot according to customer ideas addressed (x-axis),
proposed enzyme technology/enzyme class (z-axis) and accumulated score (y-axis)
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
51 | P a g e
The figure above shows how the ideas look like when they are clustered together with the proposed
technology and the customer issue addressed within the idea. The way this analysis is done is by taken
the average of the screen team score in each cluster to see which one performed the best. It is very clear
that the ideas containing the technology “deglycosylation of coloured compound through glycosyl
transferase” is by far the best rewarded when combined with the customer issues of stains from “fruits
and vegetables” and “Tea, coffee and wine” where the averages score is 102 from the screen team.
These ideas also went into further assessment after the idea campaign.
The second best rewarded area is the “Oxidoreductase” combined with the issues of “grass” stains with
an average score of 94. This suggests that Novozymes should look more into these areas since it is of
high priority and there is a lot of knowledge being shared around these clusters. However what it also
reveals is that if an idea is lacking a specific technology, or lacking the addressing of a concrete
customer issue, then it is rated very low. It shows that the ideas that do not address a specific
technology or a specific customer, scores an average of 41.5 which is almost 60% lower than the top
ideas.
5.1.5 Responsiveness to market needs:
In order to examine the market needs articulated in the posted ideas, the ideas have been examined
through to identify and classify where the market needs are articulated, the following section shows the
classification of ideas and put into 3 categories.
Category 1: No market needs articulated at all.
Category 2: A market, trend, competitor or consumers is mentioned.
Category 3: A specific customer “name” or a specific customer request is mentioned.
To explain in detail what is meant by “market needs articulation”, the following provided is a few
examples from the online ideation in the category of 2 and 3:
Idea 20: …Arabic gum is added as 0.8-2.4% into 3 of Chinese national stains (sebum, protein, carbon
black stains) as sticker and emulsifier. it is also a popular worldwide food ingredient used in e.g.
cheese, cream ,processed fruits etc. Arabic degrading enzyme like mannaway to mannose could largely
improve the stain removal for food enzymes as well as great help to Chinese specific stains and sales in
China…
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
52 | P a g e
Comments: … We have been discussing that a claim enzyme may have a hard time in the Chinese
market - but I guess that providing cleaning benefits on several of the Chinese national stains may be
the way into the Chinese market? ....
This example is in category 2, because it resonate to a market, in this case the Chinese market. The
comment supports the idea and comes with some insights on how the market situation is in this market,
and states that this could be a way for Novozymes to open up into this market where they not currently
having any business.
Idea 44: …Big soaper companies like P&G, Unilever and Henkel have divisions involved in making
colored pens or could quickly acquire them. Why not co-develop a family friendly marker system where
enzymes in the washing powder will remove any of the colors in the marker pen set? You could put a
special seal on the pens showing they are part of the "color be gone" system…
Comments: …By mixing these basic colors together, you get all the colors that a desktop printer can
make (well maybe we need black too). I think that we should get both Novozymes and P&G experts to
work the problem…
Comments: … We can include pens yes - How many pens does P&G and Novozymes give out? That is
the place to start. That is how Post-it notes started. What a great gimmick to explain to customers the
power of our enzymes…
This example is scored in category 3, because the idea is built on some direct customers business where
Novozymes could go in and contribute with some technology, in this case some of the biggest
detergents sellers in the world (Unilever, P&G). The comments responses in a great fashion, especially
the first comment suggest Novozymes should go into closer collaboration with the partner P&G. The
second comment goes a bit further and suggests that Novozymes creates a gimmick to explain the
power of the enzymes within the product.
Idea 46: …Improve swimming pool cleaning process (to degradate body grease, sun protector base) in
a more sustainable way by applying enzymes to replace chemicals…
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
53 | P a g e
Comments: …We already received 2 customer’s request on this sense in Latin America, important
idea in my point of view…
This example is in category 3 as well, because the idea takes steam from an existing problem, and
provides the technical solution, but especially the comment that Novozymes in Latin America already
have received directly indications of customer needs is important for the idea. This is a clear example
of that sharing ideas can lead to insight from elsewhere in the organization.
Idea 56: …Most of the fabrics on the market now are mixed fabrics of polyester and Cotton. However
many technical clothes like sport equipment are made of polyester only. There are no solutions on the
market for improving the feel of these fabrics and our cellulases have no effect on polyester. Esterases
have already been proven to perform fabric modification on polyesters and a combination of esterases
with cellulases has shown great effects on mixed fabrics...
Comments: …We have positive experience with Cutinase in this area. RD did work around 2004-5 if I
remember right and the idea is currently part of a surface modification project running with P&G…
Comments: … We have received a lot of customer’s request on that. Great idea…
In this example the idea is very much focusing on the technology, however the first comment directs
the attention to a project already running with a customer (P&G), that might give a clearer
understanding of if there have been developed more in-depth research that could benefit the idea. The
second comment, makes the idea even more interesting, by sharing that there have been received a lot
of direct customer request to this problem. This makes the idea qualify into category 3.
This example is taken directly from the outcome of the idea campaign, there are a lot more, so these are
just some few examples of how the classification of the ideas into the categories. It is however hard to
draw a specific line whether when ideas end up in a category 2 or 3, thus the selection and
categorization of the chosen ideas is made in together with internal experts from the Innovation Office
in Novozymes, especially in some of the more technical advanced ideas.
Identifying this allows the thesis to distinguish between if ideas are closely related to the market needs
or more focused on e.g. technology or processes within Novozymes. This enables the findings the ideas
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
54 | P a g e
that are most likely to have a concrete customer to take the idea to market. The purpose of this
investigation is to discover if the crowd can take the market needs into their ideas and to see how this is
received by the rest of the crowd and the Screen Team.
In the second part of the analysis it will be tested if there is a significant evidence and correlation
between the ideas with high market articulation and if the crowd and screen team have the same
perception of the way they rate these.
Table 6; Distribution of ideas with market articulation
Distribution of market needs articulated: No. of ideas
Category 1 43
Category 2 15
Category 3 16
Total 74
The table above shows the outcome of the distribution of market needs articulated ideas, rated and
based on the presented criteria. These variables will be used in the regression model presented in the
second part of the analysis.
5.1.6 Screen team evaluation:
As presented in the Case description, the screen team was composed in a cross-functional matter with
experts within the detergent area. The screening criteria was based on both technical and market
aspects. The conducted documents from the screen team states which arguments and scores that lay
behind the final scoring of the ideas. The evaluation was done over a short period of 2 days, the
members were assigned to score the ideas individually and later the scores was summed up and
discussed at a final review meeting.
The screen team document reveals the specific scores and the individual ratings and comments on the
ideas from the members. The six scoring criteria were divided in two categories, a commercial score
and an enzyme score. Commercial score with refers to the market probability and the enzyme score
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
55 | P a g e
refers to the technical probability. The reason this has been done is to investigate how the screen team
was viewing the online ideation and to what extent the technology and the market played a role in the
selection of the winning ideas. The two variables were distributed according to the ideas and reveals
that the relationship between the two scores in the evaluation process. (Appendix 1)
Figure 9; Statistical comparison of screen team scores given on commercial and technical criteria
The figure (9) above shows the distributed scores calculated based on the retrieved documents form the
screen team review meeting. The two scores were put into a t-Test to illustrate the difference in the
commercial and the technical scoring aspects.
When the screen team score is broken into these two series, it reveals that the customer score are rated
highest in the in the evaluation, this clearly indicates that the market is a heavy represented in the
screening process. This is also shown in the comments around these. The screen team is therefore not
only focused on the technology probabilities but mainly the market aspect of the ideas.
The two series states clear that the winning idea (idea 45) was very strong in both commercial and
technical probability, whereas a lot of the poorly rated ideas had a too strong focus on the market and
did not live up to the technical probabilities wanted to go into further assessment of the idea.
The evaluation process was not presented to the crowd, however in the following section of the
analysis this will be exposed more toughly through the regression models provided.
t-Test: Two-Sample Assuming Unequal Variances
Commercial Score Technical Score
Mean 29,405 23,338
Variance 322,546 231,925
Observations 74 74
Df 142
t Stat 2,217
P(T<=t) one-tail 0,014
t Critical one-tail 1,656
P(T<=t) two-tail 0,0282
t Critical two-tail 1,977
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
56 | P a g e
5.2 Conclusion of analysis part 1:
In the first part of the analysis there was provided an overview of the performance and the distribution
of ideas generated in the idea campaign.
Firstly, the descriptive statistics of the output has demonstrated a good insight on where the ideas came
from and how they were constructed from the campaign. It was shown that 74 ideas were submitted by
53 participants, which is fairly okay with a submitting activity of 68%. Furthermore 38 were cross-
functional ideas and 27 with high novelty and 33 ideas with high market needs articulated in them. This
could indicate that the data sample is quit representative when looking at similar research within this
research area.
Secondly, the cross-functional distribution of ideas was examined, showing that the 50/50% spilt
resulted in 38 ideas with cross-functional involvement. It was examined further how these 38 ideas
originated, which showed that ideas created out of the Technical Service Department had the highest
likelihood of being cross-functional with 11 out 12 ideas. It was also revealed that R&D presence was
strong in these ideas; they were involved in 35 of the 38 cross-functional ideas.
Thirdly, the novelty of ideas was examined, with an overview of the distribution of new ideas. All the
27 ideas was rated high (a score of >10). Also here there where a strong presence from R&D who was
responsible for 18 of the 27 ideas generated, and furthermore it was that R&D was involved in almost
every new idea except for 2.
Fourthly, Figure (9) of the clustering of ideas was provided, in order to identify where the “winning
ideas” were lying and how they were scored The grouping of ideas showed that in order to score high
on a submitted idea it needed to be related within two different customer issues, and be within one
specific technology.
The market needs articulation was examined and explained by providing examples of ideas with
medium/high market needs from within the idea or comment. Here it was concluded by the results of
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
57 | P a g e
the distribution provided, that out of the 74 ideas 15 where with medium market needs and 16 ideas
with high market needs articulated.
Lastly, the screen team evaluation documents revealed that, the screen team actually was rating the
market needs or customer issues relatively higher the technical ideas. This is a strong indicator of that
the expert team, goes into a far more specific background search to i.e. internal databases to see if the
ideas have been tested out before or if any of proposed customers actually want to pursue the ideas in
there respected markets. What is interesting here is that there seems to be an indifferent approach to
how the screen team evaluates the ideas, and how the crowd votes the ideas they think is the most
realistic to realize.
The second part of the analysis will focus on putting these findings into a regression model to see the
relations between the screen team and the crowd.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
58 | P a g e
5.3 Analysis part 2
The first part of the analysis provided a descriptive overview of the outcome of the idea campaign and
demonstrated how the crowd performed and how the screen team received and rated the ideas.
In this part of the analysis there will be provided a more in-depth investigation of implications on the
“market need” articulated in the ideas, and where the crowd and screen teams favorite ideas origin. The
hypotheses developed in the Problem statement will be tested in the regression analysis provided. The
goal is to see how these results match in terms of correlation between the crowd and the screen team.
The wanted test is created to see if the crowds and the screen teams are receptive to market needs
articulation, and if so, can this create an acceleration of the organization’s response to market needs,
faster and more accurate and deliver more customer-driven innovations.
5.3.1 Putting the numbers into relation
This section goes into investigation of how the different results from the previously analysis can be put
into use when answering the developed hypotheses. This will demonstrate how the relations are form
and is done though simple regression analysis. This will enable the thesis to identify if there is any
significant evidence or correlation, in how the perception was from the screen team results and how the
crowd voted. Secondly the investigated model will show if new ideas are significant to market needs
articulated or if they are cross-functional.
The method used for the regression analysis is described in the Chapter Method and research design.
Dependent Variables:
As dependent variables, as described earlier, the screen team score was used (mean = 1.92 and SD =
1.10) in the first model, the variable is generated into a linear string in order to use it in the regression
model. The second dependent variable is the votes from the crowd score (mean = 1.81 and SD = 1.12);
this variable is also generated into a linear string.
Independent Variables:
Market needs articulated in idea (i); is the variable that describes how many ideas had market needs
articulated within them, and the once that didn’t, (mean= 1.63 and SD= 0.82). Idea origin; (ii) As three
dummies: idea from Sales (mean = 0.22 and SD= 0.42), idea from R&D (mean = 0.48 and SD = 0.50),
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
59 | P a g e
idea from Technical Service (mean = 0.14 and SD = 0.35) and idea from Marketing (mean = 0.12 and
SD = 0.34), (Marketing is sat as a base line). (iii) New idea to Novozymes is made from the evaluation
of all the ideas (mean = 0.58 and SD 0.49). (iv) the cross-functional variables (mean = 0.51 and SD =
0.50), is create from the descriptive analyses, where all the ideas was identified and examined for
cross-functional involvement.
Table 7; Regression model of the screen team Score and the crowd score:
Model 1 Model 2
VARIABLES Screen Team Score Crowd Score
Market needs articulated in idea -0.33183 0.64742**
(0.43062) (0.25057)
Idea from Sales Dept. 0.76443 -1.10436***
(0.74417) (0.39894)
Idea from R&D Dept. 0.61392 -0.21396
(0.77153) (0.35285)
New idea to NZ 0.39057 0.44262*
(0.44965) (0.26528)
Idea from Technical Service 0.60793 -0.10676
(0.72910) (0.40117)
Cross-Functional w. involvement 0.54272 -0.05331
(0.54876) (0.30033)
Constant 2.37272*** 1.68859***
(0.78560) (0.35666)
Observations 74 74
R-squared 0.05479 0.27269
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
60 | P a g e
6. Discussion
6.1. Revisiting the research hypotheses
The overall research question of this thesis was whether the crowd of ‘New Claims’ could be
considered a ‘wise crowd’. The nine hypotheses chosen to test ‘crowd wisdom’ were grouped in four
categories: idea quality, co-creation as expressed in cross-functional idea discussions, novelty and
responsiveness to ideas which contained market needs (Table 1).
Testing hypothesis 1 it was found that 82% of all ideas contained more than 420 characters, which
corresponded to three ‘tweet-length’ statements. Given the acceptance criterion of 75%, hypothesis 1
could be accepted indicating that the crowd articulated their ideas in test strings of sufficient length to
express relatively complex thoughts (Table 4, analysis 5.1.1.)
Likewise, the second hypothesis regarding idea quality was accepted as well since it turned out that
80% of all ideas contained at least two comments, with the acceptance criterion being 75% (Table 4,
analysis 5.1.1.). In this thesis ‘one idea plus two comments’ was viewed as the minimum requirement
to justify the term ‘idea discussion’.
Hypothesis 3 concerned the number of references stated in the prposed ideas, the threshold for
acceptance being that 25% of all ideas should contain at least one internal or external reference. Only
18% of the ideas lived up to this condition whereby hypothesis 3 is not accepted (Table 4, analysis
5.1.1.).
With regards to the crowd’s capability of posting novel ideas it was found that 37% of all ideas were
considered as novel by the screen team (figure 8, table 7, and analysis 5.1.3.) whereby hypothesis 4
could be accepted considering 25% as the threshold.
Regression analysis was performed to answer the question whether novel ideas were correlated with
high scores. This turned out to be the case for the crowd whose dot-vote score correlated with novelty
at a p-value smaller than 0.044 (significance level: p<0.05). Interestingly, this was not the case for the
screen team where p < 0.390 (Table 7, analysis 5.3.1).Apparently, the crowd gave high ratings to novel
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
61 | P a g e
ideas but the screen team did not. Hence, hypothesis 5 is accepted but the discrepancy between crowd
and screen team perspective needs to be noted.
Evidence for co-development of ideas was investigated by analyzing the occurrence of cross-functional
involvement in the discussion of ideas. It was found that 51% of all ideas fulfilled this criterion
whereby hypothesis 6 could be accepted (threshold was 25%), (Table 5, analysis 5.1.2.).
Regarding the question whether cross-functional involvement also was a predictor of idea ranking it
was found that the correlation to the crowd’s dot-vote was p=0.053. Thus, hypothesis 7 is – in a strict
sense – not accepted for the crowd but still very close to the acceptance criterion of p<0.05. Even if the
criterion was formally not met it is safe to say that the crowd in ‘New Claims’ had a pronounced
tendency to rate ideas with cross-functional discussions as high. As with the novelty-test (hypothesis
5), cross-functional discussions were not a predictor for high scores as given by the screen team
(p=0.326) (Table 7, analysis 5.3.1.).
Finally, investigating the crowd’s responsiveness to ideas with articulated market needs it was observed
that 41% of all ideas contained elements and statements which reflected market needs (Table 7,
analysis 5.1.5.). Hence, hypothesis 8 was accepted. With regards to the correlation of ‘market need-
incidence’ with idea scores given by the crowd a p-value of 0.012 was found , which in sensu stricto
does not meet the acceptance criterion (p<0.05) but is still considered as substantial. Again, and in line
with the preceding observations, no correlation between ‘market need articulation’ and the screen team
score was found (p=0.444). Thus, hypothesis 9 was not accepted.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
62 | P a g e
Table 8; Summary of findings and hypotheses acceptance
Category Hypothesis
no.
Hypothesis results Acceptance? Method of
testing –
Idea quality 1) 82% of the ideas contained more than 420
characters.
Accepted Descriptive
statistics
2) 80%, thereby is the hypothesis accepted Accepted Descriptive
statistics
3) 18% therefore is this parameter not accepted. Not Accepted Descriptive
statistics
Idea Novelty
4) 37% of the ideas was rated highly novel
accepted
Accepted Descriptive
statistics
5) Crowd score p-value was p=0.044
Screen team score p-value was p=0.390.
Accepted Regression
analysis
Idea
co-
development
6) 51% of all the ideas lead to discussions with
cross-functional involvement.
Accepted Descriptive
statistics
7) Crowd score p-value was p=0.053,
Screen team score p-value was p=0.326
Not accepted in
sensu stricto
Regression
analysis
Responsivene
ss to
market needs
8) 41% thereby is the hypothesis accepted Accepted Descriptive
statistics
9) Crowd p-value was p=0.012,
Screen team p-value was p=0.444.
Not accepted
in sensu stricto
Regression
analysis
6.2 The crowd‘s excitement about the presented ideas was not shared by the screen team
When measured in relation to the nine formal criteria alone, the crowd in this thesis could indeed be
regarded as ‘wise’: they proposed many novel ideas which were of often of high quality; they
commented actively on each other’s ideas and many ensuing discussions took place across the involved
business functions, which is an indicator of co-development or co-maturation of the initial ideas.
It also appeared that many ideas brought forward by the crowd were stated in the context of - or in
relation to - market needs, which suggests a market-oriented, outward looking professional mindset
prevailing in the crowd.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
63 | P a g e
Furthermore, it was remarkable that the crowd through their collective dot-votes showed consistent
positive reactions to novel ideas, ideas developed through cross-functional discussions and – most
importantly – in relation to ideas containing market needs. The fact that hypotheses 7 and 9 were not
accepted in sensu stricto can be regarded as a mere formal detail which does not change the overall
picture of a by and large innovative, collaborative and ‘market-responsive’ crowd.
Again, this statement has to be viewed in a somewhat isolated and absolute perspective because as soon
as the comparative analysis of the screen team’s reactions comes into play one has to take a more
differentiated view on the crowd of ‘New Claims’. It is perhaps the most interesting finding of this
thesis that the screen team consistently did not share the crowd’s ‘excitement’ about novelty, co-
development and market needs. None of the three studied correlations was significant or came even
close to the significance level of p<0.05. In a sense one could say that the screen team was remarkably
irresponsive to key idea features which triggered strong reactions by the crowd.
6.2 The screen team’s perspective
The observation of consistent discrepancy between the crowd and the screen team’s perspectives called
for a closer investigation as presented in table 9. In comparison to the crowd the screen team’s
approach to the presented ideas was very different from a several point of views.
Firstly, instead of using freely distributable dot-votes, the screen team was confined to rate each and
every idea on six predetermined screening criteria using quantitative scores. Up to five points could be
given to each of the criteria. In this way a criteria-based ranking of ideas was established.
Secondly, the screen team also went to great lengths to record their reasons of their scorings in a
comprehensive spreadsheet (appendix 2). While scores were given by each of the ‘screeners’
individually, the comments to the scores were typed by the group during their screen team meetings.
These comments are the main source to explain why the screen team had significantly different
perspective on the presented ideas. It appears that the screen team checked ideas rigorously for the
presence, validity and consistency of supporting information. The quotes from the screen team
comments below are representative for their meticulous work. They recorded the absence of supporting
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
64 | P a g e
evidence, they commented on inconsistencies between claims made by idea authors and the factual
source situation. (Appendix 2)
Idea no. 17, comments from the screen team: “Not clear what the substrate is - is mustard and turmeric
the same? We have looked at curry stains in the past. Not clear if oxidation or hydrolase is the answer.
Stains are only relevant in some regions. NA - 417 records in LUNA7 could have been read before
submitting as 'new' idea. oxidoreductase?”
Idea no. 62, comments from the screen team: “Stain act as mediator, Interesting for emerging markets,
if we have the enzyme. Tested before in an old Unilever project, Reckitt Benckiser is interested. Worth
revisiting.”
In summary, the impression prevails that the screen team’s approach to establish an idea ranking was
much more rational and critical than the crowd’s. Where the crowd seems to react almost impulsively
on trigger words indicating novelty and market relevance, the screen team assumes a calm stance and
takes deeper look to validate the provided information.
However, despite the evidence pointing to a sober and rational behavior there was one particular
observation which indicated that even the screen team did not act completely free of any bias. Out of
the six screening criteria three were of commercial nature and three of technical nature (appendix 1).
The statistical analysis of the scoring distribution showed that the screen team had a significant
tendency to award higher scores on the commercial criteria than on the technical ones (figure 9). This
could suggest that the screen team was more captivated by the potential market impact and commercial
relevance of the presented concepts (ideas) than by their technical quality. With some caution it may be
argued that this commercial bias reflects an outward-looking, market- and customer- oriented
perspective prevailing in the screen team.
7 LUNA is an internal database in Novozymes
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
65 | P a g e
6.3 From singular ideas to ‘strategic idea landscapes’
A surprising and unintended but highly intriguing outcome of ‘New Claims’ was that the screen team
made an ad hoc decision to categorize and cluster all presented ideas. This was not a step foreseen in
the original procedure protocol of the ideation and by taking this step they pushed their ambition level.
Now it was not only about finding a few promising individual ideas but to look for idea patterns and for
new themes and trends. One of the screeners explained: “Reading all these ideas we realized that many
ideas were thematically related. They represented contextual clusters. Thus, we decided to take a closer
look and see if we could arrive at a systematic way of clustering”. They came up with a categorization
grid which consisted of ‘enzyme types’ as one dimension and ‘commercial claims’ as the other
(figure 8). As a next step they categorized each idea as belonging into one of the fields on the grid,
which allowed them to establish a synoptic overview.
During the research leading to this thesis it became obvious that one additional step could be taken. By
adding the screen team’s quantitative scores as a third dimension their idea-grid became an ‘idea-
landscape’. Now it was evident that the idea and score accumulation in certain fields gave rise to bona
fide ‘strategic peaks’: certain proposed technologies would allow certain new claims which were scored
as having different levels of attractiveness as viewed from the screen team’s perspective. Three-
dimensional maps like the one presented are highly intriguing because they provide a novel set of
perspectives which may complement or – even more interesting – conflict with the established strategic
paradigms prevalent in management circles of the organization.
Strategic maps derived from crowdsourcing, even in their present simple form, may helpful to
challenge conventions and to lead to discussions of new angles. In the case of ‘New Claims’ the
identification of the idea category ‘deglycosylation glycosyl transferases for the decolouring of fruit
and vegetable stains’ represented such a new angle for future development, which was hitherto not
explicitly articulated and discussed as a potential innovation vertical. A first step to explore that
innovation potential could be to run the next crowdsourcing exercise specifically on the identified new
theme or category.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
66 | P a g e
Finally, it also needs to be pointed out that the author of this winning idea was a relatively young
female scientist from Latin America who had moved recently to Copenhagen to join Novozymes. Her
significant innovation contribution was to apply scientific insights which she brought with her from
another part of the world in the context of her new company.
This finding was reminiscent of literature on combinatorial innovation and knowledge creation by
Keller, (2001) Cross-Functional Project Groups in Research and New Products Development:
Diversity. Thus, it could indicate that the theory presented by Surowiecki, 2004, is true in this case,
since the crowd was acting as a crowd with high exchange of ideas and collaboration (74 ideas with
over 200 comments). In Surowiecki, 2004, criteria regarding how to assemble the rational crowd state
that the crowd needs to have diversity of opinion, independence, be decentralized and aggregation in
there construction, in order to be a rational crowd and to gain Collective intelligence. However, since
the screen team was indifferent it can be argued that the incentives for creating ideas was not too clear
for the tasked crowd, which by Surowiecki 2004, is one of the reasons why a crowd can fail in an
attempt to be rational.
In the theory related to the amount of novelty generated in the ideas, where it was found that 37% was
in the category of a new idea and whereas R&D was responsible for over 60% of these, could indicate
that the notion of innovation still comes from R&D is still a key factor in developing new knowledge,
as argued by Bos et al. (2007), “Scientists generally work with ideas that are on the cutting edge of
what is understood. This knowledge often requires specialized expertise, is difficult to represent, may
be tacit, and changes rapidly”.
However, the regression analysis showed that there was no significant evidence that ideas coming from
R&D necessarily were better than from the other departments. In Poetz and Schreier (2012) it was
found that ideas coming from engineers had high technical feasibility, however, it was also found that
the customer benefits were represented fairly low in contrast to ideas coming from Sales or directly
from consumers. The study also showed that the ideas coming from the others than professional
engineers were much higher rated in terms of novelty and customer benefits. A similar trend was
found in the present research in relation to the distribution of cross-functional ideas, it was clear that
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
67 | P a g e
ideas from Technical Service were widely discussed, with 11 out of 12 ideas. This indicated that the
combination of involvement from Sales or Marketing on one side and R&D on the other side created a
high flow of creative novel ideas. It is important to emphasize that in order to succeed as a new idea,
R&D needs to be involved in the idea, this goes good in hand with the study from Poetz and Schreier,
2012.
Nonaka (1994:15) “… although ideas are formed in the minds of individuals, interaction between
individuals typically plays a critical role in developing these ideas. That is to say, ‘communities of
interaction’ contribute to the amplification and development of new knowledge”.
This seems to be the case in this thesis of creating new knowledge. Poetz and Schreier (2012) stated
that professional engineers outperformed customers in the creation of feasible ideas, in contrast to the
ideas generated outside the company, in this thesis the screen team is acting in the same manner, which
is an indication that the screen team was clear in their criteria of the ratings, which is understandable
since they were to a higher degree responsible for further assessment of the ideas.
One of the benefits of internal crowdsourcing as argued by Simula et. al. 2012, is that, since it is an
internal crowd there are no issues in terms discussing IP or company secrets which is good when
developing new ideas, however Simula et.al. 2012 also argue that this could lead to a crowd not being
able to generate non-obvious ideas which in turn could lead to feasible market-relevant solutions. In the
present investigation of market needs as articulated in the presented ideas, it was found that the crowd
reacted positive to ideas containing such; however, there was no significant evidence that the screen
team follows suit. This could indicate that the screen team was more focused in the on technical
feasibility. The study of Poetz and Schreier (2012) showed the same indications, namely that the
screening of the ideas came out in favor of the commercial implementation rather than the technical
feasibility.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
68 | P a g e
7. Conclusions and Perspectives - Moving from idea-hunting to ‘crowd-strategizing’
The initial intention of ‘New Claims’ was essentially idea-hunting and identifying a few trophy ideas.
Seen in this light, the bona fide discovery of a cognitive step-sequence which leads to ‘crowd-
strategizing’ was completely serendipitous. However, it also needs to be pointed out that this
unintended ‘discovery’ was essentially enabled by a highly competent and ambitious screen team. It is
not known at present whether the step-sequence which unfolded in ‘New Claims’ can give rise to a
repeatable standard protocol, but a couple of conclusions and assumptions seem to be important:
(1.) An engaged and productive crowd of ‘right size’ - The basis for any success – be it idea-hunting
or crowd-strategizing - is a highly engaged crowd which produces a sufficiently large number
of ideas with acceptable quality, novelty, diversity and market relevance. Their ideas and
interactive comments represent a somewhat chaotic and unordered universe of biased notions,
perspectives, postulates and suggestions. However, this body of ideas and comments also
represents the indispensable raw-material for the ensuing cognitive process which leads to
ordered innovation insights. In this context it becomes clear that the size of the crowd is a
critical factor to consider: in the thriving innovation culture of Novozymes oversized crowds
are likely to generate oversized idea universes which would simply overwhelm the absorptive
capacity of the screen team. The size of the crowd and their engagement level must be gauged
in relation to the culture of the campaign-hosting company.
(2.) A cautious stance on pushing innovation burden onto the crowd - It is debatable whether active
attempts should be taken to ‘discipline’ the crowd during the chaotic idea posting and
discussion phase. For instance, screen teams may be tempted to ask the crowd to include more
supporting references. By doing so they would request the crowd to share some of their
innovation burden, the likely risk of posing this request is crowd demotivation. Thus, it
becomes a question of balance: does a screen team want to risk demotivating a crowd to gain
better referenced ideas?
(3.) Crowd creativity is more important than crowd wisdom. Dot-voting as a mechanism for
establishing an idea ranking through the crowd is certainly helpful because it provides a form of
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
69 | P a g e
collective praise to popular ideas and their authors. However, the present thesis also clearly
showed that it cannot be trusted as a mechanism to identify ideas with significant innovation
impact. There was no correlation between the crowd’s perspective and the screen team’s
perspective. Thus, the main research question of this thesis can be answered such: the crowd of
‘New Claims’ can certainly not be regarded as wise when compared to the screen team.
However, it is also true that the screen team is very unlikely to have conceived the idea map
shown in Figure 9 by themselves alone - i.e. without the previous mass-input from the crowd.
(4.) Screen team wisdom and competency are essential. Clearly, a highly knowledgeable, critical
and open-minded screen team is needed to meticulously scrutinize crowd-input. Capable screen
teams need to recognize weakly supported ideas and rank them accordingly. On the other hand,
they also need to recognize the few truly new high-potential ideas. Furthermore, and even more
importantly, they must be able to elevate their perspective from the individual idea level to the
idea-universe as such in order to recognize the emerging clusters and themes. Ultimately, they
must strive to identify axioms of order which allow them to put idea clusters into a coherently
logical and systematic ‘strategic’ map.
(5.) A general protocol for cognitive performance for the screen team? The cognitive steps which
the screen team must take to transform ‘chaos into order’ and which are necessary to move from
‘notions to insights’ are undoubtedly very challenging and the right approach may differ
significantly from campaign to campaign. At this point it is unclear and speculative whether a
standard operation procedure could be developed here. However, looking at the rapid progress
and the spreading use of algorithms for dynamic tagging, visual clustering and semantic
searches it seems very likely that the next generation of idea management systems will support
such cognitive processes.
(6.) Building organizational innovation capabilities requires repeatability. Repeatability is a key
condition for successful learning cycles and continuous innovation in the organization. In the
present thesis a single campaign has been investigated in-depth but we need to remind ourselves
that Novozymes is running such internal crowdsourcing exercises on a regular basis and at a
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
70 | P a g e
monthly to bi-monthly pace. To ensure long-term repeatability one must again consider crowd
motivation in the context of the prevailing company culture. Given the fact that the crowd
composition of many Novozymes R&D campaigns is fairly constant, how can one sustain a
high level of crowd engagement while running a high campaign frequency?
In order to avoid crowd-attrition and demotivation it is recommended to conduct a crow-
debriefing meeting after the online campaign has been finalized. Current praxis in Novozymes
is that the screen team announces the winning ideas on the intranet by e-mail and to award the
winners with tokens of recognition but to secure long-term crowd engagement. While such one-
way communication praxis ensures immediate gratification of the winners, many participants in
the crowd may be left behind with unanswered questions such as the underlying reasons for
picking certain ideas as winners but not others. Such questions should be addressed in an open
concluding dialogue with the screen team. Clearly, the screen team may not be in a position to
share all their knowledge with the crowd, e.g. due to confidentiality, but it is very likely that the
crowd will appreciate the screen team’s efforts to explain their decisions and resolve potential
misunderstandings.
Conclusively, it can be stated that the present internal crowdsourcing approach has a strong potential to
accelerate customer-centric innovation because it was found that a number of impactful novel ideas and
idea categories could be identified in a much shorter time period than with conventional processes. The
main reason for this acceleration effect was the cross-functional engagement of a collaborative crowd
and the subsequent idea filtering and clustering as performed by a highly competent screen team.
Clearly, the critical enabler was not the ‘wisdom’ of the crowd but that of the screen team. However, it
also seems quite evident that the structured, cognitive performance of the screen team would have not
been possible without the preceding ‘chaotic’ creativity of the crowd.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
71 | P a g e
7.1 Further Research
The research conducted and the findings of the thesis has led to some interesting results on how
Novozymes can use internal crowdsourcing in a sustainable way in the future, however, the thesis is
not able to make any generalizing conclusion in the broad perspective of crowdsourcing. In order to do
this, a larger scale of cases analysis is necessary to conduct, i.e. with more case companies and larger
crowds. This would require a large monitor tool and a new way of screen the ideas generated.
A future initiative could include crowdsourcing together with external participants, i.e. customers or
partners, to find how the ideas will be reviewed and in that sense easier to gauge their relevance
directly against market needs. This would follow the managerial recommendations, thus the knowledge
space and the innovation burden could be shared with other parties. This seems to be a logical next step
for Novozymes.
Another perspective would be to investigate an open-source development approach in the biotech
space, and how Novozymes can tap into this knowledge pool, i.e. through social media or via
prototypes experimentation. It could be investigated how external knowledge could be applied a way
that would enable Novozymes to continue development of their key applications while still benefiting
from open source input and engagement.
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
72 | P a g e
Bibliography
Bos, N., Zimmerman, A., Olson, J., Yew, J., Yerkie, J., Dahl, E. & Olson, G. (2007) From Shared
Databases to communities of Practice: A Taxonomy of Collaboratories.
Journal of Computer-Mediated Communication. Vol. 12, pp. 652S672.
Chesbrough, H. (2011) Open Services Innovation: Rethinking your Business to Grow and Compete in a
New Era. Jossey-Bass.
Flemming, L. & Sorensen, O. (2004) Science as a Map in Technological Search.
Strategic Management Journal, Vol. 25,pp 909-911.
Flynn, M., Dooley, L., O’Sullivan, D. & Cormican, K. (2003) Idea Management for Organizational
Innovation.
International Journal of Innovation Management. Vol.7, No. 4,
Freeman, RE., Wicks, AC., Parmar, B., (2004) Stakeholder theory and “the corporate objective
revisited” Organization Science, 2004
Hine, D. & Kapeleris, J. (2006) Innovation and Entrepreneurship in Biotechnology: An International
Perspective. Edward Elgar Publishing Limited.
Howe, J. (2006) The rise of Crowdsourcing
Wired Magasin,issue 14.06, June 2006.
Lakhani, K. R & Jeppesen, L. B. (2007) Getting Unusual Suspects to Solve R&D Puzzles.
Harvard Business Review. May 2007.
Lakhani, K. R. & Panetta, J. A. (2007) The Principles of Distributed Innovation.
Innovations, Summer, pp. 97-112.
Lauto, G., Valentin, F., Hatzack, F., Carlsen, M,. (2013) Managing Front-End Innovation through Idea
Markets at Novozymes
Research-Technology Management, July - August 2013
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
73 | P a g e
Keller, R. T. (2001) Cross-Functional Project Groups in Research and New Products
Development: Diversity, Communications, Job Stress, and Outcomes.
Academy of Management Journal. Vol. 44, No. 3, pp. 547-555.
Nonaka, I. (1994) A Dynamic Theory of Organizational Knowledge Creation.
Organization Science. Vol. 5, No. 1, pp. 14S37.
Pisano, G. P. (2010) The evolution of Science-based business: Innovation how we innovate.
Industrial and Corporate Change, Volume 19, Number 2, pp. 465–482
Pisano, G. P. & Verganti R. (2006) Which Kind of Collaboration is Right for you?
Harvard Business Review, December 2006
Poetz, M. K. & Schreier, M. (2012) The Value of Crowdsourcing: Can Users Really Compete with
Professionals in Generating New Product Ideas?
Journal of Product Innovation Management. Vol. 29, No. 2, pp. 245S256.
Saunders, M., Lewis, P. & Thornhill, A. (2007) Research Methods for Business Students,
Pearson Education Limited. 4th edition.
Schumpeter, J. A. (1939) Business Cycles: A Theoretical, Historical and Statistical Analysis of the
Capitalist Process.
McGraw-Hill Book Company.
Simula, H. & Vuori, M. (2012) Benefits and Barriers of Crowdsourcing in B2B Firms: Generation
Ideas with Internal and External Crowds.
International Journal of Innovation Management, Vol 16. No. 6, December 2012.
Soukhoroukova, A., Spann, M. & Skiera, B. (2012) Sourcing, Filtering, and Evaluating New Product
Ideas: An Empirical Exploration of the Performance of Idea Markets.
Journal of Product Innovation Management. Vol. 29, No. 1, pp. 100S112.
Surowiecki, J. (2004) The wisdom of Crowds: Why the Many Are Smarter Than the Few and How
Collective Wisdom Shapes Business, Economies, Societies and Nations.
Doubleday; Anchor Publisher, 2004. ISBN: 978-0385503860
Christian Brix Tillegreen MSc. Management of Innovation & Business Development Copenhagen Business School
Confidential 2013
74 | P a g e
Van de Ven, A. H. (1986) Central Problems in the Management of Innovation.
Management Science. Vol. 32, No. 5, pp. 590S607.
Yi, S. K. M., Steyvers, M., Lee, M. D. & Dry, M. J. (2012) The Wisdom of the Crowd in
Combinatorial Problems.
Cognitive Science. Vol. 36, pp. 452S470.
Media and websites:
NZ1; Novozymes Official website (2013). Innovation, retrieved 11/12/2013 from:
http://www.novozymes.com/en/innovation/Pages/default.aspx
NZ2; Novozymes annual report from (2013), retrieved 08/09/2013 from:
http://report2013.novozymes.com/
Interview with Peder Holk-Nielsen, new CEO at Novozymes (2013) from Novozymes TV retrieved
05/06/2013, from;
http://www.novozymes.tv/video/7631519/peder-holk-nielsens-view-on.
Interview with Steve Jobs from Inc. Magazine "The Entrepreneur of the Decade Award" (1st of April
1989), retrieved 05/06/2013 from:
http://www.inc.com/magazine/19890401/5602.html
Quote by Steve Jobs in "TIME digital 50" in TIME digital archive (1999), retrieved 05/06/2013, from
http://content.time.com/time/digital/digital50/08.html