Venturing, Value Creation & the UNICEF Innovation Fundunicefstories.org/wp-content/uploads/2018/01/GMPGroup18UNICEFJuly... · Strategic Level 1: The Venturing Approach to Value Creation

Alliance Manchester Business School Venturing, Value Creation & the UNICEF Innovation Fund

__________________________________________________________________________________________

1

Venturing, Value Creation & the UNICEF Innovation Fund

Brian Meagher

Ajay Melwani

Declan McKeogh

Haitham Afifi

John Santiago-Miller

Edited on 17th January 2018

This project report is submitted in partial fulfilment of the requirements for the Degree of

Master of Business Administration

Innovation


__________________________________________________________________________________________

2

Table of Contents

List of Figures .......................................................................................................................................................... 7

List of Tables ......................................................................................................................................................... 10

List of Abbreviations ............................................................................................................................................. 12

Statements and Declarations ............................................................................................................................... 13

Confidentiality Statement ................................................................................................................................ 13

Anonymity Statement ...................................................................................................................................... 13

Declaration ....................................................................................................................................................... 13

Word Count ...................................................................................................................................................... 13

Executive Summary .............................................................................................................................................. 14

Acknowledgements .............................................................................................................................................. 15

1. Introduction ...................................................................................................................................................... 17

1.1 Problem Statement .................................................................................................................................... 17

1.2 The UNICEF Context ................................................................................................................................... 19

Vision and Mission ....................................................................................................................................... 19

Served ‘Market’............................................................................................................................................ 19

Revenue and Expenditure ............................................................................................................................ 20

1.3 The UNICEF Office of Innovation ................................................................................................................ 21

The Principals for Digital Development ....................................................................................................... 21

Office of Innovation Structure and Ecosystem ............................................................................................ 22

UNICEF Ventures and UNICEF Innovation Fund ........................................................................................... 23

1.4 Research Question and Objectives ............................................................................................................. 25

General Research Question ......................................................................................................................... 25

Research Objectives ..................................................................................................................................... 25

2. Literature Review .............................................................................................................................................. 26

2.1 Approach to Literature Review .................................................................................................................. 26

2.2 General Research Question: The Venturing Approach to Value Creation ................................................. 26

Venture Capital and Corporate Venturing ................................................................................................... 26

Venturing Business Models .......................................................................................................................... 26


__________________________________________________________________________________________

3

2.3 Research Objective 1: Investment Potential of Individual Initiatives ......................................................... 28

Proposed Approach to Evaluate Individual Initiatives ................................................................................. 29

2.4 Research Objective 2: Evaluation of UNICEF Innovation Fund Portfolio .................................................... 31

Proposed Framework to evaluate the UNICEF Innovation Fund Portfolio of Initiatives ............................. 33

Proposed Framework to manage portfolio balance .................................................................................... 33

2.5 Research Objective 3: Investigation into Potential Secondary Benefits .................................................... 34

3. Research Design and Methodology .................................................................................................................. 36

3.1 Research Design ......................................................................................................................................... 36

3.2 Research Phases and Timeline ................................................................................................................... 37

Phase 1: Defining the Research Question and Objectives ........................................................................... 38

Phase 2: Desk Based Research ..................................................................................................................... 38

Phase 3: Literature Review .......................................................................................................................... 38

Phase 4: Internal UNICEF Innovation Fund Documents ............................................................................... 39

Phase 5: Semi-structured Interviews ........................................................................................................... 39

Phase 6 and 7: Triangulation Analysis and Conclusions ............................................................................... 42

4. Findings and Analysis ........................................................................................................................................ 45

4.1 General Research Question: The Venturing Approach to Value Creation ................................................. 45

UNICEF Office of Innovation Strategy and Three Horizons .......................................................................... 45

UNICEF Ventures in the Context of Uncertainty .......................................................................................... 47

The UNICEF Innovation Fund Mission Model .............................................................................................. 48

The UNICEF Innovation Fund Strategic Staircase ......................................................................................... 50

UNICEF Innovation Fund Corporate Venturing Model................................................................................. 53

UNICEF Innovation Fund Business Model Canvas ........................................................................................ 55

4.2 Research Objective 1: Investment Potential of Individual Initiatives ......................................................... 57

Proposed Framework ................................................................................................................................... 57

Principles for Evaluation .............................................................................................................................. 58

Value Proposition ......................................................................................................................................... 59

Timing of Evaluations ................................................................................................................................... 59

Initiative 1: Care for Children Results .......................................................................................................... 61


__________________________________________________________________________________________

4

Care for Children Results Summary ............................................................................................................. 67

Initiative 2: MomConnect ............................................................................................................................ 68

4.3 Objective 2: Evaluation of UNICEF Innovation Fund Portfolio ................................................................... 76

Returns and Potential Value ........................................................................................................................ 78

Interview Findings ........................................................................................................................................ 80

Paulson et al., (2007) Portfolio Evaluation Tool........................................................................................... 81

Day (2007) Risk Matrix ................................................................................................................................. 89

4.4 Research Objective 3: Investigation into Potential Secondary Benefits .................................................... 92

Interview Results ......................................................................................................................................... 92

Organisational Learning (Often Through Failure) ........................................................................................ 94

Making the Organisation Ready for Change ................................................................................................ 96

Inspiring Staff and Attracting New Types of Talent ..................................................................................... 98

Creating a Sustainable Network of Long Term Relationships Even After Completion .............................. 100

External Mimicry ........................................................................................................................................ 102

Creating New Business Models .................................................................................................................. 104

Building Capabilities ................................................................................................................................... 107

Driving Organizational Goodwill ................................................................................................................ 109

5. Discussion & Conclusion ................................................................................................................................. 111

5.1 Synthesis of Findings ................................................................................................................................ 111

Strategic Level 1: The Venturing Approach to Value Creation ................................................................... 112

Individual Initiative Level 2: Investment Potential of Individual Initiatives ............................................... 112

Portfolio Level 3: Evaluation of UNICEF Innovation Fund Portfolio ........................................................... 113

Secondary Effect Level 4: Investigation into Potential Secondary Benefits ............................................... 113

5.2 Managerial Implications ........................................................................................................................... 114

Key Findings and Implications .................................................................................................................... 114

Innovation Investment Intensity ................................................................................................................ 118

Innovation Opportunity Cost ..................................................................................................................... 118

5.3 Limitations and Avenues for Future Research ......................................................................................... 119

5.4 Conclusion ................................................................................................................................................ 119


__________________________________________________________________________________________

5

References .......................................................................................................................................................... 120

Appendices ......................................................................................................................................................... 132

Appendix 1: UNICEF Background and Context ............................................................................................... 132

UNICEF's Seven Strategic Outcomes (Medium Term Strategic Plan 2014-2017) ...................................... 132

Innovation as an Implementation Strategy (Medium Term Strategic Plan 2014-2017) ............................ 132

List of UNICEF Programme Countries ........................................................................................................ 134

Appendix 2: UNICEF Office of Innovation Background and Context .............................................................. 135

Innovation Focus ........................................................................................................................................ 135

Innovation Focus, Initiatives and Outcomes Map ..................................................................................... 135

Principals for Digital Development ............................................................................................................ 138

Appendix 3: UNICEF Innovation Fund Background and Context .................................................................... 139

UNICEF Innovation Fund Stage of Innovation Process ............................................................................... 139

Typical Nature of UNICEF Innovation Fund Projects .................................................................................. 141

Portfolio Areas ........................................................................................................................................... 141

Returns to Investors outlined by UNICEF ................................................................................................... 141

Summary of UNICEF Innovation Fund Portfolio Areas, Returns and Value ............................................... 142

Benefits and Reasoning behind UNICEF Innovation Fund ......................................................................... 144

Innovation Fund Investors ......................................................................................................................... 144

Venture Capital Approach: Innovation Fund Eligibility and Scoring Criteria ............................................. 144

Typical Venture Capital Scoring Criteria Compared with UNICEF Innovation Fund Criteria ...................... 146

Appendix 4: Innovation Portfolio: Initiatives Supported by UNICEF Innovation Fund to Date ...................... 147

Appendix 5: List of Internal Documents received from UNICEF ..................................................................... 148

Appendix 6: Benchmarking UIF against Leading VC Innovation Investment Funds in International

Development .................................................................................................................................................. 150

Appendix 7: Additional Literature .................................................................................................................. 153

Research Objective 1: Evaluation of Individual Initiatives ......................................................................... 153

Research Objective 2 Literature Review: Evaluation of Innovation Fund Portfolio ................................... 165

Paulson (2007) Portfolio Evaluation Tool Methodology and Adaptation. ................................................. 166

Paulson et al. (2007) Portfolio Evaluation Tool Application ...................................................................... 167

Appendix 8: Interviews Findings .................................................................................................................... 192


__________________________________________________________________________________________

6

Interview Observations Relating to Objective 1 ............................................................................................. 192



Significant Insights into Secondary Benefits List ........................................................................................ 195

Additional Secondary Benefits Not Listed.................................................................................................. 197

Appendix 9: Additional Information on Organisational Learning and Lean Start-up ..................................... 198

Lean Start-up and UNICEF Innovation Fund .............................................................................................. 198

Innovation Accounting and UNICEF Innovation Fund ................................................................................ 199

Appendix 10: Participant Information Sheet .................................................................................................. 202


__________________________________________________________________________________________

7

List of Figures

Figure 1: UNICEF Innovation Fund Stage of Investment…18

Figure 2: UNICEF Vision & Mission…19

Figure 3: Definitions Sector/Arena/Industry/Market/Strategic Outcomes…19

Figure 4: 12 Year Revenue Trended by Source (2004-15)…20

Figure 5: 12 Year Expenditure Trended by Region (2004-15)…20

Figure 6: 2015 Direct Programme Expenditure by Strategic Outcome…21

Figure 7: UNICEF Principles for Digital Development…22

Figure 8: UNICEF Innovation Ecosystem…23

Figure 9: UNICEF Innovation Fund Alignment with the Wider Organisations Vision…24

Figure 10: Typical Succession of Finance Sources over Business Cycle…26

Figure 11: Value of Real Options Orientated Investment Strategies…27

Figure 12: Conceptual model on the relationship between strategic orientation, project portfolio management, and success…31

Figure 13: The Innovation Portfolio Management (IPM) Process - Requires Detailed Coordination and Dynamic Decision Making…32

Figure 14: The Innovation Ambition Matrix…32

Figure 15: Direct and Indirect UN Innovation Imperatives…34

Figure 16: Research Design Summary…36

Figure 17: Research Phases and Timeline…37

Figure 18: Process of Planning and Conducting the Interviews…40

Figure19: Proposed triangulation of single case study…43

Figure 20: Triangulation…44

Figure 21: UNICEF Office of Innovation: 3 Horizons…46

Figure 22: UNICEF Office of Innovation in the Context of Uncertainty…47

Figure 23: The Ashridge Mission Model…48

Figure 24: The UNICEF Innovation Fund Mission Model…49

Figure 25: The Strategic Staircase Framework…50

Figure 26: Iterative Strategic Maturity Framework…51


__________________________________________________________________________________________

8

Figure 27: UNICEF Innovation Fund Strategic Staircase and Strategic Maturity Level…52

Figure 28: Traditional Development Funding vs. UNICEF Innovation Fund VC Model…53

Figure 29: UNICEF Innovation Fund Corporate Venturing Model…54

Figure 30: UNICEF Innovation Fund Business Model Canvas…56

Figure 31: UNICEF Evaluation MAP…57

Figure 32: Evaluation Timing…60

Figure 33: Care for Children Value Proposition Fit…63

Figure 34: Paulson et al. (2007) framework aggregated results for Care for Children…64

Figure 35: Learning Framework for Care for Children…66

Figure 36: MomConnect – How does it work…68

Figure 37: MomConnect Value Proposition Fit…71

Figure 38: Paulson et al. (2007) framework aggregated results for MomConnect…72

Figure 39: Learning Framework for MomConnect…74

Figure 40: UNICEF Innovation Fund Portfolio Dashboard; three portfolio areas…76

Figure 41: UNICEF Innovation Fund portfolio of investments in applicants from UNICEF country offices and private sector companies…77

Figure 42: Using rubrics in evaluation design and in making evaluative judgments…78

Figure 43: The deployment of 'leading' and 'outcome' indicators across the four scaling stages…80

Figure 44: Section 2 Potential impact on the market…83

Figure 45: Section 5 Scalability…84

Figure 46: Portfolio of initiatives performance profile aggregated across sections 1 to 10…85

Figure 47: Aggregated initiative performance profile (sections 1 - 10)…86

Figure 48: UNICEF Innovation Fund Portfolio health profile…87

Figure 49: Section 11 Portfolio health…88

Figure 50: UNICEF’s Innovation Risk Matrix – Diversity of its profile…90

Figure 51: Interview Results: Secondary Benefits Experienced by Interviewees…93

Figure 52: Organisational Learning Interview Observations…94

Figure 53: Build-Measure-Learn Feedback Loop and UNICEF Innovation Fund…95

Figure 54: Making the Organisation Ready for Change Interview Observations…96


__________________________________________________________________________________________

9

Figure 55: Exponential Rate of Change in Technology vs. Intuitive Assumption…97

Figure 56: Inspiring Staff and Attracting New Types of Talent Interview Observations…98

Figure 57: Top 50 Companies Millennials Want to Work For and What they Look for in Employers…99



Figure 60: Closed vs. Open Innovation Models and UNICEF Innovation Fund…101

Figure 61: External Mimicry Interview Observations…102

Figure 62: External Mimicry Time Line from Desk Based Research…103

Figure 63: Creating New Business Models Interview Observations…104

Figure 64: Open Source Business Model Canvas: Red Hat…104

Figure 65: Conceptual Open Source Business Model Canvas for Seeded Start-ups…106

Figure 66: Building Capabilities Interview Observations…107

Figure 67: UNICEF Innovation Fund Plotted on the Open Innovation Methods Matrix…108

Figure 68: Driving Organizational Goodwill Interview Observations…109

Figure 69: Augmented Mission Delivery…109

Figure 70: UNICEF Brand Personality, Existing and New Traits…110

Figure 71: The Revised Technology Adoption Cycle – Crossing the Chasm…115

Figure 72: Strategic Outcome Description and Programme Areas…132

Figure 73: UNICEF Strategic Plan 2014-2017 Overview…133

Figure 74: UNICEF Programme Countries…134

Figure 75: UNICEF Office of Innovation Focus, Initiatives & Outcomes Map…136

Figure 76: Sample Office Innovation Projects…137

Figure 77: Principles for Digital Development…138

Figure 78: UNICEF Office of Innovation Strategic Emphasis…139

Figure 79: UNICEF Innovation Fund and Innovation Process…140

Figure 80: UNICEF Office of Innovation and Innovation Fund Focus, Portfolio Areas & Value Creation…143

Figure 81: Comparison of Portfolios Investing in Innovation for International Development…151

Figure 82: Illustration of the sources researched along with some keywords and terms for objective 1 literature review…155


__________________________________________________________________________________________

10

Figure 83: Traditional Evaluation Exemplars…157

Figure 84: Developmental Evaluation Diagnostic Checklist…160

Figure 85: Illustration of the authors researched and some keywords and terms for Objective 2 literature review on portfolio evaluation…165

Figure 86: Section 1: Potential impact on accelerating results…168

Figure 87: Section 2: Potential impact on the market…170

Figure 88: Section 3: Company Growth and Secondary Benefits…172

Figure 89: Section 4: Sustainability…174

Figure 90: Section 5: Scalability…176

Figure 91: Section 6: Innovation…178

Figure 92: Section 7: Initiative’s Impact on the Portfolio…180

Figure 93: Section 8: External Start-Up Team Capabilities and Pace of Progress…182

Figure 94: Section 9: UNICEF Innovation Fund Capabilities for this Initiative…184

Figure 95: Section 10: Financial Indicators of Value Creation…186

Figure 96: UIF Portfolio (Current) including UNICEF Innovation Funds first portfolio of investments in private sector companies…188

Figure 97: Cumulative Gross Metrics vs. Actionable Metrics using Cohort Analysis…203

List of Tables

Table 1: Methods for Assessing Social Performance…28

Table 2: Comparison of the Traditional and Integrative Approach against UNICEF Requirements…29

Table3: Comparison of Traditional vs. Developmental Evaluation…30

Table 4: Secondary Effects and Potential Secondary Benefits of UNICEF Innovation Fund…34

Table 5: Ripple Effects of UNICEF Innovation Unit’s Work…35

Table 6: Literature to be Used to Analyse Findings Related to Potential Secondary Benefits…35

Table 7: Main Sources of Evidence Utilised During Desk Based Research…38

Table 8: Process of Searching Tertiary Literature…38

Table 9: UNICEF Interview List…41

Table 10: External Interviews…42

Table 11: Bias/Error in Six Sources of Evidence linked to Four Types of Triangulation…43


__________________________________________________________________________________________

11

Table 12: Principles for Evaluation aligned with Principles for Digital Development…58

Table 13: Evaluation Do’s and Don’ts picked up during interviews with UNICEF…58

Table 14: Results from applying User Pains Questions to Care for Children…61

Table 15: Results from applying User Pains Questions to Care for Children…62

Table 16: Interview Results from applying Value Proposition to Strength of ‘Fit’…63

Table 17: Care for Children and Paulson et al., (2007) Framework Extract…65

Table 18: Care for Children Learning Objectives and Questions…67

Table 19: Results from applying Beneficiary Jobs Questions to MomConnect - Mothers and Children…19

Table 20: Results from applying Beneficiary Jobs Questions to MomConnect - Department of Health and other Ministries…20

Table 21: Results from applying gain creators Questions to MomConnect - Mother and Children…70

Table 22: Interview Results from applying Value Proposition to Strength of ‘Fit’…71

Table 23: MomConnect and Paulson et al., (2007) Framework Extract…73

Table 24: MomConnect Learning Objectives and Questions…75

Table 25: Summary of strategies that can be used for deriving the importance of evaluative criteria…79

Table 26: Sources of information for the adaptation of Paulson, et al. (2007) evaluative tool…82

Table 27: UIF Portfolio (Current) including UIF's first portfolio of investments in private sector companies…91

Table 28: Typical Extract from Semi Structured Interview Guide: Potential Secondary Benefits Question…92

Table 29: Types and Variants of Open Source Business Models…105

Table 30: Four Levels of Research and Analysis…111

Table 31: Strategic Level Key Findings and Managerial Implications…114

Table 32: Individual Level Key Findings and Managerial Implications…115

Table 33: Portfolio Level Key Findings and Managerial Implications…116

Table 34: Secondary Level Key Findings and Managerial Implications…117

Table 35: UNICEF Innovation Fund Investment Intensity Compared with R&D Investment Intensity of the Top 10 Business Innovators…118

Table 36: UNICEF Innovation Fund: Assessment Criteria for REOI…145

Table 37: UNICEF Innovation Fund: Assessment Criteria for Expressions of Interest…146

Table 38: Common qualities of transformative digital technologies compared to UNICEF Innovation Fund Criteria…154

Table 39: Social Value and Performance Measurement Research Table…156

Table 40: Interview Observations relating to Objective 1 – individual initiative…194


__________________________________________________________________________________________

12

Table 41: Interview Observations relating to Portfolio Evaluation…195

Table 42: Interview Results: Significant Quotes from Key Informants (Part 1 of 2)…198

Table 43: Interview Results: Significant Quotes from Key Informants (Part 2 of 2)…199

Table 44: Additional Secondary Benefits cited by Interviewees not Listed in the Interview Question…200

List of Abbreviations

UIF: UNICEF Innovation Fund

UOI: UNICEF Office of Innovation

WASH: Water Sanitation and Hygiene

CEE: Central and Eastern Europe

CIS: Commonwealth of Independent States

VC: Venture Capital

IDIA: International Development and Innovation Alliance

IPM: Innovation Portfolio Management

PPM: Project Portfolio Management


__________________________________________________________________________________________

13

Statements and Declarations

Confidentiality Statement

The work contained herein has been conducted by the authors as signatories of volunteer service with UNICEF.

Commitments from the authors to undertake obligations with respect to those services include the following

confidentiality clause:

‘to keep confidential any and all unpublished information made known to me during the course of my service

as a volunteer and not to publish any reports or papers on the basis of information obtained during the period

of my service, except with the authorization of UNICEF’

Any publication of all or part of this report can only be done with the authorization of UNICEF.

Anonymity Statement

A diverse group of experts both within UNICEF and external to the organisation have contributed to the data

collected in this report. We recognise their contribution as a collective. The names of individual contributors

are not listed, nor are they attributed to direct quotes or extracts of data to protect all contributors from any

misinterpretation or other unintended consequences.

All the contents of this report represent the analysis and views of the authors only. No content of this report

should be taken to represent the official opinion or position of UNICEF in any way. In particular we also draw

attention to the various evaluation tools developed by the authors and the ratings given to various initiatives.

Those evaluations again represent the analysis and views of the authors only and are not in any way an

evaluation conducted by UNICEF or any of UNICEF’s partners.

Declaration

We, the authors declare that no portion of this final project has been submitted for any other degree at the

University of Manchester or any other institution.

The research presented herein was primarily conducted between June 2016 and January 2017.

Word Count

Word count Limit: 15,000 in main body excluding cover page, table of contents, tables/figures, list of

references and appendices.

Actual word count: 14,572 in main body excluding cover page, table of contents, tables/figures, list of

references and appendices.


__________________________________________________________________________________________

14

Executive Summary

This research project has been conducted with the aim of understanding the value that corporate venturing

can potentially create in international development with a specific focus on the recently launched UNICEF

Innovation Fund. The Fund endeavours to provide an efficient mechanism to invest in open source innovation

initiatives at proof of concept and faces many challenges in its evolutionary phase of development including

demonstrating the value it can bring to the organisation. This study has researched and analysed potential

value creation by the Fund on four levels namely; strategic, individual initiative, portfolio and secondary effects

level in an endeavour to address that challenge.

Case study strategy is applied leveraging multi-method qualitative research approaches in a six month cross

sectional time horizon. Data collection was from multiple sources of evidence including desk based research,

internal documents, semi-structured interviews with key informants and further documents shared by experts.

Semi-structured interviews in particular provided a rich source of primary data collection with 11 key

informants within UNICEF and 6 external industry experts. Data, investigator and theory triangulation was

leveraged to improve the quality of analysis and conclusions.

The findings uncovered positive insights across all four levels of analysis. At strategic level the management of

innovation across three horizons was identified as key to establishing a failure tolerant space within which to

seed early stage initiatives with risk capped smart capital. The mission model uncovered a strong hypothesis

and the corporate venturing model demonstrated that five types of value could be created from the Fund

three of which would be leveraged even when initiatives failed namely; IP stack, community building and

learning. The potential for long term value through impact at scale and growth were also identified from a

small percentage of highly successful initiatives.

Two individual initiatives were evaluated; Care for Children, an open source electronic reporting system for

residential care institutions in Cambodia and MomConnect an open source maternal monitoring system in

South Africa. Both initiatives returned positive indications of potential value creation underpinning the

reasoning behind the Fund. The portfolio of initiatives with $11.2m invested in 31 initiatives to date was

evaluated demonstrating portfolio health and a spectrum of risk consistent with the Funds stage of

development with changes in mix expected as more external start-ups are seeded. Evidence of eight secondary

benefits was also identified. Organisational learning was most prominent closely followed by the creation of

sustainable networks and driving organisational goodwill. The importance of not overlooking such benefits in

order not to under value such venturing approaches was a key finding.

Managerial implications were identified across all four levels chief of which are the challenges of resource

allocation to support the vision and aspirations of management, the need to develop strategies to help

initiatives ‘cross the chasm’ to scale, further research required into time bound portfolio management and the

need to put in place mechanisms to manage and capture organisational learning.


__________________________________________________________________________________________

15

Acknowledgements

First, we would like to thank Dr. Marianna Amatullo, Co-Founder and Vice President of Designmatters, the

award-winning social impact department of the ArtCenter College of Design, in Pasadena, California. An

ethnographic case study by Dr Amatullo titled, ‘Innovation by Design at UNICEF’ offered initial insights into the

organisation's pioneering approaches. Following contact from the authors, Dr Amatullo's positive engagement

facilitated an introduction with the UNICEF Office of Innovation. Without that introduction this project would

not have been possible.

We would like thank the UNICEF Office of Innovation management for agreeing to work with us and also

making the time of their team available to facilitate a fruitful collaboration, exchange of ideas and in depth

data collection which would not have been possible without their support. Special thanks to our primary

contacts, at UNICEF Ventures for their constant collaboration over six months. The teams’ proactive approach

in contacting a diverse group of relevant experts both inside and outside UNICEF requesting their participation

in interviews was a further significant assistance to in depth data collection which improved the breadth and

quality of our findings. We also thank the Ventures team for their engagement and support during the period

including the sharing of key documents and valuable critical feedback on our interim findings.

We thank all contributors who gave up their time to be interviewed for the project. Experienced and

thoughtful contributions from the UNICEF Office of Innovation were a constant throughout. This included

many diverse perspectives at differencing ends of the UNICEF Office of Innovation spectrum.

Elsewhere within the wider UNICEF organisation we thank contributors from the UNICEF Supply Division,

UNICEF Cambodia Country Office, UNICEF South Africa Country Office, UNICEF Brazil Country Office and the

UNICEF Evaluation Office.

We also thank external experts outside UNICEF who gave up their time to be interviewed. We thank

contributors from Results for Development and the International Development Innovation Alliances (IDIA) for

a broad strategic overview of the subject area. We also thank contributors on the subject area from the USAID

Global Development Lab for comparative information regarding their Development Innovation Ventures

programme which helped provide a benchmark from which to compare the UNICEF Innovation Fund.

We also thank a number of Monitoring and Evaluation Specialists both partnered with UNICEF and those not

involved with the organisation for their time and contributions.

From the University of Manchester, firstly we would like to thank our supervisor, Professor Kate Barker, Senior

Lecturer from the Innovation, Management and Policy, Academic Unit. Kate’s continuous feedback and

guidance throughout this six month project helped the authors through the academic challenges, time

constraints and also encouraged the group to focus on the key findings of such a wide study.


__________________________________________________________________________________________

16

We also thank other faculty members at the Alliance Manchester Business School who, prior to

commencement of the project, provided early feedback on draft proposals, general advise, or simply

encouragement to develop a proposal in the field of social innovation and international development, a road

less travelled for final MBA Projects. Those contributors included Professor Kevin Jagiello, Senior visiting Fellow

and Strategic Management Director for the Global MBA; Dr. Lyndsay Rashman Senior Fellow and MBA Projects

Director; Dr. Alison Ashton, Marketing Course Coordinator; Professor Paul Taafe, Visiting Fellow, Operations

Management Course Coordinator and Dr. Anthony Merna, Risk Management Course Coordinator.

Finally we thank our families and friends for their support and patience, particularly during long periods of

preoccupation with this project over six months and indeed the MBA over two and half years.

The Authors: January, 2018.


__________________________________________________________________________________________

17

1. Introduction

1.1 Problem Statement

With the establishment of its Innovation Unit in 2007, UNICEF formally identified the need to foster

innovation. UNICEF Executive Director, Anthony Lake points out that ‘...new technology and innovations are

making it easier to reach the most deprived…’ outlining the organisation's intention to ‘...re-purpose

technology that the private sector is developing and apply it to our own purposes’ (UNICEF, 2012). The

‘identification and promotion of innovation’ is also established as one of seven implementation strategies in

UNICEF's 2014-2017 Strategic Plan (UNICEF, 2013 & Appendix 1).

Despite UNICEF’s clear commitment to innovation, the scale and complexity of its child rights mission coupled

with the organizational culture of a large international development organization makes rapid

transformational change a major challenge. Amatullo (2015) notes the ‘start-up subculture’ of the UNICEF

Innovation Unit but also presents findings that suggest the wider organization is ‘at zero in terms of

mainstreaming… innovation...’ with references to ‘bureaucratic stasis’, ‘organizational inertia’ and

‘organizational resistance to change’ which might be expected in a large public service institution.

The uncertainty of trying new things at UNICEF where its complex global mission has the most significant

consequences for the lives of its ‘customers’ requires a careful and well thought out strategy. Ramalingam and

Bound (2016) cite challenges facing international development organisation's in identifying ideas with the

largest potential impact and deciding how best to support them at each stage.

UNICEF Ventures has developed a new strategy to overcome some of those challenges by adapting approaches

from the venture capital world to seed external and internal initiatives. Such ‘corporate venturing’ is applied

by tech giants like Intel and Google in the business world (Parnell, 2016). Quasi venture capital approaches

have also been adopted by international development organisations such as USAID and The Gates Foundation

(see Appendix 6). Key lessons learned by those international development organisations include the value of

providing seed capital at proof of concept, the need to identify innovators from developing countries and the

need to integrate science and technology innovation with social and business innovation (Buchsbaum, 2016).

Grand Challenges Canada, also cite the importance of energizing the next generation of entrepreneurs along

with sourcing widely but scaling selectively (Singer, 2016).

It is the premise of the authors that innovation in international development organisation's has the potential

to create as much value as it does for successful business innovators such as Google, Apple or 3M and that

venturing in such contexts could also become as successful as Lilly Ventures, Google Ventures or Intel Capital.

The value created by innovation in businesses can be measured through the commercialisation of products

and services but international development organizations lack ‘the simple elegance of a financial measure’ to

demonstrate returns (Forbes, 1998, noted in Kaplan, 2001). External scrutiny of expenditure not related to

core programmes and the high failure rate of innovation ventures exacerbates the challenge of convincing

donors to invest without clarity of social return.


__________________________________________________________________________________________

18

Despite these challenges, Rockefeller Foundation President, Judith Rodin (2016) argues that the modern day

rate of change does not allow international development organisations ‘...to let innovation take a long,

winding course...’ citing an explosion of new innovative initiatives. The Gates Foundation echoes this view

observing‘... a robust pipeline of innovations or ‘pile up at proof of concept’ (Buchsbaum, 2016).

UNICEF Innovation Fund endeavours to provide an efficient mechanism to invest in open source initiatives at

proof of concept that have the potential to create value for its mission. This project seeks to assess the value

of the Fund and its overall process to the organisation.

Figure 1: UNICEF Innovation Fund Stage of Investment (Ramalingam and Bound, 2016, UNICEF, 2013e and UNICEF, 2016d)

6. Development & Testing

(Iteration Cycle): i. Designing, ii. Prototyping, iii. Testing, iv.

Reflecting& Re-designing.

7. Business Analysis

Financial viability of the concept(s) assessed.

8. Proof of Concept (PoC)

Initial working prototype, MVP or Pilot which demonstrates the concept works.

‘Innovation Pile Up’ (Buchsbaum, 2016)

UNICEF Innovation Fund (Early Stage Seed / Catalytic Investment)

9. Piloting

Final product concept tested to demonstrate value or otherwise to decision makers.

10. Scale

Innovation scaled with significant investment.

Scale Up Finance: Private Sector investment,

grant-funding, UNICEF networks, Global Innovation Centre (GIC), International Development Innovation Alliance (IDIA), UNICEF

Innovation Procurement Acceleration Fund & Global

Innovation Centre

Exploration & Concept Testing, Analysis, Proof of Concept & Piloting

Scale Up

1. Identifying Need

Market research.

2. Idea Generation

Creative process. Large quantity of ideas.

3. Idea Screening

The most promising ideas.

4. Concept Generation

Ideas studied in the field. Multiple product concepts.

5. Concept Selection

Prioritised highest impact potential


__________________________________________________________________________________________

19

1.2 The UNICEF Context

Vision and Mission

UNICEF is an international development organisation with a child rights and welfare mission. The

organisation's vision is ‘a world where the rights of every child are realised’. This vision is cascaded though its

mission which seeks to meet basic needs, advocate for rights and expand the opportunities of children globally

(UNICEF, 2003 and 2015).

Served ‘Market’

For the purposes of this study UNICEF’s served market is defined as the ‘International child rights and welfare

market’. Definitions of sector, arena and industry are illustrated in Figure 3 along with the organisation's

strategic outcomes, regions of operation and mode of implementation.

Figure 3: Definitions Sector/Arena/Industry/Market/Strategic Outcomes (Created by the Authors based on UNICEF, 2013b&c and Meagher, 2015)

Figure 2: UNICEF Vision & Mission (Adapted from UNICEF, 2003, 2015c & Meagher 2015)

Meet Basic Needs

Vision

A world where the rights of every child are realised

Advocate for Rights Expand Opportunities

International Child Rights and Welfare ‘Market’

Asia Middle East & North Africa

Sub-Saharan Africa

Latin America

Central & Eastern Europe (CEE) + Commonwealth of Independent States (CIS)

Health

Outcome

Education

Outcome

HIV/AIDS

Outcome

Child Protection Outcome

Social Inclusion Outcome

WASH Water Hygiene

& Sanitation Outcome

Nutrition

Outcome

International Development & Relief Industry

International Non-profit Arena

Non-profit Sector

Fundraising National

Committees & HQ

Implementation: through individual Country Programmes managed by Country Offices

Presence in over 190 Countries


__________________________________________________________________________________________

20

Revenue and Expenditure

UNICEF's global mission, strong brand, and UN mandate generates yearly revenue in excess of $5 Billion with

approximately 60% contributed by governments and 29% by the private sector.

Figure 4: 12 Year Revenue Trended by Source (2004-15) (Created by the authors based on data from UNICEF Annual Reports 2004-2015 & Meagher, 2016)

Strong positive surplus margin (revenue-expense/revenue) has been achieved for 9 of the last 12 years. Sub-

Saharan Africa, Asia and the MENA region attract the most expenditure based on customer need collectively

accounting for 84.6%.

Figure 5: 12 Year Expenditure Trended by Region (2004-15) (Created by the authors based on data from UNICEF Annual Reports 2004-2015 & Meagher, 2016)

0

1000

2000

3000

4000

5000

6000

04 05 06 07 08 09 10 11 12 13 14 15

Re

ven

ue

in m

illio

ns

$

Year

12 Year Revenue Trended by Source (2004-2015)

Governments

Private sector/NGO

Inter-organisational

Other sources

Total Revenue

Total 2015 Revenue $5.009 Billion

60.4%

29.1%

100%

8.6%

1.9%

0

1000

2000

3000

4000

5000

6000

04 05 06 07 08 09 10 11 12 13 14 15

Exp

en

dit

ure

in m

illio

ns

$

Year

12 Year Expenditure Trended by Region (2004-2015) Sub-Saharan Africa

Asia

Middle East & North Africa

Latin America

Central/Eastern Europe & CIS

Inter-regional

Total Indirect Expenditure

Total Expenditure 100%

Total 2015 Expenditure $5.112 Billion

51.7%

15.6%

17.3%

3.0%

2.8%

2.9%

6.7%


__________________________________________________________________________________________

21

Direct programme expenditure in 2015 reached $4.768 billion with the Health, Education and WASH strategic

outcomes receiving the largest percentage of funds.

Figure 6: 2015 Direct Programme Expenditure by Strategic Outcome (Created by the authors based on data from UNICEF, 2016a)

1.3 The UNICEF Office of Innovation

The UNICEF innovation Unit, founded in 2007, has evolved into the current UNICEF Office of Innovation, a

global interdisciplinary team with the scope of ‘identifying, prototyping and scaling technologies and practices

that strengthen UNICEF's work…’ (UNICEF, 2016b and Meagher, 2016a&b). The Office defines innovation as

‘something new or different that adds value and has a concrete impact’ (UNICEF, 2013d).

The Principals for Digital Development

At the core of UNICEF's innovation work are the Principals for Digital Development created by UNICEF in 2009

and now endorsed by major international development organisation's and UN agencies (UNICEF, 2016f). The

principles are also key criteria of assessment for initiatives applying to the UNICEF Innovation Fund. The nine

Principles are summarised below with further details included in Appendix 2.

Health $1288.7m

27.0%

Education $1000.5m

20.9%

WASH Water Sanitation & Hygene

$868.7m 18.2%

Child Protection $643.6m

13.5%

Nutrition $603.5m

12.6%

Social Inclusion $266.5m

5.6%

HIV/AIDS $107.1m

2.2%

2015 Direct Programme Expenditure by Strategic Outcome

Health

Education

WASH

Child Protection

Nutrition

Social Inclusion

HIV/AIDS

Total 2015 Direct Programme Expenditure $4.768 Billion

Note: Total and breakdown is not an exact match due to rounding


__________________________________________________________________________________________

22

Figure 7: UNICEF Principles for Digital Development (Created by the Authors based on UNICEF, 2014m and Meagher, 2016)

Office of Innovation Structure and Ecosystem

The UNICEF Office of Innovation is structured around three sub-divisions namely; UNICEF Futures, UNICEF

Ventures and UNICEF Scale. UNICEF Futures scans the 2-5 year horizon to evaluate emerging and trending

technologies for synergies of shared value with the private sector. UNICEF Ventures invests in early stage

solutions with the potential to positively impact children in the 0-2 year future. This investment comes from

the UNICEF Innovation Fund, the focus of this project. UNICEF Scale identifies proven solutions with the

potential to be implemented at national and regional scale. The Global Innovation Centre (GIC) provides the

leadership and technical support to deploy and scale those solutions (UNICEF, 2016i).

Collaboration with a network based innovation ecosystem is fundamental to the work of the UNICEF Office of

Innovation as illustrated below. This includes UNICEF Country Offices which provide the link to core

programmes and Innovation labs established by some offices provide a hub for innovation focus. Other UN

agencies, the private sector and academia all contribute to the wider ecosystem.

Innovative Solutions that Add Value for Children

1. Design with the User

2. Understand the Existing Ecosystem

3. Design for Scale

4. Build for Sustainability

5. Be Data Driven

6. Use Open Standards, Open Data, Open Source and Open Innovation

7. Reuse & Improve

8. Do No Harm

9. Be Collaborative


__________________________________________________________________________________________

23

Figure 8: UNICEF Innovation Ecosystem (Amatullo, 2015 - Updated by the Authors to Reflect UNICEF Office of Innovation Structure, 2016)

UNICEF Ventures and UNICEF Innovation Fund

UNICEF Innovation Fund is a pooled funding vehicle launched in 2015 that makes seed investments in early

stage open source technology solutions in programme countries (see Appendix 1) with the potential to create

value for children (UNICEF, 2016d). UNICEF Ventures also provides technical support for the solutions seeded

by the Fund for 12 months (UNICEF, 2016d).

The Fund is currently $11.2m in size with investors including the Governments of Denmark and Finland as well

as the Walt Disney Corporation. The Fund is divided into three portfolio areas, namely, (1) Real-time

Information for Decision Making, (2) Infrastructure and (3) Products for Youth Under 25. The expected returns

are growth of the initiatives, accumulation of IP stack and the establishment of a community of open source

problem solvers in programme countries.

Scale GIC

Nairobi & Bangkok

Futures

San Francisco

Ventures

UNICEF Innovation

Fund

New York

Office of Innovation Director

Start-Up

Partnerships

Scale-up


__________________________________________________________________________________________

24

The portfolio areas and expected returns of the Fund are summarised on the right below along with the major

aspects of the organisation's vision, mission and the Office of Innovations evolving focus areas.

UNICEF Vision

UNICEF Mission

UNICEF Innovation

Focus (2014)

How 2014 Focus Areas

were Addressed

UNICEF Office of Innovation Focus (2016)

UNICEF Office of Innovation

Approach to 2016 Focus Areas

UNICEF Innovation Fund Portfolio Areas &

Expected Returns (2016)

A world where the rights of

every child are realised

Meet Basic Needs

Information

Real-time Data:

Two way communication

& better service

delivery

Real-time Information &

Tracking (Information)

Tools that strengthen real-time monitoring,

transparency, & accountability to inform decision making (2-way

communication)

Real-time Information for Decision Making

Expected Returns: Growth IP Stack

Communities

Tracking: Ensuring children receive

lifesaving interventions

Expand Opportunities

Opportunity

Equitable Access:

Giving children an opportunity

to survive & thrive

Access to information

(Opportunity)

Emerging technologies to

overcome barriers to accessing

information & services

Infrastructure

Expected Returns: Growth IP Stack

Communities

Infrastructure (Opportunity)

Emerging technologies to

overcome barriers to accessing

information & services

Physical Products

(Opportunity)

Procurement of fit for purpose value for money supplies that impact programmes

& emergencies

Advocate for Rights

Choice

Youth Engagement: Bring young

people closer to government

Youth Empowerment

(Choice)

Innovations that empower young

people to become social change agents,

solve problems, connect & scale

solutions

Products for Youth under 25

Expected Returns:

Growth IP Stack

Communities

Figure 9: UNICEF Innovation Fund Alignment with the Wider Organisation's Vision (Created by the authors based on UNICEF, 2003, 2013b&c, 2014b, 2015c and 2016a-h)


__________________________________________________________________________________________

25

1.4 Research Question and Objectives

Addressing the above mentioned problem statement and specific context, this study aims to contribute to the

understanding of the potential value that venturing can create in international development with a specific

focus on the UNICEF Innovation Fund. The project has been broken down into four core sections to adequately

dissect and analyse the numerous aspects that contribute to value creation.

Firstly, the group aimed to evaluate the venturing strategy to assess the strategic reasoning behind the

approach, covered under the general research question. The research and analysis will then drill down to

uncover further insights and findings under three specific objectives covering value created from individual

initiatives, the portfolio of initiatives and potential secondary benefits.

The research question and objectives are as follows:

General Research Question

How can a large established international development organisation create value from a ventures approach to

innovation such as that which is applied in UNICEF Innovation Fund?

Research Objectives

The research objectives intend to drill down to undercover three further specific levels of analysis as follows:

1. Research suitable tools and methods to evaluate the investment potential of individual initiatives

seeded by the UNICEF Innovation Fund and apply those approaches to two initiatives to demonstrate

potential value to UNICEF's mission.

2. Research suitable frameworks to evaluate the UNICEF Innovation Fund as a portfolio and demonstrate

potential value of the Fund considering the majority of initiatives are expected to fail with only a

limited few expected to be successful.

3. Investigate the potential secondary benefits that UNICEF Innovation Fund can bring to the organisation

through doing new things, taking risks and accepting failure so as to capture learning benefits and

reduce under-valuation of its innovation initiatives and processes.


__________________________________________________________________________________________

26

2. Literature Review

2.1 Approach to Literature Review

Due to the cross cutting nature of the project bringing business thinking to international development,

applicable literature is drawn from a broad spectrum of sources and disciplines. This breadth and diversity is

necessary to make sense of the complexity that the UNICEF Innovation Fund embraces, as any one stream of

literature is insufficient to satisfy all aspects of the study. Subject areas such as innovation, strategic

management, corporate venturing, business models, evaluation, portfolio management, social innovation and

marketing are all leveraged to this end. It is not the objective of this literature review to map all publications

for every concept but rather identify and apply the most relevant literature to the context. The review is

structured around the four core sections of this project consisting of the general research question and three

research objectives and includes both academic and practitioner sources. The key literature is applied and

expanded further in the findings section of this report.

2.2 General Research Question: The Venturing Approach to Value Creation

Venture Capital and Corporate Venturing

Venture Capital defined by the National Venture Capital Association as ‘a segment of the private equity

industry which focuses on investing in new companies with high growth potential and accompanying high risk’

(NVCA, 2016 – Yearbook) offers UNICEF a proven ‘vehicle to try new things’.

Start Up Maturity

Public Debt / Equity

Long Term Debt

Late Stage PE

Early Stage VC

Business Angels

Self-Financed

Figure 10: Typical Succession of Finance Sources over Business Cycle (Khurshed et al., 2015)

Companies as diverse as Google, BMW, and General Mills are adopting corporate venturing business models,

putting money into start-ups as they look for alternative means to learn and innovate (Lerner, 2013).

International development organisations such as USAID and the Gates Foundation have also applied similar

methods (Ramalingam and Bound, 2016).

Venturing Business Models

Osterwalder and Pigneur (2010) define a business model as ‘the rationale of how an organisation creates,

delivers, and captures value’. McGrath (2010) argues that the business model concept is suitable in ‘highly

uncertain, complex and fast-moving environments’ such as UNICEF's where ‘strategies are as much about

insight, rapid experimentation and evolutionary learning as they are about traditional skills and planning’.


__________________________________________________________________________________________

27

McGrath (2010) asserts that ‘most experiments with new technologies fail, but without such failures the

eventual new victorious design would not have had a chance’ and notes the need for financial tools that make

sense with a strategy of experimentation rather than traditional deterministic concepts such as net present

value. Real-options reasoning approaches (Putan and McMillan, 2004) keeping initial investments small until

concepts are proven and cutting those that fail early are noted by McGrath (2010) as having success in

innovative organisation's such as Amazon. The benefit of options orientated investment strategies are

illustrated in Figure 9 below.

Black Hole Investment Strategies Options Orientated Investment Strategies

Figure 11: Value of Real Options Orientated Investment Strategies (McGrath, 2010)

Further literature related to corporate venturing is explored in the findings section of this report where

specific theories are used to make sense of the findings related to the UNICEF Innovation Fund. Several

strategic management frameworks are also leveraged to analyse the venturing strategy including; Baghai, et al.

(2000) Three Horizons of Growth; Campbell et al. (1990) and Campbell and Yeung (1991), Ashridge Mission

Model; Hay and Williamson (1991) Strategic Staircase; Sniukas et al. (2016) Strategic Maturity Framework;

Seelos and Mair (2016) ‘When Innovation Goes Wrong’ and Osterwalder and Pigneur (2010) Business Model

Canvas.

Positive

Unlimited Downside Risk

Downside Risk Contained at any Given Time

Negative

Positive

Negative

Cumulative Cash Flow

Cumulative Cash Flow

Time Time


__________________________________________________________________________________________

28

2.3 Research Objective 1: Investment Potential of Individual Initiatives

Although an abundance of evaluation methods exist for evaluating individual initiatives, our research is refined

by defining clearly the type of initiatives the UNICEF Innovation Fund will be executing and matched with

relevant evaluation approaches. Given the fund will continue to seed in multiple start-ups with open source

technology solutions across various countries, it expects to obtain real-time information and tracking, growth

of the initiatives seeded and accumulation of an IP stack. A fast rate of change is anticipated over 12-18

months during which the fund will support the entrepreneurial function of the start-ups that exist to focus on

creating change. The suitability any framework to assess value created by the fund must therefore be capable

of dealing with changeable and uncertain environments.

Developmental evaluation is not a set of methods, tools, or techniques. There is not a set of steps to follow.

There is no recipe, formula, or standardized procedures. Rather, developmental evaluation is a way of

approaching the challenge of evaluating social innovation through guiding principles (Patton, 2015) operates in

dynamic situations (Gamble, 2008) with uncertain outcomes, facilitates real-time feedback, a learning

framework (Dozois, 2010), and is arguably more effective for evaluating the UNICEF Innovation Funds

initiatives (Melwani et al., 2016).

Traditional evaluation on the other hand focuses on accountability as it uses a prescriptive method that relies

on available data to act as evidence through using randomized control trials (RCT’s) to prove or disprove

whether an intervention works. Traditional evaluation is arguably too premature for UNICEF Innovation Fund

with innovations that are still under development (Patton, 2011; ILO, 2014) and that they will only produce

weak results that lead to poor decision making, inefficient use of resources and thus perceived as unrelated to

the nature of the organisation (Conger, 2009).

Ebrahim and Rangin (2010) alternative classification of the methods for assessing social performance has been

adapted below which summarizes traditional evaluation, developmental evaluation and the integrative

evaluation.

Traditional Approach

Logic

Models

Expected Return and

Cost Effectiveness

Experimental Methods

Systematic Reviews

Participatory and

Relationship-Based

Developmental Evaluation & Integrative Approach

Examples of Users

1. United States (USAID) 2. W.K. Kellogg Foundation 3. Innovation Network

1. SROI Network (SROI) 2. Robin hood foundation - (BCR) 3. Millennium Challenge Corporation -

1. Abdul Latif Jameel PovertyAction Lab, MIT 3. Innovations for Poverty Action

1. Campbell Collaboration 2. International Initiative for Impact Evaluation (3ie)

1. Center for Effective Philanthropy 2. U.N. (FAO) 3. World Bank

1. ActionAid 2. Bridgespan Group 3. McKinsey & Co. Social Sector Office. 4. UNICEF

Table 1: Methods for Assessing Social Performance (Adapted from Ebrahim & Rangin 2010)


__________________________________________________________________________________________

29

A detailed literature review of evaluation approaches is included in Appendix 7 as well as a completed

questionnaire that indicates the suitability of Developmental Evaluation for UNICEF Innovation Fund.

Comparing the different methods against the characteristics of the fund we can see that integrative

frameworks are the most suitable (Table 2).

Proposed Approach to Evaluate Individual Initiatives

A further comparison of integrative Developmental Evaluation (DE) and traditional evaluation methods are

shown in Table 3. Given the possible drawbacks of applying traditional evaluation approaches to innovative

initiatives they are deemed unsuitable and an integrative approach is thus considered more appropriate. The

integrative approach together with the Paulson et al., (2007) radical innovation evaluative framework was thus

selected for application herein with the latter being further elaborated under the Objective 2 literature review.

Approach Criteria

Traditional Approaches Integrative Approaches

Logic Models

Expected Return and Cost Effectiveness

Randomized Controlled Trial

Developmental evaluation (Gamble, 2008; Patton, 2011; Dozois, 2010)

Paulson et al Framework (2007)

Framework is suitable in uncertain environments where uncertainty is high (applicable for Radical Innovation and Moon-shoots)

No No No Yes Yes

Framework is suitable in environments where ambiguity is present

No No No Yes Yes

Framework is suitable where failure of initiatives is expected and an acceptable part of the process.

No No No Yes Yes

Framework is suitable as part of a wider evaluation of a portfolio of options.

Yes Yes Yes Yes Yes

Framework is suitable where the majority of returns and value is only likely to be demonstrated in the long term.

Yes Yes No Yes Yes

Framework is suitable when changes and iterations are required regularly No No No Yes Yes

Framework facilitate on-going cycles of feedback and able to evaluate real-time data

No No Yes Yes Yes

Framework that can support the evaluation of the monetary term Yes Yes Yes Yes Yes

Table 2: Comparison of the Traditional and Integrative Approach against UNICEF Requirements (Adapted by the authors from Ebrahim and Rangin, 2014)


__________________________________________________________________________________________

30

Comparative Criteria

Traditional Evaluation Developmental Evaluation

Purpose and Situation

Evaluation Purpose

To test, prove, validate and hold accountable

To support development of innovations and adaption of initiatives into dynamic environments

Situation Static situation, innovation is a proven concept, data is available, goals well defined, variables controllable, predictable

Complex dynamic situation, solution is not 100% certain, multiple pathways, need for exploration and social experimentation

Focus and Target of Evaluation

Driving Force Outcomes driven, systems viewed as context.

Systems change driven, outcomes emergent, dynamic

Governance Top down evaluation (prove theory) Helps innovators navigate the muddled middle where top-down and bottom-up forces intersect and often collide

Modelling and Methods

Modelling approach

Design the evaluation based on linear cause– effect logic: specify inputs to activities/processes, then outputs to outcomes to impacts. Hypothesized, and predicted, then tested.

Design the evaluation using systems thinking to capture and map complex systems dynamics and interdependencies, and track emergent interconnections. Use of pattern detection.

Measurement approach

Measure performance against predetermined goals and SMART outcomes:

Develops measures and tracking mechanisms quickly as outcomes emerge; measures can change during the evaluation as the process unfolds.

Attention to unexpected Consequences

No attention, side-lined. Attention given to the unanticipated and emergent as a fundamental evaluation function.

Methods approach and Philosophy

Rigorously methods- focused: an evaluation is judged by validity and methodological criteria first and foremost. Traditional research and disciplinary standards of quality dominate.

Utilization- focused: methods are chosen in service to developmental use; methods derive from utility and pragmatic considerations.

Roles and Relationships

Ideal evaluator position

Independent. Part of the innovation team, bringing evaluative thinking to the group, supportive of the innovators’ values and vision.

Evaluation Results and Impacts

Reporting Lengthy detailed formal reports, scholarly voice, and passive.

Rapid, real-time feedback, engaged, present voice, active.

Impact on organizational culture

Evaluation often engenders fear of failure

Evaluation encourages learning and long term development.

Approach to Complexity

Approach to uncertainty

Strives for complete certainty and predictability

Expects uncertainty

Professional Qualities

Evaluator attributes

Commitment to rigor; independence; credibility with external authorities and funders; analytical and critical thinking.

Methodological flexibility and adaptability; systems thinking: creative and critical thinking balanced; high tolerance for ambiguity; open and agile.

Table3: Comparison of Traditional vs. Developmental Evaluation (Patton, 2011)


__________________________________________________________________________________________

31

2.4 Research Objective 2: Evaluation of UNICEF Innovation Fund Portfolio

Project portfolios are vehicles that link strategy formulation to strategy implementation (Danila, 1989; Cooper

et al., 1999; Meskendahl, 2010; Eggers, 2012 and Unger et al., 2012). This literature review highlights key

considerations for portfolio evaluation including the collective risk return profile, strategic alignment,

governance/decision making, project selection/prioritisation, portfolio balance, resource allocation, rate of

attrition, interdependencies, time horizon and the key challenge of balancing project specific versus

standardised portfolio performance metrics. Meskendahl's (2010) conceptual model illustrates the relationship

between strategic orientation, project portfolio management, and success.

Figure 12: Conceptual model on the relationship between strategic orientation, project portfolio

management, and success (Meskendahl, 2010)

The process of innovation operates in a dynamic and uncertain environment, whereas valuation metrics are a

snapshot in time making the development of valuation metrics for innovation in an international development

context very challenging. As illustrated below, the Innovation Portfolio Management (IPM) process

necessitates dynamic decision making and detailed coordination between the evaluation of individual projects

and the corporate strategy formulation process (Cooper, Edgett and Kleinschmidt, 1999; Anderson and

Joglekar, 2005). Decision making is challenging due to the uncertainty of information and estimates (Cooper et

al., 1999; 2002; Nagji and Tuff, 2012) and requires repeated revision of the active projects in a portfolio where

resource, market or technological interdependencies between projects can be a major source of competitive

advantage (Mathews, 2010).


__________________________________________________________________________________________

32

Figure 13: The Innovation Portfolio Management (IPM) Process - Requires Detailed Coordination and Dynamic Decision Making (Derived from Cooper et al., 1999; 2002; Anderson & Joglekar, 2005).

Cooper et al. (2001); Cooper (2009, 2014); Day (2007); Nagji and Tuff (2012); Ohr and McFarthing (2013);

Holtzman (2014) among others, stress that an organisation must have a clear vision of where their projects fall

on the spectrum of risk in order to balance their innovation portfolio. Nagji and Tuff (2012) explain that

innovation sits on a spectrum from incremental to transformational (see below) and posit that

transformational initiatives where uncertainty is greatest require unfamiliar approaches. The literature

stresses the need to develop capabilities such as the ability to define the organisation's risk appetite, to

tolerate and embrace risk and to ensure continuous learning. Prominent Project Portfolio Management (PPM)

authors explain that projects must be prioritised by their importance and strategic contribution.

Figure 14: The Innovation Ambition Matrix (Nagji and Tuff, 2012)

Core

Optimising existing products for existing customers

Adjacent

Expanding from existing business to ‘new to the company business’

Transformational Developing breakthroughs and inventing things for markets that do not currently exist

Use existing products and assets

Add incremental products and assets

Develop new products and assets

Serve existing markets and customers

Enter adjacent markets, serve adjacent customers

Create new markets, target new customer needs

How to Win

Where to Play


__________________________________________________________________________________________

33

Cooper et al. (2009) and Cooper and Edgett (2014) argue that in order to guide the portfolio and allocate

resources, top firms achieve best results not by relying on traditional financial criteria, but by applying multiple

methods such as strategic buckets, scorecards, productivity index and scoring models. Mathews (2010)

advocates keeping ‘early-phase evaluations cheap and fast' by way of qualitative and rough order of

magnitude (ROM) estimates. Holtzman (2014) and Canales, et al. (2014) explain that traditional evaluation

approaches can bias decision makers against experimentation and transformational innovations. Patton (2011)

stresses that in uncertain and changing environments, methods and tools can emerge as the situation calls for

them and ‘questioning is the ultimate method’. Figure 85 in Appendix 7 illustrates the authors which were

researched as part of this review along with some keywords.

Proposed Framework to evaluate the UNICEF Innovation Fund Portfolio of Initiatives

The innovation portfolio evaluation tool Paulson et al. (2007), a scoring model based on evaluative questions,

developed in collaboration with companies such as 3M, IBM, GE, Xerox, Intel & Shell was identified as an

appropriate framework to assess the relative value of initiatives within the context of the entire innovation

portfolio and will thus be applied herein. The framework can be applied at any point in time and can also be

used to track changes in the relative value of the portfolio over time.

Proposed Framework to manage portfolio balance

The innovation risk matrix developed by Day (2007) employs a unique scoring system to illustrate a portfolio’s

balance in terms of the risk associated with investments oriented towards innovation and will be used in

addition to the portfolio evaluation tool outlined above. The adaptation and application of both tools is

explained in the Findings and Analysis Section under Objective 2.


__________________________________________________________________________________________

34

2.5 Research Objective 3: Investigation into Potential Secondary Benefits

Research into literature to help investigate the potential secondary benefits of the UNICEF Innovation Fund,

started with a framework for internal innovation by Canales et al. (2014), the authors of which include the

UNICEF Ventures and Futures Leads. Canales et al. (2014) argue that innovation often delivers benefits from

secondary effects and present a ‘secondary effects scorecard’.

The effects were treated as potential secondary benefits in this project and two more were added as identified

by UNICEF Ventures in initial exploratory interviews. The eight potential secondary benefits outlined below

were used in interview questions to investigate if evidence of their existence was present in relation to the

UNICEF Innovation Fund.

No. Canales, et al.(2014) Secondary Effects No. 8 Potential Secondary Benefits Investigated

1 Organisational learning (often through failure)

2 Making the organisation ready for change

1 Are you able to track second and even third-level effects? How?

2 Are you attracting new types of talent? 3 Inspiring staff and attracting new types of talent

3 Do relationships remain after you go, creating a sustainable network?

4 Creating a sustainable network of long term relationships even after completion

4 Is there external mimicry? 5 External mimicry

5 Did the project create new business models? 6 Creating new business models

6 Did it result in new team/staff capabilities? 7 Building capabilities

7 Did it drive organizational goodwill (i.e. new funders/allies)?

8 Driving organisation goodwill (i.e. new funders/allies)

Table 4: Secondary Effects and Potential Secondary Benefits of UNICEF Innovation Fund (Adapted from Canales, et al., 2014 and initial exploratory interviews with UNICEF Ventures)

Bloom and Falkner (2015) identify direct and indirect innovation as dual imperatives of innovation in UN

organisations. The indirect innovation imperative in the form of Organisational Change will be assessed

through Canales et al. (2014) secondary effects scorecard.

Figure 15: Direct and Indirect UN Innovation Imperatives (Bloom and Falkner, 2015)

Organisational Change

Community Enabling

Humanitarian & Development Needs Met

Innovation Lab

Organisational Change Indirect

Imperative

Community Enabling

Direct Imperative


__________________________________________________________________________________________

35

Dean (2015), a thesis titled ‘Building innovation capability in an intergovernmental aid organisation: A case

study of UNICEF's Innovation Unit’ asked ‘What are the ripple effects of the Unit’s work, if any, in other parts

of the UNICEF system?’ and identified five different ripple effects within UNICEF and two broader effects

outside the organisation as outlined in Table 2.

No. Ripple Effects of UNICEF Innovation Unit’s Work

1 A creation of space and a language to talk about innovation

2 A spread and adoption of the Unit’s innovation principles in other units

3 Innovation is now a priority in UNICEF's Strategic Plan

4 Country Offices are now generating ideas

5 Field, country and headquarter offices are now exploring and integrating technology in their programs

6 Other UN agencies seek advice from the UNICEF Innovation Unit (broader spill over effect)

7 The innovation principles are adopted by other aid agencies (broader spill over effect).

Table 5: Ripple Effects of UNICEF Innovation Unit’s Work (Dean, 2015)

Further literature was researched under each of the eight potential secondary benefits to help make sense of

the findings uncovered in relation to UNICEF Innovation Fund as summarised in Table 3. This literature will be

leveraged further in the findings section of this report.

No. Potential Secondary Benefits Investigated Description of Main Literature to Analyse Findings

1 Organisational learning (often through failure)

Lerner (2013) Corporate Venturing Ries (2011) Lean Start-up Sniukas et al. (2016) Ventures & Strategic Innovation

2 Making the organisation ready for change Kurzweil (2001) Law of Accelerating Returns Christianson (1997) Disruptive Technology Moore (1991) Crossing the Chasm

3 Inspiring staff and attracting new types of talent

Agrawal (2014) How Companies can Attract Talent Deloitte (2016) Millennial Survey Strategy& PwC (2016) Most Innovative Companies

4 Creating a sustainable network of long term relationships even after completion

Chesbrough (2003) Open Innovation McInerney (2014) Non-profit open source software

5 External mimicry Bloom and Faulkner (2016) UN Innovation lessons UN Agency Innovation Websites

6 Creating new business models Osterwalder and Pigneur (2010) Business Model Canvas McGrath (2010) Discovery Driven Planning Meier (2016) Successful Open Source Business Models RedHat.com

7 Building capabilities Harwood (2016) Open Innovation Method Matrix Christianson (1997) Disruptive Technology Lerner (2013) Corporate Venturing Teece (2007) Dynamic Capabilities Chesbrough & Rosenbloom (2002) Business Model Value Nagji & Tuff (2012) Managing Innovation Portfolio

8 Driving organisation goodwill (i.e. new funders/allies)

Aaker (1997) Dimensions of Brand Personality Keller (1993) Customer Brand Equity

Table 6: Literature to be Used to Analyse Findings Related to Potential Secondary Benefits


__________________________________________________________________________________________

36

3. Research Design and Methodology

3.1 Research Design

The predominant research philosophy of the group is pragmatism, where the most important determinant of

epistemology, ontology and axiology adopted is the research question (Saunders et al., 2008 and Meagher, et

al., 2015). The research primarily followed a deductive approach with multiple theories identified at the outset

for testing in the context. As the research evolved and significant data was collected, abductive reasoning was

also leveraged enabling logical inference from new discoveries to theory in a methodologically ordered way

(Reichertz, 2004, noted by Flick, 2013).

Case study strategy is applied, defined by Yin (2014) as an empirical inquiry of a contemporary phenomenon

within its real-life context, when the boundaries between phenomenon and context are not clearly evident,

using multiple sources of evidence’. Multi-method qualitative research was chosen based on an early

identification of data available. The project was conducted in a cross sectional time horizon for six months

representing an early stage snap shot of a rapidly evolving new approach within UNICEF. Data collection was

from multiple sources of evidence throughout the period including desk based research, initial exploratory

interviews, internal documents, semi-structured interviews with key informants and further documents that

key informants shared.

The research design is summarised as follows.

Figure 16: Research Design Summary (Adapted from Saunders, et al., 2009)

Data Collection from Multiple Sources of Evidence

Cross-sectional Time Horizon

Multi-method Qualitative Research

Case Study Strategy

Deductive Approach

Pragmatic Research Philosophy


__________________________________________________________________________________________

37

3.2 Research Phases and Timeline

The case study research was conducted in seven phases as illustrated below.

Phase 1 Phase 2 Phase 3 Phase 4 Phase 5 Phase 6 Phase 7

Research

Questions

and

Objectives

Desk Based

Research

Literature

Review

Internal

UNICEF

Innovation

Fund

Document

Review

Semi-

structured

Interviews

&

Additional

Documents

Data

Analysis and

Triangulation

Conclusions

and

Implications

Proposed

research

questions

and

objectives

presented

by the

authors and

reviewed

by UNICEF

Ventures.

Developed

iteratively.

Desk based

research fully

exhausted

online

sources of

evidence

across the

four core

levels of

research

(research

question and

three

objectives).

Literature

review

related to

the four core

levels of

research to

uncover

relevant

theory and

applicable

frameworks.

Internal

documents

describing

processes,

procedures

and strategy

related to

UNICEF

Innovation

Fund

reviewed in

detail.

17 Semi-

structured

interviews

with key

informants to

gain deep

insights and

understanding

across the

four levels of

research.

Data,

investigator

and theory

triangulation

leveraged in

the analysis

of findings to

make sense

of the key

points of

conflict and

convergence.

Managerial

implications

and

conclusions

drafted

iteratively

overlapping

with phase 6

to improve

the rigour

and quality

of the final

report.

Figure 17: Research Phases and Timeline (Adapted from Soy, 1997 & Melwani et al., 2016)

Theory / What is known

Secondary Data Collection and Analysis

Primary Data Collection

and Analysis

Case Study Strategy


__________________________________________________________________________________________

38

Phase 1: Defining the Research Question and Objectives

The research question and objectives were developed through multiple iterations and collaboration with

UNICEF Ventures in July 2016. It was identified that ‘the challenge lies in defining the value of doing new

things, of taking risks and accepting failure’ and establishing ‘what is the contribution of each exploration, but

also, in the case of the Fund, what is the value of a portfolio (where the majority have failed, and some few

‘succeed’). What is the value of the process that leads us to these few successes?’ (UNICEF Interviews, 2016).

This feedback along with multiple iterations lead to the final research question and objectives contained

herein.

Phase 2: Desk Based Research

The requirements for transparency within international organisation's generally (Baines et al., 2011) and

UNICEF's Principle 6, ‘open data’ (UNICEF, 2016e) resulted in extensive online databases being at the disposal

of the research team. As a result a plethora of evidence was uncovered through desk based research which

helped frame the research accurately in the early stage. A summary of the main sources of evidence used in

desk based is as follows.

Website Data and Evidence Provided

http://www.unicef.org/

UNICEF website used to research organisational background and context such as mission, strategic objectives, annual reports and core programmes.

http://www.unicef.org/innovation/

UNICEF Office of Innovation website used to research its strategy, principals, focus areas, structure and processes.

http://www.UNICEF'stories.org/ UNICEF Office of Innovation blog providing numerous stories and reports about innovation at UNICEF used to gain further in depth insights.

http://www.unicefinnovationfund.org UNICEF Innovation Fund website providing public information about the Fund for potential investors, investees and partners including details of how to apply to the fund and high level criteria for assessing applications.

Table 7 : Main Sources of Evidence Utilised During Desk Based Research

Phase 3: Literature Review

The literature review presented earlier in this report first started with a wider search. A range of databases

were used to ensure extensive coverage of the literature (Saunders et al., 2012). The authors followed the

search process as follows.

No. Search Steps

1 Key words to match controlled index language

2 Search appropriate database sources

3 Note precise details including search strings used

4 Note the full reference of each relevant literature identified

Table 8: Process of Searching Tertiary Literature (Adapted from Saunders et al., 2012, noted in Meagher, et al., 2015)

http://www.unicef.org/

http://www.unicef.org/innovation/

http://www.unicefstories.org/

http://www.unicefinnovationfund.org/


__________________________________________________________________________________________

39

Phase 4: Internal UNICEF Innovation Fund Documents

Internal unpublished documents related to UNICEF Innovation Fund were made available allowing deeper

insights to be gained. A list of documents provided is outlined in Appendix 5. The internal documents provided

detailed evidence of the developing processes and procedures being applied in the management of the

UNICEF Innovation Fund. Early stage access to the Funds stakeholder website was granted wherein dashboards

of real time data related to initiatives seeded are displayed and aggregated. Detailed initiative lists with

descriptions, targets, contacts and links to further information were also provided along with the venture

capital criteria used in the Request for Expressions of Interest and Request for Proposal processes.

The aforementioned documents served as important data sources when preparing for further probing through

semi-structured interviews and in many cases provided support to triangulate and make sense of other

findings ‘…to seek convergence and corroboration through the use of different data sources…’ (Bowen, 2009).

Platt, J. (1981) notes that documents by their nature carry certain inferences associated with its form and the

audience it seeks to address, hence the importance of other forms of evidence to facilitate triangulation.

Phase 5: Semi-structured Interviews

Semi-structured interviews with key informants added considerable depth and value as a primary source of

evidence. The interview strategy focused on the one-to-one semi-structured Interviews, which were based on

predesigned open-ended questions enabling the achievement of greater quality, depth and detail as well as

the ability to uncover subtleties in attitudes (Stokes and Bergin, 2006). Semi-structured interviews also

facilitated an open and unhurried process in a non-judgmental and confidential environment (Baumbusch,

2010).

The authors were mindful that the more structured the interview the more reliable and generalizable the

findings (Cohen et al., 2007). The interview guides were therefore structured in a consistent manner around

the four core research levels namely; strategic, individual, portfolio and secondary. Although most questions

remained common to ensure corroboration, questions were also tweaked to place more emphasis on some

and less on others depending on the background of the participant, their respective area of expertise and the

focus of the research question being addressed.

Interviews were conducted remotely via teleconferencing with the research team sharing an invitation letter

with UNICEF Ventures who circulated this among the targeted participants. Prior to interviews the research

group shared an interview guide and Participant Information Sheet (see Appendix 10) ensuring all potential

participants were aware of the purpose of the interviews, the confidential nature and their right to withdraw

at any time. Each interview lasted between 60 to 90 minutes and all interviews were recorded to form

accurate transcripts. The interview process is summarised below.


__________________________________________________________________________________________

40

Figure 18: Process of Planning and Conducting the Interviews (Meagher, et al., 2015)

The researchers conducted 17 semi structured interviews in total with both participants from within UNICEF’s

organisation as well as external organisation's operating in the international development industry. UNICEF

Ventures shortlisted key informants based on knowledge of the UNICEF Innovation Fund and related expertise.

Key informants had a very high level of expertise and experience resulting in very rich insights being gained.

The diversity of the group is also worth noting with interviewees not only drawn from the UNICEF Office of

Innovation but also from recipients of the UNICEF Innovation Fund in Cambodia, South Africa and Brazil each

with their respective expertise in core areas of UNICEF's mission as well as experts in procurement, hardware

and evaluation.

Interview planning

Inform / Confirm

participants

Interview Preparation

Conducting Interview

with Recording

Interview Follow-up

+ Additional Documents

Shared

Interview review,

transcrips and

analysis

Data and Researcher

Triangulation

Follow-up Interview

as necessary

Clarification of specific points & additional

documents collected

All Interviews recorded and where possible attended by

2 researchers

Request / Confirm Schedule + Share Project

Overview & Scope

Clarification of specific points & additional

documents collected

Prepare interview guide. Taylor based on

knowledge and expertise

Selection of Interviewees with relevant knowledge

and expertise


__________________________________________________________________________________________

41

No. Department / Organisation Location No. of Interviews

1 UNICEF Office of Innovation

UNICEF Headquarters, NYC 2

2 UNICEF Ventures

UNICEF Headquarters, NYC 1 + periodic feedback

3 UNICEF Ventures UNICEF Headquarters, NYC 2 Group interviews +

Continuous coordination

and feedback

4 UNICEF Futures

San Francisco 1

5 UNICEF Scale and Global

Innovation Centre

Bangkok 1

6 UNICEF Ventures


7 New York Supply Centre (Supply

Division)


8 UNICEF Cambodia Country Office

Cambodia 2

9 UNICEF South Africa Country

Office

South Africa 2

10 UNICEF Brazil Country Office

Brazil 1

11 UNICEF Office of Evaluation

UNICEF Headquarters, NYC 1 Group Interview

Table 9: UNICEF Interview List

Relevant key informants outside UNICEF were also identified with expertise in international development,

venture capital methods as applied in the industry as well as innovation evaluation experts and consultants.

The impartial input of external experts contributed further key insights which helped balance and triangulate

the internal UNICEF contributions.


__________________________________________________________________________________________

42

No. Department / Organisation Location No. of Interviews

12 Results for Development and

International Development

Innovation Alliance

London 1

13 Results for Development

Washington, DC 1

14 Panoply Digital Ltd.

London 1

15 USAID: U.S. Global Development

Lab

Washington, DC 1

16 Utilization-Focused Evaluation

Saint Paul, Minnesota 1

17 The Knowledge Institute Te Awamutu, New Zealand

1

Table 10: External Interview List

In addition to insights gained directly from the informants through answers to semi structured interview

questions, many interviewees also shared key documents which added an extra layer of depth to the evidence

and findings. These additional documents include both publicly available information as well as information

considered confidential. No content relating to confidential information is reproduced in this paper without

first having obtained express permission of the author. To protect the anonymity of participants contributions

quotations included in this report are not referenced to individual participants or the organisation/department

they represent. However by sharing the identity of the group of key informants as a collective in this section

the authors wish to illustrate the level of seniority, expertise and diversity the group possess.

Phase 6 and 7: Triangulation Analysis and Conclusions

Figure 19 illustrates the triangulation which has been applied at each layer of research in an endeavour to

improve the quality of conclusions (Saunders et al., 2008).


__________________________________________________________________________________________

43

Figure 19: Proposed triangulation of single case study (Adapted from Saunders et al., Lewis 2009, and Meagher, et al., 2015)

To further emphasise the efforts to triangulate findings the sources of evidence have been tabulated below to

illustrate Denzin's (1978) four types of triangulation.

Source of Evidence

Type of Bias / Error (Yin, 2014)

Suitable Type of Triangulation (Denzin, 1978)

Approach adopted by researchers

(1) Desk Based Research (2) Internal Documents (4)Interviewee Documents

a. Biased selectivity, if collection is incomplete b. Reporting bias unknown bias of any given documents author

(i) Data Triangulation: Multiple data sources to reduce selectivity bias. (ii) Investigator (Observation) Triangulation: Multiple investigators to reduce reporting bias (iii) Theory Triangulation: Multiple theories used to make sense of research documents further reducing reporting bias.

(i) Cross examination of internal documents, desk based research and interviewee documents. (ii) Cross checking interpretations and recordings between investigators. (iii) Multiple theories from a plethora of disciplines to analyse findings.

(3) Interviews

a. Bias due to poorly articulated questions b. Response bias c. Inaccurate recall d. Reflexivity – interviewee gives what interviewer wants to hear

(i) Data Triangulation: Multiple respondents proving multiple perspectives and improving question design. Multiple interviewees diluting response bias and reflexivity. Recordings eliminating inaccuracy of recall. (ii) Investigator Triangulation: Improved analysis multiple investigators reducing reflexivity. (iii) Theory Triangulation: Multiple perspectives through a range of academic and practitioner frameworks.

(i) Wide range and diversity of key informants from UNICEF Office of Innovation, country office recipients, and expert internally and externally cross checked with external experts. (ii) Improved analysis by 5 investigators. (iii) Multiple theories from a plethora of disciplines.

Table 11: Bias/Error in Six Sources of Evidence linked to Four Types of Triangulation (Adapted from Yin 2014; Denzin 1978 (also noted in Meagher, et al., 2015 and Melwani, et al., 2016))

Data Triangulation

Multiple Sources of Evidence

Desk Based Research

Internal Documents

Interviews

Additional Documents

from Interviewees

Investigator Triangulation 5 Group Members all contributing to analysis of findings

Theory Triangulation Multiple Academic

& Practitioner Frameworks

Combined

Approach

Abductive


__________________________________________________________________________________________

44

Whilst precise convergence of evidence from multiple sources is not always possible the multiple sources

served as a constant benchmark and countercheck as the study progressed. Detailed understanding, concise

findings and clear conclusions could not be drawn if such triangulation did not form a constant part of the

study.

Figure 20: Triangulation (Adapted from Yin, 2014)

(1) Desk Based Research

(websites) & Archival Records

(2) Internal Documents

(3) Semi-Structured interviews

Data Triangulation +

Theory Triangulation (Multiple Academic & Practitioner

Frameworks) +

Investigator Triangulation =

Improved Quality of Findings, Analysis and Conclusions

(4) Additional Documents Shared by

Interviewees


__________________________________________________________________________________________

45

4. Findings and Analysis

4.1 General Research Question: The Venturing Approach to Value Creation

UNICEF Office of Innovation Strategy and Three Horizons

Drucker (1954) described strategy as ‘a sense of direction around which to improvise’, a relevant definition for

the iterative innovation venturing strategies of today and one which received favourable feedback in UNICEF

interviews with the ambiguous, uncertain and dynamic nature of innovation ventures noted by key informants.

The UNICEF Office of Innovation has developed a structure that ‘scans the horizons’ (UNICEF Interviews, 2016).

This strategy evolved through experience since 2007, when the then UNICEF Innovation Unit had the vision to

identify that investment in mobile technology through its RapidSMS initiatives (see Appendix 2, Figure 75 and

76 for overview of ‘typical’ initiatives) had great potential to create value in future horizons. The subsequent

successful development of its flagship open source platform, RapidPro has served as a benchmark for current

strategies.

Baghai et al., (1999) research into 30 industry pacesetters achieving higher growth, presented the separation

of growth possibilities into the Three Horizons of Growth Framework. The application of Horizon 1 ‘core

business’, Horizon 2 ‘emerging businesses and capabilities’ and Horizon 3 ‘create viable options’ can be seen in

many industry leaders today. The UNICEF Office of Innovation apply a similar approach through it’s Scale,

Ventures and Futures structure inspired by its own experiences (UNICEF Interviews, 2016).

The three innovation horizons are essential in order to provide a space for new forward looking approaches

such as venturing through the UNICEF Innovation Fund. Multiple small bets in early stage initiatives many of

which may fail is highly appropriate in Horizon 2 but would not be suitable in the Scale Horizon 1 which

requires heavier investment with failure of initiatives intergarted into core programmes having greater

adverse consequences. The Horizon 2 space allows the UNICEF Innovation Fund to trial emerging solutions

with finincial, reputational and programme related risks capped before the Scale phase.

The three horizons collectively contribute to an ‘innovation floodplain’ which is not linear or unidirectional but

rather involves multiple cycles, feedback loops, iterative meandering initiatives which evolve in different ways

within the UNICEF Office of Innovation ecosystem (UNICEF Interviews, 2016).


__________________________________________________________________________________________

46

UNICEF Office of Innovation: 3 Horizons A structure that ‘scans the horizons’ ‘How do we make this organisation change?’

‘UNICEF Innovation Floodplain’ ‘Not a pipeline... A pipeline implies a set of constraints and linearity... and a directionality’

Figure 21: UNICEF Office of Innovation: 3 Horizons (Created by Authors based on UNICEF (2016b&i), Baghai et al. (1999) and (UNICEF Interviews, 2016)

Horizon 1 (Current)

Extend Core Business

UNICEF Scale

The focus of UNICEF Scale is to identify and scale up proven innovative solutions, expanding their application by UNICEF and others to multiple countries and regions and millions of people.

The Global Innovation Centre (GIC) is at the core of this work. Large scale investment. Hard work, organisationally pushing it through.

Direct involvement why UNICEF core programmes and country offices.

Innovation + Scale = Impact (Seelos and Mair, 2016)

Initiatives currently in Scale up: U-Report; RapidPro, Digischool (MobiStation); Digital Kiosks; RasberryPi4L; Internet of Good Things.

Horizon 2 (1-2 Years)

Build Emerging Businesses & Capabilities

UNICEF Ventures

UNICEF Ventures invests in early stage solutions that show great potential to positively impact children in the one-to-two-year future.

This investment comes from the recently

launched UNICEF Innovation Fund, which is a pooled funding vehicle that invests in open source technology solutions coming from start-ups based in UNICEF’s programme countries.

The Venture Fund allows managed risk taking. Multiple small bets in the form of smart capital investments to grow companies that can contribute to UNICEFs mission. Experience tells UNICEF it takes about three years for a start-up to work but Ventures targets an acceleration of this timeline within two years. After two years it is hoped to see something that is a viable product or not.

Horizon 3 (2-5 Years)

Create Viable Options

UNICEF Futures

UNICEF Futures looks at the two-to-five-year horizon to evaluate emerging and trending technologies to see how UNICEF can work with the private sector to find shared value in this future space – for companies to do better business while improving access to essential services for children.

Future investment areas include: transportation and mobility; identity; learning; mobile financial services and digital currencies; and wearable and sensor technology.

Tech. companies, big money in the world to be made. China is investing $60 billion in Africa over the next 3 years and UNICEF is investing $17 billion in the world. Overlap between $100 billion businesses and billion person needs. Conceptualisation & understanding landscape


__________________________________________________________________________________________

47

UNICEF Ventures in the Context of Uncertainty

The Three Horizons and Uncertainty

The UNICEF Office of Innovation plots UNICEF Ventures and UNICEF Innovation Fund, highest on a scale of

technical and utility uncertainty within the organisation. The Fund is also considered high on the scale of

market and organisational uncertainty just behind UNICEF Futures. UNICEF Scale is the interface between the

UNICEF Office of Innovation and UNICEF's core programmes as illustrated below.

Figure 22: UNICEF Office of Innovation in the Context of Uncertainty (UNICEF Internal Documents, 2016; adapted by the authors based on UNICEF Interviews, 2016)

UNICEF Ventures Response to Uncertainty

The experience from development of the UNICEF Office of Innovations flagship open source initiative,

RapidPro, is the benchmark used for the potential failure rate of such initiatives. As noted by key informants:

‘we made 15 investments in various RapidSMS versions in various countries, out of that we got the knowledge

that we needed to invest... and RapidPro came out of that... Whether that’s 100x I don’t know but we were

able to get substantial benefit from it’ (UNICEF Interviews, 2016).

Adapt Existing Tech to UNICEF Needs $$

Point of Care Testing, Solar-Lights for Latrines

Scale Up New Platforms $$$

U-Report

Core Programmes $$$$$

Health,

Education, WASH,

Child Protection, Nutrition,

Social Inclusion, HIV/AIDS,

Emergency

New Needs

Climate Change,

Migration, Urbanisation,

Job Skills

Scaling Up Nutrition

(SUN)

Multiple Indicator Cluster Surveys

(MICS)

Scale

0-1 Year Horizon

$$$

Ventures Innovation Fund

1-2 Year Horizon $

Futures

2-5 Year Horizon

$

Office of Innovation

Market and Organisational Uncertainty

Tech

nic

al a

nd

Uti

lity

Un

cert

ain

ty


__________________________________________________________________________________________

48

Letts et al. (1997) observe that Venture Capital investors expect only 2 in 10 investments to produce a big

payoff. Jurveston noted by Jagiello (2016) notes Venture Capital ratios which indicate that for every 5,000

ideas; 10 will be seeded, 5 of which are likely to fail, 3 are likely to survive, 1 is likely to return twice the

investment and 1 should produce exceptional returns 50-100 times the investment.

The experience of the UNICEF Office of Innovation with RapidPro and the ratios noted by venture capital

practitioners suggest that the UNICEF Innovation Fund approach to investing, making multiple small bets in

initiatives that have satisfied venture capital assessment criteria is a prudent response to the levels of

uncertainty faced. Table 37 in Appendix 3 compares UNICEF Innovation Fund scoring criteria with that of a

‘typical’ venture capital firm and finds a high level of consistency. Appendix 6 compares UNICEF Innovation

Fund with leading international development venture funds administered by USAID and the Gates Foundation

and also finds a high level of consistency, with UNICEF Innovation Funds relatively small size and limitation to

only seed $100,000, being the most considerable differences.

The UNICEF Innovation Fund Mission Model

Campbell and Yeung (1991) cite the importance of a clearly defined, strong mission. Whilst the wider

organisation's mission is clearly defined it is intended to use the aforementioned model to assess the mission

of UNICEF Ventures and more specifically the UNICEF Innovation Fund. Campbell and Yeung (1991), following a

2 year research project, defined mission as a combination of four elements namely; purpose, strategy, values

and behaviours/standards arguing that a strong mission exists when the four elements reinforce each other.

The Ashridge Mission Model

Purpose Why UNICEF Innovation Fund Exists

Strategy Distinctive Competence (Rational and Commercial Left Brain Logic)

Values What UNICEF Office of Innovation Believes In

(Emotional Right Brain Logic)

Behaviour / Standards The policies and behaviour patterns that underpin the distinctive competence and value system

Figure 23: The Ashridge Mission Model (Campbell and Yeung, 1991)

The Mission Model of UNICEF Ventures, based on desk based research and interviews, is illustrated below.


__________________________________________________________________________________________

49

Vision

Strategy

Behaviour / Standards / Alignment

Values

Purpose

Leadership

Higher Ideal: A world where the rights of every child are realised (Wider organisation's Vision)

Purpose: Clear Hypothesis: ‘In order to solve the biggest problems in the world we need to invest in places that have those problems where you are most likely to have

creativity and solutions that can connect those problem solvers to each other’. The success of UNICEF Innovation Fund will be determined by products that work and the

establishment of communities around them that have built them. The communities that built them have tried their hardest and we can connect them together and it is all open source and

interoperable. If one does not work we learn from it.

Vision: Long Term success after 5 years: An ability to take an equity stake in

companies with several funds in the range of $10-20m. Multiple portfolios of tech. managed by different people in different ways. Private equity portfolio

buying companies, converting them to create value for children. If you are trying to solve the worlds big problems you do not need to stick with one approach and if you mix these approaches of finance, technology, development and social sector you actually have a chance to solve some of these big problems. An evolution of

UNICEF Innovation Fund is how we should invest in development in general.

Values: Risk taking, flexible, modern, data driven. Principals for Digital Development: 1. Design with the User: Context-appropriate solutions informed by user needs, user engagement,

incremental and iterative development, enhance existing and plan for org. adaptation, useful to most

marginalized; 2. Understand the Ecosystem: Participate in networks and communities of like-minded

practitioners, align to existing technological, legal and regulatory policies; 3. Design for Scale: mitigate

dependencies that might limit scale, systems approach considering implications beyond project, Replicable and customisable, demonstrate impact before scale, analyse tech. choices through lens of

national and regional scale, partnerships from beginning; 4. Build for sustainability: Sustainability

planning from the start including long term financial health, utilize and invest in local communities and developers and help catalyse their growth, engage local governments to ensure integration into national

strategy; 5. Be Data Driven: Design projects so impacts can be measured at discrete milestones,

evaluate innovative solutions and areas where there are gaps in data and evidence, use real-time information to monitor and inform management decision at all levels, leverage data as a by-product of

user actions/transactions; 6 Use Open Data, Open Standards, Open Source, Open Innovation: Adopt and expand open standards, Document APIs (Application Programming Interfaces) where use by a larger community is possible, Invest in software as a public good, Open source default with code made

available in public repositories and supported through developer communities; 7. Reuse and Improve: Use, modify and extend existing tools, platforms and frameworks, Develop in modular ways favouring

approaches that are interoperable; 8. Address Privacy & Security: Mitigate risks to security of users,

Privacy of personally identifiable information, Equity and fairness and protect the best interests of end-

users; 9. Be Collaborative: Engage diverse expertise across disciplines and industries, Work across

sector silos to create coordinated holistic approaches, Document, work, results, processes and best practices and share them, Publish material under Creative Commons license by default.

Strategy: UNICEF Ventures invests in early stage solutions that show great

potential to positively impact children in the one-to-two-year future. This

investment comes from UNICEF Innovation Fund, which is a pooled funding vehicle that invests in open source technology solutions coming from start-

ups based in UNICEF’s programme countries. UNICEF Innovation Fund is

making multiple small bets ($100,000 over 12 months), in initiatives after proof of concept stage. The majority of initiatives expected to fail but allow UNICEF

accelerate learning around what does and does not work. If venture capital ratios hold true in the development context a very small number of highly

successful initiatives may achieve Growth and Impact at scale in the long term that make all the failures worthwhile delivering actual results for children. Also

each failure will contribute to IP stack so every initiative will build this asset base

or public goods. Open source Communities of problem solvers are also built.

Behaviour / Standards (Organisational Alignment): Structure: ‘To be an amoeba. To always be changing form quickly enough so that we do not get caught in an organisational structure which will make it unable to make change. Opposite to bureaucracy. ‘To always keep moving and to change the shape of our team so we do not ever get caught by the DNA of

the organisation, by the antibodies which are built to resist change’. Keep moving and to keep shifting and changing so that we are current with the times and that we maintain enough of the feel of UNICEF that we can relate what we are doing to the organisation but that we never get too sure of ourselves, too certain of ourselves, too comfortable with ourselves or too worried

about our jobs so that we are worried about taking risk. Systems: Real-time data dashboards and website to help monitor, evaluate and provide a basis for adaptation and support.

Staff: Limited in the launch phase. Guided by Ventures Leadership. Business management and open source experts currently managing the fund but increased technical, open source expertise and managerial support planned to be recruited within 6-9 months. Skills: Organisational competencies and capabilities in innovation built up since 2007.

Figure 24: The UNICEF Innovation Fund Mission Model (Created by the authors from Campbell and Yeung, 1991; Jagiello, 2015 and UNICEF Interviews, 2016)


__________________________________________________________________________________________

50

The ‘higher ideal’ of the wider organisation is included in the model to give a full picture and facilitate an

overview of strategic alignment. Interviews uncovered a clear hypothesis in the purpose of the Fund which

considers that ‘in order to solve the biggest problems in the world we need to invest in places that have those

problems… connect those problem solvers to each other… open source and interoperable… [and] if one does

not work we learn from it’ (UNICEF Interviews, 2016). Vision also uncovered in interviews has been added to

the original model as a link between purpose and strategy (Jagiello, 2015) with 5 year success communicated

as having multiple funds between $10-20m and investing in multiple portfolios of tech. The ventures strategy

is clearly defined as pooled investment in early stage solutions with potential to create value in the one to two

year horizon and values are rooted in the Principals for Digital Development presented earlier (Figure 7 and

Appendix 2). The most interesting insight was uncovered under the Behaviour/Standards part of the model,

with the structure referred to as an ever changing ‘amoeba’ in order not to become caught in a bureaucratic

structure unable to make change (UNICEF Interviews, 2016). This is seen as a proactive response to the

‘organisational stasis’ identified by Amatullo (2015).

The UNICEF Innovation Fund Strategic Staircase

Hay and Williamson (1991) strategic staircase framework developed with major British and European

companies enables managers to break strategic agendas into bite-sized pieces working backwards from a

defined mission, guiding the selection of priorities and resources while providing a tool to communicate

strategy as illustrated below.

Figure 25: The Strategic Staircase Framework (Hay and Williamson, 1991)

Whilst UNICEF Ventures clearly communicated its mission during interviews it primarily adopts an iterative

approach to strategy development in line with Drukers (1954) ‘sense of direction around which to improvise’.

Mission Year t

Time

Expectation Capabilities to be acquired by Year t-2

Strategies which must be underway by Year t-4

Now


__________________________________________________________________________________________

51

Sniukas et al. (2016) suggests that strategic innovation goes through three phases when building new growth

models including: (1) Inception, where the initial idea of strategy is developed, (2) Evolution where the

business model is reworked, fine-tuned and adapted whilst being operated and (3) Diffusion when the strategy

works well and the focus shifts from designing to scaling the model. The ‘core business design thinking

principle’ underlying this process is ‘active iteration’, the process of ‘learning through active experience’

(Sniukas et al., 2016).

The UNICEF Innovation Fund, almost 18 months since inception in 2015 (UNICEF, 2015d) is currently in the

evolution phase. The inception, evolution and diffusion process follows an iterative cycle of act, learn and

design as illustrated below (Sniukas et al. 2016). The knowledge gained through such iterations facilitates

adaptation and the continuous pivoting of strategy in an evolving process leading to maturity of the model

(Sniukas et al., 2016).

Figure 26: Iterative Strategic Maturity Framework (Sniukas et al., 2016)

The Fund is evolving with the first 5 seed investments in external start-ups launched in November 2016.

Evolution into a pure ‘pooled funding’ portfolio is being achieved with future investor contracts to stipulate

the pooled condition (UNICEF Internal Documents, 2016).

The milestones and targets along the way to vision attainment are illustrated below which assumes both the

visionary staircase and iterative emergent strategy which will continue to change, adapt and evolve. Additional

resources will clearly be required as UNICEF Innovation Fund passes from step to step on the strategic

staircase which has been identified below based on interview findings (UNICEF Interviews, 2016).

Time

Strategic Maturity

Level

Design

Act

Learn

Design

Act Learn

Design

Act Learn

Inception

Evolution (UNICEF Innovation Fund)

Diffusion


__________________________________________________________________________________________

52

5 Years Vision Attainment

2 Years An ability to take an equity stake in companies. Several Funds in the range of $10-20m. Portfolios of tech. managed by different people in different ways. Private equity portfolio buying companies & converting them to create value for UNICEF. Buy companies, make them public goods. 'Mix approaches of finance, technology, development and social sector… you actually have a chance to solve some of these big problems.’

1 Year Secure multiple rounds of further investment from governments, companies, foundations, etc. Launch further Venture Funds to invest in other areas that have great potential to positively impact children. Additional Human Resources.

9 Months Second round of investments in the Fund. Results from the first round of investment will be used to make the case for further investment in the Fund.

Vision Attainment

&

Strategic Maturity

Level

6 Months Mid 2017 Demonstrate value creation to investors. Gartner or others could value the code base and IP stack created by 30 companies and circa 10 common source codes that most are working on. Or evaluate worth to investors by engaging them directly. Company growth may also be used. Broadly are we creating good products, good communities of problem solvers, are there a lot of people working on things together, are they working in different environments on the same set of challenges and those are all quantifiable.

2 Months 20-30 Investments Early 2017. Early to mid-2017 UNICEF Innovation Fund consists of a ‘bubbling pool of Tech. and IP.’ Hire additional human resources to manage that pool including experts in open source software that can help manage the developers as well as business management personnel to support business model development.

Current Position

Second Round of call for proposals (Requests for Expressions of Interest REOI) by End of 2016 (second batch of external investments).

Launch 2015

2.1m invested. Launched first 5 external private sector initiatives signing contracts in November 2016.

Launched internally to Country Offices in early 2015

Figure 27: UNICEF Innovation Fund Strategic Staircase and Strategic Maturity Level (Created by the authors from Hay and Williamson, 1991; Sniukas et al., 2016 and UNICEF Interviews, 2016)

Time

Design

Act Learn

Design

Act Learn

Design

Act Learn


__________________________________________________________________________________________

53

UNICEF Innovation Fund Corporate Venturing Model

Moore (2007) posits that to succeed in the long term a focus on Horizon 2 is essential arguing that innovations

are ‘better off in boot strapped start-ups’ because external investors use fairer standards of measurement.

Lerner (2013) advocates the value of corporate venturing as an accelerator citing Intel’s next generation semi-

conductor chip development leveraging an experimental, failure-tolerant mind-set outside the parent risk

adverse culture a key aspect of the UNICEF Innovation Fund.

Canales, et al. (2014) refer to disruptive innovations being difficult to anticipate in advance with ‘investments...

typically evolv[ing] in stages, starting with small bets and only increasing as key uncertainties are resolved’.

McGrath (2010) advocates ‘discovery driven planning’ in uncertain environments capping risks with the goal of

learning as much as possible at the lowest possible cost’ an approach and goal shared by the UNICEF Ventures.

Black Hole Investment Strategies Options Orientated Investment Strategies

Traditional Development Long Term Programmes Significant Investment / Grant Funding

UNICEF Innovation Fund Multiple Small Bets

Figure 28: Traditional Development Funding vs. UNICEF Innovation Fund VC Model (McGrath, 2010)

The UNICEF Innovation Fund corporate venturing model is illustrated below. The venture capital failure ratios

and the RapidPro benchmark suggest circa 90% of initiatives will fail as illustrated by the red investment line.

However the blue dotted line illustrates that all failures will contribute to: (1) IP Stack, building the open

source asset bank of potential solutions continuously; (2) communities of problem solvers will also grow even

following failures and (3) learning from those failures will continue to accrue.

The solid blue line represents the circa 10% of highly successful initiatives which may go on to (4) create

significant value and impact at scale. This limited percentage of highly successful initiatives may come from

adaptations of previous failed initiatives and may also emerge from the open source IP stack asset base. (5)

Growth of the start-ups in the longer term is also a form of value targeted by the Fund.

The model represents a sophisticated corporate venturing strategy that leverages the potential of multiple

small bets whilst managing risk in the highly uncertain Horizon 2.

Investment Investment

Value Creation Value Creation

Unlimited Downside Risk

Downside Risk Capped at $100,000 for Each Bet


__________________________________________________________________________________________

54

Figure 29: UNICEF Innovation Fund Corporate Venturing Model (Created by the Authors Based on Desk Based Research, UNICEF Interviews and Seelos and Mair, 2016)


__________________________________________________________________________________________

55

UNICEF Innovation Fund Business Model Canvas

Whilst UNICEF Innovation Fund is a vehicle for seeding innovative initiatives it is also an innovative initiative in

its own right. Launched in 2015 it is the first venture fund created by a United Nations organisation and is a

new business model for the organisation.

Osterwalder and Pigneur (2010) define a business model as describing the rationale of how an organisation

creates, delivers, and captures value’. Business modelling is considered a ‘useful approach to figuring out a

strategy’ suggesting ‘experimentation, prototyping and a job that is never quite finished’ in what McGrath

(2010) considers the ‘dynamic perspective’.

The UNICEF Innovation Fund Business Model Canvas has been developed below based on the research

conducted in this study. The value proposition with gain creators, bundle of benefits and returns have been

covered in previous sections of this report. The key partners are noted including UNICEF Country Offices which

is consistent with its traditional model. The open source development community and International

Development Innovation Alliance (IDIA) are new partnerships which continue to evolve and grow. The key

activities are dominated by the venture capital methods of selecting and supporting initiatives with key

resources including the UNICEF Ventures team providing innovation management support with strong

capabilities built in the Office of Innovation since 2007. Customer relationships are built further through the

communities and ecosystem that the UNICEF Innovation Fund intends to foster.

The cost structure leverages a pooled funding mechanism that allows the UNICEF Innovation Fund use the

committed capital of its investors on any initiative that it selects for seed funding following the venture capital

selection process. Technical and business management support from the UNICEF Ventures team which will be

expanded as the Fund grows is also a significant aspect of the cost structure. UNICEF Innovation Fund provides

a path for new revenue streams from investors interested in its new model such as the Walt Disney

Corporation and foundations which is covered further in the secondary benefits section of this report.


__________________________________________________________________________________________

56

Key Partners UNICEF Country Offices Programme countries. UNICEF Networks facilitating co-funding & Scale-up Funding International Development Innovation Alliance (IDIA), UNICEF Innovation Procurement Acceleration Fund, Global Innovation Centre (UNICEF Scale) Private Sector Partners Networks in programme countries which can provide grant funding to scale initiatives Government Partners In programme countries who may provide grant funding to scale initiatives Civil Society & NGOs Via country offices International Organisation's & the wider UN ecosystem Open Source Development Community under further development via the Fund

Key Activities

Seeking expressions of interest for the fund and advertising; Review and selection of EOI & RFP using VC selection criteria; Technical and business management support, management, coordination, support, monitoring & evaluation; Facilitating access to UN infrastructure for start-ups; Facilitating access to scale funders

Value Proposition Gain Creators (Satisfiers) & Pain Relievers (Solutions) Identify and grow open source intellectual property and number of open source solutions to improve children's access to: 1. Information; 2. Opportunity; 3. Choice; Accelerate the generation of evidence to prove open source prototypes can be scaled. Pool and connect open source IP to allow collaboration between companies/institutions developing similar solutions. Bundle of Benefits (Products & Services) Three Portfolio Areas: 1. Products for Youth Under 25: Needs – learning & youth participation; 2. Real-time information for decision making and governance: allows children be understood by decision makers with the power to take action; 3. Infrastructure to increase access to services and information: connectivity, power, finance, sensors and transport. Returns (Value): 1. Growth: Small investments in innovative initiatives has the potential to grow solutions and technologies more rapidly; 2. IP stack: Investors gain access to open-source intellectual property & benefit from UNICEF's infrastructure, expertise & connections; 3. Communities: Open source problem solvers in programme countries + connection with existing platforms e.g. U-Report with 1.8M users in 18 countries.

Customer Relationships Communities Open source community of problem solvers in programme countries which can be connected globally. Examples of previous success: RapidPro international Development Community U-Report Community (in country and across borders) Co-creation Solutions like U-Report with Youth Influencing, Decision Making through SMS/Open Source weekly polls)

Customer Segments Served ‘Market’ Children: International child welfare served ‘market’. UNICEF Country Offices with Open Source solutions consistent with the 3 portfolio areas. Small scale start-ups and academia in UNICEF Programme Countries that have: 1. Early stage open source innovative initiatives consistent with the 3 portfolio areas; 2. Prototype or pilot is complete; 3. Need a small amount of money (circa $100,000) is needed to demonstrate its value to scale funders/investors; 4. Technology generates publicly exposed real-time data. Gains, Pains & Customer Jobs to be Done Gains, Pains and customer jobs can be categorised using Seven Strategic Outcomes (Health; HIV/AIDS; WASH; Nutrition; Education; Child Protection; Social Inclusion) solutions under the 3 portfolio areas.

Key Resources & Capabilities UNICEF Ventures Expertise: Strategy; Design; Tech support; Partnerships; Open Networks; Big & Unconventional Data Sources for humanitarian applications; open source software & content management platforms; Alternative energy; Real-time Data (RapidPro Team); Access to Information; UAVs & other hardware; Blockchain; expanding expertise in new tech. Management: Innovation Fund Board (Executives & Senior Advisers); Management Team; Reference Group (major investors & experts). Real-time monitoring, evaluation & collaboration website.

Channels Communication UNICEF Office of Innovation supported by central marketing & communication channels & Country Office networks. Distribution Through Country Offices supported by UNICEF Supply Division (Copenhagen)

Cost Structure Central Pooled Funding reduce the high costs of fragmentation, projectisation and savings in staff time through consolidation of one report at portfolio level for all investors Recovery rate: 8% of investments in the fund covering administration and overheads. Operational Costs: Fund Management, Technical Assistance, Knowledge Products.

Revenue Streams Investment Streams Pledged or Secured to Date: $11.2m to date with investments by: Government of Denmark, Walt Disney Corporation (Disney/Lucas Films), Government of Finland and foundations. Other Nordic countries and other partners have expressed interest. Co-Funding Leveraged: 1.6m (Dashboard) UNICEF welcomes investments in UNICEF Innovation Fund from: 1. Governments; 2. Private sector; 3. Foundations; 4. Academic institutions; 5. Civil society; 6. International financing institutions; 7. Multilateral organisation's

Figure 30: UNICEF Innovation Fund Business Model Canvas (Created by the Authors based on Desk Based Research, UNICEF Ventures Interviews and Osterwalder and Pigneur, 2010)


__________________________________________________________________________________________

57

4.2 Research Objective 1: Investment Potential of Individual Initiatives

Proposed Framework

A collection of suitable approaches have been combined which include the business canvass (Osterwalder and Pigneur, 2010), Principles for Evaluation with a Learning

Framework (Patton, 2011; Gamble, 2008; Dozois et al., 2010), Value Proposition (Osterwalder and Pigneur, 2010; Strategyzer, 2016) and an integrative approach (Paulson

et al., 2007). A business canvass as shown in Figure 30 is to be summarized for the individual initiative, to give a baseline appreciation of key activities, resources,

capabilities etc. related specific to the initiative. To ensure there is alignment with the evaluation approach and the fund, the Principles for Digital Development are linked

with the Principles for Evaluation which embed a technical learning framework. The canvass, principles and framework set the baseline for the overall evaluation approach.

Figure 31 : UNICEF Evaluation MAP Created by authors based on Desk Based Research, Documents and UNICEF Interviews, 2016. Adapted from Patton, 2015, Paulson et al., 2007, Osterwalder & Pigneur, 2010 and Strategyzer, 2016


__________________________________________________________________________________________

58

Principles for Evaluation

Alignment of the Principles for Evaluation against Principles for Digital Development is demonstrated in Table

12.

No. Principles for Digital Development

Principles for Evaluation Evaluation Principle Description

1 Design with the user Co-creation Evaluation should not be independent from the innovation but be interdependent with it, and with those involved (as part of a team), evaluator provides feedback, facilitates conceptualization of the change process, and both captures and generates perspectives about what is happening, and why.

2 Understand the Ecosystem Developmental Purpose Illuminate, inform and support what is being developed, Identify the nature and patterns of development within the innovation

3 Design for Scale Utilisation focussed Focus on intended use by intended users from beginning to end, ensure utility and actual use

4 Build for Sustainability Developmental Purpose As above

5 Be Data Driven Real time feedback Inform on-going adaption as needs, findings, and insights emerge, rather than only at predetermined times (e.g. quarterly, mid-term or at end of project)

6 Use Open Data Track emerging interconnections

Develops measures and tracking mechanisms quickly as outcomes emerge; measures change during evaluation as process unfolds.

7 Reuse and Improve Utilisation focussed As above

8 Address Privacy & Security Evaluation Rigor Use appropriate methods and maintain confidentiality

9 Be Collaborative Co-creation As above

Table 12: Principles for Evaluation aligned with Principles for Digital Development, adapted from Patton (2011) and UNICEF Principles for Evaluation (2016)

UNICEF Interviews (2016) provided the following guidance of do’s and don’ts when undertaking evaluations.

Table 41 in Appendix 8 provides a synopsis of some of the key findings, shared by participants.

No. Evaluation

‘Don’t’ ‘Do’

1 Concretize initiatives with frameworks such as theories of change

Create agile and flexible approaches that can deal with ambiguity and change

2 Hold findings accountable such that it engenders fear of failure

Capture data in real-time to facilitate learning and decision making

3 Strive for complete certainty and predictability Capture and map change and interdependencies, and track emergent interconnections

4 Use lengthy formal reports emphasising and recording every single detail

Consider the team dynamics and team members backgrounds while setting up the evaluation team

5 Assume a static situation where variables are always controllable and measurable

-

6 Design the evaluation on linear cause-effect logic -

Table 13: Evaluation Do’s and Don’ts (UNICEF Interviews, 2016)


__________________________________________________________________________________________

59

Value Proposition

Interviewees expressed the importance to ‘design with the user’ which corresponds to the first principle of

UNICEF’s Principles for Digital Development. The value proposition provides a method to analyse whether the

product/service meets the requirements of the users and supports beneficiaries. It also supports the value

creation hypothesis (Ries, 2011) by mitigating against value destroying elements, e.g. ironing out any questions

in an application that do not add value to the inspection which could be picked up when analysing ‘customer

pains vs. customer pain relievers’. Similarly, it supports the growth hypothesis (Ries, 2011) by trying to

understand the reason behind a start-up’s (the initiative’s) growth and mitigate against value-destroying kinds

of growth (e.g. growing through mass country-wide adoption by inspectors however producing low quality

recommendations to the ministry). The value proposition can be applied at the beginning or at intervals during

any new initiative or business to track whether the user, (customer or beneficiary) requirements are being met

and if changes are needed to current product/service offerings. For the individual initiatives, it is applied at the

beginning as a forward looking tool to indicate the prospective nature of the initiative.

Timing of Evaluations

This research was conducted at a time when the two individual initiatives were just commencing their

implementation following investment from the Fund. Though testing of the evaluation approach would benefit

from being undertaken later in the process, it was applied to obtain initial feedback.

Timing of evaluations should be undertaken at suitable stages and may not always be effective if strictly

calendar based. Given the nature of innovations which experience periods of success followed quickly by

perceived failure, evaluations undertaken only at predetermined times such as 12 or 18 months may not

maximize opportunities for learning or give a representative view of all activities throughout the duration of

the initiative. Consideration should therefore be given to evaluate at other periods to leverage co-creation,

learning, inform on-going adaption as needs, findings, and insights emerge, rather than only at predetermined

times, as shown in Figure 32.


__________________________________________________________________________________________

60

Figure 32: Evaluation Timing Created by the authors based on Desk Based Research, Documents and UNICEF Interviews, 2016. Adapted from Seelos and Mair, 2016


__________________________________________________________________________________________

61

For the purpose of Research Objective 1, we have selected Care for Children (Cambodia) and MomConnect

(South Africa) which allow the evaluation approach to be tested for initiatives at different stages of investment

by the Fund and applied to initiatives that that have in-depth information via interviews with UNICEF

representatives.

Initiative 1: Care for Children Results

Out of 5,583,000 children under the age of 18 in Cambodia, one in every 350 is currently living in a residential

care institution. The initiative will replace a paper based inspection system that monitor the quality of care

given to children with hand held applications, and reduce the time for inspections and analysis from multiple

days to just 1 day, thus allowing ministries to act quickly upon low scoring institutions.

Value Proposition Analysis

The following value proposition questions were applied to understand the exact ‘user pains’ and were

reviewed with UNICEF Cambodia during interview.

No. User Pains Questions User Pains Answers Discussed in Interview with UNICEF

1 What are the frustrations of the Ministry of

Social Affairs, Veterans and Youth

rehabilitation (MoSVY)?

Unable to quickly reach all institutions because of the

time it takes under the paper based system

Taking too long to transfer the data and analyse the

information on a national level

After inspections it can take months before being able

to compile reports at a national level

2 How are current value propositions

underperforming? What features is the

inspection process missing?

Inspections can take up to 4 days and more time to

enter data. Modern digital reporting is missing.

3 What are the main difficulties and

challenges the system encounters?

Being able to quickly identify low performing

institutions, dealing with the backlog of inspections.

4 Are there upfront investment costs, a steep

learning curve, or other obstacles

preventing adoption?

No current obstacles, investment made into the devices

and application. Initial training and testing was done

with a national team of supervisors. Further testing and

training is to be undertaken with end users to ensure

the application is user friendly by 02/12/16 which may

result in additional adaptations to the application.

Table 14: Results from Applying User Pains Questions to Care for Children (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016 and UNICEF Interviews, 2016)


__________________________________________________________________________________________

62

The following value proposition questions were then applied to understand the exact ‘pain reliever answers’ to ensure that the right solution is being proposed.

No. Pain Relievers Questions Pain Reliever Answers Discussed in Interview with UNICEF

1 Could your products/services produce

savings (time, money or efforts)?

Yes it can save on efforts, human resources. Currently insufficient resources to deal with the number of inspections.

The tool aims to allow inspections twice a year rather than once.

Will enable the ministry to reach all residential institutions to ensure all children are safely taken care of.

Will also identify which children are in need of care and which are not.

2 Could the products/services fix under-

performing solutions? By introducing new

features, better performance, or enhanced

service quality.

The first reason is to ensure children are safely taken care of in an institution. Institutionalisation should only

happen if children really need to be there (e.g. children should not be there if they have biological parents).

Ensure there are good services provided in terms of social workers providing appropriate care, feeding, basic health,

good water, sanitation etc. and all other services that should be provided for a child.

3 Could your products/services put an end

to difficulties and challenges the

institutions encounter? By making things

easier or eliminating obstacles?

The tool will measure also the number of children in the institutions to see if they are reducing.

From institutions through the new system another advantage is that it will be automatically identifying what are the

weaknesses of the institutions taking consideration the minimum standards of care and will automatically generate

recommendations for improvements. This is a new feature. Previously the institutions were not getting feedback

from the inspections. So from their level they can make improvements.

When an inspection has been completed there will be automatically a report sent to the institutions to inform the

responsible person of the institution the outcome and recommendations for improvement.

There will be a follow-up in the next inspection to see if the actions from the previous inspections have been acted

upon.

4 Could your products/services allow the

potential to help the government or

institutions overcome other related

obstacles?

Potential of expanding this technology into other areas of child protection is immense

Table 15 : Results from Applying User Pains Questions to Care for Children (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016 and UNICEF Interviews, 2016)


__________________________________________________________________________________________

63

There is evidence of good fit between the digitized inspection application and the problem to be solved as

shown in Table 16 and Figure 33. As the results of inspections will be available to the public, the degree of

positive and negative feedback from the public is not entirely known at this stage; hence the Product-Market

Fit has been rated lower between 3 and 4 compared with the other ratings.

No. Type of Fit Fit Rating 1-5

1 On Paper: Problem Solution Fit

Evidence that MoSVY and institutions care about the jobs, pains and gains

The digitized inspection system addresses the jobs, pains and gains

5

5

2 In The Market: Product-Market Fit

Evidence the digitized inspection system, pain relievers and gain creators

actually create beneficiary value and gaining traction in the market

3-4

3 Financial: Business Model Fit

Evidence that the value proposition can be financially viable and scalable

5

Ratings 1-5: where 1 = strongly disagree, 5 = strongly agree, 3 = neither agree or disagree

Table 16: Interview Results - Applying Value Proposition Statements to Strength of ‘Fit’ (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016 and UNICEF Interviews, 2016)

Figure 33: Care for Children Value Proposition Fit (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016)

Digital Inspection

1. Reduce reporting to just 1 day

2. Real-time reporting to ministry 3. Immediate improvement in standards

4. Open data kit software and comes with pre-listed questions based on the minimum standards of care

Obstacles to Improving Institutions: 1. Inspection visits 1-4 days plus reporting

3. Results take months to reach ministry decision making body 2. Lack of

resources 4. Annual inspections (needed every 6 months)

Conduct case management of each child

Inspections

Reintegrate 30% of children by 2018

Inspection App

Automatic calculation

Open data kit software

Total days it takes to check 406 institutions will reduce from 1,015 to 406 resulting in increased cost efficiency

Ability of the Ministry to monitor more

children

Significantly minimize the risks children

face

Potential expansion into other areas of child protection

Reduce time to collect and analyse data

Real-time reporting

Improved livelihood of

children

Free up time to reach more

children

Fast response: improving performance of institutions


__________________________________________________________________________________________

64

Paulson et al. (2007) Analysis

Whilst the Paulson et al. (2007) innovation evaluation framework is applied in the portfolio analysis under

Research Objective 2, substantial sections of the framework are designed for individual initiative level and are

thus applied herein. The framework focuses on the impact of initiatives by analysing impact against the

organisation’s mission, sustainability, growth and financial indicators. The ratings in Figure 34 summarize the

aggregated scores.

Figure 34: Paulson et al. (2007) Framework Aggregated Results for Care for Children

Table 17 gives detailed justification of selected sections as follows.


__________________________________________________________________________________________

65

Section 1: Potential impact on Accelerating Results for Beneficiaries/Customers 1-5

1.1 ….significant market opportunity….across geographic locations…potential at global scale. 4

1.2 There are no insurmountable barriers that users will face in accessing the product/service. 3

1.3 …users show improved knowledge/awareness.. empowered to voice opinions. ‘.. we cannot comment on the end users given we are still at the early stage’

0

1.4 Value chain members are enthusiastic concerning the potential applications of this initiative. ‘….questions raised whether the end users will be a little older will they be fully comfortable with using the tablet for the inspections...’

3

Section 3: Initiatives Impact on UIF's Growth, Renewal & Secondary Benefits 1-5

2.1 This initiative is requiring the team to develop/ leverage internal / external networks. ‘So far….internal network... potential for external…’

4

2.2 The initiative contributes significantly to key secondary benefits ‘Support data collection for implementation of government policy….’

4

2.3 We are well positioned to capture sufficient value for our mission. 4

Section 4: Sustainable Enhancement of Initiative/Fund Longevity/Performance 1-5

3.1 No significant risks exist that could undermine the sustainability of the business. ‘Since we did not roll out the initiative yet we will remain cautious of the risks’

3

3.2 An appropriate vision of scale can be developed for alternative locations/contexts/apps. ‘There is an appropriate vision for scaling in other locations’

4

3.3 The program/solution is able to use existing infrastructure and/or facilities. ‘Using existing infra and facilities…don’t have any issues with access to the internet.’

4

3.4 The solution is low in complexity with few programme components. ‘Paper based questions with multiple choice answers transferred across to tablets’

4

Section 9: Firm Capabilities for this Initiative 1-5

4.1 Executive support is strong for this initiative. ‘…endorsed by the minister of social affairs, high level commitment’

4

4.2 We have the resources in this organisation to accomplish the next steps necessary for this initiative.

‘..there will still be some requirements form the ministry to support them to travel to all the residential care institutions. When it comes to the tech, we do not see major challenges as most of the costs are done in the initial phase.’

3

4.3 Cost of gaining the technical progress that is needed is appropriate given ultimate promise for this opportunity.

4

4.4 The chances of achieving the next crucial technical milestones are high. ‘Being able to implement the project as expected’

4

4.5 The initiative is able to leverage against the UNICEF brand, contributing to the initiative’s success.

4

Section 10: Financial Indicators of Value Creation 1-5

5.1 The initiative provides a cost effective solution, when addressing the problem / need. ‘Able to do it much quicker with fewer resources’

4

5.2 The initiative is attracting external attention which encourages additional investment in the fund.

‘Additional funding needed when supporting the ministry to travel to different locations’

4

5.3 The cost estimates are realistic based on local context and competitive prices/rates. 4


Table 17 : Care for Children and Paulson et al. (2007) Framework Extract


__________________________________________________________________________________________

66

Although the results indicate a positive outlook with the majority of scores at 4 one should be cautious of

potential vanity metrics that may unintentionally give a better outlook than what is actually being experienced

(Reis, 2011) and understand the risk of using incorrect metrics, the importance to question the metrics and

pivot to ensure what is being evaluated is current and relevant. For an evaluator to effectively do this they

would need to apply the principles of co-creation, developmental purpose and real-time feedback as noted

earlier.

Technical Learning Framework

A technical learning framework is recommended that is prepared by the group implementing the initiative to

obtain collective agreement on the risks and opportunities or ‘what do we need to pay attention to when care

for children takes off’ and ‘what do we need to learn from the Care for Children initiative’ (Dozois et al., 2010).

Figure 35 is intended to provide structure and direction for learning and focuses the team on producing a set

of learning objectives. Applied during evaluation, Table 18 shows how the learning objectives are further

broken down into specific questions.

Figure 35: Learning Framework for Care for Children, adapted from Dozois et al., (2010) and RADAR (2009)


__________________________________________________________________________________________

67

To facilitate real-time learning, the framework must be a live document that is updated regularly and is not

strictly fixed. Given that Care for Children had not officially commenced, the answers have not been completed

at this stage, and act as an initial set of questions that require development by an evaluator and the group

implementing the initiative.

The technical learning framework is applied at initiative level for the immediate group working to deliver the

initiative as part of the evaluation process. Organizational learning on the other hand which addresses learning

from a wider perspective in order to learn quickly and make decisions regarding future funding is discussed

under Research Objective 3.

Learning Objective

Learning Question

Inspections How effective is the application? What improvements are needed?

We note that the Inspection Application was to enable inspectors to fill up the equivalent of a 20-page paper-based form, which previously took multiple days to complete, in just one day. Can you confirm if these inspections have been completed within 1 day as envisaged?

User Feedback How user friendly is the system? Are there any changes needed to allow the users be more effective?

Implementation Whilst the data flow is much faster, how do the resources at the Ministries cope with the vast amount of increased feedback?

What challenges are there with aggregating data?

Are results of the inspections analysed manually by individual persons and solutions proposed? Or is the software able to aggregate typical problems and automatically assign solutions?

Support With the problems and areas of improvement being identified faster, are the resources able to react equally as quickly and implement improvements straight away?

How can the implementation phase be improved?

Transparency Are there any challenges encountered with being able to guard against undue influences and being able to ensure the vision is not lost with operational issues?

Table 18: Care for Children Learning Objectives and Questions (Dozois et al., 2010)

Care for Children Results Summary

The results provide in depth clarity on the problem to be solved by identifying the ‘key pains’ under the

current paper based inspection system. The product/service offered via the digitised inspection system

indicates that there is a good fit between this and the job to be done and that a strong value proposition

exists. Another assessment should be undertaken after the initiative has been officially launched in January

2017 to see if actual users at a national level are able to utilize the digitized system having been familiar with

the paper based system previously.

Whilst this evaluation is focussed on the digitised inspection application, the success of the ministries ability to

take action for low performing institutions and to verify which children should and should not be kept in the

institutions is dependent on whether the ministry has the resources and capabilities to manage the expected

influx of reports and automatic recommendations from the inspectors.


__________________________________________________________________________________________

68

Initiative 2: MomConnect

MomConnect is an SMS-based maternal reminder program in South Africa to improve access, coverage and

quality of care for pregnant women and their children for one year after the delivery. MomConnect help-desk

allows mothers to send any questions, complains or compliments about the medical service provided which

help improving the health care sector.

Figure 36: MomConnect – How Does it Work? (UNICEF South Africa, 2015)

Value Proposition Analysis

The following value proposition questions were applied to assess if there is a good ‘fit’ between the

customer/beneficiaries problems to be solved and the solution being offered. For MomConnect there are two

main types of beneficiaries. First is the direct beneficiaries ‘Mothers and Children’ and the second beneficiary

group is the Department of Health, ministries and government partners. Accordingly the questions applied to

both different types of beneficiaries and reviewed during the interviewees with UNICEF South Africa team.


__________________________________________________________________________________________

69

The direct customer job questions for mothers and children were as follows.

No. Customer Jobs Trigger Questions Customer Jobs Answers

1 What are the stepping stones that could help your beneficiary achieve this key job?

Improve pregnant women’s feeling about being connected with help-desk that supports them during the pregnancy period and answer their questions and concerns.

Dual communication both through receiving SMS and also through phone calls.

All the official languages are covered within MomConnect help desk.

2 What does your beneficiary need to accomplish that involves interaction

Engagement with the helpline for customer specific questions/concerns.

Investment into helpline response

3 What tasks are your beneficiaries trying to perform? What functional problems are your customers trying to solve?

To gain access to information and help towards challenges during / after their pregnancy.

Allows women to lodge complains whenever they do not find the right treatments or the service at the clinic.

4 What jobs, if completed, would give the user a sense of self-satisfaction?

Address concerns related to nutrition, HIV/AIDs, hypertension, breastfeeding, immunization, mother-child bonding.

Women self-satisfaction through the ability to raise a complaint whenever the service provided is not at the expected level or as per the communicated SMS.

Table 19: Results from Applying Customer Job Questions to MomConnect - Mothers and Children (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016 and UNICEF Interviews, 2016)

The second customer job questions for the Department of Health and other ministries were as follows.

No. Customer Jobs Trigger Questions Customer Jobs Answers

1 What are the stepping stones that could help your beneficiary achieve this key job?

Proactive steps for the government to support and protect maternal and infant mortality

Two ways of communication between the department of health and the pregnant women.

2 What does your beneficiary need to accomplish that involves interaction

Aggregating queries/answers.

Creating database that supports future decisions.

Investment into helpline response.

Provide feedback to the healthcare service provider about the quality of service.

3 What tasks are your beneficiaries trying to perform? What functional problems are your customers trying to solve?

Reduce the maternal and infant mortality rates.

Increase the awareness of the pregnant and new mothers across the country.

Improve the healthcare service.

4 What jobs, if completed, would give the user a sense of self-satisfaction?

Address concerns related to nutrition, HIV/AIDs, Hypertension, breastfeeding, immunization, mother-child bonding.

Support other organizations to leverage MomConnect technical and practical experience.

Utilize the women feedback to improve the quality of healthcare service.

Table 20: Results from Applying Customer Job Questions to MomConnect - Department of Health and other Ministries (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016 and UNICEF Interviews, 2016)


__________________________________________________________________________________________

70

The following value proposition questions were applied to understand the exact ‘user gains’ during the interview stage.

No. Gain Creators Trigger Questions Gain Creators Answers

1 Could your products/services….create

savings that please your customers? In

terms of time, money, and effort?

Potentially create cost efficiencies by reducing the burden on emergency care and transportation.

Save women time through communicating the needed information before visiting the clinic which increases the

awareness about potential treatments.

2 Could your products/services... create

positive social consequences?

Reduce the infant and maternal mortality rates.

Create demand for better quality of health services provided to pregnant women.

Increase the awareness of pregnant women about the expected health service.

Transforming the pregnant women to be an active partner in her medical care and pregnancy instead of passive

partner.

Increase peer-to-peer interaction within the pregnant women community through sharing information and

engagement with other pregnant women.

3 Could your products/services fulfil a desire

customers dream about? By helping them

achieve their aspirations or getting relief

from a hardship?

Helps low and medium income customers low cost and easily accessible digital health system.

Creating a community for pregnant woman where she feels always connected and supported 24/7, she can ask

questions and receive answers, in addition receive information throughout her pregnancy.

Support other organisations within the healthcare sectors for future decisions.

4 Could your products/services produce

positive outcomes matching your

customers’ success and failure criteria? In

terms of better performance or lower cost.

Free of charge service for over 1,000,000 users.

Helping women who are vulnerable to health risks and who have no access to information.

Allowing pregnant mothers to ask questions and get quick answers.

Increase the level of pregnant women awareness.

Table 21: Results from Applying Gain Creator Questions to MomConnect - Mother and Children (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016 and UNICEF Interviews, 2016)


__________________________________________________________________________________________

71

In summary there is evidence of good fit between MomConnect service (including the help-desk) and the

problem to be solved as shown in Table 22 and Figure 37.

Since MomConnect depends on SMS and phone calls as the communication tools which results in cost being

incurred by the beneficiaries and the provider hence, the business model fit has been rated at 4 compared

with the other ratings. It is worth mentioning that with the recent MomConnect Facebook application

launched and the free Wi-Fi availability in some areas in South Africa it is expected that the cost of the service

will be reduced.

No. Type of Fit Fit Rating

1 On Paper: Problem Solution Fit

Evidence that mothers care about the jobs, pains and gains

The help desk addresses the jobs, pains and gains

5

5

2 In The Market: Product-Market Fit

Evidence the sms system, help desk, pain relievers and gain creators actually

create beneficiary value and gaining traction in the market

5

3 Financial: Business Model Fit

Evidence that the value proposition can be financially viable and scalable

4


Table 22: Interview Results - Applying Value Proposition Statements to Strength of ‘Fit’ (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016)

Figure 37: MomConnect Value Proposition Fit (Osterwalder and Pigneur, 2010; Strategyzer.com, 2016)

Digital Healthcare

Personal SMS reminders sent to mothers for testing, check-ups, medication

Detailed advice and appointments for HIV

Free SMS service means each household has access to the service

High Infant and Maternal Mortality Rate High proportion of mortalities are avoidable

Must prevent HIV transmission, mother- baby

Low-middle income households, most vulnerable

Access to information

Helpdesk service engaging with customers to specific queries

Automated code written to aggregate queries for faster, efficient Communi -cation

Nutrition, AIDS/HIV, Hypertension, Breastfeeding….

Deliver targeted personalized messages

Expand area of advice

Potentially create cost efficiencies by reducing the burden on emergency care and transportation

Receiving queries allows helpdesk to understand

the highest concerns

help-desk interaction improves engagement,

promotes new users

Reduce mortality rates

Cater for other high risk issues: gestational diabetes, anaemia and hypertension and is being extended up to a child’s fifth birthday

Free, accessible, easy to use digital

SMS service

reach target

audience


__________________________________________________________________________________________

72

Paulson et al. (2007) Analysis

Whereas the value proposition analysis is more focused on evaluating the value from the customer

perspective, the Paulson et al. (2007) tool is used to evaluate the overall value of the initiative and portfolio.

More details and analysis of the tool is covered in the findings presented under Research Objective 2.

The Paulson et al. (2007) evaluation framework applied to MomConnect and the aggregated scoring

percentage of this analysis is presented in Figure 38 below. In addition, the detailed results and analysis is

presented in Appendix 7.

Figure 38: Paulson et al. (2007) framework aggregated results for MomConnect

The rating of the questions was based on the information and data collected, as well as the conducted

interviews with the UNICEF team. Table 23 presents samples of the questions applied. While the majority of

the questions present high scoring, some questions are rated lower with justification.


__________________________________________________________________________________________

73

Section 2: Potential impact on accelerating results for the International Child Rights and Welfare Market

1-5

2.1 The initiative has significant market opportunity and relevance across geographic locations (with UNICEF programs).

5

2.3 The initiative involves collaboration and engagement within the wider community/ecosystem. 5

Section 3: Initiatives Impact on UIF's Growth, Renewal & Secondary benefits 1-5

3.1 The new initiative will promote a sustainable pipeline of innovation oriented initiatives, either through the area of technology it focuses on, or the external leads developed, etc.

5

3.2 This initiative has engendered large numbers of potential applications and options for potential applications.

5

3.4 & 3.5 This initiative is requiring the team to develop/ leverage internal and external networks. 4

3.6 The initiative contributes significantly to key secondary benefits (e.g. exposure to new market partners/development of ecosystem, organisational learning, new capabilities, new technology frontiers, new business models etc.)

4

3.7 There are no insurmountable barriers that users will face in accessing the product/service. 3

3.8 The Project utilises GIC/UIF expertise. 5

3.11 The initiative makes a viable business case for private sector application. 5

Section 4: Sustainability 1-5

4.2 Initiative sustainability is enhanced given its fit within the portfolio and its linkages with the ecosystem and the system of support provided by the UIF.

4

4.3 Both the initiative team and the company sponsored observe good governance and sound financial practices, particularly with regard to budgeting, reporting and monitoring.

3

4.4 The initiative leverages economies of scale/scope. 5

4.5 Sustainability is supported given the viability and affordability of the product/service being developed.

4

Section 5: Scalability 1-5

5.3 The initiative rates highly in terms of readiness or quick wins e.g. the solution can be tested by users on a limited scale, without requiring complete adoption.

5

5.5 The solution is designed in such a way that its core components could be easily replicated, with contextualized tailoring only needed for limited aspects of the program.

5

5.6 The solution is low in complexity. 2

Section 9: UIF Capabilities for this Initiative 1-5

9.1 Executive support is strong for this initiative. 5

9.2 We have the resources within UIF to accomplish the next steps necessary for this initiative. 5

Section 10: Financial Indicators of Value Creation 1-5

10.1 The initiative provides a cost effective solution, when addressing the problem / need (e.g. reduces the costs of implementation).

5

10.3 The initiative is attracting external attention which encourages additional investment in the fund.

3

10.5 The initiative encourages efficient use of resources (value for money). 5

10.7 Both the initiative team and the company sponsored observe good governance and sound financial practices, particularly with regard to budgeting, reporting and monitoring.

4

10.8 The cost estimates are realistic based on local context and competitive prices/rates. 5


Table 23: MomConnect and Paulson et al. (2007) Framework Extract


__________________________________________________________________________________________

74

Technical Learning Framework

A technical learning framework is also recommended to be prepared by the group working on MomConnect

which will support the continuous improvement of the service and the potential implementation in other

countries. Figure 39 is intended to provide structure and direction for learning and focuses the team on

producing a set of learning objectives.

Figure 39: Learning Framework for MomConnect, adapted from Dozois et al., (2010) and RADAR (2009)

The learning objectives are further analysed and discussed during the interviewees with the UNICEF Innovation

Fund team. Table 24 shows an initial set of questions and points that requires further development and

analysis by the evaluators and the team involved in the implementation while developing the framework.


__________________________________________________________________________________________

75

Learning Objective

Learning Question

User Feedback Can we create more cost effective solution either via Whatsapp, messenger, Facebook that can avoid the SMS and phone calls costs?

Do we need to further streamline the registration process?

Are the current automatic responses to typical questions efficient and have positive feedback?

How the SMS can have embedded links that can provide more information and links to other sources of information?

Utilisation of Data

Can the current help-desk system link the electronic medical records with the provided service?

With the data collected through the Help-desk (such as number of active users, geographic coverage, timing of registration, reason for contacting the help desk, etc.) Can we better utilize this information to provide better quality or other services?

Implementation How can we improve the management function and decision making process?

How can we improve the communication process between all engaged parties?

Are there sufficient resources to help the transition and the continuous improvement of the new helpdesk?

Do we have enough resources to support the implementation of this initiative in other countries?

Collaboration Dynamics

Are there any group collaboration challenges observed between the innovation team and those implementing the application? How can we overcome such challenges?

Integrity and Momentum

Are there any challenges encountered with being able to guard against undue influences and being able to ensure the vision is not lost with operational issues?

Table 24: MomConnect Learning Objectives and Questions (Dozois et al., 2010)

MomConnect Results Summary

The applied frameworks have provided clarity on the ‘job to be done’ for both types of beneficiaries. The

service offered via MomConnect and the Help-desk indicates that there is a good fit and existence of a strong

value proposition.

Since MomConnect passed through different phases of implementation and continuous improvement, it is

essential to document the team experience and combine all types of documentation, information and process

related to this initiative in one full document, which will facilitate the learning process. In addition, the above

mentioned learning framework must be live and updated regularly with all components related to the

implementation, provided service and help-desk which will support the implementation of such initiatives in

other countries.


__________________________________________________________________________________________

76

4.3 Objective 2: Evaluation of UNICEF Innovation Fund Portfolio

This section provides an overview of the findings in relation to the papers second research objective as follows; ‘research suitable frameworks to evaluate the UNICEF

Innovation Fund as a portfolio and demonstrate the potential value of the Fund considering the majority of initiatives are expected to fail’. The UNICEF Innovation Fund

has raised $11.2 million to date with investment focusing on initiatives categorised by three portfolio ‘areas’ as illustrated below.

Figure 40: UNICEF Innovation Fund Portfolio Dashboard; three portfolio areas (UNICEF, 2016j)


__________________________________________________________________________________________

77

Internal UNICEF country office initiatives make up 26 of the 31 investments seeded to date. The remaining 5

investments represent the UNICEF Innovation Funds first private sector initiatives, seeded in November 2016.

It is planned to seed an additional 20-40 private sector initiatives in 2017 (UNICEF, 2016k). A description of all

current investments can be found in Appendix 7.

Current Investments Area Past Investments Area

1 IPAL, Information Technology University (New) SO: Nutrition/Health/HIV&AIDS

RTI 14 U-Report, Mexico PFY

2 9Needs (New) SO: Health/HIV AIDS/ Nutrition/Education/Child Protection/Social Inclusion

Infra 15 U-Report, Burundi PFY

3 SayCel (New) SO: Health/HIV AIDS/Nutrition /Education/Child Protection/Social Inclusion/WASH

Infra 16 U-Report Chile PFY

4 mPower (New) SO: Nutrition/Health/AIDS/WASH

RTI 17 U-Report, Global PFY

5 Somleng Chatterbox (New) SO: Education

RTI 18 U-Report Papua New Guinea

PFY

6 Better Teachers, Better Learners, Maldives SO: Education

PFY 19 MomConnect, South Africa RTI

7 Thaetorm Koma "Care for Children" SO: Child Protection

PFY 20 MomConnect, Uganda RTI

8 U-Report Brazil Advocacy Project SO: Social Inclusion

PFY 21 Internet of Good Things Infra

9 U-Report Burkina Faso SO: Nutrition/Health/HIV&AIDS

PFY 22 Start-up Myanmar PFY

10 U-Report Fiji SO: Health/Child Protection/Social Inclusion

PFY 23 Point of care Diagnostics PFY

11 U-Report Guatemala SO: Child Protection

PFY 24 Innovation lab PFY

12 U-Report App SO: Child Protection/Education/Health

PFY 25 Academic challenge PFY

13 MApp 4 VAC Reporting Strategic Outcomes: Child Protection

RTI 26 Vojo PFY

LEG

END

Blue = Private Start-ups Products for Youth (PFY) Portfolio Area

27 Yuudee App for Children PFY

Green = Current investments Real Time Information (RTI) Portfolio Area

28 RapidPro Immunization RTI

Off White = Past investments Infrastructure (Infra) Portfolio Area

29 RapidPro Global RTI

Strategic Outcomes: SO 30 EUM's (Global)

RTI

Past Investments: Past initiatives are no longer considered for further funding /support by UIF, hence they will not be considered in this evaluation. They do however continue to receive support from other avenues e.g. UNICEF Scale.

31 Learning Labs

PFY

Figure 41: UNICEF Innovation Fund Portfolio of Investments in Applicants from UNICEF Country Offices and Private Sector Companies (UNICEF ,2016j; UNICEF , 2016l)


__________________________________________________________________________________________

78

Returns and Potential Value

A key objective in Project Portfolio Management (PPM) is to maximize value, with resource allocation requiring

clear judgement of value across multiple perspectives, noting the subjectivity of value (Ang et al., 2007).

‘Evaluative Rubrics' offer a way to provide a structure and methodology to the evaluation process. A rubric

matrix describes different performance levels developed for each initiative, setting out the basis for making

evaluative judgments and enabling a shared understanding of what matters and of 'what good looks like' (King

et al., 2013). The aims and use of rubrics in evaluative design and in making evaluative judgements are

illustrated below.

The aims of evaluative rubrics

Transparently sets out the basis for making evaluative judgments

Provide a means for reaching a shared understanding of what matters, and what ‘good’ looks like

Help us to integrate expectations, policy, strategy and evidence about what good looks like

Provide a basis for integrating multiple data sources to reach holistic evaluative judgments

Keep evaluation focused on the things that matter

Figure 42: Using Rubrics in Evaluation Design and in Making Evaluative Judgments (King et al. 2013)

Davidson (2005) asserts that the relative importance of evaluative criteria can be derived from a number of

strategies as outlined in the table below. Interviews revealed that such judgements for 'MomConnect' and

'Care for Children' are primarily derived from the knowledge of selected stakeholders (bottom-up).

Rubrics

Whatwould• “excellent”• “verygood”• “good”• “adequate”• “inadequate”• “poor”looklike?

Newdatawecollect

Exis ngdata

Policy

Strategy

Evidence

Expecta ons

• Interviews• Focusgroups• Survey• Milestonereports• Programmedata• etc

Ar cula ngshared

understanding

Surfacingvalues Determinedataneeds

Determine data needs &

analysis

Clear overall evaluative judgement

Sense making

Articulating shared understanding

Surfacing Values


__________________________________________________________________________________________

79

Strategy Advantages and Disadvantages

Stakeholders can vote Inclusive and democratic but assumes stakeholders are all well informed.

Can draw on knowledge from selected stakeholders

Combines stakeholder and evaluator expertise but requires careful justification.

Use evidence from the literature Avoids reinventing the wheel and is a good supplement to stakeholder input, but requires good knowledge of the literature

Use specialist judgement (expert panels)

Quicker than a literature search but may reflect the prevailing expert view.

Use evidence from needs and values assessments

Provides independently verifiable evidence of importance but works only for criteria with that evidence.

Use programme theory and evidence of causal linkages

Provides independently verifiable evidence of importance but may be difficult to explain to stakeholders and or use in a participatory mode.

Table 25: Summary of Strategies that can be used for Deriving the Importance of Evaluative Criteria (Adapted from Davidson, 2005)

The literature review as well as subsequent interviews suggests a key challenge of portfolio evaluation is the

tension between the requirement for individualised (bottom-up) evaluative criteria and measurement metrics

at project level (such as those identified for 'MomConnect' and 'care for children') versus standardised criteria

and metrics (top-down) which enable aggregated evaluation of the collective impact at portfolio level (IDIA,

2016; Mathews, 2011; Cooper, 2009; Paulson et al., 2007).

UNICEF Innovation Fund currently measure real time growth in the user base, number of commits and

interactions on a project (UNICEF, 2016 j) in its evolutionary phase of development. Ries (2011) warns against

using gross or 'vanity' metrics, where these types of metrics provide little understanding of true performance

and sustainability. For example as one interviewee put it ‘if only 20% of women are using U-Report, we need

to know the why … we need to dig into the qualitative data to understand’. Another interviewee asserts that

‘real time data tells you what the pattern is, it doesn’t tell you what the pattern means and what to do about

it’ whereas ‘timely data’ is about ‘gathering additional data from users and doing case studies of applications

on the feedback’. Ries (2011) argues that actionable metrics should be developed, as explained further under

Research Objective 3 findings and Appendix 9.

The IDIA's 'Measuring Impact Working Group' identified ‘Leading’ indicators as those used to predict the

expected and projected impact of innovations that have completed the initial ‘Proof of Concept’ stage, before

‘Transition to Scale' (IDIA, 2016) and are therefore more relevant for the UNICEF Innovation Fund given the

stage at which it seeds initiatives. ‘Outcome’ indicators are used to measure the actual and projected impact

of an innovation during ‘Transition to Scale’, ‘Scale’ and beyond (IDIA, 2016). One recommendation is that

successful initiatives are tracked into scale and beyond in order to demonstrate the actual value created

through measurement with 'outcome indicators'. The deployment of 'leading' and 'outcome' indicators across

the three innovation stages is illustrated below.


__________________________________________________________________________________________

80

Figure 43: The deployment of 'leading' and 'outcome' indicators across the four scaling stages (IDIA, 2016)

Interview Findings

This section provides an overview of the interview findings in relation to this papers second objective. The

findings inherently come from the semi-structured Interviews conducted as well as any additional material

shared during the course of the interviews. Table 41 in Appendix 8 provides a synopsis of some of the key

findings, shared by informants.

The findings illustrate the complex nature of demonstrating the value of innovation oriented investments

particularly at portfolio level in international development, highlighting importantly the dos’ and don’ts when

designing a framework to evaluate the Fund.

Overall, there was broad consensus on what constitutes ‘real return’ in international developmental, with

agreement that the long term focus should be about ‘lives improved’. Key to any assessment of value requires

an ability to measure performance outcomes, and in this respect, interviewees were concerned by the lack of

evidence based measures to support the value generated by initiatives in international developmental.

Interviewees shared concerns on how performance for individual initiatives should then be aggregated to

reflect the performance of the portfolio as a whole, noting that ‘at the top level not all innovations are going to

have the same metric’. Other key points raised by interviewees include the importance of establishing a

common language (i.e. definitions), ensuring the process of evaluation is seamless when initiatives transition

from proof of concept into scale, the sometimes-unhealthy obsession with metrics, the timely reporting of

data and that much of the rich content resides in qualitative data providing deeper insight into issues, whilst

with regard to real time data we sometimes fail to see what is emergent.

UIF


__________________________________________________________________________________________

81

One evaluation specialist interviewed asserts that a portfolio’s performance, especially an innovation fund, is

reflected in its diversity and warned against standardised metrics advising that the development of any

evaluation tool must be adaptable to individual initiatives. This interviewee sees the evaluation process as a

more reflective practice and notes ‘what we want with most interventions in complex adaptive systems is

they’re adaptive, but you keep doing the same thing over and over again.’ It is crucial that evaluation should

not discourage experimentation and UNICEF Ventures emphasised that a rigorous evaluation approach for

early initiatives was not required. However the importance of getting the selection, prioritisation, balance and

diversity right remains.

In summary, the difficulty of evaluating a portfolio of innovation initiatives especially in international

development, in addition to the inadequacy of traditional evaluation approaches suggest that integrated

approaches are more suitable as explained in the findings presented in Research Objective 1. Quantitative data

facilitates easier comparison between innovations but it may not provide the full picture of an initiative or

portfolio's impact (IDIA, 2016). Paulson et al.'s (2007) portfolio evaluation tool was developed to enable

innovation hub managers assess the relative values of initiatives within the context of the entire innovation

portfolio at any point in time, and to track changes in the relative value of the portfolio over time (Paulson et

al., 2007).

Complimenting Paulson et al.’s (2007) portfolio evaluation tool, Day’s Innovation Risk Matrix (2007) enables

fund managers to assess the diversity of their portfolios in terms of the funds familiarity with its intended

market as well as its familiarity with the technology it has invested in. The following section highlights the

practicalities and usefulness of both Paulson et al.’s (2007) portfolio evaluation tool, and Day’s Innovation Risk

Matrix (2007).

Paulson et al., (2007) Portfolio Evaluation Tool

The original tool was based on evaluative questions comprising 7 'drivers of valuation' categories. Three

further categories were added based on the IDIA's 'Measuring Impact Working Group's' identification of

'Impact on Beneficiaries’, 'Scalability' and 'Sustainability' as the 3 core concepts in M&E of innovation. A

'financial indicator of value' section was also added. The evaluative questions within each valuation category

were adapted/new questions introduced to serve the context of the UNICEF Innovation Fund. Weightings were

added to emphasise the importance of some sections as recommended by an evaluation specialist. The

sources of the adaption's are as presented in Table 26.


__________________________________________________________________________________________

82

No. Source Description

1 UNICEF Selection and Assessment Scoring Criteria for REOI & RFP stages (UNICEF, 2016m)

2 UNICEF Innovation Funds expected returns and UNICEF's innovation Principles

3 PDMA tool book for new product development (Koen, 2004)

4 Portfolio Management for New Products: Picking The Winners (Cooper and Edgett, 2014)

5 UNICEF IoGT Log frame (UNICEF, 2016o) & U-Report Log frame (UNICEF, 2016p)

6 Interview findings

7 Towards a Shared Impact Measurement Framework for Innovation (IDIA, 2016)

8 UNICEF GIC Monitoring, Evaluation and Learning (MEL) Framework (UNICEF, 2016n)

Table 26: Sources of information for the adaptation of Paulson, et al. (2007) Evaluative Tool

Given the early stage at which UNICEF Innovation Fund invests, the tool mostly comprises of ‘Leading’

indicators which are more dependent on predictive modelling and a chain of interconnected assumptions

(IDIA, 2016) and in line with a recommendation from the IDIA, the tool attempts to include both a measure of

collective impact aggregated from across a portfolio of innovations, as well as indicators for a funder’s return

on investment and measures of how efficient that innovation initiative might be in delivering impact. The

'relevance' option creates a menu option (as suggested by one evaluation specialist) whereby the evaluator

can choose which questions are applicable to a particular initiative thus providing customisation as well as

aggregation.


__________________________________________________________________________________________

83

The tool helps articulate the portfolios contribution to value creation, growth strategy and assists in innovation strategy development (Paulson et al., 2007). Sections 1 to

10 were completed for each of UNICEF Innovation Funds current investments generating an index score as illustrated below in examples taken from section 2 and section 5

of the completed tool. The application of the completed tool along with the 'Framework Methodology' is explained further in Appendix 7.

Section 2

II

Potential impact on accelerating results for the International Child

Rights and Welfare Market: The extent to which this initiative is

likely to have a significant impact in bringing wholly new benefits or

significant leaps in known benefits to the market

Weighting 0.06Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

2.1The initiative has significant market opportunity and relevance

across geographic locations (with UNICEF programs).1 4 1 4 1 4 1 4 1 3 1 4 1 5

2.2

Initiative data could be used to inform policy and

programmes/solution could improves accountability of

governments and other providers of lifesaving services.

1 4 1 4 1 4 1 4 1 4 0 1 5

2.3The initiative involves collaboration and engagement within the

wider community/ecosystem. 1 3 1 3 1 3 1 3 1 3 1 3 1 4

2.4Value chain members are enthusiastic concerning the potential

applications of this initiative.0 - 0 - 0 - 0 - 0 1 3 1 4

2.5

This initiative solves a problem which is poorly addressed at this

time in the market and wider ecosystem (e.g. currently not

addressed by initiatives in the Global Innovation Exchange &

Innovation award repository).

1 3 1 3 1 3 1 3 1 4 1 4 1 4

2.6This initiative could offer a solution to an impending regulatory

threat.0 - 0 - 0 - 0 0 0 0 -

Total Value 14 14 14 14 14 14 22

Maximum Potential Value 20 20 20 20 20 20 25

Initiative Aggregate/Potential impact on the market

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

71.84% 66.7 70.0 66.7 70.0 66.7 70.0 66.7 70.0 66.7 70.0 66.7 70.0 83.3 88.0

Figure 44: Portfolio Evaluation tool Section 2:Potential impact on the market (Adapted by the authors from Paulson et al., 2007)

U-REPORT

Fiji

U-Report

Burkina Faso

U-Report

App

Care for

Children

MomCon-

nect

(South

Africa)

Response Per Individual Initiative (Initiatives 1 to 7)

U-Report

Guatemala

U-Report

Brazil

Advocacy

Project


__________________________________________________________________________________________

84

Section 5

VScalability: Potential for application at national &

international/global scale

Weighting 0.15

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0,

1)

Value

Level

(1-5)

Rele-

vance

(0,

1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

5.1 The initiative has a viable Business model1 4

1 4 1 4 1 4 1 4 1 3 1 3

5.2

An appropriate vision of scale can be developed for alternative

applications/other locations (replication of business model in

different geographies)

1 3 1 4 1 4 1 4 1 4 1 3 1 3

5.3

The initiative rates highly in terms of readiness or quick wins e.g.

the solution can be tested by users on a limited scale, without

requiring complete adoption.

1 3 1 4 1 4 1 4 1 3 1 3 1 3

5.4 The solution is able to use existing infrastructure and/or facilities. 1 4 1 4 1 4 1 4 1 4 1 4 1 3

5.5

The solution is designed in such a way that its core components

could be easily replicated, with contextualized tailoring only

needed for limited aspects of the program.

1 4 1 4 1 4 1 4 1 4 1 4 1 4

5.6 The solution is low in complexity 1 4 1 4 1 3 1 3 1 4 1 4 1 4

5.7 The initiative is designed to reach a large number of people. 1 3 1 5 1 4 1 4 1 4 1 3 1 3

5.8The initiative is cost effective to enable integration into

government or other large-scale programmes.1 4 1 4 1 4 1 4 1 4 1 4 1 4

5.9 Expected demand / market readiness is high 1 4 1 3 1 3 1 3 1 3 1 3 1 3

33 36 34 34 34 31 30Maximum Potential Value 45 45 45 45 45 45 45

Initiative Aggregate / Scalability

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import

% of

Max

Value

% of

Drivers

Import

% of

Max

Value

% of

Drivers

Import

% of

Max

Value

75.89% 100.0 73.3 100.0 80.0 100.0 75.6 100.0 75.6 100.0 75.6 100.0 68.9 100.0 66.7

Figure 45: Portfolio Evaluation tool Section 5: Scalability (Adapted by the authors from Paulson et al., 2007)

Response Per Individual Initiative (initiatives 8 to 14)Better

Teachers,

Better

MAPP VAC9Needs Mpower SomlengSaycel IPAL


__________________________________________________________________________________________

85

It is the intent for the framework that each section is completed by the project manager in collaboration with the portfolio manager. The outcome is a snapshot in time of

the relative performance profile of each initiative aggregated across the 10 value drivers as shown below.

Figure 46: Portfolio of Initiatives Performance Profile Aggregated Across Sections 1 to 10 (Adapted by the authors from Paulson et al., 2007)


__________________________________________________________________________________________

86

An aggregated performance profile is also generated for each value driver as shown in below. In this application 'sustainability', 'scalability' and 'Firm Capabilities' score

well (low-mid 70's), whereas 'projects impact on company renewal' scored low-mid 60's. The subjective and assumptive nature of application due to the time and

proximity limitations of this research project is noted however the framework represents the most comprehensive approach uncovered by the authors following, desk

based reseach, literature review, interviews with key informants and key informant documents.

Figure 47: Aggregated Initiative Performance Profile (Sections 1 - 10) (Adapted by the authors from Paulson et al., 2007)

Answering the questions takes time and effort but the information generated is perhaps the most powerful result of the process according to Paulson et al. (2007) and this

can make a valuable contribution to the feedback and learning process. Improvement in the relative value of the portfolio is expected as the fund matures. Tracking the

changes will demonstrate increased value (or lack thereof), for example more innovative/novel initiatives should emerge as the Fund gains more exposure to the

innovation ecosystem. The portfolio manager completes section 11 'Overall Portfolio Health', which in this application scores quite well with 73% as illustrated in figure 49

below.


__________________________________________________________________________________________

87

Presently the Fund could be better balanced in terms of portfolio areas addressed and internal/external sourcing of initiatives but considering it is in the start-up

evolutionary phase of development such issues are to be expected. The diversity and pooled funding efficiencies are expected to grow with the Fund as it evolves over

time. As noted above, it is recommended that successful initiatives are tracked into scale and beyond in order to demonstrate actual value created using outcome

indicators. For example, the ‘3E Value for Money (VfM) Concept’ which is based on economy, efficiency and effectiveness (Diamond, 2005), where the measurement of

effectiveness extends towards outcomes and impacts, and approaches such as SROI and Social Cost benefit analysis (CBA) could be considered.

Figure 48: UNICEF Innovation Fund Portfolio Health Profile


__________________________________________________________________________________________

88

Section 11XI Portfolio Health

Relevance

(0, 1)

Value Level

(1-5)

11.1

With the current mix of initiatives, the fund continues to maintain a strong

consensus and clear focus on the strategic direction of the portfolio and its

overall objectives (inc. target domains and competencies to developed).

1 7

11.2The mix of initiatives within the portfolio continues to reflect the current political,

economic, social, technological, environmental & legal environment (climate). 1 7

11.3UIF innovation initiatives form the basis of a continuing pipeline of potential

significant contributions to the greater UNICEF mission. 1 7

11.4The portfolio demonstrates UIF's growing sphere of influence when compared to

addressing the needs of its constituents by more traditional means.1 8

11.5

The portfolio of initiatives is well-diversified with respect to UIF portfolio areas 1.

Products for youth under 25 2. Real‐time information for decision making 3.

Infrastructure to increase access to services and information.

1 6

11.6The portfolio of initiatives is well-diversified with respect to contribution to

expected returns, namely; Growth, IP Stack, Communities.1 8

11.7The portfolio of initiatives is well-diversified with respect to internally sourced

(country offices) & externally sourced innovations. 1 6

11.8 At least 10% of the initiatives demonstrate enough potential value to be scaled

up with the aim of achieving impact. 1 8

11.9 The ‘churn rate’ of initiatives in the portfolio is appropriate. 1 7

11.10UNICEF Innovation Funds portfolio of initiatives is sufficiently diverse in terms of

key secondary benefits that UIF are trying to develop. 1 7

11.11 UIF have not spread resources too thin across too many initiatives in this portfolio. 1 7

11.13

Portfolio enables easy identification of initiatives considered low in value (which

in all likelihood will inhibit the growth of the funds outputs/outcomes) when

assessed against other initiatives within the fund.

1 8

11.14The initiatives in this portfolio are benefiting from synergistic effects on one

another.1 8

11.15 initiatives continue to transition to scale in a timely manner. 1 7

11.16

The portfolio of initiatives continues to be supported by a governance structure

which avoids waste and duplication, permitted by efficient allocation of

resources, and cost economies achieved through scope and scale.1 8

11.17Cross collaboration across initiative teams at portfolio level is enhanced,

enhancing portfolio (fund) efficiencies?1 7

Total Value 62

Maximum Potential Value 85

% of

Drivers

Important

% of Max

Value

Portfolio Health total 100.0 72.9

ResponsePortfolio

Figure 49: Section 11: Portfolio health (Adapted by the authors from Paulson, et al., 2007)


__________________________________________________________________________________________

89

Day (2007) Risk Matrix

As a company shifts from incremental to growth oriented projects, organisation's assume greater

levels of risk especially as it moves towards unfamiliar markets, with even more unfamiliar products

/technology, changing the risk/reward relationship of the portfolio. Optimising the risk return trade-

off will necessitate a balanced approach to project investment ensuring organisations ‘…distribute

innovations more evenly across a spectrum of risk’ (Day, 2007).

Day’s risk matrix provides a useful methodology for assessing a portfolio’s diversity in terms of the

risk profile assumed for innovation projects in international development, by adopting a ‘…unique

scoring system and calibration of risk to help estimate the probability of success or failure for each

project based on [the organisation's familiarity around its] intended market (x axis) and the product

or technology (y axis)…’ (Day, 2007).

Analysis presented in Figures 50, Table 27 and Appendix 7 suggests that the innovation portfolio is

currently clustered around more adjacent technology, with less incremental and few

transformational initiatives. The Portfolio is currently aligned with the Funds stage of evolution and

with the Horizon 2 medium term focus. Progression towards a more risky transformational mix is

expected as more start-ups are seeded.


__________________________________________________________________________________________

90

Figure 50: UNICEF’s Innovation Risk Matrix – Diversity of its Profile.

9Needs

Care for children

Somleng Chatterbox

U-Report App

U-Report - Brazil Advocacy Group

Saycel

IPAL

U-Report - Guatemala

U-Report - Burkina Faso

Mom Connect

Better Teachers

U-Report - Fiji

Mapp 4 VAC Reporting

mPower

0

5

10

15

20

25

30

0 5 10 15 20 25 30 35

Pro

du

ct /

Te

chn

olo

gy

Intended Market

UNICEF Innovation Risk Matrix

50% - 60% Relative Risk





New to company

Adjacent to current offerings

Same as current offerings

Same as present Adjacent to present New to company


__________________________________________________________________________________________

91

Initiative Brief introduction to the initiative. Familiarity with Intended Market Familiarity with Product/Technology

9Needs (South

Africa) (New)

Initiative improves Early Childhood

Development services in South Africa using

blockchain infrastructure and smart contracts.

9Needs has prototyped a system which

strengthens current registration, contracting,

information and management systems.

This initiative is being launched for the first time

in a new market. Distribution and uptake rates

are unknown with little information on which to

base an estimate.

Intended Market Risk is considered HIGH.

The technology is new to the company, with

block chain being introduced for the first time.

Product/Technology risk is considered HIGH.

Somleng

Chatterbox

(Cambodia)

(New)

An efficient and low cost Interactive Voice

Response (IVR) and SMS platform, to be

integrated into the RapidPro platform to extend

its reach to communities with low literacy

levels.

Will complement the existing RapidPro

technology operating in the same market.

Intended Market Risk is considered MODERATE.

The IVR technology is relatively new to the

market, and acceptance/uptake has yet to be

proven.


SayCel

(Nicaragua)

(New)

Improve low-cost open source GSM

communications suite used to run cellular

networks in rural regions by installing low-cost

technology & training local governments and to

maintain their own networks.

Initiative extends mobile cellular coverage to

less densely populated rural regions.

Relationship with stakeholders key. New

initiative in the market.


Product technology is new to the organization,

suggesting greater degree of reliance may be

placed on vendor partners.


Innovations

for Poverty

Alleviation Lab

(IPAL)

(Pakistan)

(New)

IPAL have developed an interactive voice

response (IVR) system that serves as a maternal

care hotline. Aims to provide fathers with

access to medical advice and guidance in

accessible formats.

Initiative aligns well within the real-time

information portfolio and UIF's intended market


Product is relatively new, and as a form of

engagement, some overlap with existing

products is evident.


mPower

(Bangladesh)

(New)

Open Smart Register Platform (OSRP), an

integrated digital registry platform that seeks to

improve data collection and efficiency for

frontline workers providing reproductive,

maternal, new born and child services.

New market initiative. Initiative aligns well

within the real-time information portfolio and

UIF's intended market


Product/Technology sits well within the

organisation's domain.

Product/Tech risk is considered MODERATE.

Table 27: Analysis Supporting Innovation Risk Matrix - Portfolio (Current) including First External Investments (UNICEF ,2016J; UNICEF , 2016L)


__________________________________________________________________________________________

92

4.4 Research Objective 3: Investigation into Potential Secondary Benefits

Interview Results

Potential secondary benefits of the UNICEF Innovation Fund identified for further investigation were based on

Canales et al. (2014) ‘secondary effects scorecard’. This was then expanded to include potential benefits

identified by UNIECF Ventures in initial exploratory interviews. The expanded list was converted into a

standard interview question which was asked to all key informants with knowledge of the Fund as follows.

Potential Secondary Benefits Investigated

Objective 3: Secondary Effects

1. Below is a list of potential secondary benefits from cultivating and encouraging innovation

through UNICEF Innovation Fund:

(i) Organisational learning (often through failure)

(ii) Making the organisation ready for change

(iii) Inspiring staff and attracting new types of talent

(iv) Creating a sustainable network of long term relationships even after completion

(v) External mimicry

(vi) Creating new business models

(vii) Building capabilities

(viii) Driving organisation goodwill (i.e. new funders/allies)

a. Have you observed any of the above in relation to UNICEF Innovation Fund or UNICEF

Ventures?

b. Are there any secondary benefits that you have observed from catalysing innovation

that is not listed?

Table 28: Typical Extract from Semi Structured Interview Guide: Potential Secondary Benefits Question

The replies to question ‘a’ above were aggregated into percentages of interviewees returning positive

responses on each of the eight potential secondary benefits. Organisational learning (often through failure)

was the most noted benefit with 90% of interviewees returning positive responses. Creating a sustainable

network of long term relationships and driving organisational goodwill were the second most noted with 80%

positive response. Inspiring staff and attracting new talent, creating new business models and building

capabilities was the third most noted benefit with 70%. Making the organisation ready for change with 60%

and external mimicry with 50% completed the list. A summary of the findings is illustrated below.


__________________________________________________________________________________________

93

Figure 51: Secondary Benefits Experienced by Interviewees (UNICEF Interviews, 2016)

Organisational Learning

(often through Failure)

Making the Organisation Ready

for Change

Inspiring Staff and Attracting New Types

of Talent

Creating a Sustainable Network

of Long Term Relationships even after Completion

External Mimicry Creating New

Business Models Building Capabilities

Driving Organisation Goodwill (i.e. new

funders/allies)

1 2 3 4 5 6 7 8

% Observed 90 60 70 80 50 70 70 80

0

10

20

30

40

50

60

70

80

90

100

% O

bse

rve

d

Secondary Benefits

Secondary Benefits Observed/Experienced by Interviewees

90

70

80

70

50

80

70

60

1 4 3 2 5 2 3 3


__________________________________________________________________________________________

94

Organisational Learning (Often Through Failure)

Organisational learning was the most noted secondary benefit from interviewees with 90% returning positive

observations and some informants asserting that it is a primary benefit of UNICEF Innovation Fund.

Organisational Learning: Key Interview Observations

90% Positive Response

‘I would move organisational learning and inspiring staff in particular to a primary benefit...’

‘...we’ve often talked about the importance in innovation of failing fast, but... a new iteration of that is

actually failing smart and that captures the learning element… really understanding what it was that didn’t

work, what influence your own organisation may have played in that process in particular. Because

organisation's can make it pretty systematically difficult to scale innovation effectively just by virtue of the

way they are set up, and the culture and the risk appetite.’

‘…there was a lot of learning... we did fail several, several times, we had to redesign... we had to find our

way working creatively with the government and our partners... and all of that was a huge benefit because

the national scale up was able to be done... learning from those lessons they didn’t have to go through the

same thing again... so that definitely was a plus’

‘Organisational learning, absolutely captured because we are going to be failing a lot....’

Figure 52: Organisational Learning Interview Observations (UNICEF Interviews, 2016)

Despite the high observation rate the opposite was true for interviewees asked if a systematic mechanism was

in place to track those learning benefits along with other secondary benefits. Lerner (2013) argues that, if

corporate venturing programs are to succeed, organisation's need to invest in learning and harvest knowledge

from start-ups.

Ries (2011) posits that the most vital function of organization building under conditions of extreme uncertainty

is validated learning, a rigorous method for demonstrating progress empirically, uncovering valuable truths

about present and future prospects. Ries, argues that any efforts in early stage ventures which do not

contribute to learning are waste.

The mandatory requirement of UNICEF Innovation Fund that real time data must be provided by investees

offers data for systematic validated learning. Figuring out the right solutions to build is ‘true start-up

productivity and seeks to avoid building products that customers do not want’ (Ries, 2011). This approach

combined with corporate venturing has the potential to accelerate the development of such solutions through

the UNICEF Innovation Fund and internalise the learning benefits. A proposed framework for managing the

new start-ups through the iterative learning process based on the Lean Start-up model is illustrated below and

further information is contained in Appendix 9.


__________________________________________________________________________________________

95

Figure 53: Build-Measure-Learn Feedback Loop and UNICEF Innovation Fund (Adapted from Ries, 2011, See also Appendix 9)

Ideas with Proof of

Concept & Assumptions

Build

Open Source Products / Solutions

(MVP)

Measure Using Actionable

Metrics

Real Time Data

(from actual customers)

Learn Validated Learning

Avoiding Waste

Value Creation Hypothesis Growth Hypothesis

Start-up Leap-of-Faith Assumptions (quantitative predictions)

Must be able

to measure

MVP Impact

Are you making your

product better?

How do you know?

How do you know that

the changes made to

MVP are related to the

results we’re seeing?

How do we know

we are drawing the

right lessons from

those changes?

Start building as quickly as possible to start the

learning cycle (launch early and iterate)

Innovation Accounting

Learning Milestone 1

Build 1st

MVP



Tune the Engine

from Baseline to Ideal

(Iterations, micro changes

& optimisations)



Pivot of Persevere Decision

Test Assumptions & Fundamental Business Hypothesis Systematically and Rigorously through iterations of a Minimal

Viable Product (MVP) targeting early adopters whilst maintaining long term Vision. Additional features beyond what

early adopters demand and beyond what is required to start learning are not incorporated to avoid wasted resources

and time (avoiding expense and time to build a complete product only to find that nobody wants it).

Innovation Accounting used to quantitatively establish whether the ‘engine-tuning efforts’ are adding value through

the evaluation of learning milestones; Allows UNICEF Ventures establish if failures are because the start-up learned

something critical or due to value destroying reasons; Differentiate true value creating ventures from value destroying

ventures; Based on Quantitative Financial Model / Value Creation Model to be evaluated rigorously.

Goal: Full Turn of Build-Measure-Learn Loop with Minimum Effort and in the Least Amount of Time.

UNICEF Innovation Fund VC Selection Criteria Filters Initiatives with greatest potential to be seed funded

UNICEF Office of Innovation Decision to Support Scale or ‘have a hard no’

(On Completion of UNICEF Innovation Fund Support Period)


__________________________________________________________________________________________

96

Making the Organisation Ready for Change

‘Making the organisation ready for change’ was the second least noted secondary benefit with just 60%

positive responses.

Making the Organization Ready for Change: Key Interview Observations


‘making the organisation ready for change is a big one, especially with these emerging technologies we

see... in our near future it’s important to be aware of them’

‘if this doesn’t prepare us for talking financially about things as an organisation I don’t know what will....

‘$100Bn industries meet $1Bn person needs… UNICEF tries to influence’

Figure 54: Making the Organisation Ready for Change Interview Observations (UNICEF Interviews, 2016)

The ‘Law of Accelerating Returns’ (Kurzweil, 2001) demonstrates that technological change historically has

been exponential but posits that people intuitively assume that the current rate of progress will continue in

future periods and hence dramatically underestimate the rate of change, offering some explanation for this

secondary benefits lagging acknowledgement among interviewees.

Most analysts agree that technological growth will significantly influence the way organizations will operate in

future and this presents an opportunity to develop new business models to create value in new ways not

possible previously (AMBS, 2015). Christianson (1997) demonstrates how large established organisations

became undone by such change observing ‘the firms that led the industry in every instance of developing and

adopting disruptive technologies were entrants to the industry, not its incumbent leaders’. UNICEF must move

with the times and be ready for such change or risk losing relevance. Lerner (2013) argues that the creation of

a venture fund might help deliver breakthrough ideas and the UNICEF Innovation Fund has the potential to be

such a vehicle.


__________________________________________________________________________________________

97

Exponential Rate of Technological Change Vs. Intuitive Assumption that Current Rate will Continue

Internet Exponential Rate of Change

Figure 55: Exponential Rate of Change in Technology Vs. Intuitive Assumption (Kurzweil, 2001 - Director of Google Engineering and founder of Singularity University)

Intuitive Assumption that Current Rate of progress will continue in future periods.

Exponential Rate of Technological Change (mass use of inventions)

Electricity

Telephone Radio

Television

PC

Electricity

Mobile Phone

Internet


__________________________________________________________________________________________

98

Inspiring Staff and Attracting New Types of Talent

Inspiring staff and attracting new types of talent was joint third most noted secondary benefit by interviewees

with 70% returning positive observations. Once again some informants advised that it was in fact a primary

benefit of the UNICEF Innovation Fund consistent with the findings related to organisational learning.

Inspiring Staff and Attracting New Types of Talent: Key Interview Observations


‘I would move… inspiring staff in particular to a primary benefit...’

‘...it definitely inspires staff, creates that motivating environment and yes we have attracted new types of talent’

‘..Using the digital system really brings a new energy and enthusiasm to using the new tool.. which will... simplify their lives.. they will be able to carry out other tasks in the time that has been spared... the team... is extremely enthusiastic... as a learning process but also to enable them to better do their job... in a more efficient way... and being able to address other urgent problems that children are facing.... by simplifying.... and making a much more effective system....’

‘....third one, absolutely we’re seeing that already because we’re seeing people speaking with the VC language or capital mind-set language which is very interesting...’

Figure 56: Inspiring Staff and Attracting New Types of Talent Interview Observations (UNICEF Interviews, 2016)

Desk based research and literature reviews were reasonably consistent with the positive findings. Figure 57

shows the top 50 companies young people want to work for in a study by Collegefeed polling 15,000

Millennials either in college or recently graduated (Agrawal, 2014). The key quality millennials look for in

employers is people and cultural fit (Agrawal, 2014) and the UNICEF Office of Innovation endeavours to

cultivate a new forward looking culture that embraces new ways of working.

The Collegefeed study shows that 25% of respondents look for innovative companies to work for and the top 5

companies they target are ranked the top 5 most innovative companies currently by Strategy& PwC in their

annual innovation study (Strategy& PwC, 2016). So the interview findings, that ventures such as the UNICEF

Innovation Fund inspires and attracts new types of talent is very much consistent with the Collegefeed

findings.


__________________________________________________________________________________________

99

Top 50 Companies Young People Want to Work For

What Millennials Looks for in Employers

Figure 57: Top 50 Companies Millennials Want to Work For and What they Look for in Employers (Agrawal, 2014 and Strategy& PwC, 2016)

Also Top 5 Most Innovative Companies (Strategy& PwC, 2016)


__________________________________________________________________________________________

100

Creating a Sustainable Network of Long Term Relationships Even After Completion

Creating a sustainable network of long term relationships was joint second most noted secondary benefit by

interviewees with 80% returning positive observations.

Creating a Sustainable Network of Long Term Relationships even after Completion: Key Interview Observations


‘I think creating networks of relationships especially within the tech industry regulatory bodies within UNICEF is something I’ve been involved with governments... NGO’s... other UN Agencies... that are all trying to tackle the same problem...’

‘...we’ve had so much positive outreach from people running other funds, who want to work with us because we’re not a threat and we’re investing in markets they’re interested in... we’ve had everyone from family foundations, to... venture funds which have said we’ll share pipeline to the IDIA group. Groups in all of those... have expressed interest in collaboration...’


This secondary benefit should also be considered primary as the purpose of the UNICEF Innovation Fund, as

illustrated in the Mission Model (Figure 24) and the Corporate Venturing Model (Figure 29), is to build a

community of open source problem solvers around solutions for children. UNICEF's Principals for Digital

Development, a key part of the UNICEF Innovation Fund venture capital scoring criteria (Appendix 3) also

underpins this purpose as the follow extracts illustrate.

Network Related Principals for Digital Development

Principal 2: Understand the Ecosystem

Participate in networks and communities of like-minded practitioners.

Principal 4: Build for Sustainability

Utilise and invest in local communities and developers by default, and help catalyse their growth.

Engage with local governments to ensure integration into national strategy, and identify high level government advocates.

Principal 6: Use Open Data, Open Standards, Open Source, Open Innovation

Invest in software as a public good.

Develop software to be open source by default with the code made available in public repositories and supported by developer communities.

Principal 9: Be Collaborative

Engage diverse expertise across disciplines and industries at all stages.

Work across sector silos to create coordinated more holistic approaches.

Document work, results, processes and best practices, and share them widely.

Publish materials under create commons license by default, with strong rationale if another licensing approach is taken.


Metcalfe’s Law argues that ‘the value of a network as a whole is proportional to the square of the number of

participants’ (Shapiro and Varian, 1999 and Ries, 2011). Chesbrough (2003) argues that open innovation

acknowledges that ideas that add value can come from inside and outside the organisation distinct from the

centralised R&D silos of the closed innovation paradigm as follows.


__________________________________________________________________________________________

101

The Closed Innovation Model

The Open Innovation Model: UNICEF Office of Innovation and UNICEF Innovation Fund

‘In the closed model, the lab is your world, in the open model the world becomes your lab’ (Chesbrough, 2016 at Haas Business School)

Figure 60: Closed vs. Open Innovation Models and UNICEF Innovation Fund (Chesbrough, 2003 and UNICEF, 2016c&d)

UNICEF Innovation

Fund Requesting internal and

external applications

and seed funding

multiple early stage

initiatives

Closed linear pipeline of innovation

towards clearly defined markets

Open Network of Relationships

Sustainable Open Source

IP

Built with Each Initiative.

Permanently open source

and shared on Github.

Communities of Open Source

Problem Solvers around

solutions for children in

UNICEF programme

countries



Closed Network

Relationships

Not Developed Closed linear pipeline

of innovation towards clearly defined markets


__________________________________________________________________________________________

102

External Mimicry

External mimicry was the least noted secondary benefit by interviewees with 50% returning positive

observations. Many reasons can explain this low response including the benefit not accruing but also the

humility of informants.

External Mimicry: Key Interview Observations


‘...we just had another group in the UN... launching [a] fund... that was mixed public and private funding, so they’re already doing that… which is great…’

‘internal mimicry is something that happens... people are mimicking taking risks... internally as well as externally and that helps you get to organisational change’

Figure 61: External Mimicry Interview Observations (UNICEF Interviews, 2016)

Bloom and Falkner 2015 argue that ‘the growing focus on innovation within UN agencies most prominently

began with UNICEF's initiative to establish a dedicated innovation unit in 2007’. Wishnie and Wyatt (2016) cite

UNICEF as one of the ‘few forward-thinking NGOs [that] established central tech and innovation teams’

referring to them as the ‘standout example’.

Dean (2015) noted the adoption of UNICEF's Principles for Digital Development by other organisation's as a

‘ripple effect’ and demonstrated that UN agencies seek advice from UNICEF including UNDP, UNFPA and

UNHCR with UNICEF seen as a ‘champion of innovation’ (Dean, 2015).

The identification of ‘internal mimicry’ taking place within the organisation where individuals in departments

outside the office of innovation are beginning to mimic some of the risk taking approaches was an interesting

insight surfaced in the interviews. The potential for internal mimicry to stimulate organisation change was

emphasised and can be seen as an important benefit that will need to be leveraged going forward as the

organisation seeks to build new capabilities and challenge the organisational stasis identified by Amatullo

(2015).

With regards to external mimicry, desk based research established a timeline of major actions by UNICEF and

other external organisations. Although evidence of clear cause and effect is not explicit, the timeline and

literature suggest that UNICEF is a pioneer of innovation within the UN ecosystem which has pushed forward

the innovation agenda. The UNICEF Innovation Fund is too early in its evolution to establish clear mimicry

however the WFP accelerator fund and UNHCR innovation fund are perhaps early examples.


__________________________________________________________________________________________

103

Figure 62: External Mimicry Time Line from Desk Based Research (Created by the authors based on Bloom and Faulkner, 2016; UNICEF, 2015a&b; UNHCR, 2014; WFP, 2016; UNFPA, Undated and Principles for Digital Development, 2016)


__________________________________________________________________________________________

104

Creating New Business Models

Creating new business models was the third most noted secondary benefit from interviewees with 70%

returning positive observations. The benefits of the UNICEF Innovation Fund, as a new corporate venturing

model, have been covered earlier in this report (See Figures 29 and 30). This section presents findings related

to potential new business models of start-ups seeded by the Fund.

Creating New Business Models: Key Interview Observations


‘New business models, absolutely, we actually had to come up with the open source cases and open source business models...’

Figure 63: Creating New Business Models Interview Observations (UNICEF Interviews, 2016)

Bain (2013), McKinsey (2014 & 2015) and BCG (2015) all cite the importance of business model innovation to

grow and build business. Lindgardt et al., (2009) cite BCG research demonstrating that returns to business

model innovators were four times greater than those to product or process innovators.

The classic open source freemium business model of US software company Red Hat is illustrated below with

revenue generated from enterprise clients and software also available free to self-service users. Red Hat

benefits by having its software continuously improved by the open source community reducing the traditional

high fixed costs of in-house software developers (Osterwalder & Pigneur, 2010 & Meagher, 2016).

Key Partners

(Linux) Open Source

Development Community

Key Activities

Software support

services

Software versioning &

testing.

Value Proposition

Free (Linux) open source based

software

Continuously upgraded, serviced, & guaranteed

software

Customer Relationship

Self-service &

direct access to engineers

Customer Segments

Self-service Users

Enterprise Clients

Key Resources

Red Hat (Linux)

Software

Channels

RedHat.com

Red Hat global branches

Cost Structure

Cost structure contains

elements of a service company

Revenue Streams

Professional Subscription

Free Software

Figure 64: Open Source Business Model Canvas: Red Hat (Osterwalder & Pigneur, 2010 & Meagher, 2016)

A review of the many variants of open source freemium business models has been summarised below

including professional service models, software as a service models and other variants as clear pathways for

start-ups seeded by UNICEF Innovation Fund.


__________________________________________________________________________________________

105

No. Freemium Open Source Business Model Variant

Free Open Source Value Proposition (IP Stack and Community)

Revenue Stream for Start-up Example Reference

1 Professional Service Model: Consulting /Training/ Support

Software (source code and binaries) fully open source and freely available.

Fees charged for consulting, training, installation and/or support.

Red Hat https://www.redhat.com/en

2 Professional Service Model: Selling Executable Binaries

Open source software provided in source code form only (not including executable binaries (machine code).

Certified and tested (stable and service-ready) executable software binaries (encoded instructions (machine code) that make computers perform tasks) sold to paying customers and offering commercial service of compiling and packaging the software.

Red Hat, IBM, Revolution Analytics.

https://www.redhat.com/en

3 Open Software-as-a-Service (OSaaS) Model (Cloud as a Service)

Open Source Software package provided allowing the community and users to act as their own network host and build their own customized blog, website, etc. Example: Wordpress.org

Open Software as a Service (OSaaS) provided in parallel offering internet hosting service, support, maintenance upgrades and enhancements for a fee. Service is accessible via paid subscription based on user number, transaction volume or time based. Example Wordpress.com

Wordpress.org and Wordpress.com. Drupal Gardens Management System.

https://wordpress.com/ https://wordpress.org/

4 Proprietary Software Model or Dual-licensing Model

An open-source version is licensed and continuously developed and updated.

Certified and tested (stable and service-ready) proprietary version similar to open source version which does not use the source code is licensed for a fee to enterprise clients. (Similar to traditional enterprise software revenue stream).

Oracles MySQL Database, MongoDB (Union Square Ventures)

http://www.oracle.com/technetwork/database/mysql/index.html

5 Open Core Model or Proprietary Extension Model

Core software is open source and continuously developed and updated.

Fees levied for special features, modules that extend or enhance the core product. Optional value added proprietary extensions include modules, plug-ins, add-ons or fringe benefits.

IBM Proprietary Linux, Red Hat (Linux), Cloudera Apache Hadoop, Talend.

https://www-03.ibm.com/linux/matrix/index.html https://www-03.ibm.com/linux/ltc/

6 Partnerships with Funding Organisation's

International Organisations, Governments, NGOs, Universities and companies pay for software and then release as open source.

Grant funding, salary paid to developers or venture capital style seed or catalytic funding to begin with. Further revenue streams required for long term.

UNICEF Innovation Fund Google Summer Code initiative

http://www.unicefinnovationfund.org/#about-page

7 Advertising Supported Software Model

Software provided free and open source.

Advertising banner sales. SourceForge, Google, Mozilla.

https://sourceforge.net/

8 Delayed Open-sourcing: Version Lagging

Software of previous versions provided open source.

Latest version available to paying customers. MariaDB Corporation https://mariadb.com/about http://mariadb.org/

Table 29: Types and Variants of Open Source Business Models (As referenced above)



https://wordpress.com/

https://wordpress.org/




https://www-03.ibm.com/linux/matrix/index.html



https://www-03.ibm.com/linux/ltc/

https://www-03.ibm.com/linux/ltc/



https://sourceforge.net/

https://mariadb.com/about

http://mariadb.org/


Page:106

A greater challenge than establishing open source business models with clear routes to commercialisation is

merging those models with value creation for UNICEF's child rights mission. UNICEF Ventures have identified

business model development as an area where start-ups will need significant support (UNICEF interviews,

2016). Figure 65 illustrates such a dual value creation concept which seeks to help the bottom quintile of

children out of poverty using a developers’ technology thus unlocking a future customer base with positive

feelings towards the developers’ brand.

There are of course many variants of the below concept which have the potential to be developed, highlighted

in part by Table 29 above but also considering new business models which have not yet been created. This is a

key benefit that the UNICEF Innovation Fund can leverage. Should the Fund grow to levels illustrated in the

Mission Model (Figure 24) and Strategic Staircase (Figure 27) then the development of new business models

through the seeding of external start-ups could potentially transform the way UNICEF delivers its mission in

the long term.

Key Partners

Open Source Development Community

Technical Experts

Academia

Private Sector Governments Civil Society

Country Offices Innovation Labs

Key Activities

Software

support services

Software versioning &

testing

Value Proposition

Free open source based software

creating value: 1. Products for needs of youth

under 25 (learning / participation), 2. Real-time information for

decision making 3. Infrastructure to increase access

to services & information (connectivity, power finance...)

Continuously upgraded,

serviced, & guaranteed software or other as relevant to the particular

innovation

Customer Relationship

Self-service &

direct access to engineers

other

Customer Segments

Child welfare & rights:

improved strategic outcomes

Bottom Quintile of Children lifted out of poverty =

Potential Future Customers

Enterprise Clients

Key Resources

Technical Support &

Mentoring by UNICEF and

Partners

Channels

Website

Branches

Other

Cost Structure

Cost structure contains elements of a service company Low software development costs as open source community continuously improve the software free of charge.

Revenue Streams Return (value): 1. Growth, IP Stack & 3 Communities (Potential: use 3 returns to earn revenue)

Professional Subscription (Potential)

Some Personal Subscriptions (Potential)

Free Software / Platforms / Solutions

Figure 65: Conceptual Open Source Business Model Canvas for Seeded Start-ups (Dual Value Creation) (Osterwalder & Pigneur, 2010, UNICEF, 2015a & 2016a-i & Meagher, 2016)


Page:107

Building Capabilities

Building capabilities was the joint third most noted secondary benefit by interviewees with 70% returning

positive observations.

Building Capabilities: Key Interview Observations


‘…different systems, interoperable systems and SMS technology.. it’s helping them think differently and more effectively... this has been a major change, me myself I’ve learned so much by doing this that I wouldn’t have known... that’s definitely a... secondary benefit’

Figure 66: Building Capabilities Interview Observations (UNICEF Interviews, 2016)

Whilst internal capabilities are built through innovation, the corporate venturing strategy employed through

the UNICEF Innovation Fund primarily builds external capabilities leveraging communities of open source

problem solvers. Chesbrough and Rosenbloom (2002) argue that start-ups are less likely to be constrained in

the experimentation with alternative models and Lerner (2013) posits that ‘a corporate VC fund can move

faster, more flexibly, and more cheaply than traditional R&D arguing that ‘it’s likely that developing capabilities

on their own would have taken Lilly [Ventures] far longer and been far more expensive’.

Challenges and limitations have been identified by UNICEF Ventures regarding the pace at which the wider

organisation can adopt new initiatives and build innovation capabilities citing organisational inertia (Amatullo,

2016). The UNICEF Innovation Fund can sidestep such cultural constraints, reducing the time and cost required

to build capabilities internally by investing in start-ups with the intention of building an innovation ecosystem

complimentary to the organisation's mission from which the organisation can learn.

The UNICEF Innovation Fund ‘degree of openness’ which helps build external capabilities is illustrated in Figure

67 below based on a framework by Harwood (2016). The fund can be seen plotted in the top and right of the

matrix, the zone considered highest on a scale of open innovation. The corporate venturing and open

innovation ecosystem pitches the UNICEF Innovation Fund in two zones of the matrix both of which are in

Harwood (2016) ‘exploit’ stage of innovation.


Page:108

Stage of

Innovation

Exploit Intrapreneurship

Programme e.g. Google 20% Time

Corporate Venture Fund e.g. Intel Capital, Lilliy

Ventures UNICEF Innovation Fund

Venture Model

Open Innovation Ecosystem

e.g. LEGO Ideas UNICEF Innovation Fund

Open Source Community

Extract Innovation Incubators

e.g. Lockhead Skunkworks

Co-Creation Community e.g. Ford Innovative Mobility

Accelerator Programme e.g. Telefonica Wayra

Explore Colleague Crowd e.g. Oxfam Future

Shoppers

Social Listening e.g. P&G House Proud

Crowd

Open Innovation Challenge

e.g. UBS FoF Challenge

Open Inside Outside In Inside Out

Degree of Openness

Figure 67: UNICEF Innovation Fund Plotted on the Open Innovation Methods Matrix (Adapted from Harwood, 2016)

The major challenge for UNICEF is how to internalise the external capabilities. Possible future evolution of

UNICEF's strategy of investing in start-ups identified by UNICEF Ventures and captured in the mission model

(Figure 24) and strategic staircase (Figure 27) include taking equity stakes in successful start-ups and / or

acquisition.


Page:109

Driving Organizational Goodwill

Driving organisational goodwill was the second most noted secondary benefit from interviewees with 80%

returning positive observations along with creating a sustainable network of long term relationships

mentioned earlier which had an equally positive response.

Driving Organizational Goodwill (e.g. New Funders/ Allies): Key Interview Observations


‘Public positioning of UNICEF, a lot of what Ventures and Futures does, are new things...people don’t expect shiny new things out of UNICEF but it is the Futures and Ventures group who is doing that... there is a tremendous amount of publicity for the organisation, of repositioning the organisation to be highly relevant, highly innovative... future focused... that I would capture under bullet number 8’ (driving organisational goodwill)

‘It positions... UNICEF is seen in the scenario of stakeholders as an innovative partner... dialoguing with new trends, open to adapt... and not insisting on the old type of communication... it strengthens our mission and our goal’

‘Eight yes, in that we had people.... who would never have funded UNICEF traditionally’

‘In terms of secondary benefits... we’ve attracted more donors and different partners that we otherwise would not have had, it’s also lead to many, many ideas...’

‘Other partners that we can bring to the table that are not currently partners...’

Figure 68: Driving Organizational Goodwill Interview Observations (UNICEF Interviews, 2016)

A number of informants referred to the work of the UNICEF Office of Innovation and UNICEF Innovation Fund

as ‘repositioning’ UNICEF as a ‘highly relevant, highly innovative future focused organisation’. Such

repositioning of the UNICEF brand based on a more diverse innovative value proposition can contribute

towards augmented mission delivery if it can be demonstrated to existing and new stakeholders.

Figure 69: Augmented Mission Delivery (Armstrong et al., 2009; Jobber, 2007 and Ashton, 2015)

Actual Mission Delivery UNICEF Country Programmes

Core Benefit UNICEF Mission

Meet Basic Needs, Advocate for Rights & Expand Opportunities

Augmented Mission Delivery UNICEF Innovation Fund

Start-ups & New Partners Open Source Communities of Problem Solvers


Page:110

Aaker (1997) defines brand personality as ‘the set of human characteristics associated with a brand’ and

identifies five dimensions with which to understand it. UNICEF traditionally can be considered as a sincere,

competent and rugged international development organisation tackling some of the biggest challenges to child

rights and welfare globally with a strong mandate from the United Nations. With the addition of new ways of

doing things such as the UNICEF Innovation Fund the organisation enhances the excitement and sophistication

measures which again if communicated widely to its stakeholder can be leveraged to attract new revenue

streams, partners and further advance its mission.

Figure 70: UNICEF Brand Personality, Existing and New Traits (Aaker, 1997)

UNICEF Brand Personality (Aaker (1997)

Competence

Sophistication

Excitement

Sincerity

Ruggedness

Down to Earth

Honest

Daring

Spirited

Imagination

Up to Date

Reliable

Intelligent

Successful

Upper Class

(Leading or Premium)

Tough

Existing Traits

New Traits

New Traits

Existing Traits

Existing Traits


Page:111

5. Discussion & Conclusion

5.1 Synthesis of Findings

This research has been conducted with the aim of understanding the potential value that venturing may create

in international development with a specific focus on the UNICEF Innovation Fund. The Fund is a new approach

which faces many challenges in its evolutionary phase of development including demonstrating to internal

stakeholders the value it can bring to the organisation. This study has researched and analysed the Fund on

four levels namely; (1) strategic, (2) individual initiative, (3) portfolio and (4) secondary effects level under the

following research question and objectives.

No. Level of Analysis Research Question and Objectives Revisited

1 Strategic Level General Research Question Research Question How can a large established international development organisation create value from a ventures approach to innovation such as that which is applied in UNICEF Innovation Fund?

2 Individual Initiative Level

Research Objective 1 Research suitable tools and methods to evaluate the investment potential of individual initiatives seeded by the UNICEF Innovation Fund and apply those approaches to two initiatives to demonstrate potential value to UNICEF's mission.

3 Portfolio Level Research Objective 2 Research suitable frameworks to evaluate the UNICEF Innovation Fund as a portfolio and demonstrate potential value of the Fund considering the majority of initiatives are expected to fail with only a limited few expected to be successful.

4 Secondary Effect Level

Research Objective 3 Investigate the potential secondary benefits that UNICEF Innovation Fund can bring to the organisation through doing new things, taking risks and accepting failure so as to capture learning benefits and reduce under-valuation of its innovation initiatives and processes.

Table 30: Four Levels of Research and Analysis


Page:112

Strategic Level 1: The Venturing Approach to Value Creation

The UNICEF Office of Innovation’s management through three sub-divisions which ‘scan the horizons’ (UNICEF

Interviews, 2016) is consistent with business best practice such as McKinsey’s Three Horizons of Growth

Framework (Baghai, et al., 1999). The Ventures Horizons 2 provides a space for the UNICEF Innovation Fund to

invest early stage ‘smart capital’ in a ‘failure tolerant mind-set outside the parent risk averse culture’ (Lerner,

2013).

The mission model (Figure 24) is strong considering the purpose, strategy, values and behaviour reinforce each

other. The clear hypothesis that ‘in order to solve the biggest problems... we need to invest in places that have

those problems… connect those problem solvers to each other… open source … [and] if one does not work we

learn from it’ (UNICEF Interviews, 2016) is compelling and the findings herein suggest it should be supported.

The vision of evolving to multiple funds and a private equity portfolio is suitably aspirational, an essential

ingredient of leading business innovators (McKinsey, 2014 and 2015). The evolutionary phase of strategy

development (Sniukas et al., 2016) is being managed prudently in the short term as UNICEF Ventures learn the

maximum at the least possible cost (Ries, 2011 and McGrath, 2010) prior to mobilising additional resources

which have been identified in the strategic staircase (Hay and Williamson, 1991; Figure27).

The corporate venturing model (Figure 29) is a powerful approach with three types of value created even in

failure namely; (1) IP Stack, (2) the building of an open source community of problem solvers and (3) learning

benefits. If supported with the necessary resources the model has the potential to be sustainable in the long

term and provide a regular ‘floodplain’ of new solutions that create (4) impact at scale and (5) long term

growth. The model has attracted new revenue streams and the pooled funding mechanism allows them to be

efficiently leveraged.

Individual Initiative Level 2: Investment Potential of Individual Initiatives

The individual initiative evaluation framework applied herein acknowledges key concerns raised by

interviewees such as (1) not to cconcertize initiatives with frameworks such as theories of change (2) create

agile and flexible approaches that can deal with ambiguity and change, (3) capture data in real-time to

facilitate learning and decision making and (4) design with the user to ensure the product meets the

customer’s needs. These principals are satisfied through a Developmental Evaluation approach (Patton, 2015),

combined with an adapted evaluative framework based on Paulson (2007) together with the Value Proposition

Canvas (Osterwalder and Pigneur, 2010 and Stategizer, 2016). In applying the evaluation framework, positive

results were recorded for the two individual initiatives investigated namely Care for Children (Cambodia) and

MomConnect (South Africa) suggesting the potential to create value from such initiatives is strong.


Page:113

Portfolio Level 3: Evaluation of UNICEF Innovation Fund Portfolio

Research into the portfolio level concludes that evaluation should not discourage experimentation and that

traditional evaluation methods are inappropriate for a portfolio of innovation initiatives. Evaluations should

deal with issues such as value maximisation/return, strategic alignment, project selection/prioritisation, risk

appetite, balance/mix of projects, rate of attrition, interdependencies and secondary benefits. The challenge

of balancing innovation-specific metrics versus standardized metrics remains and is ripe for further study. The

Paulson et al. (2007) innovation portfolio evaluation tool is applied in this study to assess the portfolios

contribution to value creation. The current portfolio of initiatives scored satisfactorily (66%) and the portfolio

health scored well (73%).

The Innovation risk matrix (Day, 2007) has been applied to establish where initiatives fall on the spectrum of

risk and to gauge if they are sufficiently new to the organisation. The current portfolio of initiatives typically

adapts existing technology within programme countries. The majority of initiatives straddle the 60% relative

risk or probability of failure line (Day, 2007) which can be considered in the adjacent innovation category

defined by Nagji and Tuff (2012) as ‘expanding from existing business into new to the company business’.

The portfolio is expected to embrace further risk as additional start-ups are seeded, more consistent with the

failure rate anticipated in the corporate venturing model (Figure 29). The additional start-ups also have the

potential to move the portfolio towards a transformational innovation mix defined by Nagji and Tuff (2012) as

‘developing breakthroughs and inventing things for markets that don’t exist yet’.

Secondary Effect Level 4: Investigation into Potential Secondary Benefits

Evidence of all eight potential secondary benefits identified at the outset was uncovered through multiple

methods including interviews and desk based research. Organisational learning received the highest positive

feedback (90%) by interviewees with the benefit of ‘failing smart’ through learning considered a primary

benefit of the Fund (UNICEF Interviews, 2016).

The network of relationships through the open innovation model allowing ‘the world to become your lab’

(Chesbrough cited by Haas, 2016) was the next mostly highly noted benefit (80%). Driving organisational

goodwill received equal positive feedback (80%), where the benefit of transforming UNICEF's ‘public

positioning’ as a ‘highly relevant, highly innovative... future focused’ organisation was emphasised (UNICEF

Interviews, 2016).

As all secondary benefits were noted, with many identified as primary benefits, the findings make it clear that

UNICEF cannot afford not to invest a small percentage of its revenue in order to reap such benefits. As such,

any value assessment of the UNICEF Innovation Fund or initiatives needs to consider such secondary benefits

in order not to under-evaluate the Fund.


Page:114

5.2 Managerial Implications

Key Findings and Implications

On an academic level this project has sought to contribute to the understanding of the potential value that

corporate venturing creates in international development with a specific focus on the UNICEF Innovation Fund

across four core levels of analysis. In addition to this understanding there are numerous key findings that have

management implications going forward.

The managerial implications at strategic level are outlined below. The primary importance of the three

horizons structure, support of increased investment to grow the Fund and additional technical and business

management personnel required over the longer term are identified as the main strategic implications.

No. Key Findings Managerial Implication

1. Venturing Strategy and Value Creation

a. UNICEF Office of Innovation structures and manages innovation across three horizons: Futures, Ventures and Scale.

Managing the horizons is consistent with business best practice and is essential to provide a space for UNICEF Innovation Fund to operate in a failure tolerant risk embracing environment.

b. The Funds purpose based on the clear hypothesis that in order to solve the world’s biggest problems you need to invest in places that have those problems and connect communities of problem solvers.

The hypothesis is logical and takes a bottom up economic stance to solutions by investing in start-ups. If early evidence supports the hypothesis increased investment beyond the current low level will be necessary to grow the Fund and maximise the value it creates.

c. The new UNICEF Innovation Fund corporate venturing business model is a first in the UN and leverages options orientated investment strategies. Failure of circa 90% is expected and accepted. IP stack, community building and learning will accrue even from failed investments (continuous asset building). A small percentage of highly successful initiatives are expected to create significant value/impact at scale.

Close support to investees in the form of technical and business management guidance is necessary to leverage the value created even from failures and ensure assets in the form of IP stack, communities and learning are built. The small percentage of highly successful initiatives should be tracked further into the scale phase in order to demonstrate the value of those successes to current investors but more importantly to market the Fund attracting new investors and building growth.

d. The five year vision of growing and evolving to multiple funds between $10-20m and to develop a private equity portfolio would again be a first for a UN organisation.

The vision is suitably aspirational and challenges the traditional way of delivering the wider organisation's mission. Significant additional resources will be required to attain the vision some of which have been identified in the strategic staircase.

e. UNICEF Innovation Fund investment intensity as a percentage of UNICEF revenue is significantly lower than top business innovators.

The balance between maximisation of programme expense versus long term investment in innovation needs to be reviewed. An increase in the latter would appear to be highly necessary in order to align with the strategic plan 2014-2017 that places innovation as one of seven implementation strategies. UNICEF National Committees and other fund raising divisions need to market the Office of Innovation and UNICEF Innovation Fund to leverage higher percentages of investment. Benefits of the Fund such as driving organisational goodwill by transforming UNICEF's ‘public positioning’ as a ‘highly relevant, highly innovative... future focused’ organisation need to be leveraged.

Table 31: Strategic Level Key Findings and Managerial Implications


Page:115

At individual initiative level, the potential value that could be created by initiatives such as Care for Children

and MomConnect has been demonstrated. The main managerial implication at this level is how to support

initiatives further including the small percentage of initiatives that after twelve months appear to have the

potential to create significant value and impact at scale. Initiatives after twelve months may also require

substantially more than $100,000 to ‘cross the chasm’ (Moore, 1991).


2. Individual Initiatives Investment Potential and Value Creation

a. Seelos and Mair (2016) posit that innovation alone does not create impact but rather innovation + scale = impact. Putting in place clear paths to scale for the small percentage of highly successful initiatives is an important part of the UNICEF Innovation Fund strategy that would benefit from further development. ‘Crossing the chasm’ is identified by Moore (1991) as one of the main challenges for start-up ventures.

UNICEF Ventures currently plans to link the seeded organisations to its large network and scale investor partners. Access to UNICEF’s network is indeed valuable and offers a solution but further development of this strategy in the next twelve months would aid the transition or otherwise for the first round of private sector seeds. Moore (1991) advocates a focus on niche market first to provide a platform to cross the chasm. Such strategies would benefit from further study.

Table 32: Individual Level Key Findings and Managerial Implications

UNICEF Ventures currently intends to link its seeded organisation's to its scale investor partners and in the

short term this presents the most immediate solution. Considering the vision for the Fund to progress to

multiple funds and a private equity portfolio further emerging solutions to this challenge will need to be

developed as the Fund evolves and presents opportunities for further research.

The Revised Technology Adoption Life Cycle – Crossing the Chasm

Figure 71: The Revised Technology Adoption Cycle – Crossing the Chasm (Moore, 1991)

Innovators Early Adopters Early Majority Late Majority Laggards

Chasm

UNICEF Ventures & UNICEF Innovation Fund

Scale Investors

Moore (1991): To cross the chasm ‘target very specific niche markets

where you can dominate’ Levitt (1986): Whole Product Model – Generic, Expected,

Augmented & Potential Kim and Mauborgne (2005): ‘Create uncontested market

space’ (blue ocean)

Transition to Scale

Ries (2011) Lean Start-up Model: Test leap of Faith Assumptions (Value Creation Hypothesis & Growth Hypothesis) – MVP Minimum Viable Product Build-Measure-Learn

Potential Chasm Finance / Investment Options Potential Chasm Strategies

UNICEF Partners such as International Development Innovation Alliance (IDIA): Gates Foundation, Rockefeller Foundation, USAID,

Grand Challenges Canada, Global Innovation Fund, etc.

Early Stage Strategies


Page:116

Finding suitable means to evaluate the UNICEF Innovation Fund portfolio considering the time bound nature of

initiatives (12 months) remains a challenge. This challenge also extends to the tension of balancing innovation

specific metrics with standardised metrics which is currently being explored by UNICEF Ventures and its

partners which might also benefit from further research. Establishing suitable ways of stimulating interaction

and support between the communities of problem solvers within the portfolio is also a challenge currently

being addressed by workshops and mentoring initiatives (UNICEF, 2016l).


3. Frameworks to Evaluate the Portfolio and Demonstrate Potential Value

a. Currently there is no panacea for the challenge of innovation portfolio evaluation in the international development industry following interviews with a range of experts on the subject. The early stage of initiatives, the time bound nature of the Fund, resource constraints, aversion to rigorous evaluation for early stage initiatives (which would hinder experimentation) are important challenges for the development of portfolio management approaches.

Cooper (1997, 1997a) highlights three primary innovation portfolio management goals; maximizing portfolio value, achieving balance and the linking of portfolio to business strategy. Paulson et al. (2007) and Day (2007) tools help with the latter two and can also help with maximising portfolio value but application of the corporate venturing model and its selection process is seen as more important in this regard. This cross sectional study has provided a snapshot evaluation but further empirical research into the approaches used and others is necessary to conclude the most appropriate approach for the UNICEF Innovation Fund.

b. The challenge in portfolio evaluation of balancing the tension between innovation-specific metrics (individualised, bottom-up) at initiative level and standardised metrics (aggregated top-down) to establish the collective value at portfolio level remains.

Evaluation should not discourage experimentation and as such traditional evaluation has been concluded as inappropriate for the evaluation of a portfolio of innovation initiatives. A portfolio’s performance, especially an innovation Fund, is reflected in its diversity and thus standardized metrics may not be appropriate. Therefore the development of any evaluation tool must be adaptable to individual initiatives.

c. Resource, market or technological interdependencies between projects can be a major source of competitive advantage (Mathews, 2010). The portfolio displays interdependencies and synergistic benefits. For example there are several variations of U-Report and 3 of the 5 start-ups are based on interactive voice response (IVR) systems. Both examples demonstrate how the same technology can be leveraged for different applications. Start-up workshops have commenced and will induce cross collaboration among teams & innovations. Balance (e.g. portfolio areas and technologies) and diversity of innovations is crucial for long term success and this is an area where improvements should be targeted as the Fund grows.

UNICEF Innovation Fund is intended to allow the organisation's seeded share within a community and leverage one another’s capabilities while exploring opportunities. As the Fund grows, management will require greater resources, including technical, portfolio management, and business management expertise. A forum to allow the start-up community share ideas and build on each other’s solutions needs is being nurtured. The first ‘mentors meet’ (UNICEF, 2016) where the five start-ups were hosted in workshops, introduced to each other and also introduced to experts who can mentor on business models, rapid evaluation and technical solutions is an important first step. The UNICEF Innovation Fund website, open source sharing and online webinars can be leveraged further to this end.

Table 33: Portfolio Level Key Findings and Managerial Implications


Page:117

At secondary level it was noted that such effects are indeed primary benefits of UNICEF Innovation Fund.

Organisational learning is the most significant benefit noted and needs processes and systems to be put in

place to internalise the learning from external investments. The key challenge of merging open source

business models with value creation for children in a dual value creating sustainable model requires significant

support from the UNICEF Ventures team.


4. Potential Secondary Benefits

a. Secondary benefits have been noted as primary benefits by a number of interviewees but are not systematically tracked and demonstrated by the organisation.

Secondary benefits should be emphasised when reporting to investors in order not to under value the Fund. Due to the diversity of effects and resources required to continuously track them it would not appear to be feasible to track them all regularly in the short term. Any tracking that can be automated through real time data should be leveraged and others should as a minimum be demonstrated annually to current and potential investors.

b. Organisational learning was most noted (90%) and is one of the three continuous forms of value expected from all investments along with IP stack and community building. The challenge of putting in place a framework to capture this learning needs to be addressed.

Procedures are being developed to manage the newly seeded start-ups. Processes to capture learning benefits and capabilities attained need to be established. The lean-start-up approach was explored herein as a possible framework within which to meet this challenge (see Figure 53) but further empirical research is necessary to validate its suitability.

c. A further key challenge is how to internalise the learning from external start-ups to build capabilities inside UNICEF.

A study into the development of intrapreneurship programmes within UNICEF would add value including perhaps the UNICEF Office of Innovations biggest challenge, stimulating widespread organisational change and mainstreaming innovation across the wider organisation. The ability to acquire start-ups or take equity stakes is part of the UNICEF Innovation Fund long term vision which also requires many internal hurdles to be crossed.

d. A key challenge identified by the UNICEF team is supporting the development of business models for new start-ups whose teams have stronger technical capabilities than business management capabilities.

Numerous variants of open source freemium business models were identified for possible adaptation in Table 29 and Figure 65. The key challenge remains merging those models with value creation for children in a dual value creating model. This challenge will require significant support from UNICEF Ventures and its ecosystem of experts and partners. The establishment of its first ‘mentors meet’ where start-ups met together in workshops and were advised by experts on business models and rapid evaluation is a significant first step in this regard which will require further support.

Table 34: Secondary Level Key Findings and Managerial Implications


Page:118

Innovation Investment Intensity

Table 35 below compares the current UNICEF Innovation Fund investment intensity with the R&D investment

intensity of the Top 10 business innovators (Strategy& PwC, 2016). The Funds investment intensity is

significantly lower than R&D in the organisations benchmarked. This suggests the potential to significantly

increase investment in the UNICEF Innovation Fund in line with the vision captured in the strategic staircase

(Figure 27). Further benchmarking against Gates Foundation Grand Challenges programme and USAIDs

Development Innovation Ventures is shown in Appendix 5 which also demonstrates the relatively small size of

the Fund.

If the ‘identification and promotion of innovation’ is one of seven implementation strategies identified in

UNICEF's 2014-2017 Strategic Plan (UNICEF, 2013 & Appendix 2) then a higher investment intensity along with

other additional resources are necessary to support the UNICEF Office of Innovation going forward.

Rank Company R&D Spending ($Bn)

Revenue ($Bn)

Intensity (% Revenue)

1 Apple 8.1 231.4 3.5%

2 Alphabet 12.3 75.0 16.4%

3 3M 1.8 30.27 6%

4 Tesla Motors 0.7 4.05 17.28%

5 Amazon 12.5 106.8 11.7%

6 Samsung 12.7 177.4 7.2%

7 Facebook 4.8 17.93 26.8%

8 Microsoft 12.0 176.4 12.9%

9 General Electric 4.2 117.4 3.58%

10 IBM 5.2 81.7 6.36%

- UNICEF &UNICEF Innovation Fund 0.0112 5.0 0.224%

Table 35: UNICEF Innovation Fund Investment Intensity Compared with R&D Investment Intensity of the Top 10 Business Innovators (Strategy& PwC, 2016 and UNICEF, 2016j)

Innovation Opportunity Cost

Although the case for long term potential value creation by the UNICEF Innovation Fund has been found in this

study to be very strong the short term view of opportunity cost by external parties was a concern raised by the

authors. The challenge of marketing investment in initiatives, which if venture capital ratios hold true will fail

most of the time with some significant long term successes, rather than investing in proven core programmes

that deliver results today is not small in an industry where metrics such as the percentage expenditure on

administration are heavily scrutinised. One key informant gave the following explanation:


Page:119

‘When you talk about children you don’t have a lot of time... The majority of children that die... are before the

age of five... 37% of those under five children die within thirty days... so we have to accelerate the trajectory

we are on and innovations give us the ability and opportunity to accelerate actions that add value. So we’re

not just innovating for the sake of innovating, we’re looking at that very real challenge.... Related to that are

the opportunities... [we need to improve] the quality of education. Again we don’t have a lot of time. By the

time they are 6-10 if we have not improved the quality of education it’s hard to catch up... So we need to

accelerate that trajectory’ (UNICEF Interviews, 2016).

This was perhaps one of the most compelling insights into the value of innovation in international

development and the UNICEF Innovation Fund.

5.3 Limitations and Avenues for Future Research

The six month timeline and part time study limited the research to that which is presented herein. The remote

proximity of the students to the client also limited the extent of engagement although regular conference calls

and emails were maintained. Although care was taken to be impartial the group are strong supporters of

UNICEF’s mission which can affect the conclusions. Working closely with the client, the group formed a

collegial working relationship which can also affect the framing of analysis and final output. The interviewee

list was drawn up by UNICEF Ventures which could skew the findings but the list contained those with most

knowledge of the UNICEF Innovation Fund and therefore the sample is deemed appropriate. The start-up

nature of the Fund also limited the extent to which concepts could be tested in this evolutionary stage.

The main avenues for further research include: (1) the challenges of resource allocation; (2) methods to

capture and internalise organisational learning; (3) business model innovation that provides dual value for

commercial start-ups and child rights; (4) appropriate strategies to ‘cross the chasm’ to scale; (5) portfolio

management approaches to address the time bound nature of the Fund and balancing innovation-specific

metrics and standardized metrics; (6) development of intrapreneurship programmes to stimulate

organisational change, cultural change and mainstream innovation across the wider organisation and the

related (7) internalisation of externally leveraged capabilities either through diffusion or acquisition.

5.4 Conclusion

This multi-method qualitative case study has assessed how UNICEF can create value from corporate venturing

through the UNICEF Innovation Fund and has analysed the investment vehicle on four levels namely, strategic,

individual, portfolio and secondary. The evidence uncovered suggests that the corporate venturing approach

has the potential to create value and gives UNICEF an efficient mechanism to make multiple small risk capped

bets in new initiatives. Creating a space for such a model in the Horizon 2 ‘creation of viable options’ (Baghai,

et al., 1999) is a forward looking strategy that could transform the way the organisation delivers its mission in

the long term. We hope this project can aid the understanding of such approaches in the international

development context. Finally we hope this report can be of some benefit to our client as they continue in their

most important of missions.


Page:120

References

Aaker, J. (1997) ‘Dimensions of Brand Personality’. Journal of Marketing Research, August 1997.

Agrawal (2014) How Companies can Attract the Best College Talent. March 17, 2014. Harvard Business Review. Available at: https://hbr.org/2014/03/how-companies-can-attract-the-best-college-talent (Accessed: 05 November 2016)

Amatullo, M.V. (2015) Design attitude and social innovation: Empirical studies of the return on design: Innovation by Design at UNICEF: An Ethnographic Case Study. PHD. Case Western Reserve University. Available at: https://drive.google.com/file/d/0B4VJMPZTikBga3BBVWZQUkQzOHM/view?pref=2&pli=1 (Accessed: 23 April 2016)

AMBS (2015) Entrepreneurship and Innovation Management: Study Guide. Manchester: Alliance Manchester Business School.

Anderson, E.G. and Joglekar, N.R. (2005) A Hierarchical Product Development Planning Framework. Production & Operations Management, 14, 344–61.

Ashton (2015) ‘Marketing: Study Guide’ Alliance Manchester Business School.

Baghai, M., Coley, S. and White, D. (1999) The Alchemy of Growth: Practical Insights for Building the Enduring Enterprise. London: Orion Business, The Orion Publishing Group Ltd.

Bain (2013) Taking the Measure of Innovation Performance. Available at: http://www.bain.com/publications/articles/taking-the-measure-of-your-innovation-performance.aspx (Accessed: 04 July 2015).

Baines, P., Fill, C. and Page, K. (2011) Marketing. 2nd

edn. Oxford: Oxford University Press.

Banke-Thomas, A. O., Madaj, B., Charles, A., & van den Broek, N. (2015). Social Return on Investment (SROI) methodology to account for value for money of public health interventions: a systematic review. BMC Public Health, 15, 582. Available at: http://www.ncbi.nlm.nih.gov/pubmed/26099274 (Accessed: 03/01/16)

Barahona, C. (2010) Randomised Control Trials for the Impact Evaluation of Development Initiatives: A Statistician’s Point of View, p. 4, 7 Available at: https://www.bioversityinternational.org/fileadmin/_migrated/uploads/tx_news/Randomised_control_trials_for_the_impact_evaluation_of_development_initiatives__A_statistician%E2%80%99s_point_of_view_1391.pdf (Accessed: 25 August 2016)

Bassi, A., & Vincenti, G. (2015). Toward a new metrics for the evaluation of the social added value of social enterprises. CIRIEC - Espana, (83), 9-42. Available at: http://search.proquest.com/openview/a43e254167c7ed7a80a5f4fc3707223a/1?pq-origsite=gscholar (Accessed: 03/01/16).

Baumbusch, J. (2010). Semi‐structured interviewing in practice‐close research. Journal for Specialists in Pediatric Nursing, 15(3),255–258.

BCG (2015) The Most Innovative Companies 2015: Four Factors That Differentiate Leaders. Available at: https://media-publications.bcg.com/MIC/BCG-Most-Innovative-Companies-2015-Nov-2015.pdf (Accessed: 16 April 2016).

Beesley, A., Gulemetova, M., DiFuccia, M. (2009) Georgia Governor’s Office of Student Achievement Innovation Fund SROI Analysis, Final Report for IMPAQ International, LLC. Available at: https://www.impaqint.com/sites/default/files/files/Georgia%20SROI%20Final%20Report%20June%202015.pdf (accessed: 03 January 2016)

Behfar, K., Peterson, R., Mannix, El., Trochim, W (2008) The Critical Role of Conflict Resolution in

Benjamin L (2012) Nonprofit organizations and outcome measurement: from tracking program activities to focusing on frontline work. American Journal of Evaluation 33(3): 431–47.

https://hbr.org/2014/03/how-companies-can-attract-the-best-college-talent

https://drive.google.com/file/d/0B4VJMPZTikBga3BBVWZQUkQzOHM/view?pref=2&pli=1

http://www.bain.com/publications/articles/taking-the-measure-of-your-innovation-performance.aspx

http://www.ncbi.nlm.nih.gov/pubmed/26099274

https://www.bioversityinternational.org/fileadmin/_migrated/uploads/tx_news/Randomised_control_trials_for_the_impact_evaluation_of_development_initiatives__A_statistician%E2%80%99s_point_of_view_1391.pdf



http://search.proquest.com/openview/a43e254167c7ed7a80a5f4fc3707223a/1?pq-origsite=gscholar

https://media-publications.bcg.com/MIC/BCG-Most-Innovative-Companies-2015-Nov-2015.pdf

https://www.impaqint.com/sites/default/files/files/Georgia%20SROI%20Final%20Report%20June%202015.pdf

https://www.impaqint.com/sites/default/files/files/Georgia%20SROI%20Final%20Report%20June%202015.pdf


Page:121

Biswas, K. et al. 2010. Social Return on Investment: Chaha Programme, Hove: International HIV/AIDS Alliance. Available at: https://www.bond.org.uk/data/files/SROI_CHAHA_report_04.02.11.pdf (accessed: 03 January 2016)

Bloom, L. and Faulkner, R. (2016) Innovation Spaces: Lessons from the United Nations. Third World Quarterly. Vol. 37, No. 8, pp. 1371-1387

Bowen, G.A., (2009) ‘Document analysis as a qualitative research method’, Qualitative research journal, 9(2), pp. 27-40.

Brest, P. and Born, K., (2013) ‘When can impact investing create real impact’, Stanford Social Innovation Review, 11(4), pp. 33-48.

Bubb, S. (2010) ‘From third sector to pole position; how the third sector is leading innovation in public services’, International Journal of Leadership in Public Services, 6(4), pp.38-41. Available at: http://www.emeraldinsight.com/doi/abs/10.5042/ijlps.2010.0636 (Accessed: 11/01/16)

Campbell, A. and Yeung, S. (1991), “Creating a Sense of Mission”, Long Range Planning, Vol. 24 No. 4,pp. 10-20.

Campbell, A., Devine, M. and Young, D. (1990) A Sense of Mission (Ashridge Management). London: Economist Books.

Canales, R., Cannon, C., Fabian, C., Fabricant, R., Kochi, E., and Rabison, R. (2014) ‘DIY innovation: Creating an innovation capability within your organisation’, rotmanmagazine.ca, Fall 2014, pp.66-71. Available at: https://www.rotman.utoronto.ca/Connect/Rotman-MAG/Back-Issues/2014/Back-Issues---2014/Fall-2014---The-Balancing-Act/Fall-2014-Free-Feature-Article---DIY-Innovation (Accessed 29 July 2016)

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J. and Neville, A.J., (2014) ‘The use of triangulation in qualitative research’, In Oncology nursing forum, 41(5).

CGD (2006) ‘When Will We Ever Learn? Improving Lives through Impact Evaluation’. Report of the Evaluation Gap Working Group. Washington, DC: CGD.

Chanaron, J.J. (2013) ‘Innovation Valuation: Guesswork or Formalized Framework? A Literature Review’, In Emerging Dimensions of Technology Management, pp. 17-33. Available at: http://link.springer.com/chapter/10.1007/978-81-322-0792-4_2 (Accessed: 11/01/16)

Chesbrough, H. (2004) ‘Managing open innovation’, Research-Technology Management, 47(1), pp.23-26. Available at: http://search.proquest.com/openview/7a1e934eb44ab01f9c313b7f0175f046/1?pq-origsite=gscholar (Accessed: 11/01/16)

Chesbrough, H. (2011), “The case for open services innovation: the commodity trap”, California Management Review, Vol. 53 No. 3,pp. 5-20.

Chesbrough, H. (2016) Lecture by Henry Chesbrough at Haas School of Business. Available at: https://twitter.com/HenryChesbrough (Accessed 01 November 2016)

Chesbrough, H. and Rosenbloom, R. (2002), “The Role of the Business Model in Capturing Value from Innovation: Evidence from Xerox Corporation’s Technology Spin-off Companies”, Industrial and Corporate Change, Vol. 11 No. 3,pp. 529-555.

Chesbrough, H. W. (2003). Open innovation: The New Imperative for Creating and Profiting from Technology. Boston, MA: Harvard Business School Press.

Chesbrough, H., Sohyeong, K., and Agogino, A. (2014), “Chez panisse: Building an open innovation ecosystem”, California Management Review, Vol. 56 No. 4, pp. 144-171.

Christensen, C. M. (1997) The Innovator’s Dilemma: When Technologies Cause Great Firms to Fail. Boston: Harvard Business School Press.

https://www.bond.org.uk/data/files/SROI_CHAHA_report_04.02.11.pdf

http://www.emeraldinsight.com/doi/abs/10.5042/ijlps.2010.0636

https://www.rotman.utoronto.ca/Connect/Rotman-MAG/Back-Issues/2014/Back-Issues---2014/Fall-2014---The-Balancing-Act/Fall-2014-Free-Feature-Article---DIY-Innovation

https://www.rotman.utoronto.ca/Connect/Rotman-MAG/Back-Issues/2014/Back-Issues---2014/Fall-2014---The-Balancing-Act/Fall-2014-Free-Feature-Article---DIY-Innovation

http://link.springer.com/chapter/10.1007/978-81-322-0792-4_2

http://search.proquest.com/openview/7a1e934eb44ab01f9c313b7f0175f046/1?pq-origsite=gscholar

http://search.proquest.com/openview/7a1e934eb44ab01f9c313b7f0175f046/1?pq-origsite=gscholar

https://twitter.com/HenryChesbrough


Page:122

Cohen, L., Manion, L., & Morrison, K. (2007) Research Methods in Education. 6th Edition.

Conger, S. (2009). Social inventions. Innovation Journal, 14(2). www.innovation.cc/books/conger_

Cooper, R.G. (2014) What's Next?: After Stage-Gate. Research-Technology Management, 57(1), pp.20-31.

Cooper, R.G. (2014) What's Next?: After Stage-Gate. Research-Technology Management, 57(1), pp.20-31.

Cooper, R.G. and Edgett, S. (2014) Portfolio Management for New Products: Picking The Winners. Stage-Gate International and Product Development Institute Inc. pp.3-6

Cooper, R.G. Edgett, S. and Kleinschmidt, E., 2001. Portfolio management for new product development: results of an industry practices study. R&D Management, 31(4), pp.361-380.

Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J. (1998) Best Practices for Managing R&D Portfolios. Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J.(1999) New Product Portfolio Management: Practices and Performance. Journal of Product Innovation Management, 16, 333–50. Cooper, R.G., Edgett, S.J. and Kleinschmidt, E.J.(2000) New Problems, New Solutions: Making

Cordery, C. and Sinclair, R. (2013), “Measuring performance in the third sector”, Qualitative Research in Accounting &Management , Vol. 10 Nos 3/4, pp. 196-212. Available at: http://www.emeraldinsight.com/doi/abs/10.1108/QRAM-03-2013-0014 (accessed: 03 January 2016)

Danila, N. (1989) Strategic Evaluation and Selection of R&D Projects. R&D Management, 19, 47–62. Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Thousand Oaks: Sage. (p126-127)

Day, G.S. (2007) ‘Is it real? Can we win? Is it worth doing? Managing risk and reward in an innovation portfolio’, Harvard Business Review, June 2002, pp.110-120.

Dean, H. (2015) Building innovation capability in an intergovernmental aid organization: A case study of UNICEF’s Innovation Unit. Available at: http://www.unicefstories.org/wp-content/uploads/2013/08/THESIS-Harald-Dean5b15d5b25d.pdf (Accessed: 21 April 2016)

Deloitte (2016) Deloitte Millenials Survey 2016. Available at: https://www2.deloitte.com/global/en/pages/about-deloitte/articles/millennialsurvey.html (Accessed: 05 November 2016)

Demirag, I. and Khadaroo, I. (2011) Accountability and value for money: a theoretical framework for the relationship in public -private partnerships. Journal of Management and Governance, 15 (2) (2011), pp. 271–296

Denzin, N.K. (1978) Sociological Methods: A Sourcebook. New York: McGraw Hill.

Diamond, M.J., 2005. Establishing a performance management framework for government (No. 5-50). International Monetary Fund. Dozois, E., Langlois M., & Blanchet-Cohen, N. (2010). DE 201: A practitioner’s guide to developmental evaluation. Montreal, Canada. J.W. McConnell Foundation and the International Institute for Child Rights and Development.

Drucker, P.F. (1954), The Practice of Management, William Heinemann, London

Dushnitsky, G. and Lenox, M.J., (2006) ‘When does corporate venture capital investment create firm value?’, Journal of Business Venturing, 21(6), pp. 753-772.

Ebrahim, A. 2005. Accountability Myopia: Losing Sight of Organizational Learning. Nonprofit and Voluntary Sector Quarterly, 34(1): 56-87. Available at: http://nvs.sagepub.com/content/34/1/56.abstract (Accessed 15/01/16)

http://www.innovation.cc/books/conger_

http://www.emeraldinsight.com/doi/abs/10.1108/QRAM-03-2013-0014

http://www.unicefstories.org/wp-content/uploads/2013/08/THESIS-Harald-Dean5b15d5b25d.pdf

http://www.unicefstories.org/wp-content/uploads/2013/08/THESIS-Harald-Dean5b15d5b25d.pdf

https://www2.deloitte.com/global/en/pages/about-deloitte/articles/millennialsurvey.html


Page:123

Ebrahim, A. and Rangan, V. (2010). What Impact? A Framework for Measuring the scale and scope of social performance. University Of California, Berkeley, Vol.56 (3), 2014.

Ebrahim, A., &Rangin, V. (2010). The limits of nonprofit impact: A contingency framework for measuring social performance. St. Louis: Federal Reserve Bank of St Louis. Available at: http://www.hbs.edu/faculty/Publication%20Files/10-099.pdf (accessed on 19 December, 2015)

Eggers, J.P. (2012) All Experience Is Not Created Equal: Learning, Adapting, and Focusing in Product Portfolio Management. Strategic Management Journal, 33, 315–35.

Emerson, J., (2003) ‘The blended value proposition: Integrating social and financial returns’, California management review, 45(4), pp. 35-51.

Emni, A., Ozlem, E., Maja, K., Ilan, R., & Florian, S. (2011). Value for money: Current approaches and evolving debates. London School of Economics. Available at: https://www.bond.org.uk/data/files/LSE_Value_for_Money__Current_Approaches_and_Evolving_Debates.pdf (Accessed 15/01/16)

Fabian, C & Grote, S. (2016) UNICEF Innovation Diagram. Available at: https://www.dropbox.com/s/nc8c59qiz8gbu1n/for_zack.pptx?dl=0 (Accessed: 14 June 2016) (Note: Further update received from Grote, S. via email on 17 July 2016)

Flick, U. (1992) Triangulation Revisited: Strategy of Validation or Alternative?. Journal for the Theory of Social Behaviour Vol.22:2 pp. 176-177. Available at: http://onlinelibrary.wiley.com/doi/10.1111/j.1468-5914.1992.tb00215.x/abstract (Accessed 18/01/16)

Flick, U. (2013) The SAGE Handbook of Qualitative Data Analysis, e-book, accessed 31 July 2016, <http://manchester.eblib.com/patron/FullRecord.aspx?p=1707694>.

Florida, R.L. and Kenney, M., (1988) ‘Venture capital-financed innovation and technological change in the USA’, Research Policy, 17(3), pp. 119-137.

Foster, W., (2008) ‘Money to grow On’, Stanford Social Innovation Review, Vol. 6(4), p. 50.

Gompers, P. and Lerner, J., (2000) ‘The determinants of corporate venture capital success: Organizational structure, incentives, and complementarities’, Concentrated corporate ownership, pp. 17-54, University of Chicago Press.

Greiling, D (2010) "Balanced scorecard implementation in German non‐profit organisations", International Journal of Productivity and Performance Management, Vol. 59 Iss: 6, pp.534 – 554.

Harris, I., Mainelli, M. & O'Callaghan, M. 2002, "Evidence of worth in not-for-profit sector organizations", Strategic Change, vol. 11, no. 8, pp. 399.

Harwood, R. (2016) The Open Innovation Methods Matrix – Part 1. Available at: http://www.100open.com/the-open-innovation-methods-matrix-part-1/ (Accessed 29 October 2016)

Hay, M. and Williamson, P. 1991, “Strategic Staircases: Planning the Capabilities Required for Success", Long Range Planning, vol. 24, no. 4, pp. 36 to 43.

Holtzman, Y (2014),"A strategy of innovation through the development of a portfolio of innovation capabilities", Journal of Management Development, Vol. 33 Iss 1 pp. 24 - 31

IDIA (2016). The IDIA Measuring Impact Working Group Document ‘Towards a Shared Impact Measurement Framework for Innovation’ (2016)

ILO (2014), Fooled by randomisation: why RCTs might be the real ‘gold standard’ for private sector development, The Lab Market Systems Development for Decent Work

Jagiello, K. (2016) Advanced Strategic Management (ASM): Full Workshop Slide Pack, Jagiello Associates for Alliance Manchester Business School, University of Manchester.

Jagiello, K. and Jagiello, J. (2015) ‘Strategic Management: The Strategic Management Course Manual’ Available at:

https://www.bond.org.uk/data/files/LSE_Value_for_Money__Current_Approaches_and_Evolving_Debates.pdf

https://www.bond.org.uk/data/files/LSE_Value_for_Money__Current_Approaches_and_Evolving_Debates.pdf

https://www.dropbox.com/s/nc8c59qiz8gbu1n/for_zack.pptx?dl=0

http://onlinelibrary.wiley.com/doi/10.1111/j.1468-5914.1992.tb00215.x/abstract

http://onlinelibrary.wiley.com/doi/10.1111/j.1468-5914.1992.tb00215.x/abstract

http://www.100open.com/the-open-innovation-methods-matrix-part-1/


Page:124

https://online.manchester.ac.uk/webapps/blackboard/content/listContent.jsp?course_id=_35079_1&content_id=_3383007_1 (Accessed: 20 May 2015)

Kanter, R., (1985) ‘Supporting innovation and venture development in established companies’, Journal of business venturing, 1(1), pp. 47-60.

Kaplan, R.S. (2001) ‘Strategic Performance Measurement and Management in Non-profit Organizations’, Non-profit Management and Leadership, Spring 2001, Vol. 11, No. 3, pp. 353-370.

Katy, B., Turgut, E., Holzmann, T. and Sailer, K., 2013. Innovation intermediaries: a process view on open innovation coordination. Technology Analysis & Strategic Management, 25(3), pp.295-309.

Keizer, J.A. and Halman, J.I., 2007. Diagnosing risk in radical innovation projects. Research-Technology Management, 50(5), pp.30-36.

Kester, L., Griffin, A., Hultink, E.J. and Lauche, K., 2011. Exploring Portfolio Decision‐Making Processes. Journal of Product Innovation Management,28(5), pp.641-661.

Khurshed, A., Espenlaud, S., Strong, N., Garrett, I., Goa, N., De Cesari, A. (2014) Corporate Finance Study Guide and workbook. Manchester business School.

King, J., McKegg, K., Oakden, J., and Wehipeihana, N. (2013). Rubrics: A method for surfacing values and improving the credibility of evaluation. Journal of MultiDisciplinary Evaluation, 9(21), 11–20.

King, N.(2004) ‘Chapter 2 Using interviews in qualitative research’, Essential guide to Qualitative Methods in Organisational Research. Edited by Cassell C. and Symon G. London: Sage Publications

Klingebiel, R. and Rammer, C., 2014. Resource allocation strategy for innovation portfolio management. Strategic Management Journal, 35(2), pp.246-268. London: Routledge Falmer

Koen, P.A., 2004. The fuzzy front end for incremental, platform and breakthrough products and services. PDMA Handbook, pp.81-91.

Kressel, H. and Winarsky, N. (2015), ‘Why the Fail-Fast Approach Isn’t Right for Breakthrough Ventures’, Harvard Business Review, Vol. number unavailable, [Online]. Available at: https://hbr.org/2015/11/why-the-fail-fast-approach-isnt-right-for-breakthrough-ventures (Accessed: 25 October, 2016).

Krlev, G., Bund, E. and Mildenberger, G. (2014) ‘Measuring what matters—Indicators of social innovativeness on the national level’, Information Systems Management, 31(3), pp.200-224. Available at: https://www.researchgate.net/publication/264090717_Measuring_What_Matters-Indicators_of_Social_Innovativeness_on_the_National_Level (Accessed: 11/01/16)

Kroeger, A, & Weber, C 2014, 'Developing A Conceptual Framework For Comparing Social Value Creation', Academy Of Management Review, 39, 4, pp. 513-540, Business Source Alumni Edition, EBSCOhost. Available at: https://www.researchgate.net/publication/276183507_Developing_a_Conceptual_Framework_for_Comparing_Social_Value_Creation (Accessed 15/01/16)

Kurzweil, R. (2001) The Law of Accelerating Returns. Available at: http://www.kurzweilai.net/the-law-of-accelerating-returns (Accessed: 29 October 2016)

Lacerda, F. M., Martens, C. D. P. and Freitas, H. M. R (2016) A Project Portfolio Management model adapted to non-profit organizations. Project Management Research and Practice

Lecy, J., Schmitz, H. and Swedlund, H. (2012), “Non-governmental and not-for-profit organizational effectiveness: a modern synthesis”, Voluntas: International Journal of Voluntary and Nonprofit Organisations, Vol. 23 No. 2, pp. 434-457. Available at: http://link.springer.com/article/10.1007/s11266-011-9204-6 (Accessed 15/01/16)

Lerner, J., (2013) ‘Corporate venturing’, Harvard Business Review, 91(10), pp. 86-94.

https://online.manchester.ac.uk/webapps/blackboard/content/listContent.jsp?course_id=_35079_1&content_id=_3383007_1

https://online.manchester.ac.uk/webapps/blackboard/content/listContent.jsp?course_id=_35079_1&content_id=_3383007_1

https://hbr.org/2015/11/why-the-fail-fast-approach-isnt-right-for-breakthrough-ventures

https://hbr.org/2015/11/why-the-fail-fast-approach-isnt-right-for-breakthrough-ventures

https://www.researchgate.net/publication/264090717_Measuring_What_Matters-Indicators_of_Social_Innovativeness_on_the_National_Level

https://www.researchgate.net/publication/264090717_Measuring_What_Matters-Indicators_of_Social_Innovativeness_on_the_National_Level

https://www.researchgate.net/publication/276183507_Developing_a_Conceptual_Framework_for_Comparing_Social_Value_Creation

https://www.researchgate.net/publication/276183507_Developing_a_Conceptual_Framework_for_Comparing_Social_Value_Creation

http://www.kurzweilai.net/the-law-of-accelerating-returns

http://www.kurzweilai.net/the-law-of-accelerating-returns

http://link.springer.com/article/10.1007/s11266-011-9204-6

http://link.springer.com/article/10.1007/s11266-011-9204-6


Page:125

Letts, C.W., Ryan, W. and Grossman, A., (1997) ‘Virtuous capital: What foundations can learn from venture capitalists’, Harvard business review, 75, pp. 36-50.

Levitt, T. (1986) ‘The Marketing Imagination’. New York: Free Press.

Liket, K.C., Rey-Garcia, M. and Maas, K.E., (2014) Why Aren’t Evaluations Working and What to Do About It A Framework for Negotiating Meaningful Evaluation in Nonprofits. American Journal of Evaluation, 35(2), pp.171-188. Available at: http://aje.sagepub.com/content/early/2014/03/18/1098214013517736 (Accessed 15/01/16).

Lindgardt, Z., Reeves, M., Stalk, G. & Deimler, M. S. (2009) Business Model Innovation: When the Game Gets Tough, Change the Game. Boston: The Boston Consulting Group. Available at: https://www.bcg.com/documents/file36456.pdf (Accessed: 2

nd September 2016)

Lovbrand, E., Pielke, R. and Beck, S. (2011) ‘A Democracy Paradox in Studies of Science and Technology’, Science, Technology, & Human Values, Vol. 36, No. 4 (July 2011), pp. 474-496.

MacMillan, C. I., Van Putten, B. A., Mcgrath, G. R., and Thompson, D. J., (2006) ‘Using Real Options Discipline for Highly Uncertain Technology Investments’, Research Technology Management, 49, 1, pp. 29-37, Jan/Feb 2016.

Martinsuo, M. & Killen, C. (2014) ‘Value management in project portfolios: identifying and assessing strategic value’, paper presented to the European Academy of Management, June 4-7, Valencia, Spain.

Mathews, S (2011) 'Innovation Portfolio Architecture, Part -- 2: Attribute Selection And Valuation', Research Technology Management, 54, 5, pp. 37-46, Business Source Premier, EBSCOhost

Mathews, S 2010, 'Innovation Portfolio Architecture', Research Technology Management, 53, 6, pp. 30-40, Business Source Premier, EBSCOhost, viewed 3 September 2016.

Matusik, S.F. and Fitza, M.A., (2012) ‘Diversification in the venture capital industry: leveraging knowledge under uncertainty’, Strategic Management Journal, 33(4), pp. 407-426.

McGrath, R. (2010) Business Models: ‘A Discovery Driven Approach.’ Long Range Planning. 43 (2010), pp. 247-261.

McKinsey (2014) The Seven Effective Traits of Effective Digital Enterprises. Available at: http://www.mckinsey.com/business-functions/organization/our-insights/the-seven-traits-of-effective-digital-enterprises (Accessed: 16 April 2016).

McKinsey (2015) The Eight Essentials of Innovation. Available at: http://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/the-eight-essentials-of-innovation (Accessed: 16 April 2016).

McInerney, P. (2014) ‘Networks of innovation: tracing the structures of flows among not-for-profit open source software foundations, philanthropies and intermediaries'. Voluntary Sector Review, Vol 5, No 1, pp. 47–73, (2014). Available at: http://www.ingentaconnect.com/content/tpp/vsr/2014/00000005/00000001/art00003 (Accessed: 01 January 2016)

Meagher, B (2015) Final Project Preparation Course Assessment 1: What do we mean by Triangulation? Why might triangulation be important in relation to researching an organization using a single case study approach?

Meagher, B (2016a) Advanced Strategic Management: Course Assessment 1: How can innovation capability at UNICEF be improved to add value to the organisation’s mission?

Meagher, B (2016b) Advanced Strategic Management: Course Assessment 3: UNICEF Innovation, Open Source Business Models & VC Portfolio.

http://aje.sagepub.com/content/early/2014/03/18/1098214013517736

https://www.bcg.com/documents/file36456.pdf

http://www.mckinsey.com/business-functions/organization/our-insights/the-seven-traits-of-effective-digital-enterprises

http://www.mckinsey.com/business-functions/organization/our-insights/the-seven-traits-of-effective-digital-enterprises

http://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/the-eight-essentials-of-innovation

http://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/the-eight-essentials-of-innovation

http://www.ingentaconnect.com/content/tpp/vsr/2014/00000005/00000001/art00003


Page:126

Meagher, B., McKeogh, D., Afifi, H., Miller, J. (2015) Final Project Preparation Course Assessment 2: Research Proposal: A Framework to assess value creation by innovation at UNICEF.

Meier, C. (2016) 5 Successful Business Models for Web-based Open-source Project. Available at: https://handsontable.com/blog/articles/5-successful-business-models-for-web-based-open-source-projects (Accessed 24 October 2016)

Melwani, A., Meagher, B., McKeogh, D., Afifi, H., Santiago-Miller, J., (2016) Venturing, Value Creation & the UNICEF Innovation Fund, Assessment 2, Interim Report.

Meskendahl, S. (2010) The Influence of Business Strategy on Project Portfolio Management and its Success – A Conceptual Framework. International Journal of Project Management, 28, 807–17.

Metcalfe, B. “Metcalfe’s Law: A network becomes more valuable as it reaches more users,” Infoworld, Oct. 2, 1995. See also the May 6, 1996 column, hhttp://www.infoworld.com/cgi-bin/displayNew.pl?/metcalfe/bm050696.htmi.

Micheli, P & Kennerley, M (2005) Performance measurement frameworks in public and non-profit sectors, Production Planning & Control, 16:2, 125-134, DOI: Available at: http://www.tandfonline.com/doi/abs/10.1080/09537280512331333039 (Accessed 15/01/16)

Moody, M., Littlepage, L. and Paydar, N. (2015), Measuring Social Return on Investment. Nonprofit Management and Leadership, 26: 19–37. Available at: http://onlinelibrary.wiley.com/doi/10.1002/nml.21145/pdf (Accessed 15/01/16)

Moore, G. A. (1991) ‘Crossing the Chasm: Marketing and Selling High-Tech Products to Mainstream Customers’. New York: Harper Collins.

Mulgan, G. (2016) Nesta: Good and bad innovation: What kind of theory and practice do we need to distinguish them? Available at: http://www.nesta.org.uk/sites/default/files/good_and_bad_innovation_by_geoff_mulgan.pdf (Accessed: 26 July 2016)

Nagji, B. and Tuff, G., (2012) Managing your innovation portfolio. Harvard Business Review, 90(5), pp.66-74.

Nichols J, Coldwell H, McIntosh B and Thornton G (2016) Towards a collective understanding of Social Cost Benefit and Value. The Journal of Strategy, Operations and Economics. Available at: https://www.researchgate.net/publication/241740717_Social_Return_on_Investment_SROI_and_Performance_Measurement (Accessed 15/01/16)

O'Connor, G.C. and DeMartino, R. (2006). Organizing for radical innovation: an exploratory study of the structural aspects of RI management systems in large established firms. Journal of Product Innovation Management, 23, pp. 475–497.

Ohr, R-C., and McFarhting, K. (2013) Managing Innovation Portfolios, from Innovation Management.Se, Innovation Management online concepts, https://www.innovationmanagement.se/2013/09/16/managing-innovation-portfoliosstrategic-portfolio-management/, Retrieved 20th July 2016

Osterwalder, A and Smith, A., (2016) Strategyzer.com, Value Proposition Tools, Available at: https://strategyzer.com/platform/resources (Accessed 18/09/16)

Osterwalder, A. and Pigneur, Y. (2010) Business Model Generation. Hoboken, New Jersey: John Wiley and Sons, Inc.

Parnell, B. (2016) More Corporations Are Looking to Corporate Venture Capital to Access Innovative Start-up Technologies Available [Online] at: http://www.rocketspace.com/ corporate-innovation/more- corporations-are-looking-to- corporate-venture-capital

https://handsontable.com/blog/articles/5-successful-business-models-for-web-based-open-source-projects

https://handsontable.com/blog/articles/5-successful-business-models-for-web-based-open-source-projects

http://www.tandfonline.com/doi/abs/10.1080/09537280512331333039

http://onlinelibrary.wiley.com/doi/10.1002/nml.21145/pdf

http://www.nesta.org.uk/sites/default/files/good_and_bad_innovation_by_geoff_mulgan.pdf

https://www.researchgate.net/publication/241740717_Social_Return_on_Investment_SROI_and_Performance_Measurement

https://www.researchgate.net/publication/241740717_Social_Return_on_Investment_SROI_and_Performance_Measurement

https://strategyzer.com/platform/resources

https://strategyzer.com/platform/resources

http://www.rocketspace.com/corporate-innovation/more-corporations-are-looking-to-corporate-venture-capital

http://www.rocketspace.com/corporate-innovation/more-corporations-are-looking-to-corporate-venture-capital


Page:127

Patton, M.Q. (2011) Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use (PDF e-Book). Guilford Press, 2011

Patton, M.Q. (2015) Developmental Evaluation Exemplars: Principles in Practice (EPUB e-Book). Guilford Press, 2015

Paulson, A.S., O'Connor, G.C. and Robeson, D., 2007. Evaluating radical innovation portfolios. Research-Technology Management, 50(5), pp.17-29.

Pawson, R. and Tilley, N., (1997) Realistic evaluation. Thousand Oaks, CA: Sage.

Platt, J., (1981) ‘Evidence and proof in documentary research: 1: Some specific problems of documentary research’, The Sociological Review, 29(1), pp. 31-52.

Polonsky, MJ, Landreth, s, McDonald, G (2016),"Perspectives on social impact measurement and non-profit organisations", Marketing Intelligence & Planning, Vol. 34 Iss 1 pp. 80 – 98

Putten, A. B. and MacMillan, I. A. (2004) ‘Making Real Options Really Work’, Harvard Business Review, December 2004, pp. 1-7.

Ramalingam, B. and Bound, K. (2016) Nesta: Innovation for International Development: Navigating the Paths and Pitfalls. Nesta. [Online]. Available at: http://www.nesta.org.uk/sites/default/files/innovation_in_international_development.pdf (Accessed: 30 April 2016).

Research Councils UK (2015), ‘Research Councils UK Common Principles on Data Policy’. Available at: http://www.rcuk.ac.uk/research/datapolicy/ (Accessed: 25 July 2016), Research Technology Management, 41, 20–33.

Ries, E. (2011) The Lean Start-up. New York: Crown Publishing Group, a division of Random House, Inc.

Ruff, K. & Olsen, S., (2016) The Next Frontier in Social Impact Measurement Isn’t Measurement at All, Why we need skilled analysts to improve social capital markets. Available at:

Sarasvathy, Saras D., What Makes Entrepreneurs Entrepreneurial?, Vol., pp. 1-9, Available at: http://ssrn.com/abstract=909038 (Accessed 23 September 2015).

Saunders, M., Lewis P. and Thornhill, A. (2009) ‘Research Methods for Business Students’. 5th edn. England: Person Education Limited.

Saunders, M., Lewis P. and Thornhill, A. (2012) ‘Research Methods for Business Students’. 6th edn. England: Person Education Limited.

Sculley, S. (2014) Moonshot: Game-changing Strategies to build Billion-Dollar Businesses. New York: Rosetta Books.

Seelos, C. and Mair, J. (2016), ‘When Innovation Goes Wrong’, Stanford Social Innovation Review, Fall 2016, pp. 26-33.

Shapiro, C. and Varian H. R. (1999). Information Rules: A Strategic Guide to the Network Economy. Boston, MA: Harvard Business School Press.

Sillanpää, V (2013) "Measuring the impacts of welfare service innovations", International Journal of Productivity and Performance Management, Vol. 62 Iss: 5, pp.474 – 489

Smaje, C., (1997), Not Just a Social Construct: Theorizing Race and Ethnicity, Sage Social Science, Vol 31 No. 2, 307-327, Available at: http://soc.sagepub.com/content/31/2/307.short (Accessed 21/01/16)

Sniukas, M., Lee P. and Morasky, M. (2016) The Art of Opportunity: How to build growth and ventures through strategic innovation and visual thinking. Hoboken, New Jersey: John Wiley and Sons, Inc.social_inventions1_09232009min.pdf. Available

http://www.nesta.org.uk/sites/default/files/innovation_in_international_development.pdf

http://www.rcuk.ac.uk/research/datapolicy/

http://ssrn.com/abstract=909038


Page:128

at: http://www.innovation.cc/books/conger_social_inventions1_09232009min.pdf (Accessed: 25 August 2016)

Soy, S. (1997) The Case Study as a Research Method. Uses and Users of Information -- LIS 391D.1 start-ups [Online]. Available at: http://bit.ly/InnovationFundPressRelease. (Accessed: 13 November 2016)

Stilgoe, J., Owen, R., and Macnaghten, P. (2013) ‘Developing a Framework for Responsible Innovation’, Research Policy, Vol. 42, pp. 1568-1580.

Stirling, A. (2010) ‘Keep it complex: When knowledge is uncertain, experts should avoid pressures to simplify their advice’, Nature, Vol. 468, pp. 1029-1031.

Stokes D. and Bergin R. (2006) ‘Methodology or “methodolatry”? An evaluation of focus groups and depth

Stoten, D. W., (2013) NEETs a case study in addressing the issues relating to disengaged youth in East Cleveland Vol. 56 No. 5, 2014 pp. 467-480 Emerald Group, Available at: www.emeraldinsight.com/0040-0912.htm (Accessed 24/01/16)

Strategy& PwC (2016) Global Innovation 1000: Software-as-a-Catalyst. Strategy+Business Issue 85 Winter 2016. Available at: http://www.strategy-business.com/media/file/sb85_16409_Global_Innovation_1000_SoftwareAsCatalyst.pdf (Date Accessed 31 October 2016)

Strategy& PwC (2016) The Global Innovation 1000: The Top Innovators and Spenders. Available at: http://www.strategyand.pwc.com/innovation1000#/tab-2016|VisualTabs1 (Date Accessed 31 October 2016)

Teece, D. J. (2007). Explicating dynamic capabilities: The nature and microfoundations of (sustainable) enterprise performance. Strategic Management Journal .28 (13), 1319 - 1350

Terwiesch, C, & Ulrich, K 2008, 'Managing The Opportunity Portfolio', Research Technology Management, 51, 5, pp. 27-38, Business Source Premier, EBSCOhost, viewed 3 September 2016.

The University of Manchester, (2016) ‘Research Ethics’. Available at: http://www.manchester.ac.uk/research/environment/governance/ethics/ (Accessed: 25 July 2016)

The University of Manchester, (2016) ‘University Research Ethics Committee’. Available at : http://www.manchester.ac.uk/research/environment/governance/ethics/urec/ (Accessed: 25 July 2016)

Thomas, W (2016) ‘Shooting for the Moon’, Ivey Business Journal Online, Spring 2006, Jan/Feb 2016.

Thomson, D. (2010), “Exploring the role of funders’ performance reporting mandates in nonprofit performance measurement”, Nonprofit and Voluntary Sector Quarterly, Vol. 39 No. 4, pp. 611-629.

Tuan, M.T. (2008). Measuring and/or estimating social value creation: Insights into eight integrated cost approaches. Seattle, WA: Bill & Melinda Gates Foundation—Impact planning and improvement. Available at: https://docs.gatesfoundation.org/Documents/WWL-report-measuring-estimating-social-value-creation.pdf (Accessed on: 18 December, 2015).

UNFPA (Undated) Disrupting the UN System – UNFPA’s New Vision on Youth Innovation. Available at: https://unjobfinder.org/articles/unfpa-youth-innovation-engagement-and-participation (Accessed: 04 November 2016)

Unger, B.N., Kock, A., Gemünden, H.G. and Jonas, D. (2012) Enforcing Strategic Fit of Project Portfolios by Project Termination: An Empirical Study on Senior Management Involvement. International Journal of Project Management, 30, 675–85.

Unger, B.N., Rank, J. and Gemünden, H.G. (2014) Corporate Innovation Culture and Dimensions of

http://www.innovation.cc/books/conger_social_inventions1_09232009min.pdf

http://www.strategy-business.com/media/file/sb85_16409_Global_Innovation_1000_SoftwareAsCatalyst.pdf

http://www.strategy-business.com/media/file/sb85_16409_Global_Innovation_1000_SoftwareAsCatalyst.pdf

http://www.strategyand.pwc.com/innovation1000

http://www.manchester.ac.uk/research/environment/governance/ethics/

http://www.manchester.ac.uk/research/environment/governance/ethics/urec/

https://docs.gatesfoundation.org/Documents/WWL-report-measuring-estimating-social-value-creation.pdf

https://docs.gatesfoundation.org/Documents/WWL-report-measuring-estimating-social-value-creation.pdf

https://unjobfinder.org/articles/unfpa-youth-innovation-engagement-and-participation


Page:129

UNHCR (2014) UNHCR Innovation. Available at: http://www.unhcr.org/5319ddb39.pdf (Accessed: 31 October 2016)

UNHCR (2016) UNHCR launches Fund to drive innovation in operations. Available at: http://innovation.unhcr.org/unhcr-launches-fund-drive-innovation-operations/ (Accessed: 31 October 2016)

UNICEF (2005) Annual Report 2004 [Online]. Available at: http://www.unicef.org/publications/files/Annual_Report_2004.pdf (Accessed: 10 February 2015).




UNICEF (2009) Annual Report 2008 [Online]. Available at: http://www.unicef.org/publications/files/UNICEF_Annual_Report_2008_EN_072709.pdf (Accessed: 10 February 2015).




UNICEF (2013a) Annual Report 2012 [Online]. Available at: http://www.unicef.org/publications/files/UNICEF-AnnualReport2012_8July2013.pdf (Accessed: 10 February 2015)

UNICEF (2013b) ‘The UNICEF Strategic Plan, 2014-2017: Realising the rights of every child especially the disadvantaged’ [Online]. Available at: http://www.unicef.org/strategicplan/files/2013-21-UNICEF_Strategic_Plan-ODS-English.pdf (Accessed: 23 June 2015)

UNICEF (2013c) ‘The UNICEF Strategic Plan, 2014-2017: Realising the rights of every child especially the disadvantaged [short brochure]’ [Online]. Available at: http://www.unicef.org/strategicplan/files/UNICEF_Strategic_Plan__2014-2017_e-version.pdf (Accessed: 23 June 2015)

UNICEF (2013d) UNICEF Stories: What Innovation Means to UNICEF. Available at: https://blogs.unicef.org/innovation/what-innovation-means-to-unicef/ (Accessed: 06 June 2016)

UNICEF (2013e) UNICEF innovation toolkit. Available at: https://unicefinnovationtoolkit.files.wordpress.com/2013/05/unicef_innovation_toolkit.pdf (Accessed: 25

th September 2015)

UNICEF (2014a) Annual Report 2013 [Online]. Available at: http://www.unicef.org/publications/files/UNICEF_Annual_Report_2013_web_26_June_2014.pdf (Accessed: 10 February 2015).

http://www.unhcr.org/5319ddb39.pdf

http://innovation.unhcr.org/unhcr-launches-fund-drive-innovation-operations/

http://www.unicef.org/publications/files/Annual_Report_2004.pdf




http://www.unicef.org/publications/files/UNICEF_Annual_Report_2008_EN_072709.pdf




http://www.unicef.org/publications/files/UNICEF-AnnualReport2012_8July2013.pdf

http://www.unicef.org/strategicplan/files/2013-21-UNICEF_Strategic_Plan-ODS-English.pdf

http://www.unicef.org/strategicplan/files/2013-21-UNICEF_Strategic_Plan-ODS-English.pdf

http://www.unicef.org/strategicplan/files/UNICEF_Strategic_Plan__2014-2017_e-version.pdf

https://blogs.unicef.org/innovation/what-innovation-means-to-unicef/

https://unicefinnovationtoolkit.files.wordpress.com/2013/05/unicef_innovation_toolkit.pdf

http://www.unicef.org/publications/files/UNICEF_Annual_Report_2013_web_26_June_2014.pdf


Page:130

UNICEF (2014b) UNICEF Innovation Annual Report 2014 [Online]. Available at: https://unicefstories.files.wordpress.com/2013/08/ui_ar_2014-1124.pdf (Accessed: 16 May 2015)

UNICEF (2015a) From Principle to Practice: Implementing the Principles for Digital Development: Perspectives and Recommendations from the Practitioner Community. Available at: http://www.unicefstories.org/wp-content/uploads/2013/08/From_Principle_to_Practice.pdf (Accessed: 31 October 2016)

UNICEF (2015b) Innovation in the UN: A Session of Joint Meeting of the Executive Boards of UNDP/UNFPA/UNOPS, UNICEF, UN-Women and WFP, February 2, 2015. Available at: http://www.unicefstories.org/wp-content/uploads/2013/08/Innovation-in-the-UN_04.09.2015-1.pdf (Accessed: 31 October 2016)

UNICEF (2015c) The UNICEF Vision and Mission for Children [Online]. Available at: http://www.unicef.org/publicpartnerships/files/UNICEF_Mission(1).pdf (Accessed: 11 December 2015)

UNICEF (2016a) Annual Report 2015 [Online]. Available at: http://www.unicef.org/publications/files/UNICEF_Annual_Report_2015_En.pdf (Accessed: 31 October 2016)

UNICEF (2016b) UNICEF Innovation: Focus Areas. Available at: http://www.unicef.org/innovation/innovation_73203.html (Accessed: 20 May 2016)

UNICEF (2016c) UNICEF Innovation: UNICEF Innovation Fund: About. Available at: http://www.unicefinnovationfund.org/#about-page (Accessed: 20 May 2016)

UNICEF (2016d) UNICEF Innovation: UNICEF Innovation Fund: Resources: Learn More About The Innovation Fund. Available at: http://www.unicefinnovationfund.org/learn-more (Accessed: 20 May 2016)

UNICEF (2016e) UNICEF Innovation: UNICEF Innovation Fund: Resources: List of UNICEF Programme Countries. Available at: http://www.unicefinnovationfund.org/programme-countries (Accessed: 20 May 2016)

UNICEF (2016f) UNICEF Innovation: UNICEF Innovation Fund: Resources: Principals. Available at: http://www.unicefstories.org/principles/ (Accessed: 20 May 2016)

UNICEF (2016g) UNICEF Innovation: UNICEF Innovation Fund: Resources: Short Background Note. Available at: https://drive.google.com/file/d/0BwC3tr7jXADzNXJTNzk4d01qams/view?pref=2&pli=1 (Accessed: 20 May 2016)

UNICEF (2016h) UNICEF Innovation: UNICEF Innovation Fund: Resources: UNICEF Request for Expression of Interest (REOI) for Open Source Technology Solutions for Children. Available at: https://drive.google.com/file/d/0B6-5qW-9eN2bUG9nOVA2eGpyU0k/view?pref=2&pli=1 (Accessed: 20 May 2016)

UNICEF (2016i) UNICEF Innovation: Who we are. Available at: http://www.unicef.org/innovation/innovation_73203.html (Accessed: 20 May 2016)

UNICEF (2016j) UNICEF Innovation: UNICEF Innovation Fund Dashboard. Available at: http://staging-innovation.herokuapp.com/ (Accessed: 09 November 2016)

UNICEF (2016k) - UNICEF Innovation Fund Backgrounder [Online]. Available at: http://www.unicefstories.org/wp-content/uploads/2016/11/UNICEF-Innovation-Fund-2-pager.pdf (Accessed: 13 November 2016)

UNICEF (2016l) UNICEF makes venture capital investments in emerging market tech start-ups [Online]. Available at: http://bit.ly/InnovationFundPressRelease. (Accessed: 13 November 2016)

https://unicefstories.files.wordpress.com/2013/08/ui_ar_2014-1124.pdf

http://www.unicefstories.org/wp-content/uploads/2013/08/From_Principle_to_Practice.pdf

http://www.unicefstories.org/wp-content/uploads/2013/08/Innovation-in-the-UN_04.09.2015-1.pdf

http://www.unicef.org/publicpartnerships/files/UNICEF_Mission(1).pdf

http://www.unicef.org/publications/files/UNICEF_Annual_Report_2015_En.pdf

http://www.unicef.org/innovation/innovation_73203.html

http://www.unicefinnovationfund.org/

http://www.unicefinnovationfund.org/learn-more

http://www.unicefinnovationfund.org/programme-countries

http://www.unicefstories.org/principles/

https://drive.google.com/file/d/0BwC3tr7jXADzNXJTNzk4d01qams/view?pref=2&pli=1

https://drive.google.com/file/d/0B6-5qW-9eN2bUG9nOVA2eGpyU0k/view?pref=2&pli=1

http://www.unicef.org/innovation/innovation_73203.html

http://staging-innovation.herokuapp.com/

http://staging-innovation.herokuapp.com/


Page:131

UNICEF (2016m) UNICEF Innovation fund Scoring Criteria for Request For Expression Of Interest (REOI) stage and Request For Proposal RFP stage.

UNICEF (2016n) UNICEF Global Innovation Centre: Monitoring, Evaluation and Learning (MEL) Frame work. UNICEF (2016o) IoGT Log frame Global Innovation Level Alternative Draft 20th June 2016

UNICEF (2016p) U-Report Log frame Global Innovation Level Draft 5: revised – 28th June 2016

UNICEF (Undated) UNICEF Innovation: Focus Areas: Real-time Information. Available at: http://www.unicef.org/supply/files/Day_2_5._Unicef_innovation.pdf (Accessed: 20 May 2016)

UNICEF. (2003) UNICEF's mission statement. Available at: http://www.unicef.org/about/who/index_mission.html (Accessed: 06 February 2015)

University of Manchester (2014), ‘Code of Good Research Conduct’. Available at: https://documents.manchester.ac.uk/display.aspx?DocID=2804 (Accessed: 25 July 2016)

University of Manchester, (2016) ‘University of Manchester Research Data Management Policy’. Available at: http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/policy/(Accessed: 25 July 2016).

Von Hippel, E., Thomke, S. and Sonnack, M. (1999) “Creating Breakthroughs at 3M.” Harvard Business Review 77, no. 5 (September 1999).

Von Schomberg, R. (2013) ‘A Vision of Responsible Research and Innovation’, In Own, R., Heintz, M., and Bessant, J. (eds.) Responsible Innovation. London: John Wiley.

Von Schomberg, R. (2014) ‘The Quest for the ‘right’ Impacts of Science and Technology: A Framework for Responsible Research and Innovation. In Van den Hoven, J., Doorn, D., Swierstra, T., Koops, B.J. and Romijin, H. (2014) Innovation 1: Innovative Solutions for Global Issues. Netherlands. Springer

WFP (2016) World Food Programme Launches Innovation Accelerator To Test Drive Hunger Solutions. Published 11 July 2016. Available at: http://www.wfp.org/news/news-release/world-food-programme-launches-innovation-accelerator-test-drive-hunger-solutions (Accessed: 04 November 2016)

Wilsdon, J. Willis, R. (2012) See-Through Science: Why public engagement needs to move upstream. Available at: http://www.csec.lancs.ac.uk/docs/nano%20project%20seethroughsciencefinal%20nov05.pdf (Accessed 23 July 2016)

Wood, C. and Leighton, D. 2010 Measuring Social Value – The Gap Between Policy and Practice. London: Demos. Available at: http://www.demos.co.uk/files/Measuring_social_value_-_web.pdf (Accessed on 21 December, 2015)

Yin, R. (2014) Case study research: Design and methods. (5th edn) Los Angeles: Sage.

Zappalà G, Lyons M. (2009). Recent approaches to measuring social impact in the Third sector: An overview. CSI Background Paper No. 6 July 2009.Available at: http://www.socialauditnetwork.org.uk/files/8913/2938/6375/ CSI_Background_Paper_No_5_-_Approaches_to_measuring_social_impact_-_150210.pdf (Accessed 21 December, 2015)

Zappalà G, Lyons M. (2009). Recent approaches to measuring social impact in the Third sector: An overview. CSI Background Paper No. 6 July 2009.

Zider, B., (1998). ‘How venture capital works’. Harvard business review, 76(6), pp.131-139.

http://www.unicef.org/supply/files/Day_2_5._Unicef_innovation.pdf

http://www.unicef.org/about/who/index_mission.html

https://documents.manchester.ac.uk/display.aspx?DocID=2804

http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/policy/

http://www.library.manchester.ac.uk/services-and-support/staff/research/services/research-data-management/policy/

http://www.wfp.org/news/news-release/world-food-programme-launches-innovation-accelerator-test-drive-hunger-solutions

http://www.wfp.org/news/news-release/world-food-programme-launches-innovation-accelerator-test-drive-hunger-solutions

http://www.csec.lancs.ac.uk/docs/nano%20project%20seethroughsciencefinal%20nov05.pdf

http://www.demos.co.uk/files/Measuring_social_value_-_web.pdf


Page:132

Appendices

Appendix 1: UNICEF Background and Context

UNICEF's Seven Strategic Outcomes (Medium Term Strategic Plan 2014-2017)

UNICEF Medium Term Strategic Plan 2014-2017 (UNICEF, 2013b&c) focuses on seven strategic ‘outcomes’. A

description of each outcome along with related programme areas are summarised below.

7 Strategic Outcomes Programme Areas

1 Health: Supporting global efforts to reduce under-five mortality through improved and equitable use of high impact maternal, new-born and child health interventions from pregnancy to adolescence and promotion of healthy behaviours.

Immunization; Polio eradication; maternal and new-born health; child health; health systems strengthening; health in humanitarian situations.

2 HIV&AIDS: Supporting global efforts to prevent HIV infections and increase treatment during both decades of a child’s life through improved and equitable use of proven HIV prevention and treatment interventions by pregnant women, children and adolescents.

Prevention of mother to child transmission and infant male circumcision; and treatment of young children affected by HIV&AIDS; adolescents and HIV&AIDS; protection and support for children and families; HIV&AIDS in humanitarian situations.

3 WASH: Supporting efforts to eliminate open defecation and increase use of safe drinking water through improved and equitable access to safe drinking water sources, sanitation and healthy environments and improved hygiene practices.

Water supply; sanitation; Hygiene; WASH in schools and early childhood centres; WASH in humanitarian situations.

4 Nutrition: Supporting global efforts to reduce under nutrition, with particular focus on stunting, through improved and equitable use of nutritional support and improved nutrition and care practices.

Infant and young child feeding; micronutrients; nutrition and HIV; community management of acute malnutrition; nutrition in humanitarian situations.

5 Education: Supporting global efforts to provide access to quality education for both boys and girls through improved learning outcomes and equitable and inclusive education.

Early learning; equity with a focus on girls’ education and inclusive education; learning and child friendly schools; education in humanitarian situations.

6 Child Protection: Supporting global efforts to prevent violence, abuse, exploitation and neglect through improved and equitable prevention and child protection systems.

Child protection systems strengthening; violence, exploitation and abuse; birth registration; strengthening families and communities; child protection in humanitarian situations.

7 Social Inclusion: Supporting global efforts to reduce child poverty and discrimination against children through improved policy environments and systems for disadvantaged children.

Child poverty analysis and social protection; human rights, non-discrimination and participation; public financial management; governance and decentralisation; social inclusion in humanitarian situations.

Figure 72: Strategic Outcome Description and Programme Areas (UNICEF, 2013b&c)

Innovation as an Implementation Strategy (Medium Term Strategic Plan 2014-2017)

The Medium Term Strategic Plan identifies seven implementation strategies to achieve the outputs one of

which is ‘Identification and Promotion of Innovation’ described as ‘identifying promising innovations and

supporting partners to adopt, adapt and scale up the most useful and promising approaches’ (UNICEF,

2013b&c) giving innovation a central role in the organisation's strategy.


Page:133

Demand Enhanced support for children, families and communities to promote knowledge, behaviour change,

demand for services and opportunities for participation

Supply Increased national capacity to ensure availability of, and access to, services and to strengthen systems

Enabling Environment Strengthen political commitment and national capacity to legislate, plan and budget for children

Humanitarian Action Increased country capacity and delivery of services to protect and provide for children in humanitarian

settings

Rights and Gender Equality Increased capacity of governments and partners, as duty-bearers, to identify and respond to specific

challenges related to the protection and promotion of the rights of children, to gender equality, and to the empowerment of girls and women.

Global & Regional Action Enhanced enabling environment at global and regional levels to realize child rights

Capacity Development Communication for development (C4D), training, tech., assistance.

Evidence Generation, Policy Dialogue and Advocacy Generate evidence using innovation to influence policy, dialogue and advocacy.

Partnerships National, local, civil society, academic and private sector.

South-South and Triangular Cooperation Triangular cooperation.

Identification and Promotion of Innovation Identifying promising innovations and supporting partners to adopt, adapt and scale up the most useful

and promising approaches.

Integration and Cross-Sectoral Linkages Learning from integrated programmes and partnerships.

Service Delivery Delivering essential services and commodities especially where institutional capacity is weak.

...through sound implementation strategies based on UNICEF’s comparative advantage in achieving results for children, especially the most disadvantaged and... through sound management practices and

operational excellence based on clear principles, achievement of concrete results, accountability and transparency.

Figure 73: UNICEF Strategic Plan 2014-2017 Overview (UNICEF, 2013b&c)

Health

Outcome

Education

Outcome

HIV/AIDS

Outcome

Child Protection Outcome

Social Inclusion Outcome

WASH

Outcome

Nutrition

Outcome

Realising the rights of every child, especially the most disadvantaged

Seven Strategic Outcomes

Common set of Six Outputs for each Outcome

Organizational Efficiency and Effectiveness

Seven Implementation Strategies


Page:134

List of UNICEF Programme Countries

Country Country Country Afghanistan Ghana Palau

Albania Grenada Panama

Algeria Guatemala Papua New Guinea

Angola Guinea Paraguay

Anguilla Guinea Bissau Peru

Antigua and Barbuda Guyana Philippines

Argentina Haiti Qatar

Armenia Honduras Romania

Azerbaijan India Russian Federation

Bahrain Indonesia Rwanda

Bangladesh Iran (Islamic Republic of) Saint Kitts and Nevis

Barbados Iraq Saint Lucia

Belarus Jamaica Saint Vincent & Grenadines

Belize Jordan Samoa

Benin Kazakhstan Sao Tome and Principe

Bhutan Kenya Saudi Arabia

Bolivia, Plurinational State of Kiribati Seychelles

Bosnia and Herzegovina Korea, DPR Senegal

Botswana Kosovo under UNSC res. 1244 Serbia

Brazil Kuwait Sierra Leone

British Virgin Islands Kyrgyzstan Solomon Islands

Bulgaria Lao PDR Somalia

Burkina Faso Lebanon South Africa

Burundi Lesotho South Sudan, Republic of

Cambodia Liberia Sri Lanka

Cameroon Libya Sudan

Cape Verde Macedonia Suriname

Central African Republic Madagascar Swaziland

Chad Malawi Syrian Arab Republic

Chile Malaysia Tajikistan

China Maldives Tanzania, United Republic of

Colombia Mali Thailand

Comoros Marshall Islands, Republic of Timor-Leste

Congo Mauritania Togo

Cook Islands Mexico Tokelau

Costa Rica Micronesia (Federated States) Tonga

Cote d'Ivoire Moldova Trinidad and Tobago

Croatia Mongolia Tunisia

Cuba Montenegro Turkey

Congo (DRC) Montserrat Turkmenistan

Djibouti Morocco Turks and Caicos Islands

Dominica Mozambique Tuvalu

Dominican Republic Myanmar Uganda

Ecuador Namibia Ukraine

Egypt Nauru United Arab Emirates

El Salvador Nepal Uruguay

Equatorial Guinea Nicaragua Uzbekistan

Eritrea Niger Vanuatu

Ethiopia Nigeria Venezuela (Bolivian Republic)

Fiji Niue Vietnam

Gabon Occupied Palestinian Territory Yemen

Gambia Oman Zambia

Georgia Pakistan Zimbabwe

Figure 74: UNICEF Programme Countries (159) (UNICEF, 2016c&d)


Page:135

Appendix 2: UNICEF Office of Innovation Background and Context

Innovation Focus

The UNICEF Innovation Annual Report 2014 (UNICEF, 2014d) outlines three core focus areas namely;

information, opportunity and choice as follows:

1. Information: Innovation initiatives that apply either Real-time Data for two way communication and

better service delivery or Tracking ensuring mothers and children receive necessary, lifesaving

interventions. Example: RapidPro open source platform that allows design and adaptation of

solutions that provide access to information for better service delivery and interventions.

2. Opportunity: Innovation initiatives that remove barriers to, or provide new means of, Equitable

Access to services, facilities or products that give children an opportunity to survive and thrive.

Example: Mobistation portable multimedia education solutions.

3. Choice: Innovation initiatives that facilitate or improve Youth Engagement and/or Youth

Empowerment by bringing young people closer to government and other decision makers.

The UNICEF Office of Innovation website breaking them down into five more detailed focus areas as follows

(UNICEF, 2016a):

1. Real-time Information: ‘Tools that strengthen systems by providing real-time monitoring,

transparency, and accountability, which helps inform decision making’. Example: RapidPro (UNICEF,

2016a).

2. Youth Empowerment (engagement/choice): ‘Innovations that empower young people [to] become

social change agents who can influence and lead on solving problems in their communities. The ability

for young people to connect to each other and counsel each other also allows them to share and scale

their own solutions’. Example U-Report (UNICEF, 2016a).

3. Access to Information: ‘Emerging technologies can help overcome barriers that prevent young people

from accessing information and services. These new resources provide youth with more choices and

more opportunities’. Examples: Internet of Good Things, Virtual Reality (UNICEF, 2016a).

4. Infrastructure: ‘Solutions that overcome gaps in infrastructure -- such as connectivity, power, finance,

and transportation -- increase access to services and information’. Examples: Drones, Satellites

(UNICEF, 2016a).

5. Physical Products: ‘Product Innovation is about procurement of fit-for-purpose and value-for-money

supplies that have impact on our programmes and emergencies...’ Examples: Diagnosis kits, ORS &

Zinc Co-Pack for treating diarrhoea (UNICEF, 2016a).

Innovation Focus, Initiatives and Outcomes Map

Examples of some innovation initiatives for each of the five focus areas have been summarised and mapped

below. The link between each initiative and its contribution to UNICEF's wider strategic ‘outcomes’ is also

illustrated.


Page:136

‘Market’

Definition & Broad Focus

Innovation Focus (Detailed)

How Focus is Addressed

Initiative Groups / Platforms

Initiatives/Projects/Tools

Outcomes Health HIV/AIDS WASH Nutrition Education Child Protection Social Inclusion

Figure 75: UNICEF Office of Innovation; Focus, Initiatives & Outcomes Map (Created by the authors based on UNICEF, 2013b, c & d & 2016a-i & Meagher, 2016a&b)

International Child Welfare (Served) ‘Market’ (Focusing on 7 Strategic Outcomes – See final row)

Real-time Information & Tracking

Infrastructure Youth Empowerment, Engagement & Choice

RapidSMS (Birth Reporting)

RapidFTR (Rapid Family Tracing & Reunification)

Primero (Next Gen. RapidFTR)

Innovation at UNICEF: ‘to do something new or different that adds value...’ Broad Innovation Focus: Information, Opportunity & Choice

Access to Information Physical Products

Tools that strengthen real-time monitoring, transparency &

accountability to inform decision making (2-way

communication)

Solutions to gaps in connectivity, power, transport, finance &

administration increasing access to services&

information

Innovations that empower young people to become

social change agents, solve problems, connect & scale

solutions

Emerging technologies to overcome barriers to

accessing information & services giving the

opportunity & choice

Procurement of fit for purpose value for money supplies that impact programmes &

emergencies (mostly incremental innovations developed by country offices with some radical

innovations by UNICEF Supply)

RapidPro: Open Source Software Platform: design, pilot &

scale services connecting mobile phones without a developer.

RapidSMS delivery of

HIV test results

U-Report: 2 way real-time SMS

communication: Counselling Services,

Youth Influencing, Decision Making

Internet of Good Things

IoGT: Packaged content info. for

free & on low end devices

Virtual Reality: advocacy tool

to increase empathy trough

experience

Drones to transport HIV test samples

Satellites Connecting

Remote Communities

RemindMi (Mexico) SMS reminders to mothers on antenatal care, nutrition, delivery, postnatal care,

breastfeeding & immunisation

RapidPro SMS tracking

WASH kits in Pakistan

U-Report Voice

Matters protection

against abuse

U-Report: By Youth for Youth

UPSHIFT programme

encouraging youth to solve problems in their communities

by social media, lab website, etc.

PoC HIV Diagnostic

FOND Food Online

Nutrition Database

Temporary Emergency Structures

Primary School Furniture Design &

Procurement Guidelines

DigiSchool (Mobistation)Portable

solar powered multimedia kit with laptop, projector & speakers delivering educational content

in remote or un-served areas

Digital Kiosks Solar- powered

rugged computer kiosks for

youth centres in rural settings

Raspberry Pi eLearning tool interventions for out of school kids teaching basic core

and technology skills

mTrac (pregnancy monitor)

EduTrac (education monitor)

Health Products: ORS / Zinc co-pack for diarrhoea, COTZ End-to-end ORS/Zinc value chain in

remote areas, Amoxicillin

Dispersible Tablets, CHW+ Community Health Worker

pack & system, Weight Estimation Tape for CHWs,

ARIDA Diagnostic for Pneumonia, ViVa Stock

Visibility for Vaccines Online, Vaccine Supply to Last Mile

using Phase Change Material to avoid freezing, Emergency kit

labelling

HWTS Household water treatment & safe storage, Jerry Can Collapsible water container, Latrine

Additive to aid de-composition, Disabled latrines

Vaccine Supply to Last Mile using phase change material to avoid freezing

RapidPro: Successfully Creating Value after many Failures.

Benchmark for UNICEF Innovation Fund Expected Failure Rate. Inspiration for the ‘scanning the horizons’ approach.


Page:137

Focus (Broad)

How it is Addressed

Example Initiatives

Focus (Detailed)

Approach Office if Innovation Example Projects Launch Location Indicator / Metric

Possible Indicators (Metric)

Strategic Outcome

Information

Real-time Data: Two way communication & better service delivery

mTrac Birth Reporting Real-time

Information & Tracking (Information)

Tools that strengthen real-time monitoring, transparency, & accountability to inform decision making (2-way communication)

RapidSMS Birth Registration & RemindMi: Birth registration by SMS to central data base. Health check reminders during & after pregnancy

2010 Nigeria & Zambia

% Local Gov’t Areas Reporting

Identification of teams not performing

Health

mTrac: Disease surveillance reporting, drug stock monitoring & community feedback

2011 Uganda % Health Facilities Using

% health workers & % CHW using

Health

Tracking: Ensuring mothers & children receive lifesaving interventions

RapidFTR Primero

RapidFTR (Rapid Family Tracing and Reunification): 2011 Various TBC Mobilisation time, Time to reunite 50%, 75% 100%,

Child Protection (+ Health & Nutrition)

Primero (Next generation of RapidFTR) 2014 Pilot

TBC TBC Mobilisation time, Time to reunite 50%, 75% 100%

Child Protection (+ Health & Nutrition)

Opportunity

Equitable Access: Giving children an opportunity to survive & thrive

Mobistation Digital Kiosks Rasberry Pi ORS/Zinc Co-packaging Amoxicillin Dispersible Tablets

Access to information (Opportunity)

Emerging technologies to overcome barriers to accessing information & services

Digischool (MobiStation) (Schools): Portable solar powered multimedia kit built around a laptop, projector and speakers delivering educational content in remote or un-served areas

2012 Uganda No. of locations

Increase in No., No. of children taught, Increase in no. of kids taught

Education

MobiStation (healthcare training)

2014 Pilot

Uganda No. of locations

DITTO for health workers

Health (+Education)

Digital Kiosks: Solar- powered rugged computer kiosks for rural youth centres

2011 Uganda No. of kiosks

No. of kiosks / No. of rural centres

Social Inclusion (+ Education)

Raspberry Pi4L: eLearning for out of school kids teaching basic core and technology skills

2014 Pilot

Lebanon TBC No. of children using / No. of out of school

Education

Internet of Good Things: mobile-ready web-based resources & applications that bring good to life & education to children with no internet access

2015 40 countries (2015)

TBC No. of children using / No. of children without internet

Education (+ Social Inclusion)

Infrastructure (Opportunity)

Emerging technologies to overcome barriers to accessing information & services

RapidSMS HIV Test Results: Reduced cycle time of HIV test results leading to earlier infant diagnosis, treatment and statistically higher survival rates.

2010 Zambia Cycle Time: No. of Days

Ditto HIV/AIDS

Physical Products (Opportunity)

Procurement of fit for purpose value for money supplies that impact programmes & emergencies

Oral rehydration salts (ORS) & Zinc Co-packaging: Treatment for diarrhoea the second largest treatable cause of death

2010 UNICEF Supply

TBC No. of ORS & Zinc / No. of ORS

Health (+WASH)

Amoxicillin Dispersible Tablets: Dosage tracking antibiotic packet for pneumonia treatment

2011 UNICEF Supply

TBC No. of dosage tracking packets / Total No. of Packets

Health

Choice

Youth Engagement: Bring young people closer to government

Real-time Counselling Services Youth Influencing Decision Making

Youth Empowerment (Choice)

Innovations that empower young people to become social change agents, solve problems, connect & scale solutions

U-Report: Real-time Counselling Services, Youth Influencing, Decision Making SMS/Open Source weekly polls

2011 Uganda No. of users & No. of districts

No. of children using Social Inclusion,

U-Report: DITTO 2014 Nigeria No. of users DITTO Social Inclusion

U-Report: Individualised HIV counselling 2014 Zambia No. of users DITTO HIV + Social Inclusion

Figure 76: Sample Office Innovation Projects (Created by the authors based on UNICEF, 2014b and UNICEF 2013c&d & Meagher, 2016a&b)


Page:138

Principals for Digital Development

The Principals for Digital Development are at the very core of all UOI activities and are a key aspect of

assessment for initiatives applying for funding to UNICEF Innovation Fund. The principals have been developed

following years of iteration and lessons learned by UNICEF on the implementation of technology. The

principals have been endorsed by a wide range of global partners including other UN Agencies, the Gates

Foundation, private sector partners, governments and civil society organisation's (UNICEF, 2016e and

Meagher, 2016).

Figure 77: Principles for Digital Development (UNICEF, 2016f)


Page:139

Appendix 3: UNICEF Innovation Fund Background and Context

The UNICEF Innovation Fund was identified by the UOI as a core part of its strategy to pursue emerging

opportunities in the UNICEF Innovation Annual Report 2014 (UNICEF, 2014e).

Figure 78: UNICEF Office of Innovation Strategic Emphasis (UNICEF, 2014b)

The $10.5m venture fund was launched in 2015 to invest in open-source technologies that contribute towards

the organisation’s mission (UNICEF, 2016b). The UIF provides a pooled funding mechanism to support country

offices, small scale organisations and institutions to take their tested projects to the next stage (UNICEF,

2016b). The companies must be based in a country where UNICEF have an established country programme

which confirms that there is a need for the UNICEF mission.

The Innovation Fund supports projects for a 12-18 month period (UNICEF, 2016c) by which time it is

considered the project will have either succeeded in convincing scale funders of its potential or fail to go any

further. Evaluations conducted in this project aim to capture the few successes as well as learning from failures

which are expected as part of the Venture Capital style approach.

UNICEF Innovation Fund Stage of Innovation Process

A typical process for innovation based on the UNICEF Innovation Toolkit (2013) and UNICEF Supply Division

Innovation Process (Undated) is shown below. The process illustrated below is included herein to give clarity to

the stage at which the UNICEF Innovation Fund applies. The following 3 criteria give the overall nature and

stage of projects which can apply to UIF (UNICEF, 2016c):

1. Early stage

2. Prototype or pilot is complete (Proof of concept has been achieved)

3. A small amount of money (circa $100,000) is needed to demonstrate its value to scale

funders/investors

Thus UNICEF Innovation Fund is applicable immediately after proof or concept and ceases prior to scale up

phase as illustrated.


Page:140

Figure 79: UNICEF Innovation Fund and Innovation Process (UNICEF, 2013e & 2016a-i & UNICEF Supply Division, Undated & Meagher, 2016)

Step 1 Identifying

Need

based on database

research & UNICEF Strategy

resulting in a Project

Brief

Step 6 Development & Testing

Iteration Cycle:

1. Designing a technically feasible, desired & economically viable

product(Proof of Concept) 2. Prototyping to identify what is missing & explore

different possibilities, 3. Testing in lab & field in

typical usage environment, 4. Reflection on results to

facilitate redesign.

Step 7 Business Analysis

Financial viability of the

concept(s) assessed including Market Research, Return on

Innovation (ROI), Value Framework & Business

Model Canvas

Step 2 Idea Generation

Creative process

involving multidisciplinary

teams generating a

large quantity of ideas without

quality assessment

Step 3 Idea Screening

Evaluation

process involving multidisciplinary teams isolating

the most promising &

eliminating ideas unlikely to create

value

Step 4 Concept

Generation

Needs & ideas studied in the

field by a multidisciplinary

team applying human centred design gaining a

holistic understanding &

generating multiple product

concepts

Step 5 Concept Selection

Concepts

screened by a multidisciplinary

team using simple matrix

that prioritised concepts with

the highest impact & lowest

resources required based

on educated guesses resulting in the selection of one or more

concepts

Step 9 Implementation at Scale

Innovation scaled following

implementation plan for manufacturing &

distribution + monitoring & evaluation plan including

economic impact analysis & project evaluation plan

Step 8 Piloting

Final product

concept is tested across several regions in real

usage situation to ensure adoption

rates are high enough, test

manufacturing, raise awareness & demonstrate

value to decision makers

Stage Gate 0 Stage Gate 1 Stage Gate 2 Stage Gate 3

Phase 0: Explore Phase 1: Concept Phase 2: Field Trial Phase 3: Scale Up

Proof of Concept

(PoC) achieved

+ First

Prototype complete

UNICEF Innovation Fund

Internal & External projects eligible for $100,000 seed funding

investment over 12-18 months after PoC + 1st Prototype subject to

satisfying criteria

Scale Up Finance

Options: Private Sector investment, grant-funding,

UNICEF networks, International Development Innovation Alliance (IDIA), UNICEF

Innovation Procurement Acceleration Fund & Global

Innovation Centre

Principles For Digital Development 1. Design with the User; 2. Understand the Ecosystem; 3. Design for Scale; 4. Build for Sustainability; 5. Be Data Driven;

6. Use Open Data, Open Standards, Open Source, Open Innovation; 7. Reuse & Improve; 8. Address Privacy & Security; 9. Be Collaborative

Innovation Review Board evaluation

& resource allocation





Note 1:This process is illustrated based on UNICEF Innovation Toolkit (2013), UNICEF Process for Innovation at Supply Division (Undated) & UNICEF Innovation Website (accessed: 20 May 2016).

Country Office Budget

Country Office Budget


Page:141

Typical Nature of UNICEF Innovation Fund Projects

The innovation projects can involve the following (UNICEF, 2016c):

1. Developing a new technology

2. Expanding/improving existing technology / platform

3. Piloting a new use for existing technology

4. Piloting an existing technology in a new area / country

Portfolio Areas

The initiatives or innovation projects must align with one of UNICEF Innovation’s three portfolio areas as

follows (UNICEF, 2016b):

1. Products for youth under 25: Needs – learning, youth participation, overcome barriers that prevent

access to information and services, connectivity and counsel peers.

2. Real-time information for decision making: allows vulnerable children be understood by decision

makers with the power to take action.

3. Infrastructure to increase access to services and information: connectivity, power, finance, sensors

and transport, access to information (especially real time). Examples of technologies identified as

priority for investment: Blockchain, 3D printing, wearables and sensors, artificial intelligence,

renewable energy, personal/social communications networks, satellites, mobile devices and UAVs.

Returns to Investors outlined by UNICEF

UNICEF identify three types of return from Innovation Fund as follows:

1. Growth: Small investments in innovative initiatives has the potential to grow solutions and

technologies more rapidly than simply investing all available funds in traditional means. Growth can

be measured real time measuring number of users, number of interactions by individual project or

product family

2. IP stack: Investors gain access to open-source intellectual property and benefit from UNICEF's

infrastructure, expertise, connections and environment for identification, piloting and scaling.

Investors can adapt the innovations to other contexts outside the UNICEF mission to benefit their

interests. This can be seen as a quasi ‘financial return on their investment through free access to new

tools’.

3. Communities: UNICEF is creating communities around innovation initiatives e.g. U-Report with 1.8M

users in 18 countries. UNICEF describe those communities as a ‘global change engine that can be

used to build solutions in the future’.

The foregoing returns ‘expected’ by UOI for investors in the fund will be explored further in the interview stage

of the project.


Page:142

Summary of UNICEF Innovation Fund Portfolio Areas, Returns and Value

Figure 16 endeavours to capture and summarise the main focus, portfolio areas and returns to investors the

UNICEF Office of innovation expect from the UNICEF Innovation Fund as narrated in the related documents

and websites reviewed to date. Figure 16 also introduces a means of breaking down value creation into three

parts namely value for the customer, organisation and ecosystem (Sniukas, et al., 2016). Moreover the linkage

of this value to the seven strategic outcomes of the wider organisation is also included.


Page:143

Focus (Broad)

How it is Addressed

Example Initiatives

Innovation Fund Portfolio Area Returns to Investors Value Creation Strategic Outcome

Information

Real-time Data: Two way communication & better service delivery

mTrac Birth Reporting

1. Products for youth under 25 address a range of needs including learning and youth participation. Emerging technologies can help overcome barriers that prevent young people from accessing information and services. The ability for young people to connect to each other and counsel each other also allows them to share and scale their own solutions. 2. Real-time information for decision making: allows the situation of the most vulnerable to be understood by those who have an ability to address their circumstance. Real-time data can help identify where disparities are greatest, who is being reached, who is not using essential services, and why this is the case. Products that produce real-time information strengthen systems, improving monitoring, transparency, and accountability. 3. Infrastructure to increase access to services and information, including connectivity, power, finance, sensors and transport. Access to (life enhancing) information, is a human right. Innovation that builds and strengthens infrastructure holds great promise to address challenges that negatively impact children.

1. Growth: Projects and technologies become big, in a global ecosystem increasing the scale, reach, impact and efficiency of policies, service and information for children, ultimately generating financial and efficiency savings. This can be measured by real time growth (or lack of growth) of user bases, interactions. 2. IP Stack: The Fund invests exclusively in open source, but it is not without value. E.g.: UReport is an enterprise-level, cloud-hosted, multitenancy solution for youth engagement. Investors gain access to such open-source intellectual property, while UNICEF provides the infrastructure, expertise and environment for identification, piloting and scaling. Investors are able to apply for their own use, translating them into improvements in efficiency, reach, impact and scale of their own programmes and policies for children. Investors, therefore, receive a financial return on their investment through free access to new tools. 3. Communities: Created around the products. UReport has more than 1.8M users in 18 countries. It also has more than 1,953 commits to GitHub and 302 stars. Having access to robust communities of developers and entrepreneurs is also of value to our partners – as it will allow them to either extend work that’s been done through the fund for their own purposes and also access a global change engine that can be used to build other solutions in the future.

1. Customer Strategic Outcomes: Gain Creators: Satisfy Needs & Pain Relievers: remove barriers allowing access to: Information, opportunity, choice) 2. Organisation Financial (efficiency savings), Operational, Strategic & Other benefits (learning, readiness, etc.) 3. Ecosystem (Partners & Stakeholders) Financial (efficiency savings), Operational, Strategic & Other benefits (learning, readiness, etc.)

1. Health 2. HIV/AIDS 3. WASH 4. Nutrition 5. Education 6. Child Protection 7. Social Inclusion

Tracking: Ensuring mothers & children receive lifesaving interventions

RapidFTR Primero

Opportunity

Equitable Access: Giving children an opportunity to survive & thrive

Mobistation Digital Kiosks Rasberry Pi ORS/Zinc Co-packaging Amoxicillin Dispersible Tablets

1. Products for youth under 25 2. Real-time information for decision making 3. Infrastructure to increase access to services and information

Note: These three areas are ripe for investment due to rapidly changing technologies such as blockchain, 3D printing, wearables and sensors, artificial intelligence and renewable energy (Applicable to all focus areas: Information, Opportunity and Choice)

1. Growth 2. IP Stack 3. Communities

1. Customer 2. Organisation 3. Ecosystem


Choice

Youth Engagement: Bring young people closer to government

Real-time Counselling Services Youth Influencing Decision Making

1. Products for youth under 25 2. Real-time information for decision making 3. Infrastructure to increase access to services and information

1. Growth 2. IP Stack 3. Communities

1. Customer 2. Organisation 3. Ecosystem


Figure 80: UNICEF Office of Innovation & Innovation Fund Focus, Portfolio Areas and Value Creation (UNICEF, 2013b&c, 2014b,and 2016a-i and Sniukas et al., 2016)


Page: 144

Benefits and Reasoning behind UNICEF Innovation Fund

UNICEF Innovation Fund has been designed to address the following (UNICEF, 2016c):

1. To reduce major inhibitors to innovation that can contribute to its mission in programme countries.

2. To improve risk management for both country offices and investors

3. To bridge the gap in early stage funding by providing small seed investments (circa $100,000)

4. To facilitate a portfolio style approach that provides:

a. Assistance and advice from technical experts

b. Reduce sector or country focused technical support allowing resources (cost and time) to be

balanced and shared

c. Free country level innovators from the need to individually look for funding sources reducing

costs of fragmentation and projectisation of funding

d. Provide investors with quality assurance through rigorous selection and technical support.

5. Internal applications from UNICEF Country Offices must provide matched financing doubling the size

and potential of internal investments.

Innovation Fund Investors

UNICEF welcomes investments in Innovation Fund from:

1. Governments

2. Private sector

3. Foundations

4. Academic institutions

5. Civil society

6. International financing institutions

7. Multilateral organisations.

The current investors which have provided the 10.5m include the Government of Denmark, Walt Disney

Corporation, the Government of Finland and the Page Family Fund Foundation.

Venture Capital Approach: Innovation Fund Eligibility and Scoring Criteria

UNICEF Innovation Fund applies venture capital methodologies in both its selection process and support of

initiatives thereafter. UNICEF's ‘Request for Expression of Interest (REOI) for Open Source Technology

Solutions for Children’ (UNICEF, 2016g) outlines the criteria by which individual initiatives will be assessed as

follows.


Page: 145

Criteria Possible Score

Eligibility Criteria (Mandatory Requirements)

1 Registered in a UNICEF programme country Yes/no

2 Working on open source solutions, under the following licenses:

Software: BSD, AGPL, LGPL

Content and designs: Creative Commons CC-BY-SA or CC-BY

Yes/no

3 Working on a technology solution that aligns with three portfolio areas:

Products for youth under 25 to address a range of needs including learning and youth participation;

Real-time information for decision making and governance

Infrastructure to increase access to services and information, including but not limited to connectivity, power, finance, sensors and transport.

Yes/no

4 Generating publicly exposed real-time data that can be / is measured Yes/no

Scoring Criteria

5 Relevance of solution for children and young people

Potential impact on access to information, opportunity and choice

Potential application at global scale

20

6 Novelty of the project

Alignment with Innovation ‘Principles for Digital Development’

Alignment with desired stage of development

30

7 Suitability of the team and key personnel to implement the project 30

8 Project budget and financing 20

Total Score 100

Table 36: UNICEF Innovation Fund: Assessment Criteria for REOI (UNICEF, 2016g)

Applications that pass the REOI stage are further assessed in the requests for proposals (RFP) which also

applies Venture Capital selection criteria. The criteria applied in both the REOI and RFP stages provides further

emphasis on what UNICEF determine to be key characteristics of potentially successful initiatives whilst

acknowledging that failure is still likely.


Page: 146

Typical Venture Capital Scoring Criteria Compared with UNICEF Innovation Fund Criteria

The assessment criteria for individual innovation initiatives considered for seeding by the UNICEF Innovation

Fund are modelled on those used by Venture Capitalists. Table 9 below compares ‘typical’ venture capital

assessment criteria with UNICEF Innovation Fund criteria with clear parallels established.

Typical VC Business Plan Requirements

Typical Venture Capital Assessment Criteria UNICEF Innovation Fund Criteria

Opportunity & Market Size

Identifiable Market:

Is there a clearly defined market for the product or service?

Does the product or service meet the identifiable need?

Does the company have a reasonable plan to meet the identified need in an efficient, revenue-generating manner?

Criterion No. 5 (Weight: 20) a. Relevance of solution for children

and young people b. Potential impact on access to

information, opportunity and choice

c. Potential application at global scale

Review of Competitive Landscape and Solutions

Sustainable Competitive Advantage:

Is the idea truly unique?

Has the company considered economic and technological change that may affect the business model?

Who are the competitors and what are their strengths and weaknesses?

Criterion No. 6 (Weight 30) a. Novelty of the project b. Alignment with Innovation

‘Principles for Digital Development’

c. Alignment with desired stage of development

Resumes of Management Team

Strong Management:

Does leadership inspire confidence?

Do they have vision, expertise and ability to generate significant growth?

Does the team consider best practices of those that have gone before?

Criterion No. 7 (Weight 30) a. Suitability of the team and key

personnel to implement the project

Detailed Financial Projections

Commercially Viable:

Does the company have a product or service that can be reproduced efficiently to generate revenue?

Criterion No. 8 (Weight 20) a. Project budget and financing

Capitalization Table

DITTO DITTO

Table 37: UNICEF Innovation Fund: Assessment Criteria for Expressions of Interest (MyCapital, Undated, noted by Smith, 2016 and (UNICEF, 2016g)


Page: 147

Appendix 4: Innovation Portfolio: Initiatives Supported by UNICEF Innovation Fund to Date

Figure 41 included in the main body of this report provides a summary of innovation initiatives which have

received seed investment from UNICEF Innovation Fund. The projects are listed noting the portfolio area as

defined by UNICEF Office of Innovation.


Page: 148

Appendix 5: List of Internal Documents received from UNICEF

The following is a list of all internal documents received from UNICEF Office of Innovation in relation to UNICEF

Innovation Fund. The documents were shared following the signing of ‘Conditions regarding services as a

volunteer with UNICEF’ with specific reference to obligation ‘b’ as follows: ‘to keep confidential any and all

unpublished information made known to me during the course of my service as a volunteer and not to publish

any reports or papers on the basis of information obtained during the period of my service, except with the

authorization of UNICEF’.

The list of internal documents is as follows:

Draft Terms of Reference for an Advisory Board (Not yet formally set-up).

Evaluation and Scoring Criteria for Expression of Interest stage.

Evaluation and Scoring Criteria for Request for Proposal stage.

Innovation fund agreement template between UIF and the selected companies for funding support.

Innovation fund agreement template that external investees will be asked to sign upon selection.

Innovation Fund Backgrounder document.

Investment made by UIF office for UNICEF Country Offices to date.

Open source product business model.

Real-time reporting website (continuously updated website).

Towards a Shared Impact Measurement Framework for Innovation Measuring Impact Working Group,

The International Development Innovation Alliance (IDIA), September 2016

The International Development Innovation Alliance Framework.

The request for expressions of interest (stage 1 of the external selection process).

The Terms of Reference template for the recent Request for Proposals that was issued to shortlist of

companies.

The UNICEF Office of Innovation team article co-written by Rodrigo Canales, Charlie Cannon,

Christopher Fabian and Erica Kochi, Robert Fabricant and Rebecca Rabison titled ‘DIY Innovation:

Creating an Innovation Capability within your Organisation’, which presents a framework for internal

innovation.

UIF funding agreement with the Government of Finland.

UIF office procurement strategy.

UNICEF internal memo for the establishment of a pooled fund setup.

UNICEF Request for Expression of Interest (REOI) for Open source technology solutions for children.

Independent multi-country review of innovations for scale

Innovations in programme monitoring

Open Source Brief and organisation interviews: Open Source Design

Evaluation report by the UNICEF Regional Office for Southern and Eastern Africa


Page: 149

DE Report from UNICEF OOE: Peace building, Education and Advocacy Programme in Ethiopia

Developmental Evaluation, June 2016

Various UNICEF Blogs including:

o Exponential thinking: A revolutionary mind-set for development?

o I went to Estonia, and saw the future.

o Digitalizing Cambodia’s inspection system to better protect children living in residential care

https://t.co/eb80uyoOZl?ssr=true

http://www.unicefstories.org/2016/09/07/i-went-to-estonia-and-saw-the-future/


Page: 150

Appendix 6: Benchmarking UIF against Leading VC Innovation Investment Funds in International

Development

Funds administered by leading practitioners in international development were researched to benchmark the

UNICEF Innovation Fund approach. Amalgam and Bound (2016) cite two examples of similar funds financing

innovation in international development presently, namely the Gates Foundations Grand Challenges

Programme and USAIDs Development Innovation Ventures (DIV). A comparison of UNICEF Innovation Fund

and the aforementioned funds is provided in Figure 81.


Page: 151

Comparison Criteria

Gates Foundation Grand Challenges Programme

USAID Development Innovation Ventures


Founded 2003 (Grand Challenges in Global Health) & 2007 (Grand Challenges Explorations)

2010 2015

1 Goal / Purpose

Open innovation across disciplines and fields systematically sourcing ideas and solutions to development problems through challenges. Measuring the impact / value of interventions & building a global innovation marketplace.

Bring in creative new ideas for solving problems, increase accountability, delivery of more impact for less money & application of venture capital principals.

Invest in open source technologies for children. Pooled financing mechanism for innovators in developing countries. Models of finance, methodologies of venture capital funds and principals of UNICEF's Innovation Unit.

2 Idea Generation

Systematically sourcing & harnessing creative ideas via a challenge which involves extensive consultation with experts inside and outside the foundation.

Global Open Innovation competition for ideas

Global Request for Expression of interest (REOI) and proposals (RFP) harnessing open innovation to all external small scale start-ups and internal country offices in UNICEF programme countries with prototypes consistent with UNICEF's mission.

3 Countries that can avail of Funding

Global Global Countries with a UNICEF Country Programme, hence the need for improvement in child welfare & rights is established and bottom-up user centred innovation is encouraged.

4 Selection Criteria (Portfolio Criteria)

Systematic process of calling for, examine and rewarding ideas based on 3 Questions: 1. Is idea responsive to Grand Challenge; 2. Is the proposed work related to the strategy of the Foundation & 3. If successful would the work be important & impactful?

Innovations must satisfy 3 Primary Pillars: 1. Cost effectiveness; 2. evidence of impact & 3. pathway to scale

Mandatory requirements: 1. Applicants must be small scale start-up companies (not NGOs) or academic institutions registered in programme countries; 2.Open Source; 3. Software BSD, AGPL, LGPL; 4. Content & Designs Creative Commons CC-BY-SA or CC-BY; 5. Alignment with 3 portfolio areas; a. Products for youth under 25 (learning, participation), b. Real-time information, c. infrastructure to increase access (connectivity, power, transport...); 6. Publicly exposed measurable real-time data. Assessment. Other Criteria: 7.Team suitability; 8.Budget & financing; 9. Novelty. Most interested in: Blockchain, 3D printing, wearable’s, sensors, artificial intelligence & renewable energy.

5 Portfolio Size Grand Challenges in Global Health: $100m (Animesh, 2010)

Since 2010: $70m invested in 145 solutions in 9 sectors & 36 countries

$11.2m Innovation Fund with plans to grow

6 Investment Format & Size

Reviewers internal and external permitted to champion individual ideas backed by $100,000 seed funding for proof of concept.

Grant investments from initial seed of less than $100,000 up to $15,000,000.

$100,000 early stage seed funding for compliant applications that have achieved proof of concept, have a 1

st prototype or have completed a successful pilot.


Page: 152

Comparison Criteria

Gates Foundation Grand Challenges Programme

USAID Development Innovation Ventures


7 Scaling After 1 year of work facilitated by seed award projects are eligible to request a follow-on award for up to $1 million.

Structured staged financing awards in according to milestone achievements to mitigate risk with flexibility for grantees to iterate.

Innovation Fund does not cover the scaling phase. The fund is focused on providing VC style early stage seed investments only to grow the base of open source innovations contributing to UNICEF's mission. Other Options: Private Sector investment, grant-funding, UNICEF networks, International Development Innovation Alliance (IDIA), UNICEF Innovation Procurement Acceleration Fund & Global Innovation Centre

8 Management, Monitoring & Evaluation

Experts from the ecosystem review champion-based proposals assessing: 1. Innovative relative to existing approaches? 2. Scientifically feasible & valid? 3. Team capability & unique resources? 4. Budget& timeline relative to complexity, risk & potential impact? 5. Balance picking the best individual proposals versus building a portfolio of complimentary projects.

Tiered, performance based approach ensuring failed ideas fail fast & cheaply. Pay-for-performance milestones applied.

Fund is managed by a Fund Board including the Executive Director with support of 3 senior advisors. A further Strategic Advisory Board consisting of internal & external experts in child rights, venture capital and emerging technology due to be established. Day-to-day management and support by a management team at UNICEF Ventures (NYC). Evaluation metrics: All innovations must be capable of producing real time data. Metrics such as number of users etc. are tracked real time via a website open to investors and investees.

9 Risk Management

Stage Gate financing based on rigorous evaluations. Multiple small bets. $100,000/100,000,000 = 0.1% financial risk on each seed. Financial risk on follow on investments greater due to larger size.

Stage Gate pay for performance. Multiple small bets. $100,000/70,000,000 = 0.14% financial risk on each seed. Follow on >.

One Stage investment only therefore avoiding risk of follow on investments which are funded by other mechanisms. Multiple small bets. $100,000/9,000,000 = 1.11% financial risk on each seed.

10 Evidence / Measuring Impact & Value of Interventions

Given the lengthy process to demonstrate proof of concept and subsequent barriers to achieve scale and impact a framework for intermediate evaluation of Return on Investment (ROI) is used as a proxy of lives saved & lives improved.

Outcomes: 50% use randomised control trials (RCT) (when intervention is unproven (lengthy & costly)). 50% use key performance indicators (KPIs) (quicker process-oriented).

All open source innovations must be capable of providing real time data which is easily measurable in terms of coverage, uptake, results, etc. Stage Gate 3 Pilot Analysis: 1.Social Return on Investment (positive impact per $ invested over time), 2. Cost Utility Analysis (ratio of cost to benefit in life adjusted years for health innovations) &3. Cost Effectiveness Analysis (comparing relative costs & outcomes of 2 or more courses of action).

11 Support Technical assistance provided. Tech. assistance provided. Technical assistance provided.

12 Returns Financial & social returns. Financial & social returns. 1. Growth of social technologies in global ecosystem, 2. IP stack from open source still has value e.g. U-Report an enterprise-level, cloud hosted, multi-tenancy solution for youth engagement, 3. Access to Communities of developers & entrepreneurs to extend the work or build other solutions.

Figure 81: Comparison of Portfolios Investing in Innovation for International Development (Ramalingam & Bound, 2016, UNICEF, 2016a-m, Meagher, 2016)


Page: 153

Appendix 7: Additional Literature

Research Objective 1: Evaluation of Individual Initiatives

Evaluation Uses

Evaluation can be used to satisfy accountability, legitimacy and efficiency (Moxham, 2014), to bear evidence

regarding value creation (Sillanpää, 2013), to improve organisational effectiveness and performance

benchmarking (Cordery and Sinclair, 2013), to manage risk (Benjamin, 2008) and to guide decision making

(Hatry, 1999 and LeRoux and Wright, 2010) (Melwani et al., 2016)

Various arguments state that they are too “ceremonial” in many organisation's (Hoefer, 2000), they show

legitimacy to external stakeholders (Greiling, 2010), that more comprehensive evaluations are required

Thomson (2010) vs. the argument that demands placed on time and resources are too large (Zappalà & Lyons,

2009; Benjamin, 2012) (Melwani et al., 2016).

Evaluations for the UNICEF Innovation Fund should consider the common qualities of transformative digital

technologies (Ramalingam and Bound, 2016, pp. 202-205).


Page: 154

Common Qualities of Transformative Digital Technologies

UNICEF Innovation Fund Criteria

1 Focused on appropriateness and relevance to culture and context. Focused on specific human needs grounded in context.

1. Registered in a UNICEF programme country (context). 3. Three portfolio areas: a. Products for youth under 25 to address a range of needs including

learning and youth participation; b. Real-time information for decision making and governance c. Infrastructure to increase access to services and information, including

but not limited to connectivity, power, finance, sensors and transport. 5. (a) Relevance of solution for children and young people. 6. (b) Alignment with Innovation ‘Principals for Digital Development’

Principal 1: Design with the user

Principal 2: Understand the ecosystem

2 Can be used by the poor and communities to meet their needs directly.

2. Working on open source solutions (or willing to be open source), under the following licenses: a. Software: BSD, AGPL, LGPL b. Content and designs: Creative Commons CC-BY-SA or CC-BY 6. (b) Alignment with Innovation ‘Principals for Digital Development’


Principal 2: Understand the ecosystem

Principal 6: Open data, open standards, open source, open innovation.

3 Not controlled by a single organisation.

2. Working on open source solutions 6. (b) Alignment with Innovation ‘Principals for Digital Development’


4 Often have empowerment and inclusion as key goals.

5. (b) Potential impact on access to information, opportunity and choice. 6. (b) Alignment with Innovation ‘Principals for Digital Development’


Principal 9: Be collaborative

5 Platforms which enable development of an infinite number of solutions, not just a single specific solution). Can enable adaptation and iteration.

2. Working on open source solutions (or willing to be open source), under the following licenses: a. Software: BSD, AGPL, LGPL b. Content and designs: Creative Commons CC-BY-SA or CC-BY 6. (b) Alignment with Innovation ‘Principals for Digital Development’


Principal 7: Reuse and Improve

Principal 9: Be collaborative

6 Can grow to considerable scale and lead to changes in whole systems.

5. (c) Potential application at global scale 6. (b) Alignment with Innovation ‘Principals for Digital Development’

Principal 3: Design for scale

Principal 4: Build for sustainability


Table 38: Common qualities of transformative digital technologies compared to UNICEF Innovation Fund Criteria (Ramalingam and Bound 2016, UNICEF 2016a-h) (Melwani, et al., 2016)

A literature review of social value creation in the context of innovation has been undertaken (Figure 84) and

Table 39 summarises a portion of the research and ranks the papers relevant to that portion.


Page: 155

Figure 82: Illustration of the sources researched along with some keywords and terms for objective 1 literature review


Page: 156

Summary of Best Practice Traditional Evaluation

Exemplars of best practice traditional frameworks are analysed in figure 59, though it is arguable that the

traditional method is too prescriptive for the UNICEF Innovation Fund investments. Those that adopt a more

flexible approach to the selection, definition and collection of indicators based on stakeholder interests,

program area and budget are ranked No.1 highest rank and No.5 lower ranked, as these are considered

methods that are less prescriptive and customizable to the innovations arising from the UNICEF Innovation

Fund.

1. CMS, 2015, 2. Miller & Hall, 2012. 3. Moxham, 2014 4. Micheli & Kennerley, 2005 5. Greatbanks et al., 2010. 6. Leipniz, 2014. 7. Benita and Balick, 2008. 8. Bubb, 2010.

1. Bassi & Vincenti, 2015, 2. Kroeger & Weber, 2014. 3. Moody et al., 2015, 4. Nicholls et al., 2009 5. Banke-Thomas et al., 2015, 6. Grieco, 2015. 7. Lynch-Cerullo & Cooney, 2011. 8. Emni et al., 2011, 9. Demirag & Khadaroo, 2011. 10. Krlev et al., 2014.

1. Biswas et al., 2010. 2. Ebrahim & Rangan, 2010. 3. Tuan, 2008, 4. Benjamin, 2012, 5. Liket et al., 2014. 6. Beesley et al., 2015 7. Cordery, 2013. 8. Choong, 2014 9. Carnochan, 2013 10. Chen, 2010 11.Kellogg Fdn. 2004 12.Zappalà & Lyons, 2009. 13. Sillanpää (2013)

1. Nichols et al. 2016 2. Serrano-Cinca, 2013 3. Wood & Leighton 2010. 4. Frigo, 2003. 5. Wu & Hung, 2008 6. Dawson, 2010. 7. DIFD, 2011. 8. Antadze & Westley, 2012.

1. USAID 2014 2. Polonsky et al., 2016 3. Cook et al., 1995. 4. Roper & Cheney, 2012

Peer Reviewed 2,3,4,5,6,7,8 1,2,3,5,6,7,9,10 4,5,7,8,9,10,13 2,4,5,6,8 4

Practitioner/Independent report

1 4, 8 1,3,4,6,11,12 3,7 1,3

Third sector/Public 1,2,3,4,5,6,7,8

1,2,3,4,5,6,7,8,9,10

ALL 1,2,3,4,5,6,7,8

1,2,3,4

Private sector 7 8,10,12

Case/pilot Study 6 1 1,5 6

Overview/Review of definitions, Principles, (dis)advantages of social value measurement methods/frameworks/ /metrics

2(SROI),3, 4,5,8

3(SROI), 5,6,7,8,10 3,4,7,8,9,10 (BSC),12

1,3, 4,6,7 1,2,3

Develops/implements frameworks/ methods/metrics

1,6,7 1,2,4 (SROI),5(SROI),8,9

1,2,5,10 (BSC),11 (Logic Model),13

2,5

Monetizing outcomes 4 1,3,5

Related to innovation 1,8 9,10 8 4

Refers to/develops a Logic model

4 1,3,4,6,7,8 1,2,3,5,7,9,12,13

7

Ranking First Second Third Fourth Fifth

Table 39: Social Value and Performance Measurement Research Table (Source: Meagher et al., 2016a)


Page: 157

SROI Network, 2012

Social Impact Investment Taskforce, 2014, UK presidency of G8

WBCSD Social Capital, 2013

Purpose Capital, 2013

J.P. Morgan, 2012

Geo, Grantmakers 2012

Research Method

Collection from various authors and advisory group

Review of 60 publications and 45 expert interviews, supported by Deloitte

Developed with more than 20 companies to support measurement

Literature review of key publications on social metrics, interviews with 20 Canadian investors in impact investing

Written for professional investors. Written from own investment experience and interviews with 23 institutional investors.

Conversations with other grant makers to prepare specific guide to help grant makers decision making

Measure-ment Method

Six Stage Process, detailed guidance at each one. Detailed calculations including use of Net Present Value, Payback and Sensitivity Analysis

Seven Guidelines, successfully applied against five diverse organisation's

Four Step Guide, specific for business use. Exclusive focus on socio-economic impact.

No specific Staged Process, Focuses on Non-financial social metrics

Investment Approach Measuring against Impact, Return and Risk, cross referenced against numerous case studies.

Four Essentials for Evaluation: Lead, Plan, Organize Share.

Useful for which stage? (feasibility, innovation, project, completion)

Can be applied as a forecast as final outcome

Assessment geared towards final outcome

Applicable to any stage

Applicable to any stage

Can be applied as a forecast as final outcome

Assessment geared towards final outcome

Detailed tools and metrics

Detailed tools and calculations included

None included, however industry tools are referenced for guidance

10 different tools for different socio-economic uses

11 different tools for different social impact metrics

Bespoke referenced but not included

None included or referenced

Adaptable for high innovation

Part adaptable

Part adaptable

No No No No

Ranking 1 2 3 4 5 6

Figure 83: Traditional Evaluation Exemplars (WBCSD, 2013; SIIT G8, 2014, Purpose Capital, 2013; SROI Network, 2012; JP Morgan, 2012) (Melwani, et al., 2016)


Page: 158

Traditional Evaluation Challenges

For innovative interventions Patton (2011) and ILO (2014) warn against the use of the traditional method as

the RCT’s are too premature for innovations that are still under development. Hence they will only produce

weak results that lead to poor decision making and inefficient use of resources. Social entrepreneurs (Conger,

2009) experienced that evaluation methods seem entirely unrelated to the nature of their enterprise.

Traditional evaluation frameworks are a static point in time measure, whereas, the process of innovation

operates in a dynamic environment. Hence, the development of evaluation tools for innovation in a non-profit

environment is fraught with challenges. The importance of developing reliable metrics is evident from high

failure rate associated with innovation (Chanaron, 2013, p. 18). Chesbrough (2004, p.24) highlights the

dilemma faced by many technology companies when dealing with ‘false negatives’ leading decision makers to

abandon innovations which were later proved viable (Melwani, et al., 2016)

Chanaron (2013) highlights the challenges due to the uncertainty linked with the probability of the projects

success and also explains that Traditional evaluation frameworks often fail to take account of “the [actual]

process or its ‘S’ curve representation” and often do not distinguish between the nature of innovation whether

it be incremental or disruptive (Melwani, et al., 2016)

Mulgan (2016) acknowledges the many challenges involved in assessing new technology which include the

following:

No one can predict precisely how technologies will evolve.

It is difficult to predict who will benefit.

It is extremely difficult to define the counterfactual to any given innovation.

Incumbents who stand to lose from innovation are likely to be well organised and powerful.

Despite the challenges Mulgan (2016) actively seeks input on how best to tackle such assessments and offers a

broad framework on which to build which includes the following:

Pragmatic stage gate approach to knowledge, risk and uncertainty.

Distinguish between different types of good and focus innovation to the ones with the greatest

potential (e.g. good with network effects or positive externalities that increase in value with increased

consumption).

Consider distributional effects (hard to predict).

Ethical assessment and the golden rule test (good innovations, the ones we want ourselves).

Contrary to the above recommendation of stage gates by Mulgan (2016), Nagji& Tuff (2012) warn of the

dangers of using such an approach asserting companies should use the polar opposite—a combination of

noneconomic and internal metrics—to assess transformational efforts in their early stages. They posit that this

can improve the team’s ability to learn and explore, citing the example of Google where the only hurdle such

an imitative must clear is its likelihood of learning from the process.


Page: 159

Cooper's (2014, p.20) article assesses what leading firms are doing to move beyond their current idea-to-

launch methodology and attempts to integrate these practices into a next-generation system. Cooper, who is a

creator of stage-gates, observes that in some organisations it’s an evolution of Stage-Gate to a better, faster

model; in others, it’s closer to a revolution, moving to a very different system.

Ruff & Olsen (2016) highlight that a common mistake, that evaluators seek out the best measurement tool. On

small scale this can work for many grant-making foundations who can apply a common tool across their

portfolio. However for large individual initiatives that differ in mission, theory of change within socio,

economic or geographic context the common measures do not work well as they ask the wrong questions,

measure the wrong things and miss the real impact.

Next Generation Evaluation

Gopalakrishnan and Preskill (2016) describe the Next Generation Evaluation and propose three methods which

will shape how evaluations are undertaken in the future. These include DE, Shared Measurement and Big Data.

DE as summarized above is used by leading foundations such as the Bill & Melinda Gates Foundation,

the J. W. McConnell Family Foundation, the John S. and James L. Knight Foundation, and the Grand

Rapids Community Foundation.

Shared Measurement aims to facilitate collaboration and speed to sharing information. The method

of evaluation for Project SAM was still based on traditional methods of surveys or short term

indicators to measure and track. It could have used on other methods however it arguably a way of

encouraging collaboration rather than a specific unique way of evaluation and thus it is debatable

whether it is a method in its own right. DE already encompasses collaboration and data sharing.

Big Data characterized by the “3 V's”—sheer volume, greater variety, and increased velocity will serve

to support DE and other methods such as impact measurement, RCD, and formative/summative

evaluation. As it is primarily a useful tool for generating and analysing data it is arguably another

supporting tool as interviews and personal interaction will still be needed to complete evaluations

such as DE.


Page: 160

DEVELOPMENTAL EVALUATION DIAGNOSTIC CHECKLISTS

Developmental evaluation is evaluation in support of social innovation and tackling complex issues. It is appropriate certain situations and conditions, and not others. The following assessment tool will help you assess whether a situation is suitable (or not) for a developmental evaluation, and whether your group is a good match for a developmental evaluation process. These checklists ask you to rate a series questions. The checklists are followed by a set of more open-ended questions that can help to start to plan your developmental evaluation. The checklist can be completed by an individual, or as a group, using the questions as a prompt for discussion. Once you have responded to all the statements, tally up the results, and make a judgement about if, when and how you might proceed. The sections are:

Developmental situation – is the initiative truly adaptive, requiring new and constantly adapted

approaches, or it a more conventional effort requiring efficient delivery of relatively established

responses?

Adaptive capacity – do the decision-makers, and those supporting them, have the capacity and

interest to work in the adaptive way required for social innovation and working with complex

issues?

Readiness for developmental evaluation – are the innovators involved ready to productively use

developmental evaluation?

Preparation for planning the developmental evaluation – some things to consider as you think

about how to gather information and interact with the innovators

Before you begin the assessment, keep the following things in mind:

1. In this document, ‘the group’ refers to the would-be innovators, which might be a small project

team, the board and staff of an organization, or the members of a collaboration. It may be

helpful to think of the ‘we’ in the questions from multiple perspectives (the evaluator, the

hands-on innovators, the organization’s decision makers, and the funder).

2. In this document, ‘intervention’ refers to any project, program, strategy or model that is in

some form of adaptive development (something innovative is being created and/or something

already established requires radical restructuring/rethinking.)

3. The checklists build on each other: if a situation is not innovative, there is no need to assess the

group’s readiness for adaptive leadership; similarly, if a group demonstrates weak adaptive

leadership, they will not make productive use of developmental evaluation feedback.

4. We do not describe how to gather the data or assess the data you might use to answer these

questions (we have plans to introduce this shortly). We do encourage you, however, to be

thorough, reflective and clear about the data you are using when you rate each statement.

The checklists are presented with a scoring tool. You may find the scoring a useful guide to help you see where you are at, or you may prefer to reflect on each of the statements without scoring using the questions as a way of prompting your own thinking. Please proceed in the way that you find the most beneficial.

Figure 84: Developmental Evaluation Diagnostic Checklist (Cabaj and Gamble, 2011)


Page: 161

IS IT A DEVELOPMENTAL SITUATION?

Statement Strongly Disagree

Disagree Neutral

Agree Strongly Agree

1 The challenge we want to address is difficult to define.

1 2 3 4 5

2 There are multiple causes that contribute to the challenge, and these are inter-dependent and difficult to predict.

1 2 3 4 5

3 The factors that contribute to progress on meeting this challenge are unknown or unclear.

1 2 3 4 5

4 The stakeholders involved (directly and indirectly) have diverse values, interests and perspectives.

1 2 3 4 5

5 The group is experimenting with different ways to turn their theory into reality, (e.g. a grant program, a training course) and these are not-yet-established nor tested.

1 2 3 4 5

6 The innovation’s outcomes (types, scale, speed) are uncertain and/or unpredictable.

1 2 3 4 5

7 The context in which the group is operating (e.g. funding, partners, demographics, stakeholders) is rapidly changing.

1 2 3 4 5

Sub-Total 32

Total

Results

7 to 12 Non-developmental Situation

Your intervention is very well developed and may be working in a stable environment. You may be seeking evaluation feedback for accountability which aims to find out if you are implementing it with fidelity to a well laid out and proven model.

13 to 20 Effectiveness Situation Your intervention is very well developed. You may be seeking evaluation feedback to judge the model’s effectiveness.

21 to 29 Improvement Situation Your intervention is relatively stable and/or operating in a stable environment. You may be seeking evaluation feedback to improve the model.

30 to 35 Developmental Situation

Your intervention is either not yet clear or in need of radical restructuring. You may be seeking evaluation feedback to develop the model.

Figure 84 (continued): Developmental Evaluation Diagnostic Checklist (Cabaj and Gamble, 2011)


Page: 162

DO YOU HAVE ADAPTIVE CAPACITY?

Statement Strongly Disagree

Disagree Neutral

Agree Strongly Agree

1 We have a history of innovation and tackling complex challenges.

1 2 3 4 5

2 We are comfortable with ambiguity, uncertainty, and the tension of adaptive work.

1 2 3 4 5

3 We are motivated to try something new and committed to a systematic process of innovation.

1 2 3 4 5

4 We have sufficient resources to carry out its work, and can invest more if/when promising new avenues emerge.

1 2 3 4 5

5 We are willing to “learn-by-doing”, allowing the intervention to emerge over time, rather “plan the work and work the plan”.

1 2 3 4 5

6 We have the flexibility and authority to change the emerging intervention to reflect new learnings and shifts in the environment.

1 2 3 4 5

7 We have permission and room to make “safe-to-fail” errors and mistakes in search of what works.

1 2 3 4 5

8 We are more interested in learning and getting results, than being perceived to be “right”.

1 2 3 4 5

9 We have time and patience to experiment with new approaches and generate results.

1 2 3 4 5

Sub-Total 37

Total 69

Results

9 to 18 Non-Existent Your group is working with a fairly rigid context that does not allow it to engage in an authentic process of exploration and innovation.

19 to 24 Low Your group’s ability to work adaptively is very limited. You should proceed with extreme care (if at all) and work hard at addressing your weak areas before or during the innovation process.

25 to 37 Good Your group has the adaptive capacity to move forward, though some areas may need extra attention before or during the innovation process.

38 to 45 Excellent Your group is in an excellent position to innovate and/or work on complex issues.



Page: 163

ARE YOU READY FOR DEVELOPMENTAL EVALUATION? Statement Strongly

Disagree Disagree Neutral

Agree Strongly

Agree

1 We are hungry for feedback on our work.

1 2 3 4 5

2 We understand that we all operate with cultural and cognitive biases that “shape” the way we think about and see the world.

1 2 3 4 5

3 We have a history of gathering, analysing and using evaluation to inform our work (or we are fully prepared to going forward).

1 2 3 4 5

4 The group has a culture of curiosity, inquiry and critical reflection.

1 2 3 4 5

5 We have a demonstrated commitment to “data-based” decision-making.

1 2 3 4 5

6 We’ve had positive experiences with evaluation (and evaluators) in the past.

1 2 3 4 5

7 We understand and broadly support developmental evaluation.

1 2 3 4 5

8 We are prepared to commit time and resources to the developmental evaluation?

1 2 3 4 5

9 We have someone (internal or external) clearly in the role of developmental evaluator.

1 2 3 4 5

Sub-Total 33

Total 102

Results

9 to 18 Poor Your group’s readiness is too low to proceed. You require significant work to improve the conditions for developmental evaluation before you move forward.

19 to 24 Low Your group readiness for developmental evaluation is limited. Proceed with caution. Address short-comings before you begin and/or intentionally approach the work moving forward as an opportunity to strengthen your capacity for developmental evaluation. Be prepared for the fact that you may choose to discontinue developmental evaluation mid-way through the process, or you may say you are doing developmental evaluation when you are in-fact using evaluation in more of a formative or summative mode.

25 to 37 Medium Your group is sufficiently ready for developmental evaluation to begin, though it should keep on an eye on its weaker areas of readiness and/or identify measures to strengthen them as you proceed.

38 to 45 High You group is an excellent candidate for developmental evaluation.



Page: 164

Planning the Developmental Evaluation Once you have determined if a developmental evaluation is right for your initiative, the following questions will help you begin to plan.

1. What are the major questions that the innovators involved with this initiative are interested in answering?

2. What are the decisions they anticipate making in relation to this initiative?

3. When will those decisions get made? By whom?

4. What information, findings and analysis would support them in learning more about their

innovation and inform effective decision making about it?

5. What is the frequency with which this initiative implements different iterations? Is it an activity that is carried out on a weekly basis (meaning there is opportunity for rapid feedback and on-going changes) or is it the sort of thing that only happens once or twice in a year?

6. When this innovation is being ‘tried-out’, are there opportunities to test out different variations

(for example, multiple groups, different geographies...)?

7. What is the consequence if something you try out is not successful? Would it negatively affect anyone? Impact the organization’s reputation? Would it be expensive? Are there possible harmful implications?

8. How would you characterize the information preferences and learning style of the innovators

who are involved? Do they prefer/trust certain kinds of data more than others? How do they process information when making decisions?



Page: 165

Research Objective 2 Literature Review: Evaluation of Innovation Fund Portfolio

The literature review for objective 2 in section 2.4 above summarises key considerations for portfolio evaluation from the literatures most prominent authors on portfolio

evaluation. Below is an illustration of the breadth of authors researched and some keywords and terms.

Traditional evaluation

Expected Return

Value accrues in indirect ways

High uncertainty

Economic value difficult to quantify

Paulson et al (2007)

Value maximisation

Mix and balance

Project prioritisation

Strategic alignment

multiple methods to select projects

willingness to continuously

assessoptimise the

portfolio

Risk appetite

Embracing failure

resource allocation

IPM

Interdependencies

Time horizon

Flexibility

Tailored approach

Inappropriate decision rules e.g. NPV, hurdle rates

Cost of the evaluation

coherent portfolio strategy

Day (2007)

Strategic buckets

Real options

Scorecards

Profitability Index

performance measurement

strategicmisrepresentation

Optimism bias

IRR

Sensitivity analysis

Evaluative Questions

CBASROI

CEA

Optioneering

Portfolio matrices

AHP

Scoring models

ROI

Leadership Champion

Developmental Evaluation

High Innovation

Realistic Evaluation

rate of attrition

Specific Versus Standardised

evaluation metrics

Figure 85: Illustration of the authors researched and some keywords and terms for objective 2 literature review on portfolio evaluation


Page: 166

Paulson (2007) Portfolio Evaluation Tool Methodology and Adaptation.

This tool should be completed by the portfolio manager with input from each project leader. Sections 1 to 10

are project/initiative level considerations which requires the portfolio manager with input from the project

leaders to rate a series of 10 sets of questions (called Value drivers) about each initiative in the portfolio. For

each question, two responses are required: a) an importance weighting/relevance indicating that driver of

value's relevance to the specific project, and b) an assessment of the extent to which the initiative exhibits that

driver of value. In the first column for each initiative, place a 0 next to questions which are not relevant to the

specific initiative and a 1 next to items that are important to consider for that project. There is an opportunity

to give each question higher/lower rating by lowering/increasing the relevance. This should be done for each

initiative in the portfolio. An index score is generated for each initiative as presented in the main findings

above. The resultant 'Performance Profile' bar chart indicates the areas of strengths and weaknesses for each

project, and cumulatively for the portfolio.

Subsequent to the completion of sections 1 to 10, Section 11 'Portfolio health' is completed and this is where

Portfolio level considerations are addressed. The same methodology should be followed here. Again an index

score is generated for the overall portfolio health as presented in the main findings above.

According to Paulson et al. (2007, cited in Meagher et al. 2016) when designing an innovation portfolio

evaluation tool, the following elements should be considered;

Address the interrelationships in risk among potential innovation investments and among current

businesses. Also to consider the return relationship between the initiatives within the same portfolio.

The tool must consider the learning and discovery in a flexible approach that considers all the

information and data within the company and outside the company. This Flexibility approach is

required since the innovation environment is highly dynamic.

The evaluation of a project in high-uncertainty environment cannot be summarized from single point

of view such as the dollar term of generated cash flows since this demission might be misleading at

the initial stages of evaluation.

The tool was adapted to include UIF's initiative selection and assessment criteria, UIF's expected returns UIF

and portfolio areas, UNICEF's innovation Principles, UNICEF IoGT & U-Report Log frames, Interview findings,

UNICEF GIC Monitoring, Evaluation and Learning (MEL) Framework & documentation on measurement metrics

from the IDIA. The aim was to strike a balance between maintaining the intent and theoretical basis of the

Paulson et. al. (2007) tool whilst at the same time making the tool as context specific and useful as possible.


Page: 167

Paulson et al. (2007) Portfolio Evaluation Tool Application

As highlighted above, sections 1 to 10 were completed (except for section 8 which was not attempted as the

authors have no knowledge of the make-up and dynamics of the start-up teams) for each of UNICEF

Innovation Funds current investments generating an index score for each initiative which is illustrated by the

bar chart in Figure 46 in the main body of this report. The following tables exhibit the application of the tool

for all current UNICEF Innovation Fund investments. There are two tables per section as all 14 initiatives will

not fit horizontally across one page/table.

An aggregated performance profile is generated for each value driver as shown in figure 47 above. In this

application 'sustainability', 'scalability' and 'Firm capabilities (UIF)' score well (low-mid 70's), whereas 'projects

impact on company renewal' and 'Innovativeness' scored low-mid 60's. This second score is a result of the

portfolio mix at the time of this study which includes many internal initiatives based on adjacent technology.

These initiatives were invested in first by the UNICEF Office of Innovation to launch the Fund and many of

those type of initiatives, such as U-Report and RapidPro solutions, have since moved to scale. More innovative

initiatives are expected in the next rounds of investments as the UNICEF Innovation Fund matures and as the

ecosystem grows.

As highlighted above the tool enables the assessment of the relative values of initiatives within the context of

the entire innovation portfolio at any point in time and changes in the relative value of the portfolio can be

tracked over time (Paulson et al., 2007). Mom-connect, Seycel and 9needs were the highest scoring initiatives

and whilst the subjective and assumptive nature of application due to the time and proximity limitations of

this research project is noted, the authors believe that application of the tool by those who are part of the

initiatives development process will generate valuable information thus making a valuable contribution to the

learning and feedback process.


Page: 168

Section 1

I

Potential impact on accelerating results for beneficiaries: The

extent to which this initiative is likely to have a significant impact in

bringing wholly new benefits or significant leaps in known benefits

to beneficiaries.

Weighting 0.25

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

1.1Expected/projected lives saved or improved from this initiative are

substantial1 3 1 3 1 3 1 3 1 3 1 4 1 5

1.2Solution has the potential to positively impact the lives of the most

vulnerable children 1 3 1 3 1 3 1 3 1 3 1 4 1 4

1.3 The available evidence indicates effectiveness of the initiative 1 3 1 3 1 3 1 3 1 2 1 3 1 4

1.4There are no insurmountable barriers that users will face in

accessing the product/service.1 3 1 3 1 3 1 3 1 3 1 3 1 3

1.5 The initiative does no harm. 1 5 1 5 1 5 1 5 1 5 1 5 1 5

1.6The actual number of lives saved or improved from this initiative is

significant. [Outcome indicator]0 - 0 - 0 - 0 - 0 - 0 - 0 -

1.7The initiative has brought behaviour change amongst the target

population [Outcome indicator]0 - 0 - 0 - 0 - 0 - 0 - 0 -

1.8

Engaged users show improved knowledge and awareness and more

positive attitudes/are more empowered to voice opinions.

[Outcome indicator]

0 - 0 - 0 - 0 - 0 - 0 - 0 -

Total Value 17 17 17 17 16 19 21


Initiative Aggregate /Potential impact for beneficiaries

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

70.33% 62.5 68.0 62.5 68.0 62.5 68.0 62.5 68.0 62.5 64.0 62.5 76.0 62.5 84.0

U-Report

Brazil

Advocacy

Project

MomCon-

nect (South

Africa)

U-REPORT

Fiji

U-Report

Guatemala

Care for

Children


Figure 86 (Sheet 1 of 2): Portfolio Evaluation tool Section 1: Potential impact on accelerating results (Adapted by the authors from Paulson et al., 2007)

U-Report AppU-Report

Burkina Faso


Page: 169

Section 1

I

Potential impact on accelerating results for beneficiaries: The

extent to which this initiative is likely to have a significant impact in

bringing wholly new benefits or significant leaps in known benefits

to beneficiaries.

Weighting 0.25

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

1.1Expected/projected lives saved or improved from this initiative are

substantial1 3 1 3 1 3 1 3 1 3 1 3 1 3

1.2Solution has the potential to positively impact the lives of the most

vulnerable children 1 3 1 3 1 3 1 3 1 3 1 4 1 3

1.3 The available evidence indicates effectiveness of the initiative 1 3 1 3 1 3 1 3 1 3 1 4 1 3



1.5 The initiative does no harm. 1 5 1 5 1 5 1 5 1 5 1 5 1 5

1.6The actual number of lives saved or improved from this initiative is

significant. [Outcome indicator]0 - 0 - 0 - 0 - 0 - 0 - 0 -

1.7The initiative has brought behaviour change amongst the target

population [Outcome indicator]0 - 0 - 0 - 0 - 0 - 0 - 0 -

1.8

Engaged users show improved knowledge and awareness and more

positive attitudes/are more empowered to voice opinions.

[Outcome indicator]

0 - 0 - 0 - 0 - 0 - 0 - 0 -

Total Value 18 18 17 17 17 20 17


Initiative Aggregate /Potential impact for beneficiaries

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

70.33% 62.5 72.0 62.5 72.0 62.5 68.0 62.5 68.0 62.5 68.0 62.5 80.0 62.5 68.0

Response Per Individual Initiative (initiatives 8 to 14)

Saycel IPAL

Figure 86 (sheet 2 of 2): Portfolio Evaluation tool Section 1: Potential impact on accelerating results (Adapted by the authors from Paulson et al., 2007)

Better

Teachers,

Better

Learners

9Needs Mpower MAPP VACSomleng


Page: 170

Section 2

II





Weighting 0.06Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)



2.2




1 4 1 4 1 4 1 4 1 4 0 1 5




applications of this initiative.0 - 0 - 0 - 0 - 0 1 3 1 4

2.5





1 3 1 3 1 3 1 3 1 4 1 4 1 4


threat.0 - 0 - 0 - 0 0 0 0 -

Total Value 14 14 14 14 14 14 22



% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

71.84% 66.7 70.0 66.7 70.0 66.7 70.0 66.7 70.0 66.7 70.0 66.7 70.0 83.3 88.0


U-Report

Guatemala

U-Report

Brazil

Advocacy

Project

Figure 87 (Sheet 1 of 2): Portfolio Evaluation tool Section 2:Potential impact on the market (Adapted by the authors from Paulson et al., 2007)

U-REPORT

Fiji

U-Report

Burkina Faso

U-Report

App

Care for

Children

MomCon-

nect (South

Africa)


Page: 171

Section 2

II





Weighting 0.06Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)



2.2




1 3 1 3 1 4 1 4 1 4 1 4 1 3




applications of this initiative.0 0 0 0 0 0 0

2.5





1 3 1 3 1 3 1 3 1 4 1 4 1 3


threat.0 0 0 0 0 0 0

Total Value 14 12 15 15 14 16 12



% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

71.84% 66.7 70.0 66.7 60.0 66.7 75.0 66.7 75.0 66.7 70.0 66.7 80.0 66.7 60.0

Figure 87 (Sheet 2 of 2): Portfolio Evaluation tool Section 2:Potential impact on the market (Adapted by the authors from Paulson et al., 2007)

Somleng


9NeedsIPAL

Better

Teachers,

Better

Learners

Saycel MAPP VACMpower


Page: 172

Section 3

III

Initiatives Impact on UIF's Growth, Renewal & Secondary benefits:

The extent to which this initiative leverages UIF's current strengths

to develop existing & new lines of growth (e.g. causing UIF to learn

about and move into unfamiliar domains) whilst als

Weighting 0.06

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

3.1

The new initiative will promote a sustainable pipeline of innovation

oriented initiatives, either through the area of technology it focuses

on, or the external leads developed, etc.

1 2 1 2 1 2 1 2 1 3 1 2 1 5

3.2The initiative creates significant potential for growth among new

customer segments not currently served by the organisation.1 2 1 2 1 2 1 2 1 3 1 3 1 5

3.3This initiative has engendered large numbers of potential

applications and options for potential applications.1 3 1 3 1 3 1 3 1 3 1 2 1 5

3.4This initiative is requiring the team to develop/ leverage internal

networks.1 3 1 3 1 3 1 3 1 3 1 4 1 4

3.5The initiative is requiring the team to develop/leverage external

networks.1 3 1 3 1 3 1 3 1 3 1 3 1 4

3.6

The initiative contributes significantly to key secondary benefits

(e.g. exposure to new market partners/development of ecosystem,

organisational learning, new capabilities, new technology frontiers,

new business models etc.)

1 2 1 2 1 2 1 2 1 3 1 4 1 4



3.8 The initiative utilises GIC/UIF expertise. 1 5 1 5 1 5 1 3 1 4 1 3 1 5

3.9The aggregate market potential for all applications coming from this

initiative is significant to UNICEF.1 4 1 4 1 4 1 4 1 3 1 2 1 5

3.10The timing of expected first value creation for this initiative is

reasonable.1 4 1 4 1 4 1 5 1 3 1 4 1 4

3.11Initiative makes viable Business case for the private sector

application 0 - 0 - 0 - 0 - 0 1 2 1 5

Total Value 31 31 31 30 31 32 49


Initiative Aggregate / Company Growth & Secondary benefits:

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

65.65% 100.0 62.0 100.0 62.0 100.0 62.0 100.0 60.0 100.0 62.0 100.0 58.2 100.0 89.1


U-Report

Guatemala

U-Report

Brazil

Advocacy

Project

Care for

Children

U-Report

Burkina Faso

MomCon-

nect (South

Africa)

Figure 88 (Sheet 1 of 2): Portfolio Evaluation tool Section 3: Company Growth & Secondary benefits: (Adapted by the authors from Paulson et al., 2007)

U-Report

App

U-REPORT

Fiji


Page: 173

Section 3

III

Initiatives Impact on UIF's Growth, Renewal & Secondary benefits:

The extent to which this initiative leverages UIF's current strengths

to develop existing & new lines of growth (e.g. causing UIF to learn

about and move into unfamiliar domains) whilst als

Weighting 0.06

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

3.1

The new initiative will promote a sustainable pipeline of innovation

oriented initiatives, either through the area of technology it focuses

on, or the external leads developed, etc.

1 4 1 3 1 4 1 3 1 3 1 3 1 3

3.2The initiative creates significant potential for growth among new

customer segments not currently served by the organisation.1 3 1 3 1 4 1 3 1 3 1 3 1 3

3.3This initiative has engendered large numbers of potential

applications and options for potential applications.1 3 1 3 1 3 1 3 1 3 1 3 1 3


networks.1 3 1 3 1 3 1 3 1 3 1 3 1 3


networks.1 4 1 3 1 4 1 4 1 3 1 4 1 3

3.6

The initiative contributes significantly to key secondary benefits

(e.g. exposure to new market partners/development of ecosystem,

organisational learning, new capabilities, new technology frontiers,

new business models etc.)

1 3 1 3 1 5 1 4 1 3 1 3 1 3



3.8 The initiative utilises GIC/UIF expertise. 1 3 1 4 1 4 1 4 1 4 1 4 1 3

3.9The aggregate market potential for all applications coming from this

initiative is significant to UNICEF.1 3 1 4 1 4 1 3 1 3 1 3 1 3

3.10The timing of expected first value creation for this initiative is

reasonable.1 4 1 4 1 4 1 4 1 4 1 4 1 4

3.11Initiative makes viable Business case for the private sector

application 0 0 1 4 0 0 0 0

Total Value 34 33 43 35 32 34 31


Initiative Aggregate / Company Growth & Secondary benefits:

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

65.65% 100.0 68.0 100.0 66.0 100.0 78.2 100.0 70.0 100.0 64.0 100.0 68.0 100.0 62.0


IPAL MAPP VAC

Better

Teachers,

Better

Learners

Figure 88 (Sheet 2 of 2): Portfolio Evaluation tool Section 3: Company Growth & Secondary benefits: (Adapted by the authors from Paulson et al., 2007)

9Needs Mpower SomlengSaycel


Page: 174

Section 4

IVSustainability: the extent to which sustainability in terms of

initiative/fund longevity/performance is enhanced.

Weighting 0.15

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

4.1This initiative does not threaten the sustainability of other

initiatives within the portfolio.1 5 1 5 1 5 1 5 1 4 1 4 1 4

4.2

Initiative sustainability is enhanced given its fit within the portfolio

and its linkages with the ecosystem and the system of support

provided by the UIF.

1 4 1 4 1 4 1 4 1 4 1 3 1 4

4.3

Both the initiative team and the company sponsored observe good

governance and sound financial practices, particularly with regard

to budgeting, reporting and monitoring.

0 0 0 0 0 0 1 4

4.4 The initiative leverages on economies of scale/scope. 1 3 1 3 1 3 1 3 1 3 1 5 1 5

4.5Sustainability is supported given the viability and affordability of the

product/service being developed.1 3 1 4 1 3 1 3 1 3 1 4 1 4

4.6

No significant/highly probable risks (e.g. Regulatory barriers) exist

that could undermine the sustainability of the solution (e.g. reliance

on single supplier, unsupportive policy environment etc).

1 3 1 3 1 3 1 3 1 3 1 2 1 3

4.7Robustness of results of initial prototyping/piloting/ past data

collection and initiative milestones are high.1 4 1 3 1 3 1 4 1 3 1 3 1 5

4.8To the best of our knowledge there are no competing approaches

to solve the problem that this initiative works towards solving.1 4 1 4 1 4 1 4 1 3 1 4 1 4

4.9 The initiative has the Potential to influence policy / systems change 1 4 1 4 1 4 1 4 1 3 1 5

4.10Smart partners (especially from country governments and

companies/investors) willing to co-fund0 - 0 - 0 - 0 - 0 - 0 - 0 -

4.11 Proven entrepreneurial success of the team 0 - 0 - 0 - 0 - 0 - 0 - 0 -

4.12The possibility of negative repercussions from beneficiaries in terms

of social or political implications of this initiative is low.1 4 1 4 1 4 1 5 1 5 1 5

Total Value 9 34 34 29 34 31 30 43


Initiative Aggregate /Sustainability

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

76.26% 75.0 75.6 75.0 75.6 72.7 72.5 75.0 75.6 75.0 68.9 72.7 75.0 83.3 86.0

U-Report

App

U-REPORT

Fiji

U-Report

Guatemala

U-Report

Brazil

Advocacy

U-Report

Burkina Faso


Figure 89 (sheet 1 of 2): Portfolio Evaluation tool Section 4: Sustainability (Adapted by the authors from Paulson et al., 2007)

MomCon-

nect (South

Africa)

Care for

Children


Page: 175

Section 4

IVSustainability: the extent to which sustainability in terms of

initiative/fund longevity/performance is enhanced.

Weighting 0.15

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

4.1This initiative does not threaten the sustainability of other

initiatives within the portfolio.1 5 1 5 1 3 1 3 1 4 1 4 1 3

4.2

Initiative sustainability is enhanced given its fit within the portfolio

and its linkages with the ecosystem and the system of support

provided by the UIF.

1 3 1 4 1 4 1 4 1 3 1 4 1 3

4.3




0 0 0 0 0 0 0

4.4 The initiative leverages on economies of scale/scope. 1 4 1 4 1 4 1 4 1 3 1 4 1 3

4.5Sustainability is supported given the viability and affordability of the

product/service being developed.1 4 1 3 1 4 1 3 1 3 1 4 1 3

4.6

No significant/highly probable risks (e.g. Regulatory barriers) exist

that could undermine the sustainability of the solution (e.g. reliance

on single supplier, unsupportive policy environment etc).

1 3 1 3 1 3 1 3 1 3 1 3 1 3

4.7Robustness of results of initial prototyping/piloting/ past data

collection and initiative milestones are high.1 4 1 4 1 3 1 4 1 3 1 3 1 3

4.8To the best of our knowledge there are no competing approaches

to solve the problem that this initiative works towards solving.1 5 1 4 1 3 1 3 1 3 1 3 1 3

4.9 The initiative has the Potential to influence policy / systems change 1 3 1 2 1 4 1 3 1 3 1 3 1 2

4.10Smart partners (especially from country governments and

companies/investors) willing to co-fund0 - 0 - 0 - 0 - 0 - 0 - 0 -

4.11 Proven entrepreneurial success of the team 0 - 0 - 0 - 0 - 0 - 0 - 0 -

4.12The possibility of negative repercussions from beneficiaries in terms

of social or political implications of this initiative is low.1 4 1 5 1 5 1 5 1 5 1 4 1 5

Total Value 35 34 33 32 30 32 28


Initiative Aggregate /Sustainability

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

76.26% 75.0 77.8 75.0 75.6 75.0 73.3 75.0 71.1 75.0 66.7 75.0 71.1 75.0 62.2

Figure 89 (Sheet 2 of 2): Portfolio Evaluation tool Section 4: Sustainability (Adapted by the authors from Paulson et al., 2007)

Better

Teachers,

Better

MAPP VACSomlengMpower9NeedsIPALSaycel



Page: 176

Section 5



Weighting 0.15

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

5.1 The initiative has a viable Business model 1 4 1 4 1 4 1 4 1 3 1 4 1 4

5.2




1 3 1 4 1 4 1 4 1 3 1 4 1 5

5.3




1 3 1 3 1 3 1 3 1 3 1 4 1 5


5.5


could be easily replicated, with contextualized tailoring only needed

for limited aspects of the program.

1 4 1 4 1 4 1 4 1 4 1 3 1 4



5.8The initiative is cost effective to enable integration into government

or other large-scale programmes.1 4 1 4 1 4 1 4 1 4 1 4 1 3

5.9 Expected demand / market readiness is high 1 4 1 4 1 4 1 4 1 4 1 5

33 34 34 34 32 31 37Maximum Potential Value 45 45 45 45 45 40 45


% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

75.89% 100.0 73.3 100.0 75.6 100.0 75.6 100.0 75.6 100.0 71.1 100.0 77.5 100.0 82.2

Care for

Children

Figure 90 (Sheet 1 of 2) Portfolio Evaluation tool Section 5: Scalability (Adapted by the authors from Paulson et al., 2007)

MomCon-

nect (South

Africa)

U-Report

Burkina Faso

U-Report

Brazil

Advocacy

Project

U-Report

Guatemala


U-REPORT

Fiji

U-Report

App


Page: 177

Section 5



Weighting 0.15

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

5.1 The initiative has a viable Business model1 4

1 4 1 4 1 4 1 4 1 3 1 3

5.2




1 3 1 4 1 4 1 4 1 4 1 3 1 3

5.3




1 3 1 4 1 4 1 4 1 3 1 3 1 3


5.5


could be easily replicated, with contextualized tailoring only needed

for limited aspects of the program.

1 4 1 4 1 4 1 4 1 4 1 4 1 4



5.8The initiative is cost effective to enable integration into government

or other large-scale programmes.1 4 1 4 1 4 1 4 1 4 1 4 1 4

5.9 Expected demand / market readiness is high 1 4 1 3 1 3 1 3 1 3 1 3 1 3

33 36 34 34 34 31 30



% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

75.89% 100.0 73.3 100.0 80.0 100.0 75.6 100.0 75.6 100.0 75.6 100.0 68.9 100.0 66.7

MAPP VAC9Needs Mpower SomlengSaycel IPAL

Figure 90 (Sheet 2 of 2): Portfolio Evaluation tool Section 5: Scalability (Adapted by the authors from Paulson et al., 2007)

Response Per Individual Initiative (initiatives 8 to 14)Better

Teachers,

Better


Page: 178

Section 6

VIInnovation: Novelty of initiative and alignment with Innovation Principles

and desired stage of development.

Weighting 0.60

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

6.1 The initiative is developing new technology 0 - 0 - 0 - 0 0 0 - 1 1

6.2

Initiative is expanding/ improving existing technology/ platform (i.e.

it will be in a place where the technology is being used...but will

involve new tech being built)

0 - 0 - 0 - 0 0 0 - 1 5

6.3

Initiative is a new application/ piloting new use case for existing

technology (i.e little or no new tech will be built and it will be used

in the same place)

1 2 1 2 1 2 1 2 1 3 1 4 1 4

6.4Scaling existing, initial pilot to new areas (i.e. no significant new

tech, but also this will be in a new country)0 - 0 - 0 - 0 - 0 0 - 1 4

6.5 The initiative aligns with all 9 UNICEF Innovation Principles 1 4 1 4 1 4 1 4 1 3 1 4 1 5

6.6The innovation is in line with most recent developments in related

technology space. 1 3 1 3 1 3 1 2 1 3 1 4 1 3

6.7 The business model itself is innovative. 1 2 1 2 1 2 1 3 1 2 1 3 1 5

6.8The model has relative advantage over existing practices (Current

solutions are considered inadequate).1 3 1 3 1 3 1 3 1 3 1 4 1 4

Total Value 14 14 14 14 14 19 31


initiative Aggregate / Innovation

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

68.25% 62.5 56.0 62.5 56.0 62.5 56.0 62.5 56.0 62.5 56.0 62.5 76.0 100.0 77.5

Figure 91 (sheet 1 of 2): Portfolio Evaluation tool Section 6: Innovation (Adapted by the authors from Paulson, et al., 2007)

U-Report

Brazil

Advocacy

Project

U-REPORT

Fiji


U-Report

Burkina Faso

U-Report

Guatemala

U-Report

App

Care for

Children

MomCon-

nect (South

Africa)


Page: 179

Section 6

VIInnovation: Novelty of initiative and alignment with Innovation Principles

and desired stage of development.

Weighting 0.60

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

6.1 The initiative is developing new technology 0 0 0 0 0 0 0

6.2

Initiative is expanding/ improving existing technology/ platform (i.e.

it will be in a place where the technology is being used...but will

involve new tech being built)

1 4 0 1 4 0 0 0 0

6.3

Initiative is a new application/ piloting new use case for existing

technology (i.e little or no new tech will be built and it will be used

in the same place)

0 1 4 0 1 4 1 4 1 3 1 3

6.4Scaling existing, initial pilot to new areas (i.e. no significant new

tech, but also this will be in a new country)0 0 0 0 0 0 0

6.5 The initiative aligns with all 9 UNICEF Innovation Principles 1 4 1 4 1 4 1 4 1 4 1 3 1 4

6.6The innovation is in line with most recent developments in related

technology space. 1 3 1 4 1 5 1 4 1 3 1 3 1 3

6.7 The business model itself is innovative. 1 3 1 3 1 4 1 3 1 3 1 2 1 3

6.8The model has relative advantage over existing practices (Current

solutions are considered inadequate).1 4 1 4 1 4 1 4 1 4 1 3 1 4

Total Value 18 19 21 19 18 14 17


initiative Aggregate / Innovation

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

68.25% 62.5 72.0 62.5 76.0 62.5 84.0 62.5 76.0 62.5 72.0 62.5 56.0 62.5 68.0


9Needs Mpower SomlengSaycel

Figure 91 (sheet 2 of 2): Portfolio Evaluation tool Section 6: Innovation (Adapted by the authors from Paulson, et al., 2007)

IPAL MAPP VAC

Better

Teachers,

Better

Learners


Page: 180

Section 7

VII

Initiative's Impact on the Portfolio: The extent to which the

initiative helps bring the portfolio to a healthy balance on any of

our desired dimensions for portfolio diversification.

Weighting 0.10

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

7.1There are possibilities for synergistic effects between this initiative

and other in the portfolio.1 4 1 4 1 4 1 4 1 4 1 4 1 3

7.2 If this initiative were killed, the portfolio would suffer. 1 2 1 2 1 2 1 2 1 3 1 2 1 2

7.3This initiative provides balance to the portfolio of innovations in

terms of pacing/stage of maturity of initiatives. 1 3 1 3 1 3 1 3 1 4 1 4 1 5

7.4This initiative is providing balance to the portfolio of innovations in

terms of desired risk in the portfolio.1 2 1 2 1 2 1 2 1 3 1 3 1 5


terms of technology domains we’re invested in.1 2 1 2 1 2 1 2 1 2 1 4 1 2

7.6This initiative is providing balance to the portfolio in terms of

expected returns (Growth, IP Stack, Communities). 1 2 1 2 1 2 1 2 1 3 1 4 1 4

7.7

This initiative is providing balance in terms UIF portfolio areas 1.

Products for youth under 25 2. Real‐time information for decision

making 3. Infrastructure to increase access to services and

information.

1 2 1 2 1 2 1 2 1 2 1 5 1 5

7.8At the portfolio level, committing to this initiative continues to

enhance the secondary benefits which accrue to the fund.1 3 1 3 1 3 1 3 1 3 1 5 1 4

Total Value 20 20 20 20 24 31 30


Initiative Aggregate / Impact on Portfolio

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

68.96% 100.0 50.0 100.0 50.0 100.0 50.0 100.0 50.0 100.0 60.0 100.0 77.5 100.0 75.0

U-Report

Brazil

Advocacy

Project

U-Report

Burkina Faso


Figure 92 (Sheet 1 of 2): Portfolio Evaluation tool Section 7: Initiative's Impact on the Portfolio (Adapted by the authors from Paulson et al., 2007)

MomCon-

nect (South

Africa)

Care for

Children

U-Report

App

U-REPORT

Fiji

U-Report

Guatemala


Page: 181

Section 7

VII

Initiative's Impact on the Portfolio: The extent to which the

initiative helps bring the portfolio to a healthy balance on any of

our desired dimensions for portfolio diversification.

Weighting 0.10

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

7.1There are possibilities for synergistic effects between this initiative

and other in the portfolio.1 4 1 4 1 4 1 4 1 4 1 4 1 3

7.2 If this initiative were killed, the portfolio would suffer. 1 4 1 4 1 5 1 4 1 3 1 3 1 3

7.3This initiative provides balance to the portfolio of innovations in

terms of pacing/stage of maturity of initiatives. 1 5 1 5 1 5 1 5 1 5 1 3 1 3


terms of desired risk in the portfolio.1 4 1 4 1 4 1 4 1 4 1 2 1 3


terms of technology domains we’re invested in.1 3 1 3 1 5 1 4 1 4 1 2 1 3

7.6This initiative is providing balance to the portfolio in terms of

expected returns (Growth, IP Stack, Communities). 1 4 1 4 1 4 1 4 1 4 1 3 1 3

7.7

This initiative is providing balance in terms UIF portfolio areas 1.

Products for youth under 25 2. Real‐time information for decision

making 3. Infrastructure to increase access to services and

information.

1 5 1 4 1 5 1 4 1 4 1 4 1 3

7.8At the portfolio level, committing to this initiative continues to

enhance the secondary benefits which accrue to the fund.1 4 1 4 1 4 1 4 1 4 1 3 1 3

Total Value 33 32 36 33 32 24 24


Initiative Aggregate / Impact on Portfolio

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

68.96% 100.0 82.5 100.0 80.0 100.0 90.0 100.0 82.5 100.0 80.0 100.0 60.0 100.0 60.0

Better

Teachers,

Better

Learners

MAPP VACSomlengMpower9NeedsIPAL


Figure 92 (sheet 2 of 2): Portfolio Evaluation tool Section 7: Initiative's Impact on the Portfolio (Adapted by the authors from Paulson et al., 2007)

Saycel


Page: 182

Section 8

VIII

External Start-up Team Capabilities & Pace of Progress: The extent

to which the initiative team is exhibiting the appropriate set of skills

and competencies and is making appropriate progress given the

initiative's stage of development.

Weighting

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

8.1The leader of the team is capable of taking this initiative to the next

level.0 0 0 0 0 0 0

8.2This initiative team is populated with members who exhibit a strong

experimental orientation.0 0 0 0 0 0 0

8.3 The initiative team actively supports the open innovation process. 0 0 0 0 0 0 0

8.4

The initiative team displays the characteristics of effective Teams

(e.g. Task effectiveness, inter-team co-operation, team innovation,

team viability & team member well-being).

0 0 0 0 0 0 0

8.5

This initiative team is populated with members who have previously

struggled with the entire innovation process from discovery/idea

generation through commercialisation/scaling.

0 0 0 0 0 0 0

8.6

This initiative team is making significant progress given the amount

of resources they have and the difficulty of the challenges they are

facing.

0 0 0 0 0 0 0

8.7The initiative team has a clear division of roles and responsibilities

with clear visionary.0 0 0 0 0 0 0

8.8

Team has balance and diversity of expertise, strengths and

characteristics (incl gender) relevant to technology solution, sector,

context, sustainable business.

0 0 0 0 0 0 0

8.9

Commitment/ sustainability of team members to each other & the

company (length of employment with the company; team members

employed full-time).

0 0 0 0 0 0 0

8.10 Team has strong connections with stakeholders. 0 0 0 0 0 0 0

Total Value 0 0 0 0 0 0 0


Initiative Aggregate / Team Capabilities

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

0.00% 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

U-Report

Burkina Faso

MomCon-

nect (South

Africa)

U-REPORT

Fiji

Care for

Children

U-Report

Guatemala

U-Report

Brazil

Advocacy

Project

Figure 93 (Sheet 1 of 2): Portfolio Evaluation tool Section 8: External Start-up Team Capabilities & Pace of Progress (Adapted by the authors from Paulson et al., 2007)


U-Report

App


Page: 183

Section 8

VIII

External Start-up Team Capabilities & Pace of Progress: The extent

to which the initiative team is exhibiting the appropriate set of skills

and competencies and is making appropriate progress given the

initiative's stage of development.

Weighting

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

8.1The leader of the team is capable of taking this initiative to the next

level.0 0 0 0 0 0 0

8.2This initiative team is populated with members who exhibit a strong

experimental orientation.0 0 0 0 0 0 0

8.3 The initiative team actively supports the open innovation process. 0 0 0 0 0 0 0

8.4

The initiative team displays the characteristics of effective Teams

(e.g. Task effectiveness, inter-team co-operation, team innovation,

team viability & team member well-being).

0 0 0 0 0 0 0

8.5

This initiative team is populated with members who have previously

struggled with the entire innovation process from discovery/idea

generation through commercialisation/scaling.

0 0 0 0 0 0 0

8.6

This initiative team is making significant progress given the amount

of resources they have and the difficulty of the challenges they are

facing.

0 0 0 0 0 0 0

8.7The initiative team has a clear division of roles and responsibilities

with clear visionary.0 0 0 0 0 0 0

8.8

Team has balance and diversity of expertise, strengths and

characteristics (incl gender) relevant to technology solution, sector,

context, sustainable business.

0 0 0 0 0 0 0

8.9

Commitment/ sustainability of team members to each other & the

company (length of employment with the company; team members

employed full-time).

0 0 0 0 0 0 0

8.10 Team has strong connections with stakeholders. 0 0 0 0 0 0 0

Total Value 0 0 0 0 0 0 0


Initiative Aggregate / Team Capabilities

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

0.00% 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Better

Teachers,

Better

Learners

IPAL


Somleng MAPP VAC9Needs

Figure 93 (Sheet 2 of 2): Portfolio Evaluation tool Section 8: External Start-up Team Capabilities & Pace of Progress (Adapted by the authors from Paulson et al., 2007)

Saycel Mpower


Page: 184

Section 9

IXUIF Capabilities for this initiative: The degree to which UIF has the

resources, motivation, and capability to bring this initiative to fruition.

Weighting 0.06

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

9.1 Executive support is strong for this initiative. 1 3 1 3 1 3 1 3 1 4 1 4 1 5

9.2We have the resources within UIF to accomplish the next steps

necessary for this initiative.1 4 1 4 1 4 1 4 1 4 1 3 1 5

9.3The apparent path to market and scale for this initiative is one with

which we are comfortable.1 4 1 4 1 4 1 4 1 4 1 3 1 5

9.4Cost of gaining the technical progress that is needed is appropriate

given ultimate promise for this opportunity.1 4 1 4 1 4 1 4 1 4 1 4 1 4

9.5The chances of achieving the next crucial technical milestones are

high.1 4 1 4 1 4 1 4 1 4 1 4 1 4


networks.1 3 1 3 1 3 1 3 1 3 1 3 1 4


networks.1 3 1 3 1 3 1 3 1 4 1 3 1 4

9.8 This initiative will leverage and extend our current competencies. 1 4 1 4 1 4 1 4 1 3 1 3 1 4

9.9The initiative is able to leverage against the UNICEF brand,

contributing to the initiatives success.1 4 1 4 1 4 1 4 1 4 1 4 1 4

Total Value 33 33 33 33 34 31 39


Initiative Aggregate / Firm Capabilities

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

76.67% 100.0 73.3 100.0 73.3 100.0 73.3 100.0 73.3 100.0 75.6 100.0 68.9 100.0 86.7


U-REPORT

Fiji

U-Report

App

U-Report

Brazil

Advocacy

Project

U-Report

Guatemala

Figure 94 (sheet 1 of 2): Portfolio Evaluation tool Section 9: UIF Capabilities for this initiative (Adapted by the authors from Paulson et al., 2007)

Care for

Children

MomCon-

nect (South

Africa)

U-Report

Burkina Faso


Page: 185

Section 9

IXUIF Capabilities for this initiative: The degree to which UIF has the

resources, motivation, and capability to bring this initiative to fruition.

Weighting 0.06

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

9.1 Executive support is strong for this initiative. 1 4 1 4 1 4 1 4 1 4 1 3 1 3

9.2We have the resources within UIF to accomplish the next steps

necessary for this initiative.1 4 1 4 1 4 1 4 1 4 1 4 1 4

9.3The apparent path to market and scale for this initiative is one with

which we are comfortable.1 4 1 4 1 4 1 4 1 4 1 4 1 4

9.4Cost of gaining the technical progress that is needed is appropriate

given ultimate promise for this opportunity.1 4 1 4 1 4 1 4 1 4 1 4 1 4

9.5The chances of achieving the next crucial technical milestones are

high.1 4 1 4 1 4 1 4 1 4 1 4 1 3


networks.1 4 1 3 1 3 1 3 1 3 1 3 1 3


networks.1 5 1 4 1 4 1 4 1 4 1 4 1 4

9.8 This initiative will leverage and extend our current competencies. 1 4 1 4 1 4 1 4 1 4 1 2 1 3

9.9The initiative is able to leverage against the UNICEF brand,

contributing to the initiatives success.1 5 1 4 1 4 1 4 1 4 1 4 1 3

Total Value 38 35 35 35 35 32 31


Initiative Aggregate / Firm Capabilities

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

76.67% 100.0 84.4 100.0 77.8 100.0 77.8 100.0 77.8 100.0 77.8 100.0 71.1 100.0 68.9

Figure 94 (sheet 2 of 2): Portfolio Evaluation tool Section 9: UIF Capabilities for this initiative (Adapted by the authors from Paulson et al., 2007)

9Needs


MpowerIPAL SomlengSaycel MAPP VAC

Better

Teachers,

Better

Learners


Page: 186

Section 10

IX

Financial indicators of value creation:

The degree to which the initiative contributes to the financial value

of the portfolio and UIF.

Weighting 0.06

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level (1-

5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

10.1The initiative provides a cost effective solution, when addressing

the problem / need (e.g. reduces the costs of implementation).1 3 1 3 1 3 1 3 1 4 1 4 1 5

10.2 This initiative is contributing significantly to the IP stack. 1 2 1 2 1 2 1 2 1 3 1 3 1 3

10.3The initiative is attracting external attention which encourages

additional investment in the fund.1 3 1 3 1 3 1 3 1 3 1 4 1 3

10.4The initiative replaces an existing offering/mode of delivery with

the potential for significant cost efficiencies. 1 3 1 3 1 3 1 3 1 3 1 4 1 3

10.5The initiative encourages efficient use of resources (value for

money).1 3 1 3 1 3 1 3 1 4 1 3 1 5

10.6 Strong Potential returns (1. Growth, 2. IP Stack, 3. Communities) 1 3 1 4 1 3 1 4 1 3 1 4

10.7




0 - 0 - 0 - 0 - 0 - 0 - 1 4

10.8The cost estimates are realistic based on local context and

competitive prices/rates.0 - 0 - 0 - 0 - 0 - 1 4 1 5

10.9The UIF investment fits within the company's overall strategy and

financial approach.1 3 1 3 1 3 1 3 1 4 1 4 0

Total Value 20 21 20 21 24 26 32


Initiative Aggregate / Financial indicators

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

68.94% 77.8 57.1 77.8 60.0 77.8 57.1 77.8 60.0 77.8 68.6 87.5 74.3 88.9 80.0

U-Report

Brazil

Advocacy

Project

U-Report

Guatemala


Figure 95 (sheet 1 of 2): Portfolio Evaluation tool Section 10: Financial indicators of value creation: (Adapted by the authors from Paulson et al., 2007)

U-REPORT

Fiji

U-Report

Burkina Faso

MomCon-

nect (South

Africa)

Care for

Children

U-Report

App


Page: 187

Section 10

IX

Financial indicators of value creation:

The degree to which the initiative contributes to the financial value

of the portfolio and UIF.

Weighting 0.06

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

Rele-

vance

(0, 1)

Value

Level

(1-5)

10.1The initiative provides a cost effective solution, when addressing

the problem / need (e.g. reduces the costs of implementation).1 5 1 3 1 3 1 3 1 4 1 4 1 4

10.2 This initiative is contributing significantly to the IP stack. 1 4 1 3 1 4 1 3 1 3 1 2 1 3

10.3The initiative is attracting external attention which encourages

additional investment in the fund.1 4 1 3 1 4 1 3 1 3 1 2 1 3

10.4The initiative replaces an existing offering/mode of delivery with

the potential for significant cost efficiencies. 1 4 1 4 1 4 1 4 1 4 1 4 1 3

10.5The initiative encourages efficient use of resources (value for

money).1 4 1 4 1 4 1 4 1 4 1 4 1 4

10.6 Strong Potential returns (1. Growth, 2. IP Stack, 3. Communities) 1 3 1 3 1 4 1 3 1 3 1 3 1 3

10.7




0 - 0 - 0 - 0 - 0 - 0 - 0 -

10.8The cost estimates are realistic based on local context and

competitive prices/rates.0 - 0 - 0 - 0 - 0 - 0 - 0 -

10.9The UIF investment fits within the company's overall strategy and

financial approach.1 5 1 4 1 4 1 4 1 4 1 3 1 4

Total Value 29 24 27 24 25 22 24


Initiative Aggregate / Financial indicators

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Driver

s

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

% of

Drivers

Import-

ant

% of

Max

Value

68.94% 77.8 82.9 77.8 68.6 77.8 77.1 77.8 68.6 77.8 71.4 77.8 62.9 77.8 68.6

MpowerIPALSaycel


Figure 95 (sheet 2 of 2) Portfolio Evaluation tool: Section 10: Financial indicators of value creation: (Adapted by the authors from Paulson et al., 2007)

9Needs Somleng MAPP VAC

Better

Teachers,

Better

Learners


Page: 188

Analysis supporting Day’s Innovation Risk Matrix is as follows.


9Needs (South Africa) (New)

Initiative improves Early Childhood

Development services in South Africa using

blockchain infrastructure and smart contracts.

9Needs has prototyped a system which

strengthens current registration, contracting,

information and management systems.

This initiative is being launched for the first time

in a new market. Distribution and uptake rates

are unknown with little information on which to

base an estimate.


The technology is new to the company, with

block chain being introduced for the first time.


Care for Children (Cambodia)

Country based initiative involving close cooperation with Government authorities in relation to development of digital based performance reporting of child care institutions in Cambodia. This initiative has the potential to improve accuracy and timely collection of data for decision making purposes, leading to improvement in Government response times.

Initiative is being launched in a small segment of the market and much will depend on the success of this initiative before country level impact can be determined with any degree of certainty. Also, key to the success of this initiative will be the relationships formed with Government authorities. Although this initiative has the support of the Central Government, local authorities will need to be engaged if this initiative is to be scaled across Cambodia. Other considerations: - Significant scope for improvement in services - More re-engineering of the process of data capture - High scalability potential - Will expect to see more engagement with stakeholders, particularly Government Ministry. Intended Market Risk is considered MODERATE.

Application of this technology is new to this particular industry, and much will depend on uptake by local staff within the Ministry. Having said this, early indications are that staff at the Ministry are accepting of the changes, and there is an air of optimism around improved functionality and expected efficiency gains. Other considerations: - Will require training and acceptance by stakeholders Product/Technology risk is considered HIGH.

Somleng

Chatterbox

(Cambodia)

(New)

An efficient and low cost Interactive Voice

Response (IVR) and SMS platform, to be

integrated into the RapidPro platform to extend

its reach to communities with low literacy

levels.

Will complement the existing RapidPro

technology operating in the same market.


The IVR technology is relatively new to the

market, and acceptance/uptake has yet to be

proven.



Page: 189


U-Report App Syria

Priority areas include an increase in immunisation rates, enhancing school enrolment among internally displaced children and increasing the number of children accessing child-friendly spaces. U-Report adopts text message technology and serves as a community based feedback system, amplifying the voices of many. It also serves as a form of engagement with the community, capable of polling views across a range of important issues, as well as acting as an early warning device. U-Report provides a new model for engaging with communities, effectively empowering young people.

This initiative is being launched for the first time in a new market. Distribution and uptake rates are unknown with little information on which to base an estimate. Given the current political and economic environment, adoption of U-Report at country level may vary in success. Existing stakeholder relationships do not necessarily mitigate risk, and potential threats at country level are known with less certainty. Intended Market Risk is considered HIGH.

The technology will require the use of smart phones, and as such, other forms of social media may compete with this technology. Background: The technology used in this project is an open source RapidPro. ‘At the core of RapidPro lies our exclusive Flow engine. With Flows, anybody can create complex SMS and voice applications without the need of a programmer or expensive consulting company. Using a basic Android phone, you can launch your application instantly in any country while avoiding expensive set-up costs or external technical support". Having said this, the necessary services and functions are new by comparison with what was previously available. Expectations of quality/standards will also be higher. Product/Technology risk is considered HIGH.

U-Report - Brazil Advocacy Group

Collection of real time data from young people and provide them with critical information. Overall U-Report will align national with local and regional issues and campaigns carried out over 800 youth networks spread through the most vulnerable communities in far-away communities and urban centres. Issues include effects of the Zika virus on communities and to disseminate information on prevention practices.

This initiative is being launched for the first time in this market. Unfamiliarity with market environment and competitive pressure likely to be exerted from regulatory authorities and Telco’s in relation to communication products may adversely impact the effectiveness with which this product is introduced. Intended Market Risk is considered HIGH.

Although the technology is still relatively new to the company, U-Report has been introduced in other markets, and the company’s familiarity with this technology is improving. Having said this, SMS technology is proving to be unpopular (due to current call charges imposed by Telco’s); however, Facebook is growing in popularity. The preferred medium has yet to be determined. Product/Technology risk is considered HIGH.


Page: 190


SayCel

Nicaragua

(New)

Improve low-cost open source GSM

communications suite used to run cellular

networks in rural regions by installing low-cost

technology & training local governments and to

maintain their own networks.

Initiative extends mobile cellular coverage to less

densely populated rural regions. Relationship

with stakeholders key. New initiative in the

market.


Product technology is new to the organization,

suggesting greater degree of reliance may be

placed on vendor partners.


Innovations

for Poverty

Alleviation Lab

(IPAL)

Pakistan

(New)

IPAL have developed an interactive voice

response (IVR) system that serves as a maternal

care hotline. Aims to provide fathers with access

to medical advice and guidance in accessible

formats.

Initiative aligns well within the real-time

information portfolio and UNICEF Innovation

Funds intended market


Product is relatively new, and as a form of

engagement, some overlap with existing

products is evident.


U-Report - Burkina Faso

This initiative addresses issues such as safe pregnancy and good maternal and new born health.

This initiative is being launched for the first time in this market. Intended Market Risk is considered HIGH.

U-Report has been introduced in other markets as it is being scaled. Product/Technology risk is considered MODERATE.

U-Report Fiji

Community Feedback System built on U-Report (RapidPro platform) to cover all aspects of cyclone relief, recovery and transition to development, with scope for adaptation as a community engagement and polling and/or early warning system.



Learning labs Uganda

An initiative supported by UNICEF Uganda, U-Portal is a digital content management tool developed to support schools and youth centres across Uganda. U-Portal is currently undergoing a re-design/upgrade improving the scalability of the tool across UNICEF country offices, to serve as the go-to tool for digital content management.

The project targets the youth population of rural Uganda utilising human centred design and attempts to take into consideration how young people in this region are likely to use computers. The distribution channels which relate to this initiative remain much the same, and the initiative itself does not compete with existing initiatives. This initiative appears to be growing in popularity with target audiences. Intended Market Risk is considered LOW.

This initiative represents an extension/upgrade of an existing product which sits well within UIF's levels of competence/capability. The initial product design was kept simple and cost effective, with the design process taking account of user’s interaction with the technology. From this perspective, the risk of failure associated with product/technology is considered low. Product/Technology risk is considered LOW.


Page: 191


Better teachers Maldives

E-learning platform to deliver large volumes of content to educators across the country. This has the potential to enhance the quality of teaching and possibly better learning outcomes for students in the Maldives. Content delivery across a single platform is one initiative which will address the disparity in education levels between the cities and the vast island population. It is expected that this initiative will serve 4,000 teachers across 200 islands through 220 schools.

This initiative is aligned with UNICEF's mission and the education strategic outcome and has support from the Ministry of Education The intended market aligns with well with UNICEF Innovation Fund portfolio. Intended Market Risk is considered MODERATE.

This initiative sits well with UNICEFs capabilities. Where the outcome is to improve the quality of education provided, the e-learning platform is one determinant of quality, and as such, a low to moderate level of risk does exist. Product/Technology risk is considered MODERATE.

U-Report - Guatemala

This initiative addresses issues around child marriage, adolescent pregnancy and sexual violence against girls.



Mapp 4 VAC Reporting Maldives

Increasing accessibility to information, enabling reporting of cases of violence against children from remote islands of Maldives. A mobile application that is linked to the call centre and the Maldives child protection database. The information from both calls and SMS would be centrally captured, analysed and used for timely response and action. This, enabling the social workers to respond to cases in a timely manner.

Given the vast reach required to ensure effectiveness, familiarity with intended market is considered low. Intended Market Risk is considered HIGH.

The organisation is familiar with the current product/technology. Product/Technology risk is considered LOW.

mPower

(Bangladesh)

(New)

Open Smart Register Platform (OSRP), an integrated digital registry platform that seeks to improve data collection and efficiency for frontline workers providing reproductive, maternal, new born and child services.

New market initiative. Initiative aligns well within the real-time information portfolio and UIF's intended market Intended Market Risk is considered MODERATE.

Product/Technology sits well within the organisation's domain. Product/Tech risk is considered MODERATE.

MomConnect South Africa

SMS maternal reminder program for pregnant women. Now building the technology as part of the RapidPro.

Currently in development for both the South African and Ugandan market place. Intended Market Risk is considered MODERATE.

The organisation is familiar with the current product/technology. Product/Technology risk is considered LOW.

Figure 96: UIF Portfolio (Current) including UIF's first portfolio of investments in private sector companies (UNICEF ,2016J; UNICEF , 2016L)


Page: 192

Appendix 8: Interviews Findings

Interview Observations Relating to Objective 1

No. Key Valuation Themes

Significant Quotes from Interviewees Related to Objective 1

1 Typical process of Frameworks of Evaluation that Currently Applied at the Organization / Project Level

‘We don’t have a common evaluation framework in the sense that other than very

basic things, you have to have an evaluation plan across, a costed evaluation plan,

things like that’.

‘there’s a huge degree of flexibility to the degree that there is standardization’

‘The common evaluation frameworks would be sub-organisational and even

though they might not be very strong’.

‘We’re not much into methodological development, we tend to find and adapt and

refine things on the market that were familiar with’.

‘For SROI & ROI, we often don’t capture things in that way, it might be more

qualitative and descriptive’.

‘If we’ve done these many different things, we think we have logical model or

impacts model that says that this has been accomplished and we don’t feel the

need to always go out and try to validate things through national monitoring or

through certain kinds of evaluations’.

‘We set up a long-term agreement with 8 or 9 powerful global institutions, and so

UNICEF Offices that want to invest at a high level of rigor for impact valuations can

access these firms for doing visibility studies’.

‘we’re very appreciative of the need to respect the uncertainties that the offices

were facing and the partners were facing’

‘We tend to concentrate on evaluation by theme and understand what the learning

from that theme was or is’

‘highly committed as much as possible to having shared impact measurements’

2 Technological Changes Impact on Evaluation

‘one of the big problems we keep finding with new technology implementation is

its sustainability’

‘I worry about the standardisation given the variety of the kind of innovations that

are going on and the extent to which the items generalise across different kinds of

innovation.’

‘We are very much able to deal with emergent and contingent systems change

driven’.

3 Evaluation Team

‘We consider and assess the evaluation team dynamics while setting up the

evaluation team’.

‘The organization as a whole is very accepting of formative evaluations to see if the

programs and understandings are developing as hoped for and as predicted and if

not, then what can be learned about it’.

4 Developmental Evaluation

‘We have a colleague that got quite enchanted by that [DE], saw an opportunity to

do that. She applied it within peace building. So we put evaluators in 2 countries’.

‘By UNICEF putting monitoring evaluation officers in the country offices they have

often the very same outcomes and desires or goals as a Developmental Evaluator

in the sense that he has a facilitator role throughout the process and it’s an

iterative process and we have a lot of experience within that field of analysis’

Table 40: Interview Observations Relating to Objective 1 – Individual Initiative


Page: 193




1 Real Returns as Perceived

‘You can say so many doctors we’ve trained, so many kids vaccinated, that's how we measure, and you can say the infant mortality has gone down by this much. Those are things that we measure’, and ‘there is no return on investment on the value of life that's for sure’. ‘I would classify ourselves as relatively weak on using the kind of evaluative logic that the market uses that the private sector uses. We don’t traditionally concern ourselves very much with efficiency and ROI’.

2 Indicators are a fallacy/Real Returns are Reflected in the Portfolios Diversity.

‘that’s what an innovation fund ought to be doing’ also noting ‘…folks do too much interpretation too quickly without really seeing patterns and the other is that they don’t do it often enough to see what’s emergent.’ [the evaluation process is a] ‘reflective practice to figure out what’s going on…it becomes important to set up a process for interpretation and action that is manageable and is kept up with’. ‘the diversity approach is not to say that they all get evaluated by the same criteria, but in fact, what kind of diversity do we want, and are we getting that kind of diversity, that’s what an innovation fund ought to be doing.’ ‘What we want with most interventions in complex adaptive systems is they’re adaptive, but [the problem is] you keep doing the same thing over and over again.’ ‘The appropriateness or one way to handle this is to create a menu of these kinds of items, and ask them on a project basis to pick the items that are relevant for their particular situation that they believe are appropriate to be.’

3 Evidence Based Measures

‘There isn't really much evidence around the real return on investment…how to prove whether this kind of VC approach really will influence or transform or accelerate how development is done more efficiently faster bigger better and so on and that is at the core of the challenge of…justifying innovation and models such as the VC approach’. ‘… have to have an enormous hunger for the evidence of impact and things like that in a very rapid timeframe’.

4 Value Proposition

‘…real value accrues to UNICEF/Ecosystem via learning, and being able to move ideas incrementally forward [and that] Its only once you start scaling that you’re going to see real value [accrue towards the] end beneficiary’.

5 Substance Over Form (Qualitative Data)

‘Understanding the data, it’s not just what, what, what, it’s also the why, for example, if only 20% of women are using U-Report, we need to know the why … we need to dig into the qualitative data to understand’. ‘One of the things I feel with innovation is … we’ve got this backend system with access to quantitative data all the time and its real-time data, but actually, you need to dig into that data and a lot of that is qualitative right.’ ‘real time refers to, especially in the software industry, to data that’s entered at the point of consumption, and so its immediate’. ‘…timely data that would involve follow up with users may not be real time so that to the extent that one is gathering additional data from users and doing case studies of applications the feedback is timely and so it’s both real time and timely not all real-time data is evaluative or developmental’. ‘Real time data tells you what the pattern is, it doesn’t tell you what the pattern means and what to do about it, and that’s one of the confusions I find with real-time data’.

6 Themes ‘focus on evaluation by theme and understand what the learning from that theme was or is e.g. WASH program, or social protection’ and ‘… that (high level) metrics are comparable at sector level i.e. agricultural.’


Page: 194



7 Shared Impact/Platform indicators needed Business Intelligence

‘…in general we are highly committed as much as possible to having shared impact measurements’. ‘…so we found that one of the critical elements of doing this type of work is you have to have unified shared business intelligence’. ‘The language UNICEF are beginning to use has come from the private sector i.e. business intelligence / business case, especially if you want to attract funding and bring partners on board, need to speak the same language as the private sector.’

8 Standardised metrics – Pitfalls

‘I worry about the standardisation given the variety of the kind of innovations that are going on and the extent to which the items generalise across different kinds of innovation.’

9 Aggregation - Problematic

‘at the top level not all innovations are going to have the same metric’. ‘The discussion needs to be around [the meaningful nature of these top-level indicators, what will they really tell us?’ ‘In terms of managing initiatives at an aggregate level, this currently does not happen [at country level]. Initiatives are managed at functional level i.e. child protection, education section, etc.’ ‘we don’t aggregate our portfolio’

10 UNICEF’s MEL Framework

‘at the very top level, metrics are standardised and focus on reach i.e. number of users, etc.’ whilst at the other extreme, at country level, ‘indicators are innovation specific’.

11 Resistance ‘… impression I get with a lot of these innovations is let’s just do it.’

12 Pipeline ‘One would naturally lead to the other, right, the kind of like pilots going into scale. All the M&E you would be doing at the pilot should be feeding into the scale stuff because you need proof points to know it works in order to scale it, you should be having similar indicators, right.’

Table 41: Interview Observations relating to Portfolio Evaluation


Page: 195


Significant Insights into Secondary Benefits List

Significant quotes from key informants related to each of the eight potential secondary benefits are

summarised below.

No. Secondary Benefit

Significant Quotes from Interviewees related to the Secondary Benefit List

1 Organisational Learning (often through Failure) (90%)

‘I would move organisational learning and inspiring staff in particular to a primary benefit...’ ‘Definitely number 1, there was a lot of learning... we did fail several, several times, we had to redesign... we had to find our way working creatively with the government and our partners... and all of that was a huge benefit because the national scale up was able to be done... learning from those lessons they didn’t have to go through the same thing again... so that definitely was a plus’ ‘we can more thoroughly speak about this once we have gone through this first process with the Innovation Fund... more lessons... will come out of that...’ ‘organisational learning, absolutely captured because we are going to be failing a lot....’ ‘if you point out failure and just failure and there are a lot of evaluators that have that tendency then the evaluation goes nowhere... so one thing that is quite interesting is to show the successes up front’ ‘...a good distinction recently, that we’ve often talked about the importance in innovation of failing fast, but... a new iteration of that is actually failing smart and that captures the learning element which I think has been missing from the fail fast approach which was it’s not working, just drop it quickly and find another one and try that. Fail smart is much more about really understanding what it was that didn’t work, what influence your own organisation may have played in that process in particular. Because organisation's can make it pretty systematically difficult to scale innovation effectively just by virtue of the way they are set up, and the culture and the risk appetite. When innovation goes wrong by Christian Seelos, innovation pathologies... important for you to consider from a UNICEF perspectives and how it’s structured within the UN network’

2 Making the Organisation Ready for Change (60%)

‘making the organisation ready for change is a big one, especially with these emerging technologies we see... in our near future it’s important to be aware of them’ ‘...the second one, if this doesn’t prepare us for talking financially about things as an organisation I don’t know what will....’

3 Inspiring Staff and Attracting New Types of Talent (70%)

‘..Using the digital system is really bringing a new energy and enthusiasm to using the new tool.. which will... simplify their lives.. they will be able to carry out other tasks in the time that has been spared... the team... is extremely enthusiastic... as a learning process but also to enable them to better do their job... in a more efficient way... and being able to address other urgent problems that children are facing.... by simplifying.... and making a much more effective system....’ ‘...it definitely inspires staff, creates that motivating environment and yes we have attracted new types of talent’ ‘....third one, absolutely we’re seeing that already because we’re seeing people speaking with the VC language or capital mind-set language which is very interesting...’

Table 42: Interview Results: Significant Quotes from Key Informants (Part 1 of 2)


Page: 196


Significant Quotes from Interviewees related to the Secondary Benefit List (Continued)

4 Creating a Sustainable Network of Long Term Relationships even after completion (80%)

‘I think creating networks of relationships especially within the tech industry regulatory bodies within UNICEF is something I’ve been involved with governments... NGO’s... other UN Agencies... that are all trying to tackle the same problem...’ ‘...the fourth one, we’ve had so much positive outreach from people running other funds, who want to work with us because we’re not a threat and we’re investing in markets they’re interested in... we haven’t figured out quite how to do it yet but we’ve had everyone from family foundations, to.. venture funds which have said we’ll share pipeline to the IDIA group. Groups in all of those and people in all of those have expressed interest in collaboration... We haven’t got there yet because we need to get our house in order but there is definitely something there...

5 External Mimicry (50%)

‘...we just had another group in the UN publish... that they were so proud to be launching [a] fund in the UN that was mixed public and private funding, so they’re already doing that and copying... our stuff which is great, that’s UNHCR by the way....’

6 Creating New Business Models (70%)

‘New business models, absolutely, we actually had to come up with the open source cases and open source business models so that was a legit thing that we’re doing....’

7 Building Capabilities (70%)

‘what it all means to have different systems, interoperable systems and SMS technology, I’ve not just seen this with our UNICEF staff but I’ve also seen it with our healthcare providers for instance, our government counterparts, who want to understand.. better.. it’s helping them think differently and more effectively... this has been a major change, me myself I’ve learned so much by doing this that I wouldn’t have known... that’s definitely a... secondary benefit’

8 Driving Organisation Goodwill (i.e. new funders/allies) (80%)

‘Public positioning of UNICEF, a lot of what Ventures and Futures does, are new things...people don’t expect shiny new things out of UNICEF but it is the Futures and Ventures group who is doing that... there is a tremendous amount of publicity for the organisation, of repositioning the organisation to be highly relevant, highly innovative... future focused... that I would capture under bullet number 8’ (driving organisational goodwill) ‘It positions... UNICEF is seen in the scenario of stakeholders as an innovative partner... dialoguing with new trends, open to adapt... and not insisting on the old type of communication... it strengthens our mission and our goal’ ‘Eight yes, in that we had people like the.... who would never have funded UNICEF traditionally’ ‘In terms of secondary benefits... we’ve attracted more donors and different partners that we otherwise would not have had, it’s also lead to many, many ideas...’ ‘Other partners that we can bring to the table that are not currently partners...’

Table 43: Interview Results: Significant Quotes from Key Informants (Part 2 of 2)


Page: 197

Additional Secondary Benefits Not Listed

Table 44 gives the secondary benefits noted by informants that were not listed in the interview question.

Whilst the benefits are important most are variants of the listed benefits and have been captured above.


Additional Secondary Benefits noted by Interviewees Not Listed in the Question

9 The Potential to Guide Legislation in Emerging Tech. (10%)

‘And something that’s not on this list is the potential to guide legislation in countries... where we have good relationships with governments... being privy to and part of the regulatory process.... UNICEF can be the middle man as a sort between... companies that want to use their technologies and then governments that see these requests but maybe aren’t quite sure with what to do with them... UNICEF Innovation can be part of that relationship’.

10 Value of Open Source IP and Communities (10%)

‘The value created by these investments, the open source intellectual property... and I think that... there’s a flow of value that goes across everyone... and the other is the value of the community that we create... I think that this work can have a balancing effect on an out of kilter world and I think that can happen because your connecting a group in Pakistan working on this to a group in India working on that to a group in Brazil working on this suddenly people are working together on common problems and I think that that’s really valuable...’

11 Sectoral Programmatic Maturation (10%)

‘seeing... the different sectors start to... modify their common practices... is an important one over time’

12 Characteristics, Skill Sets and Behaviour of the Modern Development Professional (10%)

‘...we have seen a large shift... in the characteristics, skill sets and behavioural element that a more modern and future looking development professional seems to need and to have... that are different. So as an example we’ve seen less of a focus in new programming and looking forward on technical skill sets and much more on, can this partner integrate programmes well, are they a good partner manager... the general... DNA of what someone who is going to be successful in this in the next 10 years as development evolves we are seeing change fairly significantly.’

13 Evolution of Risk Management through Making Small Bets over Long Traditional Programmes (10%)

‘the actual risk and monetary calculation is something that we are continuing to see evolve so that the concept of making small bets and paying for results versus doing a large five year programme... and our traditional programming is something that we continue to be looking at...’ ‘we believe that programmes like [venture fund] are about taking smart and informed risk... this is why we like to double down and things like business intelligence, you’ll note that I don’t call it data because data does not mean anything to anyone. I specifically call it business intelligence because it’s information that we are using, in order to help us mitigate risk, make better decisions etc... we don’t fund a large percentage of what we get so we spend an enormous amount of time in that due diligence and in that risk mitigation to try to figure out. Is this a viable innovation? Is this a viable team? Does it have the combination of factors that we feel are worth that initial investment.... programmes like [venture fund] are the most conscious and intentional about understanding, mitigating and making decisions around a risk... compared to any other programming that [we do]..’

Table 44: Additional Secondary Benefits noted by Interviewees not Listed in the Interview Question (Interviews)


Page: 198

Appendix 9: Additional Information on Organisational Learning and Lean Start-up

Lean Start-up and UNICEF Innovation Fund

Ries (2011) posits that the way forward is ‘to see every start-up... as a grand experiment’ and ask pertinent

questions like ‘should this product be built’ and ‘can we build sustainable business around this set of products

and services?’ (Ries, 2011) which lends itself to the Ventures Horizon 2 that seeks to ask those questions at an

early stage before they pass to the Scale Horizon 3. The Lean Start-up methodology experiments following

scientific method beginning with a clear hypothesis and then tests those predictions empirically with the goal

of discovering how to build a sustainable business around the start-ups vision (Ries, 2011). The model fits the

corporate venturing approach of UNICEF Innovation Fund allowing hypothesis of the solutions seeded to be

tested rigorously, immediately and thoroughly based on real time data feedback.

Starting with a small seed or catalytic investment prevents a tremendous amount of waste without

compromising the overall vision of the start-up or that of UNICEF (Ries, 2011). This early stage evidence is

indeed one of the core objectives of UNICEF Innovation Fund (See Figure 29 - Corporate Venturing Model) and

the minimal viable products (MVP) of the Lean Start-up model are consistent with the early stage proof of

concept solutions targeted by the Fund. The minimum viable products (MVP) allow the start-ups and UNICEF

Ventures to ‘start the process of learning as quickly as possible’. Adaptations of the MVPs over 12 to 24

months under the guidance of with the support of UNICEF Innovation Fund technical and business

management assistance is manageable within the tight constraints of $100,000 seed funding, something that

would not be possible with large scale traditional product development.

‘One of the most dangerous outcomes for a start-up is to bumble along in the land of the living dead’

maintaining a false optimism even when it is clear that the initiative requires major changes if it to succeed,

‘the myth of perseverance’ (Ries, 2011). UNICEF Innovation Fund gives the organisation a ‘vehicle to try new

things’ but also the option of concluding a ‘hard no’ at the end of the Funds support period (UNICEF

Interviews, 2016). In order for UNICEF Ventures to systematically support start-ups and make informed

decisions at the end of that support period a systematic methodology is required.


Page: 199

Innovation Accounting and UNICEF Innovation Fund

Innovation accounting has the potential to adequately support the pivot or persevere question and indeed the

scale question. Such a process will only be as effective as the metrics which are designed for tracking.

Currently the UNICEF Innovation Fund dashboard tracks gross cumulative metrics such as number of users and

number of commits but the authors posit that now that the first round of start-ups have been seeded

actionable metrics (Ries, 2011) should be developed with the start-up teams. A short review of possible types

of metrics follows to establish suitable approaches going forward.

Instead of just looking at cumulative totals or gross numbers such as total revenue and total number of users,

cohort analysis which looks at the performance of each group of customers or cohort that comes into contact

with the product independently (Ries, 2011). The tracking of conversion rates showing the percentage of new

users in a month who went on to take further action such as downloading, usage and purchase is done to

uncover the true performance of the start-up initiative over time. These funnel metrics, developed from the

traditional sales funnel track the progression or sequences of customer behaviour known as ‘flows’, allow a

deeper quantitative understanding of start-up performance and have greater predictive power than simply

relying on gross metrics alone (Ries, 2011).

Cohort analysis rules out attributing the reason for failure on any one sample of customers because ‘each

cohort represents an independent report card’ (Ries, 2011). This richer type of metric continuously updated by

the real time data requirements of UNICEF Innovation Fund provide deeper insights and facilitate more

accurate questions about why initiatives are not adding value or are not illustrating customer behaviour that

formed part of the value creation and growth hypotheses’. The data would allow UNICEF Innovation Fund and

the start-ups seeded make more informed and rapid conclusions about whether or not the right solution is

being built. If you are building the wrong solution no amount of optimization will produce the required results

without a fundamental strategic pivot aimed at achieving a sustainable solution. Innovation accounting

provides a framework that helps ensure that the requirement to pivot is made more clear and avoids relying

solely on gross metrics which Ries (2011) deems ‘vanity metrics’ and ‘success theatre’.

Funnel metrics utilising cohort analysis where possible should to be determined for each initiative. For

example funnel metrics for a UReport initiative (SMS youth advocacy tool) might be number of users, % logged

on once a week, % logged on once a day, % engaged once in surveys, % actively advocating by initiating

campaigns and % of campaigns leading to change with the latter two metrics being the most important

indicators of success.


Page: 200

Funnel Metrics using Cumulative Gross Metrics (Vanity Metrics / Success Theatre)

Funnel Metrics using Cohort Analysis (Independent Report Cards based on Random Cohorts)

Figure 97: Cumulative Gross Metrics vs. Actionable Metrics using Cohort Analysis (Adapted from Ries, 2011)

Adding New Customers (Illusion of forward motion)

Difficult to draw clear cause and effect inferences making

strategic decisions and prioritization extremely

challenging

Adding New Customers but

not Improving Yield (Little real progress)

3 A’s of Metrics (1) Actionable: must

demonstrate clear cause and effect

(2) Accessible: Simple, tangible cohort based reports breaking down

complexity to behaviours that matter & shared

widely. (3) Auditable: credible, tested in real world and

spot checked

Easier to draw clear cause and effect

inferences which aids strategic decision making

and prioritization


Page: 201

Cohort based metrics combined with split testing (A/B testing) can further strengthen the cause and effect

relationship between changes to the MVP and results observed in real time data. One version maintaining the

its previous form being tested at the same time as a new iteration with changes or new features helps

establish if those changes add value or change customer behaviour. Split testing can form part of an extremely

effective form of validated learning and is entirely possible for UNICEF Innovation Fund start-up initiatives

giving the real time data requirement. The discipline can ensure that only validated changes will be kept as the

team continues to iterate. Ries (2011) argues that failure is a prerequisite to learning (Ries, 2011) and if the

foregoing methods can be applied to the UNICEF Innovation Fund the organisation and start-ups will capture

and benefit from that learning.


Page: 202

Appendix 10: Participant Information Sheet

Venturing, Value creation & the UNICEF Innovation Fund Participant Information Sheet

You are being invited to take part in a research study as part of a student project – conducted by students working towards an award of the Masters in Business Administration (MBA) post graduate degree with the University of Manchester (UK). It is hopeful that this research study will demonstrate in clear terms that innovation does generate significant value in a non-profit setting. Before you decide, it is important for you to understand why the research is being done and what it will involve. Please take time to read the following information carefully and discuss it with others if you wish. Please ask if there is anything that is not clear or if you would like more information. Take time to decide whether or not you wish to take part. Thank you for taking the time to read this. Who will conduct the research?

Name Student Number Email Contact

Brian Meagher 9589074 [email protected]

Declan McKeogh 9583776 [email protected]

Haitham Afifi 9595933 [email protected]

John Santiago-Miller 9577704 [email protected]

Ajay Melwani 9577194 [email protected]

All students are currently undertaking their final year of study in the Masters in Business Administration program with Manchester Business School. What is the purpose of the research? The purpose of the study is to demonstrate the value attributable to innovation in terms of being able to deliver against the organisation's mission (in a non-profit setting), with the value of innovation measured both in isolation and as part of a portfolio of innovation initiatives. Why have I been chosen? You were chosen as one of number of candidates by the Head of the Innovation Fund as being someone within the management team who could provide valuable insight into the management and operation of innovation initiatives managed within the fund. What would I be asked to do if I took part? You will be asked to take part in one or more semi-structured interviews with 2 researchers from the team. The interviews will take place either in person or by some on-line channel. Interviews will be scheduled well in advance, and at a time convenient to you. The interview will take the form of a discussion with specific questions relating to the innovation process, and the management of that process. We want you to know that you have the right to refuse participation, however, if you do choose to participate, you also have the right to withdraw your participation at any stage of the interview or for any part of the interview. Any information / data shared with the researchers will be treated on a confidential basis (if requested). If you wish to protect your anonymity, we will do so. What happens to the data collected? The data collected from the interview(s) will be analysed with other information also gathered during the course of this research project. The aim of the project is to demonstrate value and how this can best be demonstrated. The data collected will therefore be used to triangulate our findings against other sources of information, and findings will be reached in a generalised manner. The final report will endeavour to report generalised findings and will protect the anonymity of participants.

mailto:[email protected]




Page: 203

Please note that this research project does not seek to obtain information that is confidential or personal in nature. Any such information acquired during the course of this project will be handled with care, ensuring anonymity, confidentiality and respecting the privacy of individuals. How is confidentiality maintained? All data of a confidential, sensitive or personal nature will be stored on the university network. All files will be encrypted. Access to this data will be restricted to the project team for the duration of the research project, and the project supervisor (appointed by the university) for a period of up to 5 years. If interviews are to be audio/video recorded (subject to permission received from participants), these recordings will be maintained and archived by the university for a period of up to 5 years. Any similar records/information maintained by the research team, will be destroyed at the conclusion of this research project. If as a result of the interviews, the researchers believe that there is any risk of harm to the participant, either physically, or by way of embarrassment, or which has the potential to affect the participant’s career prospects, the participant will be advised. The participant’s anonymity will be protected, and any personal information inadvertently shared will be respected. What happens if I do not want to take part or if I change my mind? It is up to you to decide whether or not to take part. If you do decide to take part you will be given this information sheet to keep and be asked to sign a consent form. If you decide to take part you are still free to withdraw up to a specified point (please stipulate at what point this is no longer possible e.g. time of publication) without giving a reason and without detriment to yourself. Will I be paid for participating in the research? There are no financial rewards or rewards of any kind attached to your participation in this research project. This project is unfunded, and any cost incurred is borne by the student researchers. What is the duration of the research? The duration of the research project is 4 months. The duration of interviews will be up to a maximum of 3 x 1hour interviews. Any telephone conversations will be will be brief. All interviews and telephone conversations will be scheduled in advance. Where will the research be conducted? All student researchers taking part in this research are global MBA students situated at various locations globally. Interviews will more than likely be conducted in your place of work or via the use of some multi-media platform. Will the outcomes of the research be published? The outcomes of the research will not be published for public consumption without express permission of the project sponsor. The project sponsor may wish to make the findings of the research available on their own website or the organisation's intranet. Who has reviewed the research project? The research is supervised by a member of staff from the University of Manchester and as such has received endorsement. What if something goes wrong? In the event that something does go wrong, participants can contact the student researchers by way of email. Please refer to email contacts for student researchers above. Please also ensure that when contacting the researchers, all members are copied.

Participants are also able to contact the University of Manchester. For contact details for the university please refer to: http://www.manchester.ac.uk/connect/contact-us/

http://www.manchester.ac.uk/connect/contact-us/


Page: 204

What if I want to make a complaint? Minor complaints If you have a minor complaint then you need to contact the researcher(s) in the first instance. Formal Complaints If you wish to make a formal complaint or if you are not satisfied with the response you have gained from the researchers in the first instance then please contact the Research Governance and Integrity Manager, Research Office, Christie Building, University of Manchester, Oxford Road, Manchester, M13 9PL, by emailing: [email protected] or by telephoning 0161 275 2674 or 275 2046. What Do I Do Now? If you have any queries about the study or if you are interested in taking part then please contact the researcher(s). Please refer to the contact details for student researchers above.



Page: 205

Value creation through the application of the Innovation Venture Fund at UNICEF

CONSENT FORM

Please note that example consent points 3-6 will most specifically relate to interview or focus group studies.

If you are happy to participate please complete and sign the consent form below.

Please initial box

I agree to take part in the above project

Name of participant

Date Signature

Name of researcher

Date Signature

1. I confirm that I have read the attached information sheet on the above project and have had the opportunity to consider the information and ask questions and had these answered satisfactorily.

2. I understand that my participation in the study is voluntary and that I am free to withdraw at any time without giving a reason and without detriment to my treatment/service/self.

1. I understand that my data will remain confidential. Any comments of a private nature which is revealed during the course of the research will not be disclosed to any party. Confidential data will be subject to a confidentiality agreement where one is signed.

4. I understand that the interviews may be audio-recorded.

5. I agree to the use of anonymous quotes.


Page: 206

Sample Interview Guide:

Global MBA Project


Name Student Number Email Contact

Brian Meagher 9589074 [email protected]

Declan McKeogh 9583776 [email protected]

Haitham Afifi 9595933 [email protected]

John Santiago-Miller 9577704 [email protected]

Ajay Melwani 9577194 [email protected]

Semi-structured Interview Guide

UNICEF Office of Evaluation: <enter participant name>

Profile: <insert interviewee profile>

Date: <enter date>

Background Information

2. What is your position in the organisation? Can you briefly describe your role and responsibilities?

3. We understand the focus of UNICEF Scale is to identify and scale up proven innovative solutions.

a. Can you help us make the distinction between UNICEF Ventures and UNICEF Scale?

i. How does the Office of Innovation decide what initiatives should be funded by

UNICEF Innovation Fund (UIF) or the UNICEF Global Innovation Centre (UGIC) or

the International Development Innovation Alliance (IDIA)?

ii. Does an initiative need to have a certain number of users, or be active in a number

of countries to be considered now part of UNICEF Scales scope?

b. Can you explain the interaction/cross over between UIF and UNICEF Scale?





Page: 207

Core Questions

General Research Question: Ventures Approach

4. Our understanding is that the UNICEF Office of Innovation (UOI) is adopting a ventures strategy

through UNICEF Innovation Fund (UIF) which involves making multiple small investments, or small

bets, in initiatives after proof of concept stage. The majority of initiatives are expected to fail but the

application of venture capital methods typically results in a very small percentage of highly successful

investments making all the small failures worthwhile.

a. Can you please give your views on our interpretation of UIFs strategy?

b. Would you like to add anything further?

5. We consider that UNICEF Innovation Fund can create value for multiple stakeholders including:

(i) UNICEF's beneficiaries (children and mothers);

(ii) UNICEF as an organisation and

(iii) The wider ecosystem of partners and collaborators within which UNICEF works.

a. Can you please elaborate on value created for those 3 groups of stakeholders through the

UNICEF Innovation Fund (UIF)?

Objective 1: Individual Initiatives

6. Innovative initiatives such as those seeded by the UNICEF Innovation Fund are inherently uncertain,

ambiguous and subject to change with outcomes that are often emergent and systems change driven.

As part of our study we are investigating suitable frameworks/approaches to evaluate UNICEF

Innovation Fund initiatives and the fund as a portfolio. Can you please confirm the following citing any

reports or examples that we could explore further:

a. What scalability criteria or implications are considered, when evaluating an initiative prior

to scale up (apart from the Principals for Digital Development)? How does this help in your

decision making process?

b. Does UNICEF Scale have its own M&E approach? Is it prescriptive with RCT’s or some other

approach? Are the existing methods serving your purposes presently?

c. Do you think evaluations are needed at all for such highly innovative interventions with

benefits often taking more than 5 years to accrue?


Page: 208

d. We have identified Developmental Evaluation (DE) as a suitable approach to evaluate

innovation and adaptation. DE informs and supports innovative and adaptive development in

complex dynamic environments through processes of asking evaluative questions, applying

evaluation logic, and gathering and reporting evaluative data to support development with

timely feedback. The purpose is not to evaluate the initiatives and portfolio prospectively or

retrospectively but rather in real-time.

i. Do you believe this evaluation approach could continue to evaluate initiatives

which move onto UNICEF Scale?

ii. Are there any resource constraints to applying such an approach?

e. Can you describe UNICEF Office of Innovations application of the following:

i. Real-time data for adaptive management?

ii. Iterative design thinking?

iii. Rapid prototyping?

Objective 2: Portfolio

7. Perhaps the distinction between UNICEF Ventures and UNICEF Scale can also be defined through

assessments of uncertainty and risk? The below diagram illustrates such an approach plotting

initiatives to be funded by UNICEF Innovation Fund in the zone of highest technical/utility and

market/organisational uncertainty.

a. Is UIF the planned vehicle for venturing the more uncertain innovation initiatives with the

UOI? Is it the intention that the majority of initiatives in UIF will be of this nature?

b. Can you help us understand the differences between UNICEF Ventures Initiatives and

UNICEF Scale initiatives in terms of uncertainty?


Page: 209

Figure 1: UNICEF Innovation Diagram

8. Do you have any advice for developing an evaluation method which is suited to the UNICEF

Innovation Funds Venture Capital approach where 'the innovator' is external to the organisation

and the majority of initiatives are expected to fail?

Objective 3: Secondary Effects

9. The UNICEF Office of Innovation notes the often hidden value in the secondary benefits or innovation

such as:

(ix) Organisational learning (often through failure)Making the organisation ready for change

(x) Inspiring staff and attracting new types of talent

(xi) Creating a sustainable network of long term relationships even after completion

(xii) External mimicry

(xiii) Creating new business models

(xiv) Building capabilities

(xv) Driving organisation goodwill (i.e. new funders/allies)

a. Can you elaborate on any of the above giving examples of how value was captured and

demonstrated? Are you currently able to track second and third level effects? If so, how?


Page: 210

Closing

10. Is this anything further that you wish to add that we have not asked that you feel is relevant to our

study or any other points that you wish to share?

11. We may request a possible follow up interview or correspondence depending on the findings from

other participants. Would you be open to facilitating another engagement?

Thank you for your time. It is much appreciated.

----------End----------


Page: 211


The Authors:

For further information regarding this report please do not hesitate to contact:

Brian Meagher

Email: [email protected]

LinkedIn: https://www.linkedin.com/in/brian-meagher-872a4783

Twitter: @meagherbrian

Ajay Melwani


LinkedIn: https://www.linkedin.com/in/ajay-melwani-48b5b09/

Declan McKeogh


LinkedIn: https://www.linkedin.com/in/declan-mckeogh-93635787/

Haitham Afifi


LinkedIn: https://www.linkedin.com/in/haitham-afifi-14a5bb24/

John Santiago-Miller


LinkedIn: https://www.linkedin.com/in/john-santiago-miller-0267033/

Innovation


https://www.linkedin.com/in/brian-meagher-872a4783

https://www.linkedin.com/in/brian-meagher-872a4783


https://www.linkedin.com/in/ajay-melwani-48b5b09/


https://www.linkedin.com/in/declan-mckeogh-93635787/


https://www.linkedin.com/in/haitham-afifi-14a5bb24/


https://www.linkedin.com/in/john-santiago-miller-0267033/