16
WHITE PAPER: Big Data & Predictive Analytics; THE CHALLENGES AND OPPORTUNITIES FOR ALL.

Odgers Berndtson and Unico Big Data White Paper

Embed Size (px)

Citation preview

WHITE PAPER:

Big Data & Predictive Analytics;THE CHALLENGES AND OPPORTUNITIES FOR ALL.

INTRODUCTION

ABOUT THIS DOCUMENT

UNLOCKING THE POWER OF PREDICTIVE ANALYTICS

DEFINING PREDICTIVE ANALYTICS

THE FIVE KEY UNDERSTANDINGS WITH BIG DATA AND PREDICTIVE ANALYTICS

WHAT IS POSSIBLE TODAY?

THREE PREDICTIONS FOR THE FUTURE

Q&A

APPENDIX; INDUSTRY ATTENDEES OF THE SYDNEY AND MELBOURNE EXECUTIVE BREAKFASTS

02

03

04

05

06

09

09

11

14

Contents

02

Big Data remains one of the most significant opportunities for business, but also one of the most poorly leveraged. In numbers, only 0.5 per cent of data is ever actually analysed, meaning that organisations are not only wasting precious resources collecting and storing data, they are missing out on the significant insights that data can provide back into the business.

What businesses are good at is the capture of raw data. Studies have found that 90 per cent of the world’s data has been created in just the past two years, and the rate of data collection is not slowing down – we are expecting an increase of 4,300 per cent of yearly data production by the year 2020.

This rate of data creation – and capture – has the potential to exasperate the overwhelming challenges that businesses are facing in data

analysis. Businesses are aware of the need to scale their investments into Big Data analysis. A Wikibon forecast expects the global Big Data market to grow from $18.3 billion in 2014 to $92.2 billion in 2026 – a compound annual growth rate of 14.4 per cent. IDC, meanwhile, predicts that through 2020, spending on cloud-based Big Data and Analytics technology will grow 4.5 times faster than spending on on-premises solutions.

Perhaps the most clear indicator of the raw demand for data solutions can be found in the venture capital space, where Big Data startups picked up 11 per cent of all tech venture capital in 2015. Big Data is not just a trend – it’s equal parts pain point and opportunity, and organisations are keen to resolve the former and capitalise on the latter.

Introduction“Artificial Intelligence, or more correctly Machine Learning, is a powerful new technology that must be understood by every company. The potential impact that AI/ML will have on businesses is significant and will enable companies to provide new services and drive greater insights more quickly. Every executive must understand this opportunity and what impact it could have on their operations.” –David Thodey, Chair CSIRO

03

This white paper looks at where the biggest data opportunities are, and how businesses can capitalise on these by better understanding what solutions will be of benefit for their business.

Unico in collaboration with Odgers Berndtson engaged senior enterprise executives from a wide cross-section of industries during two business roundtables in October 2016. Please see the attendance list in appendix one. The insights from these discussions are captured throughout the document.

From page 14 you will find a Q & A section, in which the industry attendees highlighted their key questions around the implementation of and use of predictive analytics and data analysis.

About this document

This white paper was generated to capture real world experience and thinking in data use and analysis.

04

A KPMG study found that for CEOs, Big Data and analytics was a top-3 investment priority. Two thirds of CEOs, according to the report, were concerned that their organisation wasn’t being disruptive enough, and this concern was in significant part driving interest in data; CEOs saw it as a path in developing new products and services, while realising greater savings and efficiencies.

For the CFO, Big Data offers the opportunity to develop a broader understanding of a businesses’ ever more complex corporate governance responsibilities. For these executives, moving from a retrospective an intuition-driven decision making process, to

one based on data, will help the business be more proactive and holistic in the way it handles its economics.

The CMO benefits too. Currently, marketers see predictive analytics as a holy grail to their work. A Forrester Consulting study found that 89 per cent of B2B marketers had identified predictive analytics as being critical to their roadmaps in 2016, and that 78 per cent of them see B2B marketing as expanding to deal acceleration from demand generation. From that statistic it’s easy to deduce that predictive analytics will play a key role in a critical redefinition of marketing in many businesses.

Unlocking The Power OF PREDICTIVE ANALYTICS

Most members of the C-suite community now regard Big Data and analytics to be a critical part of their role into the future.

“It is apparent that boards and executive teams are under enormous pressure to land on a ‘future’ operating model for their businesses. A robust and well tested strategic plan that is dynamic and data driven is essential for the new digital era and therefore the demand for executives that understand this data driven world is significant and will only continue to grow especially when we consider how wrong we can be if we don’t use all the tools at our disposal.”

–Paul Rush Partner, Odgers Berndtson

05

Once an organisation has collected and centralised its data, it is then ready to have predictive analytics applied to it, in order to derive key pieces of information. It is important that organisations do this step; without the analysis applied to the data, the ROI in collecting and storing the data is non-existent and all the data will be good for is searching. It’s in the analysis that the value of big data is unlocked.

Predictive analytics specialises in making use of unstructured data, or what we generally refer to as ‘Big Data.’ Structured data makes use of fixed fields

– spreadsheets or relational databases, for example. Unstructured data includes photos, webpages and digital documents; it’s the data that isn’t meaningful when placed into neat boxes and categories.

The concept of Big Data can also be understood in two ways. It could be related to the number of samples or observations that exceed certain threshold; or it could be the number of dimensions exceed certain number, even

Because predictive analytics includes automated processes, applications of the technology often have strong Machine Learning capabilities built in, in order to automatically “learn,” adapt, and update as new data is collected.

But Machine Learning is not always going to generate the best results from the data, either. If the data is highly linear, then an investment in Machine Learning can be wasteful and inefficient. For this reason it’s important to fully understand the nature of the data before developing the algorithms with which to analyse it.

if the number of samples is relatively small. For an easy example, researchers might have DNA genes data consisting of more than 10,000 genes per sample, but have less than 100 samples. This is still Big Data. In general, we classify small data as having equal to or less than 15 attributes, medium data of between 15 – 25 attributes, and Big Data as more than 25 attributes.

There are two primary challenges in leveraging Big Data; the first is that unstructured data is proving very difficult to leverage by organisations, , and with around 80 per cent of all data being unstructured, tools and services that help organisations make sense of it all, such as predictive analytics, are immeasurably valuable. The other challenge, however, is that for machine learning or data mining algorithms, the high dimensionality of many examples of big data is a significant technical challenge that needs to be overcome - often through custom algorithms - before an organisation will be able to derive meaningful and accurate insights from the data.

Predictive analytics can be used to: • Predict future trends/events• Identify patterns• Identify casual relationships

between things• Image recognition• Text mining and processing

Defining PREDICTIVE ANALYTICS

06

UNDERSTAND THAT NOT ALL DATA IS EQUAL

The sheer weight of data being created means that it’s simply impossible to give every byte of it equal weighting. To make effective use of data, and the solutions that are implemented to leverage it, enterprise needs to focus on the nature of the business and the kind of data it will derive best value from, and then identify how it captures and applies analytics to the datasets. In effect, we need to get back to basics and understand and redefine Big Data before we can start to work with it.

Data can be split into three distinct categories: structured, unstructured and semi-structured. Structured data is what people typically visualise when they think of data; it looks like numbers in a spreadsheet. This is relatively simple to analyse.

Unstructured and semi-structured data is more difficult to analyse, and might take more unusual forms such as:

• Any digital image has data behind it which can be analysed.

• Word documents full of words that can be analysed with text processing algorithms.

This is harder to analyse, but forms most of what we term ‘Big Data’, and is where most information is actually contained.

Five Key Understandings WITH BIG DATA AND PREDICTIVE ANALYTICS

Capitalising on the opportunities that predictive analytics enables requires business to take a measured, five-prong approach to the planning, rollout and subsequent management of technology solutions:

01

07

NOT ALL ‘ANALYSIS’ IS EQUAL (AND PEOPLE ARE A CRITICAL PART OF THE PROCESS)

There is a difference between analysis and analytics. Analysis unlocks the true value of data by extracting meaningful insights from what has been collected through the analytics strategy. To put it simply: Data analytics without analysis is just data.

It is important to have a human element involved in all analytics strategies to properly extract the insights behind what the data is telling us, and to hold the data analytics

ROBUST ALGORITHMS

Evolving technologies such as we’ve seen in Hadoop, parallel processing, and cheaper storage techniques, are combining to make the collection and storage of data easy for businesses of all sizes. However, data of that size and scale also requires advanced and robust algorithms to be able to leverage the insights in a productive and efficient manner.

algorithms to account. Machine Learning and robust algorithms are certainly able to take the busywork out of a data scientist’s role, however, it is important that the role not be made redundant; instead the role of a data scientist should transition to something more strategic to the business, and data scientists in the future will need to have a better understanding of the value and purpose of data at all other branches of the business’ operations.

Most critically, organisations need to consider developing bespoke algorithms that speak to the data that is important to their business. Algorithms and their formulas need to be able to cut through the “noise” of the sheer mass of data out there, and collect (and analyse) only the relevant economic, locational and behavioural data sets.

Five Key Understandings WITH BIG DATA AND PREDICTIVE ANALYTICS (CONT.)

02

03

08

Five Key Understandings WITH BIG DATA AND PREDICTIVE ANALYTICS (CONT.)

WE NEED TO UNLOCK THE VALUE OF MACHINE LEARNING

If the data is non-linear, it needs to have Machine Learning applications applied to be able to generate meaningful insights over the short, medium, and long term. Machine Learning is, basically, the function that helps technology get better with time.

Machine Learning is important to the analysis of large quantities of unstructured data, as we find in Big Data. Solutions that have robust

USING DATA TO PREDICT OUTCOMES

The key purpose of the collection and analysis of Data is to identify trends and opportunities, and to use that information to gain competitive advantage to better position the business.

It’s easy to fall into a trap whereby an organisation draws data into the business and then extracts insights looking backwards; at what it already understands. Properly configuring the Big Data insights strategy from the outset is essential to commercial success.

Machine Learning processes built into them are able to extract value from the multitudes of sources that input data into an organisation, and do so without human supervision. Furthermore, Machine Learning applications become better the more data that is fed into them. With this in mind, effective Data Analytics projects in the future will implement Machine Learning processes as standard.

04

05

09

Three predictions FOR THE FUTURE

What does the future hold for Big Data and predictive analytics? Based on the predictions and trends for the current state of the Big Data industry, we can expect three key themes to emerge over the coming years:

BIG DATA IS ACCELERATING INTO EVEN BIGGER DATA

The first trend, as noted earlier in this paper, is that Big Data is only going to become more overwhelming in terms of how much is captured and stored, both in terms of structured and unstructured data.

This presents challenges. When businesses as a whole are currently only analysing 0.5 per cent of the data coming into the business, whatever challenges they are facing in increasing that percentage will be exasperated as the organisation brings in more and more data. At the same time, a greater understanding of what can be done with data, as well as

01improved tools, Machine Learning, and analysis strategies, there is also going to be much greater datasets to play with, which provide in turn much better quality data to draw insights out of.

We can also expect to see an increase in complex data analysis being done in real time, assisted by automation and Machine Learning. Previously this has been difficult to achieve, but now it’s possible to generate actionable insights out of your data as it comes in; for example, model parameters can be updated in real time as data becomes available.

10

Three predictions FOR THE FUTURE (CONT.)

AUTOMATION WILL DEMOCRATISE DATA, AND MAKE IT VALUABLE TO ALL WITHIN THE BUSINESS

It will become important that all people within the organisation, from CEO to marketing and on to HR, will be able to use data in their role. Gartner’s now-infamous prediction that an organisation’s marketing team will spend more on technology than the IT team by 2017 stems in no small part from the expected increase in spending on data.

DATA SCIENTISTS WILL BECOME ONE OF THE MOST VALUABLE RESOURCES IN COMPANIES

Australia is facing a skills shortage as data scientists become more and more in demand. This is going to push up wages and movement between jobs, and given that data scientists will be in demand across most industries, businesses will need to develop a strategy to obtain – and then retain – the data scientist talent.

Third parties might be a solution, with organisations gaining access to data scientist skills by engaging with a trusted partner for building the analytics strategy. For others, 457 visas or outsourcing overseas might be the way to go.

02

03

Equally, SMEs can expect to have access to increasingly sophisticated tools for data analysis, as these technologies become more established in the market. In order to assist individuals and businesses make use of the data across the organisation, and to compensate for a lack of internal skills in SMEs, automation will become a more prominent tool that businesses invest in as past of their Big Data spend.

Data scientists will be in high demand because they will provide businesses with competitive advantage. There will be off-the-shelf analytics solutions available, but businesses will know that if their rival uses the same analytics that they are, there will be no competitive advantage. Instead, these businesses will turn to their data scientists to develop bespoke applications that will give them access to data insights that their competition does not have.

11

Q&A

How much time do you spend setting up the framework - what

are the timelines?

Why bespoke and not off-the-shelf?

Across two senior executive roundtable events held in Melbourne and Sydney in October 2016, business leaders were engaged in a discussion on how Big Data and predictive analytics can be applied to their own businesses and business practices. These questions were a key focus for enterprise:

The time frames involved in setting up and executing an algorithm can vary substantially, and can be anything from a couple of weeks to months in duration. Factors that can affect the rollout time include; the front end and the kind of

There are advantages to both bespoke and off-the-shelf solutions. If your Big Data goal is simply to be able to search and extract data, then there are off-the-shelf solutions that can comfortably meet those needs, from a variety of vendors. The added benefit of these solutions is that they are significantly more cost effective to get up and running than bespoke solutions.

Bespoke solutions have the advantage of being highly customisable; for example,

reports that are to be generated, the amount of data that needs to be input, and whether there needs to be a period of collecting new data in order to test the strength of the algorithm.

locational data is often important, and often difficult to derive real understanding from with an off-the-shelf solution. Of even more interest to a lot of organisations is that bespoke solutions provide genuine competitive advantage. You can be fairly certain that if you’re using an off-the-shelf solution, then so too is your competitors. But the insights delivered through a bespoke solution are likely to be to the benefit of your organisation alone.

12

Q&A (CONT.)

How do you embark on a proof of concept for one of these

things when it can be especially difficult to convince large

organisations to make these kinds of expenditures?

How should the data transfer be interpreted? Is it something that can be technology-driven, or is

human involvement important at this stage of the analysis cycle?

How do I deal with concerns within my organisation that staff fear of their

jobs as a consequence of Big Data analytics?

Businesses are vaguely aware of the need to invest in ‘big data’ already, so for data scientists or IT executives to sell up into management or the board, there is already a base awareness there.

The ability for predictive analytics to allow an organisation to get on the front foot with its competitive strategy is the

There is absolutely the need to have human eyeballs looking at various points in the analytics cycle. Automation and machine learning is very effective in collecting data and creating meaningful insights out of it. However, applying those insights to real-world scenarios requires a human understanding of the data as well.

It’s true that the insights generated by Big Data algorithms might highlight the redundancy or inefficiency within some roles within the organisation, however, we see Big Data as a net creator of jobs, and those staff would have the opportunity to take on new roles within the organisation.

For example, we created an algorithm for one organisation, and over a period of time the team managing it grew to include a head data scientist, and three people working underneath him. Because

salient point that needs to be made. Being able to refine margins, staffing numbers, inventory held, and so on based on real-time information is a compelling business case as organisations look to clamp down on unnecessary wastage in their spending, and find new and effective ways to reach customers.

It’s also important to have skilled data scientists looking at the insights, in order to understand whether abnormalities are signs of new trends, or otherwise whether the algorithm needs updating to meet changing dynamics in the market.

the insights generated from big data tend to be so valuable, organisations generally like to properly resource their analytics teams.

Getting the staff on board with a Big Data analytics strategy does require a change management strategy to come from the executive team, in order to get everyone on board, but over the longer term the career opportunities that these technology solutions enable will be compelling for staff at all levels of the organisation.

13

Q&A (CONT.)

Is it true that some machine learning algorithms ignore

clusters of data that it doesn’t understand or are new?

How do you determine weights for inputs into the

analytics algorithms?

What is the difference between Machine Learning and

Deep Learning?

This can happen, and this is why it is important to have skilled data scientists looking at abnormalities in clusters of data to determine whether the algorithm needs to be adjusted.

The short answer is that you shouldn’t be manually assessing weightings. The mathematical algorithm should be robust enough within the parameters of defining the relationship between the thing that you want to forecast, and the explanatory variables.

Deep Learning is “the analysis and learning of massive amounts of unsupervised data, making it a valuable tool for Big Data analytics where raw data is largely unlabelled and un-categorized.” Effectively, it is much the same thing as Machine Learning, with some variations in how the algorithm is applied.

A good algorithm will also throw up red flags when the clusters of abnormal data are significant enough.

With bespoke Big Data analytics projects, the mathematics team will approach the problem with the understanding of the outcomes, or the narrative that the analytics will create for the business. The weightings will be built into the equation with that goal in mind.

Businesses should not be relying on a single form of algorithm for their insights. With two or three different algorithms working on a single big data problem, the insights being derived from the analytics team will be more rounded and, therefore, beneficial. Organisations should therefore be investing in both Machine Learning and Deep Learning analytics solutions.

“The success of data analysis relies heavily on the relevance and quality of the source data and inputs, domain expertise is critical. Best practice analytics brings together business, science and technology to create a powerful differentiator.”

–Michael McKeon, Business Development Director, Unico Enterprise Services

14

Appendix; Industry Attendees OF THE SYDNEY EXECUTIVE BREAKFASTS

ANDY HEDGES

ANTHONY LAU

CAMERON GARRETT

CHRIS EARNSHAW-NEES

DAVID BIGHEL

JASON JUMA-ROSS

KAREN LAWSON

KYLE BUNTING

MICHAEL MCKEON

MICHELLE ZIVKOVIC

PAUL RUSH

PETER BINKS

RICHARD MCMANUS

TIM SCHNEIDEMAN

Leadership & Innovation Global Executive

ALAUD

Macquarie Group

Artis Group

ASX

Facebook

Slingshot Accelerator

TPG Telecom

UNICO

Directioneering

Odgers Berndtson

Crystal Bay Capital

Richard McManus

Hewlett Packard Enterprise

15

Appendix; Industry Attendees OF THE MELBOURNE EXECUTIVE BREAKFASTS

ADAM KYRIACOU

ANTHONY DEEBLE

ANTHONY MAGUIRE

BEN CHESTERMAN

DAVID ROBINSON

GENEVIEVE ELLIOTT

HAMISH COLEMAN

LARRY HOWARD

LOUISE HIGGINS

MARK GILBERT

MATTHEW PERRY

MICHAEL ALF

MICHAEL MCKEON

MICHELLE FITZGERALD

PAUL RUSH

PETER GAIDZKAR

RAPHAEL OWEN

RICHARD NEESON

ROBERT TURNER

Odgers Interim

Val Morgan

Telstra

Telstra

Assetic

Vicinity Centres

Vicinity Centres

Bluescope Steel

NOVA Entertainment

Telstra

DuluxGroup

KPMG Enterprise

UNICO

City of Melbourne

Odgers Berndtson

Reapit

Val Morgan

Original IT

Assetic