61
1 Turn "Big Data" into Business Value with Real-Time BI Timo Elliott, March 2012

Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

Embed Size (px)

Citation preview

Page 1: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

1

Turn "Big Data" into Business Valuewith Real-Time BITimo Elliott, March 2012

Page 2: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

2

The information in this presentation is confidential and proprietary to SAP and may not be disclosed without the permission of SAP. This presentation is not subject to your license agreement or any other service or subscription agreement with SAP. SAP has no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation and SAP's strategy and possible future developments, products and or platforms directions and functionality are all subject to change and may be changed by SAP at any time for any reason without notice. The information in this document is not a commitment, promise or legal obligation to deliver any material, code or functionality. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. This document is for informational purposes and may not be incorporated into a contract. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP´s willful misconduct or gross negligence.

All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.

Legal Disclaimer

Page 3: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

4

Top Technology Priorities of CIOs, Gartner

Rank Technology1 Cloud computing2 Virtualization3 Mobile technologies4 IT management5 Business Intelligence6 Networking, voice, and

data communications7 Enterprise applications8 Collaboration

technologies9 Infrastructure10 Web 2.0

Rank Technology1 Analytics and business

intelligence2 Mobile technologies

3 Cloud computing (SaaS, IaaS, PaaS)

4 Collaboration technologies (workflow)

5 Legacy modernization

6 IT management

7 CRM

8 ERP applications

9 Security

10 Virtualization

2011 2012

Page 4: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

5

Top 10 CIO Business and Technology Priorities, 2012

Rank Top 10 Technology Priorities Top 10 Business Priorities

1 Analytics and business intelligence Increasing enterprise growth

2 Mobile technologies Attracting and retaining new customers

3 Cloud computing (SaaS, IaaS, PaaS) Reducing enterprise costs

4 Collaboration technologies (workflow) Creating new products and services (innov.)

5 Legacy modernization Delivering operational results

6 IT management Improving efficiency

7 CRM Improving profitability (margins)

8 ERP applications Attracting and retaining the workforce

9 Security Improving marketing and sales effectiveness

10 Virtualization Expanding into new markets and geos

Page 5: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

6

Business Analytics Provides Great Value

Data is extremely important for competitiveadvantage

Data makes an important contribution to customer relations efforts

Business information has helped manage costs or improve operations

Executives believe companies can benefit greatly from using data, especially information generated within the company

Agree: 69% Agree: 77% Agree: 70%

Page 6: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

7

Business Analytics Continues Steady Growth

IDC Worldwide Business Analytics Software – $billions – 2011 and 2012 estimates

17.519.4

22.124.3 24.9

28.130.4

33.9

2005 2006 2007 2008 2009 2010 2011 2012

*“After three decades, the business analytics market is finally reaching the mainstream”

“There are few growth inhibitors in the foreseeable future”

Dan Vesset, IDC*

+8.2%

Page 7: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

8

Analytics is an Ever-Increasing Share of IT Budget

2009 2010 2011

3.9%

+4.1%

+4.3%

Gartner: worldwide BI, analytics and performance management software revenue

“BI spending has far surpassed IT budget growth overall for several years”

Dan Sommer, Gartner

Page 8: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

9

IT Spending Per Head Rising Fast

New revenue generated from IT initiatives (enterprise innovation, context-aware computing, social networks, etc.) will become the primary factor determining CIOs compensation.

Information-smart businesses will increase recognized IT spending per head by 60%.

“Enterprise leaders and stakeholders must change their way of thinking that “lower is better” for IT spending per employee”

Gartner Predicts 20112011 2015

+60%

Page 9: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

10

100%

75%

2020

50%

2014

User Growth

25%

2011

Percentage of Potential Users That Analytics Will Reach

Page 10: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

11

What’s Hot?

Collaborative, real-time, and predictive rose most, while ease of implementation, fast exploration, and ease of use remained highly important

Page 11: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

12

What’s Hard?

Poor organization of data is a challenge

Agree: 88% Agree: 81% Agree: 77%

Poor processes for sharing data between departments and employees is a hurdle

Technical issues—such as data silos or incompatible systems—represent at least some challenge

Page 12: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

13

What’s Hard?

Big Data!

Page 13: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

14

1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010

Data Scientists Wanted

Page 14: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

15

Page 15: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

16

Page 16: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

17

Page 17: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

18

BIG DATAREAL TIMEPREDICTIVE

Page 18: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

19

CRM Data

GP

S

Demand

Spe

ed

Velocity

Transactions

Opp

ortu

nitie

s

Service C

alls

Customer

Sales Orders

Inventory

Em

ails

Tweets

Planning

Things

MobileInstant M

essages

Worldwide digital content will double in 18 months, and every 18 months thereafter.

VELOCITY

In 2005, humankind created 150 exabytes of information. In 2011, 1,200 exabytes will be created.

VOLUME VARIETY80% of enterprise data will be unstructured, spanning traditional and non traditional sources.

Gartner

IDC

The Economist

Page 19: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

20

Data Qwality

(it’s Timo Elliott with two ‘t’s)

Page 20: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

21

Data Quality is a Real Problem

33% Lost a customer or new business deal as a result of missing data

30% Lost business due to the mishandling of data

30% Of operating revenue is wasted through process failure and information rework

4% Rate data quality as excellent

63% Have no idea how much data quality might be costing

17% Have no plans to start a data quality initiative

88% See data as a strategic asset

Sources: Vanson Bourne; Larry English; The State of Data Quality Today

BUT

Page 21: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

22

Applications for Data Stewards

Page 22: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

23

Ingest Kafka Flume Scribe

Process Azkaban Oozie Pig HiveHadoop MapReduceS4 Storm

Store Voldemort Cassandra Hbase

Present Big Data Applications?

Page 23: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

24

The New Data Warehousing

In-Memory

MassivelyParallel

Calculation Engine

Web, UnstructuredSensors, Real-time

Hadoop, NoSQLComplex

Event Processing New Analytic

Platforms

ERP, Replication

Cloud

External data

“Extreme data performance”

Everybody in the industry is doing some combination

Column Data Store

Page 24: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

25

Understand Big Data through Information Management LifecycleTechnologies supporting enterprise requirements naturally co-exist and co-evolve

Ingest Store Process Present

Effo

rt

Effo

rt

Unstructured

Replication/ synchronization

Extract-transform-load Event stream processing

Database Many instance formats Many storage formats ACID properties Transactional

Versatile data queries using SQL/OLAP

Scripting and UDF to process In-DB

Federate queries across DB and DFS

Low-latency processing

Mature connectors to transport data to varietyof visualization tools

Structured

File transfer Extract-load data from

variety data sources

Distributed File System (DFS) Files stored in native format Software-driven reliability model BASE properties Lack of BI tool support

Transform to pre-process or at time of request using map-reduce

Huge amount of efforts to define the SQL

Scheduled chains to create consumable information

Simple SQL-like connectivity High-latency processing

Need new storage formats for high speed and high concurrency

Emerging visualization tools

Page 25: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

26

Big Data Analytics

Mobile

TransactionProcessing

DB Engine

In‐memory Technology

DB Engine

Analytic Grid

DB Engine

MapReduceBatch Compute Framework

Ingest

Store

Process

Presen

t

Sybase Replication Server,  SAP BusinessObjects Data Services(Integrate / synchronize data across deployment options)

Sybase ESP

Stream & event processing

SAP Big Data Processing Framework

SAP HANA  Sybase IQ 

Sybase ESP

Monitor / filter streaming events

Semi‐structured Data Structured Data Unstructured Data

HadoopSybase ASE

Hive/HDFS 

Big Data Applications

Page 26: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

27

Business Intelligence

Analytic Platform

New Analytic Infrastructures

DataBusiness Applications

We now have the possibility to dramatically simplify our analytic infrastructures

SPEED & FLEXIBILITY

Calculation Engine

Query Results

Operational Data Store

Data Warehouse

Indexes

Aggregates

ETL

DataMarts

Query

Copy

Page 27: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

28

Blur Operations and Analytics

Copy

Business Applications

Analytic PlatformBusiness Intelligence

Use a single platform for both analytics and applications

Data

Page 28: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

29

The New Analytics Experience

DataDiscovery

Extranets

“Actionable Insights”

Mobile Predictive

Vizualization, Geolocation

Social

Action-oriented Context

Collaborative DecisionSupport

Page 29: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

30

What Causes Decision Problems? People!

Source: Economist Intelligence Unit

Page 30: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

31

Major Strategy Directions

BI

DEEPERINTEGRATION

RICHERINSIGHT

SOCIAL

MOBILE

SUB POINT

Methodsand

Templates

Structureto

Teamwork

Ad-hoc Teams

Page 31: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

32

Integration

HANA

Sybase IQ

SAPBW HierarchiesBICSERP InfoSetsABAP Functions…

Microsoft Sharepoint

HANAIn-Memory Computing

Page 32: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

33

Insight to ActionWeb Services

BI docs

Applications

Supported source documents: • SAP Crystal Reports• SAP BusinessObjects Dashboards

Page 33: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

34

Hadoop And Related Technologies

Doug Cutting with his son’s “Hadoop” elephant

Page 34: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

35

Accessing Hadoop Data

Page 35: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

36

Information Design Tool on Hadoop Hive

A Data Foundation against a Hive schema■ One can draw joins between the Hive tables■ We support Hive tables, aliases, derived tables, Hive views and Hive partitioned tables

Page 36: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

37

Examples

Page 37: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

38

Richer Insight

Page 38: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

39

Richer Insight

Page 39: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

40

Predictive Analysis

Page 40: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

41

Predictive Analysis

Page 41: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

42

Predictive Analysis

Page 42: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

43

Predictive Analysis for the Rest of Us

Web Intelligence Web Intelligence Explorer

StreamWorkExplorer Analysis, Edition for OLAP

The Predictive Analysis Process Designer can connect to BOE 3.x and 4.x Universes enabling them as data

Embedding Predictive Analysis Services to provide for example – trend analysis, forecasting, outlier detection etc.

Page 43: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

44

Page 44: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

45

Mobile

Page 45: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

46

Try the new “Experience SAP” Mobile Application

Page 46: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

47

Recalls Plus

Page 47: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

48

Paul Predicts

Page 48: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

49

Afaria Analytics

Devices

Applications

Number of Devices by Operating system (OS) Number of Devices by Carrier Number of Devices by Manufacturer Number of new devices added each month by OS Ownership of Devices [Corporate/Personal]

Roaming activity of Devices [Data/Messaging/Voice] International roaming activity of Devices for current billing period Number of devices that exceed the defined activity threshold Data, Voice and Messaging Usage by Carrier

Activity

Compliance

Devices that are compromised (iOS & Android) Number of Devices that have not connected in time Number of iOS Devices without a password policy Number of Devices that are out of compliance (by platform) Number of Devices that have access violations (by platform)

Number of Enterprise Applications installed by month Top 10 Enterprise Applications by installation status Volume Licensing status for Enterprise applications Top 10 Enterprise Applications by platform & installation status Number of Enterprise applications that are out of version

Page 49: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

50

Afaria Analytics

Page 50: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

51

Voice Recognition is Getting Siri-ous

Page 51: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

52

Next Up: Brain Waves!

Page 52: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

53

Page 53: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

54

Page 54: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

55

StreamWork: SAP’s Collaboration Foundation

Page 55: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

56

Collaboration in Analytics

Page 56: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

57

Page 57: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

58

SAP Social Intelligence

Expertise location — Relationship Mining — Social Network Analysis

Page 58: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

59

AnalyticApplications

Governance,Risk, and

Compliance

Enterprise Performance Management

BusinessIntelligence

DataWarehousing

Enterprise Information

Management

Business Analytics Solutions from SAP

Analytic Capabilities

Collaboration

Data Sources Access

Services and Best Practices

Page 59: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

60© SAP 2009 / Page 60

Virtual Event www.sapvirtualevents.com

Visit our Business Analytics resource center

Visit our blog http://blogs.sap.com/analytics

Learn the 4.0 solutions with 4.0 tutorials

Join our official fan page on Facebook and follow us on Twitter

SAP BusinessObjects Links

Page 60: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

61

Conclusion

Big data is not just about big dataSAP Big Data Framework for volume, velocity, variety, validity

Revolution in analytic platformsBack end: simplification, new real-time opportunitiesFront end: mobile first, collaboration and consumerization

Innovation without disruptionSAP BusinessObjects: radically change your business without having to radically change your infrastructure…

Page 61: Turn Big Data into Business Value nordic distributiontimoelliott.com/blog/docs/innovationbysapbigdata_nordic.pdf · Turn "Big Data" into Business Value ... code or functionality

Thanks!

Email:[email protected]

Analytics Blog:timoelliott.com

You Should Follow Me on Twitter: @timoelliott