Dance with the elephant & Keeping all of your toes An Architect’s Guide to Big Data Robert Stackowiak, Oracle North America, VP Information Architecture

Dance with the elephant & Keeping all of your toesAn Architect’s Guide to Big Data

Robert Stackowiak, Oracle North America, VP Information Architecture & Big Data

Doug Cackett, Oracle EMEA, Chief Architect

September 30, 2014

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. 3

Safe Harbor StatementThe following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.


Program Agenda

Framing the conversation

A methodology to deliver success

Defining future state architecture

How to avoid the oncoming elephant

Next Step

1

2

3

4

5

Copyright © 2014, Oracle and/or its affiliates. All rights reserved.

Silos Standardize Optimize Information as a Platform

• One off tools/solutions• Bottom-up• Local business unit

driven• Many versions of truth• Independent data

marts

• Enterprise standard tools/solutions

• Data warehouse and dependent data marts

• IT & LOB partnership• Secure, consistent

access to all data

• Agile, flexible architecture

• Analyze “just in time” structured & unstructured data together

• Advanced analytics & real-time recommendations

• All stage 3 capabilities delivered as a platform (service/cloud)

• Access of tools and data among broad subscriber group

Maturity & Capability

Unified Information Architecture Maturity Phases


Big Data Footprint & Scope of Architecture

zBY INDUSTRY & LINE OF BUSINESS

BIG

DAT

A AP

PLIC

ATIO

NS

DISCOVERY

BUSI

NES

SAN

ALYT

ICS

BUSINESS ANALYTICS

DATA RESERVOIR

BIG

DAT

AM

ANAG

EMEN

T

DATA WAREHOUSE

SOU

RCES


Architecture Vision Common Emerging Platform Pattern

Data Warehouse / Data Marts

Business Intelligence Tools

ERP, CRM & Other Transactional Apps

Historic Source of Truth Reporting, Query and Analysis Tools

Information Discovery Engine

Advanced Analytics

Website Logs & Data NoSQL DBSensors

Hadoop High Volume Distributed File System

Structured Data

Semi-structured Data

Real-Time Analyticsand Recommendations

Recommend Location & User Profile

R, SAS

Discoveries


Potential Oracle Products in the Footprint

Endeca Information Discovery on Exalytics

Cloudera HDFS on Big Data Appliance

Reliable, Available, Secure Source of Truth

Fast, Intuitive Data Discovery

Website Logs & Data Oracle NoSQL DB

Real-time Recommendations

Analyst Friendly Reporting Query & Analysis Tools

Unstructured Data Analysis

Sensors

Oracle Database DW on Exadata

Oracle BI Foundation Suite, Hyperion on

Exalytics

Oracle ERP & CRM Solutions on Exadata

OracleReal-Time Decisions

Structured Data Analysis

Big Data Connectors

ODI

OEP

Advanced Analytics, In-Memory, Big

Data SQL

R, SAS


Extending to the Internet of Things

Smart Sensor

IoT Comm. Agent

Application Framework

Sensor

IoT Comm. Agent


Gateway


IoT Comm. Agent &

Mgmt Proxy

IoT Comm. Gateway

Protocol Mediation

Web Services Security

Web Services Mgmt

IoT Identity & Access Mgmt

Device Access Management

Device Identity Management

Identity Directory/Database

IoT Management

Device Registration & Mgmt

Message Cache

Management Console

Applications Provisioning

Data Routing & Analysis

Distributed Data Grid

Complex Event Processing

Big Data

Big Data Store

Data Aggregator

NoSQL DB

Key Value Data Store

Database

RDBMS

Business Intelligence

Performance Management

Alerts, Dashboards & Reports

Advanced Analysis & Data Science

Tools

Data & Knowledge Discovery Tools

Enterprise Integration

Complex applications integration & SOA

DatacenterManagement

Server Management & Monitoring

Gather Enrich Stream Manage Acquire Organize & Analyze

Local Event Analysis

Ca

pil

lary

Ne

two

rks

Ne

two

rk F

ire

wa

ll

Intr

an

et

Fir

ew

all

Da

tab

as

e F

ire

wa

ll


IT Focus

Implementation Rodmap

(IT & LOBs)

Business Information Mapping (LOBs)

Visioning & Innovation (IT & LOBs)

Technical Information Architecture (IT)

Use Case Hypothesis (LOBs)

Proven Methodology for Success

Implementation (IT, SI & LOBs)


Sample Use Case including Critical Success Factors & Measures

Perspectives

Financial

Optimize Channels and measure effectiveness of new product launches

Customer

Increase self-qualification, understand customer needs then targeting

Internal

Better simplify & optimize retail channels for better

close rates

Innovation & Growth

Measure success of campaigns &

advertising, better relationship

management

Critical Success Factors

Regional financial optimization- products / sales

Accurate cost analysis

Measure new launch success


Self-serve web experience tied to right channel

Understand customers lost


Right level of complexity in sales & order process

Reduce order cycle time


Right ads and promotions at right markets / right time

Ability to measure impact

Improve Sales through All Channels (by 22 percent)

MeasuresSales, marketing expense,

markups /discounts and win rate, cost, price sold at, margin, warranty & service

MeasuresEnd-to-end sales process &

interactions, sentiment, reasons for losses, close ratio, customer retention

Measures Leads converted, time to close,

sales cost, requests for info, touch point effectiveness

MeasuresAd expense, sales, customer

satisfaction, market share potential vs. sales attained


Defining a Business Information Map / Data Flow


Defining a Roadmap with Achievable Goals & Payback


Rethinking Information Architecture

• Design for Analytics 3.0• Connect information to execution• A platform for all data• Establish an architectural framework• Enable discovery to keep it relevant• Automate to manage and govern

Architect to avoid getting trampled by the oncoming elephant


Data as the Foundation for Business… but no longer the complete answer – agility is the key

Analytics 2.0

Analytics 3.0

Analytics 1.0

• Reporting with limited use of descriptive analytics

• Limited range of tabular data • Batch oriented analysis • Analysis bolted onto limited set

of business processes

• Firms “Competing on Analytics”• Extended analytics to larger and

less structured datasets• Emergence of Big Data into the

commercial world• Recognition of Data Science

• Platform for monetisation• Deeper analysis & more data• Faster test-do-learn iterations• Different types of data & wider

business process coverage• Analysts focus on discovery and

driving business value• Agile with operational elements

incorporated in design patterns

Adapted from Tom Davenport


Designing for Analytics 3.0


Manage all the data in a Big Data Management SystemBut find the right place to put it

Tooling maturity

Availability & Business Continuity

ACID transactional requirement

Security

Variety of data formats

Data sparsity

Ingestion (coding) simplicity

Cost effectively store low value data

Ingestion rate

Straight Through Processing (STP)

0

5

HadoopRelationalMy Ap-pllication


Information Interpretation

Establish an architected frameworkBalancing agility with ingestion and interpretation costs

Virt

ua

lisa

tion

&

Qu

ery

Fe

de

ratio

n

Enterprise Performance Management

Pre-built & Ad-hoc BI Assets

InformationServices

Data Science

Data Ingestion Access & Performance Layer

Foundation Data Layer

Raw Data Reservoir

Data Sources

Data Engines & Poly-structured sources

Content

Docs Web & Social Media

SMS

StructuredDataSources

•Operational Data•COTS Data•Streaming & BAM

Master & ReferenceData Sources

• Incr

easin

g qua

lity &

enric

hmen

t

• Redu

cing c

osts

of qu

ery c

oncu

rrenc

y

• Incr

easin

g for

mali

satio

n of d

efini

tion


Recognise the value of discovery

• Discovery & exploitation require differing people, process & technologies

• Design to minimise time taken and size of governance step

• Agile BI must be part of the design thinking for ongoing commercial exploitation

Discovery is about data understanding NOT IT development

Commercial Exploitation

Discovery phase

Understanding of the data

Governance

Business Value

Time / Effort


Establish a data discovery capabilityAccelerate with automated provisioning / de-provisioning steps

Analysis Processing & Delivery

Discovery Lab & Data Science Tooling

Data Reservoir & Enterprise Data

Data Science(Primary Toolset)

Statistics Tools

Data & Text Mining Tools

Faceted Query Tools

Programming & Scripting

Data Modeling Tools

Query & Search Tools

Pre-BuiltIntelligence

Assets

IntelligenceAnalysis

Tools

Ad Hoc Query& Analysis Tools

OLAP Tools

Forecasting &Simulation Tools

Reporting Tools

Data Scientist

Vir

tua

lisa

tion

&

Info

rma

tion

Se

rvic

es

Data Factoryflow

Data Quality & Profiling

Graphical rendering tools

Dashboards & Reports

Scorecards

Charts & Graphs

Sandbox – Project 3



Data store Analytical

Processing

General BI flow


Next Step: Contact Oracle for Help

• North America– Information Architecture & Big Data: [email protected]– Oracle Enterprise Architects / Information Architecture: [email protected]

• EMEA– Information Architecture & Big Data: [email protected]

• Download the Big Data & Information Management white paper here

mailto:[email protected]



http://www.oracle.com/ocom/groups/public/@otn/documents/webcontent/2297765.pdf


Documents

Dance with the elephant & Keeping all of your toes An Architect’s Guide to Big Data Robert Stackowiak, Oracle North America, VP Information Architecture