Concepts, technologies and the IBM perspective

Alberto Ortiz Big Data & Analytics Architect

January 2015

Big Data & Analytics

What is Big Data?

Definition of Big Data

... data comes from everywhere: sensors used to gather climate information, posts to social media sites, digital pictures and videos, purchase transaction records, and cell phone GPS signals to name a few. This data is big data.

Big data is an all-encompassing term for any collection of data sets so large or complex that it becomes difficult to process them using traditional data processing applications.

In short, the term Big Data applies to information that can’t be processed or analyzed using traditional processes or tools.

"Big Data." Wikipedia. Wikimedia Foundation, n.d. Web. 05 Jan. 2015.

"Bringing Big Data to the Enterprise." IBM. N.p., n.d. Web. 07 Jan. 2015.

Zikopoulos, Paul, Chris Eaton, Tom Deutsch, Dirk Deroos, and George Lapis. Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data. New York: McGraw-Hill, 2012. 3. Print.

It's not a specific type of software

It's not a product

It's not a specific amount of data

It's not just social media

What are the characteristics of Big Data?

Velocity

Sprint

53M subscribers

10B transactions/day

Volume

Vestas

2.6 Peta Bytes

Weeks for placement analysis

Variety

Airbus

Technical aircraft

information

Information in CRMs, data

bases, ERPs, ECMs, files, etc.

Veracity

Italian Bank

6th bank in Italy

MDM+Social Media+Emails

6

Leverage more of the data being captured

7

TRADITIONAL APPROACH BIG DATA APPROACH

Analyze small subsets of information

Analyze all information

Analyzed information

All available information

All available information analyzed

Reduce effort required to leverage data

8

TRADITIONAL APPROACH BIG DATA APPROACH

Carefully cleanse information before any analysis

Analyze information as is, cleanse as needed

Small amount of carefully

organized information

Large amount of

messy information

Data leads the way—and sometimes correlations are good enough

9

TRADITIONAL APPROACH BIG DATA APPROACH

Start with hypothesis and test against selected data

Explore all data and identify correlations

Hypothesis Question

Data Answer

Data Exploration

Correlation Insight

Leverage data as it is captured

10

TRADITIONAL APPROACH BIG DATA APPROACH

Analyze data after it’s been processed and landed in a warehouse

or mart

Analyze data in motion as it’s generated, in real-time

Repository Insight Analysis Data

Data

Insight

Analysis

What is analytics?

12

Source: http://d3js.org/

https://www.google.com/search?q=analytics+definition

Why did it happen?

Descriptive analytics

13

What is happening?

14

Text analytics

Sentiment analysis

Entity extraction

Natural language processing

What could happen?

15

Predictive analytics

What should I do?

16

Prescriptive analytics

Business rules

Optimization

Who is who? Who knows who?

Entity analytics

IBM solutions for BD&A

Watson: state of the art in BD&A

19

Watson Video

https://www.youtube.com/watch?v=P18EdAKuC1U#t=35

Watson Foundations

New / Enhanced Applications

All Data

Big Data & Analytics Platform

Watson Foundations Strategy, Integration & Managed Services

Watson Foundations Infrastructure

What is happening?

Discovery and exploration

Why did it happen?

Reporting and analysis

What could happen?

Predictive analytics and

modeling

What action should I take?

Decision management

Information Integration & Governance

Landing, Exploration and Archive data zone

Operational data zone

Real-time Data Processing & Analytics

Enterprise warehouse data mart

and analytic

appliances zone

22

What did I learn,

what’s best?

Cognitive

Industry cases using BD&A

Banking Big Data & Analytics Customer Case Studies

NYSE Euronext enhances trading insights and reporting with IBM PureData platform Need

• Rapid analysis of trade data to facilitate compliance and regulatory reporting

• New functionality needed to track the value of a listed company, performing trend analysis, and searching for evidence of fraudulent activity

• The previous system trawled through large amounts of irrelevant information to complete searches

Benefits

• Reduced time to run market surveillance by 99%

• Improved ability to detect suspicious trading activity

• Enabled early investigative action, minimizing damage to investing public

26

http://www.youtube.com/watch?v=j7KwuGFF9bU

Government Big Data & Analytics Customer Case Studies

TerraEchos provides security and intelligence expansion with real-time analysis of streaming data

Need

• Technology solution that would detect, classify, locate and track potential threats to secure its perimeters and border areas

Benefits

• Only vendor providing a Big Data platform for real-time analysis from streaming data for security

• Unprecedented real-time facility surveillance with the ability to process 1.6GB of data per second for insight into any event

• Confidently identify and classify a potential security threat -the system is sensitive enough to distinguish between a bobcat and a bad guy

28

Real-Time Security

Retail Big Data & Analytics Customer Case Studies

Constant Contact gains marketing insight from behavior pattern analysis

Need

• Improve their business value proposition and differentiate themselves against other companies that provide email marketing services

Benefits

• IBM is only vendor providing big data platform with integrated text analytics capability

• Constant Contact’s combined analysis of time, date, recipient email addresses and email content resulted in up to 25% increase in email opening rate.

30

Life Sciences Big Data & Analytics Customer Case Studies

Link to the case study

http://public.dhe.ibm.com/common/ssi/ecm/en/imc14800usen/IMC14800USEN.PDF

North American Pharmaceutical leader achieves greater transparency, agility and productivity in R&D through secure access to huge volumes of valuable enterprise content

Need

• Executives wanted to transform information and data sharing capabilities by providing access to huge volumes of customer, patient, and research data stored in systems around the world

• Benefits

• Improved R&D efficiency by 90 percent and reduced search time by 50 percent, saving millions of dollars in the first year alone

• Reduced new staffing requirements by 1.2 percent, saving USD13.4 million yearly

• Improved sales productivity to increase revenue by 1.4 percent

32 32

Thank you

33