26
Jarut N. Cisco Systems Data. Does it Matter?

Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Jarut N.

Cisco Systems

Data. Does it Matter?

Page 2: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera
Page 3: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

AutomotiveAuto sensors reporting location, problems

CommunicationsLocation-based advertising

ConsumerPackaged GoodsSentiment analysis of what’s hot, problems

Financial ServicesRisk and portfolio analysis New products

Education and ResearchExperiment sensor analysis

Competitive success depends on Data and Analytics

Data & Analytics are Top of Mind in Every IndustryHigh Technology/ Industrial Mfg.Mfg. quality warranty analysis

Life SciencesClinical trialsGenomics

Media/EntertainmentViewers/advertising effectiveness

On-Line Services/ Social MediaPeople and career matchingWeb-site optimization

Health CarePatient sensors, monitoring, EHRsQuality of care

Oil and GasDrilling exploration sensor analysis

RetailConsumer sentimentOptimized marketing

Consumer Travel andTransportationSensor analysis for optimaltraffic flowsCustomer sentiment

UtilitiesSmart-meter analysis for network capacity,

Law Enforcement and DefenseThreat analysis - social media monitoring, photo analysis

Page 4: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Data Management and Agility

• Analyzed service sales opportunities in one-tenth the time, at one-tenth the cost

• $40 million in incremental service bookings in the current fiscal year as a result of this initiative

• Implemented a multitenant enterprise platform while delivering immediate business value

CHALLENGES

SOLUTION

INSIGHTS AND IMPACT• Unlock the business value of large data sets• Provide SLAs for internal customers using Big

Data analytics services• Support multiple internal users on same platform

• Implemented enterprise Hadoop platform on Cisco Integrated Infrastructure for Big Data

• Automated job scheduling and process orchestration using Cisco Tidal Enterprise Scheduler

3.8 PETABYES Under 1 management domain!

Page 5: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Big Data/AnalyticsData Management

Data Virtualization / Integration Analytics / Business Intelligence

Page 6: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

A Holistic View of a Big Data System

ETL

Real-TimeStreams

Unstructured Data (HDFS)

Real-Time StructuredDatabase

(hBase, Gemfire, Cassandra)

Big SQL(Greenplum,AsterData,

Etc…)

BatchProcessing

Real-TimeProcessing

Analytics

Page 7: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

§ Big Data is not just a technology. It is a business strategy for capitalizing on information resources.

How Do We Define Big Data?

Page 8: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Volume

of Tweets created daily.

12+ terabytes

More than just Data Warehousing: Big Data’s Value Is in the Analytics

Page 9: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Volume

of Tweets created daily.

12+ terabytes

Variety

of different types of data.

100s

More than just Data Warehousing: Big Data’s Value Is in the Analytics

Page 10: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Volume

of Tweets created daily.

12+ terabytes

Variety

of different types of data.

100s

trade eventsper second.

5+ million

Velocity

More than just Data Warehousing: Big Data’s Value Is in the Analytics

Page 11: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Volume

of Tweets created daily.

12+ terabytes

Variety

of different types of data.

100sVeracity

decision makers trust their information.

Only 1 in 3

More than just Data Warehousing: Big Data’s Value Is in the Analytics

trade eventsper second.

5+ million

Velocity

Page 12: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Volume

of Tweets created daily.

12+ terabytes

Variety

of different types of data.

100sVeracity

decision makers trust their information.

Only 1 in 3

trade eventsper second.

5+ million

Velocity

More than just Data Warehousing: Big Data’s Value Is in the Analytics

It is all about better analytics on a broader spectrum ofdata, and therefore represents an opportunity to create

even more differentiation among industry peers.

Page 13: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Data Warehousing

Data warehousing is the process of centralizing or aggregating data from multiple sources into one common repository.

Example Use Cases: Facebook and Retail

Page 14: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Data Virtualization

…is an approach to data management that allows an application to retrieve and manipulate data without requiring technical details about the data, such as how it is formatted or where it is physically located.

Example: Google

Page 15: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

BI/Analytics

Data Warehouse (DW)

Data Warehouse Optimization with Data Virtualization

Machine Device, Cloud

Documents and Emails

Relational, Mainframe

Social Media, Web Logs

Data Sources

HDFS HDFSHDFS

Server

Deliver Enriched BI/AnalyticsVirtualize data from DW and Hadoop

Deliver richer & deeper data for Analytics

Optimize Storage / CostsMigrate infrequently used data to Hadoop

Process Unstructured, New Sources in Hadoop

Offload ELT & Virtually Expand the DW

Page 16: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Data Analytics and Business Intelligence

§ …is a process of inspecting, cleaning, transforming, and modeling data with the goal of discovering useful information, suggesting conclusions, and supporting decision-making.

§ Business intelligence covers data analysis that relies heavily on aggregation, focusing on business information.

Example Use Case: Retail –Inventory Accuracy

Page 17: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Big Data vs. Data Analytics

§ Big Data:§ Save money:

§ Cost effective scale§ Consolidate many types of data§ Operational efficiencies

§ Data Analytics:§ New ways to make money:

§ Predict, understand, and monetize customer behavior

§ Fact-based decision making§ Enable real-time tactical decisions

§ Also save money: § Increase security, reduce fraud,

predict failures

Page 18: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

§ Business Intelligence:§ Rear-view mirror§ Answers what happened? § Tactical/departmental/siloed§ Predefined and canned§ What did we sell, to who, and

how?

§ Business Analytics:§ Windshield and

beyond§ Answers what is next?§ Strategic/enterprise/holistic§ Discovery and ad hoc§ What can we sell and to who?

Business Intelligence or Business Analytics? What’s the Difference?

Page 19: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

12+ TBsof tweet data

every day

25+ TBs of log data every day

2.5 exabytes of data every day

30 billionRFID tags today

4.6 billioncamera phones

worldwide

100s of millions of

GPS enableddevices

sold annually

200 million smart meters

Source of the Data?

500 / Second Data sent per 4G LTE Enabled Car

Page 20: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

How the Data Is Sourced

Page 21: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Where the Data Is Sourced

Remote Branch

Branch Office

Co-Location XaaS Provider

Retail Outlet

Manufacturing

• Localized Processing• “Intercloud” Connectivity• Data Optimization• Data Virtualization

Head Office

What We Need:

Page 22: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

The Challenge

Growing Data Volumes

Shortened Processing Windows

Escalating Costs

Hitting Scalability Ceilings

Demanding Business

Requirements

ETL Complexity

Latency in Data

Tight IT Budgets

The Classic Enterprise Challenge…

…Do More with Less

Page 23: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Data Warehouses Cannot Cost-Effectively Support Data Growth

100% DATA GROWTH100TB 100TB 100TB

Data Warehouse$20,000 – $100,000/TB

To Add Capacity to the Data WarehouseIncremental spend of $2M – $10M

Keeps the Right Data in the Data Warehouse:• Operational Analytics• Reporting• Business Analytics

Offloading Everything Else to Big Data: Saves $1.85M – $9.8M• Historical Data• Data Processing• Data Hub/Ad Hoc Exploratory• Transformation/Batch

Today, growth is accommodated by additional investment in your data warehouse

Big Data complements your data warehouse, offloading data to defer/avoid more costly spend

100TB50TB

LOWER VALUE DATA

HIGH VALUE DATA 50TB100TB

Hadoop Cluster Cost $1000 –$2000/TBIncremental Cost $240K – $300K

Page 24: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Opportunity: Turning Big Data into Wisdom

Data-driven enterprises outperform their industry peers by up to 6%, are up to 26% more profitable*Gartner’s 2013 CIO survey: Analytics/business intelligence was the number one technology priority

WISDOM (Scenario Planning)

KNOWLEDGE

INFORMATION

DATA

• Collect more accurate performance data• Analyze variability, understand root causes

• Create highly specific customer segmentation• Tailor products and services• Anticipate requirements and outcomes

• Increase productivity (MIT, 2003)• Speed delivery of new innovations to market

MORE IMPORTANT

LESS IMPORTANT

*MIT, 2013

Page 25: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

§ Big Data is a large ecosystem comprised of over ISVs. Cisco views the market as centered around three major pillars, which are data management, data warehouse optimization and expansion, and analytics and BI.

§ Data can be virtualized, warehoused, optimized, mined, analyzed, or used to produce new data.

§ CEOs, the heads of business lines, board members, marketing and financial leaders, all may be the drivers of Big Data initiatives.

Summary

Page 26: Data. Does it Matter? - MonsoonSIM THAILAND · 2020-01-25 · every day 25+ TBs of log data every day 2.5 exabytes of data every day 30 billion RFID tags today 4.6 billion camera

Q & A