Trends in Big Data

René Kuipers

Principal Consultant Big Data & Analytics

@rjlkuipers

Overview

• Wat is it (or not) ?

• What to do with it ?

• How to deal with it ?

• Impact on IT?

Big Data

Wat is it (or not)

Big data: what is it (or not)

Big data: what is it (or not)

• Social data.

• Sensor data.

• unstructured data.

• A hype

• Hadoop

• Real-time

• ………………

• The four V's: Volume, Variety, Velocity, Value

Big data: what is it (or not)

Wat exactly is the trend ?

• “a lot of data” isn’t new:

• Governments

• Stock exchange.

• Real-time isn’t new:

– Railway management

• Level of detail increases

• Data is no longer in possession

• Open Data intiatives.

Example 1:

on-board sensor data

• America’s Cup 2013

• On-board sensors

• Off-board sensors

• 30.000 dp/s

• Oracle Team USA won 9-8 after 8-1 down, because of ‘insight’.

Example 2:

medical

• personalized healthcare

• n=1 treatment.

• Bring down cohort-size by

means of more personal

data.

Example 3:

scientific

• CERN (LHC)

• 120.000 sensors

• 4 GB data/sec

Example 4:

online ads

• Real-time profiling

• Context search

• Display ad

• Billing

Big Data Social

Cloud Mobile

Een unity of 4

The value of data

Value of data

• The higher up in an

organization, the lower the

value of an individual

datapoint.

• The lower (more operational)

in an organization, the higher

the value of an individual

datapoint.

• Big data offers more (real-

time) insight on

operational level than on

strategic level.

• Big Data is mainly a CX ‘thing’.

• 360-degree view of

• Patient

• Customer

• Supplier

• …

• …

Organization Technical

Implications

– Outward.

– Openness, transparency

– Extreme short communication (outward)

– Customer focus, short reactiontimes.

Implications

Organization

Customerfocus#klm #fail

Organization Technical

Implicaties

How to embed Big Data into traditional DWH/BI ?

How to embed Big Data into traditional DWH/BI ?

How to embed Big Data into traditional DWH/BI ?

DWH and BI

changes

In-house

data

External

data

– Storage of Big Data: Hadoop

– Storage of ‘traditional’ data: RDBMS

– necessity: transparant access.

– necessity: high-end usertools: Self Service BI.

– necessity: fast back-end.

Datawarehouse, Big Data and Business Intelligence

Future

More data More

questions

Faster

answers

In-memory

– Big Data

– Large Volume (Volume)

– Fast (Velocity)

– In different shapes and sizes (Variety)

– Huge information potential (Value)

– Data grows exponentially

– Data is extern

– Data must lead to faster insights

– Fundamentallly datawarehouse redesign

– In-memory

– Demands organizational changes

Conclusions