Heise Developer World 2016 - Big Data ist tot, es lebe Business Intelligenz

Page 1Glomex GmbH – A ProSiebenSat.1 Media SE company

Dr. rer. nat. Markus Schmidberger, Data Platform Architect

@cloudHPC

Big Data ist tot – Es lebe Business Intelligenz?

CeBIT 2015


Gartner Hype Cycle 2015

Where is Big Data in 2015 / 2016 ?

2014

2013

2012

2011?


Glomex: A ProSiebenSat.1 company

No.1 Ad sales house in GSA

~2bn video views per year

50+ online & mobile platforms

20 years of technical excellence

Where We Come From

No.1 TV broadcaster in Germany


Networking effects – The power of unifying many big players in one platform

TOTAL COST OF OWNERSHIP

CONTENT PRODUCTION RISK

REACH

INNOVATION COST

eCPM

Multi-tenant traffic pooling

International syndication

CAPEX pooling

Aggregated content performance data

Hyper-targeted advertising


Glomex – The Global Media Exchange

Publishers

Content providers

Video Value Platform

Media Delivery Platform

Media Exchange Platform

Glomex

External broadcasters

Web-only content owners

Non-P7S1 publishers


CDN OperationsCDN

Operations

Glomex: The Global Media Exchange

Streaming Infrastructure

CDN Operations

Client / Player

DescriptivePrediction

Operations

User Monitoring

TechnicalVideo Streaming Experience

Analyze

Automation


Data Platform Vision

CDN Video-Player (Web, App)

Media Asset Management Transcoding

Video Platform

Data Platform

User-generated Data System-generated Data

Analytics Billing

Monitoring Content Discovery


Strategic Profile – Data PlatformLe

vel o

f Com

pete

nce

BI Big Data

BI & Big Data

Machine LearningScalabilityReal-time

Report CentricComplete Value Chain

OLAP


Data Platform - Architecture

Cloud-Based ArchitectureAmazon Web Services

Modern Analytics Workflows and Machine

Learning Methods

Modern Data ArchitectureMicroServices


Amazon Web Services

#1: Agility

#3: Continual Iteration and Innovation

#2: Platform Breadth

#4: Cost Savings and Flexibility


Big Data World

Amazon Glacier

S3 DynamoDB

RDS

EMR

Amazon Redshift

Data PipelineAmazon Kinesis Cassandra

CloudSearch

Kinesis-enabled

app

Lambda ML

SQS

ElastiCache

DynamoDBStreams


ingest /collect

store process /analyze

consume / visualize

Simplify Data Processing

data answers

Time to Answer (Latency)Throughput

Cost


Collect Store Analyze Consume

A

iOS Android

Web Apps

Logstash

Amazon RDS

Amazon DynamoDB

AmazonES

AmazonS3

ApacheKafka

AmazonGlacier

AmazonKinesis

AmazonDynamoDB

Amazon Redshift

Impala

Pig

Amazon ML

Streaming

AmazonKinesis

AWSLambda

Amaz

on E

last

ic

Map

Redu

ce

AmazonElastiCache

Sear

ch

SQL

NoS

QL

Ca

che

Stre

am

Proc

essi

ngBa

tch

Inte

ract

ive

Logg

ing

Stre

am

Stor

age

IoT

Appl

icat

ions

File

St

orag

e

Anal

ysis

& V

isua

lizat

ion

Hot

Cold

Warm

Hot

Slow

HotM

L

Fast

Fast

Amazon QuickSight

Transactional Data

File Data

Stream Data

Not

eboo

ks

Predictions

Apps & APIs

Mobile Apps

IDE

Search Data

ETL


MicroServices


Lambda Architecture

Graphic provided by http://soutier.de/


Lambda Architecture


Summary

Lambda Architecture

• Combines batch & real-time

• Enables BI & Big Data

AWS Cloud

• Enables agility

• Makes us fast and scalable

Team

• We are all engineers

• Cross-functionality provides new insights

BI & Big Data

• Enables new business insights

• Makes us faster


Dr. rer. nat. Markus Schmidberger, Data Platform Architect

@cloudHPC

Big Data ist tot – Es lebe Business Intelligenz?

CeBIT 2015

We are hiring …

•Data Scientists

•Data Engineers

•Project Managers

Science

Heise Developer World 2016 - Big Data ist tot, es lebe Business Intelligenz