Embed Size (px)
Citation preview
Antonio Alvarez | EMEA BDM for Big Data E-mail: [email protected] @A_AlvarezGarcia
AWS Big Data Pla-orm
Better Visibility of Your Business
Big Data & The Cloud
BIG DATA Platform:
Big Data Challenges:
Capacity Planning & Scalability
Lower Cost, OpEx
Experiment & learn more
Advanced profiles
IT Complexity
Data Variety…
..Volume, velocity Old Answers
& Questions
Managed Services
Fully managed,
secured & automated
services that brings agility &
focus
S3, EMR, Kinesis, Redshift,
DynamoDB:
Collect all data, do Complex
computations and processing it, both in Real-Time &
Batch
Sensors (IoT)
Social
Images
Videos
E. Apps.
Documents
Web Logs
Big Value
Machine Learning
Easy deployment of ML powerful models without the need of ML Experts ready to
be used
Virtually unlimited &
Elastic Resources
No heavy lifting & Reduced Time to Market, parallel processing on
demand
New Answers/questions &
Business Ideas Extract the
meaning from all your data & focus on new business
Ideas, Models, etc..
High Cost & Commitment
IT Challenges: SLAs, Sa;sfac;on, low u;liza;on (all?)
Massively Parallel Processing (on demand)
ON A SINGLE INSTANCE
COST: 4h x $2.1 = $8.4 RENDERING TIME: 4h
ON MULTIPLE INSTANCES
COST: 4 x 1h x $2.1 = $8.4 RENDERING TIME:
Expand to 25 instances
EMR (Steady State)
EMR (Batch Processing)
Shrink to 9 instances
EMR (Steady State)
On and Off Fast Growth
Unpredictable peaks Predictable peaks
USAGE PATTERNS: Flexibility and Agility
Fixed!
Some References
netflix
More than 25 Million Streaming Members
50 Billion Events Per Day
~10 PB of data stored in Amazon S3
S3
Data consumed in mul;ple ways
S3
EMR
Prod Cluster (EMR)
Recommenda;on Engine
Ad-‐hoc Analysis Personaliza;on
EMR
S3EMR
EMR
Prod Cluster (EMR)
Query Cluster (EMR)
EMR
EMR
Enterprise DWH
AWS Redshi; helped FT to increase performance (98% faster queries), reduce TCO (80%) and increase Agility
500,000 WRITES PER SECOND DURING SUPER BOWL
FINRA is moving its platform to the AWS Big Data Platform (AWS)
Finra: Financial Industry Regulatory Authority
• Stores and anlyses: 30B Market events per Day
• $10 to $20M annual Savings (Estimations)
• They have increase their Agility, Speed and Cost savings to operate at scale
hVp://aws.amazon.com/solu;ons/case-‐studies/finra/
How Much could this cost me? i.e. Real-time Analysis scenario
500MM tweets/day = ~ 5,800 tweets/sec
Kinesis (Ingestion) cost is $0.765/hour
Redshift (DWH) cost is $0.850/hour (for a 2TB node)
S3 (Data Lake) cost is $1.28/hour (no compression)
Total: $2.895/hour
Cost & Scale
Thank you
Contact information: Antonio Alvarez EMEA BDM for Databases & Big Data E-mail: [email protected] @A_AlvarezGarcia
