Embed Size (px)
DESCRIPTION
Real time analytics on big data applications, what is real time, what are the challenges to achieve real time complex calculation on big data and how they can be solved.
Citation preview
Big Data in REAL TIMERon Zavner
We’re Living in a Real Time World…Homeland Security
Real Time Search
Social
eCommerce
User Tracking & Engagement
Financial Services
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved2
The Flavors of Big Data Analytics
Counting Correlating Research
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved3
It takes a week for users to
send 1 billion tweets
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved4
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
On average,
140 milliontweets get sent every day
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved5
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
The highest throughput to date is
6,939 tweets/sec.® Copyright 2011 Gigaspaces Ltd. All Rights Reserved6
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
460,000 new accounts are created daily
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved7
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
Challenge – Word Count
Word:Count
Tweets
CountCount??® Copyright 2011 Gigaspaces Ltd. All Rights Reserved8
Thousands of tweets per second to process Aggregate counters for each word Latency – less than a second System needs to linearly scale System needs to be fault tolerant Querying & Persisting Data Managing the system
Analyze the Problem
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved9
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved10
Tier Based Architecture?
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved11
Data Grid
1. Just2. Found3. Love
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved13
The 3 Most Popular Words on Twitter?
- August 2012
