Big Data in REAL TIMERon Zavner
We’re Living in a Real Time World…Homeland Security
Real Time Search
Social
eCommerce
User Tracking & Engagement
Financial Services
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved2
The Flavors of Big Data Analytics
Counting Correlating Research
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved3
It takes a week for users to
send 1 billion tweets
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved4
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
On average,
140 milliontweets get sent every day
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved5
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
The highest throughput to date is
6,939 tweets/sec.® Copyright 2011 Gigaspaces Ltd. All Rights Reserved6
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
460,000 new accounts are created daily
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved7
Twitter in Numbers (March 2011)
Source: http://blog.twitter.com/2011/03/numbers.html
Challenge – Word Count
Word:Count
Tweets
CountCount??® Copyright 2011 Gigaspaces Ltd. All Rights Reserved8
Thousands of tweets per second to process Aggregate counters for each word Latency – less than a second System needs to linearly scale System needs to be fault tolerant Querying & Persisting Data Managing the system
Analyze the Problem
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved9
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved10
Tier Based Architecture?
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved11
Data Grid
1. Just2. Found3. Love
® Copyright 2011 Gigaspaces Ltd. All Rights Reserved13
The 3 Most Popular Words on Twitter?
- August 2012
Recommended
