Embed Size (px)
DESCRIPTION
Gayatri Patel, eBay, presents at the Big Analytics 2012 Roadshow The wonders of what data can do for an organization is measured in the productivity and competitiveness of their team's decisions. Some believe more data is the key. Agreed...but good decisions require more than just deriving intelligence from big data. In this dynamic market, the need to socialize and evolve ideas with other teams, quickly correlate information across sources, and test ideas to fail fast early are strong enablers to gain competitive footing. eBay¹s analytic and technology advancements garners insights and approaches that continue to help our employees tell their "data stories" and make better decisions.
Citation preview
Extreme Decision-Making at eBay
Gayatri Patel
We All Make Decisions…
Our Decision Makers
“I love to connect
the dots to find out what is
driving our growth
and why!” “I love to monitor
the health of our global
operations with
confidence and tell great
stories to our BoD”
“I love to find unique anomalies that have not been discovered yet!”
Is Customer Brand improving with last
months’ new Motors campaign from our
Motors Mobile Application?
What is the latest revenue projection for our coupon campaigns
targeted to enhance Fashion Search for
large retailer listings?
Business Marketing Manager
5
Source: McKinsey Global Institute “Big data: The next frontier for innovation, competition, and productivity”
What Makes it Happen?
Lots & Lots & Lots of Data at eBay
BIG Data VVC
9
>50 TB/day new data
>100 PB/day >100 Trillion pairs of information
Millions of queries/day
>7500 business users & analysts
>50k chains of logic
24x7x365 99.98+% Availability
turning over a TB every second Active/Active
Near-Real-time
>100k data elements
Always online
Processed
Who Makes it Happen with Extreme Analytics at eBay?
“I love to connect the dots to find
out what is driving
our growth
and why!”
“I love to monitor
the health of our global
operations with
confidence and tell great
stories to our BoD”
“I love to find unique anomalies that have not been discovered yet!”
Analyst $$
Management & Executives
$$$$
PM & BM $$$
Researcher $
Decision Makers…
IT
Supply Chain Legal
Stra
tegy
Shipping
Search
Customer Support
Procurement
Corporate
Finance
Marketing
HR
Operations
Fraud
Capacity Planning
Facilities
Independent Data Marts = Silo Intelligence…
Semi-structured
SQL++
Structured
SQL
Low End Enterprise-class System
Contextual-Complex Analytics Deep, Seasonal, Consumable Data Sets
Production Data Warehousing Large Concurrent User-base
Discover & Explore Analyze & Report
150+ concurrent users
500+ concurrent users
Enterprise-class System
5-10 concurrent users
Unstructured
JAVA / C
Structure the Unstructured Detect Patterns
Commodity Hardware System
6+PB 40+PB 20+PB
Singularity Hadoop EDW
Data Storage & Processing Platforms
There’s No Technology Silver Bullet
Opportunistic Decisions
Enlightened Reasonings
Informative Measurements
Actionable Recommendations
What is happening?*
Why is it happening?*
How to take action?
• Trust • Quality
• Timely • Context
• Iterate…Test-What If…Collaborate-Share
Key Questions to Make Informed Decisions
Harmony Symphony Melody
Opportunistic Decisions
Informative Measurements
Harmony-Class Insights
50,000 ft
• Cross-Function Metrics • Business Performance
Summaries • Personalize views: • Dimensions • Attributes
• Deep Dive to Summarized Data
Harmony Health – Finance & Marketing Harmony Vertical (Customer/Inventory)
Harmony Trading & Trust Harmony Search & Behavior
Explore Cube
Quick…Easy…”Personalizable”…”Business-blessed”
Opportunistic Decisions
Informative Measurements
Harmony Verticals Harmony Trading
Enlightened Reasonings
Symphony-class Insights 10,000 ft
• Product Performance • Drill Downs • What-If/Ad-Hoc
Symphony Shipping
Symphony Transaction
Explore Cube Explore Cube
Symphony Enthusiast
Symphony Fashion
Harmony-class Insights
50,000 ft
Quick…Easy…”Personalizable”…”Product-blessed”
The DataHub Portal
• Dashboards • Reports • Views • Metrics • Measurements
Front End Analytic Assets
• Summary Tables • Aggregate Tables • In-Memory Caches • Data Feeds • Data Files • Base Tables
Back End Analytic Assets
Customer Inventory Trading Search Data
Enabling Services
ODW
EDW Singularity Hadoop
… …
Analytic Assets
Trust Product
Customer Product
Inventory Product
Trading Product
Search Product
Health Product
Behavior Product
Symphony Back-End
Symphony Front End
(1-3)
Harmony Front End
(4 only)
Common Dimensional Model
Insights Products
Q1 Q2 Q3 Q4 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec
Health Insights
Trading Insights
Trust Insights
…
Harmony Health v3.0
Harmony Health v3.2 Harmony Health v3.4
Symphony v3.3 Symphony v3.5
Harmony Health v3.6
Symphony v3.7 Symphony v3.9
Harmony Trading
v1.0 Harmony Trading v1.2 Harmony Trading v1.4
Symphony v1.3 Symphony v1.5
Harmony Trading v1.6
Symphony 1.7 Symphony v1.17
Symphony v1.3 Symphony v1.5 Symphony v1.7 Symphony v1.9 Symphony v0.9
Symphony v0.11
Symphony 1.9
Symphony v1.11 Symphony v1.15
Symphony v1.13
3.0 4.0
2.0 1.0
Product Roadmap
• Extends value over time • Adapt & make available quickly • Continuous quality
Extreme Analytics at eBay
Advanced Analytic Infrastructures
Designing for the Unknown
>85% of eBay analytical workload is NEW & Unknown The metrics you know are cheap The metrics you don’t know are expensive – but high in
potential ROI Exploration & Testing are core pillars of an analytics-
driven organization 22
22
23
More Tests
Better Tests
Better Decisions
Ownership & Confidence
Hypothesis Algorithm
Ideas Pattern Guesses
Assumptions
Behavioral
Attribution
Advanced
Simulation
Machine Learning
Experimentation: Build From Idea Up
Platform Metrics For Sample Queries
Rapid-fire BI & Exploration
Purpose Built Applications
Desktop Self-service
Executives & Management PM & BU Managers
Analysts * Scientists & Researchers
Engineers
Executives & Management PM & BU Managers *
Analysts Scientists & Researchers
Engineers
Executives & Management PM & BU Managers
Engineers *
Enterprise BI Statistical Modeling
Analysts Scientists & Researchers *
* Primary authoring group
Business Centric Technology Centric
Executives & Management PM & BU Managers
Analysts Engineers *
BI Tools & Platforms
Self Service Analytics
30
