Upload
abigail-puckett
View
29
Download
1
Tags:
Embed Size (px)
DESCRIPTION
BIG DATA , We have a communication problem. GINORMOUS SYSTEMS April 30–May 1, 2013 Washington, D.C. Daniel Tunkelang Head of Query Understanding, LinkedIn. BIG DATA IS EVERYWHERE. BIG DATA POWERS EVERYTHING. DATA SCIENTISTS WORRY ABOUT VOLUME, VELOCITY, VARIETY, …. BUT THE BOTTLENECK - PowerPoint PPT Presentation
Citation preview
BIG DATA,We have a communication problem.
GINORMOUS SYSTEMSApril 30–May 1, 2013Washington, D.C.
Daniel TunkelangHead of Query Understanding, LinkedIn
BIG DATA IS EVERYWHERE
BIG DATA POWERS EVERYTHING
DATA SCIENTISTS WORRY ABOUTVOLUME, VELOCITY, VARIETY, …
BUT THE BOTTLENECKISN’T COMPUTATIONAL
IT’S COGNITIVE
TOOLS AUGMENTHUMAN INTELLECT
BIG DATA IS A TOOL
Doug Engelbart, inventor ofthe mouse, hypertext, etc.
NOT EVERYONE SUBSCRIBESTO THIS POINT OF VIEW…
Claudia Perlich, Chief Scientist of media6degrees, speaking atTTI/Vanguard 2012 Conference on Understanding Understanding:
SHE HAS A POINT
BUT PREDICTIVE MODELINGIS NOT ENOUGH
TRAININGDATA?
OBJECTIVEFUNCTION?
WE NEED APEOPLE-CENTRICAPPROACH TOBIG DATA
INTERPRETABILITYINTERACTION
INSIGHT
LET’S START WITHINTERPRETABILITY
EXAMPLE:SVMvs.
DECISION TREE
DECISION TREES HAVE FLAWS…
DISCRETE
BUT THEYCOMMUNICATE
(if they’re shallow)
early splits provide big picture…
fat leaves guidefeature engineering
…or reveal training data problems
WHI
CHSUPPORTS
ITERATION
INTERPRETABILITY DELIVERS
Key search leader favors rule-based approach for key scoring algorithms.
Replaced regression with decision tree in local search model: gained accuracy and insight.
Using trees to recognize spam, analyze search abandonment, model / quantify social proof.
GO DEEP vs INTERPRETABILITY
A KEY DATA SCIENCE TRADE-OFF
ON TOINTERACTION
DON’T OVERPAY FOR PRECISION
BE FAST, CHEAP, AND 98% RIGHT
http://metamarkets.com/2012/fast-cheap-and-98-right-cardinality-estimation-for-big-data/
ARE PEOPLE THAT IMPATIENT?
tolerable wait time for web users
0.1s increase in latency significantly reduces # of searches, ad revenue
tl;dr: YES
IMPATIENCE IS GOODSPEED MATTERS
INSIGHT
http://blog.takejune.com/archives/52334044.html
BE TRENDY AND NORMALIZE
vs
Sept. 11thAbu Ghraib
Weapons Inspectors
SOLVE FOR INTERESTINGNESS
COMPUTE POTENTIAL INSIGHTS
APPLY HUMAN INTUITION
SUMMARY: Let’s have a conversation with Big Data.
INTERPRETABILITYINTERACTION
INSIGHT