Embed Size (px)
DESCRIPTION
'Seeing Meaning in the age of Big Data' Probabilistic modeling and machine learning touches every industry. Extracting meaning from data allows better user interaction, finds patterns that would otherwise be obscured using traditional BI reporting, and leads to defensible decision making.
Citation preview
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Visualizing InferenceSeeing Meaning in the Age of Big Data
Alex Morrise
October 9, 2014
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
1 Big Data meets Machine Learning
2 Machine Learning Pros and ConsWhat is Business Value of ML?ML vs. BIML disclaimerPersonalization
3 Visualizing InferenceML InfancyCases Studies
4 Summary
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
“Machine Learning is the antidote to having towrite down billions of business rules”
Machine Learning Uses Include:User Personalization
Finding Predictors to a given Objective (Revenue,Churn, Volatility, Moods, Sentiment, Market Cap,Stocks, Biomedical, Risk Assessment, etc)Fraud Detection (Purchases, Identity Thief, HealthInsurance, Governments)Optimizing Decision Flows (Shipping, Markets,Robotics, Database Migrations, Team resourceallocation and management, etc...)
Key Take AwayEvery Industry is Touched by Machine Learning
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
“Machine Learning is the antidote to having towrite down billions of business rules”
Machine Learning Uses Include:User PersonalizationFinding Predictors to a given Objective (Revenue,Churn, Volatility, Moods, Sentiment, Market Cap,Stocks, Biomedical, Risk Assessment, etc)
Fraud Detection (Purchases, Identity Thief, HealthInsurance, Governments)Optimizing Decision Flows (Shipping, Markets,Robotics, Database Migrations, Team resourceallocation and management, etc...)
Key Take AwayEvery Industry is Touched by Machine Learning
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
“Machine Learning is the antidote to having towrite down billions of business rules”
Machine Learning Uses Include:User PersonalizationFinding Predictors to a given Objective (Revenue,Churn, Volatility, Moods, Sentiment, Market Cap,Stocks, Biomedical, Risk Assessment, etc)Fraud Detection (Purchases, Identity Thief, HealthInsurance, Governments)
Optimizing Decision Flows (Shipping, Markets,Robotics, Database Migrations, Team resourceallocation and management, etc...)
Key Take AwayEvery Industry is Touched by Machine Learning
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
“Machine Learning is the antidote to having towrite down billions of business rules”
Machine Learning Uses Include:User PersonalizationFinding Predictors to a given Objective (Revenue,Churn, Volatility, Moods, Sentiment, Market Cap,Stocks, Biomedical, Risk Assessment, etc)Fraud Detection (Purchases, Identity Thief, HealthInsurance, Governments)Optimizing Decision Flows (Shipping, Markets,Robotics, Database Migrations, Team resourceallocation and management, etc...)
Key Take AwayEvery Industry is Touched by Machine Learning
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
“Machine Learning is the antidote to having towrite down billions of business rules”
Machine Learning Uses Include:User PersonalizationFinding Predictors to a given Objective (Revenue,Churn, Volatility, Moods, Sentiment, Market Cap,Stocks, Biomedical, Risk Assessment, etc)Fraud Detection (Purchases, Identity Thief, HealthInsurance, Governments)Optimizing Decision Flows (Shipping, Markets,Robotics, Database Migrations, Team resourceallocation and management, etc...)
Key Take AwayEvery Industry is Touched by Machine Learning
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Companies are being inundated with data:
User Behavioral Data/Click Stream (purchases, views,engagement, interests)
Sensor Data (Internet of Things, smart meters,construction, management)B2B (SaaS management tools across all industries,Salesforce, etc)B2C (Uber, Netflix, google)
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Companies are being inundated with data:
User Behavioral Data/Click Stream (purchases, views,engagement, interests)Sensor Data (Internet of Things, smart meters,construction, management)
B2B (SaaS management tools across all industries,Salesforce, etc)B2C (Uber, Netflix, google)
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Companies are being inundated with data:
User Behavioral Data/Click Stream (purchases, views,engagement, interests)Sensor Data (Internet of Things, smart meters,construction, management)B2B (SaaS management tools across all industries,Salesforce, etc)
B2C (Uber, Netflix, google)
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Companies are being inundated with data:
User Behavioral Data/Click Stream (purchases, views,engagement, interests)Sensor Data (Internet of Things, smart meters,construction, management)B2B (SaaS management tools across all industries,Salesforce, etc)B2C (Uber, Netflix, google)
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Companies are being inundated with data. They all haveone thing in common.They want a way to capitalize on the real meaning behindthe data.
Bayesian methods allow the discovery of the latentproperties in data, while assessing our confidence inthe models certainty/ignorance.Hypothesis testing, parameter estimation, confidenceintervals, etc..
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Why do businesses need Machine Learning/Data Science?
They have plenty of data to train expert systems
Traditional BI may not be able to find the correctpatternsShifting focus from traditional 20th century businessobjectives, companies need to convert their valuepropositions into technological currency.Users & Businesses are sophisticated and wantintelligence in their applications
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Why do businesses need Machine Learning/Data Science?
They have plenty of data to train expert systemsTraditional BI may not be able to find the correctpatterns
Shifting focus from traditional 20th century businessobjectives, companies need to convert their valuepropositions into technological currency.Users & Businesses are sophisticated and wantintelligence in their applications
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Why do businesses need Machine Learning/Data Science?
They have plenty of data to train expert systemsTraditional BI may not be able to find the correctpatternsShifting focus from traditional 20th century businessobjectives, companies need to convert their valuepropositions into technological currency.
Users & Businesses are sophisticated and wantintelligence in their applications
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business value of MachineLearning and Data Science?
Why do businesses need Machine Learning/Data Science?
They have plenty of data to train expert systemsTraditional BI may not be able to find the correctpatternsShifting focus from traditional 20th century businessobjectives, companies need to convert their valuepropositions into technological currency.Users & Businesses are sophisticated and wantintelligence in their applications
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business Value of BI?
Traditional BI tools and methods (Tableaux, Splunk, etc), areamazing, sophisticated and potentially misleading
Example: Splitting demographic data into seeminglygood piles and running aggregating reporting overthose splits.
Wonderful if you want to know what women ages 24-26are purchasing in Los Angeles this month.What about Behavior?
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business Value of BI?
Traditional BI tools and methods (Tableaux, Splunk, etc), areamazing, sophisticated and potentially misleading
Example: Splitting demographic data into seeminglygood piles and running aggregating reporting overthose splits.Wonderful if you want to know what women ages 24-26are purchasing in Los Angeles this month.
What about Behavior?
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
What is the Business Value of BI?
Traditional BI tools and methods (Tableaux, Splunk, etc), areamazing, sophisticated and potentially misleading
Example: Splitting demographic data into seeminglygood piles and running aggregating reporting overthose splits.Wonderful if you want to know what women ages 24-26are purchasing in Los Angeles this month.What about Behavior?
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Businesses want Automatic Action
Finding Behavior in a Automatic Actionable WayFinding Behavior requires leveraging the power ofmachine learning to tease out the meaning behind theobservations in a crowd sourced way.
Bayesian methods such as Factorizations, HierarchicalClustering, and other Topic models, extract meaningfrom the data.Once model is fit, observations of Behavior connect toActions in the system (API), yielding an automaticintelligent way to process information and transactions.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Businesses want Automatic Action
Finding Behavior in a Automatic Actionable WayFinding Behavior requires leveraging the power ofmachine learning to tease out the meaning behind theobservations in a crowd sourced way.Bayesian methods such as Factorizations, HierarchicalClustering, and other Topic models, extract meaningfrom the data.
Once model is fit, observations of Behavior connect toActions in the system (API), yielding an automaticintelligent way to process information and transactions.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Businesses want Automatic Action
Finding Behavior in a Automatic Actionable WayFinding Behavior requires leveraging the power ofmachine learning to tease out the meaning behind theobservations in a crowd sourced way.Bayesian methods such as Factorizations, HierarchicalClustering, and other Topic models, extract meaningfrom the data.Once model is fit, observations of Behavior connect toActions in the system (API), yielding an automaticintelligent way to process information and transactions.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Disclaimer: Value of ML?
Let’s be Honest:ML can also lead to misleading results when used in thewrong hands:
Running a Decision Tree on demographic data couldlikely split the population by M/F right off the bat.
This split will lead to misleading results as it tries toexplaining the objective.Know your ML tool belt, practice makes perfect, andtreat the job as science (tests, validation, parametersearch, research, etc).Answer: Use a Random Forest instead.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Disclaimer: Value of ML?
Let’s be Honest:ML can also lead to misleading results when used in thewrong hands:
Running a Decision Tree on demographic data couldlikely split the population by M/F right off the bat.This split will lead to misleading results as it tries toexplaining the objective.
Know your ML tool belt, practice makes perfect, andtreat the job as science (tests, validation, parametersearch, research, etc).Answer: Use a Random Forest instead.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Disclaimer: Value of ML?
Let’s be Honest:ML can also lead to misleading results when used in thewrong hands:
Running a Decision Tree on demographic data couldlikely split the population by M/F right off the bat.This split will lead to misleading results as it tries toexplaining the objective.Know your ML tool belt, practice makes perfect, andtreat the job as science (tests, validation, parametersearch, research, etc).
Answer: Use a Random Forest instead.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Disclaimer: Value of ML?
Let’s be Honest:ML can also lead to misleading results when used in thewrong hands:
Running a Decision Tree on demographic data couldlikely split the population by M/F right off the bat.This split will lead to misleading results as it tries toexplaining the objective.Know your ML tool belt, practice makes perfect, andtreat the job as science (tests, validation, parametersearch, research, etc).Answer: Use a Random Forest instead.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
ML is BI
Example: The Retail VerticalBI reporting can be good for detecting aggregate trendsbut fails to personalize.
Personalization can find aggregate trends and solve thequestion,“What are the 3 shoes you are highly likely to engageand ultimately purchase, given your (sparse) purchasehistory, time of year, demographic information, etc”
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
ML is BI
Example: The Retail VerticalBI reporting can be good for detecting aggregate trendsbut fails to personalize.Personalization can find aggregate trends and solve thequestion,“What are the 3 shoes you are highly likely to engageand ultimately purchase, given your (sparse) purchasehistory, time of year, demographic information, etc”
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Intermission
We’ve stated use cases for MLBusiness will capitalize on including ML in their stack.Let’s move on to see how we see the meaning behindthe data?
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Machine Learning in it’s Infancy
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Visualizing Inference
To see the latent properties in your data, construct a GraphG as follows
Form your data matrix M
Factor it using your favorite algorithm, M = WHCluster in W and H t
Use the cluster assignments (or some similarity metricon factors) to make graph G.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Visualizing Inference
To see the latent properties in your data, construct a GraphG as follows
Form your data matrix MFactor it using your favorite algorithm, M = WH
Cluster in W and H t
Use the cluster assignments (or some similarity metricon factors) to make graph G.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Visualizing Inference
To see the latent properties in your data, construct a GraphG as follows
Form your data matrix MFactor it using your favorite algorithm, M = WHCluster in W and H t
Use the cluster assignments (or some similarity metricon factors) to make graph G.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Visualizing Inference
To see the latent properties in your data, construct a GraphG as follows
Form your data matrix MFactor it using your favorite algorithm, M = WHCluster in W and H t
Use the cluster assignments (or some similarity metricon factors) to make graph G.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
8tracks.com, the Best Music Service on PlanetEarth
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
8tracks.com
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
8tracks.com
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
8tracks.com
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Boomtrain.com
Boomtrain.com uses machine learning to inform decisions.Boomtrain offers an end to end solution using, in part, a realtime novel view into the user base of a given company. Bylearning
Users Proclivity to a set of TopicsUser ArchetypesUsers Derived Meta-Properties
Boomtrain.com exposes this knowledge in an actionableframework, allowing clients to drive engagement andretention.
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Boomtrain.com
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Quid.com (Assessing metrics in Technology andInnovation)IdleGames.com (Behavior as Predictor of Demographicand Monetization)BeatsMusic.com (Contextualized MusicRecommendation – Understanding the Heart of theMusic)
VisualizingInference
Alex Morrise
Outline
Big DatameetsMachineLearning
MachineLearning Prosand ConsWhat is BusinessValue of ML?
ML vs. BI
ML disclaimer
Personalization
VisualizingInferenceML Infancy
Cases Studies
Summary
Machine Learning is the Future
We are just at the onset of a radical transformation in theway we do, and see, everything
Every business is transforming into a technologycompanyThey all need intelligence powering their core offeringsFinding better ways to see the meaning behind the datawill drive each of those offerings.
