Embed Size (px)
Citation preview
Accounting for EverythingIoT, Big Data, and Analytics
Leif Ulstrup
[email protected], @lulstrup
www.linkedin.com/in/leifulstrup/
Self-Driving Cars - How Are They Evolving So Quickly?
Source: https://waymo.com/
Self-Driving Cars - How Are They Evolving So Quickly?
Source: https://www.tesla.com/
Self-Driving Cars - How Are They Evolving So Quickly?
Source: http://www.nvidia.com/object/drive-px.html
Why Is This A Major Milestone?
Why is a Computer Winning @ Poker Even More Interesting?
What Is Common Across These Stories?
Sensors (powerful, inexpensive, abundant)High Speed Communications
Connected DevicesBig Data (to Train Models)
Deep Learning (modern AI) ModelsGPUs
Cloud Computing (APIs)Open Source Software
Consumer Scale Technology (vs Enterprise)
Multiple Exponential Technologies Interacting
Topics ● Motivation
● Background and Context
● Exploration of Decision Making
● Building Models
● Applications to Business
Problems
● “Prediction as a Service”
● What’s Next?
Motivation - Why So Much Attention on Analytics?● Next frontier in productivity improvement is in automating decisions
● C-level executives realizing “digital exhaust” can be an asset
○ Reduce operating costs by optimizing
○ Revenue growth via new customers and new lines of service
● Business research showing analytics-savvy businesses yield greater profits and
growth
● The cost of building “prediction systems” declining
● The rapid rate of quality improvements in “human-like” prediction systems
● The explosion in sensor and log data and low cost to store it
● Disruptive startup platform business models harnessing “big data”
○ Uber, AirBnB, Facebook, Twitter, …
● Fear of missing out (FOMO), Fear of disruption
Background and Context● Digitization of business and consumer activities growing rapidly
● Log files recording actions behind the scenes
● Preferences and “likes” being recorded
● Smart physical objects with low-cost sensors -> Internet of Things (IoT)
● Open Source software, hardware, and frameworks
● Open Collaboration platforms and open problem solving
● Rapid advances in “machine learning” & “deep learning” via APIs and open source
● Cloud technology for unlimited storage and compute @ low cost
Data and Computing Abundance!
Management 101
“You can’t
manage what
you can’t
measure.”
Peter
Drucker
W. Edwards
Deming
Analytics (via Wikipedia)
“...is the discovery,
interpretation, and
communication of
meaningful patterns in
data…”
Key Influencers of C-level Execs - Big Data and Analytics
Tom Davenport
HBS
Erik Brynjolfsson
MIT Sloan
Andrew McAfee
MIT Sloan
Other Influential Books
Big Data...https://en.wikipedia.org/w
iki/Big_data
- Volume
- Variety
- Velocity
- Veracity
- ...
What Is the Most Fundamental Task of a
Manager?
Digital Age ‘Microscope’“...Data measurement, Professor Brynjolfsson
explains, is the modern equivalent of the
microscope. Google searches, Facebook posts
and Twitter messages, for example, make it
possible to measure behavior and sentiment in
fine detail and as it happens…”
Erik Brynjolfsson, The Age of Big Data, NY
Times, Steve Lohr, 2/11/2012
Instrumenting the World, “Accounting for Everything”
“Instrumentation” of an Enterprise
What does that mean?
How do you do that?
It Is Already Happening - Example - Computer Log Filedefault 09:09:20.186536 -0500 assistant_service <private> can sync <private>
default 09:09:20.186583 -0500 assistant_service staging at <private>
default 09:09:20.186655 -0500 assistant_service No latest vocabulary file
default 09:09:20.188274 -0500 assistant_service No file at <private>
default 09:09:20.188438 -0500 assistant_service finalAnchor = '<private>' cleaning up=1
default 09:09:20.188488 -0500 assistant_service Saving latest vocab <private> to <private>
default 09:09:20.214834 -0500 assistant_service Removing staged vocabulary at <private>
default 09:09:21.688082 -0500 Dock notifyBestAppChanged:(null) UASuggestedActionType=0 <private>/<private> opts=(null) when=2017-02-03
14:09:21 +0000 confidence=0 from=<private>/<private>
default 09:09:22.689751 -0500 lsd CSStore: <private>
default 09:09:22.701678 -0500 lsd _LSSessionSave: result = 0 saving database for UID 501
default 09:09:22.710286 -0500 gamed GKClientProxy: clientForBundleID:
default 09:09:22.758054 -0500 gamed GKClientProxy: updateIfRecentlyInstalled
default 09:09:22.776365 -0500 gamed GKCacheTransactionGroup: transactionGroupWithContext
default 09:09:22.776435 -0500 gamed GKCacheTransactionGroup: initWithName:
default 09:09:22.776558 -0500 gamed GKCacheTransactionGroup: performOnManagedObjectContext
default 09:09:22.793980 -0500 gamed GKCacheObject: fetchRequestForContext
Example - Smartthings (Samsung) Home Automation Sensor
Date Source Type Name Value User Displayed Text
2017-02-03 6:24:47.399 AM EST3 hours ago
DEVICE temperature 45 Water Leak Sensor was 45°F
2017-02-03 6:09:52.763 AM EST3 hours ago
DEVICE temperature 46 Water Leak Sensor was 46°F
2017-02-03 5:35:04.211 AM EST4 hours ago
DEVICE temperature 45 Water Leak Sensor was 45°F
2017-02-03 5:10:12.434 AM EST4 hours ago
DEVICE temperature 46 Water Leak Sensor was 46°F
2017-02-03 4:35:23.918 AM EST5 hours ago
DEVICE temperature 45 Water Leak Sensor was 45°F
2017-02-03 1:16:28.745 AM EST8 hours ago
DEVICE temperature 46 Water Leak Sensor was 46°F
2017-02-03 12:56:34.968 AM EST8 hours ago
DEVICE temperature 47 Water Leak Sensor was 47°F
2017-02-03 12:26:44.748 AM EST9 hours ago
DEVICE temperature 46 Water Leak Sensor was 46°F
Example - Technology Focused on Mining Log Data
https://www.splunk.com/
Heavily used in cybersecurity analytics...
Logging Everything -> “Digital Exhaust” -> Cloud
Internet of Things (IoT) - converting the physical world to a digital representation and logging everything in the cloud
Link and @GilPress
Have One Of These Yet?
Source: https://madeby.google.com/home/Source: https://www.amazon.com/Amazon-Echo-Bluetooth-Speaker-with-WiFi-Alexa/dp/B00X4WHP5E
Personal Assistants - Anticipating Your Needs, Responding to Your VoiceGoogle Now https://www.workfit.com/about-us/
What are the Implications Inside Industry and Government?
How Do Managers Make Critical Decisions
in Real Life?
How Should a Manager Decide?
Gut Instinct Analytical
Techniques
Can Both Intuition and Analytical Decisions Be Automated?
Classic Examples of Automated Business Decisions● Loan Approval Decisions
● Payment Collections “Treatment” Strategies
● Credit Card Offer Mailings with Custom Terms Tailored to
the Individual Consumer’s Risk Score (prediction)
● Fraud Detection
● ...others you can think of?
Davenport’s Taxonomy of Analytics❏ Descriptive Analytics
“Mean Revenue/Customer”
❏ Predictive Analytics
“92% probability with high confidence level that customer X will switch at
contract renewal date”
❏ Prescriptive Analytics (“automated policies”)
“If new customer fails to make payment within 5 days after due date, talk to
customer on the phone to arrange payment commitment”
Simulation & Modeling (via Wikipedia)
“...process of creating and analyzing
a digital prototype of a physical
system to predict its performance in
the real world…”
Human Frailties - Affecting Perception and DecisionsThinking, Fast and Slow
Daniel Kahneman
Loss Aversion (Wikipedia)
loss aversion refers to people's tendency to prefer avoiding losses to acquiring equivalent gains: it's better to not lose $5 than to find $5. Some studies have suggested that losses are twice as powerful, psychologically, as gains.[1] Loss aversion was first demonstrated by Amos Tversky and Daniel Kahneman.
What other cognitive biases have you heard about?
Cognitive Biaseshttps://en.wikipedia.org/wiki/List_of_cogniti
ve_biases
- Hindsight bias
- Anchoring
- Framing
- “Hot hand” fallacy
- Loss aversion
- ...
Digital Models to Make Predictions and Improve Operations (examples)
● Operations
○ How many call center calls can we anticipate when we make this policy change?
○ Can we anticipate equipment failures and make repairs before the system fails?
○ How likely is this program to meet schedule and budget commitments?
● Sales
○ When will the deals in the pipeline close?
○ What will our win rate be with this pipeline?
○ Which customers are likely switch at renewal time?
○ Should we invest in a proposal for opportunity X?
● Marketing
○ How many and what quality will the sales leads we generate be from marketing program X?
● Human Resources
○ Will this potential hire be a good fit with our culture and be productive?
○ Which employees are most likely to leave and how soon?
○ Who are the best candidates to work on a new project we just won?
● ...
How Are Predictive Digital Models Constructed?
Building Digital Models for Prediction and Learning
“System”Stimulus(inputs)
Response(outcomes)
RealSystem
Digital Representation
of System
Model Building
Approach
Rea
l Sys
tem
Tran
sfor
mat
ion
to D
igita
l
Constructing a Digital Model (Forecast, Predictions, …)Traditional Approach Training a “Machine Learning/Deep
Learning” Model with Real-World Input, Outcome Pairs
Can You Rethink Your Challenge in the form of a Prediction Problem?
“The first effect of machine intelligence will be to lower the cost of
goods and services that rely on prediction. This matters because
prediction is an input to a host of activities including transportation,
agriculture, healthcare, energy manufacturing, and retail.
When the cost of any input falls so precipitously, there are two other
well-established economic implications. First, we will start using
prediction to perform tasks where we previously didn’t. Second, the
value of other things that complement prediction will rise.”
Source: https://hbr.org/2016/11/the-simple-economics-of-machine-intelligence
Machine Learning -> Deep Learning -> AI
Deep Learning (AI) APIs - Quickly Evolving and Improving Technology-> “AI as a Service”, “Prediction as a Service”
API - application programming interface
https://www.microsoft.com/cognitive-services/en-us/computer-vision-apihttps://cloud.google.com/vision/
https://cloud.google.com/natural-language/ https://developer.amazon.com/alexa-voice-service
...
Image Interpretation via Deep Learning, Cloud-based “API”
Source: https://www.microsoft.com/cognitive-services/en-us/computer-vision-api
Feature Name Value
Description { "type": 0, "captions": [ { "text": "a group of
people posing for a photo", "confidence":
0.9400386124726199 } ] }
Tags [ { "name": "outdoor", "confidence":
0.99739670753479 }, { "name": "person",
"confidence": 0.9943854808807373 }, { "name":
"posing", "confidence": 0.9544038772583008 },
{ "name": "group", "confidence":
0.7542804479598999 }, { "name": "crowd",
"confidence": 0.01921566016972065 } ]
Recent Analytical Study - Linking Employing Engagement to Activity Found in Electronic Calendars, Timekeeping, etc
We Are At the Early Stages in a Revolution that will Affect All Aspects of Business, Government
and our Lives as Prediction and Decision Making is “Digitized” and Automated in Machines Capable
of Learning by Example(s).
CLOSING - Why So Much Attention on IoT, Big Data, and Analytics?● Next frontier in productivity improvement is in automating decisions
● C-level executives realizing “digital exhaust” can be an asset
○ Reduce operating costs by optimizing
○ Revenue growth via new customers and new lines of service
● Business research showing analytics-savvy businesses yield greater profits and
growth
● The cost of building “prediction systems” declining
● The rapid rate of quality improvements in “human-like” prediction systems
● The explosion in sensor and log data, stored in “the cloud” at low cost
● Disruptive startup platform business models harnessing “big data”
○ Uber, AirBnB, Facebook, Twitter, …
● Fear of missing out (FOMO), Fear of disruption
Now, turbocharged by an explosion in IoT Sensors and Cloud-scale services
Extras
Refresher - the Scientific Method - A Rational Approach to Learning
1. Observations
2. Theory & Assumptions
3. Hypothesis (If X, then Y) <a prediction of
behavior>
4. Measures
5. Experiment
6. Compare Measures with Hypothesis from Theory
7. Learn (via Reflection and Dialogue)
8. Adjust Theory &/or Assumptions
9. RETURN TO #3
______________________________
The underlying idea behind Lean (aka TPS) and
recent codification of business startup methods.
