Microsoft Analytics: The Next Wave
Simon LidbergBenjamin Wright-JonesData Insights Center of Excellence
How do we move from this…
…to this
Delivering on one of the old dreams
of Microsoft co-founder Bill Gates:
Computers that can see, hear
and understand
John PlattDistinguished scientist at
Microsoft Research
What is Machine Learning?
Computing systems that improve with experience
“
”
The United States Postal Service processed 158.4 billion pieces of mail in 2013—far too much for efficient human sorting.
But as recently as 1997, only 10% of hand-addressed mail was successfully sorted automatically.
The challenge in automation is
enabling computers to interpret
endless variation in handwriting.
By providing feedback, the Postal
Service was able to train computers
to accurately read human
handwriting.
Today, with the help of machine
learning, over 98% of all mail is
successfully processed by machines.
SQL Server
enables data
mining of
databases
Computers
work on users
behalf, filtering
junk email
Microsoft
Kinect can
watch users
gestures
Microsoft
search engine
built with
machine
learning
Bing Maps
ships with ML
traffic-
prediction
service
Successful,
real-time,
speech-to-
speech
translation
Microsoft & Machine Learning20 years of realizing innovation
John Platt, Distinguished scientist at
Microsoft Research
1999 201220082004 20102005
Machine learning is pervasive throughout
Microsoft products.“
”
Data Science is far too complex today
• Access to quality ML algorithms, cost is high.
• Must learn multiple tools to go end2end,
from data acquisition, cleaning and prep,
machine learning, and experimentation.
• Ability to put a model into production.
This must get simpler, it simply won’t scale!
Data ScienceComplexity
Azure Machine Learning, Makes years of innovation available to everyone
Easily manage
and monitor
Ensure enterprise-
grade security
Amplify your
investments
Operationalize
in minutes
Streamline with
one portal to view
and update
Peace of mind with
best-in-class data and
identity security features
Get more from your
machine learning and
Azure solutions
Tooled for quick
deployment, hand-off
and updates
Reduce complexity to broaden participation
Features and Benefits
• Accessible through a web browser, no software to install;
• Collaborative work with anyone, anywhere via Azure workspace
• Visual composition with end2end support for data science workflow;
• Best in class ML algorithms;
• Extensible, support for R OSS.
Rapid experimentation to create a better model
Microsoft Azure Machine LearningFeatures and Benefits
• Immutable library of models, search discover and reuse;
• Rapidly try a range of features, ML algorithms and modeling strategies;
• Quickly deploy model as Azure web service to our ML API service.
Imagine what machine learning could do for your business.
Churn
analysis
Equipment
monitoring
Spam
filtering
Ad
targeting
Recommendations
Fraud
detection
Image
detection &
classification
Forecasting
Anomaly
detection
demo
We are especially pleased
that our analysts can focus
on the results and not
worry about the complex
algorithms behind the scenes.
Andrew Laudato
Pier 1 Imports
Customer recommendations with Azure Machine Learning
Pier 1 partnered with MAX451 to help loyalty customers by using historical and behavioral data to predict what products they want next.
Key Benefits
• Ease of use across skillsets
• Fast time to meaningful results
• Accessible via the cloud
“
”
Azure ML Service
Assign drivers into categories - sports
driver, Grandma/pa driver
Provide special offerings based on driver
type - sports package for the sports driver
Make a special offer for the next car
Analytical Task: Cluster the relevant diagnostics data to find groups of drivers with common
driving habits
1
3
2
No sensor for measuring the thickness of
a brake disk or a tire - Need to predict
when a failure will occur.
Inform the driver - change of parts might
be necessary in a few weeks
Order parts and prepare an individual
maintenance package before the driver
comes to the garage.
Tie customers to the dealer by offering
them attractive maintenance and spare
parts packages.
Analytical Task: Predict the possibility of next break pad failure using data collected from
sensors and telemetry.
1
3
2
Telemetry Data
HDInsight ClusterMapReduce/Hive
Service Bus
Gateway (S2S or P2S)
Developer
Data Scientists
Blob Storage (Containers) Data Mart (IaaS)
Staging (IaaS)
Web Worker Role
Internal Data Sources
Mobile Workers
External Data Scientists
HTML5Dashboard
HTTPSQL DatabaseTable StorageBlob Storage
Hive QueryCSV Files
PowerQuery
Administrator
CSV Files
Publishing Modules (SDK)
Data Conditioning (Iaas)
Database Replica / Data Backups
Data Sets Modelling
Sharing Experiments
TrainingScoring
ML Studio and the Data Scientist
• Access and prepare data
• Create, test and train models
• Collaborate
• One click to stage for
production via the API service
Business users easily access results:from anywhere, on any device
HDInsight
Desktop Data
Azure Storage
Mobile AppsEnterprise
DashboardsWeb Apps
ML API service and the Developer
• Tested models available as an url that can be called from any end point
Azure Portal & ML API serviceand the Azure Ops Team
• Create ML Studio workspace
• Assign storage account(s)
• Monitor ML consumption
• See alert when model is ready
• Deploy models to web service
"Execute R Package"
Option 1 – Results to Output Log
print(rownames(installed.packages()))
Returns results in “Visualize” or "View in Output logs" (currently 410 packages….)
Option 2 - Results to CSV
data.set = data.frame(installed.packages())
maml.mapOutputPort("data.set");
Are you a
developer or a
DBA?
Are you a business
analyst?
Are you a senior
business
stakeholder?
Are you an
architect?
Are you a SQL Server developer or
DBA and wish to broaden your
knowledge and skills?
• Online courses (Coursera/Udacity)
• Hands-on learning (curiosity)
• Recommended reading
• Subscribe to dsc/kdnuggets
• Graduate and post-graduate
study
Are you an analyst with a math's or
stats background and business
domain knowledge?
• Some of the online courses are
specifically targeted at non-IT
roles and do not have pre-
requisite knowledge requirements
Are you a senior business
stakeholder interested in the
leveraging analytics to reduce time
to value?
• Increased interest in organisations
establishing a COE function
• Governance, information
management, program
management, strategy, solution
validation, knowledge transfer,
role readiness.
Are you a solution architect /
application / data architect?
• Broad awareness of the data
science domain and business
value
• Encouraged to follow the same
learning path as a DBA or
developer
Where are the Data Scientists?
Coursera offerings (Andrew Ng/John Hopkins) are recommended
Internal Communities Forming e.g. Data Platform / Business
Analytics professionals (broadening skills), formal role definition
Specialist Data Science Division Group (DSDG)
Data Insights COE (focusing on analytics scenarios)
Algorithms Extensibility Rest API Collaboration
Classification, Clustering,
Regression and
Recommender methods
SDK for custom module
development?
Execute R module
Web Service endpoint
for client connectivity
Easily collaborate on the
same experiments, Store
for modules
Contacts
Simon Lidberg
Benjamin Wright-Jones
http://azure.microsoft.com/en-us/documentation/services/machine-learning/tutorials/
http://azure.microsoft.com/en-us/documentation/articles/machine-learning-azure-ml-customer-churn-scenario/
http://azure.microsoft.com/en-us/documentation/articles/machine-learning-overview-of-azure-ml-process/