Embed Size (px)
Citation preview
Jeff FriedCTO, BA Insight
Use Office 365 and Azure Cognitive Services for Intelligent Search
October 2017
Longtime
Search Nerd
• CTO, BA Insight
• Senior PM, Microsoft
• VP, FAST
• SVP, LingoMotors
Jeff Fried
Passionate About
• Search
• Search-driven applications
• Information Strategy
• Text Analytics
• Machine Learning
Email: [email protected]
Twitter: @jefffried
Blog: www.DoMoreWithSearch.com
Outline
Wait, isn’t search
ALREADY ‘intelligent’?
Matching IntentContent
Content Intent
LookupLearnExploreMessy
Human
Language
A new wave of Intelligent Search Engines?
Machine Learning (ML), Natural Language Processing (NLP),
Semantics, Cognitive, Artificial Intelligence (AI), Self-Learning, …
http://www.cmswire.com/information-management/intelligent-cognitive-ai-based-search-separating-hype-from-reality/
Forrester: “Keyword search of the past has evolved into Cognitive Search”
Forrester, June 2017
Evolution of enterprise search technology
TODAYYESTERDAY TOMORROW
Relevancy based on general criteria that
compare how well a single item matches a
query, compared to the rest of the index.
Matching Content & Popularity
By leveraging your personal activity, we
can signal highly relevant content that
would otherwise drown in the noise
Personalized & behavior based Drive actionable insights
Take action on data and empower
decision making by understanding
intention across many facets
Scope & scale of Microsoft Graph
Use machine learning to create
insights from your data
Construct pairs for questions & answers
Query match quality in content or page
Popularity of document or page
Content metadata such as created date
Bing entity extraction
Understanding organizational priorities
as well as your extended network
Using AI for semantic understanding
+ +
Microsoft, September 2017
11
Semantics! Machine Learning! AI!
Latest Enhancements to SPO search
Making O365 search intelligent using Azure Cognitive Services
more
ˇ
Start with understanding how it works
Matching IntentContent
Context
ContentEnrichment
UX
Connections
QueryAugmentation
Insight Data Sets
Trending Documents
People I Work With
UsedDocuments
Files
Contacts
Groups
Meetings
Org
Tasks
File Enrichments
ItemMetadata
# ActivityHistory
myGraph
Sites I Use
Suggested Sites
Usage Statistics
How does the search box work?
Used
Documents
Trending
Documents
Sites
I Use
Suggested
Sites
Beautiful
Bergen
Contoso
Norway
Kathrine
Hammervold
Metadata Drives Great User ExperiencesDocuments from many sourcesAll client or matter-relevant documents are integrated. Rich MetaData
Content annotated automatically – concepts,
categories, citations, matters, clients, etc
Navigation Controls
Explore, Discover, Drill-down
Where is the Metadata in Delve?
So….I don’t need metadata, right?
Delve doesn’t have refinement
Chart courtesy of xkcd
Outline
The Cognitive Arms Race
Agent Bots Cognitive Machine Learning Augmented RealityMicrosoft Cortana Bot Framework Cognitive Services
- Vision- Speech- Language- Knowledge- Search- Labs
Azure MLCognitive Toolkit
Hololens
Google Google Assistant API.AI Cloud Vision APICloud Video IntelligenceCloud Speech APICloud Natural Language APIGoogle Knowledge GraphGoogle Custom SearchML Advanced Solutions Lab
Cloud ML EngineTensorFlow
Glass at WorkDaydream/Tango
IBM Watson Virtual Agent Watson Conversation Watson APIs- Vision- Speech- Language- Data Insights
Watson ML ServiceApache SystemML
Amazon Alexa Amazon Lex RekognitionPolly
Amazon ML
Apple Siri SiriKitSpeech
Core ML ARKit
Facebook Facebook M ParlAI FastTextCommAI
Torch Oculus Rift
The Cognitive Arms Race
Matching IntentContent
Context
Content
Enrichment
UXConnections
Query
Augmentation
Making SharePoint search intelligent with Cognitive Services
more
ˇ
BA Insight Product Portfolio
Smart Analytics
AutoClassifier
Content from 60+ Enterprise Systems
Cognitive Services (AI, Machine Learning)
SearchIndex(es)
Search forDynamics
Connectors
ExpertiseLocator
SmartPreviews
Sm
art
Hub
Content
ContentProcessing
SearchIndex
/ Graph
Search UI
Enhancing Content with Cognitive Services
QueryProcessing
Content
Enrichment
AutoClassifierrules-driven core, combined with learning-based modules
Enhanced Content
Enriched with
Metadata and
Content Types
Search Visualization Workflow
Adding Text Analytics to ContentUser
Uploads Content Event
Fires
Content Text
Analysis
Sentiment - Is text positive or negative?
Key Phrases - What are people discussing in a single article?
Topics - What are people discussing across many articles?
Language - What language is text written in?
Add metadata to item
Shows in O365 search
Adding Translation to ContentUser
Uploads Content Event
Fires
Content Text
Analysis
Translation – Copy existing content and translate to specified language
save in SharePoint
Shows in O365 search
Create new file/content for specific language
Image SearchUser
Uploads Content Event
Fires
Computer Vision
Image Processing – recognize objects and text inside images
Shows in O365 search
Add metadata to item
Extract image(s)
from documents
Image Classifier and Search
project libraries
and
issue detection
Search Visualization Workflow
Feed content from over 60 enterprise systems
SearchIndex(es)
Search UI
Connectors
SmartAnalytics
Extensiblecontent processing
Pluggable query and results pipelines
Auto Classifier
Cognitive Services
SmartHub
“Architectural” View
37
Content is spread across lots of systems• Aderant
• Amazon S3
• Alfresco
• Box
• Confluence
• CuadraSTAR
• Elite / 3E
• EMC Documentum
• EMC eRoom
• Google Drive
• HP Consolidated Archive
• (EAS, aka Zantaz)
• HPE Records Manager/HP TRIM
• IBM Connections
• IBM Content Manager
• IBM DB2
• IBM FileNet P8
• IBM Lotus Notes
• IBM WebSphere
• iManage Work
• Jive
• LegalKEY
• LexisNexis Interaction
• Lotus Notes Databases
• Microsoft Dynamics CRM
• Microsoft Exchange
• Microsoft Exchange Public Folders
• Microsoft SQL Server
• MySQL
• NetDocuments
• Neudesic The Firm Directory
• Objective
• OpenText LiveLink/RM
• OpenText eDOCS DM
• Oracle Database
• Oracle WebCenter
• Oracle WebCenter Content (UCM/Stellent)
• PLC/Practical Law
• ProLaw
• Salesforce.com
• SAP ERP
• ServiceNow
• SharePoint Online
• SharePoint 2016
• SharePoint 2013
• SharePoint 2010
• SharePoint 2007
• Sitecore
• Any SQL-based CRM system
• Veeva Vault
• Veritas Enterprise Vault
(Symantec eVault)
• West km
• Xerox DocuShare
• Yammer
List of standard connectors, plus a proven architecture and process for creating new connectors to complex systems
Feed content from over 60 enterprise systems
SearchIndex(es)
Search UI
Connectors
SmartAnalytics
Extensiblecontent processing
Pluggable query and results pipelines
Auto Classifier
Cognitive Services
SmartHub
Bots
AR (hololens) apps
Bots and AR
Better Search- data enrichment
- query augmentation
- relevance
- recommendations
New Types of Search
and Discovery- multi-media search
- multi-lingual search
- spoken language search
- search bots
- search in AR
AI for Intelligent Search & Discovery
Content
ContentProcessing
SearchIndex
Search UI
ML applied to search
QueryProcessing
Learning about content structure Learning about user intent
Analytics
Behavioral data to drive learning
Learning to rank
BA Insight Product Portfolio
Smart Analytics
AutoClassifier
Content from 60+ Enterprise Systems
Cognitive Services (AI, Machine Learning)
SearchIndex(es)
Search forDynamics
Connectors
ExpertiseLocator
SmartPreviews
Sm
art
Hub
![[Microsoft Cognitive Toolkit (CNTK) on Azure ハンズオン] Microsoft Azure の AI 関連サービス](https://img.pdfslide.net/doc/110x75/5a6478a37f8b9a3b568b4657/microsoft-cognitive-toolkit-cntk-on-azure-microsoft-azure.jpg)
