Techdays 2017: Give your Xamarin Apps eyes, ears and a brain with Cognitive Services

GEERT VAN DER CRUIJSEN

GIVE YOUR XAMARIN APPS

EYES, EARS AND A BRAIN WITH

COGNITIVE SERVICES

AboutGeert

Geert van der CruijsenMobile First Cloud First ArchitectXpirit Netherlands@geertvdc – geert@vdcruijsen.net

AI is the biggest new hype in software development

Should I care when I’m just building mobile apps?

AI HYPE

AI HYPE

But AI is not new

AI IS NOT NEW

Alan TuringInvited the Turing test in 1950

AI IS NOT NEW

Captcha’sAre reverse Turing tests

AI IS NOT NEW

Marvin MinskyThe “father” or AI

AI IS NOT NEW

1955 Dartmouth Research Project

REASONING

KNOWLEDGE REPRESENTATION

PLANNING(INCLUDING

NAVIGATION)

NATURALLANGUAGE

PROCESSINGPERCEPTION

AI IS NOT NEW – DARTMOUTH RESEARCH PROJECT

REASONING

KNOWLEDGE REPRESENTATION

PLANNING(INCLUDING

NAVIGATION)

NATURALLANGUAGE

PROCESSINGPERCEPTION

AI IS NOT NEW – DARTMOUTH RESEARCH PROJECT

GENERALIZED INTELLIGENCE

General Intelligence is still far away

And maybe that’s a good thing

Why AI now?

WHY AI NOW?

The arrival of machine learning Performance for decades-old challenges has been transformed

Image recognition

Speech recognition

October 2016

Microsoft researchers reach human parity in conversational speech recognition

Let’s do a small testHow many dogs can you find in 2 seconds?

How many did you count?Did anyone find all 10?

WHY AI NOW?

Old way: Rules

• Build systems to look for ears,noses, legs, fur…

• Hire linguists and writegrammar rules

New way: 2012 Machine Learning

• Huge catalogue of images(way more data)

• Huge computing power

• Over time it learns ratherthan need rules

AI & Mobile?Sure I believe the hype but why care as a mobile dev?

AI & MOBILE

User Experience

AI is making natural language and interaction possible

Rize of the Chatbots

Rize of the Personal Assistants

AI & MOBILE

Sensors Everywhere

But mostly in your phones

Mobile is the interface to the intelligent cloud and intelligent edge

AI & MOBILE

Innovation in hardware & mobile OS

Apple adds A11 chip focussing on AI Processing

Apple adds CoreML APIs to iOS 11

Android Tensorflow Lite coming later this year

Microsoft Cognitive ServicesLets see what we can do infuse our apps with AI

Machine Learning as a ServiceCognitive services are trained models ready to use

MICROSOFT COGNITIVE SERVICES

MICROSOFT COGNITIVE SERVICES

Language

LUISSpell CheckLinguistic AnalysisText AnalyticsTranslatorWeb Language Model

Vision

Computer VisionContent ModeratorEmotionFaceVideoVideo Indexer

Speech

SpeechSpeaker RecognitionTranslatorCustom

Search

AutosuggestImageNewsVideoWebCustom

Knowledge

AcademicEntity LinkingExplorationQnA MakerRecommendations

Bot Framework

Inspirational use casesWhat kind of ideas can you steal/borrow and add to your own apps

MICROSOFT COGNITIVE SERVICES

INSPIRATION: VISION

UBER:Driver verification using vision API

INSPIRATION: SPEECH

Speech to TextBuild your own personal assistant or voice recorder

INSPIRATION: LANGUAGE

Spell check User input

SHOW ME SOME CODE ALREADY!github.com/geertvdcgithub.com/xpiritbv

AboutGeert

Thank you!

https://mobilefirstcloudfirst.net@geertvdc – geert@vdcruijsen.net