Augmented intelligence pietro_leo_sole24_ore_school

Artificial Intelligence to make precise decisions

July 13, 2017Pietro Leo Executive Architect & CTOChief scientist, and research strategist IBM ItalyIBM Academy of Technology Leadership Teampieroleo.com

DATA

INFORMATION

KNOWLEDGE

WISDOM

DECISION

July 18, 20176

July 18, 20177

July 18, 20178

July 18, 20179

10

You shared your position with me and can guess your mobility need. I can take you where you need to be

Just enjoy your new experience. Stay safe as in your friends home

I know what is needed for you, even before you order it

Please, come with me and stay by me.I know your content I can take care of all your digital life

11

Video: http://www.digitaltrends.com/home/grush-toothbrush-wins-americas-greatest-makers/

http://www.grushgamer.com/

HYPERDATAWORLD

Source: http://www.bloomberg.com/video/meet-the-world-s-most-connected-man-Vs~LzkbkR7yhjza~7nji1g.html

Meet theWorld's Most Connected Man

16Image source: http://personalexcellence.co/blog/i deal-beauty/

17Image source: http://personalexcellence.co/blog/i deal-beauty/

City

Lifestyle

ZIPcode

Costal vsInland Maritalstatus

Generation

Location

FamilySize

Gender

Income Level

Competitors

Age

Loyalty&CardActivity

Revenue Size

Life Stages

Eductation

Legalstatus

Sector

Industry

18

Image source: http://personalexcellence.co/blog/i deal-beauty/

City

Lifestyle

ZIPcode


Generation

Location

FamilySize

Gender

Income Level

Competitors

Age


Revenue Size

Life Stages

Eductation

Legalstatus

Sector

Industry

SubscriptionsDate on Site

Wish List

Size of Network

Check-ins

App usage duration

Number of Apps on Device

Deposits/Withdrawals

Device UsagePurchase History

FollowingFollowers

Likes

Number of Hashtags used

History of Hashtags

Search Strings entered

Sequence of visits

Time/Day log in

Time spent on site

Time spent on page

Frequency of Search

Videos Viewed

Photos liked

19


City

Lifestyle

ZIPcode


Generation

Location

FamilySize

Gender

Income Level

Competitors

Age


Revenue Size

Life Stages

Eductation

Legalstatus

Sector

Industry


Wish List

Size of Network

Check-ins

App usage duration




FollowingFollowers

Likes


History of Hashtags


Sequence of visits

Time/Day log in

Time spent on site

Time spent on page

Frequency of Search

Videos Viewed

Photos liked

Sentiment

Tone

Euphemisms

Hedonism

Extroversion

Face Recognition

Openess

Colloquialism

Reasoning Strategies

Language Modeling

DialogIntent

Latent Semantic Analysis

Phonemes

Ontology Analysis

Linguistics Image Tags

Question Analysis

Self-transcendent

Affective Status

20


City

Lifestyle

ZIPcode


Generation

Location

FamilySize

Gender

Income Level

Competitors

Age


Revenue Size

Life Stages

Eductation

Legalstatus

Sector

Industry


Wish List

Size of Network

Check-ins

App usage duration




FollowingFollowers

Likes


History of Hashtags


Sequence of visits

Time/Day log in

Time spent on site

Time spent on page

Frequency of Search

Videos Viewed

Photos liked

Sentiment

Tone

Euphemisms

Hedonism

Extroversion

Face Recognition

Openess

Colloquialism

Reasoning Strategies

Language Modeling

DialogIntent

Latent Semantic Analysis

Phonemes

Ontology Analysis

Linguistics Image Tags

Question Analysis

Self-transcendent

Affective Status

X-rays (CT scans) sound (ultrasound), magnetism (MRI), Radioactive (SPECT, PET)light (endoscopy, OCT)

Bio-Images

Clinical/Biochemical DataMicrobiome

EnvironmentDNAProteome

Steps

Nutrition

Genetics

Runs

Food

Source: Bipartisan Policy Center, F as in Fat: How Obesity Threatens Americas Future (TFAH/RWJF, Aug. 2013)

Internet of Body

BMI

Rapid growth of exogenous data is transforming healthcare

6 Terabytes

60%Exogenous Factors

1100 TerabytesVolume, Variety, Velocity, Veracity:Educational records, Employment Status, Social Security Accounts, Mental Health Records, Caseworker Files, Fitbits, Home Monitoring Systems, and more

0.4 TerabytesElectronic Medical / Health Records, Physician Management Systems, Claims Systems and more

30%Genomics Factors

10%Clinical Factors

IBM Watson Health // SOURCE: 2015 J.M. McGinnis et al., The Case for More Active Policy Attention to Health Promotion, Health Affairs 21, no. 2 (2002):7893

Data Generated per Life

Leveraging Exogenous Data for Chronic Care

60%Exogenous Factors

30%Genomics Factors

10%Clinical Factors

SOURCE: 2015 J.M. McGinnis et al., The Case for More Active Policy Attention to Health Promotion, Health Affairs 21, no. 2 (2002):7893

Glucose Monitoring

Calorie Intake

Stress LevelsPhysical Activity

Other vital signs SocialInteraction

Affinity (retail)

Sleep Pattern

> 2.5 Trillion PDF Files in the World

Majority with public and private enterprises and institutions.

Enterprise HYPERDATA

23

Multi-Modal Rich data: Text, Tables, Images, Audio, Video, Formats, Hierarchy.

PRECISION

Leveraging the Explosion of Data in Medicine An Impossible Task Without Analytics and New advanced Artificial Intelligence Computing Models

1000

Fact

s pe

r Dec

isio

n

10

100

1990 2000 2010 2020

Human Cognitive Capacity

Electronic Health Records (Clinical Data)

Internet of Things (Exogenous Data)The Human Genome (Genomic Data)

Capturing the Value of Data: Big Changes Ahead

Medical errorthe third leading cause of death in the US

Source: BMJ 2016; 353 doi: http://dx.doi.org/10.1136/bmj.i2139 (Published 03 May 2016) Cite this as: BMJ 2016;353:i2139

26

Body Mass Index (BMI)

Mass (weight - Kg) / height (cm) x height (cm)

You are Normal if your BMI is between 18.5 and 24.99

Adolphe Quetelet, 1832

27

Practice Pearls: BMI - Body mass index is a strong and independent risk factor for being diagnosed with type 2 diabetes mellitus Type 2 diabetes risk may be incrementally higher in those with a higher body mass index Understanding the risk factors helps to shorten the time to diagnosis and treatment

How precise could be a simple signal

2017 International Business Machines Corporation

The way to find information

The way to make precise decisions

Big

Dat

a ++


Technology ingredients to make precise decisions: driving new Capability for Business

Artificial IntelligenceRange of techniques including natural language understanding,knowledge, reasoning and planning, for advanced tasks

Cognitive ComputingLeverage a combination decision-makingand reasoning strategies over deep domain models and evidence-based explanations, using AI/ Machine Learning tools.

Machine LearningStatistical analysis forpattern recognition to make data-driven predictions


Research at the heart of core AI

3

Comprehension: From video and text to rich human perception

Learning and Reasoning:From scalable machine learning to making a case

Interaction:Understanding language, tone, emotion and context

A green bird sitting on top of a bowl

Hype Cycle for Emerging Technologies, 2016 (Gartner)

https://www.ibm.com/annualreport/2016/images/dow nloads/IBM-Annual-R eport-2016.pdf

Augmenting DECISIONS

Assistant

Tools

Collaborator

Coach

Mediator

Emerging types of Cognitive Systems

Augment Decision Making is opening to new forms of collaboration between humans and machines

34

Radiologist Oncologist

Sales Assistant Tax Advisor

35

Chef Designer

Musicist Movie Director

Opportunity for decision-making

support2025

Augmenting decisions opens new opportunities on top of traditional IT

Traditional globalIT spend

Source: IBM analysis presented to the Investor Briefings

~$2T

~$1.2T

37

Top outcomes from cognitive initiatives vary by industry

Finance49% Increased market agility46% Improved customer service43% Increased customer

engagement43% Improved productivity &

efficiency42% Improved security &

compliance, reduced risk

Retail56% Personalized customer / user

experience56% Increased customer engagement56% Improved decision making &

planning 56% Reduced costs55% Improved customer service

Health66% Accelerated innovation of

new products / services66% Improved productivity & efficiency64% Improved security & compliance,

reduced risk62% Reduced costs59% Improved customer service

Manufacturing 64% Improved decision making

& planning 58% Improved productivity &

efficiency54% Improved security &

compliance, reduced risk52% Improved customer service49% Enhanced the learning

experience

Government/Education54% Personalized customer / user

experience50% Improved customer service37% Improved decision making &

planning 36% Improved productivity & efficiency33% Increased customer engagement

Professional Services40% Reduced costs36% Personalized customer/user

experience36% Improved customer service36% Expanded ecosystem34% Accelerated innovation of new

products / services

% achieving outcome with cognitive

Source: An IBM study of over 600 early cognitive adopters - 2016 Full report: http://www.ibm.com/cognitive/advantage-reports/

IBM Watson is the most advanced Artificial Intelligence & Machine Learning platform to support Decision Making in Business

Toward a Precise Decision Making to reduce the wasteful spend as well as the risk in every industry

Watson:Cognitive System

IBM Cognitive Computing

45Nazioni

100+Applicationsgi nel mercato

6.000Ricercatori e Specialisti in IBM

8Lingue

200Universitorganizzano corsi su Watson

500+PartnersChe integrano Watson

API & HybridCognitive Frameworks

20Industrie

80.000Sviluppatoricostruiscono applicazionicon Watson

Watson Health5.000 Dipendenti, 6B$ di investimento

Watson InternetOf Things1000 Dipendenti, 3B$ di investimento

Watson FinantialServices

3 Unit di BusinessVerticali

200MCittadini

60MPazienti

30BImmagini

1.2MAbstractMedici

60+Soluzioni

Who: Current top players (prevalent) competitive directions and approaches

Personalized Service /Content Aggregation

Industry-oriented / Professions SpecificOutcomes via cognitive Solutions

Core Business Cognitive /Enhance Experiences

IBM (Health,Finance, )

API SERVICES /PLATFORM

AWS

Microsoft

Goggle

Amazon (Alexa)Facebook

IBM BlueMix

42

Keyword Extraction, Entity Extraction, Sentiment Analysis, Concept Tagging,

Conversation Intents Entities Dialogues

Personality Big5 Personality Traits Needs Values

Language Tone Emotion Social propensities Language styles

Translate Conversational News Custom TranslationPatents

Language DeepUnderstanding

Relation Extraction, Taxonomy Classification, Author Extraction.. Custom Analysis

Speech-to-text

Custom pronunciations Voice TransformationExpressive Voice

Voice synthesis

Keyword Spotting Telephony Broadband

Vision Face Recognition Image Similarity Image ClassificationCustom eyes

Source: https://www.ibm.com/watson/developercloud/services-catalog.htmlWATSON

Kind of skills

43https://www.technologyreview.com/s/603895/customer-service-chatbots-are-about-to-become-frighteningly-realistic/

The movements of Soul Machiness digital faces are produced by simulating the anatomy and mechanics of muscles and other tissues of the human face.

Soul Machines

The avatars can read the facial expressions of a person talking to them, using a devices front-facing camera

Soul Machinesmade NADIA, a chatbot for the Australiangovernment to help people getinformation aboutdisabilityservices.

45

Conversation

46

I am going to New York next May

Man

Walking, go around

vest

Where and When will you be using this jacket?

I'll find a jacket that fits those conditions. Are you looking for a men's or women's jacket?

Okay, I got it. What will you use this jacket for?

What styles are you looking for?

Conversation

https://www.thenorthface.com/xps

47

I am going to New York next May

Where and When will

you be using this jacket?

I'll find a jacket that fits those

conditions. Are you looking for a

men's or women's jacket?

https://www.thenorthface.com/xps

Man

Okay, I got it. What will you

use this jacket for?

Walking, go around

What styles are you

looking for?

vest

48

It will be more and more a bots vs bots marketing battle!

Our personal BOTS will buy for us, #Brands should convince them NOT us!


Watson OncologyA collaboration between IBM and Memorial Sloan Kettering (MSK). Watson for Oncology utilizes MSK curated literature and rationales, as well as over 290 medical journals, over 200 textbooks, and 12 million pages of text to support decisions.

Analyzes the patient's medical record Identifies potential evidence-backed treatment options Finds and provides supporting evidence from a wide variety of sources

51

The Medical Sieve Build a fast anomaly detection engine

Quickly filters irrelevant imagesHighlights disease-depicting regionsFlags coincidental diagnosis

Intended as a radiology assistant Clinicians still do the diagnosisMachine reduces workload Machine performs triage/decision support

Given history of the patient and images of a study

Is there an anomalous image here?If so, where is the anomaly ?Describe the anomaly

The Medical Sieve


86%Accuracy


Identification of masses in breast MRI images >93% (1)

Detection of calcified plaques in coronaryarteries from CT images > 90% (2)

Automatic Detection of Aortic Dissectionin Contrast-Enhanced CT > 83% (3)

Melanoma recognition in DermoscopicImages >84% Roc curve (sensivity>95%)

IBM Research Works from the International Symposium on Biomedical Imaging 2017

(1) Hadad, Omer at ali - (2) Tang, Hui at ali. (3) Dehghan, Ehsan at ali (4) Moradi, Mehdi at ali. (5) Ben-Ari, Rami at ali. (6) Roy, Pallab at ali. (7) edai, Suman at ali (8) Coleccala et ali.

.

Labeling Doppler images with aortic stenosis>78% (4)

Detection of Architectural distortion in Mammograms > 80% (5)

Diabetic Retinopathy detection in ColourFundus Images >86% (6)

Multi-Stage Segmentation of the Fovea in Retinal Fundus Images Error

7/18/1755

I.R.C.C.S. CASA SOLLIEVO della SOFFERENZAOpera di San Pio da Pietrelcina

7/18/1756

7/18/1757

7/18/1758

7/18/17

Stories


Memories are a bridge among generations

7/18/17Tales

Weather is the secret to understanding how consumers feel and cook

A brand able to gain a spot in the daily routines and rituals of consumers creates a not only a relation but a deep intimacy with them

63

https://watsonads.com

Watson Ads

16

65

CREATIVECOMPUTING

66

MARCHESAA dress that think

JASONGRECH Fashion zeitgeist

Food Knowledge Database

CombinatorialDesigner

Cognitive Assessor

DynamicPlanner

Peer Produced Inspiration Set

Novel Customized

Recipe

Cognitive Cooking System

67

How does Cognitive Cooking work?

Raw Data- Recipes- Recipes contexts- Chemical/Flavour Data- Hedonic psychophysics- Background knowledge (e.g. Wikipedia for regional cuisines, etc)...

- Bayesian surprise- Flavor Pleasantness...

Data-drivenDecisions

106 >1015-23

Watson Chef with Bon Apptit

Live at: https://www.ibmchefwatson.com/tupler

70

Creations from the Cognitive Collection Designed by JASONGRECH and IBM Watson

Source: https://www.ibm.com/blogs/think/2016/08/cognitive-fashion/

71Source: https://www.ibm.com/blogs/think/2016/08/cognitive-fashion/

Source: https://www.ibm.com/blogs/think/2016/08/cognitive-movie-trailer/

1) A visual analysis2) An audio analysis3) An analysis of each scenes composition

IBM Research Takes Watson to Hollywood with the First Cognitive Movie Trailer

Watson / Presentation Title / Date74

WatsonPlatform

75 IBM Cognitive Cloud | Electrolux Digital Summit 2017

Cloud InfrastructureA highly scalable, security enabled infrastructure

DataTools to prepare data for cognitive

AICognitive building blocks for developers

Applications, solutions and servicesTargeted solutions for enterprise businesses

IBM delivers an architecture engineered for disruption

Cognitive Systems leverage machine learning to predict meaning in features of human language (spoken, written, visual) and related forms of human reasoning


Cloud InfrastructureA highly scalable, security enabled infrastructure

DataTools to prepare data for cognitive

AICognitive building blocks for developers

Applications, solutions and servicesTargeted solutions for enterprise businesses

Ingestion

ConversationA

PI

Storage Analytics Deployment Governance

WatsonHealth

Solutions

WatsonCyber

SecurityWeather

IBM Services & Ind.

Solutions

WatsonVirtual Agent

Watson Explore

and Discover

IBM Risk and

Compliance

Asset Mgmt.

(Maximo)

Visual RecognitionA

PI

Discovery

AP

I

Speech

AP

I

Compare and ComplyA

PI

Document ConversionA

PI

DLaaS

AP

I

Nat Language UnderstandingA

PI Nat Language

ClassifierAP

I

ToneAnalyzerA

PI Personal

InsightAP

I

KnowledgeQueryA

PI

IBM delivers an architecture engineered for disruption

Cloud Integration

Networking SecurityCore

Enterprise Infrastructure

CognitiveSystems

Virtual Servers File Storage

Object Storage

Cognitive Micro-services DevOps Tooling

ISV Solutions Client Solutions


Data analyticsServe modelTrain model

Cognitive technologies transform data into augmented intelligence that drives differentiated experiences and outcomes

Cognitive micro-services driven tooling

CurateTraining data

ConversationAPI

ToneAnalyzerAP

I

Document ConversionAP

I

DiscoveryAPI

PersonalInsightAP

I

Nat Language UnderstandingAP

I

Compare & ComplyAP

I

Visual RecognitionAPI

Nat Language ClassifierAP

I

DLaaSAPI

SpeechAPI

KnowledgeQueryAP

I

AI

https://developer.ibm.com/academic/ https://www.ibm.com/developerworks/


IBM Academic Initiativehttps://developer.ibm.com/academic/

References

Bluemixhttps://www.ibm.com/cloud-computing/bluemix/

79

CLOSING

Chief ArtificialIntelligence Officer

Chief Data Scientist

Chief InformationOfficer

Chief DataOfficer

DATA INFORMATION KNOWLEDGE WISDOM

A number A STREET number

A map of a City

A GPS root recommendationto go from A to B

https://www.theguardian.com/technology/2016/sep/08/artificial-intelligence-beauty-contest-doesnt-like-black-people

https://www.partnershiponai.org/

Cognitive Principles

1. Purpose

2. Transparency

3. Skills

Source: https://www.ibm.com/ibm/responsibility/ibm_policies.html

The purpose of AI and cognitive systems developed and applied by the IBM company is to augment human intelligence.

The IBM company will make clear: a) When and whatpurpose of a cognitive solution; b) Major Data Used; c) Protect Customer Data & Insights ownership.

IBM company will work to help students, workers and citizens acquire the skills and knowledge to engagesafely, securely and effectively in a relationship with cognitive systems, and to perform the new kinds of work and jobs that will emerge in a cognitive economy.

Thank youfor your attention.

Pietro Leo Executive Architect & CTO

Chief scientist, and research strategist IBM ItalyIBM Academy of Technology Leadership Team pieroleo.com

July 18, 201785

1st Place Image

Source: COCO Challengehttps://www.ibm.com/blogs/bluemix/2016/12/watsons-image-captioning-accuracy/

1st Place Speech

Watson says: A green bird sitting on top of a bowl

IBM Leadership in AI to understand our world

Watson error rate: 5.5%

Source: Switchboard conversational corpushttps://www.ibm.com/blogs/watson/2017/03/reaching-new-records-in-speech-recognition/



Ready to use Affective Computing services in the Watson Platform


CurateTraining data

ConversationAPI

ToneAnalyzerAP

I


I

DiscoveryAPI

PersonalInsightAP

I


I

Compare & ComplyAP

I



I

DLaaSAPI

SpeechAPI

KnowledgeQueryAP

I

AI

= Affective Service

Emotional Tone: joy, fear, sadness, disgust, anger

Social Tone: openness, conscientiousness, extraversion,

agreeableness, emotional range or neuroticism

Language Tone:Analytical, confidence, tentative

Customer Engagement Tone:Sad, Frustrated, Satisfied, Excited, Polite,

Impolite, Sympathetic





CurateTraining data

ConversationAPI


I

DiscoveryAPI

PersonalityInsightAP

I


I

Compare & ComplyAP

I



I

DLaaSAPI

SpeechAPI

KnowledgeQueryAP

I

AI

= Affective Service

Big Five dimensions Emotional Range, Consciousness, Openness,

Introversion/Extroversion, Agreeableness,

Big Five facets(30 sub dimensions)

NeedsStructure, Curiosity, Challenge, Ideal, Stability

ValuesStimulation, Tradition, Helping others, Taking

pleasure in life, Achievement

ToneAnalyzerAP

I





CurateTraining data

ConversationAPI

ToneAnalyzerAP

I


I

DiscoveryAPI


I


I

Compare & ComplyAP

I



I

DLaaSAPI

SpeechAPI

KnowledgeQueryAP

I

AI

= Affective Service

ExpressivenessGoodNews, Apology, Uncertainty

Voice TransformationYoung, Soft

Custom: Pitch, pitch range,, glottal tension, breathiness, rate timbre (sunrise, Breeze)





CurateTraining data

ConversationAPI

ToneAnalyzerAP

I


I

DiscoveryAPI


I


I

Compare & ComplyAP

I



I

DLaaSAPI

SpeechAPI

KnowledgeQueryAP

I

AI

= Affective Service

Emotionsjoy, fear, sadness, disgust, anger

Target Emotions for Entities

(24 main types of entities)

(433 subtypes)Custom entities

Keywords