24
[email protected] Qualitative Research in the Age of Artificial Intelligence Andrew Konya

Andrew Konya - Al and qualitative moderation and analysis

  • Upload
    baqmar

  • View
    68

  • Download
    2

Embed Size (px)

Citation preview

Page 1: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Qualitative Research in the Age of Artificial Intelligence

Andrew Konya

Page 2: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Fundamental Question Gather Data Analyze Data Report.ppt

How to scale for Qual?

Page 3: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Fundamental Question Gather Data Analyze Data Report.ppt

Data

“I just can’t vote for a corrupt politician. Trump is at least blunt and honest, but how he talks about women makes me feel really

uncomfortable. I am still undecided.”

Natural Language

Audio Video

Page 4: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

I enjoy walks with my dog

I like to stroll with my k9

fresh pastry in the morning is the best

warm donuts when I wake up are awesome

Concept Space

my dog walks with me to get donuts

Page 5: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

I enjoy walks with my dog

Language Embedding

Language Embedding

ie. 1 billion google news articles

concept space

Train on corpus

Page 6: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

I enjoy walks with my dog

Language Embedding

Language Embedding

0.1, 0.50.8, 0.6

concept space

high dimensional vector

Page 7: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Semantic Similarity

concept space

I enjoy walks with my dog

I like to stroll with my k9

Language Embedding

Language Embedding

Page 8: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Semantic Similarity

concept space

I enjoy walks with my dog

I like to stroll with my k9

Language Embedding

Language Embedding

= Similarity (high)

Page 9: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Semantic Similarity

concept space

I enjoy walks with my dog Language Embedding

Language Embedding

= Similarity (low)

warm donuts when I wake up are awesome

Page 10: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Semantic Clustering

I enjoy walks with my dog

I like to stroll with my k9

fresh pastry in the morning is the best

warm donuts when I wake up are awesome

concept space

LanguageEmbedding

LanguageEmbedding

LanguageEmbedding

LanguageEmbedding

Page 11: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Semantic Clustering

I enjoy walks with my dog

I like to stroll with my k9

fresh pastry in the morning is the best

warm donuts when I wake up are awesome

concept space

•k-means•DBSCAN•hierarchical clustering•spectral clustering•etc.

Relevant Algos

Page 12: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Semantic Attribute Analysis

Trump is blunt and honest, and has some skills for success, but how he talks about

women makes me feel really uncomfortable.

Language Embedding

concept space

Neural Network

Semantic Attributes

ability

empathy

personality

authenticity

similarity

Page 13: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Semantic Attribute Analysis

Says it like it is. Honest. Blunt.

concept space

Neural Network

Semantic Attributes

ability

empathy

personality

authenticity

similarity

Language Embedding

Page 14: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Semantic Attribute Analysis

Page 15: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Linguistic Reasoning

•Memory Networks (MN)•Neural Turring Machines (NTM)

When does Sally cook?

A.I.

Sally cooks on weekends and holidays

Sally cooks when she has free time

Sally has free time over the holidays

Sally has free time on the weekendsRelevant Algos

Page 19: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Conversation Automation

Templated Conversation

A.I.

A.I. A.I.

•Preprogramed flow by a human•Assisted by NLP/ML algos•Limited scope of conversation•Templated responses

•Semantic Classification •Entity Extraction

Page 20: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Conversation Automation

A.I.

the taste is weird

LSTM LSTM LSTM LSTM

Thought Vector (in concept space)

LSTM LSTM LSTM

explain the taste

Seq2seq RNN•Trained on real conversation data•Can cover wide scope of dialogue•Does not use context

Page 21: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Conversation Automation

A.I.

Seq2seq RNN with context•Trained on real conversation data•Can cover wide scope of dialogue•Does use context via context vector

Previous Context Vector

the taste is weird

LSTM LSTM LSTM LSTM

Thought Vector (in concept space)

LSTM LSTM LSTM

explain the taste bob

Current Context Vector

LSTM

Page 22: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Conversation Automation

A.I.

Memory Networks•Trained on real conversation data•Can cover wide scope of dialogue•Context via NN addressable memory

Addressable Memory

the taste is weird

LSTM LSTM LSTM LSTM

Thought Vector (in concept space)

LSTM LSTM LSTM

explain the taste bob

Memory Vector

LSTM

NN

Page 23: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

Fundamental Question Gather Data Analyze Data Report.ppt

Automated conversation with participants

Automated analysisof conversations

Qualitative Research in the Age of Artificial Intelligence

Page 24: Andrew Konya - Al and qualitative moderation and analysis

[email protected]

A.I.

Researcher Employees,Consumers,

Experts

Thanks