Deceptive Speech

Frank Enos • April 19, 2006

Defining Deception

Deliberate choice to mislead a target

without prior notification (Ekman‘’01)

Often to gain some advantage

Excludes: Self-deception Theater, etc. Falsehoods due to ignorance/error Pathological behaviors

Why study deception?

Law enforcement / Jurisprudence

Intelligence / Military / Security

Business

Politics

Mental health practitioners

Social situations Is it ever good to lie?

Why study deception?

What makes speech “believable”?

Recognizing deception means recognizing

intention.

How do people spot a liar?

How does this relate to other subjective

phenomena in speech? E.g. emotion,

charisma

Problems in studying deception?

Most people are terrible at detecting deception

— ~50% accuracy

(Ekman & O’sullivan 1991, Aamodt 2006, etc.)

People use subjective judgments —

emotion, etc.

Recognizing emotion is hard

People Are Terrible At This

Group #Studies #Subjects Accuracy %

Criminals 1 52 65.40

Secret service 1 34 64.12

Psychologists 4 508 61.56

Judges 2 194 59.01

Cops 8 511 55.16

Federal officers 4 341 54.54

Students 122 8,876 54.20

Detectives 5 341 51.16

Parole officers 1 32 40.42

Problems in studying deception?

Hard to get good data Real world (example) Laboratory

Ethical issues Privacy Subject rights Claims of success

But also ethical imperatives: Need for reliable methods Debunking faulty methods False confessions

20th Century Lie Detection

Polygraph http://antipolygraph.org

The Polygraph and Lie Detection (N.A.P. 2003)

Voice Stress Analysis Microtremors 8-12Hz Universal Lie response http://www.love-detector.com/ http://news-info.wustl.edu/news/page/normal/669.html

Reid Behavioral Analysis Interview Interrogation

Frank Tells Some LiesAn Example…

Frank Tells Some Lies

Maria: I’m buying tickets to Händel’s Messiah for me

and my friends — would you like to join us?

Frank: When is it?

Maria: December 19th.

Frank: Uh… the 19th…

Maria: My two friends from school are coming, and

Robin…

Frank: I’d love to!

How to Lie (Ekman‘’01)

Concealment

Falsification

Misdirecting

Telling the truth falsely

Half-concealment

Incorrect inference dodge.

Frank Tells Some Lies

Maria: I’m buying tickets to Handel’s Messiah for me

and my friends — would you like to join us?

Frank: When is it?

Maria: December 19th.

Frank: Uh… the 19th…

Maria: My two friends from school

are coming, and Robin…

Frank: I’d love to!

• Concealment

• Falsification

• Misdirecting

• Telling the truth falsely

• Half-concealment

• Incorrect inference dodge.

Reasons To Lie (Frank‘’92 )

Self-preservation

Self-presentation

*Gain

Altruistic (social) lies

How Not To Lie (Ekman‘’01)

Leakage Part of the truth comes out Liar shows inconsistent emotion Liar says something inconsistent with the lie

Deception clues Indications that the speaker is deceiving Again, can be emotion Inconsistent story

How Not To Lie (Ekman‘’01)

Bad lines Lying well is hard Fabrication means keeping story straight Concealment means remembering what is omitted All this creates cognitive load harder to hide emotion

Detection apprehension (fear) Target is hard to fool Target is suspicious Stakes are high Serious rewards and/or punishments are at stake Punishment for being caught is great

How Not To Lie (Ekman‘’01)

Deception guilt Stakes for the target are high Deceit is unauthorized Liar is not practiced at lying Liar and target are acquainted Target can’t be faulted as mean or gullible Deception is unexpected by target

Duping delight Target poses particular challenge Lie is a particular challenge Others can appreciate liar’s performance

Features of Deception

Cognitive Coherence, fluency

Interpersonal Discourse features: DA, turn-taking, etc.

Emotion

Describing Emotion

Primary emotions Acceptance, anger, anticipation, disgust, joy,

fear, sadness, surprise

One approach:

continuous dim. model (Cowie/Lang)

Activation – evaluation space

Add control/agency

Primary E’s differ on at least 2 dimensions of this

scale (Pereira)

Problems With Emotion and Deception

Relevant emotions may not differ much on

these scales

Othello error People are afraid of the police People are angry when wrongly accused People think pizza is funny

Brokow hazard Failure to account for individual differences

Bulk of extant deception research…

Not focused on verifying 20th century

techniques

Done by psychologists

Considers primarily facial and physical cues

“Speech is hard”

Little focus on automatic detection of

deception

Modeling Deception in Speech

Lexical

Prosodic/Acoustic

Discourse

Deception in Speech (Depaulo ’03)

Positive Correlates Interrupted/repeated words References to “external” events Verbal/vocal uncertainty Vocal tension F0

Deception in Speech (Depaulo ’03)

Negative Correlates Subject stays on topic Admitted uncertainties Verbal/vocal immediacy Admitted lack of memory Spontaneous corrections

Problems, revisited

Differences due to: Gender Social Status Language Culture Personality

Columbia/SRI/Colorado Corpus

With Julia Hirschberg, Stefan Benus, and colleagues from SRI/ICSI and U. C. Boulder

Goals Examine feasibility of automatic deception

detection using speech Discover or verify acoustic/prosodic, lexical,

and discourse correlates of deception Model a “non-guilt” scenario Create a “clean” corpus

Columbia/SRI/Colorado Corpus

Inflated-performance scenario

Motivation: financial gain

and self-presentation

32 Subjects: 16 women, 16 men

Native speakers of Standard American English

Subjects told study seeks to identify people who

match profile based on “25 Top Entrepreneurs”

Columbia/SRI/Colorado Corpus

Subjects take test in six categories: Interactive, music, survival, food,

NYC geography, civics

Questions manipulated 2 too high; 2 too low; 2 match

Subjects told study also seeks people who can convince interviewer they match profile Self-presentation + reward

Subjects undergo recorded interview in booth Indicate veracity of factual content of each utterance using

pedals

CSC Corpus: Data

15.2 hrs. of interviews; 7 hrs subject speech

Lexically transcribed & automatically aligned

lexical/discourse features

Lie conditions: Global Lie / Local Lie

Segmentations (LT/LL):

slash units (5709/3782), phrases

(11,612/7108), turns (2230/1573)

Acoustic features (± recognizer output)

um i was visiting a friend in venezuela and we went camping

Columbia University– SRI/ICSI – University of Colorado Deception Corpus: An Example Segment

Breath GroupSEGMENT TYPE

LABEL

ACOUSTIC FEATURES

LEXICAL FEATURES

LIE

max_corrected_pitch 5.7mean_corrected_pitch 5.3pitch_change_1st_word -6.7

pitch_change_last_word -11.5normalized_mean_energy 0.2unintelligible_words 0.0

Obtainedfrom subject

pedal presses.

has_filled_pause YESpositive_emotion_word YESuses_past_tense NO

negative_emotion_word NOcontains_pronoun_i YES verbs_in_gerund YES

Produced usingASR output

and otheracoustic analyses

Produced automaticallyusing lexicaltranscription.

LIEPREDICTION

CSC Corpus: Results

Classification (Ripper rule induction, randomized 5-fold cv) Slash Units / Local Lies — Baseline 60.2%

Lexical & acoustic: 62.8 %; + subject dependent: 66.4% Phrases / Local Lies — Baseline 59.9%

Lexical & acoustic 61.1%; + subject dependent: 67.1%

Other findings Positive emotion words deception (LIWC) Pleasantness deception (DAL) Filled pauses truth Some pitch correlation — varies with subject

Example JRIP rules:

(cueLieToCueTruths >= 2) and (TOPIC = topic_newyork) and (numSUwithFPtoNumSU <= 0) and (wu_ENERGY_NO_UV_STY_MAX__EG_ZNORM-ENERGY_NO_UV_STY_MIN__EG_ZNORM-D <= 5.846) => PEDAL=L (231.0/61.0)

(cueLieToCueTruths >= 2) and (numSUwithFPtoNumSU <= 1) and (wu_ENERGY_NO_UV_STY_MAX__EG_ZNORM-ENERGY_NO_UV_STY_MIN__EG_ZNORM-D <= 5.68314) and (wu_ENERGY_NO_UV_RAW_MAX-ENERGY_NO_UV_RAW_MIN-D >= 8.41605) and (wu_F0_SLOPES_NOHD__LAST >= -2.004) => PEDAL=L (284.0/117.0)

(cueLieToCueTruths >= 2) and (wu_F0_RAW_MAX >= 5.706379) and (wu_DUR_PHONE_SPNN_AV <= 1.0661) => PEDAL=L (262.0/115.0)

CSC Corpus: A Perception Study

With Julia Hirschberg, Stefan Benus, Robin Cautin and colleagues from SRI/ICSI

32 Judges Each judge rated 2 interviews Judge Labels:

Local Lie using Praat Global Lie on paper

Takes pre- and post-test questionnaires Personality Inventory Judge receives ‘training’ on one subject.

By Judge

58.2% Acc.

By Interviewee

Personality Measure: NEO-FFI

Costa & McCrae (1992) Five-factor model Openness to Experience Conscientiousness Extraversion Agreeability Neuroticism

Widely used in psychology literature

Neuroticism, Openness & Agreeableness correlate with judge performance

WRT Global lies.

These factors also provide

strongly predictive

models for accuracy at global lies.

Other Perception Findings

No effect for training Judges’ post-test confidence did not correlate

with pre-test confidence Judges who claimed experience had

significantly higher pre-test confidence But not higher accuracy!

Many subjects used disfluencies as cues to D. In this corpus, disfluencies correlate with TRUTH!

(Benus et al. ‘06)

Our Future Work

Individual differences Wizards of deception

Predicting Global Lies Local lies as ‘hotspots’

New paradigm Shorter Addition of personality test for speakers Addition of cognitive load