人類內在感受與外在表現落差時之語音、生理特徵表現

Group 05

B05901192 張晁維

B05901017劉昶樂

B05901021賴明緯

問題意識

Stefano Sandrone et al.” Weighing brain activity with the balance: Angelo Mosso’s original manuscripts come to light”, Brain, Volume 137, Issue 2, 1 February 2014, Pages 621–633

初代測謊機

International League of Polygraph Examiners

J E REID; F E INBAU. TRUTH AND DECEPTION - THE POLYGRAPH ('LIE-DETECTOR') TECHNIQUE, 2D ED.

認知心理學

Vrij A. et al.( 2008). Increasing cognitive load to facilitate lie detection: The benefit of recalling an event in reverse order. Law and Human Behavior, 32, 253–265.

研究現況與面向

• Deception and behavior

• Cognitive load approaches

• Systematic verbal lie detection approaches

• Psychophysiological detection of deception

Deception and Behavior

• observation of behavior is hard to detect deception

(Charles F. Bond Jr., Bella M. DePaulo (2006))

• connection between lying and non verbal cues is weak

(Sporer, S. L., & Schwandt, B. (2007))

• meta analytical evidence question the utility of behavior as a source of deception markers

Cognitive load approaches

• Aim at designing interview strategies oriented to produce behavioral differences between truth tellers and liars

• Vrij et al. (2010) argued that creating a lie might require more cognitive effort than just describing an episodic memory

• Strategies like imposing cognitive load(e.g. describe the event in reverse order, stare at the interviewer’s eyes, etc.), encouraging interviewee to say more, and asking unexpected questions.

(Vrij, Fisher, & Blank, 2017)

• Pros: stronger theoretical bases (Walczyk et al.’s (2014))

Systematic verbal lie detection approaches

• Reality Monitoring(RM)

• Criteria-Based Content Analysis(CBCA)/Statement Validity Assessment(SVA)• a clinical assessment procedure rather than a standardized psychometric test.

• Both RM and CBCA/SVA are based on the notion that the verbal descriptions of self-experienced events differ from those of imagined or invented events.

• Pros: high-quality field studies for forensic cases

Psychophysiological detection of deception

• Comparison Question Test (CQT) and Concealed Information Test (CIT) with Polygraph• CQT: examinee are asked with a series of relevant, irrelevant, and comparison questions, and

they are expected to display strong physiological responding just after relevant one.• CIT: examinee are asked with a series of multiple-choice questions, and only those who have

knowledge about the crime details show stronger physiological action to correct alternatives than incorrect ones.

• Event-Related Potentials(EPRs):• examine EEG waves that appears accompanied with the recognition of meaningful.(Iacono, 2015; Rosenfeld, 2011)

• fMRI:• Compare and identify truth-teller and liar by observing fMRI brain activated area during

deception(Langleben et al., 2016)

Psychophysiological detection of deception

• Cons: • All of these method are vulnerable to countermeasures, and thus measure

methods need revisions

• sensitivity and specificity of polygraph depends on examiner using CQT or CIT

• ERPs and fMRI classification rate are not always better that those obtained by polygraph.

• Pros: • fMRI and ERPs provides more data for further analysis

• Developing potentials

語音壓力偵測Voice Stress Analysis (VSA)

• Jitter• 微肌群震顫(micro-muscle tremors,MMT)-Lippold 1971• 隨意肌的收縮正常時，以每秒大約 10 個週期小幅振盪• 說謊時，由於壓力，使肌肉協調性改變(喉嚨肌肉拉緊)，導致在 8~12Hz 的生理性震顫減少

• PSE、CVSA、LVA系統

• Pitch• 75~500Hz• (a)無壓力 (c)有壓力(平坦)

Lippold, O., "Physiological tremor", Scientific American, vol. 224, pp. 65-73, 1971.

CSC DatasetDistinguishing deceptive from non-deceptive speech – Hirschberg et al.

• 32 hours of speech data from 32 English speakers

• Task: To convince the interviewer that they possessed thecharacteristics of a successful entrepreneur. Anytime they answeredthe interviewer’s question with a lie, the subject would press a pedalunder the table indicating the truth value of their statement.

• Without ”high stakes” scenarios (the subject is very likely toexperience fear or shame, such as testifying in court after swearing onthe Bible)

• Only able to marginally improve their error rate with further featureconstruction/selection

Detecting Lies via Speech Patterns – Chow et al

LR = Logistic regression, SVC = support vector classifier, GB = gradient boosting classifier, MLP = multi-layer perceptron classifier, BAG = bagging classifier and RNN = recurrent neural network;

Neural Lie Detection with the CSC Deceptive Speech Dataset-Shloka Desai et al.• PCA features

• RNN-based

Neural Lie Detection with the CSC Deceptive Speech Dataset-Shloka Desai et al.

A Deep Learning Approach for Multimodal Deception Detection – Krishnamurthy et al(Mar/2018)

• Use textual, audio and visual features.• Visual Feature Extraction-3D CNN

• Textual Features Extraction- pretrained Word2Vec + CNN

• Audio Feature Extraction – openSMILE (open-source toolkit used to extract high dimensional features from audio)

• Micro-Expression Features

• MLP_U: Unimodal

• Data Fusion: MLP_C – Concatenation; MLP_H+C - Hadamard + Concatenation

What We Want To Do?

• Detecting deception use ONLY audio signals by deep learning.

Detection of Pathological Voice Using CepstrumVectors: A Deep Learning Approach - Fang et al

預估使用器材

全指向性電容麥克風 NTD 289 3支

USB音效卡 NTD 80 3件

HUB集線器 NTD 339 1件

Reference

• https://www.ncbi.nlm.nih.gov/pubmed/26794034

• https://onlinelibrary.wiley.com/doi/full/10.1111/lcrp.12088

• http://www.psicothema.com/PDF/4376.pdf

• https://www.ncbi.nlm.nih.gov/pubmed/26787599

• http://psycnet.apa.org/record/2007-13995-005

• http://psycnet.apa.org/record/2007-01724-001

• http://journals.sagepub.com/doi/abs/10.1207/s15327957pspr1003_2

• https://people.ok.ubc.ca/stporter/Publications_files/Pitfalls%20and%20Opportunities-FINAL.pdf

• https://www.sciencedirect.com/science/article/pii/S0732118X14000142