1

Finding the Truth: Interview and Interrogation Training Simulations

Ron Punako Senior Software Engineer

2

CTC Overview • 501(c)(3) nonprofit established in 1987

• Staff of 1,400+ professionals

• More than 50 locations

• 900,000 sq. ft., including labs & demonstration space

• Top 100 Government Contractor

• Quality/EH&S Management System comprised of industry-best models: ISO 9001 (Quality) and 14001 (Environmental), AS9100 (Aerospace), and CMMI-SE/SW (Systems/Software Engineering)

• Nationally recognized security capabilities with 300,000+ sq. ft. of Top Secret/Sensitive Compartmented Information Facility Space, JWICS, SIPRNet, and NIPRNet access

3

Interview System

• Goals • Subject • Environment • Simulation of Body Language • Simulation of Expression • Proxemics • Attention to Child’s Needs

4

Interview ILS Goals

• Develop a proof-of-concept simulation to train investigators to conduct interviews of child abuse victims.

• Develop a training scenario to benefit interagency organizations for both military and civilian applications.

• Provide a virtual interactive training module avatar system, with all necessary motion and appearance to project the behavioral indicators of abuse.

• Enable investigators to practice interviewing child abuse victims and to receive feedback after the training has reached completion.

5

Subject: Cynthia Baker

Physical profile • 6 years old • Petite • Brown hair • Brown eyes

6

Environment: Child’s Interview Room

Playroom environment • Sofa, adult chairs, child chairs • Toys

• Posters • One way glass

7

Simulation of Body Language

Shrugging off Coloring

Casual dialogue Closing off/hiding

Challenge: Communicate emotional state using body language • Designed 20 animations to

communicate emotional state • Body animations designed to be

blended across 2 layers including idle and emotive

8

Simulation of Facial Expression

Happy

Challenge: Attempt to accurately portray facial expression

• Designed for sadness, fear, anger, indignant, disgust, happiness, confusion and shame across 4 layers including: blink, idle, lip-sync and emotive

• Selected expressions modeled after Facial Action Coding System action unit combinations.

Angry

9

Proxemics

• First question deals with seating arrangements relative to Cynthia

• Experimented with the Xbox 360 Kinect controller to provide Proxemics

10

Attention to Cynthia’s Needs

Challenge: Simulating the special needs of a child interviewee • Talking at the child’s level • Sitting at the child’s level • Paying attention to the child's special

needs (requests for mother, playing and coloring)

Playing under the table

11

Interrogation System

• Goals

• Subject

• Environment

• Simulation of Body Language

• Simulation of Voluntary Facial Expression

• Simulation of Micro Expression

• Detection of Micro Expression

• Continuation of Disposition/Personality

12

Interrogation ILS Goals

• Develop a proof-of-concept simulation to train investigators to conduct interrogation of criminals suspected of sexual assault.

• Develop scenarios that can be manipulated to provide challenging interrogation exercises that are real and relevant to the current threat of perpetrators of serious crimes against persons.

13

Subject: Sergeant Mike Hagan Physical profile • 30 years old • 71 inches • 156 lbs • Brown hair • Brown eyes

Criminal record • First time offender

14

Environment: Interrogation Room

Minimalist design • One table

• Security Camera

• Two chairs

15

Simulation of Body Language

Shrugging off Stop pressuring me

Defensive rebuke Closing off/hiding

Challenge: Communicate emotional state using body language

• Designed 35 animations to

communicate emotional state • Body animations designed to be

blended across 2 layers including idle and emotive

16

Simulation of Voluntary Facial Expression

Happy Angry

Worried Disgusted

Challenge: Attempt to accurately portray facial expression with a fidelity model.

• Designed for sadness, fear,

anger, contempt, disgust, happiness and surprise across 4 layers including: blink, idle, lip-sync and emotive

• Selected expressions modeled after Facial Action Coding System action unit combinations.

17

Simulation of Micro Expressions Challenge: Can the architecture handle extremely fast animation transitions and blending?

• Designed for sadness, fear,

anger, contempt, disgust, happiness and surprise

• Achieved required 1/25 second micro expression duration.

Worried micro expression

18

Detection of Micro Expressions Challenge: How to integrate a micro expression mini-game into the simulation? • A possible micro expression occurs while

a detect button is shown • Not a test of dexterity • False-positives possible.

Worried micro expression

19

Continuation of Disposition/Personality Disposition Threshold

Disposition Visual Output

5+ Angry/ Uncooperative

2-4 Upset

0-1 Idle/ Cooperative

Disposition Modifiers (First time offender) Red = +1 Yellow = +0 Green = -1 Disposition Modifiers (Repeat offender) Red = -1 Yellow = +0 Green = +1

20

Common Design Elements: Decision Sequencing

• Linear - sequencing of events with no opportunity for deviation from sequence trunk.

• Branching - sequencing of events that allows deviation from the sequence trunk.

• Recursive – revisit previous events.

• Any order – events presented in a discreet group in any order.

21

Common Design Elements: Decision Making

Select a response which would help assess the child’s episodic memory level. A. You were telling me about your Christmas (Green = Best) B. I’ll bet Christmas day was fun, right? (Yellow = Average) C. Specifically, where were you and what did you do Christmas day (Red = Poor)

22

Common Design Elements: Performance Scoring

Learner makes a choice Score is averaged by objective and recoded

Final score is averaged across objective scores

Objective 1: 50% Objective 2: 100% …

Final: 75%

1. Choice A 2. Choice B 3. Choice C

Learner makes a choice Feedback is recorded by objective.

Feedback is provided to learner

Objective 1: Good Job! …

Good Job! You chose choice A. Choice A was best because…

1. Choice A 2. Choice B 3. Choice C

Quantitative

Qualitative

23

Common Design Elements: Confession or Disclosure

Challenge: How to simulate a meaningful confession/disclosure? • Culminating point in final act • Reached through majority green and

yellow path decisions • Must avoid terminal red path

decisions

24

Common Design Elements: Notebook

Challenge: How to provide the learner with an effective record of kinesics exhibited? • Running log of all kinesics, dialogue,

choices, decisions and mentor feedback

• Designed for learner reflection, performance review and memory/analysis aid

25

Common Design Elements: After Action Review

Summary Panel • Rollup of objectives • Scored or un-scored against objective thresholds

Details Panel • For each decision displays screen shot, decision

response, related objective and mentor feedback

26

Technical and Integration Details

• Work with internal and client SME to develop a script that details what content (objectives, voice over, feedback, remediation, interactivity, others) to integrate

• Staff works to produce content • Custom development pipeline built on the Unity Game Development Tool

supports integration of content. • Simulations currently deployed to Web, PC, Mac, iPad

27

Next Steps

• System under evaluation for a Certificate of Networthiness • Pursuing evaluation by learners at Fort Leonard Wood, MO • Advanced feature integration

– Full implementation of continuation of disposition/personality – Implementation of motion capture for Proxemics – Photo-realistic body language and facial expression – Improve development pipeline

• Evaluate potential for application in broader industry.

28

Questions?

Demo?

29

Technical Point of Contact:

Ron Punako Senior Software Engineer

814-269-6538 punakor@ctc.com

Business Development Point of Contact:

David A. Kingston, P. E. Director, Learning and Human Performance Solutions

573-329-8548 kingstod@ctc.com