Research on Development of Intelligent Tutoring Systems (ITS)
to Support Embedded Training (ET)in Future Army Systems
presented by: Henry Marshall
RDECOM Simulation and Training
Technology Center (STTC)
2
Outline
• Background in ET
• Proposed ITS Design/ Intelligent Structured Training Concept
Applications
• C2V Robotics Testbed
• CAT ATD Testbed Experiment
• Virtual Warrior Testbed
• Issues and Conclusion
3
Embedded Combined Arms Team Training and Mission
Rehearsal ATO
ATO Researching Solutions to TRADOCs #1 Training Technology Gap – Mounted/Dismounted Interoperable Embedded Training
Vision: Embedded Training & Mission Rehearsal for Combined Arms, Mounted & Dismounted Forces with
Embedded AAR
Current
• Sand Tables for Mission Rehearsal• Training systems that are difficult or impossible to deploy
Future – Interoperable Mounted & Dismounted Embedded Training
MountedEmbedded
Training
MountedEmbedded
Training
DismountedEmbedded
TrainingSolutions
DismountedEmbedded
TrainingSolutions
IntelligentTutoring
IntelligentTutoring
Embedded Mission Rehearsal & AAR
Embedded Mission Rehearsal & AAR
Low-CostInnovative
Dismounted EmbeddedTraining Solutions
Low-CostInnovative
Dismounted EmbeddedTraining Solutions
Supports FCS & FFW Programs
RoboticEmbedded Training
RoboticEmbedded Training
4
ITS and ET, a Good Match?• Training (to include Embedded Training) is a Key
Performance Parameter for the Future Combat System (FCS) and Ground Soldier Systems (GSS). Also requirement for Abrams, Bradley and Stryker.
• ET intent is to fully embed training system on the operational platforms
• The instructional staff at current Army fixed sites will likely not be available for deployed forces
• Can ITS-based technology be integrated with current simulation common components to replace the role of instructors for ET?
5
FCS/ET – Training Challenge
• New paradigm requires scenario-based practice for FCS warfighters
• CCTT has successfully used a Structured Training concept for Basic Techniques, Task and Procedures Structured Training for CCTT STRUCTT
• Formal tactical doctrine for FCS operational concept is still evolving
• Desirable to minimize costs of developing and administering training – reduce requirements for human instructors and simplify scenario definition
• ITS are effective for simulating some of the benefits of a human instructor, especially for a domain with focused, task-based exercises
• Enter our ITS research to prototype possible solutions
6
Intelligent Structured Training?
• Goals are to– Maximize Simulation Common Components in developing ITS
system
– Operate in typical virtual training environment
– Develop a system capability of replacing instructors where possible
• Assumption – Because of the complexity of free play exercises and the number of
possible solutions, ITS would be useable for only a limited set of predefined training scenarios
• Answer– Develop an ITS-based system that supports finite state transitions
and provides prompts and feedback, operating in a virtual training scenario. We have named this Intelligent Structured Training (IST)
7
Finite State Machine Evaluations
What are they?• Transition networks executing in coordination with a simulation to
gather data about instructionally significant events and states, and make evaluation conclusions in real time
Why use them in an ITS?• Several benefits:
– Modularity – they can be used separately or in conjunction for a variety of scenarios
– Instructional correspondence – individual instructional principles can be associated with independent evaluations
– Integration – the FSM structure is easily integrated with free-play simulations and maps well to diagnostics for widely varied outcomes
– Authoring ease – they can be represented visually, making them easy for non-programmers to create, maintain, and revise
– Application implements as a Behavior Transition Network (BTN)
8
ITS FSM Evaluation Example
TACTICAL: Before cresting hills, halt unmanned ground vehicles (UGV) and use mast sensors to scan for enemy
9
Intelligent Structured Training Concept
• Goal: Provide the benefits of
instructor-led training, in an
embedded setting
• Methods
– Based on Intelligent Tutoring System (ITS) technology
– Intelligent agents perform automated evaluation during execution
• Subject matter experts define agent behaviors
• Behaviors defined in hierarchical behavior transition networks (BTN)
– Real-time feedback, hinting, or coaching presented in Soldier Machine Interface
• Must operate on small “footprint” of embedded computer systems
10
C2V Experiment
• Credit LTC Mike Sanders FA 57, [email protected]
• Research based on needs of the FCSprogram to provide embedded trainingto deployed forces w/o the instructorsof a training facility
• Prototype based on virtual task training for a robotics NCO duty station
• Explore mechanisms for feedback to the trainee• Explore interfaces to OneSAF Testbed for ITS control of
OPFOR / BLUFOR to facilitate training
11
Command and Control Vehicle Crewstation Based on CAT ATD
C2V Testbed
CAT ATD Testbed
12
Command and Control Vehicle Crewstation
UAV Sensor view of the SyntheticTraining Environment (STE)
UGV Sensor view of the STE (Driver’s Position)
UGV Sensor view of the STE (Gunner’s Position)
13
Command and Control Vehicle Crewstation
Robotic Assets/Mission Status Tool
OCU/Situational Awareness Map
Tele-Operation Asset Tool
14
ITS Generated Feedback Prompt
Immediate Directive Feedback (Procedural Error Prompt)
15
Approach and Scenario Overview
• Task Analysis for FCS equipped Unit of Action (UA)
• Required Functional Capabilities include sensor fusion and engagement techniques
• User ~ Robotics Operator in the C2V at the Company level
• Scenario ~ Route Reconnaissance
• ITS modified to trigger OneSAF behaviors via DIS
16
Robotics Operator Tasks• Coordinated use of robotic assets
– Example: Maintain proper separation between air asset and ground vehicle
• Proper reporting procedures– Example: Send SITREP after reaching a control measure
• Proper engagement procedures– Example: Lase a target before sending call for fire
• Proper use of asset control tools– Example: Make sure a vehicle is currently
being controlled before issuing commands in the control interface.
17
Experimental Design• Test group: 20 subjects
• Comparison conditions– Immediate Directive Feedback (IDF) only
• ITS generated feedback through prompts
– AAR Only (Delayed Feedback) • Human facilitated AARs used open-ended, content neutral prompts
• Experiment phases– Training and test phase
• Initial human-tutored and computer-aided instruction
• Two-phased execution
• Paper and pencil test
– Retention and Post-test phase after 1 week delay
18
Experimental Results
Retention Scores for Procedural Knowledge Following Feedback
M SD
IDF Only 3.72 .92
AAR Only 3.80 1.21
Procedural Errors Following Feedback
M SD
IDF Only 28.22 19.50
AAR Only 42.08 24.59
Retention Scores for Conceptual Knowledge Following Feedback
M SD
IDF Only 3.10 2.03
AAR Only 5.60 2.12
19
Analysis
• The timing and type of feedback received during training does affect the acquisition, retention and transfer of knowledge
• Procedural knowledge – Retention scores elevated in both comparison conditions
– Lower number of errors with Immediate Directive Feedback
• Conceptual knowledge– Retention scores elevated in both comparison conditions
– Higher retention scores with AAR
• Bottom line – proven learning from embedded ITS feedback!
• This could provide deployable training and save $$$ if authoring cost were economical
• Improved Pre-Brief or Postbreif capability could help improve the ITS Conceptual Knowledge retention
20
TARDEC CAT ATD Integration
• Goal Integrate ITS system in actual FCS suggatate
• Considerable TARDEC interest in ITS
• Performed interface evaluation via Engineering Evaluation Tests (EET)
• CAT ATD uses AKit/BKit architecture where A kit relates to vehicle unique, B Kit training simulation unique.
• Issues with deep integration and getting messages to displays.
21
Virtual Warrior Experiment
• Credit Major Jason Sims FA 57, [email protected]
• Explore integration of Intelligent Structured Trainer to the Virtual Warrior Dismounted ET Man Wearable Prototype
• Examine ITS scenario authoring tools to construct training scenarios
22
Authoring Tool Design
• Provide Visualization tool for 3D Placement of scenario objects
• Correlate objects to evaluations
• Format to exchange ITS related data to OTB was a issue
• Terrain collelation was a issue
• Used S2 Focus because of relation to VW
23
VW-ITS Design
• ITS system integrated into VW System to evaluate Ground Soldier System (GSS) ET prototype
• Evaluation to move as a fireteam w/VW playing one of the team rest OTB
• Evaluation of Sending Reports as needed• Evaluation of room clearing • Evaluation of separation and sectors of fire
24
VW-ITS Issues
• Numerous problems with OTB SAF behaviors filling out the rest of the Squad and OPFOR
• Evaluations limited by troop availability• Most prefer system that would allow most of the
squad to be live (e.g. team training) as opposed to interacting with SAF
• Virtual Locomotion of VW not liked, e.g. ability to move around database. Exposure to fire also a concern.
• Difficultly sending messages w/ C2 system
25
Issues
• Integration/Interoperation with OOS and Training Common Components
• Best Future Direction, Game SDK vs. Training Common Components?• Mix with a Operational Coach Mentor?• Experiment with Enhanced / Muti-Modal feedback for ITS• Integration into vehicles information/ET systems• Transition with focus to PM - systems we developed were focused on
experimentation• Authoring systems for ease of production and usability by topic SMEs.
Low Software License Costs.• Production costs per tasks will drive economics of ITS acceptance.• Explore Team Training• Improved CGF for ITS for control of OPFOR/BLUFOR behaviors • Exercise automated Pre-Brief/Post Brief • Ability to adjust difficultly level either before execution or dynamically
26
Discussion??
Recommended
