Virtual Environment Tutoring Engine: ITSs for 3D Game Environments

Shrenik DevasaniStephen Gilbert, Ph.D.

Suhas ShettyNandhini Ramaswamy

Stephen Blessing, Ph.D.

Problem

Authoring game-based tutors is challenging due to:

– Inherent domain complexity

– Dynamic nature of the environment, non-player inputs

What’s missing:

– Authoring tools that help non-programmers develop game-based ITSs

– Generic authoring tool to create tutors for multiple game engines / simulated environments

Previous Work - XPST

Extensible Problem Specific Tutor (xPST) – ITS authoring tool that supports tutoring within synthetic environments.

Tutor author lists the sequence of steps or “subgoals” to be performed by the trainee and describes the feedback associated with each step.

xPST could be used by non-programmers.

Previous Work - XPST

sequence {(shoot-the-kork then pickup-the-crossbows) then All-Done;

}

shoot-the-kork {answer: "1";Hint: "Go find the Kork and shoot it once.";Hint: "A green crosshair shows you have correctly aimed at the Kork.";OnComplete: "You have shot the Kork!";

}

Previous Work - XPST

Goalnodes lack sufficient power to model a dynamic environment

Non-player entities’ actions were difficult to represent

Events that occur without the player’s knowledge were difficult to account for

Solution

Tutoring model is a collection of states

– Atomic state Represents the physical state of the real world

– Complex stateRepresents a cognitive state

Helps model complex tutoring strategies

Can account for full range of possibilities within the environment

Designed for non-programmers

DesignAn atomic state is described by the values of the properties of the entities in the scenario.

Every property can take a special value called “don’t care” (DC).

Complex states help model events that happen over time.

A complex state is a pattern of one or more atomic states, described by a regular expression.

– e.g. S1 (S2 | S3)* S4

One complex state can encapsulate another.

– e.g. (S1 S2)* S3 encapsulates S2 S3

Every state is associated with a priority.

Design

Every state is classified as one of:

– Start State

– Failed State

– Goal State

– Intermediate State

– Response State

Design

Design

Each state is associated with 3 kinds of feedback:

– Hints

– Just in time messages

– Prompts

DesignCounter variables:

– Help keep track of time and speed

– Updated when the learner enters or leaves a state

– Can be used as pre-conditions to enter a state

Learner model:

– Every state can be tied one or more skill variables

Simulation Engine: VBS2Virtual Battlespace 2 – a commercial-off-the-shelf 3D military simulator

Scenarios can be created using the VBS2 Mission Editor by simple drag-drop of entities

Architecture

Tutor Authoring Process

– First step is to create the scenario using the VBS2 Mission editor

– Next, Define locations in the scenario that would be interest

– Once the scenario is ready, the tutor-author can start building the tutoring model using VETE

ExampleConsider an example scenario consisting of the learner located outside a building (B5), an insurgent, a civilian and an IED all located inside B5.

In order to complete the mission, the learner must:

– Run towards the nearest wall

– Approach B5, staying close to the wall at all times

– Kill the insurgent

– Diffuse the IED

ExampleStart State:

LearnerisAlive {true}Location {!B5}Action {stand}

InsurgentisAlive {true}Location {B5}Action {DC}

CivilianisAlive {true}Location {B5}Action {DC}

IEDDiffused {false}Exploded {false}Location {B5}

ExampleInsurgentDead (Intermediate State) :

LearnerisAlive {true}Location {DC}Action {DC}

InsurgentisAlive {false}Location {DC}Action {DC}

CivilianisAlive {true}Location {B5}Action {DC}

IEDDiffused {false}Exploded {false}Location {DC}

ExampleIEDDiffused (Goal State) :

LearnerisAlive {true}Location {DC}Action {DC}

InsurgentisAlive {false}Location {DC}Action {DC}

CivilianisAlive {true}Location {!B5}Action {DC}

IEDDiffused {true}Exploded {false}Location {DC}

ExamplePlayerDead (Failed State) :

LearnerisAlive {false}Location {DC}Action {DC}

InsurgentisAlive {DC}Location {DC}Action {DC}

CivilianisAlive {DC}Location {DC}Action {DC}

IEDDiffused {DC}Exploded {DC}Location {DC}

Example

Complex State: “LearnerConfused”

– (Start NearWindow) {3, }

– Represents the learner being in a confused state of mind, when he switches between the two states, three or more times

– “LearnerConfused” can be assigned a higher priority than “NearWindow”

Example

Discussion

State-transition definitions are not required.

Tutor-author only needs to define a subset of possible states.

VETE’s architecture allows extension to other simulation engines.

Future Work

Empirical evaluation study to demonstrate that VETE is feasible for non-programmers

Training based on real-time physiological data

Support for “programming by demonstration”

Learner Management System

Questions?

Shrenik DevasaniStephen Gilbert, Ph.D.

Suhas ShettyNandhini Ramaswamy

Stephen Blessing, Ph.D.