Automated UI Evaluation based on a Cognitive Architecture and UsiXML

Jan-Patrick Osterloh, Rene Feil, Andreas Lüdtke, Juan Gonzalez-Calleros

OFFIS – Institute for Information TechnologyR&D Division Transportation

Human-Centred Design GroupOldenburg, Germany

Motivation2Target UIs

07.09.2011

Motivation

► Main Cause for Aircraft Accidents is HUMAN ERROR

► Increasing Automation ► Pilots “forget” to Fly

3

► Over-reliance on Automation and non-understanding of Automatione.g. Air-France Accident

► Automation Surprises

► Taking Human Factors into account during System Design is major Challenge� HUMAN Project

07.09.2011

Photo credit: © Lars Langemeier/A.B./zefa/Corbishttp://www.pbs.org/wgbh/nova/space/pilot-room-for-error.html (Posted 2006/10/17; Boeing Study)

The HUMAN Project4

Objective: to develop a methodology with techniques and prototypical tools supporting the prediction of human errors in ways that are usable and practical for human centred designResults:

Overview

07.09.2011

• a cognitive crew model (PF/PNF) ⇒⇒⇒⇒ to reduce the effort of flight simulator tests by model-based simulation

• software tools supporting model application by human factor experts or system designers

• a methodology to evaluate cockpit system designs based on the cognitive model

PilotNon Flying

Pilot Flying

Virtual PilotNon Flying

Virtual Pilot Flying

Real Crew Interactswith the GECO system

Virtual Crew Interactswith the GECO system

GECO system incorporating a 4D Navigation System

Actual Activities Predicted Activities?

==

The HUMAN Project

► Cognitive Architecture CASCaS used as virtual Testers

► CASCaS = Cognitive Architecture for Safety Critical Task Simulation

► Implements psychological and physiological sound models of

5Virtual Testers - CASCaS

Cognitive Layer

Associative Layer

Short Term and Long Term

Memoryphysiological sound models of human behaviour, e.g. for► Calculation of eye-movements► Memory retrieval and forgetting► Knowledge Processing► Error Prediction:

► Routine Learning

► Cognitive Lockup

07.09.2011

Per

cept M

otor

Simulation Environment Wrapper

Autonomous LayerMemory

eyes

voice

gazehands

gazefeet

Auditory VisualTactile

The HUMAN Results6

Characteristic EvaluationTask Execution Time (AHMI Task) CRU

Task Execution Time (AHMI Task) APR

Gaze Distribution (all) CRU

Gaze Distribution (all) APR

14 Characteristics� 11 positive� 3 negative----------------------------

.78

Overall Fitness Value

Gaze Distribution (all) APR

Gaze Distribution (all) FIN

Gaze Distribution (AHMI Task) CRU

Gaze Distribution (AHMI Task) APR

Dwell Time (all) CRU

Dwell Time (all) APR

Dwell Time (all) FIN

Dwell Time (AHMI Task) CRU

Dwell Time (AHMI Task) APR

Reaction Time (AHMI Task) CRU

Reaction Time (AHMI Task) APR

Model Fitness = .78

0 1

CASCaS22.06.2011

07.09.2011

Automated UI Evaluation

► Standard in Industrial System Development:

► Model based System Design (e.g. Matlab, UML, State-Charts)

► Testing and Formal Verification

► Simulation

7Proposed Method

► HUMAN: Model Based Evaluation of System Design

► UsiXML: Model Based Design of User Interfaces

► IDEA: Integrate this in an Model Based Automated UI Evaluation Process

07.09.2011

8 Automated UI EvaluationProposed Method

UsiXMLDesign Toole.g. SketchiXML

System Design Tool

e.g. Matlab, Scade

UsiXML UI Prototype

SystemModel/Prototype

07.09.2011

Sim

ulat

ion

Env

ironm

ent

Sim

ulat

or

Online E

valuationCognitiveModel

UsiXMLRendering

Engine

PrototypeModel/Prototype

UsiXML

Resultse.g. Execution Time,

Workload, Guideline

violations

UsabilityA

dviser

9 Rendering Engine

RenderingEngineUsiXML

Repository

UI OntologyUsiXML Application Core (MVC)

ExternalLibraries

OpenGL

Rendering Objects

CUI

07.09.2011

UsiXML Concrete

User Interface

UsiXML Parser Application Core (MVC)

InterProcess

Com-munication

MathLibCore

Rendering Engine

OpenGL

GTK+

GLEW

QueryInput

Read/Write

Use

Use

10 Automated UI EvaluationRendering Engine

07.09.2011

Open Issues / Next Steps

► Open Issues:

► How to integrate System Model and UsiXML Model?

11

► Next Steps:

► Adding a “Rendering Library”

► Connection to CASCaS

► Online Transformation of Current Status for evaluation

► Connection to Online Evaluation / UsabilityAdviser

07.09.2011

The End

► Thanks for your attention!

► Questions?

12

► Questions?

07.09.2011

Outline

► Motivation

► System Development in Industry

► Target UIs

► The HUMAN Project

► HUMAN Method

► Virtual Testers

13

► Virtual Testers

► HUMAN Results

► Proposed Method Extension

► Current Status and Next Steps

07.09.2011

HUMAN Results - Example14Task Execution Time

07.09.2011