CREW EXPERT TOOL IO - spaceflight.esa.int · CREW EXPERT TOOL IO: Adjusting the Direction of the CET IO ... Cued-Recall-Debrief Final Presentation, ESTEC, 15 Sept 09 22. Day 1: Experts

Final Presentation, ESTEC, 15 Sept 09 [email protected]

CREW EXPERT TOOL IOto Support Crew Autonomous Operations in

Complex Human Spacecraft

Systems Engineering & Assessment Ltd in the framework of

European Space Agency contract 21584/08/NL/HE

Iya Whiteley

Olga Bogatyreva

Ed Soldat

Chris Johnson

Anne Bruseberg

Martin Townend

- SEA

- Univ. of Bath

- SEA

- Univ. of Glasgow

- SEA

- SEA

Final Presentation

ESTEC, 15 Sept 09

CREW EXPERT TOOL IO

WHY:

To investigate how to support crew in

problem-solving

To enable ESA to support autonomous

operation of the crew on long-duration

missions to the Moon and Mars

2Final Presentation, ESTEC, 15 Sept 09

CREW EXPERT TOOL IO:

Adjusting the Direction of the CET IO

ESA Parallel studies awarded

EADS Astrium, Science & Technology B.V., Space Applications

Services & TNO

SEA with Glasgow University & University of Bath

Direction of SEA’s study re-defined

WHY: To avoid duplication of the effort

CORE REMAINING:

To enable ESA to support autonomous operation of the crew on

long-duration missions

DIRECTION CHANGE TO:

To investigate how to support crew in problem-solving


Stages of the Study

TN1 – Problem solving process & review of techniques

Defined a range of problem solving crew functions in

current & future autonomous operations; confirmed

through user interviews

Reviewed tool & techniques to support them

TN2 – Iterative design & assessment process for

the Crew Expert Tool (CET IO)

Devised a matrix for definition of scenarios

Defined iterative design & assessment process with

user involvement


Stages of the Study

TN3 – Requirements baseline & technical specification for CET IO

Defined preliminary functional requirements for interactive CET IO

Defined preliminary user interface requirements for interactive CET IO

TN4 – Design & assessment of the CET IO proof-of-concept demonstrator

Designed & evaluated CET IO proof-of-concept demonstrator with astronauts

Provided recommendations for development & evolution process of CET IO


TN1: Expert Tool to Support Crew

Autonomous Operations in Complex

Human Spacecraft Context Definition & Literature Review

Columbus Manuals & Procedures – studying current problem-

solving related procedures & practices

General Problem-Solving theory

Studying Problem-Solving techniques & how experts execute them

in complex systems & LEO (e.g. Apollo, MIR, Shuttle, ISS)

User Interviews

Three Semi-structured interviews with astronauts

Identified current problem-solving activities in LEO

Identified user tasks for future autonomous operations

Definition of the initial Crew Expert Tool


TN1: Context of Autonomous Operations

in Complex Human Spacecraft

Future Operations Context

Failures will not always be covered by easily accessible

spare parts

Crew may lack the repair tools on-board

Mission control will not be always accessible, especially

as time-delay increases or due to communication failure

Crew will be mentally & physically stretched on a long-

duration mission

Missions do bring surprises:

• Crew may be cut off from life-critical resources - e.g. Apollo 13

• Space debris – e.g. Shuttle window cracked on STS-007


TN1: Context of Autonomous Operations

in Complex Human Spacecraft

Example of Crew Quotes:

Astronaut on where issues could originate:

“… other problems can come from issues of team performance rather than

component failure. Members of the crew may have missed a step in a key

procedure (e.g. release of the satellite) …

…or where miscommunication has occurred (e.g. capture of the satellite at

the wrong moment) .”

Astronaut on problem-solving tasks:

“… to solve a problem…

…The crew needs to have the capability to analyse the situation and…

…to analyse the situation they need to go through six steps with sub-steps…


TN1: Problem-Solving Cycle

in Theory & Practice

Comment about current Problem Solving strategy:

“… go through six steps with sub-steps…

Identify that something is wrong;

What is it that‟s wrong?

What is the criticality?

If time permits… solve

the problem or compensate

for side-effects;

What can I do eliminate the

progression of the problem;

What is the next failure?”

Define the nature of the

problem

Allocate mental &

physical resources

Represent the problem

mentally

Develop a strategy to

solve it

Evaluate the solution for

appropriateness &

correctness

Monitor progress

towards the solution

Recognise or identify the

problem

Organise knowledge

about the problem

1

2

3

4

5

6

7

8

PROBLEM SOLVING

CYCLE


TN1: Problem-Solving Cycle more suited

for autonomous crew


problem

Allocate mental &

physical resources


mentally


solve it


appropriateness &

correctness

Monitor progress



problem

Organise knowledge

about the problem

1

2

3

4

5

6

7

8

PROBLEM SOLVING

CYCLE

MonitorMaintain

Detect

Diagnose

Predict

Prevent

1

Additional Crew Activities

embedded in the cycle

based on

Literature Review &

User Interviews,

i.e. Astronauts


CET IO Design Concept

Selection of TRIZ for CET IO

Theory of Inventive Problem Solving

Relies on utilising existing

resources

Versatile range of techniques

to systematically identify

potential problems

Permits the definition of

multiple solutions



Contains seven problem

solving stages

Systematically takes the

expert from problem

definition to generation of

numerous solutions

Utilises & maximises users’

own expertise & creativity



problem

Allocate mental &

physical resources


mentally


solve it


appropriateness &

correctness

Monitor progress



problem

Organise knowledge

about the problem

1

2

3

4

5

6

7

8

PROBLEM SOLVING

CYCLE


CET IO Seven Problem-Solving Stages

Problem Detection

System & Context

Definition

Problem Location

Ideal Result

Resource Identification

Solution Definition

Solution Revision


TN2: CET IO – Iterative Design Process


problem

Allocate mental &

physical resources


mentally


solve it


appropriateness &

correctness

Monitor progress



problem

Organise knowledge

about the problem

1

2

3

4

5

6

7

8

PROBLEM SOLVING

CYCLE

Requirements

Specification Phase

Test PhaseDesign Phase:

Critical Design Review

Refurbish Phase

Operation Phase

Concept &

Feasibility Phase

Build Phase

1

2

3

4

5

6

7

8

ITERATIVE DESIGN

& LIFE CYCLE

Design Phase:

Preliminary Design

Review

Human Computer Interaction

Requirements (HCIR)

HCI Integration

HCI Evaluation

1

Operation Analysis

Training Design

HCI Evaluation &

Recommendations

Operation Log

Lessons

Learned

Camera &

Microphone

Field of

views

Analyst

Crew 1

Crew 2

Cre

w 3



CET IO Working Group

Objective: Produce & agree on the final design of CET IO

Represents all of stakeholders’ interest throughout the design process:

• Chief Designer

• Potential Users

• Customers

• System Engineers

• Cognitive Engineers

• Human Computer Interaction Experts



CET IO Working Group Activities:

Evaluation of operational effectiveness of the proposed design

Clarification of operationally required functionality & scenarios

related to mission objectives

Development of the overall design & operational philosophy

with the ground & flight crew

Review of proposed designed changes & monitor iteration

throughout the design process

Monitor human-system integration & compatibility issues with

others tools & systems

Request feedback & concerns from individuals – through

incident reporting


TN3: Crew Expert Tool IO concept

Based on Theory of Inventive Problem Solving techniques & principles

Envisaged the use of structured database of spacecraft’s & habitat’s systems & its components

Smart tagging will be integrated with a structured database which will help the crew to locate required components

Smart tagging will have each component’s specific data that will allow identifying systems & components primary & secondary functions once disassembled

Structured database & Smart tagging will need to be updated with changes made to the system throughout the mission – to allow the crew to trace changes


problem

Allocate mental &

physical resources


mentally


solve it


appropriateness &

correctness

Monitor progress



problem

Organise knowledge

about the problem

1

2

3

4

5

6

7

8

PROBLEM SOLVING

CYCLE

MonitorMaintain

Detect

Diagnose

Predict

Prevent

1

17

CET IO system boundaries

Roles

Crewmember

Ground Controller

Ships‟ Monitoring

Systems

Reference

Information

Repository

Simulation Tools

uc CET Context Diagram

CET

Crewmember

Ships Monitoring

Systems

Solve Problem

Ground Controler

Reference Information Repository

Simulation Tools


CET IO Composite Structure

CET IO

CET IO Laptop

Handheld Scanner

Location Model

composite structure CET

CET

Handheld Scanner

Location Model

CET Laptop


TN4: CET IO - Design & Evaluation of a

Complex Human Spacecraft Problem Solving Tool

CET IO Design

Expert User Evaluation

Recommendation for Development


Experts Problem Solving

Working with Astronauts

Day 1: Investigated how the existing crew

would solve unanticipated issues

autonomously (without CET IO)

Day 2: Evaluated & gained feedback on the

initial proof-of-concept tool – CET IO


Day 1: Experts Problem Solving

Objective:

Investigate how crew would currently address

anticipated problems (without MC) to inform CET IO

In three different contexts:

Souyz, Shuttle & ISS

Observation Method:

Cued-Recall-Debrief



Cued-Recall-Debrief Setup

Recording Team

Problem-Solving Session

1 hr exercise

3 team members

Uninterrupted

All wearing Head-

Mounted-Cameras

Team explores

possibilities of dealing

with a broken smoke

alarm situation, which

cannot be replaced

Camera &

Microphone

Field of

views

Analyst

Crew 1

Crew 2

Cre

w 3



Cued-Recall-Debrief Method Cue-Recall-Debrief

Individual Debrief Session

In real-time & non-intrusive

Accurate recollection of their thought processes Recollection of pre-verbal experience & episodic memory

User doesn‟t take it personally, „out-of-sight, out-of-mind‟

Avoid distortion of user experience

Field of

view

Analyst Crew 1

Laptop


Recommendations from the Crew

DAY 1: Concept Development

Exercise Observations

Followed a generally accepted problem solving

process

Problem solving stages were followed at

random

Explored the problem using interchangeably

three different approaches


Day 2: Experts Evaluating CET IO

Short introduction to TRIZ principles & concept of the tool

Joint Short Demo of the CET IO

Crew individually problem solving using CET IO

Individual Cued-Recall-Debrief Field of

view

Analyst Crew 1

Laptop



DAY 2:

Evaluation & Feedback

User interface evaluation (Proof-of-Concept)

• Improving data input & editing options, including

voice

• Refinement of cues & questions

• Provide more insightful overview of the whole CET

IO tool structure

• Provide access to databases via the tool



DAY 2:


Concept development suggestions

• Extending from individual to team to two remotely

collaborating teams

• Incorporation of existing troubleshooting & problem-

solving procedures

• Make the CET IO a crewmember



DAY 2:


General Recommendations

• Using the same group of astronauts

• Allow longer time for training & evaluation

• Maintain Human Factors/Human Computer

Interaction experts involvement throughout the tool

design


Recommendations

for CET IO Development

Future Studies

Concept refinement and development

Scenario generation matrix

Database taxonomy development


Recommendation for Future Studies:

Concept refinement & development

CET IO Iterative Design &

Life Cycle process

Resources:

TIMESCALE

• 12 to 18 months

EXPERTISE –

• Human Factors

• TRIZ Consultant

• Database Engineer

• Software Engineer

• Users as Consultants

Requirements

Specification Phase

Test PhaseDesign Phase:

Critical Design Review

Refurbish Phase

Operation Phase

Concept &

Feasibility Phase

Build Phase

1

2

3

4

5

6

7

8

ITERATIVE DESIGN

& LIFE CYCLE

Design Phase:

Preliminary Design

Review

Human Computer Interaction

Requirements (HCIR)

HCI Integration

HCI Evaluation

1

Operation Analysis

Training Design

HCI Evaluation &

Recommendations

Operation Log

Lessons

Learned



Concept refinement & development


Assess recommended improvements

Assess improved functionality

Assess interface improvements

Refine cues & questions

Commence detailed design

Cognitive Work Analysis

Working with scenarios

CET IO Working Group



Develop Systematic Scenario Generation

Process

Resources:

TIMESCALE –

18 to 24 months for initial matrix structure

set up, followed by periodic updates up to

the mission

EXPERTISE –

Human Factors, TRIZ Consultant, Incident &

Accident Investigator, Future Mission

Experts, Spacecraft and Mission Equipment

Specialists, Database Engineer, Software

Engineer, Users as Consultants




Retrieve specialists’

experience & expertise

in specific systems

Integrate lessons

learned from analogous

environment studies,

accident & incidents

Scenarios to be used

throughout crew

mission training to

populate the tool’s

database with solutionsWear and tear of the

habitat

Scenario Generating Matrix (SG-Matrix)

Definition of Environment

& Protective Shells


Super-system

System

Sub-System

SPACE

DEBRIS

Super-system

COLUMBUS

EXTERNAL

ARCHITECT.

System

WELDED

SHELL

PANELS

Sub-system

PANELS

Super-system

System

Sub-System

RUPTURE

THE PANEL

PRE-CURSOR STRESS EFFECT

Energy &Information

Substance

Space

Structure

Time

Definition of Environment

& Protective Shells

Context-defining

dimensions

Scenario Generating Matrix (SG-Matrix)


Database Taxonomy Development

Resources:

TIMESCALE –

12 to 18 months for initial setup of the

database taxonomy structure,

followed by periodical updates up to

the mission as new equipment

becomes available

EXPERTISE –

Human Factors, TRIZ Consultant,

Database Engineer, Software

Engineer, Users as Consultants




Investigation of usefulness

of existing databases

Assessment of their

structure & how they can be

adapted for the CET IO

access

Establish requirements

for hardware equipment

developers to inform

databases at the level of

detail required to help the

crew




Structure database to provide answers on use

of systems resources e.g.:

What parts of the system‟s components can be

disassembled & used?

What other functions they can perform once

dismantled?

What materials they are composed of?

What physical & chemical properties they posses?


Iya Whiteley [email protected]

Chris Chaloner [email protected]

Crew Expert Tool (CET IO) team:

Dr Iya Whiteley – SEA

Dr Olga Bogatyreva – University of Bath

Ed Soldat – SEA

Prof Chris Johnson – University of Glasgow

Dr Anne Bruseberg – SEA

Martin Townend – SEA


CET IO Proof-of-Concept

Documents

CREW EXPERT TOOL IO - spaceflight.esa.int · CREW EXPERT TOOL IO: Adjusting the Direction of the CET IO ... Cued-Recall-Debrief Final Presentation, ESTEC, 15 Sept 09 22. Day 1: Experts