The Cognitive Engineering of Human-Agent-Robot Systems

The Cognitive Engineering of Human-Agent-Robot Systems

Peter BendaPhD CandidateDepartment of Information Systems

2010 Department of Information Systems

Key messages Thinking in ‘ecological systems’ sense

can provide ‘engineering/design leverage’

Cognitive systems engineering might provide representations or models thatcan be shared by humans and robots at

an ‘interface’ levelbe useful in designing ‘resilient’ work

systems Propose doctoral research that may

provide leverage in designing ‘resilient’ HART systems


What I want from HART learn what interesting and relevant

work is out there

guidance for next steps in the PhDPlease don’t tell me to quit!Focus . . .

look for opportunities for collaboration


My Background BAppSc + MAppSc Ind. & Mech

Engineering (Toronto) ~10 years HCI & HF consulting,

corporate work 4 years research fellow:“Maximising the effectiveness of

interactive automated programs for smoking cessation”


Quit Smoking Support (Briefly)

expert system model, ‘basic messaging’/advice system based on modified TTM

5 conditions (5 variations of coaching system/controls)

iterative development model—ethnographic studies

RCT 3800+ participants (quitting smokers)


Where to start? Transition from work in HF (CogEng),

HCI to a desire to work with ‘Agents’

Initial reviews of HF/Cog Eng literature and Agent literature; is there any common ground?

Perspective is of a ‘systems design problem’


Systems Engineering & Design Perspective

Broadly:

“How do we design a human-agent-robot system?”

. . . . what does it mean to design a ‘robust’ or ‘resilient’ HAR system?


Agent Lit Generally CoveredR-CAST

BRAHMS

Meta analyses of agent modelling approaches

ACT-R/ SOAR

Social Simulation

Game Theory

Cohen & LevesqueShared

Plans

Evolutionary ‘Ecological’ Agents

HRI

Joint Intentions

BDIBradshaw

Kaminka

Wayne Gray

John Yen


HF/CSE Lit ReviewedKlein’s RPD

Woods & Hollnagel’s Joint

Cognitive Systems

Hutchins

Lintern

Human Perception of AutomationParasuraman &

Sheridan s levels of automation

Perrow’s Normal Accidents, Complex Systems

Vicente & Rasmussen’s

Cognitive Work Analysis

Zieba on resilient H-A-R

Systems


Perspectives on HF and Agent literature

HF literature typically deals with Understanding human behaviour with automation,understanding human ‘perception’ of automation

• User acceptance issues Optimising human use of automation Analysis of system error training outcomes,

prevention, interface design Agent literature

Models upon which synthetic agents can be based• Cognitive, decision making , perceptual, behavioural, social

psych etc.• Provide insight into ‘human behaviours’

Optimisation of multi-agent systems (typically synthetic) Problem-solving systems etc


BUT


Klein et al.: 10 key challenges Klein et al.’s (2004) 10 key challenges facing

such H-M systems (I’m looking at you, Bradshaw!)

1. Basic Compact2. Adequate Models3. Predictability: Human-agent team members must be mutually predictable4. Directability: Agents must be directable.5. Revealing Status and Intentions: Agents must be able to make pertinent

aspects of their status and intentions obvious to their team-mates.6. Interpreting Signals: Agents must be able to observe and interpret pertinent

signals of status and intentions.7. Goal Negotiation: Agents Must Be Able to Engage in Goal Negotiation8. Collaboration Support technologies for planning and autonomy must enable a

collaborative approach.9. Attention Management: Agents must be able to participate in managing

attention.10. Cost Control: All team members must help control the costs of coordinated

activity.


Consider principles behind Distributed Cognitive Systems

Ed Hutchins + others (1995+):1. knowledge possessed by members of

the cognitive system is both highly variable and redundant

2. Individuals working together on a collaborative task possess different kinds of knowledge, will engage in interactions that will allow them to pool the various resources to accomplish their tasks.

ref Rogers (1997)


Distributed Cognitive Systems (2)

3. Distributing and sharing access and knowledge enables the coordination of expectations to emerge which in turn form the basis of coordinated action

ref Rogers (1997)


Building System Resilience Three meanings (Zeiba et al, 2009):

1. foresight and avoidance of events

2. reaction to events

3. recovery from occurrence of events.


Recovery from occurrence of (unanticipated) events

“[Affordances allow] for a common representation for the opportunities of action between the automated system and its environment.”


Can Cog Sys provide leverage?

Rasmussen and Vicente developed and refined ‘Cognitive Work Analysis’ (CWA)

Focus was on the engineering of complex, time-critical H-M systems that exploit human decision making effectively during ‘normal’ operation (i.e. predictable situations) the occurrence of unpredictable events (often

emergency situations) Goal was to provide a framework for

resilient systems designanalytical tools & design tools (e.g., EID)


Cognitive Engineering With Lintern’s (2009) modifications to

include RPD, the CWA ‘outcomes’ to be focussed on includingAbstraction-Decomposition Space

(affordance and constraint mapping of a work system)

Contextual Activity Matrices (desired and potential spans of action)

Decision Ladder(s) (potential strategies)


One potential approach Utilise Abstraction Hierarchy & related

Work Domain Analysis as a basis for a shared ‘system’ model

Development of shared system representation (interface) that Can be understood, interrogated, acted

upon by humans, agents, robots efficiently


In other wordsDevelop a specification of a human–agent-robot

shared representation (interface) supporting affordance based communication, that

can be directly ("efficiently") perceived by H-A-R

can be used as a basis for coordinated H-A-R action

as a means of collaboratively and dynamically 'resolving degrees of freedom in the work system' in unanticipated situations.


10 Challenges Redux Potentially addresses

Challenge 5 (Revealing Status and Intentions): Agents must be able to make pertinent aspects of their status and intentions obvious to their team-mates.

Challenge 6 (Interpreting Signals): Agents

must be able to observe and interpret pertinent signals of status and intentions.


Research approach1. Take a candidate H-A-R or H-A system

1. human-in-the-loop simulation1. e.g., MIL-C2 DSS in some of the R-CAST

work2. must be able to introduce unanticipated

events2. develop mechanics of integration and

human- + agent- interface(s)3. integrate the proposed model into said

simulation4. run experiments versus control

(original)


Other ideas inspired by HART

BW4T – is that a possible candidate micro-world?

NIFTI search and rescue workaugments? user-centred development

work


What I want from HART Redux

learn what interesting and relevant work is out there

guidance for next steps in the PhDPlease don’t tell me to quit!Focus . . .

look for opportunities for collaboration


A Related Approach . . . ?

Johnson , Bradshaw et al (2010) “Coactivity” & Interdependence

“Critical design feature of HR system is ‘the underlying interdependence of joint

activity’”

closely following this work . . . think there is ‘common ground’


Questions, comments?

Documents

The Cognitive Engineering of Human-Agent-Robot Systems