PSY2403 Human Technology Interaction: Method I...PSY2403 Human Technology Interaction: Method I Cato A. Bjørkli Psychologist PSY2403 HumanTechnologyInteraction MethodI CatoABjørkli

1

PSY2403Human Technology Interaction:

Method ICato A. BjørkliPsychologist

PSY2403 Human Technology Interaction Method I Cato A Bjørkli 06 02 15

Background for the lecture

• S.G.Charlton (1996) Mental Workload Test ...

• B.Nardi (1997) The Use of Ethnography Methods...

• (X.Faulkner (2000) Usability Engineering ...)


Art is ‘I’

Science is ‘we’

Claude Bernard

(why method?)

2


-- method --

a way of doing something,especially a systematic way;implies anorderly logical arrangement(usually in fixed steps)

-- science --

ability to produce solutionsin some problem domain

approximate d(f)


The Science ofHuman Technology Interaction

(MMI)

What characterizes a good product?

What characterizes a safe system?

What characterizes a efficient system?

How do we know that somethingis what we designed it to be?

(examples?)


Field of Study

Product

Social

Individual

Systems

3


Life of Products

• How do a product come to life?• How does an artifact come in to existence?• When is a product completely designed?

CS60/65


Methods for all phases of product cycles


Evaluation & Redesign Innovation & Design

Methods underlying all aspects of HTI

A way of gaining insight into the system studied

Measures, Experiments,

Analysis,Ethics,Skills

4


Method is choices

“... Theoretical interpretations serve to provide much of thedirection for the research questions and methodologies.”

S.G.Charlton, 1996, p.181. (vs Flach & Warren, 1995)

How you approach thephenomenon is guiding forthe methods you choseto investigate it.

So, beyond the type of project, youmust consider your perspectiveto the system i question.


Traditional Information Processing

Embodiment

Social Matrix

Engineering

Ergonomics

?


Fact:There are a number of ways for modeling cognitionand performance - thus are there equally many waysof measuring it.

Consequence:You must be able to provide arguments for yourspecific approach and make sure that it correspondswith your choice of measurement.

5


Considerations

• What system?– Complex– Product– Social

• What phase and focus of your project?– Evaluation of existing system– Redesign– Innovation & Development

• What theoretical perspective?– Traditional Infor Pro– Embodiment– Cognitive Engineering


W O R K L O A DCharlton, S. (1996) Mental Workload Test and Evaluation


The concept of load

6


How is workload expressed or relevant in these systems & tasks?


Workload

• Quantitative measurement of a specific transitorysubjective state or temporal feature of mentalprocesses describing human technology interaction.

• What is ‘workload’?– Definition?– By analogies or underlying mechanisms?– Multifaceted?– Related to performance– Related to motivation– Related to strategies/timing– Situatedness / Contextuality / Lab versus world


Workload is a multidimensionalconcept, and should not beestimated with single indicators,but with multivariate measures

NASA TLXSWAT

Yet, some univariate global measures yieldgood information

Strengths and weaknesses ?

How to describe a

house

Height? Weight?Shape?

7


Describe a house


1. Sensitivity: Can the test show changes in workload?

2. Intrusiveness: Do the test disturb the operator during work?

3. Diagnostical: Which aspect of several is the most important?

4. Convenience: How much logistics are necessary?

5. Relevance: How well do the measurement fit the work?

6. Operator acceptance: Is the test ok for the operator?

Criteria for good workload tests


Types of Tests:Overview

Projective tests• Timeline Analysis• Scheduling Theory• Pro-SWAT

Empirical tests• Modified Cooper-Harper (MCH)• SWAT• NASA TLX• Crew Status Survey

8


Projective Tests


PROJECTIVE TESTS

Time available

T-0 T-N

a b c

Time-Line Analysisand Scheduling

Task Time versus Time Available

Task Difficulty

Demanded skills

Frequencies

Percept/Motor channel

Criteria: task time < 75% of total time


PROJECTIVE TESTS

Pro-SWAT

Task Enactment w/prototype

Detailed Briefing

SWAT scores

Debriefing

(During prototyping)

Cashpoint Interface

Cellphone Interface

9


Where do projective tests fit in?


Empirical Tests


Empirical Techniques

“... collection of data from one or more subjects actually

performing the task(s) of interest.”

Q: What is relevant data?

10


Types of Data

• Primary task: Objective variables:– Speed, lateral position, observable

performance

• Secondary task: Additional tasks– Mental calculus, counting backwards

• PsychoPhys: Changes in patterns– Heart, eye, startle, skin conducance

• Subjective methods– Self-reports (NASA, SWAT, etc)

Workload by Inference

Workload by Measure

?


Examples from different projects


Example I:Adaptive Front Lights

The objectives of the CLARESCO project is to carry outan extensive assessment of advanced lighting technologiesby studying advanced front-lighting systems for both carand truck lighting.

11


Example I: Adaptive Front Lights

Objective variablesSpeed, position

PhysiologicalHeart, eye movement

Subjective variablesNASA TLX, Acceptance

Secondary TaskAlarm detection


Example II:Invehicle Information System

Objective variablesSpeed, position

Subjective variablesNASA TLX

GUI vs Speech Recognition


Example III:HSC Sailing

How can this beapproached?

-Workload?

-Performance?

-Secondary task?

-Primary Task?Focus:

Design of electronic chartsand other navigation equipment

12


Example III:HSC Sailing

PhysiologicalHeart, Skin Conduct

Subjective variablesNASA TLX, SWAT

Secondary TaskAdHoc Attention Task

Video RecordingLink Analysis


What kind of data do we need?


Specific empirical tests

13


Assumes a three dimensional workload construct

1. Time Load 2. Mental Effort 3. Psych Stress

Procedure:1) Normalize for subjects (scale development)2) Workload Scores (data collection)

Evaluation1) Subscales2) Total score in accordance to scale development

Empirical Technique:SWAT


• Simple dimension structure– Subjective fatigue– Workload estimate

• Seven point Likert-scale (1 to 7)– 1 = Fully Alert / Nothing to do– 7 = Completly exhausted / Unacceptable

• Compared to other system information– Length of shifts– Task Complexity– Multitasking

• Later elaborated with more dimensions in the ARWES Questionnaire

Empirical Technique:Crew Status Survey


Empirical Technique:NASA TLX

Procedure

1. Rate all six subscales from 1 to 1002. Make pairwise comparison of subscales3. Multiply rating with weigthing

Evaluation

A) Add all 6 subscales, divide by 15 (comparisons) for total scoreand/or

B) Evaluate workload profile of all 6 subscales

14


NASA TLX RATING SCALE DEFINITIONS

MENTAL DEMAND (Low/High)How much mental and perceptual activity was required (e.g., thinking, deciding, calculating,remembering, etc.)? Was the task easy or demanding, simple or complex?

PHYSICAL DEMAND (Low/High)How much physical activity was required (e.g., pushing, pulling, turning, controlling, activating,etc.)? Was the task easy or demanding, slow or brisk, slack or strenuous, restful or laborious?

TEMPORAL DEMAND (Low/High)How much time pressure did you feel due to the rate or pace at which the tasks or task elementsoccurred? Was the pace slow and leisurely or rapid and frantic?

EFFORT (Low/High)How hard did you have to work (mentally and physically) to accomplishyour level of performance?

PERFORMANCE (Good/Poor)How successful do you think you were in accomplishing the goals of the task set by theexperimenter? How satisfied were you with your performance in accomplishing these goals?

FRUSTRATION LEVEL (Low/High)How insecure, discouraged, irritated, stressed and annoyed versus secure,gratified, content, relaxed and complacent did you feel during the task?


see example from excel and program

Empirical Technique:NASA TLX


One step back, again

15


Types of Tests:Overview

Projective tests• Timeline Analysis• Scheduling Theory• Pro-SWAT

Empirical tests• Modified Cooper-Harper (MCH)• SWAT• NASA TLX• Crew Status Survey


1. Sensitivity: Can the test show changes in workload?

2. Intrusiveness: Do the test disturb the operator during work?

3. Diagnostical: Which aspect of several is the most important?

4. Convenience: How much logistics are necessary?

5. Relevance: How well do the measurement fit the work?

6. Operator acceptance: Is the test ok for the operator?

Criteria for good workload tests

SWAT vs NASA TLX?


Alternatives?

Beyond quantitative methods ...

16


Nardi and Ethnography

• Sometimes you don’t know what you are looking at

• Cognition in the wild (Welcome to the Jungle)

• Golden Rule: Before measuring a phenomenon, youshould understand it.

• Main ethnographic activities:– Interviews– Observation– Participation


Ethnography:The Choices

• How long should I study this?

• What role should I have?

• What enables me to access information?

• What tools do I have?

• What data do I collect? How? When?

• How do I report ethnographic data?


Ethnography, cont

Your role and your mission

• Goal: Reflect and state your goal for observation. What do Iwant to achieve? What is success?

• Method: How do can I get to the information and by whatmeans? Tools? Methods?

• Conditions: What resources do I have available? Time andmoney? Equipment?

• Analysis: What will be the format of data? How do I makesense of experiences? What do I bring back?

17


Ethnography:How to report?

Analysis is difficult with qualitative data (show dvd)


The choice of methods ...... in accordance to what?


Questions?

18

PSY2403 Man Machine Interaction:Method I

Cato A. BjørkliPsychologist

Documents

PSY2403 Human Technology Interaction: Method I...PSY2403 Human Technology Interaction: Method I Cato A. Bjørkli Psychologist PSY2403 HumanTechnologyInteraction MethodI CatoABjørkli