UEM College 9 mei, 2006 The Paradox of the Assisted User: Guidance can be Counterproductive Christof van Nimwegen

UEM College 9 mei, 2006

The Paradox of the Assisted User: Guidance can be Counterproductive

Christof van Nimwegen


Having a plan…..the basis of success?

O’Hara & Payne: Plan-based : Use strategies from long term memory Display based: Rely on interface information

Making an interface more costly provoked - More plan-based strategies- Better transfer to same domain problems

Zhang 1997:Externalizing information can be useful for cognitive tasks; the more information is externalized, the easier it is to

solve the problem


Background: Externalization vs. Internalization

Important for interaction designhow much information should be available on the interface (externalization) as opposed to having users encode and store this information in memory (internalization).

We investigate conditions whereby externalizing the information in the interface, or requiring it to be internalized respectively, enhances task performance of users.

This research contributes to formulate guidelines for GUI design in the context of problem solving tasks.


Externalization vs. Internalization: Example

Externalization Representation with relevant feedback Remembering certain task-related

knowledge is unnecessary This relieves working memory

(low cognitive load)

Internalization No support: Task-related information

is unavailable on the interface It needs to be internalized (high

cognitive load) and stored in memory Knowledge in memory of user is more

solid and detailed (recall, not recognition)


General research questions

In literature, there is no real agreement, but externalization in modern GUI’s is more or less default

But does this “support” of externalization … really result in better “performance” after all? make a task more pleasant in the long run? help when “learning something” is the aim?

(think of software in education) with transfer tasks? result in better knowledge of the underlying

task? what about interruptions?


Earlier experiment: Problem choiceClassical: Missionaries & Cannibals

Bring all of them to the other side

Rules: - Maximum 3 in boat - Minimum to sail is 1 (not empty)- Cannibals never in the majority or they

will eat the missionaries

Abstract version: Ball & Boxes

Transfer: Realistic M&C


Path


Material first experiment (with greyed-out items)

Abstract puzzle: variation on Missionaries & Cannibals

INTEXT

A more realistic transfer task: Missionaries & Cannibals


Results experiment 1

Internalization resulted in better problem knowledge afterwards better performance after long interval better performance on transfer task after

interval

Externalization had no advantages at all!


Externalization and planning?

Externalization

> Relying on interface discourages planning > Shallow thinking

> Worse performance

Internalization

> Stimulates planning> More engagement

> Better performance


Second experiment: more realistic task

Independent variables

Interface style (INT-EXT)We used more realistic material. Instead of abstract puzzles a conference planning task

Cognitive styleWe included an attitude trait. Before we tried to externally induce some state of mind, now we want to look at stable trait in people. We chose for the “Need for Cognition” (NFC). NFC is a well proven construct measuring the tendency of individuals to engage in and enjoy effortful cognitive tasks


Hypotheses

H1: Internalization leads to a more plan-based strategy and better performance than externalization. Having information externalized tempts users not to form plans and rely on the interface. The internalized condition lacks this guidance and is encouraged to plan and think before acting.

H2: High NFC leads to a more plan-based strategy than low NFC. People with high NFC have high intrinsic motivation to think and engage in effortful cognitive tasks, will show more plan-based behavior, and perform better


Method

Subjects 43 subjects, 17 (40%) male, 26 (60%) female. Aged 19 to 32 years Following or recently having followed higher education The experiment took at most one hour, 5 euro reward

Design 2*2Independent variables: Interface style (INT <> EXT)

Cognitive style (HIGH <> LOW) (median split)

Dependent variables: Various performance measures (times, moves)

StrategyKnowledge & Opinions


The application

The “Conference Planner” application simulated the planning of speakers (it logged all the moves)

Each of the speakers had different demands. They had to be scheduled in one of three conference rooms

The conference rooms each had its own facilities and

availability

Without planning, the scheduling will not be optimal and extra moves besides the optimal path must be made

The difference between EXT and INT was implemented by providing information concerning the legality of moves/actions


Conference planner: externalization and internalization

In EXTERNALIZATION it showed where speakers can be placed.

Slots in the timetable (satisfying constraints being available) turn green

Not the best slot, just legal ones

In INTERNALIZATION the interface, this feedback was absent

Subjects need to look more atthe constraints and think and plan themselves


Measures…..


Procedure

1. NFC2. Conference Planner

a) Trial 1b) Trial 2c) Trial 3d) Trial 4e) Trial 5

3. Post Experimental Questionnaire


Results: Time based measures (av. per 5 trails)

Time before first moveINT took longer than EXT

(p<.05)INT:M=19.8, SD=7.4EXT:M=15.3, SD=7.2

High NFCLow NFC

Tim

e b

efo

re f

irst

mo

ve

24

22

20

18

16

14

Internalisation

ExternalisationHigh NFCLow NFC

T

ime

be

twe

en

mo

ves

5.5

5.0

4.5

4.0

3.5

Internalisation

Externalisation

Time between movesINT took longer than EXT

(p<.05)

INT: M=4.8, SD=1.4EXT: M=3.9, SD=1.3

Total time: no significant difference


Results: Move based measures (av. per 5 trails)

Superfluous moves:INT made fewer than EXT

(p<0.05)INT: M=2.5, SD=0.6EXT: M= 4.3, SD=0.6

High NFCLow NFC

S

up

erf

luo

us

Mo

ves

5.0

4.5

4.0

3.5

3.0

2.5

2.0

1.5

Externalisation

Internalisation

Reconsidered moves: INT made fewer than EXT

(p<0.01)INT: M=0.5, SD=0.4EXT: M=2.1, SD=0.4

High NFCLow NFC

R

econ

side

red

mov

es

3.0

2.5

2.0

1.5

1.0

.5

0.0

Externalisation

Internalisation

Invalid moves : no significant difference


Strategy & opinion

Internalization subjects used the ‘most constraints first’ strategy more often. Tendency, (F(1,39)=3.21, p=0.08).

INT : 2.4 times out of 5 (SD=1.76) EXT : 1.5 times out of 5 (SD=1.5)

“I sometimes did not know how to proceed with the arrangement of the speakers”, (F(1,39)=5.91, p<0.02).

EXT :M= 5.24, SD =1.61INT :M= 6.23, SD=0.97)

(the item was scored reversely)


Conclusion

Hypothesis accepted: The interface requiring more internalization

resulted in

longer thinking time before subjects started more time between moves fewer superfluous moves and deviations, thus with

greater efficiency

This indicates that when information has to be internalized, more planning, active thinking and contemplation from the users is provoked


Conclusions

H2: rejected. Cognitive style along the dimensions of NFC had no influence.

It was Interface style again that predicted problem solving behavior

quite strongly, and could overrule possible effects of pre-existing individual differences. Also, our subjects were volunteering

university students, where the NFC average was very high.


Conclusions

If plan-based behavior is preferred, designers should be careful in giving users too much assistance (what is the goal?)

With multimedia being present in all corners of society, our findings can be valuable in the development of applications in the realm of education, multimedia learning or game based learning


Further research

Understanding reactions to interface information (based on cognitive findings) is important in tuning software, thereby allowing it to achieve its goal.

We will continue the research, and broaden the types of problem solving activities, and include other variables related to attitudes to problem solving.


Future research

Severe Interruptions(PC Crash)

Transfer Task(same task, different feeling)

Use eye tracking(distraction? Where are they really

looking?)


Deel 2Wat jullie gaan doen!?

Deel 2


Week 20

Maandag 13-15 Werkgroep Usability lab (CGN,C201)

Groep 1: 13:00Groep 2: 13:30 4 proefpersonenGroep 3: 14:00 4 proefpersonenGroep 4: 14:30 6 proefpersonen

_____________ 14 pp totaal


Dinsdag - donderdag

Experimentatie 2 personen aanwezig steeds 25-30 minuten per proefpersoon Jullie krijgen een tekst protocol Just drinks

Nieuw in dit experimentje:Logsqare wordt gebruikt voor video capture, in activity-

maps

Procedure1. De NFC-lijst2. De planningstaken (korte versie experiment (3 taken)3. Questionnaire (drastisch ingekort, alleen opinions, op

papier)


Data (stap 1)

1. CSV-file van NFC scores2. CSV-file van taken (raw data)3. Ingevulde korte (papieren) eind-

vragenlijst 4. Videos en met activity chart

afbeelding


Data (stap 2)

Runnen macros

Raw data > preciezere data

Fill up the gaps > mbv. videos

Preciezere data > SPSS


11121314151617181920

21222324252627282920

31323334353637383930


Data (stap 3) & verwerking

Voeg data NFC scores en Vragenlijst toe aan SPSS file Jullie krijgen van mij nog instructies tzt. Welke variabelen ik

erin wil hebben Stuur ALLE databestanden naar mij per mail Ik krijg dus per proefpersoon:

Nfc-csv PET-csv Pet-csv.xls (preciezer) Pet csv.xls.final (nog preciezer,no gaps) SPSS-bestand compleet (alles!)

Alleen de videos en activity charts kunnen op de Usabilitylab PC blijven staan. Zet dit duidelijk in en folder met groepnummer, daarin mapjes met proefpersoon nummer/naam


En dan….analyse!

Jullie krijgen dan zo snel mogelijk van mij een samengevoegd spss-bestand terug (19-20 mei)

26 mei, 23:59 (een week later) is de deadline voor het groeps rapport

Wees precies!! Je kunt tussendoor altijd mailen als er een probleem

is

SUCCES!!

Documents

UEM College 9 mei, 2006 The Paradox of the Assisted User: Guidance can be Counterproductive Christof van Nimwegen