Designing for human cognitive capabilities. Memory ...cs349/s17/... · Designing for human cognitive capabilities. Memory, perception and cognition. 1 CS349 -- Memory, Perception

Cognition

Designing for human cognitive capabilities.

Memory, perception and cognition.

CS349 -- Memory, Perception & Cognition1

So

urc

es


Jef Raskin, The Humane

Interface (2000).

Jeff Johnson, Designing with

the Mind in Mind (2010)

Erg

on

om

ics

an

d H

um

an

Fac

tors • Designing devices that fit human abilities:

physical and cognitive.


Co

gn

itiv

e E

ng

ine

eri

ng

, “C

og

ne

tic

s”

• The study of how people mentally work

– vs. dominant branch of ergonomics which

studies how people physically work

• “We must master an ergonomics of the

mind if we want to design interfaces that are

likely to work well.” (Raskin, p. 10)

• “…well-known computer interfaces … are

designed as though their designers expect

us to have cognitive abilities that experiment

shows we do not possess.” (Raskin, p. 10)


Co

gn

itio

n • Faculty for processing

information and applying prior

knowledge using mental

processes like:

– attention

– memory

– learning

– reasoning


Card, Moran, and Newell’s Model Human Processor

HC

I G

uid

elin

es • The study of “cognetics” has resulted in many sets of HCI

guidelines.

• They are all substantially the same

• Why?

– Unlike fashion design, they are not based on the designer’s

tastes and whims. They are all based on human

psychology: how people perceive, learn, reason,

remember, and convert intentions into action.


Memory


Sh

ort

vs. L

on

g-T

erm

Mem

ory • Short-term Memory: situations in which information is

retained for very short intervals (ε – several seconds)

• Long-term Memory: information is retained over longer

periods (minutes, hours, days, years, a lifetime)

• A tempting view:


PerceptionsShort-Term

Memory

Long-Term

Memory

Input RAM Disk

Lo

ng

-Term

Mem

ory

= A

cti

vate

d N

eu

ron

s • Perceptions enter via visual, auditory, olfactory, gustatory, or tactile

sensory systems

– Activate neurons in areas of the brain are dedicated to each sense

– Detect low-level features: dark-light edge, diagonal line, high-

pitched tone, sour taste, red colour, etc.

• Triggers activation of neurons in other areas of the brain that are not

specific to the sensory system.

– Combine lower-level features into higher-level features: animal,

Uncle Kevin, minor key, threat, etc.

– The set of triggered neurons depends on the features of the input

and the context.

– The more similar two perceptual stimuli are (features and context),

the more overlap there is in the sets of neurons that fire


Lo

ng

-Term

Mem

ory

= A

cti

vate

d N

eu

ron

s • Memory formation: long-lasting and perhaps permanent changes to

the neurons involved in a neural activity pattern which make it easier

to reactivate in the future.

• Remembering: reactivating the same pattern of neural activity that

occurred when the memory was formed.

– Somehow the brain distinguishes new activations from

reactivations.

– Recognition: New perceptions that trigger an existing pattern.

– Recall: Triggering an existing pattern without the perception.

• Strength of a memory depends on

– How often it’s reactivated

– Strength of the original perceptions

– Sleep


Sh

ort

-Term

Mem

ory • Not simply neural activations that don’t last long.

• Short-term memory is a combination of perception and

attention.

– Perceptions make available a collection of neural activations

– Attention selects just several of the many activations that

are currently available.

– Short-term memory is the currently activated neural patterns

of which we are aware.


Ch

ara

cte

risti

cs o

f S

ho

rt-T

erm

Mem

ory • Low capacity

– Used to say “7 items plus or minus 2” (George Miller, 1956)

– Most current researchers say this is too high

– Depends on similarity of items, ability to “cluster” items

• e.g. phone number as 3 sets of digits (vs. 10 digits)

– Capacity perhaps best measured in “item-features” rather

than items

• Volatility

– Turning attention to a different neural pattern turns it away

from some of what it had been focusing on

– Information can be easily lost from short-term memory if not

rehearsed to constantly bring it back into attention


Imp

lic

ati

on

s f

or

Use

r In

terf

aces User interfaces should help people remember information from

one moment to the next.

Examples:

• Modes:

– Allows UIs to have more functions than controls

– But… people make “mode errors” – do the wrong thing

because they forget which mode they’re in.

– Avoid modes or make it obvious which mode is engaged

• Search results: keep the search terms on screen to help

evaluate search results

• Instructions: keep a sequence of instructions visible while

being followed


Ch

ara

cte

risti

cs o

f L

on

g-T

erm

Mem

ory • Error-prone:

– Think of it as heavily compressed information

– Some features not recorded

• Weighted by emotions

• Retroactively alterable


Imp

lic

ati

on

s f

or

Use

r In

terf

aces • Don’t burden long-term memory more than you have to.

• Example:


Perception

How we take in information


Perc

ep

tio

n • Our perception of the world is not a true depiction of “reality”.

• To a large extent, we perceive what we expect to perceive.

• Our expectations – and therefore our perceptions – are biased

by:

– the past: our experience

– the present: the current context

– the future: our goals


Perc

ep

tio

n B

iase

d b

y E

xp

eri

en

ce Imagine you own a large insurance company. You are meeting

with a real estate developer to discuss plans for a new campus

of company buildings. The campus consists of a row of five

buildings, some of which include courtyards providing light for

the cafeteria and fitness centre. The developer shows you an

initial plan:


Perc

ep

tio

n B

iase

d b

y E

xp

eri

en

ce Now imagine that instead of a real estate manager you’re

meeting with the marketing manager about a billboard ad. The

advertising manager shows you a suggestion that shows a

single, value-laden word to be placed on the billboards:


Perc

ep

tio

n B

iase

d b

y E

xp

eri

en

ce


Perc

ep

tio

n B

iased

by C

urr

en

t C

on

text

• The same phrase is perceived differently depending on the list in which it appears:

• “The pattern of neural activity that corresponds to recognizing a letter, a word, a face, or any object includes input from neural activity stimulated by the context. This context includes other nearby perceived objects and events, and even reactivated memories of previously perceived objects and events.”

-- Designing with the Mind in Mind, p. 5


Fold napkins. Polish silverware. Wash dishes.

French napkins. Polish silverware. German dishes.

Pe

rcep

tio

n B

ias

ed

by G

oals • In the following “toolbox”, is there a pair of scissors?

• Was there a wrench?


Pe

rcep

tio

n B

ias

ed

by G

oals • This is a fairly famous video. If you’ve already seen it, please

don’t blow it for those who haven’t.

• https://www.youtube.com/watch?v=vJG698U2Mvo


https://www.youtube.com/watch?v=vJG698U2Mvo

Pe

rcep

tio

n B

ias

ed

by G

oals There are two ways in which our current goals bias our

perception:

1. Influencing where we look

– We focus on our goal, ignoring things that are unrelated.

2. Sensitizing our perceptual system to certain features

– Our brain will focus on particular features that meet out goal

(e.g. if we’re looking for a “red car”, we’ll scan for red).


Cognition

How we process information


Tw

o S

tate

s o

f C

og

nit

ive

Pro

ce

ss

es Cognitive unconscious: processes of which you are not

aware at the time they occur. The resources they make

available to us are HUGE.

Cognitive conscious: processes focused on the relatively few

things that you are aware of at the time they occur. The

resources they make available to us are tiny (but powerful).


(examples)

movement from unconscious to conscious …

from conscious to unconscious …

Co

gn

eti

cs “Understanding that we possess these two distinct sets of limited

mental abilities and understanding how they work in relationship

to human-machine interfaces is as essential to designing

interfaces as is knowing the size and the strength of the human

hand when we are designing a keyboard.”


-- Jef Raskin, The Humane Interface, p. 11

Co

mp

ari

ng

Co

gn

itiv

e S

tate

s


Property Conscious Unconscious

Engaged by Novelty, emergencies, danger Repetition, expected events, safety

Used in New circumstances Routine situations

Can handle Decisions Non-branching tasks

Accepts Logical propositions Logic or inconsistencies

Operates Sequentially Simultaneously

Controls Volition (your “will”) Habits

Capacity Tiny Huge

Persists for Tenths of seconds Decades

Lo

cu

s o

f A

tten

tio

n • Locus: The feature, object, or idea about which you are

intently and actively thinking (in cognitive conscious).

– Related to “focus” but...

• Cannot completely control where our locus will be

• We can have at most one locus of attention

– Maybe none

– More than one locus is actually multi-tasking, not multi-

processing

• Why is this not optimal?

– Multi-processing can be achieved by combining one locus

and one or more automatic activities...

• Driving and talking, …


See

ing

Do

esn

’t M

ean

We S

ee • A single locus of attention filters out many perceptions

– Tuning out sounds and smells...

– Implications: some aspects of a UI may be tuned out as

well

• Perceptions persist briefly and then decay quickly

– Many perceptions do not automatically become memories

– Implications:


Seein

g D

oes

n’t

Mean

We S

ee:

Ch

an

ge B

lin

dn

ess • Change Blindness: a change in a visual stimulus is

unnoticed by observer

– http://www2.psych.ubc.ca/~rensink/flicker/download/


http://www2.psych.ubc.ca/~rensink/flicker/download/

Co

nte

xt

Sw

itch

es • There is a significant cost to switching from one locus of

attention to another

– Context switch: about 10 seconds

• More for intense tasks, less for lightweight tasks

• If the same context switch is performed repeatedly...

• Implications:

– Protect users (yourself!) from context switches

– Research in interruptability

– Exploit single locus/context switches

• Magician

• Canon Cat/Some Apps (next slide)


New message from

Feridun …

Can

on

Cat


Ab

so

rpti

on • You can be more or less absorbed in your locus of attention

– More absorbed:

• More difficult to transition to another locus

• Requires greater stimulus to effect such a change

• Examples: reading a book, playing a game

• Absorption is essential to productivity

– Implications for UI: don’t interrupt this state!

– Encourage flow (being “in the zone”)


Ab

so

rpti

on

’s N

eg

ati

ve

Co

nseq

uen

ces

• Eastern Airlines Flight 401 story

• Proportional to stress: “As stress increases, people

concentrate more and more on but a few features of their

environment, paying less and less attention to others... You

become less likely to see hints, help messages, or other user

aids as you become increasingly agitated about the problem.”

-- Raskin, p. 27

• Implications for UI: prevent errors, easy-undo


Au

tom

ati

c A

cti

on

s • Automatic Action: any activity done without conscious

thought (routines, habits, skills) … in cognitive unconscious

• Performing a task repeatedly makes it easier

– Eventually, don’t need to give it conscious thought

– Conscious thought may impede the action

• Automatic tasks enable simultaneous actions

– If we are doing more than one activity at a time, all but at

most one of them are automatic

• Automatic actions are essential to higher life forms


Lea

rnin

g a

nd

Co

ntr

oll

ing

Au

tom

ati

c A

cti

on

s • Humans cannot avoid developing automatic responses

– “Practice makes perfect” vs. “Practice makes permanent”

• Sequences of actions become clumped into a single action

– Cannot interrupt a sequence of less than 1-2 sec.

– Long sequences can only be interrupted with conscious

thought

• Unlearning automatic

actions


Go

od

Au

tom

ati

c A

cti

on

Im

plic

ati

on

s • Persistent use of any interface will develop automatic actions

that are difficult to unlearn

– Can be good (e.g. consistency and similarity leads to quick

interaction)


Ba

d A

uto

ma

tic

Ac

tio

n Im

plic

ati

on

s • Can be bad (e.g. dangerous automatic actions)

– Example: File deletion confirmations

– Any confirmation step that elicits a fixed response soon

becomes useless

– Alternatives?


The action you have requested cannot

be undone. It will cause permanent loss

of the file. If you are sure you wish to

delete the file forever, type backward the

tenth word in this box.

Su

mm

ary • Introduction to perception and how it is biased

• Introduction to human cognition capabilities

– cognetics

– cognitive conscious vs. cognitive unconscious

– locus of attention

– automatic actions

• Implication of cognition for design


Documents

Designing for human cognitive capabilities. Memory ...cs349/s17/... · Designing for human cognitive capabilities. Memory, perception and cognition. 1 CS349 -- Memory, Perception