CS422 - Human Computer Interaction

Instructor: Mehwish AzizSpring: Fall 2010Lecture 4-6

CHAPTER – 1HUMAN

Model Human Processor• Way back in 1983, Card, Moran and Newell gave a Model Human Processor

- Simplified view of human processing involved on interaction with computer system. This view comprises of:

Perceptual System (Input-Output) – handling sensory stimulus from outside world

Motor system (Memory) – controlling actionsCognitive system (Processing) – providing processing required for the

two

• Inspired from the above analogy, we consider today human user as information processing system

• External factors like social and organizational environment

• Moving ahead – let’s see what we will cover on ‘HUMAN’

Outline• How Limited Capacity to process information is a major

implication for design?

• Limited Capacity as Human possess:Limited numbers of receivable and reception

channelsLimited Memory Varying criteria to process and apply informationEmotions

Outline

• Limited numbers of receivable and reception channelsvisual channel auditory channel movement haptic channel

• Limited Memory sensory memory short-term (working) memory long-term memory

Outline• Varying criteria to process and apply information:

reasoning problem solving skill acquisition error

• Emotion influences human capabilities

• Users share common capabilities but are individuals with differences, which should not be ignored.

Input-Output Channels• Input-Output Channels of Human:

Input Channels – SensesOutput Channels – Motor control of the effectors

• Out of all senses – Vision, Hearing and Touch are major ones for HCI

• Effectors – limbs, fingers, eyes, head and vocal system

Input-Output Channels• Simplest Example – when you use PCs

Sight to view informationTouch to prompt input to PC through h/w interfaceHearing – when a system beeps on your mistakeVocals – when giving a command through speech or

say video presentation

• Is computers Interaction today on a balance for both disabled and normal human?

Human Vision• Vision has two stages

Physical reception of stimulusProcessing and interpretation of stimulus

• Physical Reception of stimulusMechanisms for receiving light and transforming it into

electrical energyLight reflects from objectsImages are focused upside-down on retinaRetina contains rods for low light vision and cones for

color visionGanglion cells (brain!) detect pattern and movement

Processing and Interpretation of Stimulus• Identification of Size & Depth

Visual angle indicates how much of view object occupies (relates to size and distance from eye)

Visual Acuity is ability to perceive details (limited)

Familiar objects perceived as constant size (in spite of changes in visual angle when far away)

Cues like overlapping help perception of size and depth

Processing and Interpretation of Stimulus• Brightness

Subjective reactions to levels of lightAffected by luminance of objectsMeasured by just noticeable differencesVisual Acuity increases with luminance as does flicker

• ColorComposed of hew, intensity and saturationCones sensitive to color wavelengthsBlue Acuity is lowestColor Blindness - 8% Males and 1% Females

Processing and Interpretation of Stimulus

• Visual Systems compensates for:Movement Changes in Luminance

• Context is used to resolve ambiguity

• Optical illusions sometimes occur due to over compensation

Optical Illusions

the Ponzo illusion the Muller Lyer illusion

Is this text Correct?

the quick brown fox jumps over the

the lazy dog

Design Focus• Optical Illusion highlight the difference between way things are

and way we perceive them

• In interface design we need to be aware that we will not perceive things exactly as they are!

• Optical Illusion also affects page symmetry

• In graphic design there is a phenomena known as optical center saying that See the center of page a little above than actual centerBottom page margins tend to be increased by 50% to

compensate incase page is arranged by actual center

Reading• Reading is performed with certain stages

Visual Patterns perceivedDecoded using internal representation of language Interpreted using knowledge of syntax, semantics, and pragmatics

• Reading involves saccades and fixations

• Perceptions occurs during fixations

• Word shape is important to recognition

• Negative contrast improves reading from computer screen

• For negative contrasts - Lets review some web-pages

The question persists and indeedgrows whether the computer will makeit easier or harder for human beings to

know who they really are, to identifytheir real problems, to respond more

fully to beauty, to place adequate valueon life, and to make their world safer

than it now is.

Norman Cousins – The Poet and the Computer, 1966

The question persists and indeedgrows whether the computer will make

it easier or harder for human beings toknow who THEY REALLY ARE, TO IDENTIFYTHEIR REAL PROBLEMS, to respond morefully to beauty, to place adequate value

on life, and to make their world saferthan it now is.

Norman Cousins – The Poet and the Computer, 1966

Hearing• Provides information about environment

distances, directions, objects etc.

• Physical apparatus:outer ear – protects inner ear and amplifies soundmiddle ear – transmits sound waves as vibrations to inner earinner ear – chemical transmitters are released and cause

impulses in auditory nerve

• SoundPitch – sound frequencyloudness – amplitudeTimbre – type or quality

Hearing• Humans can hear frequencies from 20Hz to 15kHz

less accurate distinguishing high frequencies than low

• Auditory system filters sounds can attend to sounds over background noise

for example, the cocktail party phenomenon

• Different applications are using sound speech toconvey information or generate requests for user

commands

Hearing• Uses of non-speech sound include the following:

Attention – to attract user’s attention to an alarming state or end of the process

Status Information – continuous background sounds can be used to convey status information especially where continuous visual attention is difficult

Confirmation – a sound with an action to confirm that the action has been carried out

Navigation – using changing sound to indicate where the user is in a system say a sound to support navigation hypertext

Touch• Provides important feedback about environment

• Can be a key sense for one who is visually impaired

• Stimulus received via the following receptors in the skin Thermo-receptors – heat and coldNociceptors – painMechanoreceptors – pressure (some instant or

continuous)

• Some areas are more sensitive than others e.g. fingers

Touch• Kinaesthesia - awareness of body position

affects comfort and performance

• E-Commerce seems to be very successful when it comes to shop for travel services, books and Computer accessories/ Softwares ….. What about clothes shopping through e-commerce? Do

you know about TouchCity?

Movement• Time taken to respond to stimulus say hitting a button in response to

a questionreaction time + movement time

• Reaction time depends on informationReceived by sensory receptors transmitted to brainBrain’s processing to generate a response

• Movement time is dependent on age, fitness etc.

• Reaction time - depends on stimulus type Visual ~ 200ms Auditory ~ 150 ms Pain ~ 700ms

Movement• Increasing reaction time decreases accuracy only in the

unskilled operator

• How pie chart-shaped menus are preferable to list since all options are equidistant?

• Fitts' Law describes the time taken to hit a screen targetMt = a + b log2(D/S + 1)

Here; a and b are empirically determined constantsMt -> movement time, D -> Distance , S -> Size of target

=> targets as large as possible distances as small as possible

Touch + Movement• Haptics is the science of applying tactile sensation to human

interaction with computers.

• Haptic device is one that involves physical contact between the computer and the user

This enables human to feed information to computer & also receive information through felt sensation on some part of the body – Haptic Interface

Through an input/output device, such as a joystick or data gloves, that senses the body's movements

For example, in a virtual reality environment, a user can pick up a virtual tennis ball using a data glove

Memory• There are three types of memory function:

Sensory memories (Iconic, Echoic, Haptic)

Short-term memory or working memory

Long-term memory

Sensory Memory• Buffers for stimuli received through senses

iconic memory: visual stimuli echoic memory: aural stimuli haptic memory: tactile stimuli

• Examples“sparkler” trail / fireworks persistent imagestereo sound / ascertain the direction of sound origin

• Continuously overwritten or lost

Short Term Memory (STM)• Scratch-pad for temporary recall

rapid access ~ 70msrapid decay ~ 200mslimited capacity - 7± 2 chunksExamples

212348278493202

0121 414 2626

HEC ATR ANU PTH ETR EET (move last to one)

Long Term Memory (LTM)• Repository for all our knowledge

slow access ~ 1/10 secondslow decay, if anyhuge or unlimited capacity

• Two typesEpisodic – serial memory of eventsSemantic – structured memory of facts , concepts, skills -> derived from episodic memory based

experiences

• semantic LTM derived from episodic LTM

Long Term Memory (LTM)• Semantic memory structure

provides access to informationrepresents relationships between bits of

informationsupports inference (deductions)

• Model: semantic networkinheritance – child nodes inherit properties of parent

nodesrelationships between bits of information is explicitsupports inference through inheritance

LTM – Semantic Network

Models of LTM - Frames• Information organized in data structures using slots and

frames• Slots in structure instantiated with values for instance of data• Frames slots may contain default, fixed or variable

information• Type–subtype relationships DOG Fixed legs: 4

Default diet: carnivorous sound: bark

Variable size: color:

COLLIE Fixed breed of: DOG type: sheepdog

Default size: 65 cm

Variable color

Models of LTM - Scripts• Model of stereotypical information required to interpret situation• Script has elements that can be instantiated with values for

context• “John took his dog to the surgery. After seeing the web , he left.”

Script for a visit to the vetEntry conditions: dog ill

vet openowner has money

Result: dog betterowner poorervet richer

Props: examination tablemedicineinstruments

Roles: vet examines diagnoses treatsowner brings dog in pays takes dog out

Scenes: arriving at receptionwaiting in roomexaminationpaying

Tracks: dog needs medicinedog needs operation

Models of LTM – Production Rules• Representation of procedural knowledge• Condition/action rules

if condition is matchedthen use rule to determine action

IF dog is wagging tailTHEN pat dog

IF dog is growlingTHEN run away

LTM – Storage of Information• rehearsal

information moves from STM to LTM

• total time hypothesisamount retained proportional to rehearsal time

• distribution of practice effectoptimized by spreading learning over time

• structure, meaning and familiarityinformation easier to remember

LTM - Forgetting• decay

information is lost gradually but very slowly

• interferencenew information replaces old: retroactive interferenceold may interfere with new: proactive inhibition

• so may not forget at all memory is selective …

… affected by emotion – can subconsciously `choose' to forget

LTM - Retrieval• recall

information reproduced from memory can be assisted by cues, e.g. categories, imagery

• recognitioninformation gives knowledge that it has been seen

beforeless complex than recall - information is cue

Thinking - Reasoning• Reasoning

deduction, induction, abduction• Deduction: derive logically necessary conclusion from given

premisese.g. If it is Friday then she will go to workIt is FridayTherefore she will go to work.

• Logical conclusion not necessarily true:e.g. If it is raining then the ground is dryIt is rainingTherefore the ground is dry

Thinking - Reasoning• Deduction: When truth and logical validity clash …

e.g. Some people are babiesSome babies cry

• Inference - Some people cry• Correct?

• Induction: generalize from cases seen to cases unseen e.g. all elephants we have seen have trunkstherefore all elephants have trunks.

• Unreliable: can only prove false not true

… but useful!

• Humans not good at using negative evidence e.g. Wason's cards.

Thinking - Reasoning• Abductive Reasoning: reasoning from event to

causee.g. Sam drives fast when drunk.

If I see Sam driving fast, assume drunk.

• Unreliable: can lead to false explanations

Thinking – Problem Solving• Process of finding solution to unfamiliar task using

knowledge.

• Several theories exist.

• Gestalt Theory:problem solving both productive and reproductiveproductive draws on insight and restructuring of problemattractive but not enough evidence to explain `insight' etc.move away from behaviourism and led towards information

processing theories

Thinking – Problem Solving• Problem space theory:

problem space comprises problem statesproblem solving involves generating states using legal

operatorsheuristics may be employed to select operators

e.g. means-ends analysisoperates within human information processing system

e.g. STM limits etc.largely applied to problem solving in well-defined areas

e.g. puzzles rather than knowledge intensive areas

Thinking – Problem Solving• Analogy

analogical mappingnovel problems in new domainuse knowledge of similar problem from similar domain analogical mapping difficult if domains are semantically

different

• Skill acquisitionskilled activity characterized by chunkinglot of information is chunked to optimize STMconceptual rather than superficial grouping of problemsinformation is structured more effectively

Errors & Mental Models• Slips

right intention, but failed to do it right causes: poor physical skill , inattention etc. change to aspect of skilled behaviour can cause

slip

• Mistakes wrong intention cause: incorrect understandinghumans create mental models to explain behaviour.if wrong (different from actual system) errors can occur

Emotion• Various theories of how emotion works

James-Lange: emotion is our interpretation of a physiological response to a stimuli

Cannon: emotion is a psychological response to a stimuliSchacter-Singer: emotion is the result of our evaluation

of our physiological responses, in the light of the whole situation we are in!

• Emotion clearly involves both cognitive and physical responses to stimuli

Emotion• The biological response to physical stimuli is called

affect

• Affect influences how we respond to situationspositive creative problem solvingnegative narrow thinking

“Negative affect can make it harder to do even easy tasks; positive affect can make it easier to do difficult tasks”

(Donald Norman)

Emotion + Individual Differences• Implications for interface design

Stress: increase difficulty of problem solvingRelaxed users: more forgiving of shortcomings in designaesthetically pleasing and rewarding interfaces: increase

positive affect

• Individual Differences:long term: sex, physical and intellectual abilitiesshort term: effect of stress or fatigueChanging: age

• Ask yourself: will design decision exclude section of user population?

Psychology & Designing Interactive Design• Some direct applications

e.g. blue acuity is poor blue should not be used for important detail

• However, correct application generally requiresunderstanding of context in psychology, understanding of particular experimental conditions

• A lot of knowledge has been distilled in:guidelines (chap 7)cognitive models (chap 12)experimental and analytic evaluation techniques (chap 9)

CHAPTER – 2COMPUTER

Computers • Computers consisting of I/O Devices, memory and processing – just

like human being.• Richer Interaction everywhere and everywhen!

sensorsand deviceseverywhere

Text Entry Devices• keyboards (QWERTY et al.):

QWERTY arrangement not optimal for typinglayout to prevent typewriters jamming!

Alternative designs allow faster typing but large social base of QWERTY typists produces reluctance to change.

• Alternative - Alphabetic: keys arranged in alphabetic ordernot faster for trained typistsnot faster for beginners either!

2 3 4 5 6 7 8 9 0Q W E R T Y U I

1O P

S D F H J LA G K

Z X C V B N M , .SPACE

Text Entry Devices• Dvorak

common letters under dominant fingers, biased for right handcommon combinations of letters alternate between hands10-15% improvement in speed and reduction in fatigueStill, large social base of QWERTY typists produce market

pressures not to change• Special Keyboards: for one handed use e.g. Maltron left-handed

keyboard

Text Entry Devices• Chord keyboards: only a few keys - four or 5

letters as combination of key presses compact size - ideal for portable

applications short learning time – key presses reflect

letter shape Fast - once you have trained Social resistance still exists, plus fatigue

after extended use

Text Entry Devices• Phone pads:

use numeric keys with multiple presses

surprisingly fast!

• T9 predictive entrytype as if single key for

each letteruse dictionary to ‘guess’

the right wordbut 26 -> ‘am’ or ‘an’ etc.

Text Entry Devices• Handwriting Recognition:

Text can be input into the computer, using a pen and a digesting tablet

natural interactionTechnical problems:

capturing all useful information - stroke path, pressure, etc. in a natural manner

segmenting joined up writing into individual lettersinterpreting individual letterscoping with different styles of handwriting

Used in PDAs, and tablet computers etc.

Text Entry Devices• Speech Recognition:

Improving rapidlyMost successful when:

single user – initial training and learns peculiaritieslimited vocabulary systems

Problems withexternal noise interferingimprecision of pronunciationlarge vocabulariesdifferent speakers

Positioning, Pointing & Drawing• Mouse: Handheld pointing device: very common, easy to use

Two characteristics: planar movement, buttons

Mouse located on desktop: requires physical space, no arm fatigue

Relative movement only is detectable.

Movement of mouse moves screen cursor

Screen cursor oriented in (x, y) plane, mouse movement in (x, z) plane …

Positioning, Pointing & Drawing• Mouse: Two methods for detecting motion:• Mechanical:

Ball on underside of mouse turns as mouse is movedRotates orthogonal potentiometersCan be used on almost any flat surface

• Optical:light emitting diode on underside of mousemay use special grid-like pad or just on deskless susceptible to dust and dirtdetects fluctuating alterations in reflected light intensity

to calculate relative motion in (x, z) plane

Positioning, Pointing & Drawing• Mouse: some experiments with the foot-mouse

controlling mouse movement with feet …not very common but foot controls are common elsewhere:

car pedalssewing machine speed controlorgan and piano pedals

Touchpad• Small touch sensitive tablets:

‘stroke’ to move mouse pointerused mainly in laptop computers

• Good ‘acceleration’ settings importantfast stroke

lots of pixels per inch movedinitial movement to the target

slow strokeless pixels per inchfor accurate positioning

Trackball & Thumbwheels• Trackball

ball is rotated inside static housinglike an upside down mouse!

relative motion moves cursorindirect device, fairly accurateseparate buttons for pickingvery fast for gamingused in some portable and notebook computers.

• Thumbwheels …for accurate CAD – two dials for X-Y cursor positionfor fast scrolling – single dial on mouse

Touch Sensitive Screen• Detect the presence of finger or stylus on the screen

works by interrupting matrix of light beams, capacitance changes or ultrasonic reflections

direct pointing device

• Advantages: fast, and requires no specialised pointergood for menu selection suitable for use in hostile environment: clean and safe from damage

• Disadvantages:finger can mark screen imprecise (finger is a fairly blunt instrument!)

difficult to select small regions or perform accurate drawing lifting arm can be tiring

Stylus & Light Pen• Stylus

small pen-like pointer to draw directly on screenmay use touch sensitive surface or magnetic detectionused in PDA, tablets PCs and drawing tables

• Light Pennow rarely useduses light from screen to detect location

• BOTH …very direct and obvious to usebut can obscure screen

Digitizing Tablet• Mouse like-device with cross hairs

• used on special surface - rather like stylus

• very accurate- used for digitizing maps

Eye Gaze• control interface by eye gaze direction

e.g. look at a menu item to select it

• uses laser beam reflected off retina… a very low power laser!

• mainly used for evaluation

• potential for hands-free control

• high accuracy requires headset

• cheaper and lower accuracy devices available….sit under the screen like a small webcam

Cursor Keys• Four keys (up, down, left, right) on keyboard• Extremely cheap, but slow. • Useful for not much more than basic motion for

text-editing tasks• No standardised layout, but inverted “T”, most

common

Discrete Positioning Controls• phones, TV controls etc.

cursor pads or mini-joysticksdiscrete left-right, up-downmainly for menu selection

Display Devices• Bitmap Displays:

screen is vast number of coloured dots

Resolution and Color Depth• Resolution used (inconsistently) for

number of pixels on screen (width x height)e.g. SVGA 1024 x 768, PDA perhaps 240x400

density of pixels (in pixels or dots per inch - dpi)typically between 72 and 96 dpi

• Aspect ratioRatio between width and height4:3 for most screens, 16:9 for wide-screen TV

• Colour depthhow many different colours for each pixelblack/white or greys only8 bits each for red/green/blue = millions of colours256 from a palette

Anti-Aliasing• Jaggies

diagonal lines that have discontinuities due to horizontal faster scan process

• Solution to jaggies is Anti-aliasingsoftens edges by using shades of line colourused for text

Cathode Ray Tubes

• Cheap display device • Fast response time for rapid animation coupled with

high colour capability• Fairly bulky in comparison to flatter displays• Stream of electrons emitted from electron gun, • focused and directed by magnetic fields,• used in TVs and computer monitors• View has a high rate of flicker• Way of reducing flicker from screen

Higher scan ratesInterlacing – tracing first all odd then even lines

Liquid Crystal Display• Liquid Crystal Display:

Smaller, lighterFound on portables and notebooks … and increasingly on desktop

and even for home TVUsed in dedicated displays: digital watches, mobile phonesRequires refreshing at usual rates but slow response to the crystals

Flicker isn’t usually noticeableHow it works:

Top plate transparent and polarised, bottom plate reflecting.Light passes through top plate and crystal, and reflects back to

eye. Voltage applied to crystal changes polarisation and hence colourN.B. light reflected not emitted => less eye strain

Special Displays• Random Scan (Directed-beam refresh, vector

display)draw the lines to be displayed directlyno jaggieslines need to be constantly redrawnrarely used except in special instruments

• Direct view storage tube (DVST)Similar to random scan but persistent => no flickerCan be incrementally updated but not selectively erasedUsed in analogue storage oscilloscopes

Large Displays• Large Displays - used for meetings, lectures, etc.• Technology used is:

plasma – usually wide screen video walls – lots of small screens together projector – RGB lights or LCD projectorhand/body obscures screenmay be solved by 2 projectors + clever software back-projectedfrosted glass + projector behind

Situated Display• displays in ‘public’ places

large or smallvery public or for small group

• display only - for information relevant to location• interactive - use stylus, touch sensitive screen• in all cases … the location matters

meaning of information or interaction is related to the location

Hermes – A Situated Display

small displaysbeside

office doors

handwrittennotes left

using stylusoffice ownerreads notes

using web interface

Digital Paper• what?

thin flexible sheetsupdated electronicallybut retain display

• how?small spheres turned orchannels with coloured liquid

and contrasting spheresrapidly developing area

appearance

crosssection

Virtual Reality and 3D Interaction

• Positioning in 3D Space:cockpit and virtual controls

steering wheels, knobs and dials … just like real!3D mouse

six-degrees of movement: x, y, z + roll, pitch, yaw data glove

fibre optics used to detect finger positionVR helmets

detect head motion and possibly eye gazewhole body tracking

accelerometers strapped to limbs or reflective dots and video processing

Virtual Reality and 3D Interaction

pitch

yaw

roll

Pitch, Yaw & Roll

Virtual Reality and 3D Interaction

• 3D Displaysdesktop VR

ordinary screen, mouse or keyboard controlperspective and motion give 3D effect

seeing in 3Duse stereoscopic visionVR helmetsscreen plus shuttered specs, etc.

• VR Headsets – 3D Effectssmall TV screen slightly different angles

Virtual Reality and 3D Interaction• VR Motion Sickness:

time delaymove head … lag … display movesconflict: head movement vs. eyes

depth perceptionheadset gives different stereo distancebut all focused in same planeconflict: eye angle vs. focus

conflicting cues => sicknesshelps motivate improvements in technology

Virtual Reality and 3D Interaction• Simulators and VR Caves

scenes projected on wallsrealistic environmenthydraulic rams!real controlsother people