se4d03 slide L31-1

SE4D03 – Computer User Interfaces• The Science of Visualization – continued

– Visual Attention• Attention comes in two types of considerations:

1. Low-level as considered previously, and2. High-level as considered next

– In high-level interactive visualizations, the human is considered bound to the computer interface, such that a problem solving capability is realized which is beyond unaided human or single computer power.

– Therefore there are two considerations to be made when conceptually we problem solve:

1. The economics of cognition, and2. The cognitive cost of knowledge.

– These aspects are important for computer-human interface design since human attention is a very limited resource and its visual parsing ability slow. The computer must match speeds and delete any unnecessary, or irrelevant ,visual information.

se4d03 slide L31-2

The Science of Visualization -- I– High-Level Visual Attention

– Current physiological understanding describes visual thinking as a set of embedded processes involving multiple human memory systems:

1. Iconic memory is a very brief image store, holding what is on the retina until it is replaced by something else of until several hundred milliseconds have passed

2. Long-Term memory is information that is retained for even a lifetime, but must be integrated consciously and actively with existing knowledge.

3. Visual Working memory holds the visual objects of immediate attention and can be composed of either external or mental images, awaiting application of visual cognition operations.

– Visual Working Memory – properties1. Visual working memory is separate from verbal working memory,

but does compete with each for long term memory access and use. (See next slide for a unified extended cognitive model that includes both.)

2. Capacity is limited to a small number of simple visual objects and patterns, about three to five in number.

3. Positions of objects are stored in an egocentric map, where 9 locations may be stored but only 3 to 5 are linked to specific objects

se4d03 slide L31-3

The Science of Visualization -- II

• High-Level VisualAttention – cont‘d

• Properties of Visual Working Memory – cont’d

4. Attention controls whatvisual information isheld and stored

5. The time required to change attention is about 100 ms.

6. The semantic meaning or gist of an object or scene can be activated in about 100 ms.

7. For items to be processed into long-term memory, deeper semantic coding is required.

• Contents of Working Memory– A temporary grouping of a collection of visual features together with

other links to brain centres, which are called object files.• Object files consist of a neural activation pattern having the equivalent of pointers

reaching into the brain where visual features are processed

se4d03 slide L31-4

The Science of Visualization -- III• High-Level Visual Attention – cont‘d

• Contents of Working Memory – cont’d– Also the concept of gist is needed to provide a summary of properties

of an object where it is composed of the properties that are pulled from long-term memory as the image is recognized.

– Change Blindness• One of the results of having such a small visual working memory space is the fact that

because so small a memory is retained that large changes in the scene go unnoticed• Example: an unsuspecting person is approached by a stranger holding a map and

asked for directions. As this is going on, two people carry a door by the conversation and while this occurs, the stranger is substituted for another who is completely dressed differently and this goes unnoticed by the subject – change blindness.

• Therefore, if a change is made while the display is being fixated, the inevitable blink will draw attention to it; however, if changes are made mid-eye movement, or mid-blink or after a short blanking of the screen, the change will not generally be detected by the observer.

– Spatial Information in Working Memory• For objects acquired in one fixation to be

identified in the next requires some kind of buffer that holds locations in egocentric coordinates as opposed to retina-coordinates. This also allows for the synthesis of information obtained from successive fixations. This is shown at the lower left..

se4d03 slide L31-5

The Science of Visualization -- IV– High-Level Visual Attention – cont‘d

• Contents of Working Memory – cont’d– Perception Issues

• Perception is tuned by the task requirements to give us what is most likely to be useful. It occurs through a sequence of active visual queries operating through a focusing of attention to provide what we need. This is done through a rapid tuning of the of the pattern perception networks to respond best to patterns of interest – see next slide. This instantaneous grouping of information by attentional processing is described by the term nexus.

• Model of Visual Attention• At the lowest level are elementary

visual features that are processed in parallel and automatically. These correspond to elements of colour, edges, motion & stereoscopic depth.

• From these elements, prior to focused attention, low-level precursors of objects, called proto-objects exist in a continual state of flux.

• At the top level, the mechanism of attention forms different visual objects from the proto-objects flux.

• This is shown at the left..

se4d03 slide L31-6

Visualization -- V– High-Level Visual Attention• Tuned Receptive Fields

– In neural visual processing, use is made of tuned filters, where certain cells respond to certain types of visual patterns and no others. For instance, some cells respond to elongated blobs, with a particular position and orientation, others most strongly to blobs of a particular position moving in a particular direction at a particular velocity and still others respond to colour.

– In general the following properties can usually be distinguished by filters1. Orientation and size (with luminance) via the Gabor Processor described in

the next lecture2. Colour3. Elements of local stereoscopic depth, and 4. Elements of motion

• Attracting Attention• There are, from the preceding work, four basic visual requirements, when

an interface needs to create a user interrupt, to bring the attention of the user to a certain situation:

1. A signal should be easily perceived, even if it is outside the area of immediate focal attention

2. If the user wishes to ignore the signal and attend to another task, the signal should continue to act a reminder

3. The signal should not be so irritating that it makes the computer useless to use

4. It should be possible to endow the signal with various levels of urgency.

se4d03 slide L31-7

The Science of Visualization -- VI– High-Level Visual Attention

• Key Features of the Visual Thinking Process1. Problem components are identified that have potential solutions

based of visual patter discovery. These are formulated into visual queries consisting of simple patterns

2. Eye-movement scanning strategies are used to search the display for the query patterns

3. Within each fixation, the query determines which patters are pulled from the flux of pattern-analysis subsystems.

a. Patterns and objects are formed as transitory object files from proto-object and proto-pattern space.

b. Only a small number of objects or pattern components are restrained from one fixation to the next. These object files also provide the links to verbal-propositional information in verbal working memory

c. A small number of cognitive markers are placed in a spatial map of the problem space to hold partial solutions where necessary. Fixation and deeper processing are necessary for these markers to be constructed.

4. Links to verbal-propositional information are activated by icons of familiar patters, bringing in other kinds of information.

se4d03 slide L31-8

GOOEY du Jour -- I– More on Navigation

• Too many options (or links), 15 here, on one page makes it very difficult to remember your navigation path as you proceed.

12

34

7

6 5

1098

1214

13

11

15

se4d03 slide L31-9

GOOEY du Jour -- II– More Navigation –

cont’d• If user misses the

<Submit Information> and looks to <Continue> at the bottom of the page, then a completely unrelated and totally unexpected popup appears!

se4d03 slide L31-10

GOOEY du Jour -- III– More Navigation – cont’d

• Breadcrumbs that leave an active self-link, are poor navigation since if selected, just refreshes the screen and goes nowhere.

se4d03 slide L31-11

GOOEY du Jour -- IV– More Navigation – cont’d

• Self-links can be embedded as a link in the content of the displayed link

se4d03 slide L31-12

GOOEY du Jour -- V– More Navigation – cont’d

• Breadcrumb should not have last link activated as shown below:

se4d03 slide L31-13

GOOEY du Jour -- VI– More Navigation – cont’d

• Options, Options, Options, …