Diagram Narratives

Barbara Tversky

Stanford University

Gratitude due:

• Office of Naval Research

• Julie Heiser, Julie Morrison, Marie-Paule Daniele, Mireille Betrancourt, Jeff Zacks, Gowri Iyer, Sandra Lozano, Sonny Kugelmass, Atalia Winter, Paul Lee, and many others

QuickTime™ and aPlanar RGB decompressor

are needed to see this picture.

(Hegarty, 1993)

(Mayer & Gallini, 1990)

Mechanical Systems

Experiment 1: Describing diagrams

• Participants given one diagram

• either car brake, pulley, or bicycle pump• either arrows or no arrows

QuickTime™ and aPlanar RGB decompressor

are needed to see this picture.

0

0.5

1

1.5

2

2.5

3

Structural Functional Structural Functional

No Arrows Arrows

Experiment 1: Descriptions from Diagrams M

ean

nu

mb

er

of

un

its

Experiment 2: Diagrams from Descriptions

How did participants depict the descriptions?

Structural Functional Description Description

0

10

20

30

40

50

60

70

80

90

100

Number of Participants

Structural description Functional description

No Arrows Arrows No Arrows Arrows

Experiment 2: Diagrams from Descriptions

Learning complex systems

• Diagrams with/without arrows

• Structural/function text

• Structural questions

• Functional questions

• Hi/lo mechanical ability

Results

• Hi-ability: learn structural & functional from either diagram

• Lo-ability: learn structural from diagrams, learn functional from text

Diagram narrative: linked graphics

• Temporal

• Causal/Logical

• Whole to part/part to whole

• Structure to function

• Variations/types

Animation

• Review: Animated graphics no better than equivalent static for instruction

(Tversky, Morrison, & Betrancourt, 2001)

Animation

• Review: Animated graphics no better than equivalent static

• Animations: hard to perceive• Too fast, too complex• Disappear

• Animations: hard to conceive• Conceive of events as sequence of discrete

steps (Zacks, Tversky, and Iyer, 2001)

Cognitive Design Principles

• Principle of Apprehension: structure & content of graphic should be readily & accurately perceived & comprehended

• Principle of Congruity: structure & content of graphic should match structure & content of desired mental representation

Route maps & directions

• Generalize turn angle to ~ 90

• Generalize curves in roads

• Diminish long straight distances

• Enlarge short tricky turns

Linedrive map at mapblast.com

How to make effective animations

• Distort time and space to match desired mental representation

• Study demonstrations for clues: How does demonstrating differ from doing?

(Lozano and Tversky)

Assemble TV cart from photo

Heiser & Tversky

Introduce

Segment actions into steps

Step Initiation Step Completion

Preview steps

QuickTime™ and aVideo decompressor

are needed to see this picture.

Exhibit large parts

Point to small parts

Make actions visible

Reassembler Demonstrator

Model structure

Model action

Conclude

Creating effective spaces for thought

• Survey examples for devices that work

• Discover desired mental representations of space and time

• Discover graphic devices that convey them

• Test

• Repeat….

External representations of thought

• Cognitive tools to augment mind• Increase memory• Facilitate information processing

• Uniquely human

Some ways graphics augment cognition

• Record information

• Convey information

• Promote inferences

• Enable new ideas

• Facilitate collaboration

Animations tell stories

• Links are temporal

Two kinds of graphics

• Visualizations of inherently visual• Maps• Ancient

• Visualizations of metaphorically visual• Graphs, charts, diagrams• Modern

Babylonian clay map

Eskimo coastal map

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

South Sea Islanders’ map

18th c. Graph

QuickTime™ and aPhoto - JPEG decompressor

are needed to see this picture.

Good graphics schematize

• Not “realistic;” omit irrelevant

• Emphasize, simplify, distort relevant

Creating effective spaces for thought

• Survey ancient and spontaneous examples; these have undergone natural user-testing

• Discover underlying mental representations

• Discover graphic devices that convey them

• Test

• Repeat….

Graphics consist of

• Elements

• Spatial relations among them

• These can convey meaning directly

Spatial Relations

• Categorical: spacesbetweenwords, ( )

• Ordinal: lists, outlines, trees

• Interval: graphs

Producing “graphs”

• 3 language cultures: USA, Hebrew, Arabic

• K-College

• Space, time, quantity, preference Tversky, Kugelmass & Winter

Meaningful use of space

• Proximity in space signifies proximity on abstract dimension

• Directionality• Vertical loaded: up is more, better, stronger• Horizontal neutral• Parallels in language & gesture

Mixing spatial metaphors

Elements

• Iconic

Elements

• Iconic

• Metaphoric: Figures of depiction• Synecdoche: part for whole• Metonymy: associate for whole

Elements

• Iconic

• Metaphoric: Figures of depiction

• Schematic (meaningful abstract forms): lines, curves, crosses, blobs, bars, and arrows

Meaningful abstract forms

• Lines, crosses, arrows, blobs

• Like classifiers: stick, sheet, roll

• Context-specific meanings

• Meanings related to mathematical/Gestalt properties

Meaningful abstract forms

• Arrows in mechanical diagrams (Heiser)

• Bars and lines in graphs (Zacks)

• Curves, lines, crosses, & blobs in route maps (Lee)

Arrows

• Natural interpretation of directionality• Arrow heads• River beds

• Many uses, interpretations

QuickTime™ and aPlanar RGB decompressor

are needed to see this picture.

Arrows for Mechanical Systems

Arrows: temporal to causal reasoning

• No arrows: structural descriptions

• Arrows-->functional descriptions

• Structural descriptions: no arrows

• Functional descriptions-->arrows

Arrows

• Asymmetric--> symmetric <--> extent <-->• Connecting, pointing, labeling• Temporal: sequence• Causal• Movement: direction, manner• Change over time, increases/decreases• Forces

Producing Descriptions from Graphs

Please describe in a sentence what is shown in the graph above:

X

Y

A

B

•

•

X

Y

B

A

Two Categories of Description

• Discrete• higher—lower

• greater—less

• more—fewer

• stronger—weaker

• Trend• rising—falling

• increasing—decreasing

• function

• relationship

• trend

Proportion of Trend Responses

•

•

Female Male0

10

20

30

40

50

60

Height (inches)

Female Male0

10

20

30

40

50

60

Height (inches)

10-year-olds 12-year-olds0

10

20

30

40

50

60

Height (inches)

•

•

10-year-olds 12-year-olds0

10

20

30

40

50

60

Height (inches)

Discrete Domain

Bar Graph

Continuous Domain

Line Graph

Proportion of Trend Responses

•

•

Female Male0

10

20

30

40

50

60

Height (inches)

Female Male0

10

20

30

40

50

60

Height (inches)

10-year-olds 12-year-olds0

10

20

30

40

50

60

Height (inches)

•

•

10-year-olds 12-year-olds0

10

20

30

40

50

60

Height (inches)

12%0%

61%7%

Discrete Domain

Bar Graph

Continuous Domain

Line Graph

Producing Graphs from Descriptions

Height for males is greater than for females.

Height for 12-year-olds is greater than for 10-year-olds.

Height increases from females to males.

Height increases from 10-year-olds to 12-year-olds.

Discrete Domain

Discrete Description

Continuous Domain

Trend Description

Proportion of Line Graphs

Height for males is greater than for females.

Height for 12-year-olds is greater than for 10-year-olds.

Height increases from females to males.

Height increases from 10-year-olds to 12-year-olds.

65%30%

100%56%

Discrete Domain

Discrete Description

Continuous Domain

Trend Description

Bars and Lines: Summary

• Constructing descriptions from depictions• Bars--> discrete comparisons • Lines--> trends

• Constructing depictions from descriptions• Discrete comparisons-->bars

• Trends-->lines

Route Maps & Route Directions

• Are the schematizations similar?

• Ask hungry students how to get to Taco Bell

Original Map 1

Original Map 2

Directions 1

• From Roble parking lot• R onto Santa Theresa• L onto Lagunita (the first stop sign)• L onto Mayfield• R onto Bowdoin• L onto Stanford Ave• R onto El Camino• go down a few miles. It’s on the right.

Directions 2

Go down street toward main campus (where most of the buildings are as opposed to where the fields are) make a right on the first real street (not an entrance to a dorm or anything else). Then make a left on the 2nd street you come to. There should be some buildings on your right (Flo Mo) and a parking lot on your left. the street will make a sharp right. Stay on it. That puts you on Mayfield Rd. The first intersection after the turn will be at Campus Dr. Turn left and stay until you get to El Camino. Turn right (south) and Taco Bell is a few miles down on the right.

Segmentation according to Denis

• Start point: “you leave the building”

• Reorientation: “you turn left”

• Path/progression: “you follow Main St.”

• End point: “until you get to the restaurant”

• Make, take, turn

• Go down

Semantics of depictions & descriptions

• Follow around

• Landmarks

• Direction

Graphic “Language”

• Semantics: schematic elements• Not iconic• Gestalt/geometric meanings• Categorical• Combinatoric

• Syntax: combine elements by rules

Gesture alone was best

• Gesturers received higher ratings

• Gesturers included more essential features

• Gesturers assembled faster/more accurately

Some ways diagrams augment cognition

• Record information

• Convey information

• Promote inferences• Structural, functional• Bars/lines; arrows

• Enable new ideas

• Facilitate collaboration

How to stop cholera epidemic Snow,

Where to reinforce bombers? Wald, WWII

Some ways diagrams augment cognition

• Record information

• Convey information

• Promote inferences

• Enable new ideas: ambiguity useful

• Facilitate collaboration

Suwa & Tversky

Some ways diagrams augment cognition

• Record information

• Convey information

• Promote inferences

• Enable new ideas

• Facilitate collaboration• Externalize common ground• Platform for inferences/plans/decisions

Heiser & Tversky

Why do graphics communicate effectively?

• Spatial inferences easy (e. g., proximity, distance, direction)

• Spatial metaphors available (e. g., time, value, strength)

• Elements interpretable (icons, figures of depiction, schematic abstract forms)

Spatial Thinking

• Began with world as we perceive it and interact with it

• Perception & action differ for different spaces, yielding different conceptions of spaces

• Unlike geometer’s view of space

• Unlike classic view of imagery

Multiple mental spaces

• Space around the body: biased retrieval

• Space of navigation: systematic errors

• Space of thought: meaning through metaphor

Conceptions of space have profound effects

• Augment memory, inference, discovery, collaboration

• Distort memory, bias information representation and retrieval

Onward and upward !