E19.2174 Cognitive Science and Educational Technology I Visual Learning Jan L. Plass New York Univerisity Jan L. Plass New York Univerisity Center for

E19.2174 Cognitive Science and Educational Technology IE19.2174 Cognitive Science and Educational Technology IVisual Learning Visual Learning

E19.2174 Cognitive Science and Educational Technology IE19.2174 Cognitive Science and Educational Technology IVisual Learning Visual Learning

Jan L. PlassNew York Univerisity

Jan L. PlassNew York Univerisity

Center for Research and Evaluation of Advanced Technologies in EducationCenter for Research and Evaluation of Advanced Technologies in Education

Overview

Visual Learning

You are asked to design a visualization for educational purposes, for example, to learn about

• The ideal gas laws• The carbon cycle• Bird migration patterns• The system of voting districts in the U.S.• Air traffic control• The history of the Gulf War

Overview

Visual Learning

Visual LearningVisual LearningVisual LoadVisual LoadVisual EnvironmentVisual Environment

Emotional DesignEmotional Design

Cognitive DesignCognitive DesignCognitive VariablesCognitive Variables

Attitudes, MotivationAttitudes, Motivation

Visual Learning

• Learning from primarily visual materials• Text only as labels or brief statements

• Examples: – Graphs, charts, maps, networks,

pictures, video, animation

Visual Learning

Visual Learning

How do Visual and Verbal Information differ from one another?

Visual Learning

• Visual information:• analogous representations • inherently relational• encoded simultaneously

• Verbal information: • discreet units of symbolic information • propositional• processed sequentially

Visual v. Verbal Information

Visual Learning

Dual Coding Theory(Paivio, 1986, 1990)

VERBAL STIMULI NONVERBAL STIMULI

SENSORY SYSTEMS

VERBAL SYSTEM NONVERBAL SYSTEM

REPRESENTATIONAL CONNECTIONS

REFERENTIALCONNECTIONS

Logogens

VERBAL RESPONSES NONVERBAL RESPONSES

Imagens

REFERENTIALCONNECTIONS

AssociativeConnections

AssociativeConnections

Visual Cognitive Load

• Intrinsic LoadLoad related to complexity of the informationElement interactivity

• Extraneous LoadLoad pertaining to format and design of the interface (presentation mode, modality, temporal & spatial arrangement, representation type)

• Germane LoadMental effort expended by learner

Cognitive Load Components (Sweller, 1999)

IntrinsicLoad

ExtraneousLoad

GermaneLoad

Working Memory

Free


• Cognitive Load for Visual Representations:

• Intrinsic Visual LoadVisual element interactivity

• Extraneous Visual LoadVisual format and design of the interface (presentation mode, modality, temporal & spatial arrangement, representation type)

Lee, Plass, & Homer (2006)


Visual Learning Environments

• Highly visual learning environments

• Examples– Simulations, virtual worlds,

microworlds, games

Visual Learning Environments

Introduction

Examples

Ideal Gas Law (Oklahoma State University)

Introduction

Examples

Odyssey Simulation Package

Introduction

Examples

Gizmo/ ExploreLearning

Introduction

Examples

Molecular Workbench

Introduction

Examples

Schnotz & Rasch (2005)

Introduction

Examples

Ideal Gas Laws(NYU Molecules & Minds project, IES)

Introduction

Examples

Virtual Patient(Abdominal Exam)

NYU Schoolof Medicine

Overview

Visual Learning





Visual Learning

Group Discussion (3-4 students, 15min)

Discuss Design Principles that increase the effectiveness of visual representations for learning (Animations and Simulations)?

–List and Discuss principles from the assigned reading

–Find and discuss examples

Cognitive Design Factors

• Representation of information (Information Design)• Instructional Approach (Interaction Design)• Interactivity (Interaction Design)• Function of Visuals (in support of cognitive processes)• Scaffolds• Feedback• Narrative structure



Which mode of relationship between signs and their referents best facilitates learning?

• Icon: Most basic representation, relies on physical resemblance to convey meaning

• Symbol: Abstract, arbitrary, relies on social conventions for meaning (Peirce, 1956)

Question of Interest: • Comparison of Iconic v. Symbolic representations

Representation of Information (Semiotics)

Research Materials

• Ideal Gas Law

Chemistry Simulations

Research Materials

• Ideal Gas Law


Does adding icons facilitate learning in chemistry simulations?

• Study with 93 11th grade students in a NYC high school:• Adding icons increased recall• Icons especially helped learners with low prior

knowledge

(Lee, Plass, & Homer, 2006; Plass et al., 2007)

Representation of Information (Semiotics)

Results: Representation

Visual Design of Simulations

Which instructional approach best facilitates learning?

Consider:• Difficulty of content: Intrinsic Cognitive Load• Complexity of interactions: Extraneous Load• Educational goals / Cognitive Function of materials• Learner characteristics

Option• Direct instruction v. guided exploration

Instructional Approach: Level of Learner Control


Which instructional approach best facilitates learning?

• Comparison of direct instruction v. guided exploration

In other words:• Comparison of Worked-out example (Animation) v.

Exploration (Simulation)

Or, in even different terms:• Kirschner, Sweller, & Clark (2006) v. Everybody Else

Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching

Instructional Approach: Level of Learner Control

Research Materials

• Kinetic Theoryof Heat


Research Materials

• Ideal Gas Law

Worked-outExample

Does ability to manipulate parameters facilitate learning?

• Study with 93 11th grade students in a NYC high school:

• For comprehension:Simulation Direct Exploration > Instruction

(Plass et al., 2007)

Results: Simulation (exploratory) vs. Animation (worked-out)

Results: Instructional Format

Worked Out/Explore

ExploreWorked Example

Mean M.C. Posttest

4.0

3.0

2.0

1.0

0.0


Example

Level of Interactivity


Example

Level of Interactivity


What purpose does the visual information serve in the construction of mental models? (Plass, 1998)

• Levin, Anglin, & Carney (1987): 5 functions (heuristic)• Decorative motivational function; little relation to content

• Representative depicts content of the instruction

• Organizing depicts knowledge structures • Interpreting visualizes abstract concepts

• Transforming supports higher-level cognitive processes

Cognitive Function of Visual Information


Cognitive Theory of Multimedia Learning (Mayer, 2001)

SelectImages

SelectWords

Organize Images

Organize Words

Sounds

Images

Integrate

Verbal Model(Verbal Mental Representation)

Pictures

Words

Multimedia Presentation

Ears

Eyes

Sensory Memory

Pictorial Model (Visual Mental Representation)

Long-Term Memory

Prior Knowledge

Working Memory


What purpose does the visual information serve in the construction of mental models? (Plass, 1998)

• Our approach: Define function based on Mayer’s CTML• Selecting• Organizing• Integrating

• Different types of visuals support different learning outcomes (recall, comprehension, transfer)(Plass, Hamilton, & Wallen, 2004; Wallen, Plass, & Brünken, 2005)

Cognitive Function of Visual Information


Function of Multimedia Aids in Text Comprehension


• Split Attention PrincipleAvoid requiring learners to split their attention between, and mentally integrate, several sources of physically or temporally disparate information, where each source of information is essential for understanding the material.’’ (Ayres & Sweller, 2005)

• Modality PrinciplePresent animation with narration rather than with on-screen text (Mayer, 2001)

• Contiguity PrinciplePresent related information near to each other in time and space (Mayer, 2001)

Established Cognitive Design Principles


• Cueing Adding design elements that direct learners’ attention to the important part of a simulation reduces cognitive load and enhances learning(Dwyer, 1978, Jeung et al., 1997; Tabbers et al., 2004; de Koenig el al., 2007)

• Representation of InformationAdding iconic representations can enhance learning, especially for learners with low prior knowledge (Lee et al., 2006; Plass et al., 2009)

• Color CodingUse color to highlight important features and attributes of the visual display (Dwyer and Moore, 1991; Keller et al., 2006)

• Multiple Dynamic Visual RepresentationsMultiple dynamic representations should be integrated and linked (van der Meij & de Jong, 2006)

Emerging Visual Design Principles


• Learner Control of SegmentingLearner control over the advancement from one segment of visual materials to the next improves learning(Mayer & Chandler, 2001; Mayer et al., 2003; Moreno, 2007)

• Guided Discovery Principle Provide guidance in discovery-based learning environments(de Jong, 2006; de Jong & van Joolingen, 1998; Kirschner et al., 2006; Mayer, 2004)

Established Interaction Design Principles


• Learner Control of Pacing Learner control over the pace of the presentation of visual materials improves learning(Hasler et al., 2007; Schwan & Riempp, 2004; Tabbers et al., 2004)

• Task-Appropriate Representations –Simulations need to prepare learners for future tasks to be performed–Facilitating, Enabling, or Inhibiting Effects –Cognitive Function of Simulations (Retention, Understanding, Transfer) (Carney & Levin, 2002; Levin et al., 1987; Plass, Wallen, & Hamilton, 2004)

• Content-Manipulating Interactivity Learner control over the content of visual materials improves learning(Chandler, 2004; Hegarty, 2004; Rieber, 1990, Wouters et al., 2007)

Established Interaction Design Principles

Emotional Design Factors

• Visual Design (Information Design)• Control (Interaction Design)• Feedback• Intrinsic motivation v. Extrinsic motivation• Social interaction• Social Presence/Telepresence

Emotional Design Factors

Overview

Visual Learning






• Personalization PrincipleLearning more deeply when words in a multimedia presentation are in conversational rather than formal style(Mayer, 2005)

• Social Presence HypothesisLearning is facilitated by giving learners a sense of the presence of others in a learning environment This effect is expected to be especially strong in self-learning

Other Emotional Design Principles

Visual Learning

Group Activity (3-4 students, 30min)

Apply the Design Principles we discussed to your own projects by designing a simulation or animation.

–Select which topic to cover-Discuss which principles apply–Describe how you will apply the principles for the information design and interaction design of the simulation

Documents

E19.2174 Cognitive Science and Educational Technology I Visual Learning Jan L. Plass New York Univerisity Jan L. Plass New York Univerisity Center for