Analysis of Student Learning in Global Change

Eugene S. Takle1, Heather Moser1, and Elsebeth K. Sorensen2

1Iowa State University, Ames, IA2Alborg University, Denmark

gstakle@iastate.edu

Outline

• Context for Dialog

• Theoretical Background

• Dialog Implementation Strategies

• Diagnosis by “Theory of Language Games”

• Evaluation of student learning

Context for Dialog

• Global Change course– 3 credit , senior-level, on-campus or remote– 42 GC topics (1 each MWF), 3 5-week blocks– Online with dialog since 1995– Currently enrolls 50 students from 23 disciplines

• Course publicly available:

http://www.meteor.iastate.edu/gccourse

Characteristics of “Structure-less” Dialog

• Superficial remarks

• Off-the-cuff comments

• Personal agendas

• Religious fervor

• Cutsey comments

• Occasional glimmer of critical thinking

The Challenge:

How do we stimulate learning in online dialog?

Theoretical Framework:Learning in Virtual Environments

• Wenger (1998) : true collaborative learning occurs through “negotiation of meaning”

• Sorensen and Takle (2002)– mutual exploration of issues– mutual examination of arguments, agreements,

and disagreements– mutual questioning of positions– dynamic interaction– weaving of ideas– convergence of perspectives

“Genuine Collaboration”

“Genuine collaboration” (Salomon, 1995) is a condition of “genuine interdependence” between individuals that calls for

– sharing knowledge/information– adopting complementary roles– a “pooling together of minds”

So how do we structure our virtual environment for dialog to ensure that learning takes place?

I. Active Participation

Although some learning occurs in passive observation, active participation is essential for true collaborative learning

Put requirements on minimum number of postings: 6 per block

Table 1. A = # students, B = # comments per student per learning unit, C = #comments per student per block.____________________________________________Year A B C____________________________________________1995 32 0.069 1.01996 31 0.106 1.61997 32 0.350 5.31998 33 0.390 5.91999 26 0.320 4.82000 45 0.480 7.22001 44 0.382 5.72002 46 0.528 7.6____________________________________________

Result:

Increased volume of comments, but content of comments was shallow and did not give evidence of critical thinking

II. Quality

• Define critical thinking skills (CTS), give examples, and demand that student dialog must demonstrate CTS

• Implement an online threaded dialog structure that requires student to state, in advance of posting, the CTS being used

• Ask student in end-of-block self assessment to defend their use of CTS

Time

Quiz

ResearchPaper

Disc. Sum.

Dialog

Exam

Simulation

Animation

Hypothesis

Calc

QuizDialog

WebSearch

Disc. Sum.S

tud

ent

Pro

du

cts

Animation

Hypoth.

Exam

WebSearch

Hypothesis

Simulation

ResearchPaper

Calc

Elements of the Virtual Portfolio

Block 1 Block 2

Time

Stu

den

t P

rod

uct

s

Evalu

ate Pro

du

ct

Elements of the Virtual Portfolio

Block 1 Block 2

Time

Stu

den

t P

rod

uct

s

ProcessEvaluate

Elements of the Virtual Portfolio

Block 1 Block 2

Result:

By a subjective measure (0-10), quality went from 4.4 (1995) to 3.2 (1997) to 5.3 (2000)

It worked, but online entries became a collection of monologs

III. Action-Reaction

Require students to respond to 3 other students per block and to elicit comments from 3 other students per block.

Require one “social” or “community-building” comment per block.

Table 2. Comparison of comment characteristics when requirements for social comments were added.

___________________________________________2001 2002 Chg(%)

___________________________________________Number of students 44 46 +4Required # of posts/st. 15 18 +20Total number of posts 760 1,045 +38Actual # posts/student 17.3 22.7 +31% actual to required 115 126 +10Length of post (words) 140 155 +11______________________________________

True collaborative learning?

IV. Collaborative Learning Through Substantive Interaction

• Close, but not there yet• Not sufficient evidence we have true

“negotiation of meanings” (Wenger)• Or “true interdependence” (Salomon)• Next Step:

– Analyze dialog by use of the theory of “language games”

Language Game Theory

}}}}

Closes language game opened by

• A language game must have, as a minimum, the following elements:– Initiation (open a new game)– Expectation (offers that new understanding may emerge if

response is provided)• Clarify a misconception, resolve a conflict• Might result in initiation of a subgame (requesting more information,

clarifying an assumption, expanding the game beyond the initiation concept)

– Closure • All expectations are met, conflicts resolved

Language Game Theory

Assumption and Hypothesis

Assumption:The number of completed Language Games The number of completed Language Games

developed within a dialog in some way (yet to bedeveloped within a dialog in some way (yet to bedetermined) will be proportional to the amount of determined) will be proportional to the amount of collaborative learning that takes place.collaborative learning that takes place.

HypothesisHypothesisThe amount of collaborative learning that takes placeThe amount of collaborative learning that takes place

will increased by use of the knowledge-building process.will increased by use of the knowledge-building process.

• A random number generator was used to generate numbers that directed which comments would be selected• 1,350 comments from 1996-2004 were selected for consideration• Of these, 689 comments described a situation suitable for analysis• Of these 689 comments, 356 (52%) were considered to be candidates for initiating a language game• Of the 356 potential games, 242 (68%) were actually created• Of the 242 games created, 217 (90%) were closed• Of those comments (689-356=333) for which no game was created, only 124 (37%) received follow-up

comments

Experimental Procedure and Analysis

Interpretation% of candidate comments for

Year which a game was completed

1996 17.21997 15.71998 24.71999 38.12000 36.52001 48.82002 35.72003 30.12004 31.6

Mean = 19%

Mean = 37%

Conclusions• Online dialog is a powerful option for

promoting reflective thinking and use of critical thinking skills

• Students will respond if we tell them what we want them to do

• Developing online learning environments based on sound pedagogical principles improves likelihood of success

• Implementation of the knowledge-building process enhanced student learning

http://www.meteor.iastate.edu/gccoursegstakle@iastate.edu