Ming Ming Chiu State University of New York – Buffalo

Effects of Social Metacognition on Micro-Creativity:

Statistical Discourse Analyses

of Group Problem Solving

Ming Ming ChiuState University of New York – Buffalo

mingchiu@buffalo.edu I appreciate the research assistance of

Choi Yik Ting and Kuo Sze Wing

Under the Universal Texting plan, each text message costs $.10. Budget Texting costs $.01 per text message, but charges a monthly fee, $18.

1) How many text messages do you send each month?

2) Which company costs less for you?

3) How many texts should you send for the Universal plan and the Budget plan to cost the same?

Solving problems & Micro-creativity

• Difficult problem for students learning algebra• To solve this problem, novice students create

new ideas and check/justify their utility (micro-creativity processes).

• More micro-creativity processes

Solve problem• What group processes

micro-creativity processes?

Solving problems & Micro-creativity

Micro-Creativity Processes • Creativity processes

–Generate ideas– Identify/Justify utility

( Sternberg & Lubart, 1999 ) • Big “C” creativity affects society• Small “c” creativity affects person

( Gruber & Wallace, 1999 ) • Micro-c creativity processes occur at a

moment in time( Chiu, 2008 )

What Affects Micro-creativity?

• Social Metacognition?

• Face / Rudeness?

Social MetacognitionMetacognition

Monitoring and control of one’s knowledge and actions

( Flavell, 1971; Hacker, 1998 )

Social Metacognition Group members’ monitoring and control of one another’s knowledge and actions

( Chiu, in press)

Most individuals have poor metacognition.( Hacker & Bol, 2004 )

Social MetacognitionQuestions indicate knowledge gaps

Identifies gap in someone’s understanding Motivates and points out a way to fill the gap

to create a new idea (+) Use old or new info to explain/justify (+)

(Coleman, 1998; Webb, Troper & Fall, 1995; DeLisi & Goldbeck, 1999 )

Disagree Identify obstacles

Overcome via new ideas and/or justifications (+)

(Doise, Mugny & Perret-Clermont, 1975; Piaget, 1985)

Face / Rude• Disagree Rudely

• Excessive Agreement

• Command !

Face / RudeFace = Public Self-image

Disagree rudely (attack face) vs. Disagree politely (save face)

( Brown & Levinson, 1987 ) “Ten times two hundred.”

Disagree Rudely “No, you’re wrong, it’s one tenth times two hundred.”

Previous speaker more likely to retaliate Emotional argument

Reduce new ideas & justifications () End cooperation

( Chiu & Khoo, 2003; Gottman & Krokoff, 1989 )

Face / RudeDisagree politely “if we want it in dollars,

we can multiply two hundred by one tenth.”

• “if” – Hypothetical distances error away• No “you” – No direct blame• “we” – Shared positioning & common cause

Save previous speaker’s face Listen & understand obstacle Overcome via new ideas & justifications (+)

( Chiu & Khoo, 2003 )

Face / RudeAgree too much

Concern for social relationship

Reluctant to disagree with wrong ideas

Fewer new ideas & justifications (–)( Person, Kreuz, Zwaan, & Graesser, 1995; Tann, 1979; Tudge,1989 )

Face / RudeCommand ! Demand implementation of an old idea

Suggest that speaker has higher status than audience

Ruder than question

Threaten face

Distract from problem solving

Fewer new ideas & justifications (–)(Brown & Levinson, 1987; Chiu,2008 )

Micro-creativity processes

New ideas

JustificationsFace / RudenessPolitely Disagree (+)Rudely Disagree (–)Excessively Agree (–)Command (–)

Social MetacognitionAsk Questions (+)Disagree (+)

Control variablesMath gradePeer FriendshipGender, ethnicity, …Group mean grade, Group gender variance …

Videotape Group Problem Solving• 84 9th grade, average ability students in US city

–Work in 21 groups of 4• Not friends

• Introducing 2 variable algebraic equations– 1st day of group work– No group work preparation– Work on problem for 30 minutes

• Videotape & Transcripts– Two RAs coded each student turn – Krippendorf’s

Content analysisJay: A hundred eighty dollars.Ben: If we multiply by ten cents, don’t we get

a hundred and eighty cents?• Ben

– Disagrees politely– New information– Correct– Justifies– Question

Multi-dimensional CodingEvaluation of the previous action

– Agree ( + ), Neutral ( n ), Ignore/New topic ( * ), Disagree rudely (––), Disagree politely (–)

Knowledge content regarding problem– New idea, Old idea, Null-content ( {} )

Validity– Correct ( ), Wrong ( X ), Null-content ( {} )

Justification – Justify ( J ), No justification ( [] ), Null-content ( {} )

Invitation to participate– Command ( ! ), Question ( ? ), Statement ( _. )

Invitational Form Decision TreeMinimize Number of Coding Decisions to inter-coder reliability

• Minimize Depth of decision tree • Put highly likely actions at the top

Do any of the clauses proscribe an action?• Yes, code as command (imperative)• No, is the subject the addressee?

– No, are any of the clauses in the form of a question?• No, code as statement (declarative)• Yes, code as question (interrogative)

– Yes, is the verb a modal?• No, should the described action have been performed, but not

done? – Yes, code as a command– No, code as a question

• Yes, Is it a Wh- question (who, what, where, why, when, how)?– Yes, code as an question– No, is the action feasible?

• Yes, code as a command• No, code as an question Based on Labov (2001), Tsui (1992)

Coded TranscriptID Action EPA KC Valid? Justify IFFay Do ten times eighteen. * C !

Ben Ten times eighteen is– + R _.

Eva Twenty-eight. + C X _.

Jay Wrong. A hundred eighty dollars. — C X _.

Ben If we multiply by ten cents, don’t we get a hundred and eighty cents?

- C J ?

Fay Yep. + _.

Add other variables at each speaker turn: Student: Gender, ethnicity, mid-year algebra grade, … Group: Group’s mean mid-year algebra grade, …

4 types of Analytical Difficulties

• Time

• Outcomes

• Explanatory variables

• Data set

Statistical Discourse Analysis

Statistical Discourse AnalysisDifficulties regarding Time· Time periods differ (T2 T4)

· Serial correlation (t8 → t9)

Strategies· Breakpoint analysis

Identify BreakpointsBreakpoints• Critical events radically change interactions

• Statistically identify breakpoints– Test possible combinations of breakpoints– Model with smallest Bayesian Info Criterion (BIC) Explain the most variance w/ fewest breakpoints

Breakpoints in 1 group%

Mic

ro-c

reat

ivity

0%

20%

40%

60%

80%

100%

0 10 20 30Time (mins)

% N

ew id

eas

Statistical Discourse AnalysisDifficulties regarding Time· Time periods differ (T2 T4)

· Serial correlation (t8 → t9)

Strategies· Breakpoint analysis · Multilevel analysis (MLn, HLM)

· Test with Q-statistics· Model with lag outcomes

e.g. Justify (-1)

Statistical Discourse AnalysisOutcome Difficulties

· Discrete outcomes (Yes / No)

· Multiple outcomes (Y1, Y2)New idea & Justify

Strategies

· Logit / Probit

· Multivariate, multilevel analysis

Statistical Discourse AnalysisExplanatory model Difficulties

· People & Groups differ

· Mediation effects (X→M→Y)

· False positives (+ + + +)

· Effect across turns (X6→Y9)

Effects across several turnsBen: 10 times 18 is

Eva: 28.

Jay: Wrong, 180 dollars.

2 speakers ago = (– 2)

1 speaker ago = (– 1)

Statistical Discourse AnalysisExplanatory model Difficulties

· People & Groups differ

· Mediation effects (X→M→Y)

· False positives (+ + + +)

· Effect across turns (X6→Y9)

Strategies

· Multilevel cross-classification · Multilevel mediation tests

· 2-stage linear step-up procedure

· Vector Auto-Regression (VAR)Lag explanatory

variablese.g., Disagree (-1), Girl

(-1) Disagree (-2)

VAR models effects across turnsID Action Justify Disagre

eDisagree (-

1)Fay Do ten times eighteen. 0 0 -

Ben Ten times eighteen is– 0 0 0

Eva Twenty-eight. 0 0 0

Jay Wrong. A hundred eighty dollars.

0 1 0

Ben If we multiply by ten cents, don’t we get a hundred and eighty cents?

1 1 1

Fay Yep. 0 0 1

Statistical Discourse AnalysisData Difficulties

· Missing data (101?001?10)

· Robustness

Strategies

· Markov Chain Monte Carlo

multiple imputation

· Separate outcome models

· Use data subsets

· Use unimputed data

Results: Breakpoints • 2.65 new idea breakpoints per group

3.65 time periods per group (min=1; max =6)

• 2.05 justification breakpoints per group3.05 time periods per group (min=1; max

=6)

• Number of breakpoints did not differ across groups that solved vs. did not solve the problem

3 Types of Breakpoints• Creativity process generators

– Sharply increase new ideas or justifications

• Creativity process dampeners– Sharply decrease new ideas or justifications

• On-task Off-task transitions

Creativity generatorAna How can they be equal?Bob I don’t know Cate Try another number?Dan Which number?

[8 seconds of silence; each student looks at own paper] Cate [looks at Ana’s paper] Yours is much closer.

So, try a number close to yoursDan [looks at Ana’s paper] Mine’s even closerAna [looks at Dan’s paper] Oh! More messages get us

closer

Creativity dampenerKay Let’s try a hundred.

Lee Ok. That’s a thousand.

Tom And that’s one, so nineteen.

Kay That’s like over nine hundred away.

Jan Maybe it’s one of those trick questions.

Tom Yeah, like it can’t be done.

Kay So, maybe there’s no answer.

Lee Then, we’re done.

New Idea

Justify

Agree

Rudely Disagree

Politely Disagree

Peer Friendship

Rudely Disagree (-1) * Unsolved

Rudely Disagree (-1) *Wrong (-2)

Rudely Disagree (-1)

Math grade (-1)

Math grade (-1) *Unsolved

Command (-1)

Previous turn (-1) Current turn OutcomesExplanatory model: New Idea & Justify

Group + Time Period DifferencesUnsuccessful groups:

Negative effect of Rudely disagree (-1) on new ideasNegative effect of Math grade (-1) on justifications

Mathematics grade’s effect on justificationsDiffered across both time periods and across groups-2% to +1% in unsuccessful groups -1% to +3% in successful groups

Unsupported HypothesesQuestions were not linked to

New idea or Justifications

Rudely disagreements were not linked to Justifications

Increase Group Micro-creativity

• Ask questions rather than issue commands !

• Disagree politely to encourage justifications

• Listen to rude disagreements and use the content

to develop new ideas

Implications for Teachers & Students

Implications for Researchers• Statistically identify critical moments

(breakpoints) that radically change subsequent processes

• Effects differ across groups, time periods, turns– Use statistical model to compute specific effect

• Effects of sequences – Look beyond the effects of single actions

• New method for statistically modeling conversations

Further applications… What major or momentary events affect people’s behaviors over time during …

–Classroom conversations?–Online discussions?–A student’s think-aloud problem solving? –An infant’s learning of a new word?–Basketball games?–Stock market transactions? –Wars?

Thank you!

ID Action Turn # Valid?Previous

TurnValid (-1)

Ana Do three times four. 1 – –Ben Three times four is seven 2 X 1 Eva Three times four is nine. 3 X 2 XJay Three times four is twelve. 4 3 X

ID Action Turn # Valid?Respondto turn #?

Valid (-1)

Ana Do three times four. 1 – –Ben Three times four is seven 2 X 1 Eva Three times four is nine. 3 X 1 Jay Three times four is twelve. 4 3 X

Statistical Discourse AnalysisAnalytical Difficulty· Differences across topics

· Time periods differ (T2 T4)

· Serial correlation (t8 → t9)

· Parallel talk (→→ )

Strategy

· Multilevel analysis

· Breakpoint analysis & Multilevel analysis

· I2 index of Q-statistics; Model with lag variables

· Store path: ID prior turn, Vector Auto-Regression

· Discrete outcomes (Yes / No)

· Multiple outcomes (Y1, Y2)

· Infrequent outcomes (00010)

· Logit / Probit

· Multivariate outcome models

· Logit bias estimator

· People & Groups differ· Mediation effects (X→M→Y) · False positives (+ + + +)

· Multilevel analysis· Multilevel mediation tests · 2-stage linear step-up procedure

· Missing data (101?001?10)

· Robustness

· Markov Chain Monte Carlo multiple imputation· Separate outcome models; Data subsets & unimputed data

Knowledge content, Validity, and Justification

Does the speaker express any mathematics or problem-related information?

• No, code as null content• Yes, is all the info on the group's log/trace of problem

solving?– Yes, code as repetition– No, code as contribution and write specific info in

group's log– Does this info violate any mathematics or problem

constraints?• Yes, code as incorrect• No, code as correct

– Does the speaker justify his or her idea? • Yes, code as justification• No, code as no justification

Mathematics

Bayesian Information Criterion

Regression specification

nnk

nL )ln(2

ijk = F(0 + f0jk + g00k+ 00sS00k +00tT00k+ujkUijk

+ vjkV(i-1)jk +vjkV(i-2)jk +vjkV(i-3)jk +vjkV(i-4)jk)