Embed Size (px)
Citation preview
I T E R AT I V E P E E R C O L L A B O R AT I O N S Y S T E M
E S S A Y W R I T I N G I N O N L I N E E D U C A T I O N
E U N K W A N G J O O & K E L LY PA R K M I M S 2 0 1 4 F I N A L P R O J E C T
Enhance the students’ experience by focusing on scalable student-to-student interaction in large size classes in online education!
O N L I N E E D U C AT I O N G R O W S U P FA S T
Babson Survey Research Group, 2013http://www.onlinelearningsurvey.com/reports/changingcourse.pdf
0%
10%
20%
30%
40%
2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Online Enrollment as a Percent of Total Enrollment
L I M I TAT I O N O F O N L I N E E D U C AT I O N
L I M I TAT I O N O F O N L I N E E D U C AT I O N
Peer Interaction
Collaborative learning
L I M I TAT I O N O F O N L I N E E D U C AT I O N
Asynchronous learning
L I M I TAT I O N O F O N L I N E E D U C AT I O N
Large class size
Huge online discussion board
L I M I TAT I O N O F O N L I N E E D U C AT I O N
Large class size
Huge online discussion board
W H Y C O L L A B O R AT I V E L E A R N I N G ?
• Result in increase of students’ interest in learning
• Encourage to become critical thinkers
• Help students retain the information longer and be more satisfied
F O R E S S AY W R I T I N G
Iterative Peer Collaboration System (IPCS)
W H Y I P C S ?
• Simulate dynamics among students in a real class online
• Discuss with peer students in a format of essay writing and review process
• Provide quantified measures of how their arguments are developed
• Learning through iterative idea developments
H O W I P C S W O R K S
WRITE
H O W I P C S W O R K S
WRITE
H O W I P C S W O R K S
WRITE
H O W I P C S W O R K S
FEEDBACK
H O W I P C S W O R K S
FEEDBACK
H M M . . M Y P E E R T H I N K S D I F F E R E N T LY F R O M M I N E . B U T,
WA I T, H I S A R G U M E N T H A S A G R E AT P O I N T.
H O W I P C S W O R K S
FEEDBACK
H E R E I S M Y F E E D B A C K
H O W I P C S W O R K S
FEEDBACK
H E R E I S M Y F E E D B A C K
H O W I P C S W O R K S
DEVELOP
H O W I P C S W O R K S
DEVELOP
H O W I P C S W O R K S
DEVELOP
N O W I U N D E R S TA N D T H E Q U E S T I O N B E T T E R
H O W I P C S W O R K S
DEVELOP
N O W I U N D E R S TA N D T H E Q U E S T I O N B E T T E R
H O W I P C S W O R K S
ITERATE
F E E D B A C K A N D D E V E L O P
H O W I P C S W O R K S
ITERATE
F E E D B A C K A N D D E V E L O P
H O W I P C S W O R K S
ITERATE
F E E D B A C K A N D D E V E L O P
H O W I P C S W O R K S
H O W I P C S W O R K S
H O W I P C S W O R K S
H O W I P C S W O R K S
I P C S I N A C T I O N
• Finding the right fit for the IPCS
O N L I N E C L A S S
E S S AY A S S I G N M E N T A R G U M E N TAT I V E E S S AY
M O O C ( L A R G E C L A S S )
T I M E L I N E P R E - S E T C U R R I C U L U MN E V E R B E E N T E S T E D
E X P E R I M E N TA L D E S I G N
I P C S I N A C T I O N
• Finding the right fit for the IPCS
E S S AY A S S I G N M E N T A R G U M E N TAT I V E E S S AY
T I M E L I N E P R E - S E T C U R R I C U L U MN E V E R B E E N T E S T E D
P I L O T S T U D Y O N I P C S
• 6 students used IPCS for an assignment on INFO 205 'Information Law and Policy' at School of Information
• The assignment required an about 2,000 words long argumentative essay using related court cases or course materials
P I L O T S T U D Y O N I P C S
P R O C E D U R E
A S S I G N M E N T D U E I N C L A S S
W R I T E F E E D B A C K D E V E L O P
F E E D B A C K S U B M I S S I O N
2 N D S U B M I S S I O N
DD-1D-2D-3D-4
1 S T S U B M I S S I O N
S E L F - , P E E R - R AT I N G
S C O R I N G M A T R I X F O R E S S A Y
C R I T E R I O N 4 3 2 1
D O E S T H I S E S S AY I N C L U D E K E Y I S S U E S A N D
T H E M E S ?
R E L E VA N T I S S U E S I N C L U D E D
M O S T O F R E L E VA N T I S S U E S I N C L U D E D
S O M E R E L E VA N T I S S U E S I N C L U D E D
R E L E VA N T I S S U E S R A R E LY I N C L U D E D
D O E S T H I S E S S AY H AV E C O H E R E N T
G E N E R A L P I C T U R E ?
T H O R O U G H U N D E R S TA N D I N G O F H O W E V E N T S
A R E L I N K E D
U N D E R S TA N D I N G O F H O W E V E N T S
A R E L I N K E D
S O M E U N D E R S TA N D I N G O F H O W E V E N T S
A R E L I N K E D
N O U N D E R S TA N D I N G O F H O W E V E N T S
A R E L I N K E D
D O E S T H I S E S S AY S H O W
I N D E P E N D E N T T H I N K I N G ?
I N D E P E N D E N T T H I N K I N G A N D
A N A LY T I C A P P R O A C H
S O M E I N D E P E N D E N T
T H I N K I N G
L I T T L E I N D E P E N D E N T
T H I N K I N G
N O I N D E P E N D E N T T H I N K I N G
D O E S T H I S E S S AY S H O W C R I T I C A L
T H I N K I N G ?
C R I T I C A L E VA L U AT I O N A N D
T H I N K I N G
AT T E M P T S AT C R I T I C A L
E VA L U AT I O N
V E R Y L I T T L E E F F O R T I N C R I T I C A L E VA L U AT I O N
N O E F F O R T I N C R I T I C A L
E VA L U AT I O N
D O E S T H I S E S S AY U S E T H E
L I T E R AT U R E ?
S E V E R A L R E F E R E N C E S ,
A C T I V E S E A R C H O F R E F E R E N C E S
I N C L U D E S R E F E R E N C E S O T H E R
T H A N ‘ T H E M A I N R E F E R E N C E ’
O N LY ‘ T H E M A I N R E F E R E N C E ’
N O R E F E R E N C E S , E X C E P T
D I S C U S S I O N S
A N A LY S I S M E T H O D S
Cosine similarity
• Assign Term Frequency-Inverse Document Frequency (TF-IDF) score for each word
a writing = bag of words [{word1: tf-idf}, {word2: tf-idf}, {word3: tf-idf}, ...]
A N A LY S I S M E T H O D S
Grammatical similarity
• analyze up to 50 grammatical relations of each sentence of a writing (powered by Stanford CoreNLP tool)
I.E. "NSA violates restrictions."
"direct object" dobj(violates, restrictions)
I N C R E A S E D P E E R T O P E E R E S S AY S I M I L A R I T Y
• Average of pair-wise comparisons between peers by cosine similarity increased 54% • (t (58) = 6.23, p < 0.05)
C O S I N E S I M I L A R I T Y B A S E D O N T F - I D F
0
0.07
0.14
0.21
0.28
1st Essay 2nd Essay
IPCS helps students learn from each other
I N C R E A S E D P E E R T O P E E R E S S AY S I M I L A R I T Y
• Average of pair-wise comparison between peers by F score on increased 36% • (t (58) = 2.23, p < 0.05).
F S C O R E B A S E D O N G R A M M AT I C A L D E P E N D E N C Y
0
0.015
0.03
0.045
0.06
1st Essay 2nd Essay
IPCS helps students learn from each other
I N C R E A S E D S E L F - A S S E S S M E N T S C O R E O N E S S AY S
• Average self assessment score on essays significantly increased over the feedback and review process (t (10) = 2.78, p < 0.05)
IPCS helps students improve their essays
1
2
3
4
Self assessment 1 Self assessment 2
C H A N G E S I N S E L F A S S E S S M E N T N E G AT I V E LY C O R R E L AT E S W I T H E S S AY S I M I L A R I T Y
• The more changes in essays over the review and feedback process were made, the students assess their essay higher
• Correlation coefficient = -0.88
IPCS helps students improve their essays
0.00
0.28
0.55
0.83
1.10
0 0.25 0.5 0.75 1
Cosine similarity between 1st and 2nd essays
Cha
nges
in s
elf a
sses
smen
t
F I N D I N G S F R O M P I L O T S T U D Y
• IPCS propagates ideas among students, resulting in essays in the second iteration become more similar
• Self-rating at their second iteration of essay significantly higher
D I S C U S S I O N
• IPCS helps students
• establish his/her ability of critical thinking
• have a chance to critically review his/her essay
• review his/her own essay in a different view
D I S C U S S I O N
• Limitation of pilot study
• Number of participants
• Not a part of curriculum: low commitment
• Offline class
• A case study, not experiments
D I S C U S S I O N
• Future research
• MOOC classes with writing assignments
• Integrate IPCS into original curriculum using for assignment submission system
S P E C I A L T H A N K S T O Z A C H A R Y, D E I R D R E , & O U R PA R T I C I PA N T S I N P I L O T
S T U D Y
THANK YOU !
