The Writing Transfer Project

Promoting Transfer through Reflection: A Cross-Institutional Study of Metacognition, Identity, and RhetoricDr. Dana Lynn Driscoll, Oakland UniversityDr. Gwen Gorzelsky, Wayne State UniversityDr. Carol Hayes, George Washington UniversityDr. Ed Jones, Seton Hall University

Special Thanks• The ERS Seminar on Critical Transitions: Writing and the

Question of Transfer• George Washington University for Year 1 funding for raters

and coders• Oakland University, Seton Hall University, and Wayne State

University for additional funds for research and travel• Conference on College Composition and Communication and

the Spencer Foundation for Grant Funding for Year 2 data analysis

• Our students and faculty involved in the study

Reflection as a Means to Understand Transfer and Metacognition: Pedagogy, Assessment, and Cross-Institutional ResultsDr. Dana DriscollDepartment of Writing and RhetoricOakland Universitydriscoll@oakland.edu

Background: Transfer• Students substantially struggle in transferring writing skills from high

school to college, throughout college courses, and from college to the workplace (Carson, Chase, & Gibson, 1993; Beaufort, 2007; Driscoll, 2011; Wardle, 2009; Beaufort, 2007).

• New views of transfer (Driscoll & Wells, 2012; Lobato, 2003; Schwartz, Bransford, & Sears; Slomp; Wardle) stress the interaction between individual and context as key to investigating how learning transfers from original to subsequent contexts. Like traditional theories, these new views highlight the role of metacognition in promoting transfer.

• Most of the transfer research we have is from single-institution studies and describes what is happening; the goal of our project is to approach transfer research from a cross-institutional, longitudinal perspective.

Background: WAW, Metacognition, and Reflection• Metacognition (described as one of the eight “Habits of Mind”) is a

student’s ability to recognize learning processes and to regulate their learning processes (Schraw and Dennison, 1994)

• Scholars contend that students’ struggle concerning transfer results, in part, from the entrenched approach to teaching FYW which has traditionally lacked content knowledge about writing (Beaufort, 2007). • Some argue that FYW should emphasize metacognition and acquaint

students with research and theory on writing by using the writing about writing (WAW) approach (Downs & Wardle, 2007).

• Scholars like Yancey(1998) and Taczak (2011) advocate using reflection in composition pedagogy because it both fosters transfer and reveals writers’ metacognition.

Research QuestionsOverarching research question: What is the effect of a WAW/highly rhetorical curriculum on transfer of learning, as explored in diverse writing course contexts?

This presentation addresses the following questions:• Are the courses in our study effective in producing better

writers? • Are the courses in our study effective in encouraging

metacognition and transfer in terms of: • transfer-focused thinking to the future?• transfer while students are enrolled in their initial course?• adaptation of prior knowledge to tasks while in the course?

• How does transfer-focused thinking show up beyond the courses?

Methodology – Study Sites & Participants• Year 1 participants are from….• 2 FYW sections at a mid-sized Northeastern Catholic University (Seton Hall

University, New Jersey) • 25 sections of FYW courses and 1 section of an upper-level course at a

suburban mid-sized public Midwestern university (Oakland University, Michigan)

• 9 sections of a sophomore/junior-level writing course at a large urban public Midwestern urban university (Wayne State, Michigan)

• 14 sections of FYW courses at a large private Northeastern urban university (George Washington University, Washington D.C.)

• Year 2 participants are a sampling of students from Year 1, and include….• 7 students from the two FYW sections at SHU • 15 students from the sophomore/junior-level sections at WSU• 7 students from the upper-level writing course at OU• 22 students from the FYW sections at GWU

Methodology – Data Collection• During Year 1, 2011-2012, we collected the following data:• pre- and post-semester writing samples to examine change over

time • self-reflections from across the semester to understand the role of

reflective writing in transfer and related areas • pre- and post-semester Writing Knowledge Transfer survey

responses about student writing strategies, beliefs, perceptions, and dispositions to understand self-reported beliefs/attitudes related to transfer

• During Year 2, 2012-2013, we collected the following data:• 60 minute in-depth interviews on writing, transfer, and related areas• WID writing samples from each interviewee• Follow-up Writing Knowledge Transfer Survey responses

Methodology – Rating and Coding• Development of shared research methods, analysis methods, and

rating/coding approaches• Met at ELON seminar for preparation and analysis work (2011, 2012, 2013);

met at GWU for a coding week in August 2012 & June 2013• Raters/coders were trained graduate student teachers working in the WID

program at GWU (24 in year 1; 31 in year 2)• They rated:• Student writing:

• 208 Year 1 writing samples (68 matched pairs)• 66 Year 2 writing samples• Examined: Audience, Role of Sources, Use of Genre: Purpose and Development,

Contextualizing; Genre-driven organization, style

• Student reflective writing: • 398 reflective pieces of writing with 98 possible codes; codes were applied 14,127

times• Coded 38 interviews in Year 2• Examined: Transfer-focused thinking, writing knowledge, metacognitive strategies, sense of

identity as writers, writing-related dispositions

Methodology – Data Analysis• Inferential and descriptive statistical analysis of the pre- and

post- writing samples from year 1 (paired samples t-test, effect size, means)

• Inter-rater reliability calculations were done to see how closely our raters and coders coded (on the fly during year 2, and post-rating during year 1)

• Codes were analyzed both qualitatively and quantitatively• Quantitatively, we examined patterns in the number of codes, the

kinds of assignments, and the kinds of students over time• Qualitatively, we examined the nature of the comments, how

students were framing their experiences• We are in the process of this analysis work now, and expect to

find much more as we continue to dig into the data (especially from year 2)

Year 1 Results: Writing Analysis• We were able to compare 68 sets of student papers from writing

collected prior to the beginning of the semester to the end of the term (across four universities). Students showed significant improvement in writing while enrolled in our courses (p<0.001; effect size 0.385).

Mean Paper Score2.5

2.6

2.7

2.8

2.9

3

3.1

3.2

2.775

3.1607

Prior to Start of TermEnd of Term

Year 1 Results: Writing Analysis

Audience

Use of Sources

Genre

Contextualizing

Organization

Style

Mean of all scores

0 0.5 1 1.5 2 2.5 3 3.5 4

2.9

2.74

2.71

2.63

2.69

3.06

2.78

3.19

3.13

3.15

3.04

3.03

3.37

3.16

End of WAW Semester Pre-Semester

All of the results are significant at the p<0.02 level or above.

Key areas for a Metacognitive Understanding of Transfer• Emphasis in this talk is on transfer-focused metacognitive

processes, which we call “transfer-focused thinking”• These include three areas (generated from our dataset):

• Anticipating connections to future contexts• What (learning strategies, writing and rhetorical knowledge, content

knowledge)• Where (educational, professional, personal, general connections)

• Prior Knowledge (from previous course to current writing course)• Anticipating prior knowledge’s usefulness in present setting• Using/adapting prior knowledge and dealing with challenges• Negative transfer

• Using Knowledge and Skills (from current course to other context)• What (learning strategies, writing and rhetorical knowledge, content

knowledge)• Where (educational, professional, personal, general connections)• Negative transfer / Challenges

Transfer-focused Thinking: Anticipating Future Connections

Type of Connection (What, where)

Number of times coded in reflective writing in 398 documents (year 1)

Number of times coded in 36 interviews (year 2)

Connection to future educational experiences (where) 119 12

Connection to future personal life (where) 10 1Connection to future professional life (where) 82 8

General connection to future (where) 94 4

Learning Strategies (what) 67 2

Non-writing content (what) 12 3

Unclear/unknown connection 3 2

Writing & Rhetorical knowledge (what) 212 13

Total Anticipating Connections Made 599 45

Transfer: Prior KnowledgeType of Prior Knowledge Number of

times coded in 398 documents (Y1)

Number of times coded in 38 interviews (Y2)

Anticipating Prior Knowledge 108 6

Challenge with Prior Knowledge 68 37

Negative/Unsure About Prior knowledge 46 14

Using prior knowledge unsuccessfully 6 5

Using/Adapting Prior knowledge successfully 29 66

Transfer: Using knowledge/skillsType of knowledge/skill use Number of

times coded in 398 documents (Y1)

Number of times coded in 38 interviews (Y2)

General use of knowledge/skills 31 122

Adapting knowledge 2 6Connection to current educational life (where) 23 76Connection to current personal life (where) 8 6Connection to current professional life (where) 4 12

Learning Strategies (what) 0 5

Negative Transfer 1 7

Non-writing content (what) 1 7

Writing & Rhetorical knowledge (what) 10 98

Reflective Example• “At the beginning of the semester after being introduced to the

framework of this course I was somewhat skeptical. The idea of being a pilot course for what sounds to be a study on our class to see how it can be beneficial to students in the future did not really seem too inviting….As time has gone on my perspective towards our English 3010 course has changed for the better. After being taught more in depth about the meaning of the ideas for this course such as “discourse community” and how it relates to our professional outlook I was able to grasp a better understanding for what the purpose of this course really is…For example, I feel as though being a novice in a particular field such as accounting, the idea of genre analysis and its function helps me to identify the different areas of writing within that field and its purpose and convention that I would not have been able to before.”

Discussion - ResultsTo summarize, our initial analysis of data suggests the following:• Reflective writing prompts that scaffold in a WAW course can be

useful in encouraging transfer-focused thinking, especially towards future activity

• Students report a *lot* of transfer-focused thinking and anticipating use of prior knowledge in year 1, but we see less evidence in their reflections of actual use in year 1 (by that we mean descriptions of what they did and how they used it)

• In year 2, however, students report transferring knowledge in their courses.

• We also saw very positive results in students’ growth as writers in all of these courses

Discussion – Reflective Prompts• Reflective writing prompts can be shared across

very diverse courses to better compare and study student writing and thinking at different levels and institutions• Reflection can be used as a pedagogical,

research, and assessment tool

Multi-Institutional Research• Need to go beyond single-site or single-class studies of

transfer. • Multi-institutional research allows for

generalizabilty/aggregability and understanding nuances in context.

• A multi-level study has the potential to reveal which factors related to knowledge transfer are more important at different points in a student’s development.

• In addition, we are developing a methodology for conducting multi-institutional writing transfer research that can be replicated and expanded by other scholars (as described in “Multi-Institutional Research and Generalizability: The Case for the Writing Transfer Project” currently in preparation)

References

• Beaufort, A. (2007). College writing and beyond: A new framework for university writing instruction. . Utah: Utah State UP.

• Carson, J. G., Chase, N. D., and Gibson, S. U. (1993). Academic demands of the undergraduate curriculum: What students need. Washington, DC: Fund for the Improvement of Postsecondary Education.

• Downs, D., & Wardle, E. (2007). Teaching about writing, righting misconceptions: (Re)Envisioning “first-year composition" as “introduction to writing studies.” College Composition and Communication, 58(4), 552-584.

• Driscoll, D. L. (2011). Connected, disconnected, or uncertain: Student attitudes about future writing contexts and perceptions of transfer from first-year writing to the disciplines. Across the Disciplines, 8(2).

• Driscoll, D. L., & Wells, J. (2012). Beyond knowledge and skills: Writing transfer and the role of student dispositions. Composition Forum, 26.

• Lobato, J. (2003). How design experiments can inform a rethinking of transfer and vice versa. Educational Researcher, 32(1), 17-20.

• National Research Council (1999). How people learn: brain, mind, experience, and school. Washington D.C.: National Academy Press.

• National Science Foundation. “Transfer of Learning: Issues and Research Agenda.” NSF Workshop Report, 2002. Web. http://www.nsf.gov/pubs/2003/nsf03212/nsf03212.pdf

• Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460-475.

• Schwartz, D. L., Bransford, J. D., and Sears, D. (2005). Efficiency and innovation in transfer.” Transfer of Learning from a Modern Multidisiplinary Perspective. Ed. Jose P. Mestre. Information Age Publishing, 1-51.

• Taczak, K. (2011). "Connecting the dotsaching about Writing, Righting Misconceptions: (Re)Envisioning "First-Year Composition" as "Introduction to : Does reflection foster transfer?” Ph.D., Florida State University, Tallahassee.

• Wardle, E. (2009). "Mutt genres" and the goal of FYC: Can we help students write the genres of the university? College Composition and Communication, 60(4), 765-789.