Upload
ciee
View
104
Download
1
Embed Size (px)
Citation preview
Measuring Linguistic Gains in Immersion Settings: Empirical Studies from French and
Spanish as a Second Language
Dennis M. Wiseman
Council for International Education Exchange Annual Conference
Breaking Barriers: Cost, Curriculum, Culture Baltimore, Md
November 19-22, 2014
Wofford College
• Four-year, private, liberals arts college • Founded in 1854 • Methodist affiliated • Phi Beta Kappa • # 4 in the Open Doors survey • 1600 students
Linguistic Gains: High Impact Practices
All French or Spanish students in these data have engaged in at least one of the following activities in study abroad. – Home stay – Language course work – Internship – Volunteer or service learning
Methodology • OPIc • Immediately upon return from study abroad,
set / proctored • French students – Paris, Nantes, Rennes, Bordeaux, Aix-en-
Provence, Dijon, Arles • Spanish students – Spain, Argentina, Panama, Chile, Dominican
Republic, Peru, Ecuador, Bolivia, Honduras, Nicaragua
LAT. AMER.
SPAIN
0 20 40 60 80 100 120 140
LAT. AMER. SPAIN Series1 118 77
SPANISH_STUDY_ABROAD_2007_'13
0 20 40 60 80 100 120 140
LAT. AMER.
LAT. AMER DNT
SPAIN
SPAIN DNT
LAT. AMER. LAT. AMER DNT SPAIN SPAIN DNT Series1 118 16 77 29
ACTFL_OPI_DID_NOT_PARTICIPATE_#
0 5 10 15 20 25 30 35 40 45 50
AH
AM
AL
IH
IM
IL
DNT
AH AM AL IH IM IL DNT Series1 1 6 31 44 18 2 16
SPANISH_LAT._AMER._ACTFL_2007_2013_#
0 5 10 15 20 25 30 35
AH
AM
AL
IH
IM
IL
DNT
AH AM AL IH IM IL DNT Series1 1 7 25 15 29
SPANISH_SPAIN_ACTFL_2007_2013_#
0 5 10 15 20 25 30 35 40
AH
AM
AL
IH
IM
IL
DNT
AH AM AL IH IM IL DNT Series1 0.8 5.17 26.27 37.28 15.25 1.6 13.56
SPANISH_LAT._AMER._ACTFL_2007_2013_%
0 5 10 15 20 25 30 35 40
AM
AL
IH
IM
IL
DNT
AM AL IH IM IL DNT Series1 1.3 9.1 32.47 19.48 37.66
SPANISH_SPAIN_ACTFL_2007_2013_%
French_ACTFL_Advanced_2007_2013
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
2007_2013
2009_2013
2011_2013
2012_2013
2013
Spanish_ACTFL_Advanced_2007_2013
0% 5% 10% 15% 20% 25% 30% 35%
2007_2013
2009_2013
2011_2013
2012_2013
2013
What is COPI telling us about Spanish
improvement in the study abroad context?
CIEE Conference
Baltimore, November 2014
Vera Cerqueiras, BA Spanish Coordinator
CONTENTS
• Describe the process of training, implementation and correction of the exam;
• Analyze the progress made by students between their entrance COPI and their exit COPI;
• Show some of the challenges and the subsequent effects (washback) resulting from this evaluation.
2
What is COPI (Computer based Oral Proficiency Interview)?
3
• The COPI exam is a computerized semi-adaptive exam aimed at determining the ability for using oral spanish language at a specific moment taking into account the ACTFL Guidelines.
• It is intended for high school
students, college students, and professionals whose native language is English or who have an advanced English level
• Standarized exam • Measuring linguistic production
holistically • Providing feedback to both the
test takers and their professors.
• Exam results can be used to show the progress of a student after a semester of study.
• It cannot be used to make
decisions to pass or fail Spanish courses
Characteristics
Evaluation Levels
4
1. Superior: The candidate gives opinions, hypothesizes, and uses abstract language in unfamiliar language situations 2. Advanced: The candidate narrates and describes situations with some level of complication 3. Intermediate: The candidate demonstrates creative use of the language in simple and everyday situations 4. Novice: The candidate uses fixed expressions, short phrases and lists of memorized words The three latter levels are subdivided into 3 sublevels: High: The candidate does not have the ability to reach and maintain the next level Medium: Represents the prototypical level Low: The candidate has difficulty in meeting the level requirements
Administration of the Exam
The exam has the following components:
• A CD for the administration of the COPI exam
• An application for the certification of the exam
• Lab with PC`s ,Computers, headset with microphone and Speaker, keyboards
• The candidates must select their Spanish level taking in account the summary description of each level given by the program
• The program selects from 7 to 11 tasks
• The candidates must complete the exam by recording their answers
• Teachers / Raters should be in the computer
lab to assist.
• The exam lasts between 45/60 minutes
5
Interview Evaluation and Grading and Final Report of the Evaluation
of the level reached by the candidate.
6
• The program saves the candidate’s activities.
• A double assessment is performed by two certified professors who correct all the activities completed by each candidate.
• The evaluator assigns a corresponding level for each answer he/she listens to, taking into account the level assigned by the student or the program
• The evaluator has the option to write comments about the activity, which the program then saves as part of the evaluation.
• The program establishes an overall average after the evaluator assigns a level for each of the activities performed by the candidates.
• A final report with the
certification of each candidate’s obtained level is generated by the program including the overall average and a description of the proficiency
Exam and Certification of COPI Evaluators • Evaluators are trained by using a kit provided by CAL
(Center of applied linguistics)
• They have to pass an exam according to ACTFL Guidelines
• This exam consists of 15 comprehensive COPI exams that should be rated
• Evaluators approve the exam if at least 60% (9/15) scores coincide with those indicated by the CAL experts.
7
Why COPI implementation represents a challenge?
12
• BECAUSE it implies working with isolated language samples that must be analyzed without consideration of neither its production context nor its acquisition process.
• BECAUSE the evaluator's work needs permanent training and peer feedback in order to adjust and perfect the administration, correction and considerations of the exam.
• The internalization of the practices incorporated by giving the COPI exam allow us to focus on the elaboration of evaluation instructions and on the importance that the "knowledge of the world" has towards the successful achievement of that instruction.
• The progress data between entry exams and exit exams tests the results obtained by the very same students in the evaluations of the Spanish courses they take as part of their immersion program studies. Until now they are consistent. In other words, those students who advance the most are the same students who achieve the highest grades in their Spanish courses.
Washback Effects
Effects of International
Service-Learning on the
Development of Spanish Oral
Proficiency
CIEE Conference: Breaking Barriers
November 19-22, 2014, Baltimore
Francisco Salgado-Robles, Ph.D.
The City University of New York
Overview of presentation
① Goal ② Community Service-Learning
a) General notions b) SL in higher education c) SL in the Spanish classroom d) SL at-home: Linguistic gains e) SL abroad: Linguistic gains
③ Research Questions ④ Instrument ⑤ Participants ⑥ Blogging & Service-Learning ⑦ Results ⑧ Discussion ⑨ Conclusions ⑩ Challenges & Recommendations
GOAL
¡ Gain further insight into the effects of context (i.e., service-learning in an international setting) on the L2 acquisition process (i.e., the development of Spanish oral proficiency) through a case-control study analysis of participants’ experiences enrolled in a Language for Specific Purposes (LSP) course (i.e., Spanish for the working environment) during a summer semester abroad.
3
¡ SL? “Experiential education that engages students in activities that address human and community needs together with structured opportunities intentionally designed to promote student learning and development” (Jacoby, 1996)
¡ Students’ service experiences are tied to the academic content of an on-campus course or curriculum (Giles, Honnet, & Migliore, 1991)
¡ From the tenets of experiential education and constructivist theories that advocate for learning through first-hand discovery (Furco, 2001)
¡ “One of the most effective approaches for engaging students and boosting learning” (Learn and Serve America, 2011)
¡ Shape learners’ knowledge and cultural beliefs, professional aspirations, and community engagement.
SERVICE-LEARNING (SL): GENERAL NOTIONS
4
¡ Based on statistics from 2009-2010, ½ + of community colleges and approximately ¼ of all universities integrate SL in different academic curricula (Learn and Serve America, 2011)
¡ SL pedagogy has been adopted not only due to
the overwhelming amount of research in support of this type of educational experience, but also because it has been witnessed how life-changing these opportunities can be for their students (Jeandron & Robinson, 2010)
SL IN HIGHER EDUCATION
5
¡ The incorporation of SL in Spanish curricula follows similar trends observed in other disciplines in higher education
¡ Community engagement has been a “growing
presence in Spanish classes” and that there is presently “significant activity and interest within our ranks (Hellebrandt & Jorge, 2013)
¡ Rapidly expanding Hispanic population in the United
States (Rabin, 2009; Zlotkowski, 1999) ¡ Convincing evidence of the positive impact SL can
have on students taking Spanish as a second or heritage language (Petrov, 2013)
SL IN THE SPANISH CLASSROOM
6
¡ Offer students authentic communicative contexts in the target language
¡ Communicative competence improved from regular interaction with NSs after participating in SL (Barreneche, 2011; Navarro, 2012; Olazagasti-Segovia, 2004; Salgado-Robles, 2014)
¡ Facilitate linguistic improvements in the L2 (Hale, 1999; Mullaney, 1999; Over field, 1997)
¡ Spanish SL students scored higher on L2 reading, speaking, and writing tasks (Malkin, 2010)
¡ Students increased their use of Spanish outside the classroom and felt more confident speaking in Spanish (Pellettieri, 2011)
SL AT-HOME: L2 GAINS
7
¡ SL as an effective way enhance student’ language skil ls, knowledge of the host country, intercultural sensitivity, and global competence (Kiely, 2011)
¡ Students gained confidence in their Spanish skil ls,
developed intercultural skil ls (Kopischke Smith & Moreno-López, 2012)
¡ Facilitate linguistic improvements in the L2 (Cubillos,
2013) ¡ Students perceived a positive relationship between
their participation in SL projects and the development of their l inguistic and cultural proficiency (Brown & Purmensky, 2014)
8
SL ABROAD: L2 GAINS
RESEARCH QUESTIONS
① What is the effect of a community-enhanced Spanish for Specific Purpose course on the oral proficiency at the Intermediate-level?
② What aspects of oral proficiency are most and least influenced by the community engagement experience?
9
PARTICIPANTS (CONT’D)
Experimental group
22 students (14 f / 8 m)
19-21 yrs old
3rd year Spanish
Spanish minors/majors
Spanish for the Professions
Control group
18 students (11 f / 7 m)*
19-23 yrs old
3rd year Spanish
Spanish minors/majors
Spanish for the Professions
* 1 heritage speaker removed 11
RESULTS
*** High Statistical Significance
ü Paired t-tests run to determine significance of trends over time
Pre Mean
Post Mean
p-value
Mastery sentence 43.10 50.00 0.0003*** Experimental Vocabulary 36.77 51.22 <0.0001***
Group Fluency 45.52 54.19 <0.0001***
(+SL) Pronunciation 46.18 57.30 0.0003***
Mastery sentence 43.94 46.04 0.0002***
Control Vocabulary 39.75 52.76 <0.0001***
Group Fluency 47.00 49.15 0.1320
(-SL) Pronunciation 44.60 48.63 0.0302
18
DISCUSSION
① What is the effect of a community-enhanced Spanish for Specific Purpose course on the oral proficiency at the Intermediate-level? ü A statistically significant impact on the proficiency level of
Intermediate-level learners. ü Just one participant experienced L2 gains towards the
advanced-low proficiency level in this experiment.
② What aspects of oral proficiency are most and least influenced by the community engagement experience? ü SL had an observable and statistically significant impact on all
sub-skills of oral proficiency measured by the Versant Test.
19
CONCLUSIONS
① The incorporation of the SL component into a Spanish for Specific Purposes course in an international setting had a significant effect on oral proficiency;
② The learning conditions provided by the community
engagement facilitated oral skil l development across all sub-skil ls;
③ Overall, the SL pedagogy worked successfully for this LSP course and, thus, should be further promoted, since:
ü Provide more opportunities for L2 use in meaningful ways; ü Practice different L2 skills: speaking, reading, writing, listening, cultural understanding, etc.; ü Debrief and reflect on SL experience with others; ü Introduction to the professional field in a hands-on way; ü Give more confidence to students when they are in their services.
20
q Challenges: § Time and effort: More so than the ‘normal’ course; § Technological problems and conquering the learning curve; § Lots of smaller tasks (visit report via Google Docs; blog entry
and comments; in-class led discussions); some students had a hard time keeping up with due dates.
q Recommendations:
§ Cooperative efforts among researchers, students, instructors, administrators, and community supervisors are plainly needed;
§ The development and dissemination of SL in LSP pedagogic materials are warranted;
§ More large-scale studies and longitudinal assessment of L2 acquisition (both qualitative and quantitative) within the SL (and LSP) contexts would help to further understand a new strand of experiential learning (Kolb, 1984).
CHALLENGES & RECOMMENDATIONS
21
ACKNOWLEDGEMENTS
² Department of Hispanic Studies, U of Kentucky
² Service-Learning Council, U of Kentucky
² Education Abroad Office, U of Kentucky
² Spanish Studies Abroad, Seville, Spain
² Non-profit organizations
² 40 students