University of Minnesota Proficiency Project · The participants in this project are university students taking foreign language courses at the University of Minnesota during the spring

University of Minnesota Proficiency Project

Spring 2015 Evaluation Report

Center for Applied Linguistics

Margaret E. Malone, Ph.D.

Anne Donovan, M.S.

John Chi, M.S.

18 DECEMBER 2015

Table of Contents

INTRODUCTION ........................................................................................................................................ 1

Research Questions ................................................................................................................................... 1

METHODOLOGY ....................................................................................................................................... 2

Participants ................................................................................................................................................ 2

Procedures ................................................................................................................................................. 4

Instruments ................................................................................................................................................ 4

Data Collection ......................................................................................................................................... 4

Data Analysis ............................................................................................................................................ 5

FINDINGS .................................................................................................................................................... 6

RQ 1: What levels of proficiency do students’ test performances demonstrate at which course levels in

which languages? ...................................................................................................................................... 6

RQ 2: Do students in higher level courses demonstrate higher levels of proficiency than students in

lower levels? ........................................................................................................................................... 29

RQ 3a: Is there a relationship between the students’ proficiency ratings and their motivating factors? 36

RQ3b: Is there a relationship between the students’ proficiency ratings and their language practice/use?

................................................................................................................................................................ 38

RQ3c: Is there a relationship between the students’ proficiency ratings and the number of courses taken

at the University of Minnesota? .............................................................................................................. 41

RQ4: Are there correlations between self-assessments and actual ratings?............................................ 43

RQ 5: What factors contribute to advanced proficiency in study abroad experiences? .......................... 49

CONCLUSIONS......................................................................................................................................... 57

REFERENCES ........................................................................................................................................... 59

APPENDIX ................................................................................................................................................. 60

Table of Figures

Figure 1. Overall means for Arabic courses ................................................................................................. 6 Figure 2. Counts of proficiency levels in ARAB 1102 ................................................................................. 7 Figure 3. Counts of proficiency levels in ARAB 3102 ................................................................................. 8 Figure 4. Counts of proficiency levels in ARAB 5102 ................................................................................. 9 Figure 5. Overall means for French courses ............................................................................................... 10 Figure 6. Counts of proficiency levels in FREN Study Abroad courses..................................................... 11 Figure 7. Counts of proficiency levels in FREN Advanced courses ........................................................... 12 Figure 8. Counts of proficiency levels in FREN Senior/Capstone courses ................................................ 13 Figure 9. Counts of proficiency levels in FREN Self-Select courses ......................................................... 14 Figure 10. Overall means for German courses............................................................................................ 15 Figure 11. Counts of proficiency levels in GER Self-Select courses ......................................................... 16 Figure 12. Overall means for Korean courses ............................................................................................. 17 Figure 13. Counts of proficiency levels in KOR 3022................................................................................ 18 Figure 14. Counts of proficiency levels in KOR 3032................................................................................ 18 Figure 15. Counts of proficiency levels in KOR 4042................................................................................ 19 Figure 16. Overall means for Portuguese courses ....................................................................................... 20 Figure 17. Counts of proficiency levels in PORT 1104 .............................................................................. 21 Figure 18. Counts of proficiency levels in PORT 3502W .......................................................................... 21 Figure 19. Overall means for Russian courses ............................................................................................ 22 Figure 20. Counts of proficiency levels in RUSS 3002 .............................................................................. 23 Figure 21. Counts of proficiency levels in RUSS 3102 .............................................................................. 24 Figure 22. Overall means for Spanish courses ............................................................................................ 25 Figure 23. Counts of proficiency levels in SPAN Study Abroad courses .................................................. 26 Figure 24. Counts of proficiency levels in SPAN 3972W .......................................................................... 27 Figure 25. Counts of proficiency levels in SPAN Certificate courses ........................................................ 27 Figure 26. Counts of proficiency levels in SPAN Self-Select courses ....................................................... 28 Figure 27. LTP proficiency counts by Arabic course levels ....................................................................... 29 Figure 28. RTP proficiency counts by Arabic course levels ....................................................................... 30 Figure 29. OPIc proficiency counts by Arabic course levels ...................................................................... 30 Figure 30. OPIc proficiency counts by Korean course levels ..................................................................... 31 Figure 31. LTP proficiency counts by Portuguese course levels ................................................................ 32 Figure 32. RTP proficiency counts by Portuguese course levels ................................................................ 32 Figure 33. OPIc proficiency counts by Portuguese course levels ............................................................... 33 Figure 34. LTP proficiency counts by Russian course levels ..................................................................... 34 Figure 35. RTP proficiency counts by Russian course levels ..................................................................... 34 Figure 36. OPIc proficiency counts by Russian course levels ................................................................... 35

Table of Tables

Table 1. Course codes and course titles for Spring 2015 .............................................................................. 2 Table 2. Number of participants by language and PACE level .................................................................... 3 Table 3. PACE codes for course levels ......................................................................................................... 3 Table 4 Consolidation of course codes for undergraduate and graduate students ........................................ 4 Table 5. ACTFL sublevel conversion scale .................................................................................................. 5 Table 6. N-sizes and unanalyzable data for Arabic courses .......................................................................... 7 Table 7. Consolidation of French courses ..................................................................................................... 9 Table 8. N-sizes and unanalyzable data for French courses ....................................................................... 10 Table 9. N-sizes and unanalyzable data for German courses ...................................................................... 15 Table 10. N-sizes and unanalyzable data for Korean courses ..................................................................... 17 Table 11. N-sizes and unanalyzable data for Portuguese courses ............................................................... 20 Table 12. N-sizes and unanalyzable data for Russian courses .................................................................... 22 Table 13. Consolidation of Spanish courses ............................................................................................... 25 Table 14. N-sizes and unanalyzable data for Spanish courses .................................................................... 26 Table 15. Top motivating factors selected for learning the target language ............................................... 36 Table 16. Proficiency levels by top motivating factors............................................................................... 36 Table 17. Proficiency levels by top choice of motivation and course level ................................................ 37 Table 18. Rates of language practices outside class by course level .......................................................... 38 Table 19. Proficiency levels by top contexts for language use by course level .......................................... 39 Table 20. LTP ratings by amount of language use and course level ........................................................... 39 Table 21. RTP ratings by amount of language use and course level .......................................................... 40 Table 22. OPIc ratings by amount of language use and course level.......................................................... 40 Table 23. Average LTP scores by course level and previous UMN courses .............................................. 41 Table 24. Average RTP scores by course level and previous UMN courses .............................................. 41 Table 25. Average OPIc scores by course level and previous UMN courses ............................................. 42 Table 26. Self-assessment ranges by level .................................................................................................. 43 Table 27. Level 1 listening self-assessments and LTP ratings .................................................................... 43 Table 28. Level 3 listening self-assessments and LTP ratings .................................................................... 44 Table 29. Level 1 reading self-assessments and RTP ratings ..................................................................... 45 Table 30. Level 3 reading self-assessments and RTP ratings ..................................................................... 45 Table 31. Level 1 speaking self-assessment and OPIc ratings .................................................................... 46 Table 32. Level 2 speaking self-assessments and OPIc ratings .................................................................. 46 Table 33. Level 3 speaking self-assessments and OPIc ratings .................................................................. 47 Table 34. Summary of accuracy in self-assessments .................................................................................. 47 Table 35. Summary of accuracy in self-assessments by self-assessment level .......................................... 48 Table 36. Duration of study abroad by course level ................................................................................... 49 Table 37. Use of target language during instruction while studying abroad by course level ..................... 49 Table 38. Living situation while studying abroad by course level ............................................................. 50 Table 39. Frequency of target language use at home by living situation .................................................... 50 Table 40. Frequency of target language use outside of the home ............................................................... 50 Table 41. Duration of study abroad ............................................................................................................ 51 Table 42. Target language use during instruction ....................................................................................... 51 Table 43. Living situation while studying abroad ....................................................................................... 51 Table 44. Frequency of target language use at home by living situation .................................................... 52 Table 45. Frequency of target language use outside of the home ............................................................... 52 Table 46. Duration of study abroad ............................................................................................................ 53 Table 47. Target language use during instruction ....................................................................................... 53 Table 48. Living situation while studying abroad ....................................................................................... 53

Table 49. Frequency of target language use at home by living situation .................................................... 53 Table 50. Frequency of target language use outside of the home ............................................................... 54 Table 51. Duration of study abroad ............................................................................................................ 54 Table 52. Target language use during instruction ....................................................................................... 55 Table 53. Living situation while studying abroad ....................................................................................... 55 Table 54. Frequency of target language use at home by living situation .................................................... 55 Table 55. Frequency of target language use outside of the home ............................................................... 56 Table 56. Arabic LTP scores by course level and previous UMN courses ................................................ 60 Table 57. Arabic RTP scores by course level and previous UMN courses ................................................ 60 Table 58. Arabic OPIc scores by course level and previous UMN courses................................................ 60 Table 59. French LTP scores by course level and previous UMN courses ................................................ 61 Table 60. French RTP scores by course level and previous UMN courses ................................................ 61 Table 61. French OPIc scores by course level and previous UMN courses ............................................... 61 Table 62. German LTP scores by course level and previous UMN courses ............................................... 62 Table 63. German RTP scores by course level and previous UMN courses .............................................. 62 Table 64. German OPIc scores by course level and previous UMN courses .............................................. 62 Table 65. Korean OPIc scores by course level and previous UMN courses ............................................... 63 Table 66. Portuguese LTP scores by course level and previous UMN courses .......................................... 64 Table 67. Portuguese RTP scores by course level and previous UMN courses .......................................... 64 Table 68. Portuguese OPIc scores by course level and previous UMN courses ......................................... 64 Table 69. Russian LTP scores by course level and previous UMN courses ............................................... 65 Table 70. Russian RTP scores by course level and previous UMN courses ............................................... 65 Table 71. Russian OPIc scores by course level and previous UMN courses .............................................. 65 Table 72. Spanish LTP scores by course level and previous UMN courses ............................................... 66 Table 73. Spanish RTP scores by course level and previous UMN courses ............................................... 66 Table 74. Spanish OPIc scores by course level and previous UMN courses .............................................. 66

1 | P a g e

University of Minnesota

Language Proficiency Evaluation Report

Spring Semester 2015

INTRODUCTION

In 2014, the National Security Education Program (NSEP)’s Language Flagship awarded

contracts to three major state universities (Michigan State University, the University of Minnesota, and

the University of Utah) to investigate language proficiency outcomes of university students in both

commonly taught languages (CTL) and less commonly taught languages (LCTL). NSEP in general and

the Language Flagship in particular have as their mission to “graduate students who will take their place

among the next generation of global professionals, commanding a superior level of fluency in one of

many languages critical to U.S. competitiveness and security” (NSEP). At the University of Minnesota,

the project is known as the Proficiency Assessment for Curricular Enhancement Project (PACE).

In Fall 2014, the first semester of data collection for this project, data were collected from 306

students of French, German, Portuguese, and Spanish at the University of Minnesota. In addition to

administering language proficiency tests to students, the University of Minnesota collected demographic,

experiential, attitudinal and self-assessment data. The Center for Applied Linguistics (CAL) completed a

report on the research questions provided by the university based on the data provided by the University

of Minnesota.

In Spring 2015, the second semester of data collection, data were collected from 363 students of

Arabic, French, German, Korean, Portuguese, Russian, and Spanish at the University of Minnesota. The

same data collection methodology was followed in Spring 2015 and Fall 2014.

In this report, the Center for Applied Linguistics provides an evaluation of the Spring 2015

students’ data and attempts to answer the following research questions provided by the University of

Minnesota.

Research Questions

The research questions addressed in this report are:

1. What levels of proficiency do students’ test performances demonstrate at which course levels in

which languages?

2. Do students in higher level courses demonstrate higher levels of proficiency than students in

lower levels?

3. Is there a relationship between the students’ proficiency ratings and:

a. Their motivating factors

b. Their language practice/use?

c. The number of courses taken at the University of Minnesota? 4. Are there correlations between self-assessments and actual ratings?

5. What factors contribute to advanced proficiency in study abroad experiences?

2 | P a g e




METHODOLOGY

This section presents the methodology utilized for data collection at the University of Minnesota,

including details about the participant population, the procedures of research, the instruments employed,

the data collection, and analysis of the data.

Participants

The participants in this project are university students taking foreign language courses at the

University of Minnesota during the spring semester of 2015. Students were drawn from several course

levels across seven languages. Table 1 labels the course codes with course titles.

Table 1. Course codes and course titles for Spring 2015

Course Code Course Title

ARAB 1102 Beginning Arabic II

ARAB 3102 Intermediate Arabic II

ARAB 5102 Advanced Arabic II

FREN 3017W Advanced Writing in French: Genre, Style, Rhetoric

FREN 3101W Methods in French and Francophone Studies

FREN 3541 Oral Discourse of French

FREN 4109W Senior Project in French and Francophone Studies

FREN 4110V Honors Thesis

FREN Self-Select

FREN Study Abroad

GER Self-Select

KOR 3022 Intermediate Korean

KOR 3032 Third Year Korean

KOR 4042 Advanced Readings in Modern Korean

PORT 1104 Intermediate Portuguese

PORT 3502W Global Portuguese II

RUSS 3002 Intermediate Russian II

RUSS 3102 Advanced Russian II

SPAN 3972W Graduation Seminar

SPAN Certificate

SPAN Self-Select

SPAN Study Abroad

Participating students were enrolled at a variety of course levels. In order to better understand the

relationship between proficiency levels and course levels across languages, students’ data were coded

based on the type of course level. If a student was enrolled in multiple courses during the target semester,

particularly likely for senior or upper-level students, their data were included in the highest level course.

Table 2 shows the breakdown of the 363 participants by language and course level.

3 | P a g e




Table 2. Number of participants by language and PACE level

Course Arabic French German Korean Portuguese Russian Spanish Total

PACE 1002/1022 37 -- -- -- -- -- -- 37

PACE 1004 42 -- -- 47 6 35 -- 130

PACE 3XXX 26 45 9 19 9 8 66 182

PACE Capstone/

Certificate -- 7 -- -- -- -- 7 14

Total 105 52 9 66 15 43 73 363

As Table 2 shows, a total of 363 students participated in data collection in Arabic (N=105),

French (N=52), German (N=9), Korean (N=66), Portuguese (N=15), Russian (N=43), and Spanish (N=73).

The largest number of students were enrolled in fourth semester (1004) levels and the sixth semester

(3XXX) levels. The smallest group is the students who are capstone or certificate candidates and study

abroad participants (N=14 and N=12, respectively).

Table 3 displays the PACE codes for each course level for every language.

Table 3. PACE codes for course levels

PACE

1002/1022

PACE

1004

PACE

3XXX

PACE

Capstone/

Certificate

ARAB 1102 x

ARAB 3102 x

ARAB 5102 x

FREN 3017W x

FREN 3101W x

FREN Self-Select x

FREN 3541 x

FREN 4109W x

FREN 4110V x

FREN Study Abroad x

GER Self-Select x

KOR 3022 x

KOR 3032 x

KOR 4042 x

PORT 1104 x

PORT 3502W x

RUSS 3002 x

RUSS 3102 x

SPAN 3972W x

SPAN Certificate x

SPAN Self-Select x

SPAN Study Abroad x

Table 3 shows that for Spring 2015 semester, most course levels were in the PACE 3XXX tier.

Four course levels are in PACE 1004, four course levels in PACE Capstone/Certificate, and only one

course level in PACE 1002/1022.

4 | P a g e




The Spring 2015 course levels did not all follow a linear progression. Participants studying

Spanish, French, and German were enrolled in higher level courses that have similar prerequisites, but

need not be taken in a specific sequence. For Arabic, Korean, Portuguese, and Russian, there is a clear-cut

progression between the course levels (e.g., beginner, intermediate, advanced). Some languages have

parallel courses for graduate and undergraduate students with different course numbers, but with the same

content. For example, both ARAB 1102 and ARAB 4102 are Beginning Arabic II courses; the only

difference being the first is for undergraduates and the latter is for graduate students. Because of the

limited number of graduate students in these course levels, the graduate level courses were consolidated

into the course code for the undergraduate students in the same class. Table 4 shows a complete list of the

consolidation of course codes for course levels that have both undergraduate and graduate students in the

same course.

Table 4 Consolidation of course codes for undergraduate and graduate students

Language Course Title Course Codes Consolidation

Arabic Beginning Arabic II ARAB 1102 (U), ARAB 4102 (G) ARAB 1102

Korean Intermediate Korean KOR 3022 (U), KOR 4004 (G) KOR 3022

Portuguese Intermediate Portuguese PORT 1104 (U), PORT 4104 (G) PORT 1104

Russian Intermediate Russian II RUSS 3002 (U), RUSS 4104 (G) RUSS 3002

Advanced Russian II RUSS 3102 (U), RUSS 4112 (G) RUSS 3102

Key: U = undergraduate students; G = graduate students

Table 4 demonstrates that all of the courses that are equivalent in progression and that meet

together were combined into one common course code in order to more easily analyze the proficiency of

students who are taking courses at the same level. In the findings, the course codes under the

‘Consolidation’ column will be used to account for both the undergraduate and graduate students in their

respective courses.

Procedures

The University of Minnesota used several instruments (details in the next subsection) to collect

data from the 363 students across 7 languages, then sent all data to CAL in an Excel spreadsheet for

analysis. Upon receiving this data, CAL began organizing and analyzing the data to respond to the five

previously stated research questions provided.

Instruments

In order to answer the research questions, researchers at the University of Minnesota used a total

of seven instruments. In order to assess language proficiency across modalities, PACE utilized three

instruments developed by the American Council on the Teaching of Foreign Languages (ACTFL): the

ACTFL Listening Test of Proficiency (LTP), the ACTFL Reading Test of Proficiency (RTP), and the

Oral Proficiency Interview-computerized (OPIc). In addition to the tests, students completed self-

assessments of their abilities in listening, reading, speaking, and writing as well as a language experience

background and motivation questionnaire created by the university. All material selection/design and

implementation was conducted by the University of Minnesota.

Data Collection

The data collection process, including research design, identifying participants, materials

creation, testing participants, and completing surveys, was conducted by the University of Minnesota.

5 | P a g e




After the data collection process was complete, CAL received the data from the University of Minnesota

for analysis and evaluation.

Data Analysis

To analyze student outcomes, ratings on the ACTFL Proficiency Guidelines (ACTFL, 2012) were

converted to numerical values using conversions that have been previously used in many research studies

(see e.g. Dandonoli & Henning, 1990; Vande Berg, Connor-Linton, & Paige, 2009). Table 5 shows the

conversion scale.

Table 5. ACTFL sublevel conversion scale

ACTFL Sublevel Numeric Conversion

Novice Low 0.1

Novice Mid 0.3

Novice High 0.8

Intermediate Low 1.1

Intermediate Mid 1.3

Intermediate High 1.8

Advanced Low 2.1

Advanced Mid 2.3

Advanced High 2.8

Superior 3.0

For reporting purposes, means will be reported numerically and/or in terms of the closest ACTFL

sublevel. To assign participant and group means on test performances, the numerical ratings will be

rounded to the closest sublevel. For example, a mean of 1.15 would be reported as Intermediate Low,

whereas a mean of 1.22 would be reported as Intermediate Mid.

Some students’ test performances result in unanalyzable data. The LTP and RTP are leveled tests,

and students took a test consistent with the level of their expected performance. For example, students

enrolled in a 1002 course were given an LTP for the Novice High to Advanced Low levels. However, if a

student’s actual proficiency falls below that range, the test is scored “BR”, or “Below Range”. Similarly,

on the OPIc, students whose responses were not consistent or did not fit the scoring profile of the test

could have a test that is unratable (UR). Such tests were regarded as unanalyzable, and thus are not

included in the group means. In Fall 2014, the data contained a large amount of unanalyzable tests, but

the Spring 2015 data includedfar fewer unanalyzable tests.

Students also completed a self-assessment that consisted of a series of “can do” statements on

which students gave themselves a rating to indicate the extent to which they could do each described

statement, for example, “When I hear the language that I study and the topic is generally familiar to me, I

can understand the gist of what I hear.” Student responses to each of the statements were converted to a

numeric total score for each skill, and converted to an ACTFL level using a conversion chart provided by

the University of Minnesota. Like the LTP and RTP, the self-assessments are leveled, so students could

receive a limited range of sublevels consistent with course enrollment.

6 | P a g e




FINDINGS

RQ 1: What levels of proficiency do students’ test performances demonstrate at which course

levels in which languages?

Participants’ proficiency ratings (means) were calculated for each language, skill test (i.e., LTP,

RTP, OPIc), and course level or grouping in order to address this first research question. Using the means

of the groups, this report addresses: 1) patterns for a specific language, 2) patterns for specific skill tests,

and 3) patterns for course levels or groups in each language. Divided into each of the seven languages, the

overall proficiency scores of each language course in said language are reported along with any

unanalyzable data, followed by a narrower view at each individual course level/group with the number of

participants at each proficiency level.

I. Arabic

A total of three Arabic course levels were included in the analysis with 105 participants. ARAB

1102 is a Beginning Arabic II course level that includes of both undergraduates (ARAB 1102) and two

graduate students (ARAB 4102). The next progression is ARAB 3102, which includes the Intermediate

Arabic II courses, followed by ARAB 5102, the Advanced Arabic courses. Figure 1 displays the average

proficiency levels for performance of students of Arabic at each course level for each skill.

Figure 1. Overall means for Arabic courses

As Figure 1 shows, as the courses progress from ARAB 1102 to ARAB 3102, the listening scores

steadily increase: +0.10 from Beginner to Intermediate Arabic (NH), and +0.27 from intermediate to

advanced Arabic (from NH to IL). Reading scores also demonstrated a somewhat steady increase between

the beginning and intermediate Arabic course levels: +0.06 from ARAB 1102 to ARAB 3102 (from NM

to NH). Between the intermediate and advanced courses, the increase was larger: +0.37 (NH to IL). For

oral proficiency, the figure shows a decrease from the beginning to intermediate Arabic courses: -0.18

from ARAB 1102 to ARAB 3102 (from IL to NH). However, the next progression of courses (ARAB

3102 to ARAB 5102) shows an increase of +0.34 (from NH to IM).

7 | P a g e




Table 6 shows the total sample size for each Arabic course level and the number of participants

whose test performances were not analyzable at each level.

Table 6. N-sizes and unanalyzable data for Arabic courses

N-

size

LTP RTP OPIc

Course BR UR NR Total BR UR NR Total BR UR NR Total

ARAB

1102

37 0 0 0 0 0 0 0 0 0 1 0 1

ARAB

3102

62 0 0 1 1 0 0 1 1 0 0 0 0

ARAB

5102

6 0 0 0 0 0 0 0 0 0 0 0 0

Total 105 0 0 1 1 0 0 1 1 0 1 0 1

When reviewing the data, it is important to consider a few factors. First, note that there is unequal

distribution between total participants in the three course levels. As Table 6 shows, the number of

participants varies greatly between each of the courses, with large numbers in ARAB 1102 (N=37) and

ARAB 3102 (N=62), and only 6 in ARAB 5102. For the Arabic courses, there were only a few

unanalyzable data points: one UR for ARAB 1102 (in OPIc), and two NRs for ARAB 3102 (one in LTP

and one in RTP).

A. ARAB 1102: Beginning Arabic II

Figure 2 shows the distribution of proficiency ratings for students (N=37) in ARAB 1102.

Figure 2. Counts of proficiency levels in ARAB 1102

Figure 2, which displays the test results for all three skills together, shows that all the

participants’ ratings range from NL to AL. The most frequent proficiency rating is NH (29.1%), followed

by NM (24.5%), IL (20.0%), and IM (14.5%). The remainder of the scores fall under NL (5.5%), IH

(4.5%), and AL (1.8%). It is also notable that the oral proficiency ratings were generally higher than

listening and reading.

8 | P a g e




All participants in ARAB 1102 received analyzable scores with the exception of one UR in the

OPIc.

B. ARAB 3102: Intermediate Arabic

Figure 3 shows the distribution of proficiency ratings for the students (N=62) in ARAB 3102.


Figure 3 shows that 174 out of 184 proficiency ratings (94.6%) fall between NL and IM. The

majority of ratings are NH (29.3%), followed by IL (24.5%), NM (20.7%), IM (11.4%), and NL (8.7%).

The remainder fall between IH and AH with only 1-4 ratings (0.1-2.2%) in each of those proficiency

levels. In this course level, the skills are quite evenly distributed with only a few outliers in the higher

ranges (IH to AL), most of which are in speaking skills.

In ARAB 3102, only 2 out of 186 proficiency ratings had unanalyzable scores, and thus

approximately 98.9% of the data (N=61 in LTP and RTP; and N=62 in OPIc) is represented in Figure 3.

C. ARAB 5102: Advanced Arabic

Figure 4 displays the distribution of proficiency ratings (N=6) in ARAB 5102.

9 | P a g e





As Figure 4 shows, all of the ratings for ARAB 5102 fall between NH and IH, centering on IL

(55.6%), followed by IM (27.8%), NH (11.1%), and lastly, IH (5.6%). Proportionally, the ratings are

higher in oral proficiency than listening, and higher in listening than reading. However, there are only six

participants in this course level, so these findings must be read with caution.

All of the ratings in ARAB 5102 were analyzable (N=6 for all three skills).

II. French

Students of French were enrolled in eight different courses. These courses do not all follow a

linear progression, and the number of courses is quite large. Therefore, based on communication with the

PACE staff at the University of Minnesota, courses were consolidated into the following categories:

FREN Study Abroad, FREN Advanced courses, FREN Senior/Capstone courses, and FREN Self-Select

courses. The specific course levels that were regrouped are found in Table 7. The few participants (N=3)

enrolled in two courses were either in FREN Advanced courses or FREN Senior/Capstone courses. For

analysis, the participants that were enrolled in both a FREN Advance course and a FREN

Senior/Capstone course were grouped into the higher level course level, which in this case would be

FREN Senior/Capstone.

Table 7. Consolidation of French courses

FREN Study Abroad FREN Adv Courses FREN Sr/Capstone FREN Self-Select

FREN Study

Abroad:

Montpellier

FREN 3017W

FREN 3101W

FREN 3541

FREN 4109W

FREN 4110V

FREN Self-Select

FREN Self-Select:

DirecTrack

Figure 5 displays the overall average proficiency scores of the LTP, RTP, and OPIc for the

French students in the four aforementioned course groups.

10 | P a g e




Figure 5. Overall means for French courses

Because there is no obvious progression of course levels in the French data, this section will

describe participants’ performance at each course grouping, rather than displaying increases or decreases

between the groups. The FREN Study Abroad group performed consistently, on average, at IH in

listening, AL in reading, and IH/AL in speaking. The FREN Advanced courses group received an average

of IH in listening, AL in reading, and IH in speaking. The FREN Senior/Capstone courses group received

an average of IH in listening, AL in reading, and AL in speaking. Lastly, the FREN Self-Select group

received an average similar to the FREN Senior/Capstone courses (IH in listening, AL in reading, AL in

OPIc) but with lower means.

Table 8 displays the total sample size for each French course group and the number of

participants with unanalyzable data.

Table 8. N-sizes and unanalyzable data for French courses

N-

size

LTP RTP OPIc


FREN

SA

2 0 0 0 0 0 0 0 0 0 0 0 0

FREN

Adv

Course

29 3 0 0 3 0 0 0 0 0 0 1 1

FREN

Senior/

Capstone

7 0 0 0 0 0 0 0 0 0 0 0 0

FREN SS 14 0 0 2 2 0 0 0 0 0 0 2 2

Total 52 3 0 2 5 0 0 0 0 0 0 3 3

SA = Study Abroad; SS = Self-Select

As Table 8 shows, the distribution of participants in each course group are again quite uneven:

N=2 in FREN Study Abroad courses, N=29 in FREN Advanced courses, N=7 in FREN Senior/Honors

courses, N=14 in FREN Self-Select courses. In addition to the uneven cell sizes, there is are also some

unanalyzable data. In the LTP, five participants had unanalyzable test data (3 BRs in FREN Advanced

11 | P a g e




courses and 2 NRs in FREN Self-Select courses). In the OPIc, there were three participants without test

ratings (1 NR in FREN Advanced courses and 2 NRs in FREN Self-Select courses). All RTP results were

included in the analysis.

A. FREN Study Abroad: Montpellier

Figure 6 shows the distribution of proficiency levels in the FREN Study Abroad courses.

Figure 6. Counts of proficiency levels in FREN Study Abroad courses

As Figure 6 displays, the ratings in the FREN Study Abroad courses are between IH and AL with

one rating at IL in listening. For these two students, their listening scores vary greatly – one IL and the

other AL, both of their reading scores are AL, and their speaking ratings are at IH and AL.

All of the data for the FREN Study Abroad courses were analyzable.

B. FREN Advanced courses: FREN 3017W (Advanced Writing in French), FREN 3101W

(Methods in French and Francophone Studies), FREN 3541 (Oral Discourse in French)

Figure 7 displays the distribution of proficiency levels of the participants (N=29) in the FREN

Advanced courses, which is inclusive of FREN 3017W, FREN 3101W, and FREN 3541.

12 | P a g e




Figure 7. Counts of proficiency levels in FREN Advanced courses

Figure 7 shows that all of the participant ratings in the FREN Advanced courses fall between

IM and AM, with most at AL (51.8%), followed by IH (26.5%), AM (13.3%), and IM (8.4%). The ranges

and distributions of ratings for each skill are similar. There are slightly more ratings, proportionally, in

AL and AM for the RTP than the other skills, but no one skill shows stronger ratings than the others.

As shown in Table 8, in the FREN Advanced courses, there are a few unanalyzable data points.

There were 3 BRs in the LTP, 1 NR in the OPIc. None of the RTP data were unanalyzable.

C. FREN Senior/Honors courses: FREN 4109W (Senior Project in French and Francophone

Studies), FREN 4110V (Honors Thesis)

The distribution of proficiency levels for the students enrolled in the FREN Senior/Honors

courses (N=7), which includes the FREN 4109W and FREN 4110V course levels, can be seen in Figure

8.

13 | P a g e




Figure 8. Counts of proficiency levels in FREN Senior/Capstone courses

Figure 8 indicates that all of the performances on the three tests were between IM and AH. 76.2%

of the participants’ scores fall between IH and AM, with the most in AL (28.6%) and AM (28.6%),

followed by IH (19.0%). The remainder fell under IM (14.3%) and AH (9.5%). Similar to the FREN

Advanced courses group, the skills are quite evenly distributed.

All data in the FREN Senior/Honors courses were included in the analysis.

D. FREN Self-Select: FREN Self-Select, FREN Self-Select: DirecTrack

Figure 9 displays the distribution of proficiency ratings for the participants in the FREN Self-

Select courses group.

14 | P a g e




Figure 9. Counts of proficiency levels in FREN Self-Select courses

As Figure 9 shows, the majority of the participants’ scores (92.1%) are between IM and AL, with

the highest number of participants’ scores in AL (55.3%), the second highest in IH (23.7%) and the third

in IM (13.2%). Only a single participant rating was in IL, AM, and S (2.6% in each). Overall, the

listening scores are the lowest out of the three skills. The reading scores mostly fall under AL with only 2

ratings in IH. The OPIc ratings cover a wider range, as there is one outlier in S as well as IM and AM, but

the majority of the speaking ratings are between IH and AL.

There were two participants missing LTP scores and two participants missing OPIc scores. The

remaining data were all analyzable.

III. German

For German, there were a total of two courses with nine participants in two different Self-Select

courses. Figure 10 displays the overall means for the performance of students in German courses: GER

Self-Select. No comparisons can be made to other course levels.

15 | P a g e




Figure 10. Overall means for German courses

As Figure 10 indicates, the GER Self-Select participants’ ratings were highest in reading at

IH/AL (1.94), followed by speaking at IH (1.86), and lastly listening at IH (1.76). These results need to be

read with careful consideration because the sample size is small (N=9).

Table 9 displays the sample sizes and amounts of unanalyzable data for the German course.

Table 9. N-sizes and unanalyzable data for German courses

N-

size

LTP RTP OPIc


GER SS 9 0 0 1 1 0 0 0 0 0 0 2 2

Total 9 0 0 1 1 0 0 0 0 0 0 2 2

SS = Self-Select

As can be seen in Table 9, the sample sizes for the GER Self-Select course level is only nine

participants. In addition, there is some missing data for the course (1 NR in the LTP, 2 NRs in the OPIc).

A. GER Self-Select (DirecTrack)

Figure 11 displays the number of participants’ ratings in each proficiency level for the GER Self-

Select courses.

16 | P a g e




Figure 11. Counts of proficiency levels in GER Self-Select courses

Figure 11 illustrates that 95.8% of the ratings fall between IM and AL with one outlier in NH.

The majority of the students received a rating of AL (54.2%). The second highest number of ratings were

at the IH proficiency level (29.2%), followed by IM (12.5%).

As previously mentioned, there are three data points missing from this analysis: 1 for the LTP and

2 for the OPIc. All RTP data was used in the analysis.

IV. Korean

A total of three Korean courses were included in the analysis with a total of 66 participants.

Figure 12 shows the average proficiency on tests at each course level. KOR 3022 is the Intermediate

Korean course level that consists of both undergraduate (KOR 3022) and graduate students (KOR 4004);

both meet in the same classes. The next course level is KOR 3032, which is the Third Year Korean course

level, followed by KOR 4042, Advanced Readings in Modern Korean. For all of the Korean courses, the

only data that was collected was the OPIc. No data for LTP or RTP was collected for the Korean courses.

Figure 12 displays the overall means for all the Korean courses on the OPIc.

17 | P a g e




Figure 12. Overall means for Korean courses

Figure 12 shows that in the Intermediate Korean courses, the participants’ ratings averaged at IM.

As the course progresses to KOR 3032, the oral proficiency falls to IL (-0.07). However, as the course

progresses from KOR 3032 to KOR 4042, there is a dramatic increase (+0.53) from IL to IM/IH. Again,

the small sample should be taken into consideration when reading the analysis for the Korean courses.

Again, no LTP or RTP scores were recorded for the Korean courses.

Table 10 displays the N-sizes and unanalyzable data for the Korean courses.

Table 10. N-sizes and unanalyzable data for Korean courses

N-

size

LTP RTP OPIc


KOR

3022

47 0 0 47 47 0 0 47 47 0 0 0 0

KOR

3032

13 0 0 13 13 0 0 13 13 0 0 0 0

KOR

4042

6 0 0 6 6 0 0 6 6 0 0 0 0

Total 66 0 0 66 66 0 0 66 66 0 0 0 0

As previously mentioned, Table 10 shows an uneven distribution of participants in the four

course levels: N=47 in KOR 3022; N=13 in KOR 3032; and N=6 in KOR 4042. Again, no data was

recorded for listening or speaking proficiencies, but all of the data in the OPIc were analyzable and

therefore included in the analysis.

A. KOR 3022: Intermediate Korean

Figure 13 displays the number of participants’ ratings (total N=47) in each proficiency level for

oral proficiency in KOR 3022.

18 | P a g e




Figure 13. Counts of proficiency levels in KOR 3022

Figure 13 shows that the range of oral proficiency ratings fall between NH and IH. 24 of the

participants’ ratings (51.1%) were at IM, 13 ratings (27.7%) at IL, 6 ratings (12.8%) at NH, and 4 ratings

(8.5%) at IH. No comparison between skills can be made since only the OPIc was available.

All data was analyzable in the KOR 3022 course level.

C. KOR 3032: Third Year Korean

Figure 14 displays the counts of proficiency levels for the OPIc in KOR 3032.


Figure 14 shows that all of the participants’ ratings (N=13) in the KOR 3032 range fall between

NH and IM. 46.2% of the participants’ ratings are consistent in IM, 38.5% in IL, and 15.4% in NH.

19 | P a g e




All of the data in KOR 3032 were analyzable and included in the analysis.

D. KOR 4042: Advanced Readings in Modern Korean

Figure 15 presents the counts of oral proficiency levels for the KOR 4042 course level.


As shown in Figure 15, the participants’ scores (N=6) fall in the range of IM to AL. 3 out of the 6

participants’ scores are in the IH proficiency level; 2 of them are in IM; and one participant in the AL

proficiency level.

None of the data were excluded for the KOR 4042 class because all of the data were analyzable.

V. Portuguese

The analysis for the Portuguese test scores included two courses with a total of 15 students.

PORT 1104 is the Intermediate Portuguese course level, which consists of both undergraduate (PORT

1104) and graduate students (PORT 4104). PORT 3502W is Global Portuguese II. For all the Portuguese

course levels, the LTP, RTP, and OPIc were recorded. Figure 16 displays the average proficiency for the

participants for each course level for each skill.

20 | P a g e




Figure 16. Overall means for Portuguese courses

Figure 16 shows that the PORT 1104 participants’ average scores are around IM for RTP and

OPIc, and at IL for the LTP. Global Portuguese II students received higher average scores in all three

skills (+0.49 in LTP, +0.71 in RTP, +0.50 in OPIc). However, the sample sizes for these two course

levels are quite small.

Table 11 presents the N-sizes and unanalyzable data for the Portuguese courses.

Table 11. N-sizes and unanalyzable data for Portuguese courses

N-

size

LTP RTP OPIc


PORT

1104

6 1 0 0 1 0 0 0 0 0 0 0 0

PORT

3502W

9 1 0 0 1 0 0 0 0 0 0 0 0

Total 15 2 0 0 2 0 0 0 0 0 0 0 0

Table 11 shows that there were six participants in the PORT 1104 course level, which includes

the one graduate student (PORT 4104), and nine participants in PORT 3502W. There were only two data

points that were unanalyzable for the LTP (1 in PORT 1104 and 1 in PORT 3502W). All of the RTP and

OPIc data were included in the analysis.

A. PORT 1104: Intermediate Portuguese

Figure 17 displays the participants’ ratings (N=6) for the PORT 1104 course level.

21 | P a g e




Figure 17. Counts of proficiency levels in PORT 1104

As Figure 17 shows, all of the participants’ ratings fall between NH and IH. The most frequent

ratings are in IM, followed by IL, NH, and lastly, IH and AL. The results of the three tests are somewhat

similar, but the number of LTP ratings appear to be more in the lower range of the proficiency levels.

One data point in the LTP was excluded from the data for being unanalyzable (BR). All other

data for the RTP and OPIC was included in the analysis.

B. PORT 3502W: Global Portuguese II

Figure 18 displays the ratings for the participants (N=9) in PORT 3502W.

Figure 18. Counts of proficiency levels in PORT 3502W

22 | P a g e




As Figure 18 shows, the participants’ scores fall in the range of IM to AM, with the majority in

the IH and AL range. The highest number of scores in PORT 3502W were in IH, while the next highest

are the AL level. The remainder are in IM and AM.

All but one rating in the LTP was included in the analysis for PORT 3502W.

VI. Russian

Two Russian course levels with a total of 43 students were included in the analysis. RUSS 3002

is the Intermediate Russian II course levels, which consists of undergraduate (RUSS 3002) and graduate

students (RUSS 4104). The next progression of Russian courses is the Advanced Russian II course level,

which again consists of undergraduate (RUSS 3102) and graduate students (RUSS 4112). The

corresponding courses for Intermediate Russian II and Advanced Russian II meet together (i.e., RUSS

3002 meets in the same class as RUSS 4104; RUSS 3102 meets in the same class as RUSS 4112). LTP,

RTP, and OPIc ratings were recorded for the Russian courses. Figure 19 presents the mean proficiency for

each course level for each skill in Russian.

Figure 19. Overall means for Russian courses

Figure 19 indicates that average proficiency scores are higher for Advanced Russian II (3102)

students. The average RUSS 3002 ratings are NH for the LTP, IL for the RTP, and IM for the OPIc. The

RUSS 3102 participant ratings are higher than the RUSS 3002 courses in all three skills: +0.92 on the

LTP (from NH to IH), +0.78 on the RTP (from IL to IH), and +0.53 on the OPIc (from IM to IH).

However, the uneven N-sizes must be taken into account.

Table 12 presents the N-sizes and unanalyzable data for the Russian courses.

Table 12. N-sizes and unanalyzable data for Russian courses

N-

size

LTP RTP OPIc


RUSS

3002

35 0 0 0 0 0 0 0 0 0 0 0 0

23 | P a g e




RUSS

3102

8 1 0 0 1 0 0 0 0 0 0 0 0

Total 43 1 0 0 1 0 0 0 0 0 0 0 0

Key: T = Total

Table 12 shows that there were 35 participants in the RUSS 3002 course level and 8 participants

in the RUSS 3102 course level. There was only 1 data point that was unanalyzable for the LTP in the

RUSS 3102 course level. All of the RTP, the OPIc, and the remainder of the LTP data were included in

the findings.

A. RUSS 3002: Intermediate Russian II (for undergraduate students)

Figure 20 presents the counts of proficiency ratings in the RUSS 3002 course level.

Figure 20. Counts of proficiency levels in RUSS 3002

Figure 20 indicates that there is much variation within the RUSS 3002 course level, with most of

the participants’ scores (93.3%) between NL and IM and a few oral proficiency ratings (6.7%) in IH to

AM. 32.4% of the participants’ ratings were at IM, 25.7% of them in IL, 20.0% in NH, 10.5% in NM, and

4.8% in NL. The distribution of the LTP and RTP scores are similar, ranging between NL and IM, but the

oral proficiency ratings are higher, ranging from NH to AM.

None of the data was unanalyzable, and thus all of the participants’ data was included in the

analysis.

B. RUSS 3102: Advanced Russian II

Figure 21 presents the counts of proficiency ratings from the participants (N=7) in the RUSS

3102 course level.

24 | P a g e




Figure 21. Counts of proficiency levels in RUSS 3102

Figure 21 shows a sporadic set of results for RUSS 3102, with scores ranging from NH-AL on the

LTP, NH-AH on the RTP, and NH-S on the OPIc. The language background questionnaires that the

participants completed provide information that may have influenced the results. The questionnaires

revealed that, with the exception of two RTP ratings and one LTP rating in AL, the remaining ratings in

AL, AM, and S were all heritage language learners of Russian. This demonstrates for this group that the

learners with a heritage background in Russian typically outperform the non-heritage Russian learners

who are in the same level.

One LTP rating was excluded from the RUSS 3102 group.

25 | P a g e




VII. Spanish

For the Spanish courses, a total of 6 different courses were included in the analysis (N=73).

Similar to the French courses, the Spanish courses were consolidated into four courses, as follows: SPAN

Study Abroad, SPAN 3972W, SPAN Certificate, and SPAN Self-Select. The details of how the courses

were consolidated are in Table 13.

Table 13. Consolidation of Spanish courses

SPAN Study Abroad SPAN 3972W SPAN Certificate SPAN Self-Select

SPAN Study

Abroad: Ecuador

SPAN Study

Abroad: Toledo

SPAN 3972W SPAN Certificate SPAN Self-Select

SPAN Self-Select:

DirecTrack

Figure 22 displays the average scores on the LTP, RTP, and OPIc for the Spanish participants in

the four course levels.

Figure 22. Overall means for Spanish courses

These courses do not represent a sequential progression of courses, as is the case with some of the

other languages. SPAN Study Abroad participants received ratings at the IH proficiency level for all three

skills. SPAN Self-Select were rated at IM/IH in LTP and OPIc and AL in RTP. SPAN 3972W averaged

at IH/AL in the LTP, AL/AM in RTP, and IH in OPIc. The SPAN Certificate participants’ scores were

slightly higher in all three skills: +0.22 (from IH/AL to AL/AM) in LTP, +0.02 (from AL/AM to AM) in

RTP, and +0.17 (from IH to IH/AL) in OPIc.

Table 14 displays the total sample size for each Spanish course group and the number of

participants with unanalyzable data.

26 | P a g e




Table 14. N-sizes and unanalyzable data for Spanish courses

N-size

LTP RTP OPIc


SPAN

SA

10 0 0 0 0 0 0 0 0 0 0 0 0

SPAN

3972W

30 0 0 0 0 0 0 0 0 0 0 0 0

SPAN

Cert

4 0 0 0 0 0 0 0 0 0 0 0 0

SPAN

SS

29 0 0 2 2 0 0 1 1 0 0 0 0

Total 73 0 0 2 2 0 0 1 1 0 0 0 0

Key: T = Total; SA = Study Abroad; SS = Self-Select

Table 14 shows that there are two unanalyzable data points in the LTP and one unanalyzable data

point for the RTP, both of which are from the Spanish Self-Select group. These data points were excluded

from the analysis. Only nine participants were included in the analysis for the WPT ratings, which come

from the SPAN 3972W course (N=5), and the SPAN Certificate track (N=4). The remaining data were

excluded from the analysis.

A. SPAN Study Abroad: Ecuador & Toledo

Figure 23 presents the SPAN Study Abroad participants’ (N=10) ratings at each proficiency level.

Figure 23. Counts of proficiency levels in SPAN Study Abroad courses

Figure 23 shows that all of the scores in the SPAN Study Abroad group are between IM and AL.

Proportionally, the study abroad students scored highest in reading proficiency, followed by listening, and

then speaking.

All data in the SPAN Study Abroad track were analyzable and included in the analysis.

B. SPAN 3972W: Graduation Seminar

Figure 24 displays the counts of proficiency levels by the SPAN 3972W participants (N=30).

3 4 3

22

62

8

0

2

4

6

8

10

12

14

16

NL NM NH IL IM IH AL AM AH

SPAN Study Abroad

LTP RTP OPIc

27 | P a g e




Figure 24. Counts of proficiency levels in SPAN 3972W

Figure 24 indicates that all of the ratings in the SPAN 3972 course level range between IM and

AM; however, the majority of them (92.6%) actually fall between IH and AM with seven ratings (7.4%)

in IM. The number of ratings in IH and AL are somewhat even: 34.7% of the ratings are in AL and 33.7%

in IH. The remaining 24.2% are in AM. Based on the counts, the participants’ ratings are highest in

reading, followed by listening, and lastly speaking.

C. SPAN Certificate

Figure 25 presents the number of SPAN Certificate participants (N=4) at each proficiency level

for each skill.

Figure 25. Counts of proficiency levels in SPAN Certificate courses

28 | P a g e




Figure 25 shows that the participant ratings in SPAN Certificate courses are between IH and AM.

The distribution of ratings between each proficiency level is somewhat even. The participants’

performance are rated highest in reading, followed by listening, then speaking.

All of the SPAN Certificate data were analyzable and included in the analysis.

D. SPAN Self-Select: SPAN Self-Select, SPAN Self-Select: DirecTrack

Figure 26 displays the counts of proficiency levels for the participants (N=29) in the SPAN Self-

Select courses.

Figure 26. Counts of proficiency levels in SPAN Self-Select courses

Figure 26 shows that 95.2% of all of the participants’ scores fall in the range of IM to AL, with a

few ratings in IL (N=3, 3.6%) and AM (N=1, 1.2%). The AM proficiency level contains the most ratings

(40.5%), followed by IM (33.3%), and then IH (21.4%). The participants in the SPAN Self-Select course

level performed best in reading. Their listening and speaking proficiencies are somewhat even, which is

also indicated in Figure 22.

Three ratings were excluded from the analysis for being unanalyzable (2 from LTP, and 1 from

RTP), which is approximately 3.4% of the total data for LTP, RTP, and OPIc.

29 | P a g e




RQ 2: Do students in higher level courses demonstrate higher levels of proficiency than

students in lower levels?

The findings for Research Question 2 will exclude French, German, and Spanish results because

the course levels included in the Spring 2015 data collection in these languages are not in a linear

progression. The second research question will be answered by analyzing the result of each skill by

course level. The results will be displayed with all course levels for the remaining languages – Arabic,

Korean, Portuguese, and Russian – in one figure for comparison of proficiency levels across levels. The

figures present only test instances that received a rating on the ACTFL Guidelines and excludes all other

unanalyzable data. For details on the unanalyzable data for each course, refer back to: Table 6 (Arabic),

Table 10 (Korean), Table 11 (Portuguese), Table 12 (Russian), which can be found in the findings for

Research Question 1.

In addition to taking unanalyzable data into account, the data is also quite unevenly distributed in

terms of sample size. Because the N-size from course level to course level varies greatly, it is difficult to

compare proficiency levels at each course level.

I. Arabic

Figure 27 displays the LTP proficiency counts for all the Arabic participants by course level.

Figure 27. LTP proficiency counts by Arabic course levels

Figure 27 shows that the ARAB 1102 (both undergraduate and graduate students) LTP ratings are

spread somewhat evenly from NM through IM, with a few in NL. The ARAB 3102 LTP ratings have a

larger range (from NL to IH) but with a higher proportion of students’ ratings at the NH and IM. Lastly,

the ARAB 5102 LTP ratings, with far fewer students, cover a narrow range (between IL and IM). Thus,

the listening proficiency for the Arabic courses does tend to rise as the course levels rise (Beginning to

Intermediate to Advanced Arabic). Again, it is important to remember that the number of ratings from

participants varies greatly between courses.

Figure 28 displays the RTP proficiency counts for all the Arabic participants by course level.

30 | P a g e




Figure 28. RTP proficiency counts by Arabic course levels

As Figure 28 shows, the RTP ratings for the Beginning Arabic II course (ARAB 1102) range

from NL to IH, with the majority between NM and NH (mean is 0.64), and only a few ratings in NL, IM,

and IH. The participants’ ratings in the Intermediate Arabic course (ARAB 3102) appear similar to the

Beginning Arabic II students’ ratings with the majority of the ratings spread evenly between NM and IL.

Additionally, there is a single rating at AM. Lastly, student performances in Advanced Arabic II (ARAB

5102) were mostly rated at IL with only one participant’s rating at NH. From Beginner to Intermediate to

Advanced Arabic, there is somewhat of an increase in reading proficiency as the course levels progress;

however, the increase from Beginner to Intermediate Arabic is slight.

Figure 29 displays the OPIc rating counts for all the Arabic participants by course level.

Figure 29. OPIc proficiency counts by Arabic course levels

31 | P a g e




As Figure 29 shows, the range of OPIc ratings across all course levels is quite broad. The

Beginning Arabic II ratings (ARAB 1102) spread from NL to AL. The Intermediate Arabic ratings spread

from NL to AH, but with only one each at AM and AH. Lastly, the ARAB 5102 participants’ scores are

ranged from NH to IH with the most in IM, but with a total of only six students. Figure 29 shows that

students’ ratings in the Beginning Arabic course are actually higher than the ratings in the Intermediate

Arabic course on the OPIc. On the other hand, the Advanced Arabic II participants’ ratings are

proportionally higher than the scores of both the beginning and intermediate participants.

II. Korean

For the Korean courses, there are no LTP or RTP scores, so only the OPIc ratings will be

discussed. Figure 30 displays the OPIC proficiency counts for all the Korean participants by course level.

Figure 30. OPIc proficiency counts by Korean course levels

As Figure 30 shows, ratings from the Intermediate Korean course level (KOR 3022) range

between NH and IH with the majority in IM, followed by IL. The Third Year Korean (KOR 3032)

students’ proficiency ratings range between NH and IM. Thus, it appears that the Intermediate Korean

participants’ ratings are higher than those of the Third Year Korean participants, which is also displayed

in the means from Figure 12. Finally, the Advance Readings in Modern Korean (KOR 4042) participant

ratings fall between IM and AL, which puts their proficiency higher than the other two courses

(Intermediate and Third Year Korean). For the Korean courses, the proficiency ratings actually decreases

from Intermediate to Third Year Korean, and then increases from Third Year to Advanced Readings.

III. Portuguese

Both of the Portuguese courses (PORT 1104 and PORT 3502W) have small N-sizes: 6

participants in PORT 1104 and 9 participants in PORT 3502W. For this reason, the findings must be read

with caution.

Figure 31 displays the LTP proficiency counts for all Portuguese participants by course level.

32 | P a g e




Figure 31. LTP proficiency counts by Portuguese course levels

As seen in Figure 31, the ratings for the Intermediate Portuguese course level (PORT 1104) range

from NH to IH on the LTP. The Global Portuguese II (PORT 3502W) participant ratings were between

IM and AL, with the majority in IM and IH. From PORT 1104 to PORT 3502W, there is improvement in

listening as the course levels progress.

Figure 32 displays the RTP proficiency counts for all Portuguese participants by course level.

Figure 32. RTP proficiency counts by Portuguese course levels

Figure 32 shows that the participants’ ratings in the Intermediate Portuguese course level (PORT

1104) range between NH and AL on the RTP. It should be noted that participants’ scores are spread

across this range with one in each of the ratings and two falling at IM. The Global Portuguese II (PORT

3502W) participants’ ratings range between IH and AM, with the majority of the ratings in AL. In terms

33 | P a g e




of reading skills, there seems to be an improvement as the course levels progress from Intermediate to

Global Portuguese II.

Figure 33 displays the OPIc proficiency counts for all Portuguese participants by course level.

Figure 33. OPIc proficiency counts by Portuguese course levels

As Figure 33 shows, the Intermediate Portuguese students’ (PORT 1104) ratings fall mostly

between IL and IM in the OPIc, with one participant rated at AL. The participant who was rated at AL is

the graduate student taking the Intermediate Portuguese course (PORT 4104 before consolidation). The

Global Portuguese II (PORT 3502W) participants’ ratings ranged from IM to AM, with the majority of

the ratings in IH and AL. Overall, test scores show that there is clear improvement from the Intermediate

to Global Portuguese II course levels.

IV. Russian

Figure 34 displays the LTP proficiency counts for all the Russian participants by course level.

34 | P a g e




Figure 34. LTP proficiency counts by Russian course levels

Figure 34 shows that the Intermediate Russian II course (RUSS 3002) participants’ ratings range

from NL to IM on the LTP. The Advanced Russian II (RUSS 3102) ratings, on the other hand, are

somewhat sporadic with one rating in NH, one in IL, and the remaining 5 in AL. Despite the individual

ratings that fell in NH and IL, the Advanced Russian II course level participants’ scores are higher than

the Intermediate Russian II students in listening skills.

Figure 35 displays the RTP proficiency counts for all the Russian participants by course level.

Figure 35. RTP proficiency counts by Russian course levels

As shown in Figure 35, the ratings for Intermediate Russian II (RUSS 3002) fall between the

range of NL and IM on the RTP, with the majority in IL and IM. The Advanced Russian II (RUSS 3102)

ratings again are sporadic with one rating in NH, one rating in IL, and the remaining ratings in AL.

35 | P a g e




Despite the irregular pattern for RUSS 3102, there is still an improvement in the RTP from Intermediate

to Advanced Russian course levels.

Figure 36 displays the OPIc proficiency counts for all Russian participants by course level.

Figure 36. OPIc proficiency counts by Russian course levels

Figure 36 does not show a pattern by course level for the OPIc results. The Intermediate Russian

II (RUSS 3002) participants’ speaking proficiency ratings are spread between NH and AM with the

majority of the participants’ ratings in NH, IL, and IM. However, the eight Advanced Russian II (RUSS

3102) participants’ speaking ratings are similarly spread across levels, without a clear pattern. As

discussed in Research Question 1, the AM and S ratings actually belong to students in the course who

share the common trait of being heritage language learners of Russian. The remainder of the ratings

belong to the Advanced Russian II students who are not heritage language learners of Russian.

36 | P a g e




RQ 3a: Is there a relationship between the students’ proficiency ratings and their motivating

factors?

In addition to taking language proficiency tests, students completed surveys detailing several

different aspects of their language and cultural learning experiences, including: previous language and

cultural learning experiences, current language practices, motivation for learning the target languages.

The next three sections (Research Question 3) provide selected data from these surveys to show the

distribution of student survey responses and how they relate to proficiency scores. The first subsection for

Research Question 3 pertains to the participants’ motivating factors for learning their target language.

This section of the report shows results on the ‘motivation’ part of the survey, where students to

select and rank their top three reasons for learning the target language. Table 15 shows the number of

students who cited a particular reason as their first, second, or third reason, respectively, for pursuing

study of their target language.

Table 15. Top motivating factors selected for learning the target language

Rank

Motivating factor 1 2 3 Total

Expand cultural knowledge 77 98 68 243

I like languages 64 58 69 191

Travel 45 44 56 145

Professional reasons 39 61 27 127

Graduation requirement 71 10 23 104

Communicate with relatives / friends 19 24 28 71

Study abroad 14 23 16 53

Personal reasons 14 16 22 52

Academic reasons 6 11 12 29

Widen social circle 2 3 15 20

Table 15 shows that the motivating factor most often ranked as the most important was to

“expand cultural knowledge,” with 22% of the total first rank responses. This was also the most listed

response as a motivating factor with 243 respondents (69.2%) selecting it. The next motivating factor that

was most often ranked as most important was to “complete a graduate requirement,” with 20% of the total

first rank responses. The second most frequently selected motivating factor for learning the target

language was “I like languages,” with 191 total participant responses (54.4%).

Table 16 displays the participants’ mean proficiency ratings by their first-ranked motivating

factor.

Table 16. Proficiency levels by top motivating factors

Top motivating factor LTP N-size RTP N-size OPIC N-size

Expand cultural knowledge 1.48 N=62 1.61 N=63 1.57 N=77

I like languages 1.62 N=51 1.80 N=51 1.61 N=64

Travel 1.39 N=36 1.50 N=36 1.37 N=45

Professional reasons 1.36 N=36 1.54 N=36 1.48 N=39

37 | P a g e




Graduation requirement 0.95 N=61 0.93 N=61 1.15 N=71

Communicate with relatives/friends 1.48 N=14 1.34 N=14 1.55 N=19

Study abroad 0.98 N=9 1.01 N=9 1.32 N=14

Personal reasons 1.50 N=7 1.71 N=7 1.36 N=14

Academic reasons 1.02 N=5 0.82 N=5 1.22 N=6

Widen social circle 1.10 N=2 1.20 N=2 1.05 N=2

The top three categories most frequently selected as their top motivating factor by participants

were: 1) expand cultural knowledge, 2) graduation requirement, and 3) I like languages. Table 26 shows

the proficiency level by the top three choices of motivating factors by course level.

Table 17. Proficiency levels by top choice of motivation and course level

Course

level

1. Expand cultural knowledge 2. Graduation requirement 3. I like languages

LTP (N-size)

RTP (N-size)

OPIc (N-size)

LTP (N-size)

RTP (N-size)

OPIc (N-size)

LTP (N-size)

RTP (N-size)

OPIc (N-size)

PACE

1002/

1022

1.02

(6)

0.63

(6)

1.20

(6)

0.79

(11)

0.61

(11)

1.23

(11)

1.00

(3)

1.00

(3)

1.07

(3)

PACE

1004

0.99

(15)

0.95

(15)

1.24

(25)

0.77

(38)

0.69

(38)

0.99

(45)

1.02

(10)

0.98

(10)

1.23

(20)

PACE

3XXX

1.74

(37)

1.97

(38)

1.78

(42)

1.74

(10)

1.96

(10)

1.48

(13)

1.79

(35)

2.07

(35)

1.82

(38)

PACE

Cap/

Cert

1.70

(4)

2.15

(4)

1.93

(4)

1.70

(2)

2.05

(2)

2.05

(2)

2.30

(3)

2.30

(3)

2.10

(3)

These data show no meaningful differences in proficiency levels across the three categories;

however, the data does show that the students at the lower levels are more often motivated by graduation

requirements than the other two categories.

38 | P a g e




RQ3b: Is there a relationship between the students’ proficiency ratings and their language

practice/use?

This next section highlights the ‘language practice and use’ section of the survey, where participants are

asked about their language practices and use outside of the target language classroom. For the first

question, the survey presented a slider scale that respondents adjusted to reflect their language practices,

from 1 (Rarely or Never) to 3 (Sometimes) to 5 (Often). To best represent these responses, averages were

calculated based on the number selected, so that an average of “1” corresponds to “rarely or never” and an

average of “5” corresponds to (Often). Table 18 shows the rate of language practices across all languages

by course level.

Table 18. Rates of language practices outside class by course level

Context for language use

outside of class

PACE

1002/

1022

(N=36)

PACE

1004

(N=147)

PACE

3XXX

(N=154)

PACE

Cap/Cert

(N=14)

Total

(N=351)

Watching visual media: YouTube,

TV, or movies 2.06 2.34 2.56 2.79 2.42

Listening to news broadcasts,

podcasts, or music 1.92 2.09 2.68 2.71 2.36

Interacting with instructors and

fellow students 2.56 2.16 2.48 2.57 2.36

Reading books, newspapers, or

magazines (could be online) 1.64 1.58 2.37 2.86 1.98

Talking with friends and family 1.92 1.77 2.02 2.21 1.91

Interacting with people who share

my academic or personal interests

(in person, at events, online)

1.81 1.67 1.95 1.86 1.82

Engaging in social media:

Facebook, Twitter, etc. 1.56 1.53 2.03 2.29 1.78

Writing emails, texts, blogs, or

online discussion posts 1.64 1.44 1.83 1.79 1.65

Communicating with an assigned

conversation partner

(TandemPlus, etc.)

2.03 1.47 1.42 1.36 1.50

Participating in organized

language events (clubs, coffee

hour, etc.)

1.53 1.32 1.47 1.71 1.42

In religious contexts 1.97 1.50 1.21 1.07 1.40

At my job 1.19 1.14 1.53 1.86 1.35

Playing games online 1.11 1.16 1.18 1.29 1.17

As Table 18 shows, the top three contexts for language use outside of the target language

classroom were: 1) Watching visual media: YouTube, TV, or movies; 2) Listening to news broadcasts,

podcasts, or music; and 3) Interacting with instructors and fellow students. With an exception to a few

contexts (‘In religious contexts’ and ‘Communicating with an assigned conversation partner

(TandemPlus, etc.’), the higher course level participants (PACE 3XXX and PACE Capstone/Certificate)

39 | P a g e




on average tended to practice their target language outside the classroom more often than the lower course

level participants (PACE 1002/1022 and PACE 1004). Interestingly, for the religious context, there were

a total of 36 respondents who rated that item at a 3 or higher. Of those 36 participants, 25 of them were

Arabic learners, 5 were Korean learners, 3 were Spanish learners, 2 were Russian learners, and 1 was

Portuguese learners.

A further analysis was done to investigate the proficiency levels of the respondents who rated a 3

or a higher on the top three contexts for language use outside of the target language classroom, which can

be seen in Table 19.

Table 19. Proficiency levels by top contexts for language use by course level

Course

level

1. Watching visual media:

YouTube, TV, or movies

2. Listening to news

broadcasts, podcasts, or music

3. Interacting with instructors

and fellow students

LTP (N-size)

RTP (N-size)

OPIc (N-size)

LTP (N-size)

RTP (N-size)

OPIc (N-size)

LTP (N-size)

RTP (N-size)

OPIc (N-size)

PACE

1002/

1022

0.92

(12)

0.73

(12)

1.13

(12)

0.97

(13)

0.73

(13)

1.08

(13)

0.79

(20)

0.55

(20)

1.11

(20)

PACE

1004

1.05

(16)

1.14

(16)

1.32

(42)

1.05

(21)

1.05

(21)

1.26

(51)

0.86

(29)

0.82

(29)

1.13

(53)

PACE

3XXX

1.79

(65)

2.02

(66)

1.73

(81)

1.77

(53)

2.05

(53)

1.77

(68)

1.80

(67)

2.00

(67)

1.79

(76)

PACE

Cap/

Cert

2.07

(6)

2.20

(6)

2.24

(6)

2.07

(7)

2.30

(7)

2.07

(7)

2.18

(8)

2.25

(8)

2.19

(8)

Table 19 shows that there is not a meaningful difference of proficiency level across the top three

contexts for language use outside of the target language classroom.

In addition to asking participants about the context of language use outside of the classroom, the

survey also asked participants about the amount of language use outside of the classroom. Table 20, Table

21, and Table 22 show the LTP, RTP, and OPIc ratings, respectively, in relation to the number of contact

hours outside of the target language classroom.

Table 20. LTP ratings by amount of language use and course level

Course

level

0-1 hours 1-3 hours 3-5 hours 5-8 hours 8+ hours

LTP N-size LTP N-size LTP N-size LTP N-size LTP N-size

PACE

1002/1022

0.81 10 0.92 17 0.75 4 0.40 3 0.70 2

PACE

1004

0.85 52 0.88 32 0.95 11 0.97 3 1.30 2

PACE

3XXX

1.75 48 1.75 55 1.83 19 1.98 5 1.67 6

PACE

Cap/Cert

1.30 2 2.00 7 2.20 2 2.10 1 2.20 2

Total -- 112 -- 111 -- 36 -- 12 -- 12

40 | P a g e




Table 21. RTP ratings by amount of language use and course level

Course

level


RTP N-size RTP N-size RTP N-size RTP N-size RTP N-size

PACE

1002/1022

0.68 10 0.54 17 0.75 4 0.63 3 0.60 2

PACE

1004

0.87 52 0.73 32 0.97 11 0.57 3 1.20 2

PACE

3XXX

2.00 49 2.00 55 2.12 19 2.10 5 1.83 6

PACE

Cap/Cert

2.05 2 2.24 7 2.30 2 2.10 1 2.30 2

Total -- 113 -- 111 -- 36 -- 12 -- 12

Table 22. OPIc ratings by amount of language use and course level

Course

level


OPIc N-size OPIc N-size OPIc N-size OPIc N-size OPIc N-size

PACE

1002/1022

1.12 10 1.10 17 1.38 4 0.67 3 1.30 2

PACE

1004

1.10 75 1.10 50 1.24 16 1.33 3 1.73 3

PACE

3XXX

1.61 56 1.74 64 1.92 22 1.85 6 1.93 6

PACE

Cap/Cert

1.55 2 1.81 7 2.55 2 2.80 1 2.10 2

Total -- 143 -- 138 -- 44 -- 13 -- 13

Table 20, Table 21, and Table 22 do not show a consistent pattern of proficiency ratings as

participants’ self-reported amount of language use outside of the classroom increases. Due to the

disproportional number of participants at each course level, it is also difficult to see any patterns in the

amount of language use by course level.

41 | P a g e




RQ3c: Is there a relationship between the students’ proficiency ratings and the number of

courses taken at the University of Minnesota?

In addition to the survey data, some data was automatically pulled in from students’ records,

including the previous courses taken at the University of Minnesota. This data was used to analyze the

influence of courses taken at the University of Minnesota on proficiency. Participants had previously

enrolled in between zero and fourteen courses at the University of Minnesota in the target language. This

section presents results of the analysis with learners of all languages grouped together; language-specific

tables are presented in the appendix. Table 23 shows the average proficiency scores across all languages

for each course.

Table 23. Average LTP scores by course level and previous UMN courses

Spring 2015 Course Level

Previous

courses at

UMN

PACE

1002/1022

(N=37)

PACE

1004

(N=102)

PACE

3XXX

(N=111)

PACE

Capstone/Certificate

(N=41)

0-1

0.8

(N=37)

1.14

(N=13)

1.72

(N=25) -

2-3 -

0.84

(N=80)

1.7

(N=41)

1.95

(N=2)

4-5 -

0.9

(N=6)

1.6

(N=27)

1.92

(N=5)

6+ -

0.83

(N=3)

1.79

(N=18)

1.99

(N=34)

Table 23 shows that, while some cells have a small count, there does not appear to be a clear

pattern of listening proficiency by course level based on the number of courses taken at UMN. At the

1004 level, the 13 participants who had taken one or no courses at the University of Minnesota, had

higher average scores that those who had taken 2 or more, but this is a small cell. At the 3XXX level

participants who had taken six or more courses at the University of Minnesota had the highest levels of

proficiency, but the pattern was not linear. At the 1002/1022 and Capstone/Certificate letters, students

were grouped in similar numbers of courses, such that comparisons are not easily made. Table 24 shows

the average reading proficiency scores for participants by course level and previous courses at UMN.

Table 24. Average RTP scores by course level and previous UMN courses


Previous

courses at

UMN

PACE

1002/1022

(N=37)

PACE

1004

(N=102)

PACE

3XXX

(N=115)

PACE


(N=41)

0-1

0.62

(N=37)

1.13

(N=13)

2.04

(N=25) -

2-3 -

0.77

(N=80)

1.98

(N=44)

2.3

(N=2)

42 | P a g e




4-5 -

0.85

(N=6)

1.85

(N=28)

2.12

(N=5)

6+ -

1.17

(N=3)

1.99

(N=18)

2.23

(N=34)

As shown by Table 24, there is again no clear pattern as the number of courses students have

taken at the University of Minnesota increases. Table 25 shows participants’ OPIc scores by course level

and previous UMN course.

Table 25. Average OPIc scores by course level and previous UMN courses


Previous

courses at

UMN

PACE

1002/1022

(N=37)

PACE

1004

(N=150)

PACE

3XXX

(N=130)

PACE


(N=41)

0-1

1.11

(N=37)

1.55

(N=23)

1.93

(N=27) -

2-3 -

1.04

(N=117)

1.75

(N=46)

1.8

(N=2)

4-5 -

1.21

(N=7)

1.55

(N=38)

2

(N=5)

6+ -

1.17

(N=3)

1.66

(N=19)

1.84

(N=41)

As Table 25 shows, the students with the highest average ratings on the OPIc in the 1004 and

3XXX course levels appear to be those with the fewest courses taken at the University of Minnesota.

43 | P a g e




RQ4: Are there correlations between self-assessments and actual ratings?

Participants completed self-assessments for listening, reading, and speaking. These consisted of

“can-do” statements for which students indicated their level of ability on a four-point scale:

I can seldom do this, or I can’t do it yet.

I can do this some of the time.

I can do this most of the time.

I can do this almost always, or always.

For listening and reading, there were two different levels of the self-assessment: level one was

given to the PACE 1002/1022 course levels, and level three was given to all of the other students. For

speaking, there were three self-assessments: level one for students in the PACE 1002/1022 course level,

level two for students in PACE 1004 , and level three for students in the PACE 3XXX and PACE

Capstone/Certificate courses. However, there were no self-assessment responses for the Spanish

Capstone/Certificate participants.

Self-assessments were specifically tied to individual proficiency levels, so that the ratings were

limited for each, as seen in Table 26.

Table 26. Self-assessment ranges by level

Lowest rating Highest rating

Level 1 Novice High Intermediate Mid

Level 2 Intermediate Low Intermediate High

Level 3 Intermediate Mid Advanced Low

I. Listening

For listening skills, students took one of two levels of the self-assessment. Level one had a range

of NL to IM, but all ratings between NL and NH were grouped together. Level three had a range of NL to

AL, but all ratings from NL to IM were grouped together. The tables presenting the results of the

comparison between self-assessment and actual test ratings are color-coded to indicate if there was a

match (green), a one-level difference (yellow), or a two-level or more difference (red). Cells in which a

relationship between the self-assessment and test rating cannot be classified due to the ambiguity from

clustering (i.e., NL-NH and NL to IM in levels one and three, respectively) are shaded gray. Table 27 and

Table 28 show the results of the listening self-assessments in relation to the actual LTP test scores.

Table 27 shows the results of the level one self-assessments for listening compared to the

examinees’ LTP ratings.

Table 27. Level 1 listening self-assessments and LTP ratings

LTP rating

Self-assessment

Total NL-NH IL IM

NL 3 3 1 7

NM 9 9 1 19

NH 18 19 1 38

IL 8 7 10 25

IM 4 6 16 26

44 | P a g e




IH 0 0 1 1

AL 0 0 1 1

Total 42 44 31 117

Table 27 shows that students whose LTP rating was in the Novice Low to Novice High range

taking the level one listening self-assessment most often assess themselves at a higher proficiency level

than their actual LTP scores indicate, despite the fact that for many, the relationship between LTP and

self-assessment could not be directly determined. Forty-four of the participants’ (37.6%) self-assessments

placed them at a higher level than their LTP scores, 29 of them (24.8%) at one level above their scores

and 15 of them (12.8%) at two or more. Only 23 participants (19.7%) accurately self-assessed their

listening proficiency. Twenty participants (19.7%) assessed themselves at a lower proficiency level than

their LTP ratings. The remaining participants’ (N=30) self-assessments were grouped into the NL-NH

proficiency level in the data and their LTP scores fell in that same range; therefore, for these participants,

it is unknown how far apart the self-assessments are from the LTP scores due to the clustering.

Table 28 displays the comparison of the level three self-assessments for listening to the

examinees’ LTP ratings.

Table 28. Level 3 listening self-assessments and LTP ratings

LTP rating

Self-assessment

Total NL-IM IH AL

BR 1 4 1 6

NL 1 0 0 1

NM 2 1 0 3

NH 5 2 2 9

IL 5 9 0 14

IM 2 23 6 31

IH 3 18 6 27

AL 1 13 27 41

AM 0 0 2 2

Total 20 70 44 134

Table 28 shows similar patterns to the students who took the level one listening self-assessment,

with a bit more accuracy in matching self-assessment with their LTP scores. The results indicate that 54

of the participants (40.3%) assessed themselves above their actual LTP ratings, 29 (21.6%) of whom

assessed one level above their LTP scores and 25 (18.7%) self-assessed at two or more levels above. A

total of 45 of the participants (33.6%) rated themselves accurately, which is 10.0% higher than the level

one participants. Lastly, 19 participants (14.2%) rated themselves at one or more proficiency level below

their actual LTP ratings.

II. Reading

For reading skills, participants also took one of two levels of self-assessments: level one or level

three. Similar to the listening self-assessments, the scores were restricted based on the level of self-

assessment they took. For level one, the self-assessment rating ranged from NL to IM, but all ratings

between NL and NH were grouped together. Level three had a range of NL to AL, but all ratings from NL

to IM were grouped together. The color schematics are the same as the listening self-assessment tables

(i.e., Table 27 & Table 28). Table 29 and Table 30 display the results of the participants’ self-assessments

for reading in comparison to their RTP ratings.

45 | P a g e




Table 29 shows the results of the level one self-assessments for reading compared to the

examinees’ RTP ratings.

Table 29. Level 1 reading self-assessments and RTP ratings

RTP rating

Self-assessment

Total NL-NH IL IM

NL 6 5 1 12

NM 16 19 1 36

NH 11 16 6 33

IL 8 10 10 28

IM 0 6 11 17

IH 0 0 1 1

AL 0 0 1 1

AM 0 0 1 1

Total 41 56 32 129

Table 29 shows that the participants taking the level one reading self-assessment most often

assess themselves at a higher proficiency level than their RTP ratings, similar to their listening skills.

Fifty-eight of the participants (45.0%) assess themselves at a higher level than their RTP scores, 26 of

whom (20.2%) placed themselves at one level above their RTP ratings and 32 (24.8%) at two or more

levels above their RTP ratings. Twenty-one participants’ self-assessments (16.3%) accurately matched

their RTP scores. Seventeen participants (13.2%) underestimated their reading skills by rating themselves

below their RTP ratings.

Table 30 shows the results of the level three self-assessments for reading in relationship to the

examinees’ RTP ratings.

Table 30. Level 3 reading self-assessments and RTP ratings

RTP rating

Self-assessment

Total NL-IM IH AL

NH 0 1 1 2

IL 2 5 0 7

IM 0 5 4 9

IH 0 2 10 12

AL 1 19 59 79

AM 0 3 9 12

Total 3 35 83 121

Table 30 shows that the participants taking the level three self-assessment are quite different from

the level one participants. 61 participants (50.4%) taking the level three self-assessment accurately rated

themselves in comparison to their actual RTP ratings, and 43 of them (35.5%) were only one proficiency

level away (above or below) from their RTP scores. Only 15 participants (12.4%) rated themselves two or

more proficiency levels away (above or below) from their RTP scores. Additionally, there were more

level three participants who underestimated their reading skills (N=32, 26.4%) than those who

overestimated it (N=26, 21.5%).

III. Speaking

46 | P a g e




For speaking skills, participants took one of three self-assessments. The level one self-assessment

had a range between NL to IM, with NL to NM grouped together (NL-NM); level two had a range

between NL to above IM (>IM), with NL to NH grouped together (NL-NH) as well as anything above IM

(>IM); and lastly, level three had a range between NL to AL, with NL to IL grouped together (NL-IL).

The color schematics of the tables in the speaking section are similar to the previous self-assessment

tables. Table 31, Table 32, and Table 33 show the level one, level two, and level three speaking self-

assessments in relation to their OPIc scores.

Table 31 displays the results of the level one self-assessments for speaking in relationship to the

examinees’ OPIc ratings.

Table 31. Level 1 speaking self-assessment and OPIc ratings

OPIc rating

Self-assessment

Total NL-NM NH IL IM

NL 0 0 2 0 2

NM 0 1 1 0 2

NH 1 7 10 1 19

IL 0 9 22 3 34

IM 0 9 30 4 43

IH 0 1 6 3 10

AL 0 0 3 2 5

AM 0 0 1 0 1

S 0 0 0 1 1

UR 0 1 0 0 1

Total 1 28 75 14 118

Table 31 shows quite a different pattern from the listening and speaking self-assessments in that

the majority of the participants (N=66, 55.90%) underestimate their speaking abilities in comparison to

their OPIc scores. Thirty-three of the participants (28.0%) accurately assessed themselves at the level of

their actual OPIc rating. Only 18 participants (15.3%) assessed their speaking skills above their OPIc

ratings, for which 14 of them (11.9%) rated only one proficiency level higher.

Table 32 shows the results of the level two self-assessments for speaking in relationship to the


Table 32. Level 2 speaking self-assessments and OPIc ratings

OPIc rating

Self-assessment

Total NL-NH IL IM >IM

NL 1 4 0 0 5

NM 4 7 2 0 13

NH 2 13 3 0 18

IL 2 14 5 0 21

IM 0 5 14 0 19

IH 0 0 5 2 7

AL 0 0 3 4 7

AM 0 1 1 2 4

AH 0 0 0 1 1

S 0 0 0 1 1

Total 9 44 33 10 96

47 | P a g e




Table 32 shows that for the level two self-assessments, 34 of the students (44.2%) overestimated

their speaking abilities, 15 of them (35.4%) underestimated their speaking abilities, and 28 (29.2%)

accurately assessed themselves in speaking skills.

Table 33 displays the results of the level three self-assessments for speaking compared to


Table 33. Level 3 speaking self-assessments and OPIc ratings

OPIc rating

Self-assessment

Total NL-IL IM IH AL

NH 0 0 1 0 1

IL 1 0 0 0 1

IM 0 7 14 3 24

IH 0 4 27 15 46

AL 0 0 9 16 25

AM 0 1 1 5 7

AH 0 0 0 2 2

S 0 1 0 0 1

Total 1 13 52 41 107

Table 33 shows that the participants taking the level three self-assessment more accurately

assessed their own speaking abilities. For this group, 50 participants (46.7%) assessed their speaking

abilities at their OPIc proficiency level. 33 of the participants (30.8%) assessed themselves above their

OPIc scores, of which 29 (27.1%) were one level above their scores. Lastly, 23 participants (21.5%) self-

assessed below their OPIc ratings, for which 18 of them (16.8%) were only one level below.

Summary

In this section, a summary of all the self-assessment results will be discussed. All unanalyzable

data was excluded from this summarized analysis (i.e., BR, UR, AR, not completed, and self-assessments

in non-specific range), and the total N-sizes were adjusted accordingly. As with the previous tables, the

self-assessments were categorized as being either: accurate (same as their actual proficiency ratings), one

sublevel away, or two or more sublevels away. Table 34 summarizes the accuracy in the self-assessments

in all three skills in relation to their LTP, RTP, and OPIc proficiency levels.

Table 34. Summary of accuracy in self-assessments Listening / LTP

(N=194)

Reading / RTP

(N=207)

Speaking / OPIc

(N=299)

Accurate 68 (35.1%) 82 (39.6%) 111 (37.1%)

+/- 1 sublevel 80 (41.2%) 76 (36.7%) 131 (43.8%)

+/- 2 or more sublevels 46 (23.7%) 49 (23.7%) 57 (19.1%)

Table 34 shows that when self-assessing the three skills, the participants’ accuracy rates were

generally similar: 35.1% accurate in listening, 39.6% in reading, and 37.1% in speaking. It appears that

the proportion of participants who self-assessed one sublevel above or below their proficiency ratings

were also quite similar: 41.2% in listening, 36.7% in reading, and 43.8% in speaking. The same goes for

the proportion of participants who self-assessed at two sublevels above or below their proficiency ratings:

23.7% in listening, 23.7% in reading, and 19.1% in speaking.

48 | P a g e




To delve deeper into the self-assessments, a comparison analysis was conducted to investigate the

relationship between the level of self-assessment taken and self-assessment accuracy. Table 35 displays

the summary of accuracy in the self-assessments by level of the self-assessment the participants

completed.

Table 35. Summary of accuracy in self-assessments by self-assessment level Listening / LTP

(N=194)

Reading / RTP

(N=207)

Speaking / OPIc

(N=299)

L1

(N=79)

L3

(N=115)

R1

(N=88)

R3

(N=119)

S1

(N=116)

S2

(N=77)

S3

(N=106)

Accurate 23

(29.1%)

45

(39.1%)

21

(23.9%)

61

(51.3%)

33

(28.4%)

28

(36.4%)

50

(47.2%)

+/- 1 sublevel 36

(45.6%)

44

(38.3%)

33

(37.5%)

43

(36.1%)

56

(48.3%)

28

(36.4%)

47

(44.3%)

+/- 2 or more

sublevels

20

(25.3%)

26

(22.6%)

34

(38.6%)

15

(12.6%)

27

(23.3%)

21

(27.3%)

9

(8.5%)

Note: L = listening; R = reading; S = speaking

According to Table 35, there is an increase of accuracy from each level test score in each skill:

+10.0% from level one to level three in listening skills; +27.4% from level one to level three in reading

skills; +8.0% from level one to level two and +10.8% from level two to level three in speaking skills. This

suggests that as the participants progress through PACE levels, they become more accurate in self-

assessing their language proficiency. However, the data also shows that for the +/- 1 or 2+ sublevels,

there is no clear pattern, with some increases and some decreases as the levels progress. This suggests that

the degree of inaccuracies are somewhat inconsistent as the PACE levels progress.

49 | P a g e




RQ 5: What factors contribute to advanced proficiency in study abroad experiences?

To answer Research Question 5, responses to questions related to study abroad experiences were

analyzed. The topics of these questions include:

Duration of study abroad (less than 6 months, 6-12 months, 1-2 years)

Target language instruction (at least 50% of the time)

Living situation while studying abroad (apartment with others, dormitory, host family, other)

Frequency of target language use in the living situation

Frequency of target language use outside of the living situation

First, a broad analysis of all participants who studied abroad will be described, using responses

from the survey listed above. Following that, a descriptive analysis of participants who were rated at

advanced proficiency levels on the LTP, RTP, and OPIc will be shown to highlight their study abroad

experiences. The latter analysis will be organized first by skill (i.e., listening, reading, speaking), followed

by the topics above.

I. All Study Abroad Participants

Table 36 shows the distribution of students who studied abroad by duration of study abroad and

course level.

Table 36. Duration of study abroad by course level

Duration of study abroad

PACE

1002/1022

PACE

1004

PACE

3XXX

PACE

Cap/Cert Total

Less than 6 months 2 15 44 22 83

6 - 12 months 1 1 10 6 18

1 - 2 years 0 1 2 3 6

Total 3 17 56 31 107

A total of 107 participants reported having previously studied abroad. More than half of the total

participants who studied abroad (52.3%) were in the PACE 3XXX levels, 29% of the students in PACE

Capstone/Certificate, 16% in PACE 1004, and only 3% from the PACE 1002/1022 course level.

Moreover, approximately 78% of the participants who have study abroad experience were abroad for less

than 6 months. Nearly 17% were abroad for 6-12 months, and the remaining (6%) for 1-2 years.

The students who had study abroad experience also responded to survey questions about their

living situations and experiences with the target language during their time abroad. Table 37 shows

participants’ responses to a question about the use of target language during instruction while studying

abroad.

Table 37. Use of target language during instruction while studying abroad by course level

Was the target language the primary language

of instruction (more than 50% of the time)?

PACE

1002/1022

& 1004

PACE

3XXX

PACE

Cap/Cert Total

Yes 7 43 30 80

No 13 13 1 27

Total 20 56 31 107

50 | P a g e




Table 37 shows that the majority of the students (75%) were in environments where the primary

language of instruction was the target language. A higher ratio of participants in higher course levels had

the target language as the primary language of instruction – 97% of PACE Capstone/Certificate, 77% of

PACE 3XXX, and 35% of PACE 1002/1022 and 1004 had instruction in the target language. The survey

also asked participants about their living situations while studying abroad, which can be seen in Table 38.

Table 38. Living situation while studying abroad by course level

Living situation

PACE

1002/1022

PACE

1004

PACE

3XXX

PACE

Cap/Cert Total

Apartment with others 2 4 3 2 11

Dormitory 1 7 10 7 25

Host family 0 5 42 19 66

Other 0 1 1 3 5

Total 3 17 56 31 107

Table 38 indicates that the majority of the participants (62%) lived with a host family while

studying abroad, with only a few staying in apartments (10%), dormitories (23%), or other living

environments (5%). Participants were also asked the amount of time the language was spoken in their

living situations, as seen in Table 39.

Table 39. Frequency of target language use at home by living situation

What percentage of the time did you use the

target language in your living environment? < 25%

25% to

50%

50% to

75%

75% to

90%

90% to

100% Total

Apartment with others 9 1 1 0 0 11

Dormitory 11 6 6 1 1 25

Host family 1 2 6 6 51 66

Other 2 2 0 1 0 5

Total 23 11 13 8 52 107

Table 39 shows that nearly half of the participants (49%) reported using the target language over

90% of the time in their respective living situations. Almost one-third of the participants (32%) reported

using the target language less than half of the time in their living situations. The table also shows that

most of the participants staying with a host family claimed to be using the target language (50-100% of

the time) more frequently than those living in apartments, dormitories, or other living environments.

Table 40 shows the participants’ responses about frequency of target language use outside of the living

situation.

Table 40. Frequency of target language use outside of the home

Frequency (%) Count

Less than 25% 12

25% to 50% 21

50% to 75% 39

75% to 90% 17

90% to 100% 18

Total 107

51 | P a g e




Table 40 shows that approximately 69% of the participants used the target language over 50% of

the time outside of the living situation.

In addition to the 107 participants who had study abroad experience, a narrower analysis was

done to look into the study abroad experiences of participants who were rated at advanced levels (i.e., AL,

AM, AH, or S) in listening, reading, and speaking.

II. Participants with Advanced Listening Proficiency

Table 41 displays the duration of study abroad for students who had advanced listening

proficiency, based on the LTP.

Table 41. Duration of study abroad

Duration of Study Abroad

Advanced Proficiency

Participants

All Participants

Total

Less than 6 months 28 83

6 - 12 months 9 18

1 - 2 years 4 6

Total 41 107

As Table 41 shows, there were 41 total participants who achieved advanced proficiency in

listening and had study abroad experience. Of that 41, 28 of them (68.3%) study abroad for fewer than 6

months, 9 of them (22.0%) for 6-12 months, and 4 of them (9.8%) for 1-2 years. Table 42 displays the use

of target language during instruction throughout their study abroad experience.

Table 42. Target language use during instruction

Was the target language the primary language of

instruction (more than 50%) of the time?

Advanced

Proficiency

Participants

All

Participants

Total

Yes 37 80

No 4 27

Total 41 107

Table 42 shows that out of the 41 participants who achieved advanced proficiency in listening,

over 90% of them claimed that the target language was the primary language of instruction more than

50% of the time. Table 43 shows the living situations of the advanced listening participants during

their study abroad experience.

Table 43. Living situation while studying abroad

Living situation


Participants

All Participants

Total

Apartment with others 2 11

Dormitory 6 25

Host family 29 66

Other 4 5

Total 41 107

52 | P a g e




Table 43 indicates that the majority of the participants with advanced listening proficiency

(70.7%) stayed with a host family during their time abroad with only a few staying in an apartment (N=2)

dormitory (N=6), other living situation (N=4). Table 44 displays the frequency of target language use in

the participants’ living environments.

Table 44. Frequency of target language use at home by living situation What percentage of the time did you use the


25% to

50%

50% to

75%

75% to

90%

90% to

100% Total


Dormitory 1 2 2 0 1 6


Other 2 1 0 1 0 4

Total 4 3 5 3 26 41

Table 44 shows all of the participants who stayed with a host family (N=29) spoke in the target

language more than half the time, and approximately 86% of those participants spoke the target language

90-100% of the time. It appears that a larger proportion of participants spoke the target language less than

50% of the time when living in the other environments. However, it should be noted that the N-sizes were

quite small and varied within those other groups. Table 45 displays the frequency of target language use

outside of the home.


Frequency %


Participants

All Participants

Total

Less than 25% 0 12

25% to 50% 3 21

50% to 75% 18 39

75% to 90% 7 17

90% to 100% 13 18

Total 41 107

As shown in Table 45, almost all of the participants with advanced proficiency who studied

abroad (92.7%) claimed to use the target language over 50% of the time, and only a few participants

(N=3, 7.3%) claimed to use the target language 25-50% of the time.

The following section will discuss the study abroad experiences of participants who received a

rating of advanced or higher on the RTP. The participants in the next section are not exclusive from the

participants in this previous section, in that the same participants who received advanced proficiency in

reading may have also received advanced proficiency in listening and/or speaking.

III. Participants with Advanced Reading Proficiency

Table 46 shows the duration of study abroad for participants who had advanced reading

proficiency, based on the RTP.

53 | P a g e







Participants

All Participants

Total


6 - 12 months 13 18

1 - 2 years 4 6

Total 67 107

As Table 46 shows, a total of 67 participants who achieved advanced proficiency in reading had

some study abroad experience. Of those participants, 50 of them (74.6%) were abroad for less than 6

months, 13 of them (19.4%) studied abroad for 6-12 months, and 4 of them (6.0%) studied abroad for 1-2

years. Table 47 displays the use of target language during instruction throughout their study abroad

experience.




Advanced

Proficiency

Participants

All

Participants

Total

Yes 60 80

No 7 27

Total 67 107

As Table 47 indicates, 60 of the 67 participants who achieved advanced reading proficiency

(89.6%) responded that the target language was the primary language of instruction for more than 50% of

the time. Table 48 shows the living situations of the advanced listening participants during their

study abroad experience.


Living situation


Participants

All Participants

Total


Dormitory 11 25

Host family 48 66

Other 4 5

Total 67 107

Table 48 shows that the majority of the participants who achieved advanced proficiency in

reading (71.6%) also stayed with a host family during their time abroad, while the remaining stayed in

apartments (N=4), dormitories (N=11), or other living environments (N=4). Table 49 displays the

frequency of target language use in the participants’ living environments.



25% to

50%

50% to

75%

75% to

90%

90% to

100% Total



54 | P a g e





Other 2 1 0 1 0 4

Total 6 4 11 4 42 67

As shown in Table 49, all of the participants who achieved advanced reading proficiency and

were staying with a host family used the target language more than half of the time at home, with most of

them (41 out of 48) claiming to use it 90-100% of the time. For the participants staying in a dormitory,

most of the participants (8 out of 11) claimed to be using the target language between 25 to 75% of the

time. Lastly, for the participants staying in an apartment or other living situation, most of the participants

(3 out of 4) claimed to use the target language less than 50% of the time. Table 50 displays the frequency

of target language use outside of the home.


Frequency %


Participants

All Participants

Total

Less than 25% 0 12

25% to 50% 10 21

50% to 75% 30 39

75% to 90% 12 17

90% to 100% 15 18

Total 67 107

Table 50 shows that a variety of answers in response to their frequency of target language use

outside of the living environment. Over 85% of the respondents claimed to use the target language over

50% of the time, with most of them claiming to use it between 50% and 75% of the time outside the

home.

The next section will discuss the study abroad experiences of participants who received advanced

proficiency on the OPIC. Again, the participants in the next section are not exclusive from the participants

in the previous two sections and may overlap by being advanced in more than one skill.

IV. Participants with Advanced Speaking Proficiency

Table 51 displays the duration of study abroad for participants who had advanced speaking

proficiency, based on the OPIc.




Participants

All Participants

Total


6 - 12 months 8 18

1 - 2 years 1 6

Total 27 107

Table 51 shows that a total of 27 participants who received a rating of advanced or higher on the

OPIc also spent some time studying abroad. Of these 27 participants, 18 of them (66.7%) studied abroad

55 | P a g e




for less than 6 months, 8 of them (29.6%) studied abroad for 6-12 months, and only one of them (3.7%)

studied abroad for 1-2 years. Table 52 displays the use of target language during instruction throughout

their study abroad experience.




Advanced

Proficiency

Participants

All

Participants

Total

Yes 21 80

No 6 27

Total 27 107

Table 52 shows that 21 out of the 27 participants who achieved advanced speaking proficiency

(77.8%) claimed that for more than half of the time the target language was the primary language of

instruction. Table 53 shows the living situations of these participants during their time abroad.


Living situation


Participants

All Participants

Total


Dormitory 2 25

Host family 19 66

Other 3 5

Total 27 107

Table 53 shows that over 70% of the participants with advanced speaking proficiency

stayed with a host family during their study abroad experience. Table 54 shows the frequency of target

language use in the participants’ living environments.



25% to

50%

50% to

75%

75% to

90%

90% to

100% Total




Other 1 1 0 1 0 3

Total 4 1 2 3 17 27

Table 54 shows that all of the participants with advanced speaking proficiency who stayed with a

host family used the target language more than half of the time in their living situation, with over 84% of

them using it 90 to 100% of the time. Most of the participants in the other living environments did not use

the target language as frequently in their living situations. However, it is important to keep in mind that

the N-sizes are quite different between the living situations. Table 55 displays the frequency of target

language use outside of the home.

56 | P a g e





Frequency %


Participants

All Participants

Total

Less than 25% 0 12

25% to 50% 5 21

50% to 75% 10 39

75% to 90% 1 17

90% to 100% 11 18

Total 27 107

Table 55 shows that over 80% of the participants claimed to use the target language over 50% of

the time, with most of them claiming to use between 50 to 75% of the time (N=10) and 90 to 100% of the

time (N=11).

57 | P a g e




CONCLUSIONS

Spring 2015 data collection at the University of Minnesota contributed an additional 363 student

test performances, self-assessment data and background information to the database of proficiency scores

and survey responses from Fall 2014. In addition to the four more commonly taught languages included

in the Fall 2014 data (French, German, Portuguese, and Spanish), Spring 2015 data collection added

Arabic, Korean, and Russian. In Spring 2015, there were far fewer unratable tests, which allows for

proportionately more useable data to be included.

In terms of the research questions, student proficiency levels were documented at each course level in

each language by domain (listening, speaking and reading). As students move through the course

progressions, there were differences across languages and domains in the number of students who

increased in proficiency. In Arabic, there was a great spread in proficiency outcomes across levels and

domains. There was no demonstration of marked improvement between Beginning and Intermediate level

students, but students in Advanced Arabic demonstrated proportionately higher scores than students in

both Beginning and Intermediate level courses. For Korean, in which students were only assessed

according to the OPIc, there was a decrease in student performance from Intermediate to third-year

Korean and then an increase between third year and Advanced Reading. Students enrolled in Portuguese

courses generally showed an increase in proficiency from Intermediate to higher-level courses. For

Russian, student results varied widely and sporadically; however the language background questionnaire

indicated that, for Russian, learners with a heritage background in Russian typically outperformed the

non-heritage Russian learners who are in the same course level. Students enrolled in Spanish classes

showed somewhat of an increase as they moved to higher level courses; however, the variety of similarly-

leveled courses makes comparisons challenging.

Furthermore, the correlation between many survey questions’ responses and proficiency levels

can be difficult to evaluate because of confounding variables such as level and language of study. For

example, the effect of previous language study on proficiency level is difficult to establish because a

clearly confounding factor is the number of university courses in which a student has participated, a level

of analysis that further decreases cell sizes. However, it was noted that students with the fewest courses at

the University had the highest levels of proficiency. This is likely attributable to their previous study,

which may have resulted in arriving at University of Minnesota with relatively higher levels of

proficiency to begin. However, students who took six or more courses at the university appeared to then

demonstrate similarly high levels to students with few courses. Therefore, this outcome suggests that long

sequences of language learning may lead to higher levels of proficiency.

The self-assessment questions showed that students generally assessed themselves most

accurately in speaking and with the least accuracy in listening. It was also noted that students enrolled in

higher level courses were more accurate in their own self-assessment than students in lower level courses.

Finally, the ability to look in depth at the effects of study abroad on proficiency was affected by

small cell sizes. While many students reported prior study abroad experience, the level of detail collected

on those experiences cannot provide much illumination to the factors that contribute to language learning.

Students indicated the length of study abroad in large increments with the smallest increment identified as

less than six months, a timeframe that could encompass many models of study abroad. The great majority

of respondents who had studied abroad indicated this duration. Future data collection will expand these

cell sizes to allow for more meaningful comparisons.

These data allow a window into one semester of student proficiency outcomes across languages.

However, longitudinal studies with data collected from the same students over time (semesters and years

58 | P a g e




of study) would complement these data and present more than a snapshot of student outcomes but also a

long-term picture of progress and the factors that can contribute to progress.

59 | P a g e




REFERENCES

American Council on the Teaching of Foreign Languages. (2012). ACTFL Proficiency

Guidelines 2012. Alexandria, VA: Author.

Dandonoli, P. & Henning, G. (1990). An investigation of the construct validity of the ACTFL

proficiency guidelines and oral interview procedure. Foreign Language Annals, 23(1),

11-22.

National Education Security Program. The Language Flagship. Retrieved from:

http://www.nsep.gov/content/language-flagship. 31 July 2015.

Vande Berg, M., Connor-Linton, J., & Paige, J. M. (2009). The Georgetown Consortium Project:

Interventions for student learning abroad. Frontiers: The Interdisciplinary Journal of

Study Abroad, 18, 1-75.

60 | P a g e




APPENDIX

Table 56. Arabic LTP scores by course level and previous UMN courses


Previous courses at

UMN

PACE 1002/1022 (N=37) PACE 1004

(N=61)

PACE 3XXX

(N=6)

0-1 0.80 (N=37)

1.19 (N=8)

1.30 (N=1)

2-3

- 0.85 (N=52)

1.10 (N=1)

4-5

- 1.10 (N=1)

1.17 (N=3)

6+

- - 1.10 (N=1)

Table 57. Arabic RTP scores by course level and previous UMN courses


Previous courses at

UMN

PACE 1002/1022 (N=37) PACE 1004

(N=61)

PACE 3XXX

(N=6)

0-1 0.62 (N=37)

1.05 (N=8)

0.80 (N=1)

2-3 - 0.63 (N=52)

1.10 (N=1)

4-5 - 0.30 (N=1)

1.10 (N=3)

6+ - -

1.10 (N=1)

Table 58. Arabic OPIc scores by course level and previous UMN courses


Previous courses at

UMN

PACE 1002/1022 (N=37) PACE 1004

(N=62)

PACE 3XXX

(N=6)

0-1 1.11 (N=37)

1.70 (N=9)

1.30 (N=1)

2-3 - 0.80 (N=52)

1.30 (N=1)

4-5 - 0.80 (N=1)

1.40 (N=3)

6+ - -

0.80 (N=1)

61 | P a g e




Table 59. French LTP scores by course level and previous UMN courses


Previous courses at UMN PACE 3XXX (N=43) PACE Capstone/Certificate (N=7)

0-1 1.80 (N=10)

-

2-3 1.84 (N=14)

-

4-5 1.76 (N=9)

1.80 (N=2)

6+ 1.83 (N=10)

1.88 (N=5)

Table 60. French RTP scores by course level and previous UMN courses



0-1 2.08 (N=10)

-

2-3 2.09 (N=15)

-

4-5 1.99 (N=10)

2.05 (N=2)

6+ 2.16 (N=10)

2.22 (N=5)

Table 61. French OPIc scores by course level and previous UMN courses



0-1 2.09 (N=10)

-

2-3 1.91 (N=14)

-

4-5 1.98 (N=8)

2.30 (N=2)

6+ 1.91 (N=10)

2.06 (N=5)

62 | P a g e




Table 62. German LTP scores by course level and previous UMN courses


Previous courses at UMN PACE 3XXX (N=8)

0-1 2.10 (N=2)

2-3 1.73 (N=3)

4-5 -

6+ 1.57 (N=3)

Table 63. German RTP scores by course level and previous UMN courses



0-1 2.10 (N=2)

2-3 1.83 (N=4)

4-5 -

6+

2.00 (N=3)

Table 64. German OPIc scores by course level and previous UMN courses



0-1 2.10 (N=2)

2-3 1.90 (N=3)

4-5 -

6+

1.55 (N=2)

63 | P a g e




Table 65. Korean OPIc scores by course level and previous UMN courses


Previous courses at UMN PACE 1004 (N=47) PACE 3XXX (N=19)

0-1 1.42 (N=9)

2.10 (N=1)

2-3 1.19 (N=37)

1.50 (N=4)

4-5 0.80 (N=1)

1.19 (N=12)

6+ - 1.30 (N=2)

64 | P a g e




Table 66. Portuguese LTP scores by course level and previous UMN courses



0-1 - 1.70 (N=2)

2-3 1.16 (N=6)

1.70 (N=6)

4-5 - 1.30

(N=1)

6+ - -

Table 67. Portuguese RTP scores by course level and previous UMN courses



0-1 - 2.20 (N=2)

2-3 1.40 (N=6)

2.08 (N=6)

4-5 - 2.10

(N=1)

6+ - -

Table 68. Portuguese OPIc scores by course level and previous UMN courses



0-1 - 2.20 (N=2)

2-3 1.37 (N=6)

1.77 (N=6)

4-5 - 1.80

(N=1)

6+ - -

65 | P a g e




Table 69. Russian LTP scores by course level and previous UMN courses



0-1 1.06 (N=5)

2.10 (N=2)

2-3 0.74 (N=22)

2.10 (N=1)

4-5 0.86 (N=5)

1.33 (N=3)

6+ 0.83 (N=3)

2.10 (N=2)

Table 70. Russian RTP scores by course level and previous UMN courses



0-1 1.26 (N=5)

2.10 (N=2)

2-3 0.92 (N=22)

2.10 (N=1)

4-5 0.96 (N=5)

1.67 (N=3)

6+ 1.17 (N=3)

1.60 (N=2)

Table 71. Russian OPIc scores by course level and previous UMN courses



0-1 1.52 (N=5)

2.65 (N=2)

2-3 1.25 (N=22)

2.30 (N=1)

4-5 1.38 (N=5)

1.47 (N=3)

6+ 1.17 (N=3)

1.30 (N=2)

66 | P a g e




Table 72. Spanish LTP scores by course level and previous UMN courses


Previous courses at UMN

PACE 3XXX (N=37) PACE Capstone/Certificate

(N=34)

0-1 1.48 (N=8) -

2-3 1.59 (N=16)

1.95 (N=2)

4-5 1.73 (N=11)

2.00 (N=3)

6+ 2.10 (N=2)

2.01 (N=29)

Table 73. Spanish RTP scores by course level and previous UMN courses




(N=34)

0-1 2.09 (N=8) -

2-3 1.92 (N=17)

2.30 (N=2)

4-5 1.95 (N=11)

2.17 (N=3)

6+ 1.95 (N=2)

2.23 (N=29)

Table 74. Spanish OPIc scores by course level and previous UMN courses




(N=34)

0-1 1.56 (N=9) -

2-3 1.64 (N=17)

1.80 (N=2)

4-5 1.67 (N=11)

1.80 (N=3)

6+ 1.70 (N=2)

1.80 (N=29)

Documents

University of Minnesota Proficiency Project · The participants in this project are university students taking foreign language courses at the University of Minnesota during the spring