Tracking Pathways to Success

Identifying Learning Success Factors Across Course Delivery Formats

Peter Usinger, State Assessment Meeting 2013

Student Success in College Environment

High W/F

As high as 50% attrition

of FTIC cohort

45% of Withdrawal Reasons are

Course Related

Student Attribute

sFinancially

Limited

Non-traditional

Academically Underprepare

d

Study Context

Reviewing High F/W

Engagement in College Prep

F2F vs. Online Instruction

Nursing OrientationSLS Course

Support

Research Questions

Q1• Delivery format based differences between self-

directed behaviors and learning strategies.

Q2

• Similarities and Differences across subject domains/disciplines and their sensitivity to course delivery formats.

Q3

• Relationships among MSLQ constructs, and with student characteristics, course characteristics, and delivery formats.

Q4

• “Predictive” model to explain the relationships for developing support mechanisms and change curricular design for greater success/completion

(Online) Course Completion

Student Characteristics

Demographic variables

Previous e-learning experience

Self-Directed Learning Habits

Metacognitive skillsMotivationSelf-disciplineAutonomySelf-regulated behaviors

Sample drawn from…

Mixed withdrawal/failure rates136 Courses• Humanities, English/Letters, Mathematics and Statistics,

Sciences, Social Sciences, Nursing; Hybrids and Workforce Dev. excluded from analysis due to low frequencies.

Delivery• Online (216 Sections)• Face to Face (931 Sections)

>20,000 StudentsPopulation• Invited via email to complete MSLQ online (15 min.)

1,179 Course sections (total)

Sample by Academic Area

2,200 Participants:11% Response RateDiscipline F2F Online

Humanities 43 29

Letters 314 84

Math 957 82

Nursing 136 33

Sciences 207 56

Social Sciences 85 64

MSLQ Scales and Reliability

Motivated Strategies for Learning Questionnaire (MSLQ)

Original MSLQ Applied in Study

Items α Items α

A. Motivational Constructs1. Intrinsic Goal Orientation 4 0.74 4 0.752. Extrinsic Goal Orientation 4 0.62 2 0.633. Task Value 6 0.90 2 0.904. Control of Learning Beliefs 4 0.68 3 0.765. Self-Efficacy for Learning & Performance 8 0.93 8 0.966. Test Anxiety 5 0.80 4 0.75 B. Learning Strategies Constructs 1. Rehearsal 4 0.69 2 0.692. Elaboration 6 0.75 2 0.583. Organization 4 0.64 2 0.544. Critical Thinking 5 0.80 5 0.805. Metacognitive Self-Regulation 12 0.79 10 0.846. Time/Study Environmental Management 8 0.76 7 0.827. Effort Regulation 4 0.69 4 0.718. Peer Learning 3 0.76 3 0.779. Help Seeking 4 0.52 3 0.72Total Items in Questionnaire 81 61

Motivational Correlations

Pearson Correlation Coefficients (N=2,200)

 …between Grades Passing Online+

…and Motivational Constructs      

1. Intrinsic Goal Orientation 0.19*** 0.16*** 0.04

2. Extrinsic Goal Orientation 0.03 0.08*** -0.04

3. Task Value 0.24*** 0.19*** 0.04*

4. Control of Learning Beliefs 0.30*** 0.25*** 0.10***

5. Self-Efficacy for Learning & Performance 0.46*** 0.37*** 0.07**

6. Test Anxiety -0.23*** -0.15*** -0.01

Note 1: *p < .05. **p < .01. ***p < .001. Note 2: +Method: F2F=0, Online=1.

Learning Strategies

Pearson Correlation Coefficients (N=2,200) …between Grades Passing Online

…and Learning Strategies Constructs      

1. Rehearsal 0.00 0.02 -0.06**

2. Elaboration 0.09*** 0.07** 0.02

3. Organization 0.04 0.03 -0.06*

4. Critical Thinking 0.04 0.01 0.06*

5. Metacognitive Self-Regulation 0.04 0.04* 0.00

6. Time/Study Environmental Management 0.19*** 0.13*** 0.07**

7. Effort Regulation 0.27*** 0.21*** 0.06*

8. Peer Learning -0.04 -0.02 -0.19***

9. Help Seeking -0.04 0.00 -0.32***

(where *p < .05. **p < .01. ***p < .001.)

Due to the very minor statistical differences in the results for grades vs. course success, and the fact that a significantly higher proportion of C and D students across most discipline areas are subsequently failing, we’ve created a Student Performance aggregate variable that combines student’s Course Performance into three levels:

• Level 0: Failure or Withdrawal from Course• Level 1: Course Grade Equals C or D• Level 2: Course Grade Equals A or B

This Performance variable is used in the student success analysis following.

Data Aggregation Rules

Comparison by Delivery Type

MSLQ Correlation w/ Course Performance

Online Face to FaceMean Correlation Mean Correlation

Intrinsic Goal Or. 5.11 0.18*** 4.99 0.19***Extrinsic Goal Or. 5.82 0.01 5.94 0.06*Task Value 4.99 0.23*** 4.79 0.23***Control Beliefs 5.83 0.31*** 5.49 0.28***Self-Efficacy 5.54 0.43*** 5.20 0.44***Test Anxiety 4.42 -0.21*** 4.47 -0.21***Rehearsal 4.79 -0.04 5.04 0.03Elaboration 5.28 0.12* 5.20 0.06*Organization 5.00 0.04 5.21 0.03Critical Thinking 4.33 0.07 4.13 0.01Metacog. S-Reg. 4.86 0.03 4.84 0.04Time/Study Mgmt. 5.48 0.22*** 5.26 0.15***Effort Regulation 5.86 0.25*** 5.69 0.24***Peer Learning 2.58 -0.07 3.41 0.00Help Seeking 3.10 -0.02 4.52 0.01

N= 366 1,751

Comparison by Course Level

MSLQ Correlation w/ Course Performance

Developmental College LevelMean R Coeff. Mean R Coeff.

Intrinsic Goal Or. 4.97 0.21*** 5.03 0.17***Extrinsic Goal Or. 6.04 0.07 5.84 0.05*Task Value 4.59 0.24*** 4.94 0.21***Control Beliefs 5.48 0.31*** 5.57 0.28***Self-Efficacy 5.15 0.43*** 5.32 0.44***Test Anxiety 4.56 -0.23*** 4.40 -0.19***Rehearsal 5.13 0.07 4.92 0.00Elaboration 5.14 0.06 5.26 0.09***Organization 5.33 0.05 5.10 0.04Critical Thinking 4.13 -0.02 4.21 0.04Metacog. S-Reg. 4.94 0.10* 4.80 0.03Time/Study Mgmt. 5.42 0.23*** 5.24 0.17***Effort Regulation 5.79 0.27*** 5.69 0.25***Peer Learning 3.33 0.00 3.22 -0.04Help Seeking 4.45 0.00 4.19 -0.01

N= 551 962

Important Note by Author:Please note that the following slide has been corrected since it contained a transfer error due to a truncated spreadsheet field. Affected is one of the interesting findings with regard to the role of test anxiety in the course performance of Nursing students. The original presentation slides showed a positive correlation between Test Anxiety and Course Performance for the whole Nursing group in this sample. However, this is only true for the 2nd year cohort of the RN program involved. Pre-admission, 1st year, and BSN students show patterns similar to other academic areas. This is an extremely valuable research result since this correlation also goes along with a positive correlation between Test Anxiety and Rehearsal Strategies and Peer Learning. This indicates that our 2nd year Nursing students were not negatively overwhelmed with anxiety, but had learned to offset their anxiety levels successfully by rehearsing the subject matter at hand, and working with their peers to master the course content. We will follow-up on these findings with an appropriate discipline-specific pathway model in the next phase of this longitudinal research project.

Correlations by Academic Area

Pearson Correlation with Course Performance

Humanities (N=75)

Letters (N=416)

Math (N=1043)

Nursing (N=174)

Sciences (N=276)

Soc. Sci. (N=156)

Motivational Constructs1. Intrinsic Goal Orientation 0.14 0.07 0.22 0.22 -0.04 0.242. Extrinsic Goal Orientation -0.01 0.11 0.06 0.06 0.00 0.123. Task Value 0.25 0.10 0.21 0.27 -0.03 0.324. Control of Learning Beliefs 0.30 0.16 0.31 0.48 0.15 0.275. Self-Efficacy Learn. & Perf. 0.32 0.29 0.46 0.55 0.34 0.336. Test Anxiety -0.15 -0.04 -0.27 -0.20 -0.16 -0.09Learning Strategies 1. Rehearsal -0.06 0.03 0.01 0.06 -0.08 0.122. Elaboration 0.23 0.11 0.04 0.13 -0.04 0.123. Organization -0.05 0.10 0.02 0.02 -0.04 0.114. Critical Thinking 0.09 0.00 -0.02 0.07 -0.13 0.005. Metacognitive Self-Reg. 0.02 0.03 0.07 0.11 -0.12 0.116. Time/Study Management 0.20 0.25 0.15 0.28 0.09 0.317. Effort Regulation 0.19 0.22 0.27 0.39 0.05 0.328. Peer Learning -0.12 -0.11 0.02 0.02 -0.17 0.029. Help Seeking -0.15 0.03 0.02 -0.03 -0.14 0.08

Success Comparison in Math

Example: Math Differences Failure/Withdrawal Passing Grade Difference

Motivational ConstructsIntrinsic Goal Orientation 58.3% 66.8% 8.5%Extrinsic Goal Orientation 79.6% 84.0% 4.5%Task Value 48.1% 59.7% 11.6%Control Beliefs 63.4% 77.9% 14.5%Self-Efficacy 50.0% 73.8% 23.7%Test Anxiety 67.6% 55.6% -11.9% Learning Strategies ConstructsRehearsal 65.1% 67.1% 2.0%Elaboration 65.9% 68.3% 2.4%Organization 67.6% 70.1% 2.5%Critical Thinking 50.0% 48.9% -1.1%Meta-Cognitive Self-Regulation 61.5% 64.8% 3.3%Time/Study Management 67.1% 73.4% 6.3%Effort Regulation 70.4% 81.2% 10.7%Peer Learning 36.0% 37.3% 1.4%Help Seeking 53.7% 56.1% 2.5%

Multivariate Regression I

Multivariate Regression Analyses (Total Sample)

Predicting Course Performance

F-Value Pr > F R-Square

Fit of Model 42.1 <.0001 0.24Parameter Estimates t-Value Pr > |t| St-B

Intrinsic Goal Orientation -1.97 0.05 -0.06Extrinsic Goal Orientation -2.52 0.01 -0.06Task Value 0.51 0.61 0.01Control of Learning Beliefs -1.84 0.07 -0.06Self-Efficacy for Learning 14.84 <.0001 0.52Test Anxiety -1.46 0.14 -0.03Rehearsal 1.27 0.20 0.03Elaboration -0.47 0.64 -0.01Organization -1.09 0.27 -0.03Critical Thinking -0.75 0.45 -0.02Metacognitive Self-Regulation -1.84 0.07 -0.07Time/Study Management 2.26 0.02 0.07Effort Regulation 2.95 0.00 0.09Peer Learning -1.07 0.29 -0.03Help Seeking -0.42 0.67 -0.01

Multivariate Regression II

Multivariate Regression Analyses w/ Performance

Online Courses Face-to-Face Classes

F-Value Pr > F R-Square F-Value Pr > F R-Square

Fit of Model 7.1 <.0001 0.22 33.2 <.0001 0.24Parameter Estimates t-Value Pr > |t| St-B t-Value Pr > |t| St-B

Intrinsic Goal Orientation -0.35 0.73 -0.03 -1.47 0.14 -0.05Extrinsic Goal Orientation -0.54 0.59 -0.03 -1.66 0.10 -0.04Task Value -0.25 0.80 -0.02 0.27 0.78 0.01Control of Learning Beliefs 0.57 0.57 0.04 -2.30 0.02 -0.08Self-Efficacy for Learning 5.36 <.0001 0.44 13.60 <.0001 0.54Test Anxiety -0.40 0.69 -0.02 -1.83 0.07 -0.05Rehearsal -0.77 0.44 -0.05 2.14 0.03 0.07Elaboration -0.26 0.80 -0.02 -0.76 0.45 -0.02Organization -0.18 0.86 -0.01 -1.17 0.24 -0.04Critical Thinking 1.54 0.13 0.11 -1.88 0.06 -0.06Metacognitive Self-Regulation -1.67 0.10 -0.15 -1.12 0.26 -0.05Time/Study Management 2.37 0.02 0.17 0.85 0.39 0.03Effort Regulation -0.05 0.96 0.00 2.92 0.00 0.10Peer Learning -0.99 0.33 -0.07 -0.48 0.63 -0.01Help Seeking -0.04 0.97 0.00 -0.14 0.89 0.00

Multivariate Regression III

Multivariate Regression Analyses w/ Performance

Developmental College Level

F-Value Pr > F R-Square F-Value Pr > F R-Square

Fit of Model 14.0 <.0001 0.25 30.0 <.0001 0.24Parameter Estimates t-Value Pr > |t| St-B t-Value Pr > |t| St-B

Intrinsic Goal Orientation -0.51 0.61 -0.03 -2.07 0.04 -0.07Extrinsic Goal Orientation -2.20 0.03 -0.09 -1.21 0.23 -0.03Task Value 0.34 0.73 0.02 -0.27 0.78 -0.01Control of Learning Beliefs -0.42 0.67 -0.02 -2.08 0.04 -0.08Self-Efficacy for Learning 7.31 <.0001 0.48 13.29 <.0001 0.55Test Anxiety -1.90 0.06 -0.08 -0.72 0.47 -0.02Rehearsal 1.64 0.10 0.08 0.74 0.46 0.02Elaboration -1.97 0.05 -0.11 -0.30 0.76 -0.01Organization -0.36 0.72 -0.02 -0.39 0.70 -0.01Critical Thinking -2.03 0.04 -0.11 0.14 0.89 0.00Metacognitive Self-Regulation 1.08 0.28 0.07 -2.55 0.01 -0.11Time/Study Management 2.33 0.02 0.13 1.44 0.15 0.05Effort Regulation 0.69 0.49 0.04 3.07 0.00 0.11Peer Learning -0.33 0.75 -0.02 -1.04 0.30 -0.03Help Seeking -0.77 0.44 -0.04 0.47 0.64 0.02

• MSLQ scales: excellent reliability & context sensitivity• Motivational factors clearly driving student success• Learning Strategies (cognitive skills) show multi-faceted instructional improvement (learning) opportunities 

• Peer Learning/Help Seeking: Underrepresented within cohorts, not an integrated/facilitated college activity

• Many underprepared students succeed largely via intrinsic motivation and confidence/self-efficacy, combined with hard work and study management

• The lack of exposure to meta-cognitive abilities and critical thinking skills nurtures course environments in support of compliant learners instead of self-directed learners!Conclusions

Study Impact / Next Steps

• Online: Assess Quality Matters impact across delivery formats and feed information back into process

• 1st Year/EWS: Compare demographic differences associated with the various success patterns

• SLS: Evaluate impact with SLS courses when applying the MSLQ as formative assessment tool early in Student Success or Developmental Ed classes

• Overall: Conduct additional focus groups at college sites; seek expansion of study to inform interventions

• FCS: Seek study replication across service areas

Thank You!pusinger@polk.edu