Using The College Persistence Questionnaire To Enhance Commitment Among NGCSU

Cadets

Back In The Antediluvian Days

Why the Increased Emphasis on Retention

1. Financial and legislative pressures on colleges.2. Global economy and automation: Heighten the need for

skilled versus semi- and unskilled labor3. Schools are becoming more sensitive to students’ needs.4. Death of a dream: Failure to graduate often leads to

reductions in students’ income and quality of life

Some Students Have Special Requirements

What Do We Need To Know About Our Students?

It Depends Upon Your Responsibilities

Working With Individual Students: What Counselors, Advisors and Faculty Need To Know

1. Which cadets are most at-risk?2. What factor(s) are causing a particular cadet to leave the

Corps?3. We need scale scores and responses to individual items.

CPQ Advisor Portal

https://www.beckdavidson.com/welcome/welcome.aspx

Working With Groups of Students: What Policy-Makers and Instructors Of

Persistence-Oriented Courses Need To Know

1. What factor(s) distinguish students at their schools who will persist and will not persist in their educations?

2. Are our retention programs effective?

Six Revelations On The Path To An Effective Retention Program (Part 1)

Revelation What That Means1. We have a retention problem.

Yes, every school has a retention problem. Some schools have a greater problem than others. Nevertheless, every school includes students that do not graduate who have the capacity to graduate.

2. Someone needs to do something.

Retention problems will not just disappear; they cannot be ignored except at great cost, they must be addressed. Following this revelation, a policy maker often tasks someone or a group to deal with the retention issue.

3. We need to know what works.

You know there is a problem. Now you need to determine what it is and what to do about it. A common practice is for schools to model their initial retention activities after a sister school that claims success or a program reported in the empirical literature.

Six Revelations On The Path To An Effective Retention Program (Part 2)

Revelation What That Means4. We need to know if our activities decrease attrition.

Many schools believe that their retention-reducing activities are effective. Few schools, however, could provide a skeptic with information demonstrating that their activities are attenuating attrition or having beneficial effects. Realizing this is a big step forward.

5. We need to focus our resources, time, and energy on those activities that will have the greatest impact on our students.

Knowing what affects retention at your school is not enough. We must prioritize. There are more variables that affect persistence decisions than any school can address. Resources are limited so we must concentrate our energies on those variables that most strongly influence retention at our institution.

6. We need to establish an ongoing data collection system to evaluate our progress, augment our strengths, and overcome our weaknesses.

Students change, schools change, programs change. An in-place assessment system will allow us to demonstrate our achievements, make steady improvements, and accommodate changes to our educational systems.

Strategies For Improving Retentioin

Rely On Someone’s Professional Expertise

Subject matter experts (SMEs) have invaluable experience. Most of us were employed because we are military and/or academic SMEs.

Rely On Someone’s Professional Expertise

Problems With SMEs: SMEs can make mistakes. Some SMEs may not fully appreciate the consequences of their decisions.

Rely On Someone’s Professional Expertise

King MacArthur MarshallProblems With SMEs: Even the most expert of SMEs sometimes disagree. How do you determine whose position to support?

Look To The Literature For Guidance

In comparison to continuing-generation (CG) students first-generation (FG) students tend to be:1.Female 2. Older3.Have dependents 4. From a lower SES

FG students also tend to have:1.Lower standardized test scores2.Poor math, reading and critical thinking skills3.Low self-images4.Higher attrition rates5.Poor grades6.Low levels of academic and social integration

Problem With Relying On The Literature

The Golden Path To Effective Intervention: Attitude

Aiming for 100% Retention is an unrealistic and counterproductive goal. Not everyone should be a military officer.

Can You Imagine Someone’s

Photograph In This Box?

The Golden Path To Effective Intervention: Attitude

Take the offensive! We have nothing to be defensive about regarding retention. Our mission is to clear a path for others to follow.

The Golden Path To Effective Intervention: Empirical Evidence

Data Driven Decision Making

Goals In Developing the CPQ

1. Greatest Unmet Need: A better way of determining how students respond to the first 6 or 8 weeks of college

2. We need an instrument of great breadth that is applicable at many schools and many types of students.

3. The instrument must provide information pertinent to personnel working with individuals as well as groups of students.

4. The instrument must identify the factors that most strongly govern retention at a particular school or for a particular individual.

5. The instrument must not be too long.6. Data must be immediately available; we can’t wait.7. The instrument must have a mechanism for addressing

issues pertinent to a particular school.

Development of the CPQ-1: Procedure

1. Conducted an extensive review of the retention literature, at least 150 studies

2. Identified at least 85 variables (WOW) that had been associated with retention

3. Wrote items addressing these 85 variables4. Automated the administration and data recording

system5. Administered the questionnaire to more than 2000

students

Simplify, Simplify, Simplify: Why Thoreau Would Like Factor Analysis

1. Reduced 85 variables to 6 factors2. Scales tend to be more reliable than single items3. Findings reported in Journal of College Student

Retention

Reasons For Developing CPQ-2

1. Several CPQ-1 scales could benefit from additional items.

2. Version 1 did not assess financial pressures, the motivation to perform academic work, or the student’s belief that he or she was capable of performing successfully.

CPQ Components: Student Background Form

1. Demographic and Family: (e.g., sex, ethnicity, marital status, parent’s education)

2. Work and Financial Resources: (e.g., hours working, sources of income)

3. Reasons For Attending: (e.g., reputation, location, friends)

The Student Background Form consists of three types of items:

Principal Components Analysis: Participants and Procedure

1. Two thousand eight hundred and thirty-eight undergraduates from Angelo State University, Appalachian State University, Greenville Technical College, and Troy University-Montgomery served.

2. Students responded online, most taking less than 40 minutes to complete the 82-item questionnaire.

Results

1. Responses were converted to 5-point “favorability” scores, depending on whether the answer indicated something positive or negative about the student’s college experience. The solution produced ten factors with eigenvalues greater than 1.0.

2. A principal components analysis was performed on the favorability scores of the 82 items using a direct oblimin rotation.

3. All items with pattern coefficients of .40 or higher were retained for further analysis.

Results Just Keep On Coming

5. A second principal components analysis with a direct oblimin rotation was conducted on the resultant 54 items to ensure that the deletion of questions did not cause substantial changes in the pattern coefficients. 6. The findings of the two analyses were similar. Item deletion did not have a pronounced effect on the coefficients. 7. All correlations between components were less than .32. 8. Alpha levels were in excess of .70 for all factors.

CPQ Components: Student Experience Form

1. Institutional Commitment: Loyalty, intention to reenroll, confidence in school choice

2. Degree Commitment: Personal importance, students’ supportive network, value of degree

3. Academic Integration: Positive views of instruction, instructors, and own intellectual growth; awareness of connections between academics and career.

4. Social Integration: Sense of belonging, shared values, and similarity to others; positive involvement behaviors.

5. Collegiate Stress: Feelings of distress, pressure, and sacrifice.

CPQ Components: Student Experience Form

6. Academic Motivation: Interest and enjoyment in academic tasks; willingness to spend extra time.

7. Scholastic Conscientiousness: Timely performance of academic responsibilities.

8. Academic Efficacy: Confidence in academic skills and outcomes.

9. Financial Strain: Financial worries and difficulties; sense of disadvantage relative to others.

10. Advising Effectiveness: Positive views of advising and school communication processes.

Did Anyone Notice?

That The Ten Student Experience Scales Of The CPQ Assess Many Of The Variables

Composing Institutional Effectiveness

CPQ Components: Institution Specific Form

This form addresses questions not covered by the CPQ but are important to the Corps of Cadets.

1. How confident are you that the Corps of Cadets is right for you?

2. How do you feel about the Corps’ PT requirement?

3. How likely is it that you will participate in the Corps next semester?

4. How do you feel about the policy of requiring cadets to wear a military uniform while on campus?

CPQ Components: Institution Specific Form

5. How do you feel about mandatory quarters for beginning Cadets?

6. During your last year in high school, on the average how much time did you spend in exercise or sports?

7. How likely is it that you will complete eight semesters (or equivalent) of military science?

8. How do you feel about the residency policy where you must withdraw from the university for one academic year if you drop out of the Corps of Cadets?

How Well Does The Field Predict Retention?ACT Meta-Analysis

13 academic and non-Academic variables explain 17% of the variability of college retention across students.

Validation Study:Participants and Procedure

1. Seven hundred and forty-nine freshmen from three comprehensive universities responded online to the CPQ.

2. CPQ was administered six to eight weeks into the first semester of the freshman year.3. Most students took less than 40 minutes to

complete the questionnaire. 4. Return for next Fall semester was the criterion

variable.

Does the CPQ Predict Retention? Identifying At-Risk Students (Individual Items)

Measures Nagelkerke R2

HSR + SAT + Ethnicity +

Sex + Age + Class Size (df = 6).08

CPQ – Background Variables, not usually in student database (df = 12)

.18

Student Experience Items (df = 54)

.56

Using The CPQ To Understand An At-Risk Group: Individual Items

Compared to other freshmen Student Support Services students reported that:1. They had a bigger influence on such matters as course offerings, rule and regulations, and registration procedures. 2. They were more satisfied with the academic advisement they received.3. They more strongly preferred tasks that make extensive use of their own ideas and allowed expression of their individuality.4. Fewer of their close friends were at Appalachian State.5. It was more difficult for their families to handle the cost of college.

CPQ Items Predicting Institutional Commitment: Community College Data Argues For Targeting

Item Business Nursing Arts & Sciences

1 Degree Commitment Degree Commitment Academic Integration

2 Degree Commitment Degree Commitment Academic Integration

3 Degree Commitment Degree Commitment Support Services

4 Academic Integration Academic Integration Acad. Conscientiousness

5 Academic Integration Academic Integration Support Services

6 Academic Integration Academic Integration Social Integration

The Importance Of Student Experiences: Participants and Procedure

1. 701 freshmen from three comprehensive universities. 2. CPQ was administered six to eight weeks into the first

semester of the freshman year.3. Most students took less than 40 minutes to complete the

questionnaire.

How Well Does The CPQ Predict Retention?

Predictor Variables Nagelkerke R Square

Variables typically in student database (df=15).

.13

Variables from Background Form not typically in database (df=12)

.23

CPQ Student Experience Scales .38

Regress retention (return the following Fall semester) on the following predictor variables

Support Services Satisfaction

Institutional Commitment

.11

-.07

.30

.38

.24

.22

.05

.33.23

.19

.10

.14

Social Integration

Academic Integration

AcademicConscientiousness

DegreeCommitment Where We Are

Going

Summary and Implications

1. CPQ is a valid predictor of whether freshmen will return for their sophomore years.

2. CPQ identified why individual students dropped out.

Summary and Implications

3. CPQ identified those variables that best distinguished those students who would from those students who would not continue their educations at these three institutions.

4. This information would be very helpful to policy makers deciding what retention programs to develop.

What Retention Activities At Your School Do You Believe Have Been Successful? A Sample

Mentoring programs have been successful.

Small class sizes, instructors taking personal interest in students-increasing engagement. Low student to advisor ratios in many programs.

Reaching more students by e-mail rather than US mail.

Tutoring

The Student Solutions Center has shown some success.

Meeting 1:1 with students to discuss their method of study, getting them to set up study plans, teaching them to underline or otherwise ID key terms in test questions, eliminating wrong answers by crossing them out, dealing with test anxiety.

All efforts have had some impact, however, all could be improved.

The Summer Bridge Program.

Academic support in freshman dorms has been very beneficial for at-risk or students – We have increased the number of dorm activities to foster the growth of community and sense of home for the students.

What Retention Activities At Your School Do You Believe Have Been Successful? A Sample

Setting The Stage For Tomorrow

Let Me Invite You To List Three Retention Activities That You Believe Have Been Successful at NGCSU

College Persistence Questionnaire Identifies Factors Influencing Commitment Of Military Cadets

Goals Of This Investigation

1) Determine if the CPQ predicts commitment to the Corps2) Identify those factors that are most strongly associated

with commitment3) Offer guidelines for increasing the retention of Cadets at

NGCSU4) Identify those cadets most at-risk for discontinuing their

educations

CPQ Components

1. Student Background Form: Consists of three types of questions: Demographic and Family items (sex, ethnicity, marital status, parent’s education), Work and Financial Resources items (hours working, sources of income), and Reasons for Attending Higher Education items (reputation, location).

2. Student Experience Form: Ten psychometrically validated scales that assess students’ interactions with the academic and social environments.

More About CPQ Components

3. Ten Student Experience Form Scales: Institutional Commitment, Degree Commitment, Academic Integration, Collegiate Stress, Social Integration, Academic Motivation, Scholastic Conscientiousness, Academic Efficacy, Financial Strain, and Advising Effectiveness.

4. Institutional Specific Form: Additional questions pertaining to the Corps of Cadets.

Participants and Procedure

1. 582 Cadets from NGCSU responded in groups online to the CPQ.

2. CPQ was administered six to eight weeks into the first semester of the freshman year.

3. Most students took less than 40 minutes to complete the questionnaire.

Commitment To Corps And NGCSU Overlap

1. Created Commitment To Corps scale from Institution specific items.

2. Correlation of Commitment to Corps and Institutional Commitment scale was .71.

3. Created single scale, Commitment Scale (CS), from corps and institutional items.

4. Cronbach’s alpha of CS was .78.

Does The CPQ Predict Commitment To The Corps?

Concurrent Validity: Regress CS scores on items from the Student Background and Student Experience Forms as well as three items from the Institution Specific Form, excluding items composing the CS scale.

F(68, 513) = 6.75, p < .001, R = .69, R2 = .47

Student Background Form

Cadets who chose NGCSU because of the following factors were more committed than cadets not endorsing these attributes.

1.Academics2.Overall reputation3.Appealing location

Student Background Form

Although these correlations were not strong, they may: 1. Aid in recruiting and identifying the types of prospective students who are likely to stick.2. Indicate that cadets with certain background characteristics may need additional services once they matriculate.

Student Experience Form

1. Eight of nine Student Experience scales were statistically significant, p < .01, two-tailed.

2. The most robust correlations with CS scores were the Academic Integration, Degree Commitment, and Social Integration scales. This result provides a guide to those seeking to augment commitment to the Corps. Improvements in the qualities underlying these scales will be particularly effective in enhancing commitment.

3. A complementary approach is to develop an intervention for a variable that is “networked” with other variables (e.g., advising).

Support Services Satisfaction

Institutional Commitment

.11

-.07

.30

.38

.24

.22

.05

.33.23

.19

.10

.14

Social Integration

Academic Integration

AcademicConscientiousness

DegreeCommitment Where We Are

Going

Student Background Form, Student Experience Form, Institution Specific Form: Comparisons

1. Student Background Form: Questions entered in isolation accounted for 16% of the variance.

2. Student Experience Form: Questions entered in isolation accounted for 43% of the variance.

3. Institution Specific Items: Questions assessing attitudes about requiring cadets to wear their uniform on campus, their views on PT, and whether they exercised in high school accounted for 15% of the variance when entered in isolation.

Two Important Takeaways

1. The Student Experience Form was a much better predictor of commitment than any items on the Student Background Form. This finding indicates that experiences during the first six to eight weeks have a profound impact on commitment.

2. It also argues against attempting to determine commitment during orientation or at an earlier date.

Limitations And Recommendations

1. This study needs to be followed by an analogous investigation in which retention (return for the sophomore year) is the outcome measure.

2. The sample size was small for those regressions employing individual items as predictors. Nevertheless, given the strength of the findings, we can be confident of the reliability of the outcomes.

3. It is encouraging that Student Experience items tend to be better predictors of commitment than Student Background items. Student Experience items are more subject to change.

Limitations And Recommendations

4. Efforts to augment cadet retention should focus on increasing Academic Integration, Social Integration, and Degree Commitment.

5. An alternative strategy is to network these variables and commitment by improving Advising Effectiveness.

6. The CPQ was administered only once in the Fall semester. Many cadets drop out after the Spring semester. Administering the CPQ in the Spring might identify students who were now at-risk but were not at-risk in the Fall. Furthermore, changes in scales scores could prove diagnostic.

What Does This Tell You About These Cadets?

Intervention Strategies: Transforming Cadets Into Officers

Working With Individual Students: What Counselors, Advisors and Faculty Need To Know

1. Which cadets are most at-risk?2. What factor(s) are causing a particular cadet to leave the

Corps?3. We need scale scores and responses to individual items.

Two Measures of Retention

Institutional Commitment ScoreAdvantages: Provides a means of identifying at-risk students after 6 weeks of college. The Student Experience variables that are correlated with Institutional Commitment are subject to change.Disadvantage: Is not perfectly correlated with who actually returns

Returning SophomoresAdvantage: Indicates who actually returns.Disadvantage: It is too late to help at-risk students when this data becomes available.

Beck’s List of Invariably Violated Principles

1. The earlier the diagnosis the better the prognosis2. More information generally enhances the prediction

of at-risk students3. The more time you have to gather information the

more useful information you can acquire

What Do You Need To Do To Reduce Attrition? Not An Easy Task

1. Identify at-risk students within first 6 to 8 weeks of college

2. Determine why a student is at-risk

3. Craft a powerful message that addresses that student’s issues

4. Quickly alter the direction of that student’s life

A Tale Of Two Portals

The Student and Advisor Portals were developed because early users of the CPQ requested an online system.

Student Portal: Administers the CPQ to the student and saves the information to a database.

Advisor Portal: Retrieves information from the database and uses this information to identify at-risk students and indicate why that individual is at-risk.

Advisor Portal: Identifying At-Risk Students

The Advisor Portal provides school personnel with a tool for making one-on-one sessions with students helpful, effective, and fruitful in augmenting retention rates. It identifies individual students who are most at-risk and supplies an in-depth analysis of the problems each one is experiencing. Testimonials of advisors and counselors who have used it are uniformly and overwhelmingly positive in describing its value in their sessions with individual students.

Working With Groups of Students: What Policy-Makers and Instructors Of

Persistence-Oriented Courses Need To Know

1. What factor(s) distinguish students at their schools who will persist and will not persist in their educations?

2. Are our retention programs effective?

Meta Analysis: Intervention Effects On College Performance And Retention

1. Robbins, Oh, Le, and Button (2009)2. Examined the effects of interventions (academic skill,

self-management, socialization, and first-year experience) on academic performance and retention.

Results: Intervention Effects On College Performance And Retention

1. Academic skill training (e.g., study habits, learning strategies) had the strongest effects on academic performance. Self-management skill training (e.g., self-regulatory, emotional control) had a lesser effect.

2. Self-management training had the strongest effect on retention. Socialization interventions also showed a significant effect on retention.

3. First-year experience programs had “weak to negligible effects on academic performance and retention.”

Why Do First Year Courses Have Such Modest Effects On Retention?

Interpreting The Data

They may:1.Be ineffective.2.Have very broad goals which are difficult to measure.3.Be very diverse. Programs may very different goals.4. Have very little structure or guidance as to objectives. Instructors may simply do what they want to do and the institution may hope that this has some benefit.

What Works In Student Retention? Student And Institutional Characteristics

Item Mean

Level of student preparation for college-level work 3.90

Adequacy of personal financial resources 3.90

Student study skills 3.80

Amount of financial aid available to students 3.65

Level of student motivation to succeed 3.64

Level of student commitment to earning a degree 3.56

Level of job demands on students 3.52

Student low socio-economic status 3.49

Student educational aspirations and goals 3.36

Student personal coping skills 3.34

What Works In Student Retention? Retention Activities

Item Mean

Academic advising center 3.98

Increased number of academic advisors 3.98

Advising interventions with selected populations 3.93

Comprehensive learning assistance center/lab 3.92

Supplemental instruction 3.91

Programs for first-generation students 3.90

Required on-campus housing for freshmen 3.86

Reading center/lab 3.86

Tutoring 3.84

Summer bridge program 3.83

What Activities Differentiate Schools With High Versus Low Retention Rates?

Item Top Bottom

Increased number of academic advisors 43% 30%

Advising with selected populations 96% 82%

Supplemental instruction 75% 64%

Summer bridge program 75% 49%

Program for honors students 91% 73%

Integration advising and first-year programs 72% 48%

Training for non-faculty academic advisors 84% 67%

Faculty mentoring 74% 45%

Living/learning communities (residential) 87% 60%

Staff mentoring 57% 31%

What Are The Three Most Important Retention Activities?

Item % Selected

Freshman seminar/university 101 (credit) 24%

Supplemental instruction 16%

Tutoring 15%

Living/learning communities (residential) 14%

Advising with selected populations 14%

Placement in courses based on test scores 13%

Academic advising center 12%

Summer orientation 11%

Early warning system 10%

Which Attrition-Reducing Activities Should We Use?

That depends upon:1. What variables differentiate cadets who become military officers from those who dropout. 2. What the empirical literature indicates about the effectiveness of various attrition-reducing practices.3. What resources and personnel we can direct towards retention.

How Do We Know If Our Actions Are Hitting The Mark?

That Brings Us To Measurement Issues

Your Main Outcome Measures

1. Grades 2. Institutional Commitment

3. Sophomore Retention

Your Palette Of Measures

There must be a logical relationship between your objectives and your measures. Your available indices include:

MeasuresRetention Academic Integration

Graduation Rates Financial Strain

GPA Social Integration

Institutional Commitment Degree Commitment

High School Rank, Standardized Tests Collegiate Stress

Background (e.g., sex, parental education) Academic Motivation

Work and Finances (e.g., hours working) Scholastic Conscientiousness

Reasons To Attend (e.g., friends, location) Academic Efficacy

Institution Specific Items Advising Effectiveness

Activities With Multiple Objectives

Activities

Training for non-faculty academic advisors

Faculty mentoring

Staff mentoring

Freshman seminar/university 101 (credit)

Early warning system

Measures: These activities are very generally stated. Therefore, they should have very diverse effects. If successful, a broad range of indices will be impacted. The measures taken should depend upon the particular objectives of the activities.

Some Attributes, Activities And Their Measures

Activities Measures

Financial resources Financial Strain

Motivation to succeed Academic Integration, Academic Motivation, Scholastic Conscientiousness

Commitment to earning a degree

Degree Commitment, Academic Integration

Job demands Work and Finances Section, Financial Strain

Educational aspirations

Degree Commitment, Academic Integration

Coping skills Collegiate Stress, Scholastic Conscientiousness, Academic Efficacy

Preparation for college-level work

High School Rank, Standardized Tests, Multiple Student Experience Scales

Assessing At-Risk Or Special Populations

Activities

Students of low socio-economic status

Advising interventions with selected student populations

Programs for first-generation students

Programs for honors students

Programs for minority students

Measures: These activities attempt to address problems common to at-risk or special populations. Measures on the Student Background Form (e.g., parents’ education, ethnicity) and student database could help identify these groups. The goals of these programs vary considerably, therefore, the measures used to assess their effectiveness varies accordingly.

Academic Enhancement Activities

Activities

Student study skills

Comprehensive learning assistance center/lab

Supplemental instruction

Reading center/lab

Tutoring

Placement of students in courses based on test scores

Program for honors students

Measures: Academic Integration, Academic Efficacy, Academic Motivation, Scholastic Conscientiousness, Collegiate Stress.

Social Integration Activities

Activities

Required on-campus housing for freshmen

Living/learning communities (residential)

Program for international students

Summer bridge program

Integration of advising with first-year transition programs

Summer orientation programs

Measures: A major objective of these activities appears to be to enhance the extent that students identify with and feel part of the academic community. Other objectives of these programs should be measured, in addition to the Social Integration scale of the CPQ.

Advising Activities

Activities

Academic advising center

Increased number of academic advisors

Advising interventions with selected student populations

Measures: Advising is critical because it networks with so many other variables. The Advising Effectiveness scale of the CPQ should be used to assess these endeavors. Other variables, depending on the goals of the program should also be measured..

What Is A Reasonable Assessment Strategy?

1. Gather attrition-related information from your Institutional Research Office.

2. Administer the CPQ to obtain Student Background and Experience variables.

3. Combine the Institutional Research and CPQ variables into a dataset.

4. Determine who returns the following Fall semester.5. Annually prepare Institutional Commitment and

Retention reports.6. Examine data across years to anticipate trends.

For Your Attention. Do Not Be A Stranger. This Is A beginning,

Not An Ending!