New SERU Graduate Student Survey

October 9, 2014Gregg Thomson

(SERU PR)Daniel WhiteOlena HornerRon HuesmanTiffany Thayer

(University of Minnesota) SERU Consortium

2014 SERU GSS Workgroup 2014 SERU GSS Workgroup MembersChair Ron Huesman – Managing Director, SERU-AAU Consortium, UMN*

Melissa Anderson – UMN Graduate School*

Igor Chirikov – Higher School of Economics, Moscow

Ken Doxsee – Oregon

Louis Myers – Virginia

Gregg Thomson – SERU PR/CSHE*

Daniel White – UMN*

Staff Support Olena Horner – UMN Research Assistant*

Shelva Hurley – CSHE SERU Research Associate*

Workgroup Consultants/Advisors:Tom Dohm – UMN

Anne Maclachlan – CSHE

Maresi Nerad – University of Washington* Denotes Planning Team responsible for developing a preliminary draft for the Workgroup

Graduate SERU Workgroup Goals

− Primary:

a. Done: Developed the core of the survey

b. Done: Populations, survey design (e.g., census, across academic levels),

c. Identified central data elements for exchange

a. Seed file (Done)

b. Supplemental file (tbd)

d. Fall 2014 SERU pilot: launches late October, 2014 - 6 weeks– University of Minnesota: PhD and MA students

– University of Virginia: PhD only

• Winter 2015 International pilot:– International: Higher School of Economics (Moscow)

– Unicamp(Brazil) Winter

RATIONALE & NEED

Challenges Faced by Graduate Education in the U.S.

• Ph.D. Attrition− 6.6% leave in their first year

− 30.6% leave without a degree by year ten

• Ph.D. Time to Degree− 56.6% complete their degree within ten years

− 7.7 years – the national median time to degree in 2008

• International Competition− 33,000 - S&E doctoral degrees awarded in the U.S.

− 28,000 - S&E doctoral degrees awarded in China

− 15,000 - S&E doctoral degrees awarded in Russia

− 11,000 – S&E doctoral degrees awarded in Germany

Note: data on doctoral degrees are presented for 2007-2008

Source: Council of Graduate Schools (2008), National Science Foundation (2009, 2012)

Survey Rationale

A survey can provide vital information about graduate/professional student experiences while enrolled that may help improve graduate education outcomes (i.e., increased graduation rates, faster time to degree, and help students find successful careers).

• Existing Surveys− External exit surveys: Survey of Earned Doctorates &

Survey of Doctorate Recipients (NSF)− Institutional exit surveys: encouraged to use Graduate

Education Exit Survey (AAUDE) core as a component− Locally developed institutional student experience

surveys: (UC-Berkeley, the University of Texas at Austin, the University of Michigan, Rutgers)

• Benefits of the New Survey− a joint effort between AAU and SER-I− the survey will focus on graduate student experiences

while enrolled in school− the survey will provide data for cross-institutional and

international comparisons

The Need for the New Graduate Student Survey

THE PROCESS

An Opportunity

GradSERU

Administration:Demand for high quality information about the

graduate student experience at UMN.

Graduate Students:Desire for resources to

help with the analysis of student developed survey of graduate experience.

SERU Consortium:2012 an opportunity for a survey of the enrolled Graduate student

experience survey in the Research University and desire

for multi-institutional data

Consensus Building: Identifying Stakeholders and Expertise

Graduate School

Council of Graduate Students

Graduate and Professional

Student Association

Office of Institutional

Research

Office of Measurement

Services

Academic Health Center

Law School

Office of Public Engagement

Boynton Health Center

Center for Teaching and

Learning

Office of Student Affairs

Graduate Review and Improvement

Process

Carousel Brainstorming Session• What opportunities would an enterprise-wide graduate and

professional student survey create for the University and individual units?

• What challenges would an enterprise-wide graduate and professional student survey create for the University and individual units?

• What benefits would result from an enterprise-wide graduate and professional student survey for the University and individual units?

• What concerns or fears do you have in regard to an enterprise-wide graduate and professional student survey for the University and individual units?

• What impact would an enterprise-wide graduate and professional student survey have on the University and individual units?

Consensus WorkshopIf we were to create an effective enterprise-wide graduate and professional student survey, what

information would need to be collected about students and their experiences?

Outcomes Expectation Fulfillment

Finances Development Opportunities

Getting to Started

1. What is out there?• Survey of Earned Doctorates (NSF).• Survey of Doctorate Recipients (NSF/NIH).• PhD Completion Project (CGS)• Pew Trusts Survey on Doctorate Educ.• Ivy League Graduate Student Survey.• Assessment of Research Doct. Prog. (NRC).• Graduate Education Survey (AAUDE).

2. What is missing?

Developing the Survey: A Parallel Process

UMN Workgroup collaboration on initial draft.

Open comment period for stakeholders to raise questions/issues.

Dozen one-hour interviews with stakeholders.

Revisions.

SERU/CSHE collaboration on initial draft.

Open comment period for stakeholders to raise questions/issues.

Teleconference review of the draft.

Revisions.

THE INSTRUMENT

Review of Current Research Literature on Student Experience

Graduate SERU Survey

Conceptual Framework

Tinto’s theory of graduate

persistence (1993)

Astin’s model of student development in

higher education (1970) Research literature on key

individual- and institution-related factors that impact

on student success (degree completion & time

to degree) in graduate school

Graduate SERU Survey Conceptual Framework

Conceptual framework: provides a basis/blue-print for survey development; mapping of final items to each domain will provide an item rationale document and a useful guide for conducting research with the survey in the future.

INPUT PROCESS OUTCOMES

Attributes Entry Orientations Student Experience

Student Background/

Current Status • Demographics• Parents’

educational attainment

• Current program• Type of degree• Stage in the

program

Previous Higher Education

Experience• Degree, field,

institution• Time elapsed

since completing a previous degree

Financial Resources

• Ability to pay• Undergraduate/

graduate loans

Admissions & Entrance

• Goals• Reasons for

selecting a program

• Program communica-tions

External Commitments

• Work• Family

Financial Support

• Sources• Degree of

support

Socialization into the Profession &

Professional Development

• Professional values and ethical issues

• Opportunities offered by the program/

college/school/ Graduate School

Other Institutional Support

• Quality of amenities at the institution

Program Climate• Satisfaction with

the program climate

Proficiency Levels

• Development of academic, research, and professional skills

Advising & Mentoring

• Quality of advising• Areas of support

from an advisor• Areas of support

from a mentor

Dissertation/Thesis Stage

• Dissertation topic• Dissertation research

process

Graduate/Professional

Degree Completion

• Commitment to complete a degree in the current program

Financial Support• Satisfaction with

financial support• Impact of

employment outside of the university on degree progress

• Anticipated consequences of debt burden

Research Experience• Experience with

research-related activities

• Sources of financial support

• Interdisciplinary research and its challenges

Teaching Experience

• Training• Teaching

experience• Impact on degree

progress

Overall Satisfaction

• Satisfaction with various aspects of a program

• Fit between students’ values, expectations and the program

• Choose the same field, program, university

Outcomes

Obstacles to Degree Progress

• Factors that hinder degree progress

Career Plans & Changes

• Changes in career plans during grad/prof studies

• Careers toward which a degree program is oriented

• Factors influencing career choices

Note. Derived from Tinto (1993) and Astin (1970).

Graduate SERU SurveyStructure

The survey covers 16 areas of student experience:• A. Your Graduate/Professional Program• B. Previous Higher Education Experience• C. Admissions and Entrance• D. Socialization into the Profession and Professional Development• E. Financial Support• F. Other Institutional Support• G. Program Climate• H. Proficiency Levels• I. Advising and Mentoring• J. Dissertation/Thesis Stage (for doctoral students only)• K. Research Experience• L. Teaching Experience• M. Career Plans & Changes• N. Obstacles to Degree Progress• O. Overall Satisfaction• P. For International Students Only

Graduate SERU SurveyStructure

Core Survey• Factual questions: 42• Opinion questions: 37• Demographics: 12

International Students only• Factual questions: 5• Opinion questions: 4

Examples of Survey QuestionsE. FINANCIAL SUPPORTTo what extent do you agree or disagree with: “Having a job outside the university

while going to schoola. helped me with career preparation

b. helped me secure a job

c. helped me advance my career

d. restricted my choice of classes

e. limited the number of classes I could take

f. slowed my degree progress

g. limited my access to campus facilities or services”?

Scale: Strongly disagree, Disagree, Agree, Strongly agree, Not applicable

 

When you complete your degree, how much do you think your debt burden will

affect:h. the type of job you will seek

i. the part of the country/world in which you will live

j. your family planning

k. your life style?

Scale: Not at all, Little, Some, A great deal

Current Version

• Graduate SERU

– CORE within academic level (Ph.D. and not) and across all post-baccalaureate students

• Items tailored to the academic level (similar items with term-specific references to the level or completely different items aimed at all students at a specific academic level)

Future

• Pilot test of the survey– Reviews (staff and students)

− Additional “think-aloud” interviews or verbal probing

– Post collection: Item Response Theory analysis of items; examples of competency items

Revise and focus survey: Winter/spring of 2015

SERU consortium: Vision

• Graduate SERU and Undergraduate SERU, (and their associated modules) become part of a suite of surveys available to members for administration.

• Costs? TBD information from pilot will be useful in estimating actual costs

• SERU institutional reps, same?

Examples of Survey QuestionsPROFICIENCY LEVELS (v1)To what extent have your skills in the following areas developed during your graduate/professional program?a. Analytical and critical thinking skills

b. Understanding your specific field of study

c. Ability to speak effectively in your field

d. Ability to write effectively in your filed

e. Ability to design and conduct original research

f. Ability to write grant proposals

g. Ability to work collaboratively on a project

h. Ability to work internationally

i. Ability to collaborate across disciplines

j. Ability to critically analyze the research literature (or performance products) from your

field

k. Having transferable skills for various career opportunities

l. Adherence to high standards of ethics and professional responsibility

m. Ability to teach in your field

n. Ability to innovate, be entrepreneurial

o. Other (please specify): ___________________

Scale: Not at all, Not very much, To some extent, To a great extent, Not applicable

Examples of Survey QuestionsPROFICIENCY LEVELS (v2)Please rate your level of proficiency in the following areas when you started your graduate/professional program and now. Currently When you started

a. Analytical and critical thinking skills

b. Understanding your specific field of study

c. Ability to speak effectively in your field

d. Ability to write effectively in your filed

e. Ability to design and conduct original research

f. Ability to write grant proposals

g. Ability to work collaboratively on a project

h. Ability to work internationally

i. Ability to collaborate across disciplines

j. Ability to critically analyze the research literature (or performance products) from your field

k. Having transferable skills for various career opportunities

l. Adherence to high standards of ethics and professional responsibility

m. Ability to teach in your field

n. Ability to innovate, be entrepreneurial

o. Other (please specify): ___________________

Scale: Poor, Fair, Good Excellent, Not applicable

Examples of Survey QuestionsADVISING AND MENTORING

To what extent do you agree or disagree with the following statements concerning your current, primary advisor?

a. My advisor has expertise in the area I’m studying.

b. My advisor knows how to effectively help me.

c. My advisor provides me with information I need to help me think about my career.

d. My advisor has time for me when I need help or advice.

e. My advisor helps me get financial support.

f. My advisor assists me in writing for presentations/publications.

g. My advisor advises me about teaching.

h. My advisor teaches me the details of good research practice.

i. My advisor respects me as an individual.

j. My advisor considers my personal abilities, talents, and interests when advising me.

k. My advisor is interested in having students help with his/her research.

l. My advisor prompts me to seek out opportunities that I would not have otherwise considered.

Scale: Strongly disagree, Disagree, Agree, Strongly agree, Not applicable

Examples of Survey QuestionsOBSTACLES TO DEGREE PROGRESS

Rate the extent to which the following factors have been an obstacle to your degree progress?

a. Difficult coursework and academic requirements

b. Diminished interest in the field of study

c. Family responsibilities or obligations

d. Inadequate advising

e. Inadequate financial support

f. Need to work

g. Nonsupportive or unfriendly environment for students like me

h. Physical or emotional health problems

i. Poor or uncertain employment prospects after degree completion

j. Immigration issues

k. Other (please specify): ______________________________

Scale: Not an obstacle, A minor obstacle, A major obstacle, Not applicable

Questions

Contact– Ron Huesman huesm003@umn.edu – Daniel Jones-White djwhite@umn.edu – Olena Horner glus0005@umn.edu

References

Astin, A. W. (1970). The methodology of research on college impact, part one. Sociology of Education, 43(3), 223-254. Retrieved

from http://www.jstor.org/stable/2112065

Abedi, J., & Benkin, E. (1987). The effects of students’ academic, financial, and demographic variables on time to the doctorate.

Research in Higher Education, 27(1), 3-14.

Ampaw, F. D., & Jaeger, A. J. (2012). Completing the three stages of doctoral education: An event history analysis. Research in

Higher Education, 53, 640-660. doi: 10.1007/s11162-011-9250-3

Andrieu, S. C., & St. John, E. P. (1993). The influence of prices on graduate student persistence. Research in Higher Education,

34(4), 399-425.

Baird, L. L. (1990). Disciplines and doctorates: The relationships between program characteristics and the duration of doctoral

study. Research in Higher Education, 31(4), 369-385.

Baird, L. L. (1993). Using research and theoretical models of graduate student progress. New Directions for Institutional

Research, 80, 3-12.

Bowen, W. G., & Rudenstine, N. L. (1992). In pursuit of the Ph.D. Princeton, NJ: Princeton University Press.

Bowen, W. G., Lord, G., & Sosa, J. A. (1991). Measuring time to the doctorate: Reinterpretation of the evidence. Proceedings of

the National Academy of Sciences, 88(3), 713–717.

Council of Graduate Schools. (2008). Ph.D. Completion and attrition: Analysis of baseline program data from the Ph.D.

Completion Project. Washington, D.C.: Council of Graduate Schools.

de Valero, J. F. (2001). Departmental factors affecting time-to-degree and completion rates at one land-grant research

institution. The Journal of Higher Education, 72(3), 341-367.

Ehrenberg, R. G., & Mavros, P. G. (1995). Do doctoral students’ financial support patterns affect their times-to-degree and

completion rates? The Journal of Human Resources, 30(3), 581-609.

Ethington, C., & Pisani, A. (1993). The RA and TA experience: Impediments and benefits to graduate study. Research in Higher

Education, 34(3), 343-354.

Gardner, S. K. (2009). The development of doctoral students: Phases of challenge and support. ASHE Higher Education Report,

34(6), 1-127. doi: 10.1002/aehe.3406

Gillingham, L., Seneca, J. J., & Taussig, M. K. (1991). The determinants of progress to the doctoral degree. Research in Higher

Education, 32(4), 449-468.

Girves, J. E., & Wemmerus, V. (1988). Developing models of graduate student degree progress. The Journal of Higher Education,

59(2), 163-189.

Golde, C. M. (2000). Should I stay or should I go? Student descriptions of the doctoral attrition process. The Review of Higher

Education, 23(2), 199-227.

Golde, C. M. (2005). The role of the department and discipline in doctoral student attrition: Lessons from four departments.

The Journal of Higher Education, 76(6), 669-700.

Groen, J. A. (2012). Time to the doctorate and labor demand for new PhD recipients. U.S. Bureau of Labor Statistics. Retrieved

from http://www.ilr.cornell.edu/cheri/upload/cheri_wp146.pdf

Groen, J. A., Jakubson, G. H., Ehrenberg, R. G., Condie, S., & Liu, A. Y. (2008). Program design and student outcomes in

graduate education. Economics of Education Review, 27(2), 111–124.

Haldaway, E., Deblois, C., & Winchester, I. (1995). Supervision of graduate students. The Canadian Journal of Higher Education,

XXV(3), 1-29.

Jiranek, V. (2010). Potential predictors of timely completion among dissertation research students at an Australian faculty of

sciences. International Journal of Doctoral Studies, 5, 1-13.

Nerad, M., & Miller, D. S. (1996). Increasing student retention in graduate and professional programs. New Directions for

Institutional Research, 92, 61-76.

Ott, M. D., Markewich, T. S., & Ochsner, N. L. (1984). Logit analysis of graduate student retention. Research in Higher Education,

21(4), 439-460.

Park, C. (2005). War of attrition: patterns of non-completion amongst postgraduate research students. Higher Education

Review, 38(1), 48-53.

Pascarella, E .T., & Terenzini, P. T. (2005). How College Affects Students: A Third Decade of Research. San Francisco: Jossey-Bass.

Picciano, J., Rudd, E., Morrison, E., & Nerad, M. (2008). Does time-to-degree matter? CIRGE Spotlight on Doctoral Education #3.

CIRGE: University of Washington, Seattle, WA. Retrieved from www.cirge.washington.edu

Seagram, B. C., Gould, J., & Pyke, S. W. (1998). An investigation of gender and other variables on time to completion of doctoral

degrees. Research in Higher Education, 39(3), 319-335.

Smith, R. L., Maroney, K., Nelson, K. W., Abel, A. L., & Abel, H. S. (Spring, 2006). Doctoral programs: Changing high rates of

attrition. Journal of Humanistic Counseling, Education and Development, 45, 17-31.

Stricker, L. J. (1994). Institutional factors in time to the doctorate. Research in Higher Education, 35(5), 569-587.

Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. Chicago and London: The University of

Chicago Press.

Wendler, C., Bridgeman, B., Cline, F., Millett, C., Rock, J., & McAllister, P. (2010). The Path forward: The future of graduate

education in the United States. Princeton, NJ: Educational Testing Service.

Wilson, S. B., Mason, T. W., & Ewing, M. J. M. (1997). Evaluating the impact of receiving university-based counseling services on

student retention. Journal of Counseling Psychology, 44(3), 316-320.