New SERU Graduate Student Survey
October 9, 2014Gregg Thomson
(SERU PR)Daniel WhiteOlena HornerRon HuesmanTiffany Thayer
(University of Minnesota) SERU Consortium
2014 SERU GSS Workgroup 2014 SERU GSS Workgroup MembersChair Ron Huesman – Managing Director, SERU-AAU Consortium, UMN*
Melissa Anderson – UMN Graduate School*
Igor Chirikov – Higher School of Economics, Moscow
Ken Doxsee – Oregon
Louis Myers – Virginia
Gregg Thomson – SERU PR/CSHE*
Daniel White – UMN*
Staff Support Olena Horner – UMN Research Assistant*
Shelva Hurley – CSHE SERU Research Associate*
Workgroup Consultants/Advisors:Tom Dohm – UMN
Anne Maclachlan – CSHE
Maresi Nerad – University of Washington* Denotes Planning Team responsible for developing a preliminary draft for the Workgroup
Graduate SERU Workgroup Goals
− Primary:
a. Done: Developed the core of the survey
b. Done: Populations, survey design (e.g., census, across academic levels),
c. Identified central data elements for exchange
a. Seed file (Done)
b. Supplemental file (tbd)
d. Fall 2014 SERU pilot: launches late October, 2014 - 6 weeks– University of Minnesota: PhD and MA students
– University of Virginia: PhD only
• Winter 2015 International pilot:– International: Higher School of Economics (Moscow)
– Unicamp(Brazil) Winter
RATIONALE & NEED
Challenges Faced by Graduate Education in the U.S.
• Ph.D. Attrition− 6.6% leave in their first year
− 30.6% leave without a degree by year ten
• Ph.D. Time to Degree− 56.6% complete their degree within ten years
− 7.7 years – the national median time to degree in 2008
• International Competition− 33,000 - S&E doctoral degrees awarded in the U.S.
− 28,000 - S&E doctoral degrees awarded in China
− 15,000 - S&E doctoral degrees awarded in Russia
− 11,000 – S&E doctoral degrees awarded in Germany
Note: data on doctoral degrees are presented for 2007-2008
Source: Council of Graduate Schools (2008), National Science Foundation (2009, 2012)
Survey Rationale
A survey can provide vital information about graduate/professional student experiences while enrolled that may help improve graduate education outcomes (i.e., increased graduation rates, faster time to degree, and help students find successful careers).
• Existing Surveys− External exit surveys: Survey of Earned Doctorates &
Survey of Doctorate Recipients (NSF)− Institutional exit surveys: encouraged to use Graduate
Education Exit Survey (AAUDE) core as a component− Locally developed institutional student experience
surveys: (UC-Berkeley, the University of Texas at Austin, the University of Michigan, Rutgers)
• Benefits of the New Survey− a joint effort between AAU and SER-I− the survey will focus on graduate student experiences
while enrolled in school− the survey will provide data for cross-institutional and
international comparisons
The Need for the New Graduate Student Survey
THE PROCESS
An Opportunity
GradSERU
Administration:Demand for high quality information about the
graduate student experience at UMN.
Graduate Students:Desire for resources to
help with the analysis of student developed survey of graduate experience.
SERU Consortium:2012 an opportunity for a survey of the enrolled Graduate student
experience survey in the Research University and desire
for multi-institutional data
Consensus Building: Identifying Stakeholders and Expertise
Graduate School
Council of Graduate Students
Graduate and Professional
Student Association
Office of Institutional
Research
Office of Measurement
Services
Academic Health Center
Law School
Office of Public Engagement
Boynton Health Center
Center for Teaching and
Learning
Office of Student Affairs
Graduate Review and Improvement
Process
Carousel Brainstorming Session• What opportunities would an enterprise-wide graduate and
professional student survey create for the University and individual units?
• What challenges would an enterprise-wide graduate and professional student survey create for the University and individual units?
• What benefits would result from an enterprise-wide graduate and professional student survey for the University and individual units?
• What concerns or fears do you have in regard to an enterprise-wide graduate and professional student survey for the University and individual units?
• What impact would an enterprise-wide graduate and professional student survey have on the University and individual units?
Consensus WorkshopIf we were to create an effective enterprise-wide graduate and professional student survey, what
information would need to be collected about students and their experiences?
Outcomes Expectation Fulfillment
Finances Development Opportunities
Getting to Started
1. What is out there?• Survey of Earned Doctorates (NSF).• Survey of Doctorate Recipients (NSF/NIH).• PhD Completion Project (CGS)• Pew Trusts Survey on Doctorate Educ.• Ivy League Graduate Student Survey.• Assessment of Research Doct. Prog. (NRC).• Graduate Education Survey (AAUDE).
2. What is missing?
Developing the Survey: A Parallel Process
UMN Workgroup collaboration on initial draft.
Open comment period for stakeholders to raise questions/issues.
Dozen one-hour interviews with stakeholders.
Revisions.
SERU/CSHE collaboration on initial draft.
Open comment period for stakeholders to raise questions/issues.
Teleconference review of the draft.
Revisions.
THE INSTRUMENT
Review of Current Research Literature on Student Experience
Graduate SERU Survey
Conceptual Framework
Tinto’s theory of graduate
persistence (1993)
Astin’s model of student development in
higher education (1970) Research literature on key
individual- and institution-related factors that impact
on student success (degree completion & time
to degree) in graduate school
Graduate SERU Survey Conceptual Framework
Conceptual framework: provides a basis/blue-print for survey development; mapping of final items to each domain will provide an item rationale document and a useful guide for conducting research with the survey in the future.
INPUT PROCESS OUTCOMES
Attributes Entry Orientations Student Experience
Student Background/
Current Status • Demographics• Parents’
educational attainment
• Current program• Type of degree• Stage in the
program
Previous Higher Education
Experience• Degree, field,
institution• Time elapsed
since completing a previous degree
Financial Resources
• Ability to pay• Undergraduate/
graduate loans
Admissions & Entrance
• Goals• Reasons for
selecting a program
• Program communica-tions
External Commitments
• Work• Family
Financial Support
• Sources• Degree of
support
Socialization into the Profession &
Professional Development
• Professional values and ethical issues
• Opportunities offered by the program/
college/school/ Graduate School
Other Institutional Support
• Quality of amenities at the institution
Program Climate• Satisfaction with
the program climate
Proficiency Levels
• Development of academic, research, and professional skills
Advising & Mentoring
• Quality of advising• Areas of support
from an advisor• Areas of support
from a mentor
Dissertation/Thesis Stage
• Dissertation topic• Dissertation research
process
Graduate/Professional
Degree Completion
• Commitment to complete a degree in the current program
Financial Support• Satisfaction with
financial support• Impact of
employment outside of the university on degree progress
• Anticipated consequences of debt burden
Research Experience• Experience with
research-related activities
• Sources of financial support
• Interdisciplinary research and its challenges
Teaching Experience
• Training• Teaching
experience• Impact on degree
progress
Overall Satisfaction
• Satisfaction with various aspects of a program
• Fit between students’ values, expectations and the program
• Choose the same field, program, university
Outcomes
Obstacles to Degree Progress
• Factors that hinder degree progress
Career Plans & Changes
• Changes in career plans during grad/prof studies
• Careers toward which a degree program is oriented
• Factors influencing career choices
Note. Derived from Tinto (1993) and Astin (1970).
Graduate SERU SurveyStructure
The survey covers 16 areas of student experience:• A. Your Graduate/Professional Program• B. Previous Higher Education Experience• C. Admissions and Entrance• D. Socialization into the Profession and Professional Development• E. Financial Support• F. Other Institutional Support• G. Program Climate• H. Proficiency Levels• I. Advising and Mentoring• J. Dissertation/Thesis Stage (for doctoral students only)• K. Research Experience• L. Teaching Experience• M. Career Plans & Changes• N. Obstacles to Degree Progress• O. Overall Satisfaction• P. For International Students Only
Graduate SERU SurveyStructure
Core Survey• Factual questions: 42• Opinion questions: 37• Demographics: 12
International Students only• Factual questions: 5• Opinion questions: 4
Examples of Survey QuestionsE. FINANCIAL SUPPORTTo what extent do you agree or disagree with: “Having a job outside the university
while going to schoola. helped me with career preparation
b. helped me secure a job
c. helped me advance my career
d. restricted my choice of classes
e. limited the number of classes I could take
f. slowed my degree progress
g. limited my access to campus facilities or services”?
Scale: Strongly disagree, Disagree, Agree, Strongly agree, Not applicable
When you complete your degree, how much do you think your debt burden will
affect:h. the type of job you will seek
i. the part of the country/world in which you will live
j. your family planning
k. your life style?
Scale: Not at all, Little, Some, A great deal
Current Version
• Graduate SERU
– CORE within academic level (Ph.D. and not) and across all post-baccalaureate students
• Items tailored to the academic level (similar items with term-specific references to the level or completely different items aimed at all students at a specific academic level)
Future
• Pilot test of the survey– Reviews (staff and students)
− Additional “think-aloud” interviews or verbal probing
– Post collection: Item Response Theory analysis of items; examples of competency items
Revise and focus survey: Winter/spring of 2015
SERU consortium: Vision
• Graduate SERU and Undergraduate SERU, (and their associated modules) become part of a suite of surveys available to members for administration.
• Costs? TBD information from pilot will be useful in estimating actual costs
• SERU institutional reps, same?
Examples of Survey QuestionsPROFICIENCY LEVELS (v1)To what extent have your skills in the following areas developed during your graduate/professional program?a. Analytical and critical thinking skills
b. Understanding your specific field of study
c. Ability to speak effectively in your field
d. Ability to write effectively in your filed
e. Ability to design and conduct original research
f. Ability to write grant proposals
g. Ability to work collaboratively on a project
h. Ability to work internationally
i. Ability to collaborate across disciplines
j. Ability to critically analyze the research literature (or performance products) from your
field
k. Having transferable skills for various career opportunities
l. Adherence to high standards of ethics and professional responsibility
m. Ability to teach in your field
n. Ability to innovate, be entrepreneurial
o. Other (please specify): ___________________
Scale: Not at all, Not very much, To some extent, To a great extent, Not applicable
Examples of Survey QuestionsPROFICIENCY LEVELS (v2)Please rate your level of proficiency in the following areas when you started your graduate/professional program and now. Currently When you started
a. Analytical and critical thinking skills
b. Understanding your specific field of study
c. Ability to speak effectively in your field
d. Ability to write effectively in your filed
e. Ability to design and conduct original research
f. Ability to write grant proposals
g. Ability to work collaboratively on a project
h. Ability to work internationally
i. Ability to collaborate across disciplines
j. Ability to critically analyze the research literature (or performance products) from your field
k. Having transferable skills for various career opportunities
l. Adherence to high standards of ethics and professional responsibility
m. Ability to teach in your field
n. Ability to innovate, be entrepreneurial
o. Other (please specify): ___________________
Scale: Poor, Fair, Good Excellent, Not applicable
Examples of Survey QuestionsADVISING AND MENTORING
To what extent do you agree or disagree with the following statements concerning your current, primary advisor?
a. My advisor has expertise in the area I’m studying.
b. My advisor knows how to effectively help me.
c. My advisor provides me with information I need to help me think about my career.
d. My advisor has time for me when I need help or advice.
e. My advisor helps me get financial support.
f. My advisor assists me in writing for presentations/publications.
g. My advisor advises me about teaching.
h. My advisor teaches me the details of good research practice.
i. My advisor respects me as an individual.
j. My advisor considers my personal abilities, talents, and interests when advising me.
k. My advisor is interested in having students help with his/her research.
l. My advisor prompts me to seek out opportunities that I would not have otherwise considered.
Scale: Strongly disagree, Disagree, Agree, Strongly agree, Not applicable
Examples of Survey QuestionsOBSTACLES TO DEGREE PROGRESS
Rate the extent to which the following factors have been an obstacle to your degree progress?
a. Difficult coursework and academic requirements
b. Diminished interest in the field of study
c. Family responsibilities or obligations
d. Inadequate advising
e. Inadequate financial support
f. Need to work
g. Nonsupportive or unfriendly environment for students like me
h. Physical or emotional health problems
i. Poor or uncertain employment prospects after degree completion
j. Immigration issues
k. Other (please specify): ______________________________
Scale: Not an obstacle, A minor obstacle, A major obstacle, Not applicable
Questions
Contact– Ron Huesman [email protected] – Daniel Jones-White [email protected] – Olena Horner [email protected]
References
Astin, A. W. (1970). The methodology of research on college impact, part one. Sociology of Education, 43(3), 223-254. Retrieved
from http://www.jstor.org/stable/2112065
Abedi, J., & Benkin, E. (1987). The effects of students’ academic, financial, and demographic variables on time to the doctorate.
Research in Higher Education, 27(1), 3-14.
Ampaw, F. D., & Jaeger, A. J. (2012). Completing the three stages of doctoral education: An event history analysis. Research in
Higher Education, 53, 640-660. doi: 10.1007/s11162-011-9250-3
Andrieu, S. C., & St. John, E. P. (1993). The influence of prices on graduate student persistence. Research in Higher Education,
34(4), 399-425.
Baird, L. L. (1990). Disciplines and doctorates: The relationships between program characteristics and the duration of doctoral
study. Research in Higher Education, 31(4), 369-385.
Baird, L. L. (1993). Using research and theoretical models of graduate student progress. New Directions for Institutional
Research, 80, 3-12.
Bowen, W. G., & Rudenstine, N. L. (1992). In pursuit of the Ph.D. Princeton, NJ: Princeton University Press.
Bowen, W. G., Lord, G., & Sosa, J. A. (1991). Measuring time to the doctorate: Reinterpretation of the evidence. Proceedings of
the National Academy of Sciences, 88(3), 713–717.
Council of Graduate Schools. (2008). Ph.D. Completion and attrition: Analysis of baseline program data from the Ph.D.
Completion Project. Washington, D.C.: Council of Graduate Schools.
de Valero, J. F. (2001). Departmental factors affecting time-to-degree and completion rates at one land-grant research
institution. The Journal of Higher Education, 72(3), 341-367.
Ehrenberg, R. G., & Mavros, P. G. (1995). Do doctoral students’ financial support patterns affect their times-to-degree and
completion rates? The Journal of Human Resources, 30(3), 581-609.
Ethington, C., & Pisani, A. (1993). The RA and TA experience: Impediments and benefits to graduate study. Research in Higher
Education, 34(3), 343-354.
Gardner, S. K. (2009). The development of doctoral students: Phases of challenge and support. ASHE Higher Education Report,
34(6), 1-127. doi: 10.1002/aehe.3406
Gillingham, L., Seneca, J. J., & Taussig, M. K. (1991). The determinants of progress to the doctoral degree. Research in Higher
Education, 32(4), 449-468.
Girves, J. E., & Wemmerus, V. (1988). Developing models of graduate student degree progress. The Journal of Higher Education,
59(2), 163-189.
Golde, C. M. (2000). Should I stay or should I go? Student descriptions of the doctoral attrition process. The Review of Higher
Education, 23(2), 199-227.
Golde, C. M. (2005). The role of the department and discipline in doctoral student attrition: Lessons from four departments.
The Journal of Higher Education, 76(6), 669-700.
Groen, J. A. (2012). Time to the doctorate and labor demand for new PhD recipients. U.S. Bureau of Labor Statistics. Retrieved
from http://www.ilr.cornell.edu/cheri/upload/cheri_wp146.pdf
Groen, J. A., Jakubson, G. H., Ehrenberg, R. G., Condie, S., & Liu, A. Y. (2008). Program design and student outcomes in
graduate education. Economics of Education Review, 27(2), 111–124.
Haldaway, E., Deblois, C., & Winchester, I. (1995). Supervision of graduate students. The Canadian Journal of Higher Education,
XXV(3), 1-29.
Jiranek, V. (2010). Potential predictors of timely completion among dissertation research students at an Australian faculty of
sciences. International Journal of Doctoral Studies, 5, 1-13.
Nerad, M., & Miller, D. S. (1996). Increasing student retention in graduate and professional programs. New Directions for
Institutional Research, 92, 61-76.
Ott, M. D., Markewich, T. S., & Ochsner, N. L. (1984). Logit analysis of graduate student retention. Research in Higher Education,
21(4), 439-460.
Park, C. (2005). War of attrition: patterns of non-completion amongst postgraduate research students. Higher Education
Review, 38(1), 48-53.
Pascarella, E .T., & Terenzini, P. T. (2005). How College Affects Students: A Third Decade of Research. San Francisco: Jossey-Bass.
Picciano, J., Rudd, E., Morrison, E., & Nerad, M. (2008). Does time-to-degree matter? CIRGE Spotlight on Doctoral Education #3.
CIRGE: University of Washington, Seattle, WA. Retrieved from www.cirge.washington.edu
Seagram, B. C., Gould, J., & Pyke, S. W. (1998). An investigation of gender and other variables on time to completion of doctoral
degrees. Research in Higher Education, 39(3), 319-335.
Smith, R. L., Maroney, K., Nelson, K. W., Abel, A. L., & Abel, H. S. (Spring, 2006). Doctoral programs: Changing high rates of
attrition. Journal of Humanistic Counseling, Education and Development, 45, 17-31.
Stricker, L. J. (1994). Institutional factors in time to the doctorate. Research in Higher Education, 35(5), 569-587.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition. Chicago and London: The University of
Chicago Press.
Wendler, C., Bridgeman, B., Cline, F., Millett, C., Rock, J., & McAllister, P. (2010). The Path forward: The future of graduate
education in the United States. Princeton, NJ: Educational Testing Service.
Wilson, S. B., Mason, T. W., & Ewing, M. J. M. (1997). Evaluating the impact of receiving university-based counseling services on
student retention. Journal of Counseling Psychology, 44(3), 316-320.