Annual Report - California State University, Stanislaus...of university-wide projects for administrative, academic, and system-wide purposes. Before joining CSU Stanislaus, Dr. Sanchez

ANNUAL REPORT

OFFICE OF

INSTITUTIONAL RESEARCH 2011-12 ANNUAL REPORT

PREPARED BY:

ANGEL A. SANCHEZ, DIRECTOR

KELLI PAYNE, ADMINISTRATIVE ANALYST/ SPECIALIST

WITH CONTRIBUTIONS FROM:

LISA MEDINA, RESEARCH TECHNICIAN

LISA SAUCEDO, RESEARCH ANALYST

This annual report highlights the activities and accomplishments of the Office of Institutional Research within the 2011-12 college year, followed by an analysis of internal activities required to establish, maintain, and guide the office as well as to support leadership of the university.

TABLE OF CONTENTS

Introduction ......................................................................................................................................................................... 1

Mission and Vision ......................................................................................................................................................... 1

The IR Team ................................................................................................................................................................... 1

Past Year Activity: College Year 2011-12 ....................................................................................................................... 5

Client Feedback .............................................................................................................................................................. 7

Workload Trend Analysis: College Year 2009-10 to 2011-12 ..................................................................................... 9

Future Goals and Objectives: College Year 2012-13 .................................................................................................. 12

Continue to prioritize and adjust goals................................................................................................................. 12

Expand professional development opportunities for IR team members ....................................................... 13

Continue efforts in collaboration .......................................................................................................................... 13

Raise the profile of IR ............................................................................................................................................. 14

Concluding Remarks ........................................................................................................................................................ 14

Appendices ......................................................................................................................................................................... 15

Appendix A. Colleges, Academic Departments, Committees, and other Academic and Administrative Units

Served During 2011-12. ............................................................................................................................................... 15

Appendix B. Office of Institutional Research Website .......................................................................................... 16

Appendix C. Survey Research and Assessment Projects Supported by IR: 2009-10 to 2011-12 .................... 17

Office of Institutional Research 8/2/2012 1 2011-12 IR Annual Report

INTRODUCTION

During 2011-12, the California State University, Stanislaus Office of Institutional Research (IR) continued on new directions and initiatives in fulfillment of its mission to serve the university community. This annual report highlights the activities and accomplishments of IR within the past 2011-12 college year, followed by an analysis of internal activities required to establish, maintain, and guide the office as well as to support university leadership; a final section outlines potential new projects and looks at priorities, goals, challenges, and opportunities for the future. We should note that many of the activities and accomplishments reported below for IR were, in fact, undertaken collaboratively, involving staff in various departments and divisions across campus.

Mission and Vision

IR is dedicated to supporting the student-centered teaching and learning mission of the university. IR provides client-centered services and plays a vital role in the provisioning of information for planning, assessment of institutional effectiveness, and data driven decision-making. The vision of IR is to provide direct and value-added analytical support to the administration and the academic community in areas of program assessment, accreditation, academic program review, and strategic planning, including the design and development of surveys, primary data collection, and data analysis.

The IR Team

The current configuration of IR began in July 2008 under the leadership of the current director. Since that time, the office has introduced a client-centered service model to support administrative and academic affairs’ needs. The success of IR this past year was accomplished through the dedication of a strong and committed team. As in prior years, 2011-12 witnessed increased demand on IR for information analysis, survey design and administration, assessment and evaluation, strategic planning, and report writing. The IR team worked diligently to meet the demand. Below is the current configuration of IR, followed by brief summaries of each team member’s duties and responsibilities, work experience, and educational background:

Angel A. Sanchez, Director: Currently in his fourth year as the Director of IR, Dr. Angel Sanchez leads a professional team of analysts with expertise in survey design, assessment, planning, consulting, and data analytics. Dr. Sanchez is also assistant professor of Sociology and serves as co-leader of graduate education assessment for the Title V Commitment to Excellence in Graduate Education (CEGE) project and a member of the leadership team for the Title V HSI STEM CVMSA Project (i.e., Central Valley Math and Science Alliance). Dr. Sanchez brings extensive experience and knowledge of social research methods, strategic

Director

Angel A. Sanchez, Ph.D

Research Analyst

Lisa Saucedo

Research Analyst

Ling Zhao

Research Technician

Lisa Baysinger

Title V HSI STEM Grant Research Analyst

Jungwon Huh

Administrative Analyst/Specialist

Kelli Payne

Student Assistant (s)


planning, assessment, and data analysis, and leads IR in providing extensive support services to a wide-range of university-wide projects for administrative, academic, and system-wide purposes. Before joining CSU Stanislaus, Dr. Sanchez served as co-director of the Division of Analytic Studies at the California State University Office of the Chancellor, where he was responsible for managing professional staff on compiling system-wide student data from all campuses of the California State University; for publishing and disseminating statistical information in Statistical Reports and the Statistical Abstract; for responding to California or Federal reporting requirements; for providing research analysis and information reports in support of the academic policies enacted by the CSU Board of Trustees; and for providing information related to budget allocations and fee revenues. Dr. Sanchez also served as Chief Administrative Officer of the Division of Information Resources and Technology Services at the Kern County Office of Education, serving 42 school districts. Dr. Sanchez also served as Administrative Research Coordinator at the Los Angeles County Office of Education, serving 82 school districts. Dr. Sanchez was also Research Manager at the National Center for Bilingual Research. Throughout his professional career Dr. Sanchez has demonstrated strong visionary and leadership skills in service to education. While serving as chief administrative officer, he was appointed by the California Department of Education and State Allocation Board, to serve as project administrator of the statewide Building the Future Project as well as the subsequent Project GoldenNet—technology implementation projects designed to help inform policy and bridge the digital divide in K-12 schools. He was active and successful in building strategic alliances with Silicon Valley Technology CEOs in support of statewide efforts. Subsequently, Dr. Sanchez was invited by the Governor’s Office to serve as a member of the Governor’s K-12 technology advisory committee to help inform policy and strategic planning for schools. Dr. Sanchez earned his bachelor’s degree in Sociology from CSU Stanislaus, and his doctorate degree in Sociology at the University of California, Los Angeles. He teaches upper division courses in research methods and research analysis. His research interests include student engagement and assessment, education and society, and college persistence. Lisa Saucedo, Research Analyst: Ms. Lisa Saucedo has worked in IR for over 10 years in various positions with increasing responsibility; initially as a student research assistant and currently as a Research Analyst. Ms. Saucedo’s duties include collecting, analyzing, and reporting quantitative and qualitative data to inform campus decision-making and support planning in areas such as admissions, curriculum, enrollment management, student life, athletics, and alumni relations. She also responds to ad hoc data requests from departments, administrators, and committees on a wide-range of topics including academic performance, outcomes assessment, university and department accreditation, self-studies, grant reporting, survey development, and various student enrollment and persistence reports. Ms. Saucedo is also responsible for routine reporting to external constituents, including the CSU Office of the Chancellor, government bodies (i.e., National Center for Education Statistic’s Integrated Postsecondary Education Data System, National Collegiate Athletic Association) to the public (i.e., Voluntary System of Accountability’s College Portrait, Common Data Set), various college guide publishers (i.e., U.S. News and World Report, Princeton Review), and regional and national accreditation organizations. She performs these tasks utilizing her thorough knowledge of reporting requirements, statistics, research methods, and computer-based reporting tools (i.e., IBM SPSS Statistics, Microsoft Excel, Hyperion Designer, PeopleSoft Query, and Qualtrics Research Suite). Ms. Saucedo is an alumnus of CSU Stanislaus, having received her bachelor’s degree in Organizational Communications in December 2002. She will continue her graduate studies at CSU Stanislaus in fall 2012 in the Public Administration master’s degree program. Ling Zhao, Research Analyst: Ms. Ling Zhao has served as a Research Analyst since January 2004 and, bringing with her the knowledge of SAS software programming that the unit had been lacking for some time. Ms. Zhao’s primary responsibilities include extracting historical and current student data from the university’s


various databases (i.e., Brio Reports, Hyperion Designer, and PeopleSoft), and extensively utilizing SAS to merge data files and perform data analyses. Ms. Zhao routinely updates large-scale datasets including retention and graduation rates and grade distribution for longitudinal analyses and various research studies. She is also responsible for updating the Campus Data Portfolio, a comprehensive set of data tables that fulfill a multitude of reporting requirements, and are critical in the academic program review process. These tables display student admission, enrollment, retention, and graduation trends, including demographic information, at the university, college, and program levels. Ms. Zhao also is responsible for providing data to comply with various reporting requirements, developing and maintaining CSU Stanislaus IR SAS software programs and complex data files, and preparing ad hoc reports and tables for analysis and decision-making. Prior to working at CSU Stanislaus, Ms. Zhao worked as a Major Data Analyst in the Office of Institutional Research at Mississippi State University for two years and served as Research Associate in the Microbiology Institute at the Chinese Academy of Science for ten years. Ms. Zhao holds a bachelor’s degree in Chemistry and a master’s degree in Statistics. Lisa Medina, Research Technician: Ms. Lisa Medina has served as a Research Technician in IR at CSU Stanislaus since January 2009. Ms. Medina is responsible for the coordination and submission of Academic Planning Database (APDB) term reports to the CSU Office of the Chancellor. As part of the IR Survey Research Team, other duties include a joint responsibility for the management and on-campus support of the university-wide contract for Qualtrics Survey Research Suite, a web-based survey and enterprise feedback tool to conduct qualitative and quantitative survey research, as well as supporting the design, development and administration of web-based survey; providing support to program and unit reviews as well as program assessment activities. In addition, Ms. Medina provides Qualtrics training support and consultation to a wide variety of campus clients. Ms. Medina also provides analytic support using IBM SPSS Statistics and IBM SPSS Text Analytics for Surveys. Key support is also provided in completing ad-hoc data requests regarding faulty workload, student/faculty ratio, course subject analysis and numerous other requests primarily using APDB data and PeopleSoft. Ms. Medina previously worked for the City of Modesto, Finance Department where she was responsible for the collection, disbursement and accounting of all city monies received in accordance with Generally Accepted Accounting Principles as well as a variety of other accounting and budget duties. Ms. Medina earned a bachelor’s degree in Statistics from CSU California Polytechnic State University, San Luis Obispo. She also earned a master’s degree in Business Administration from CSU Stanislaus. Jungwon Huh, Research Analyst: Ms. Jungwon Huh has served as a Research Analyst for IR at CSU Stanislaus since May 2012. Ms. Huh’s primary responsibility is to support the data analytics and reporting needs of the Science, Technology, Engineering and Mathematics (STEM) Articulation and Transfer Project. She will also serve as the primary resource in providing analytical support, including coordination of data collection for various reporting tasks as may be required by the STEM project team. Ms. Huh earned a bachelor’s degree from Seoul National University and a master’s degree in Social Work from the University of Wisconsin at Madison, and is a doctoral candidate in Social Work at the University of North Carolina at Chapel Hill. Kelli Payne, Administrative Analyst/Specialist: Ms. Kelli Payne has served as Administrative Analyst/Specialist for IR at CSU Stanislaus since 2007. In addition to her primary duties that include a full range of administrative in support of IR functions such as budgeting, accounting, and purchasing, her other duties include coordinating and organizing efforts and activities involved in the research, review, analysis,


interpretation, and reporting of a variety of data and information used in assessing institutional effectiveness; assisting with strategic planning, accreditation, grant support, decision-making, program reviews, the assessment of student learning outcomes, and determining implications of university practices, policies, measures and procedures as well as campus-wide client relations for IR. Other responsibilities include the use of a variety of software (e.g., IBM SPSS Statistics, IBM SPSS Text Analytics for Surveys) to analyze statistical, demographic, and other educational data in order to draw conclusions, develop proposals, and test research hypotheses. Ms. Payne also supports and creates presentations, institutional reports and other documents used for all aspects of strategic planning and program review that are associated with statistical reports for a variety of nontechnical and technical audiences; prepares complex reports for broad dissemination using a variety of desk-top publishing applications (e.g., MS Word, MS Publisher, MS Excel); synthesizes data into various report formats, including summaries of statistical studies, narrative reports including tables, graphs, and charts. Ms. Payne is also responsible for the design, development and maintenance of the IR website that supports a variety of key institutional research and planning activities and serves the needs of diverse audiences. As the other half of the IR Survey Research Team, Ms. Payne is also jointly responsible for management and on-campus support of the university-wide contract for Qualtrics Survey Research Suite including training and consultation with various members of the university community in the selection, design, and validation of survey tools and other qualitative research instruments and methodologies. Ms. Payne’s research and development passion began as an undergraduate student working as an Interviewer and Data Collection staff for the Public Research Institute at San Francisco State University where she was responsible for assisting in social science policy research and data collection to service SF State University as well as the Bay Area and California governmental agencies, non-profit organizations, community groups, and businesses. She continued with her enthusiasm for research by working as a Staff Research Associate in Neuropsychology in the Department of Psychology at SF State where she aided in several clinical trials and administered various human psychophysiological measures of workers and residents who may have incurred occupations toxic exposures; as a Market Researcher and Data Analyst for Better World Advertising, a social marketing firm in San Francisco, CA where she assisted with needs assessment, formative research, and impact monitoring and evaluation; and as a Staff Research Associate for the University of California, San Francisco where she was responsible for gathering, monitoring, and analyzing critical data for clinical research studies in areas such as psychopharmacology, as well as human physiology and behavior. Prior to working at CSU Stanislaus, Ms. Payne also served as an Administrative Assistant of Communications and Missions Advancement for a local non-profit organization where she was key leadership of mission advancement, fundraising events, and plans for growth. Ms. Payne earned a bachelor’s degree in Psychology at San Francisco State University. She then went on to complete master’s coursework in Human Cognitive Psychophysiology and worked as a graduate researcher in the SF State Cognitive Psychophysiology Lab. Michelle Machado, Undergraduate GIS Analyst: As part of a collaborative effort between IR and the CSU Stanislaus Geography Department, Ms. Michelle Machado joined the IR team as a Geographic Information System (GIS) Student Intern, in October 2011. The purpose of this internship was to offer the opportunity to strengthen a student’s knowledge and skill in research methods, data analysis, and academic writing as well as to utilize GIS applications (i.e., ArcGIS) to visually display institutional data. Ms. Machado is a third-year undergraduate student, double-majoring in Geography and Sociology. Her concentrations within Geography are Geospatial Technologies, Cultural/Social and Globalization and Development and her concentration within Sociology is Human Services. Her relevant coursework includes: GEOG 4750-Geographic Information Systems, GEOG 3700 Cartography and Visualization, as well as many other geography and sociology courses. Ms. Machado’s primary duties included the development of data maps by integrating GIS data with other information systems and linking to institutional data (e.g., enrollment files) for the purpose of visually


exploring data and geographic patterns or correlations within the educational pipeline. Using this information, Ms. Machado created and presented a poster titled Using GIS to Visually Understand the CSU Stanislaus Pipeline at the California Geographical Society’s Annual Conference, UC Davis (2012, April).

PAST YEAR ACTIVITY: COLLEGE YEAR 2011-12

IR executed a demanding workload during 2011-12. Members of the IR team were involved with many different projects as well as provided support and services to the various colleges and divisions within the university. See Appendix A for a full list of organizational units served during the 2011-12 college year. Services performed include a variety of outcomes assessment projects, consultation on research design, survey development and administrations, preparation of decision support materials, retention and graduation rate studies, and program evaluations. In addition to addressing many ad-hoc requests for institutional data, IR also provided support to faculty and administrators in the preparation of several major grant proposals.

During 2011-12, IR completed a variety of internal reports and data summaries for use by governance

committees and administration as well as responded to ad-hoc requests for information from both within and

outside the university. This past year IR received more than 100 ad-hoc requests for information, covering

such topics as student success, diversity, and enrollment trends. Some examples of ad-hoc requests submitted

to IR include:

▪ Prepared data summaries in support of the Graduation Rate Initiative Team as part of the effort to improve graduation rates at CSU Stanislaus;

▪ Provided the Office of the Provost with institutional data in support of evidence-based assessment and data driven decision-making;

▪ Supported doctoral student research by providing data summaries and reports to Ed.D students;

▪ Provided and disseminated large-scale datasets from internal and external data sources in support of the Holistic Academic Program Review Committee.

IR also worked with key campus offices on preparation of mandated reports for the California State

University Office of the Chancellor and other external agencies. Data on various topics are reported,

including general institutional characteristics, admissions, financial aid, enrollment, graduation, retention, and

faculty & staff demographics. Listed below are some examples for 2011-12:

▪ Assisted in providing CSU Chancellor’s Office with periodic Full-Time Equivalent Student (FTES) target reports in order to comply with monitoring system-wide target projections of FTES enrollment for budget purposes;

▪ Collaborated with Enrollment Services to provide enrollment targets to the CSU Chancellor’s Office via the Enrollment Workbook;

▪ Submitted census faculty workload and course load information from the Academic Planning Database (APDB) to the CSU Chancellor’s Office.1 Also, secured pilot phase of APDB master files from the Chancellor’s Office, saving considerable programming work for IR;

▪ Provided and/or verified institutional data for federally-mandated reporting requirements such as the National Center for Education Statistics (NCES) Integrated Postsecondary Education Data System (IPEDS), and submitted student-athlete graduation rates to the National Collegiate Athletic Association (NCAA);

1 On many levels IR works with the Chancellor’s Office throughout the year. The census files submitted each term which are the basis for analysis and production of tabular data for the university include: APDB, Enrollment Reporting System--Applications (ERSA), Enrollment Reporting System--Off-Campus Centers (ERSO) (state-supported off-campus [Stockton] student enrollment), and Enrollment Reporting System—Student (ERSS) (state-supported student enrollment). Data files that are submitted annually include, Enrollment Reporting System—Degrees (ERSD) (degrees conferred), Enrollment Reporting System—Teachers (ERST) (teaching credentials recommended), and sERSS (self-supported student enrollment).


▪ Supported the various colleges and overall university accreditation efforts by providing institutional data for several state and specialized accrediting agencies, and the university’s regional accrediting agency (WASC);

▪ Provided institutional data for several college guidebook publishers, including Princeton Review and U.S. News and World Report, resulting in increased recognition for the university.

An important function of IR is to provide readily available institutional data via the IR website for both

internal and external audiences. See Appendix B for a screenshot of the current IR website homepage. During

2011-12, IR restructured its website around seven main categories of data found on the left-navigation bar.

The main page provides a search function within the IR site as well as notices of any new data posted or

updated. The seven categories include: Admissions, Enrollment, New Students, Degrees Conferred,

Retention and Graduation Rates, Course Enrollments, and Faculty & Staff.

In partnership with many of the academic programs, student support service units, and governance

committees, IR continues to make excellent progress in supporting student learning assessment projects

across the various academic departments and programs. During 2011-12, IR developed, administered, and/or

managed more than 40 web-based assessment surveys for university clients. Principal accomplishments

include the following:

▪ Served as survey administrators for the Strategic Planning Working Group, the Retention, Promotion, and Tenure Committee, the Ad-hoc College Reorganization Committee, and the Athletics Department in support of institutional effectiveness, institutional planning, and policy formation;

▪ Collected evidence of student learning and program impact for the Doctoral Program in Education Leadership, the Department of Biological Sciences, and the School of Nursing, to name only a few;

▪ Collaborated with the Office of Information Technology (OIT) and the English Department to develop and launch a web-based Directed Self-Placement tool;

▪ Provided support for assessment by gathering student satisfaction data from current and former students for various student support services including but not limited to Housing and Residential Life, Library Services, Learning Services, and Psychological Counseling Services;

▪ Collected and analyzed evidence for program effectiveness and planning for University Extended Education.

Another assessment activity of IR is to conduct regular surveys - of students, faculty, staff, and alumni - and to communicate results of these surveys to various university constituencies. Most surveys are conducted on a regular basis - annually, bi-annually (i.e., fall and spring semester), or less frequently as appropriate. Some surveys (such as NSSE and FSSE) are externally prepared and compiled, while many are developed internally. Externally prepared surveys, while generic in character, offer the advantage of comparative benchmarks that, when properly used help to measure growth or change in the quality of the academic and student support services environment. During the normal course of its work, IR also executed the following student engagement studies and assessment projects university-wide:

▪ Graduating Senior Survey

▪ Graduate School Exit Survey

▪ Collegiate Learning Assessment (CLA)

During the past year IR also introduced and initiated a web-based survey research platform to support faculty research, scholarly, and creative activities (RSCA), as well as to support faculty sponsored doctoral and master’s thesis research. Below is a list of such survey projects for 2011-12.


▪ Trained and consulted with Social Work faculty and their masters students to develop a web-based survey that was distributed to various counties throughout California;

▪ Served as research and data analysis consultant for faculty in the Department of Teacher Education researching high school teachers and student learning;

▪ Trained and consulted with faculty from the Department of Sociology & Gerontology and Gender Studies to develop a web-based survey assessing social networking and student-faculty interaction at CSU Stanislaus;

▪ Trained faculty to utilize a web-based survey research tool in the classroom to enhance course curriculum.

IR provided data summaries and reports for grant proposals, annual grant reporting requirements and internal tracking as well as assisted with the preparation of the evaluation section of grant projects.

▪ Assisted with grant project evaluation planning and preparation for the McNair Scholarship Grant Proposal;

▪ Secured a Special Consultant contract with the Psychology Department to serve as a research analyst for the Title V Part B (Center for Excellence in Graduate Education) grant-funded project;

▪ Hired a research analyst for the Title V HSI STEM grant project.

In addition, IR’s time and attention was devoted to supporting several major initiatives:

▪ Initiated collaboration with faculty members from the College of Natural Sciences on supporting undergraduate student research activities through the development of student research project teams that focused on development of a Writing Proficiency Screening Test (WPST) database;

▪ Negotiated with the Department of Anthropology, Geography and Ethnic Studies for an undergraduate student intern to develop research skills in IR by introducing Geographic Information Systems (GIS) into the IR analytical environment;

▪ Negotiated university-wide site license for the Qualtrics Survey Research Suite;

▪ Initiated focus of support from OIT on SQL database solutions for the Teacher Credentialing FileMaker Pro database, the Title V Part A CEGE project Access database, and the WPST FileMaker Pro database;

▪ Met with CSU Long Beach, CSU Fullerton, and the CSU Chancellor’s Office for IR planning as well as to obtain information regarding campus initiatives, operations management and relationships relevant to enrollment services.

Client Feedback

IR is a vital component resource in the policy-practice feedback loop of the university, and thus a key anchor point for information flow and dissemination in support of informed decision-making processes. An ongoing task of IR is to solicit feedback through various one-to-one and group meetings with administrative and faculty leadership. The feedback is used in IR strategic planning efforts regarding the development and adaptation of client service needs for the university community. In addition to the one-to-one meetings, IR administered its annual Point of Service Survey in spring 2012, with the goal of gathering feedback from primary clients regarding their level of satisfaction with a full range of IR products and services. IR maintains internal records of departments and clients served throughout the year; in 2011-12 there were a total of 222 primary clients served. Results from the spring 2012 sample include a total of 106 responses from CSU Stanislaus faculty, staff, and administrators; 40 percent staff, 22 percent administrators, and 38 percent faculty members. The full survey report is available for download at the IR website: www.csustan.edu/IR. The responses from the IR Point-of-Service Survey appear to affirm IR’s contribution to the campus community; that IR is responsive, knowledgeable and experienced in serving clients’ needs. Data also

http://www.csustan.edu/IR


indicated a positive perception of IR’s client-centered services. The results provided IR with some excellent insights into what we are doing well, where we may improve, and which priority actions we may take during the coming academic year. Table 1 summarizes a portion of the findings from this past year’s survey administration, with regards to client ratings of IR customer service orientation. Table 1. Summary of Responses Regarding IR Professionalism and Customer Service Orientation

Please rate the following client service aspects of IR: Number of Responses

Excellent/Good Responses (%)

Ability of IR to meet my data needs 94 91.5%

Ability of staff to demonstrate creativity and initiative 81 93.8%

Ability to schedule a meeting within a reasonable time period 75 93.3%

Client service oriented (i.e., places a high value on customer service) 90 93.4%

Delivery of information or services in a timely manner (i.e., prompt) 93 94.6%

Dependability and accuracy of data 90 88.9%

Dissemination of information 83 87.9%

Friendliness, courteousness 99 97.0%

Helpful 100 95.0%

Knowledgeable staff 98 95.9%

Professionalism of IR staff 99 97.0%

Quality of information and services 95 94.7%

Usefulness of IR data 95 91.6%

Willingness to listen and understand client’s needs 96 94.8%

Overall quality of service 100 95.0% Note. Responses include spring 2012 sample of primary clients only.

IR often responds to complex and detailed ad-hoc requests for specific types of tabular data. In the survey,

clients were asked to respond to questions regarding the types of data they requested from IR. Below is a

sample of the most frequently requested types of data:

▪ Full-Time Equivalent Student (FTES) enrollment

▪ Graduation rates

▪ Student retention rates

IR also provides a variety of professional support services to clients; the responses from the survey revealed

that the most frequently requested types of services from IR include:

▪ Assessment

▪ Research consultation

▪ Survey development, design or administration

▪ Accreditation assistance


WORKLOAD TREND ANALYSIS: COLLEGE YEAR 2009-10 TO 2011-12

IR’s focus on client and university needs has resulted in greater use of institutional data for data driven

decision-making purposes. Although internal reporting of institutional data represent a continued year-to-year

increase in output to internal campus constituencies such as faculty and governance committees for

university-wide planning, decision-making, and program assessment, the following IR workload trend analysis

focuses primarily on the increasing demand for survey research services from IR with regards to the Qualtrics

Survey Research Suite; it highlights where such requests are primarily coming from within the university and

what type of services are being requested. For a full list of survey research and assessment projects over the

past three years, see Appendix C.

During spring and summer of 2009, IR strategic planning efforts resulted in the decision to implement a web-based survey tool, Qualtrics Survey Research Suite, as a solution for reducing cost and workload efforts of hard-copy, paper surveys. Qualtrics was first housed in IR as a departmental license with limited user accounts during 2009-10 and 2010-11; however, in 2011-12, the license for Qualtrics was re-negotiated into a university-wide site license with unlimited user accounts. Since the implementation of Qualtrics at CSU Stanislaus in 2009-10, it has proven to be a mission-critical

resource; IR has executed a wide-range of survey research and assessment projects and expanded its work in

activities related to strategic planning, program assessment, faculty RSCA, course curriculum development, as

well as supported data driven decision-making processes university-wide. Figure 1 displays the year-to-year

growth in survey research and

assessment projects supported by IR

beginning in 2009-10 through 2011-

12; the number of projects increased

from 18 to 50 by 2011-12,

representing an increase greater than

170 percent.

As shown in Figure 2, in each of the reporting years, survey research and assessment projects that occur

annually display the strongest increase followed by special or ad-hoc projects; these types of surveys have

nearly tripled since 2009-10.

0

5

10

15

20

25

AnnualBi-annual

Mid-YearOn-going

Special project

10

2

1 5

14

6

1 1

11

25

4

1 1

19

Fre

quen

cy C

oun

t

Figure 2. Year-to-Year Growth in Administration of Survey Research and Assessment Projects during 2009-10 to 2011-12

2009-10 2010-11 2011-12

0

20

40

60

2009-10 2010-11 2011-12

18

33

50

Fre

quen

cy C

oun

t

Figure 1. Year-to-Year Growth of Survey Research and Assessment Projects during 2009-10 to 2011-12


Survey research and assessment project service requests span the university campus divisions, from the

Provost & Academic Affairs, to Business & Finance, to Faculty Affairs & Human Resources, and to

Enrollment & Student Affairs. Table 2 displays the basic organizational division or unit served during the last

three years. Of note, although the colleges are positioned in the division of the Provost and Academic Affairs,

in order to better evaluate trends, the table displays results for the current six colleges separately from the

Provost and Academic Affairs division.

Since 2009-10, the range of clients has expanded across the university; in 2009-10, there were only seven areas

within the university that requested survey project services for IR; by 2011-12 the divisional areas or units

increased. Currently, only one of the six colleges has not requested survey project services from IR.

Table 2. Division, College and/or Unit of Survey Research and Assessment Project Services Provided during CY 2009-10 to CY 2011-12

College

Year Division, College and/or Unit

Number

of Projects %

2009-10

College of Education 5 27.8

College of Humanities & Social Science 1 5.6

College of Natural Sciences 1 5.6

Enrollment & Student Affairs 6 33.3

Faculty Affairs & Human Resources 1 5.6

Governance Committees 1 5.6

Provost & Academic Affairs 3 16.7

Total 18 100.0

2010-11

Business & Finance 1 3.0


College of Human & Health Sciences 4 12.1







Total 33 100.0

2011-12

Business & Finance 2 4.0

College of Business Administration 1 2.0


College of Human & Health Sciences 5 10.0







Total 50 100.0


IR offers three levels of service or involvement in client survey research and assessment projects. At Level 1-

Consultation and Support, IR acts as consultants on all aspects of a survey project by providing advice as

needed on project planning, data collection, analysis, and the like. IR also provides review and feedback on,

but does not directly produce the survey instrument.

At Level 2-Technical Assistance, IR acts as survey

administrators and managers by providing consultation on all

aspects of a project and, in addition, conducts the technical

aspects of survey research and assessment, such as developing

the survey instrument, collecting data, and producing a basic

statistical analysis for the project’s findings.

At Level 3-Research Partnership, IR acts as professional

partners on a survey research and assessment project and is

involved in every stage of a project, from the beginning by

assisting with defining the research question through data

collection to interpretation of data.

As shown in Figure 3, client survey research and assessment projects requiring technical assistance has

substantially increased from two projects in 2009-10 to 23 projects in 2011-12. Likewise, client projects

requiring consultation and support has grown from two projects in 2009-10 to 12 projects in 2011-12, also a

large increase over the past three years; while clients requesting IR as research partners has remained relatively

consistent. This increase can be attributed to the awareness of and demand for these types of services from

IR. The figure also suggests a desired result, that is, in the brief period shown, the distribution of service

activity is leveling out proportionately. It is a desired outcome of IR to see, for example, the increase in

competencies and knowledge of our clients to learn how to design and administer surveys via Qualtrics, and

therefore lessen the impact on IR staff’s “time-on-task” to perform all of the work for clients.2 However, in

spite of the increase placed on IR over the past three years with regards to requests for survey research

project services, IR has continued its efforts to meet the needs of its clients by providing direct and value-

added products and services.

2 While not measured in this client survey, anecdotally, there appears to be a reduction in the number of surveys among campus

faculty that use some other off-campus survey system such as Survey Monkey (often with associated costs) which may be used for free, but with limitations on instrument complexity and sample size. In future IR client surveys, IR will attempt to measure the cost-saving impact to the university departments by using Qualtrics, as well as the impact on capacity development and knowledge across faculty and staff in using one common web-based system.

0 5 10 15 20 25

Consultation and support

Technical assistance

Research partnership

Frequency Count

Lev

el o

f Ser

vic

e

Figure 3. Survey Research and Assessment Project Support Service Provided by IR during CY 2009-10 to CY 2011-12

2009-10

2010-11

2011-12

Level 1 - Consultation and Support

Level 2 - Technical Assistance

Level 3 - Research Partnership


Table 3 displays trends in regards to the general purpose of the survey project services requested from IR.

Most notable is the increase in projects for program assessment, and institutional strategic planning. Also

prominent is the increase in the range of needs, particularly the emerging presence of faculty RSCA, which

includes faculty sponsored research activities with students using Qualtrics.

Table 3. Purpose of Survey Research and Assessment Services during CY 2009-10 to CY 2011-12 College Year Purpose Number of Projects %

2009-10

Academic support services assessment 3 16.7

Institutional effectiveness 2 11.1

Institutional strategic planning 2 11.1

Program assessment 6 33.3

Student learning outcomes 1 5.6

Student services assessment 4 22.2

Total 18 100.0

2010-11


Faculty and student research and scholarship 4 12.1






Total 33 100.0

2011-12


Faculty professional development 3 6.0

Faculty and student research and scholarship 5 10.0



Placement and testing 1 2.0




Total 50 100.0

FUTURE GOALS AND OBJECTIVES: COLLEGE YEAR 2012-13

Future goals include the ongoing development of IR’s capacity to nurture the growth of a data driven

decision-making environment. This means improving operational efficiencies that directly improve office

productivity. An additional objective is for IR to establish a strong professional presence by elevating the

research and scholarly activities of the individual team members. Together these actions will enhance the

service-provisioning excellence of IR to the university community. The following is a brief outline of goals

and objectives for 2012-13.

CONTINUE TO PRIORITIZE AND ADJUST GOALS

▪ Identify priorities and goals for the upcoming 2012-13 college year during a series of strategic

planning and development meetings during summer 2012;

▪ Conduct business process mapping of IR functions to improve operations;

▪ Conduct annual administration of the Office of Institutional Research Point of Service Survey, a client

satisfaction survey in which results are used in annual IR strategic planning as well as in the IR

Annual Report submitted to the Provost;


▪ Participate in one-on-one meetings with college deans and department chairs and inventory

needs;

▪ Engage in periodic briefings to Deans Council and Academic Affairs Council on data and

reports of key interest for planning and decision-making;

▪ Share strategic planning ideas with key leaders regarding the development of IR services to support faculty RSCA;

▪ Update the IR Master Calendar of project activity, task scheduling, and reporting;

▪ Continue to improve delivery time of ad hoc data requests and accuracy of information collected and distributed;

▪ In collaboration with the appropriate faculty committee, explore the feasibility of developing a replacement solution for IDEA course evaluations using Qualtrics;

▪ Develop Qualtrics Survey Research Suite workshops for faculty members at the Faculty Development Center.

EXPAND PROFESSIONAL DEVELOPMENT OPPORTUNITIES FOR IR TEAM MEMBERS

▪ Engage IR Strategic Planning meeting at sister CSU Sacramento campus to propose and facilitate joint learning meetings with IR staff in order to build relationships and share in learning and professional development;

▪ Create training opportunities for team members to advance their analytic skills using IBM SPSS Statistics and Statistical Analysis Software (SAS);

▪ Engage IR staff to attend and/or perform professional presentations at the CSU Stanislaus

Faculty Development Center, the IR director’s meeting, including professional organizations

such the Association for Institutional Research (AIR), and California Association for

Institutional Research (CAIR);

▪ Establish IR’s representation on university committees such as the Institutional Review Board

(IRB), Dean’s Council, Assessment Leadership Team, etc. in order to cultivate ongoing

involvement with strategically important campus activities and to gain a robust view of the

university through multi-organizational involvement.

CONTINUE EFFORTS IN COLLABORATION

▪ Engage IR staff in collaborative work with faculty members or staff members on IR projects;

▪ IR will continue its close working relationship and collaboration with Enrollment Services on enrollment management and planning;

▪ Collaborate with the Department of Geography on use of ArcGeographic Information Systems

(ArcGIS) in IR;

▪ Develop a data collection tool used for the assessment, analysis, monitoring, and reporting of

university-wide research and scholarship activities with ORSP in collaboration with Library

Services;

▪ Continue work with the ORSP in development of grant funding opportunities, as well as in

development of cost-recovery or revenue through services opportunities for IR;

▪ Continue work with OIT on SQL solutions to control the integration of outlier database systems that are not integrated into PeopleSoft.


RAISE THE PROFILE OF IR

▪ Engage IR team members in authoring and co-authoring reports leading to publications;

▪ Introduce the publication of statistical reports based on data from the APDB system. The objective is to provide relevant information on faculty workload activities to Academic Affairs, deans, managers-analysts, and department chairs;

▪ Resume publication of IR Analysis Brief;

▪ Publish online Campus Data Portfolio, displaying academic year enrollment and trend tables;

▪ Publish online university and program retention and graduation rates;

▪ Continue posting university reports and data summary tables on the IR website.

CONCLUDING REMARKS

The IR staff is enthusiastic about the accomplishments of 2011-12 and looks forward to continuing efforts toward advancing both the university’s mission and the mission of the IR office. With the increasing demands for information for planning and data driven decision-making comes an increase in the breadth of tasks, skills, and functions placed on IR. IR staff members are encouraged by the progress made over the past year, and are equally excited about the future challenges and opportunities to serve the university during the coming year.


APPENDICES

Appendix A. Colleges, Academic Departments, Committees, and other Academic and Administrative Units Served During 2011-12.

Colleges ▪ College of the Arts ▪ College of Business Administration ▪ College of Education ▪ College of Human and Health Sciences ▪ College of Humanities and Social Sciences ▪ College of Natural Sciences

Academic Departments or Programs

▪ Accounting and Finance ▪ Advanced Studies (including Ed.D. program) ▪ Anthropology, Geography and Ethnic Studies ▪ Art ▪ Biological Sciences ▪ Chemistry ▪ Communication Studies ▪ Computer Information Systems ▪ Computer Science ▪ Criminal Justice ▪ Economics, Agricultural Studies, and Social

Sciences ▪ English ▪ History ▪ Honors Program ▪ Kinesiology ▪ Liberal Studies ▪ Management, Operations, and Marketing ▪ Mathematics ▪ MBA ▪ Music ▪ Nursing ▪ Philosophy and Modern Languages ▪ Physics, Physical Sciences, and Geology ▪ Political Science and Public Administration ▪ Psychology and Child Development ▪ Sociology and Gender Studies ▪ Social Work ▪ Teacher Education ▪ Theatre

Other Academic and Administrative units

▪ Advising Resource Center ▪ Associate Vice President, Academic

Affairs/Accreditation Liaison Officer ▪ Admissions and Records ▪ Athletics Department ▪ Auxiliary Business Services ▪ Business and Finance

▪ Communications and Public Affairs ▪ Disability Resource Services ▪ Early Assessment Program ▪ Enrollment and Student Affairs ▪ General Education ▪ Graduate Studies ▪ Faculty Center for Excellence in Teaching and

Learning ▪ Faculty Mentor Program ▪ Financial Aid and Scholarship ▪ Housing and Residential Life ▪ Human Resources and Faculty Affairs ▪ International Education ▪ Library Services ▪ Office of Information Technology Client Services ▪ Office of Information Technology Learning

Services ▪ Office of Information Technology Information

Services ▪ Office of the President ▪ Office of Research and Sponsored Programs ▪ Provost and Vice President, Academic Affairs ▪ Psychological Counseling Services ▪ Public Safety ▪ Retention Services ▪ Service Learning ▪ Student Leadership and Development ▪ Title V CEGE grant program ▪ Title V PACE grant program ▪ Title V HSI STEM grant program ▪ Tutoring Services ▪ University Advancement ▪ University Extended Education ▪ Writing Proficiency Screening Test (WPST) office

Committees

▪ Academic Affairs Council ▪ Academic Senate ▪ Assessment Leadership Team ▪ Council of Deans ▪ Early Start Program ▪ Graduation Rate Initiative Team ▪ Strategic Plan Working Group ▪ Student Success Committee ▪ Ad-hoc College Reorganization Committee ▪ Holistic Academic Program Review Committee ▪ Retention, Promotion, and Tenure Committee


Appendix B. Office of Institutional Research Website


Appendix C. Survey Research and Assessment Projects Supported by IR: 2009-10 to 2011-12

College Year 2009-10 ▪ Academic Senate Staff Representative

Election ▪ Client Evaluation of Psychological Counseling

Services ▪ College of Education Faculty Survey ▪ College of Education Student Survey ▪ Department of Biological Sciences Senior

Learning Goals Assessment Survey ▪ Department of Kinesiology Program

Assessment Evaluation ▪ Economic Impact Survey ▪ Economics Department Senior Seminar Exit

Survey ▪ Faculty Mentor Program Mentor Survey ▪ Faculty Mentor Program Protégé Survey ▪ Graduate School Exit Survey ▪ Graduating Senior Exit Survey ▪ Housing and Residential Life Year-End Non-

Returning Resident Exit Survey ▪ Housing and Residential Life Year-End

Satisfaction Survey ▪ Kinesiology Senior Focus Group Survey ▪ Liberal Studies Senior Survey ▪ Library Instruction Assessment Survey

(Student Research Skills) ▪ UEPC Proposed Spring Break Calendar

Survey College Year 2010-11

▪ Academic Senate Staff Representative Election

▪ Client Evaluation of Psychological Counseling Services

▪ College of Education Faculty Survey ▪ College of Education Student Survey ▪ Department of Biological Sciences Senior


Assessment Evaluation ▪ Economics Department Senior Seminar Exit



Satisfaction Survey ▪ Liberal Studies Senior Survey ▪ Library Instruction Assessment Survey

(Student Research Skills) ▪ Ad Hoc College Reorganization Survey -

Phase 1 ▪ Social Work Graduate Student Survey: Dating

Violence Survey

▪ Department of Biological Sciences New Student Assessment Survey

▪ Ed.D Student Perception Survey ▪ English Department End of Semester

Questionnaire (English 1000/1001) ▪ English Department Senior Exit Survey ▪ Faculty Survey of Student Engagement

(FSSE), Faculty ▪ High School Teaching and Learning ▪ Housing and Residential Life Mid-Semester

Resident Survey ▪ IR Point-of-Service Survey ▪ Library User Assessment Student Survey ▪ National Survey of Student Engagement

(NSSE), Freshmen ▪ National Survey of Student Engagement

(NSSE), Seniors ▪ Nursing Cell Phone Survey ▪ Social Work Graduate Student Survey:

Corporal Punishment ▪ Social Work Graduate Student Survey: Stress

and Graduate School Survey ▪ Student-Athlete Priority Registration Survey -

Phase 1 ▪ University Bookstore Survey

College Year 2011-12

▪ Academic Senate Staff Representative Election

▪ Client Evaluation of Psychological Counseling Services

▪ College of Education Faculty Survey ▪ College of Education Student Survey ▪ Department of Biological Sciences Senior


Assessment Evaluation ▪ Economics Department Senior Seminar Exit



Satisfaction Survey ▪ Liberal Studies Senior Survey ▪ Library Instruction Assessment Survey

(Student Research Skills) ▪ Department of Biological Sciences New

Student Assessment Survey ▪ Ed.D Student Perception Survey ▪ English Department End of Semester

Questionnaire (English 1000/1001) ▪ English Department Senior Exit Survey ▪ Faculty Survey of Student Engagement

(FSSE), Faculty


Appendix C. Survey Research and Assessment Projects Supported by IR: 2009-10 to 2011-12 (cont.)

College Year 2011-12 (cont.) ▪ High School Teaching and Learning ▪ Housing and Residential Life Mid-Semester

Resident Survey ▪ IR Point-of-Service Survey ▪ Ad Hoc College Reorganization Survey -

Phase 2 ▪ BlogTalkRadio Survey ▪ CLA Participant Survey ▪ College of Education Chair Election ▪ Corporal Punishment Survey for Child

Welfare Workers ▪ Doctoral Program in Educational Leadership

Exit Survey ▪ Faculty Survey on Student Response Systems ▪ First Year Experience Program Survey ▪ First-Year Composition (Stretch) Survey

(Directed Self-Placement Survey) ▪ Learning Services Workshops ▪ Lecture Capture Survey ▪ Library User Assessment Faculty Survey

▪ Math Department Physical Space Needs Survey

▪ Networks, Tweeps, and Friends - CSU Stanislaus Faculty Survey

▪ Networks, Tweeps, and Friends - CSU Stanislaus Student Survey

▪ Nursing ADN to BSN Senior Survey ▪ Nursing Pre-licensure Senior Survey ▪ Nursing Senior Survey Contact Sheet ▪ Political Science Student Exit Survey ▪ President's Executive Cabinet Staff

Representative Election ▪ Professional Writing Program Partners Survey ▪ QSEN Student Evaluation ▪ Retention, Promotion, and Tenure (RPT)

Survey ▪ Strategic Plan Survey ▪ Student-Athlete Priority Registration Survey -

Phase 2 ▪ University Extended Education Summer

Session Survey ▪ WASC SVRT Phase 1B


IR Office of Institutional Research

Mary Stuart Rogers Educational Services Gateway Building, MSR 250

One University Circle

Turlock, CA 95382

209.667.3281

209.664.7069

[email protected]

www.csustan.edu/ir

Angel A. Sanchez, Ph.D.

Director

[email protected]

Ms. Kelli Payne

Administrative Analyst/Specialist

[email protected]

Ms. Lisa Medina

Research Technician

[email protected]

Ms. Lisa Saucedo

Research Analyst

[email protected]

Ms. Ling Zhao

Research Analyst

[email protected]

Ms. Jungwon Huh

Title V HSI STEM Grant Research Analyst

[email protected]

mailto:[email protected]

http://www.csustan.edu/ir







Documents

Annual Report - California State University, Stanislaus...of university-wide projects for administrative, academic, and system-wide purposes. Before joining CSU Stanislaus, Dr. Sanchez