What We Know and How We Know it FIPSE Results

Prepared by: Jill Perry Debby Zambo Susan Wunder

Contributors: Ray R. Buss, Ron Zambo, and Tiffany R. Williams

Fund for the Improvement of Post-secondary Education (FIPSE) grant

• received in 2010• focused on 21 original Phase I members • sought to document and evaluate:

1. change in the structure of graduate schools 2. change in the signature learning processes,

learning environments, and patterns of engagement of faculty and candidates in CPED-influenced EdD programs

3. fidelity to a set of guiding principles developed in Phase 1

Disseminate lessons learned and best practices.

Overall Data and Analytical ProcessDATA: 21 cases and 3 surveys (student, faculty, researcher) with both close- and open-ended items

Analysis of all measures aimed at ensuring credibility/trustworthiness/validity/reliability

ANALYSIS OF CASES: Focused on finding commonalities and complexities within and across CPED institutions.• performed f-2-f and virtually• entailed multiple iterations• cases read reread, examined through theoretical framework (Rogers) for answers

to RQs• cases coded and re-coded• matrix created for each case• from matrices themes developed and from these claims/assertions made

Understanding How Schools of Education have Redesigned the Doctorate of Education

Jill Alexa PerryDebby Zambo Susan Wunder

Paper presented at the 2014 American Educational Researchers Association Annual Meeting

Theoretical Frame and Methodology

• Rogers (1995) Diffusion of Innovation

• original data collected and analyzed by by 38 researchers – wrote 21 cases

• cross-case analysis conducted by 3 researchers

• proceeded through multiple levels

Prior to Joining CPED Institutions Experienced Issues and Pressures

Internal issues and confusion:• coursework not distinct• low quality dissertations• students in wrong programs• declining enrollments • ABDs

External pressures:• state level – improved leadership preparation • districts and organizations – better prepared employees and

research partners • students – programs to prepare them to take on leadership

roles

CPED Influenced Policy

• time to degree • number of required degree credits • dissertation format • dissertation oversight

CPED Influenced ProgramsPrograms incorporated and used (in varied ways) • CPED’s six principles • CPED’s six design features scholarly practitioner

CohortsCourses • Content, sequence and focused on practiceNew pedagogies Collaborative learning environments Intensified patterns of engagementDissertations in practice

CPED had an Impact on Deans

• new ways to bargain and collaborate• communication opportunities and status with

upper administration and other deans• cache that allowed them to introduce the idea

of programmatic change• support required

CPED had an Impact on Faculty• shift in workload and faculty positions• shift in pedagogy • shift in relationships with students • a national network (convenings)• cache• not all faculty open to change and could slow

change• junior tenure-track faculty fit into tenure• some practitioners hired as clinical faculty did

not feel welcome

CPED had an Impact on Students• clearer direction - sequence of courses • focused on their own problems of practice and

professional goals • respect for their practitioner knowledge• extended communication and interaction with

faculty• cohorts and support groups • satisfaction with their programs

Cross Case Conclusions • Schools of education adopted the CPED design

features and principles and diffused them throughout their organizations to create innovative and distinct EdD programs.

• Changes occurred in the signature learning processes, learning environments, and patterns of engagement.

• Lessons learned and best practices are emerging.

Seven Years After the Call: Students’ and Graduates’ Perceptions of the Re-envisioned

Ed.D.

Ron ZamboDebby Zambo

Ray R. BussJill Alexai Perry

Tiffany R. Williams

Innovative Higher Education 2013

Student Survey1. What are students in newly designed Ed.D.

programs learning? Does what they are learning align with CPED’s principles?

2. How are students in Ed.D. programs learning? Does this type of learning/teaching align with CPED’s design concepts?

3. Do students in Ed.D. programs see themselves as scholarly practitioners? If so, what does this mean?

4. Why are students pursuing an Ed.D.?

InstrumentOnline questionnaire 6-point Likert scale (Strongly Agree to Strongly Disagree)• 32 close-ended items based on CPED’s vision of a

scholarly practitioner, its principles, and design features

• 1 open-ended item asking participants why they pursued an Ed.D. from a CPED-influenced program

Participants• 296 respondents - 14 (67%) of the 21 institutions • 266 students currently enrolled in a program • 30 recent graduates • 64% female - 36% male• held education related positions for 14 years

143 (65.9%)PK-12 69 (31.8%) post secondary education(2.3%) professions outside of education

 Construct with CPED

Principle

 Mean

 SD

 Alpha

Rangeby

Institution

Learning to collaborate and form partnerships (Prin. #3)

 5.26

 0.81

 0.78

 4.75-5.71

Learning to apply what they learn to solve problems of practice (Prin. #6)

5.11 0.84 0.80 4.42-5.68

Learning to connect theory to their practice (Prin. #5)

5.08 0.88 0.83 4.27-5.82

Becoming leaders working toward positive change (Prin. #2)

5.06 0.83 0.65 4.36-5.45

Becoming scholarly practitioners (Broad Goal)

5.02 0.76 0.79 4.31-5.54

Learning to engage with diverse communities and work toward social justice (Prin. #1)

4.73 0.91 0.74 4.25-5.70

Learning through authentic experiences (Prin. #4)

4.55 1.21 0.72 3.53-5.36

Open-Ended Item: Why an Ed.D.?• professional, career related advancement• personal reasons • development and growth• because of the degree itself

Faculty Members’ Responses to Implementing New EdD programs

Ray R. Buss Ron Zambo

Debby Zambo Jill Alexia Perry

Tiffany R. Williams

under review

Faculty Survey1. How and to what extent have variables associated with

Rogers’ theory on diffusion and adoption of an innovation influenced program changes, implementation, and outcomes in newly designed/redesigned EdD programs?

2. How and to what extent have CPED principles related to EdD program improvement influenced program changes, implementation, and outcomes in newly designed/redesigned EdD programs?

InstrumentOnline questionnaire 6-point Likert scale ( Strongly Agree to Strongly Disagree)

55 close-ended items that asked about:• changes to their programs• benefits resulting from their participation in CPED• use of the six CPED principles• conceptualizations and outcomes of various redesign efforts• communication channels, time, social system of faculty members and

adoption of redesign efforts

two open-ended items: • Describe two important changes that occurred in your program because of

your participation in CPED• Do you have any comments or questions?

Participants

61 faculty members from 12 institutions who had shaped, developed, and worked in their EdD program

Criterion variable df, F test statistic and p Adjusted R2 Individual Predictor Variables that Were

Statistically Significant

Program changes F(2, 58) = 76.65,p < .001

.72 Communication channels, CPED Principle 4

Innovation implementation F(3, 57) = 40.65,p < .001

.67 Social system of faculty members, Communication channels, CPED Principle 1

Program orientation 

F(3, 57) = 44.13,p < .001

.68 CPED Principle 1,Social system of faculty members, CPED Principle 4

Program attractiveness F(2, 58) = 11.20,p < .001

.25 Social system of faculty members, Time

Program learning environment F (2, 58) = 29.97,p < .001

.49 Communication channels,CPED Principle 4

Program benefits from participating in CPED

F(3, 57) = 40.23,p < .001

.66 CPED Principle 2,Communication channels,CPED Principle 4

All construct scales demonstrated acceptable levels of reliability (0.70or higher) except for principle #4

Open-ended ItemsChanges and Comments

Six themes:• program focus/orientation• program changes• logistics of program implementation • faculty members’ perspectives• program outcomes because of changes• value of participating in CPED

Researching the Researchers: The Influence of a Sense of Belonging on Faculty and Student

Research Volunteers

Debby ZamboRay R. BussRon Zambo

Jill Alexia Perry

Paper presented at the 2014 American Educational Researchers Association Annual Meeting

Paper under review

Researcher Survey1. What were the CPED-FIPSE researchers’

motivations to volunteer?

2. What did the researchers learn through their participation in the research project?

3. What was the greatest benefit the researchers gained as a result of participating?

4. Would the researchers participate again?

InstrumentOnline questionnaire 6-point Likert scale ( Strongly Agree to Strongly Disagree)22 close-ended items • six constructs loosely based on Nambisan and Baron (2007) sense of

belonging• reliability of each construct ranged from .71 to .96.

five opened-ended items • Why did you volunteer for the FIPSE research project? • What did your learn from your participation in the research project? • What did your learn about CPED? • What was the greatest benefit you gained as a result of participating?• If you were asked to volunteer again would you do it?

Participants

• twenty-seven (out of possible 38) completed the questionnaire (71% response rate)

• from various CPED-institutions

Construct Group Faculty

Fellows Overall Cognitive Benefits 4.38 (1.40)* 5.90 (0.32)

4.94

Connectedness to Research Group 4.85 (0.68) 4.50 (1.44) 4.72(Social-integrative Benefits) Connectedness to CPED 4.94 (0.87) 4.78 (0.95)

4.88(Social-integrative Benefits) Personal Expectations 4.85 (0.77) 5.40 (0.70)

5.05(Personal-integrative Benefits) Needs Fulfilled (Hedonistic) 5.12 (0.60) 4.83 (1.25) 5.01 Usefulness of Training Materials 4.65 (0.89) 5.50 (0.50) 4.96

*Note: SD are in parentheses.

Open-ended ItemsQuestion 1—Reasons for volunteeringFaculty• learn about research, programs, and change• reciprocate• network

Fellows (students)• learn about qualitative research and case study

methodology• strengthen research skills• apply what they had learned • socialize• encouraged to join

Question 2—What participants learnedFaculty• information about CPED as an organization• variation in programs• struggles associated with making changes Fellows• learned about CPED in ways that were different from

faculty • variations in programs• faculty relationships• change process

Question 3—Benefits of participatingFaculty• learning about change • validation of changes in their programs

(reputation)• self-efficacy Fellows• research • relationships/networks

Question 4—would they participate in the future? Faculty 78% would, 17% would not, and 5% would, but had reservations

would - they and their programs benefitedwould with reservations - if it fit their research agenda

(tenure) would not – too time consuming

Fellows50% would, 20% would not, and 30% would, but had reservations• would - relationships, exposure to like-minded people, work

with faculty members • would with reservations , time and resources• would not – not enough involvement, took too much work

Conclusionscognitive benefits (4.94)• learned about CPED, research, programs, and

change from like-minded individuals closest to the source

social-integrative benefits (4.72)• network • socialize –form relationships personal-integrative Benefits (5.52)• reputation and self-efficacy (faculty)hedonistic (5.01)• most would volunteer again

Copyright 2014 by the Carnegie Project on the Education Doctorate, Inc. (CPED). The foregoing material may be used for noncommercial educational purposes, provided that CPED is acknowledged as the author and copyright holder. Any other use requires the prior written consent of CPED.