REGIONAL EDUCATION REGIONAL EDUCATION LABORATORY- SOUTHEASTLABORATORY- SOUTHEAST

REL-SEREL-SE

Executive Director

Dr. Ludwig van Broekhuizen

GAEL presentation 2/4/08

Operated by SERVE Center

at the University of North Carolina at Greensboro

2

The Regional Educational The Regional Educational Laboratory (REL) SystemLaboratory (REL) System

3

NATIONAL LABORATORY NATIONAL LABORATORY INFORMATIONINFORMATION

To access the work and products of all ten RELs:

http://ies.ed.gov/ncee/edlabs/

&

Dr. Kim Anderson, Senior Policy Research Analyst, Georgia

404-657-6174, kanderson@serve.org

4

REL-SE’s MISSIONREL-SE’s MISSION

To serve the educational needs of the

Southeast using applied research,

development, dissemination, and training

and technical assistance to bring the best

available evidence and research into local,

district, state, and regional school

improvement efforts.

5 components of work:

5

REL-SE WORK PLANREL-SE WORK PLANComponent #1: needs Component #1: needs

assessmentassessment

• Find out: On what topics do educators

need high quality evidence? E.g. at

GAEL

6

REL-SE WORK PLAN REL-SE WORK PLAN Component #2: State outreachComponent #2: State outreach

• Policy Analyst at GaDOE

• Evidence-based Education Request Desk

7

REL-SE WORK PLAN REL-SE WORK PLAN Component #3: Fast Response Component #3: Fast Response

briefsbriefs

• On issues of high priority for educators

• Secondary Data Analysis, Literature Review, State Scan, Other

• E.g.: Literacy across the Curriculum, High School Reform briefs

8

REL-SE WORK PLAN REL-SE WORK PLAN Component #4: rigorous Component #4: rigorous

researchresearch

• Experimental design research studies.

• Random assignment:

– Control group (no intervention)

– Treatment group (intervention)

• Data Analysis

• E.g., Georgia SIOP study

9

REL-SE WORK PLAN REL-SE WORK PLAN Component #5: build capacity Component #5: build capacity for evidence-based educationfor evidence-based education

• Consortium of Educators for Evidence-Based Education (CEEBE)

• Training and Technical Assistance

10

The Big Question for The Big Question for CEEBE: CEEBE:

What needs to happen for local districts and schools to more effectively and consistently utilize research, to improve

student achievement?

11

CEEBE CEEBE • Increase capacity and provide support for

educators to use Evidence Based Decision Making (EBDM)

• Disseminate EBDM products and services

• CEEBE-GA has 5 participating districts (Bibb, Clayton, Columbia, Floyd, Stephens) and representation from GaDOE

• Would you like more information?

12

How Do We Make Evidence-Based

Decisions?

The EBDM CycleThe EBDM Cycle

14

Types of EvidenceTypes of Evidence

• Research Study to advance knowledge, seek

credible explanations of phenomena

• Evaluation Study to determine merit or worth

• Data E.g.: Formative, summative assessments Demographic Process

15

RESEARCH: critiquing RESEARCH: critiquing qualityquality

When we look at a study on an intervention, how do we know it is of high quality?

REL-SE Critical Reading Protocol for Studies about Interventions

Critical Reading ProtocolCritical Reading Protocol

1) Authorship:

Who is the author? Who funded the study? Is there an obvious bias?

16

Critical Reading ProtocolCritical Reading Protocol

2) Sample, Relevance:

Does the study involve an intervention and outcome similar to the one you need?

How similar are the participants and settings in this study to your system?

17

Critical Reading ProtocolCritical Reading Protocol3) Research Design:

Does the study include a control or comparison group?

Are the research design, instruments, and procedures appropriate for answering the research questions?

Does the study appear in a peer-reviewed journal, or was it reviewed by a panel of independent experts?

18

Sample sources for expert Sample sources for expert research reviewsresearch reviews

• What Works Clearinghouse

• Best Evidence Encyclopedia

• Regional Educational Laboratories

• Center on Instruction

• Research organizations (e.g., MDRC, CPRE, Mathematica, etc.)

*handout

19

Critical Reading ProtocolCritical Reading Protocol

4) Results: What kinds of outcomes were measured?

Is there statistical significance?

Is there practical significance?

Has this study been replicated, and did the conclusions hold true in other settings?

20

21

Critical Reading ProtocolCritical Reading Protocol

5) Implementation:

What information is provided about implementation?

Does the study connect implementation to the results?

DISCUSSIONDISCUSSION

• To what extent have we asked these kinds of questions about interventions we have undertaken in the past?

• What interventions could we use the Critical Reading Protocol with in the future?

• What would we need to do to make this happen?

22

The EBDM CycleThe EBDM Cycle

24

Collective Professional Collective Professional WisdomWisdom

• Beliefs

• Attitudes

• Pedagogical Knowledge

• Content Knowledge

The EBDM CycleThe EBDM Cycle

26

Context and ConstraintsContext and Constraints

Context: The conditions within which something occurs (setting, leadership capacity, student needs, etc.)

Constraints: restrictions and limitations such as funding, facilities, etc.)

The EBDM CycleThe EBDM Cycle

28

If a strong evidence base If a strong evidence base does not exist . . .does not exist . . .

• Adopt and experimental attitude: Initiate Your Own “Action” Research on the Intervention!

• DISCUSSION: What experiences have we had with this?

29

Monitor and Assess Monitor and Assess ImplementationImplementation

Monitor: • Basic intervention components• Classroom observations• Formative assessments• Collegial problem-solving

Assess: • Is it going the way we planned? How can we

improve?

30

Evaluate outcomesEvaluate outcomes

What are the measurable outcomes (formative and summative assessments, and other data)?

How do they compare to our expectations?

Is it worth continuing with this intervention?

How can we improve our outcomes as we move forward?

DISCUSSIONDISCUSSION

1.How are we already using EBDM in our school or system?

2.How could we strengthen our use of EBDM?

3.What EBDM support do we need to do this?

31

The EBDM CycleThe EBDM Cycle

REL-SE needs assessment REL-SE needs assessment formform

• Thank you for filling out this form.

• It will help the REL-SE to design future research and technical assistance projects to meet your needs.

• Remember the request for more information on CEEBE on the back.

33