130726 i3 portfolio review

Update on the Investing in Innovation (i3) Fund

July 2013

Update on i3 ProgramPresentation Objectives

2

To date, the U.S. Department of Education (the Department) has funded 92 unique i3 projects that seek to provide innovative solutions to common education challenges.

This presentation aims to:Provide an overview of the progress i3 grantees are making towards their goals as well as what the Department is doing to help them be successfulHighlight the potential of i3 grantees to generate rigorous education researchUse data on grantee performance thus far to inform the design of future i3 competitions

Overview of the i3 Program

ApplicantsEligible applicants are: (1)Local educational agencies (LEAs)(2)Non-profit organizations in partnership with either (a) one or more LEAs or (b) a consortium of schools

PurposeTo generate and validate solutions to persistent educational challenges, to support the expansion of effective solutions across the country, and to serve substantially larger numbers of students.

FundingFY2010: Approx. $646 million in grant awardsFY2011: Approx. $148 million in grant awardsFY2012: Approx. $143 million in grant awardsFY2013: $135 million (est.) to be awarded by 12/31/2013

3

1. Aligns amount of funding with level of evidence and explicitly aims to scale effective programs by creating a pipeline of funding especially for those programs.

2. Builds a portfolio of innovative solutions to address key education challenges.

3. Provides funding for each project to complete an independent evaluation in order to expand understanding of “what works” in education reform.

What Makes i3 Unique?Core Design Features

4

1. Aligning Funding to Evidenceand Creating a Pipeline of Funding for Scaling Effective Programs

5

i3

Types of Awards Made Under i3

Funding Amount

Up to $5MM/award (2010)Up to $3MM/award (2011-2012)

Up to $30MM/award (2010)Up to $15MM/award (2011-2012)

Up to $50MM/award (2010)Up to $25MM/award (2011)*

Evidence Required

Evidence of promise or strong theory

Moderate evidence of effectiveness Strong evidence of effectiveness

Scaling Required Able to further develop and scale Able to scale to the regional level Able to scale to the national level

6

*In 2012, no Scale-up awards were made.

i3 Grants By Type 64% of i3 grants to date are in

the Development category and support the development and initial evaluation of promising practices.

53% of i3 funds to date are directed toward Validation grants, the purpose of which is to build organizational capacity to deliver the i3-supported practice, particularly early in the funding period, to reach a greater scale (as specified in the funded application).

Percentage of i3 Grants by Type

Percentage of Funds Allocated by Type

7

i3 Grants Serve High-Need StudentsPerformance reports from the 2010 and 2011 i3 grantee cohorts indicate that i3 grants are serving over 1 million students/year.*The 2010 i3 cohort reported serving 1,023,020 students in their 1st year.

93% of students served were high-need.

The 2010 i3 cohort reported serving 1,546,759 students in their 2nd year.

92% of students served were high-need.The 2011 i3 cohort reported serving 85,857** students in their 1st year.

80% of students served were high-need. Reminder: 2011 funding was ¼ of 2010 funding

*Note: The figures on this slide only represent grants that reported the number of high-need students served.**Note: 5 out of 23 grants in the 2011 cohort treated Year 1 as a planning year and did not serve students.

8

2. Building a Portfolio of Innovative Solutions

9

i3 Grants By PriorityPercentage of Funds Allocated By PriorityPercentage of i3 Grants by Priority

10

Building a Portfolio of Solutions for Education Challenges

With 92 grants funded to date, i3 projects are employing varied approaches to address education issues, such as:

Developing online training, coaching, standards-aligned resources, and videos of effective teaching practices

Improving the teacher and principal pipeline by creating alternate pathways to certification, developing rigorous evaluation systems, and building district capacity to train and place effective teachers and principals

Implementing college- and career- ready curriculum and dual enrollment programs

Creating or adapting digital tools and incorporating technology to support teaching and learning

11

Key Elements of Successful i3 Projects: Continuous Improvement Process

As grantees identify impediments to successful implementation, they have made appropriate mid-stream corrections, such as:

Improving the response time to meet participant needs by developing applications that improve data sharing practices and allow for immediate transmission of teacher performance data

Redirecting services to include school administrators in order to improve staff “buy-in” and the quality of support provided to teachers

Building technological infrastructure in rural communities to support effective implementation

Revising implementation plans to better align with current research in the field Incorporating program elements found to be successful with one participant

population into preparation efforts for another participant population

12

Key Elements of Successful i3 Projects: Quality Leadership and Delivery of i3-Funded Services

Based on i3 staff’s assessments: Over 90% of grantees made progress toward stated goals and objectives Over 80% of grantee leadership was perceived as both strategic and effective

in executing the project Almost 70% of grantees provided information demonstrating participant

satisfaction Most grantees’ expenditures of Federal and Non-Federal funds are on track

with their planned budgets and their progress to date

13

Snapshots of Promising i3 Projects

These snapshots provide examples for each type of i3 grant award (i.e., Development, Validation, and Scale-up). All snapshots are based on data from each grantee’s most recent performance report and include a hyperlink to each grantee’s original application narrative.

Inclusion of these projects within this presentation does not constitute an endorsement by the Department of any particular practice, intervention, or activity being carried under these projects, or imply that other i3 projects are not carrying out important and high-quality activities.

14

Education Connection Bay State Reading Institute Children Youth and Family Collaborative The New Teacher Project New Visions for Public Schools KIPP Foundation

Early Impact: What Has i3 Enabled Grantees to Do?

Education Connection’s i3 project enabled the facilitation of an interactive “blended learning” environment that engages students in online coursework guided by teachers. Through STEM21, students earn up to 15 college credits in the CT state college and university system at no cost to the student, facilitating college access and reducing financial barriers. In year 2 of its grant, Education Connection successfully provided four experiential learning opportunities, held at industry sites and college sites, to take students and teachers beyond the classroom and into real world STEM environments.

2010 Development GrantPriority: Standards and AssessmentsTotal Federal Funding: $4,473,481Area served: 15 LEAs in Connecticut; 1 in MassachusettsStudents served to date: 516 (target: 1,960 by grant end)Courses developed to date: 14

15


2010 Development GrantPriority: Use of DataTotal Federal Funding: $4,997,492Areas served: 12 schools in MassachusettsStudents served to date: 6,315 to date (target: 6,800 by grant end)Teachers demonstrating clear understanding of curricula to date: 330 (target: 472 by end of Year 2)

Bay State Reading Institute’s i3 project aims to help schools use data to positively impact student achievement. Theystate that i3 will enable them to determine whether their data-driven coaching for teachers is improving student results at those schools. Thus far, data shows that teachers are responding enthusiastically, and interim assessments of student performance indicate that student results are improving.

16


Advancement Through Opportunity and Knowledge- Children Youth and Family Collaborative’s i3 project shares data across sectors in an integrated service model that provides social workers a comprehensive intake assessment, individualized learning plan and pre-emancipation planning for foster youth. The project has been recognized as a model worthy of replication and they are preparing for county-wide expansion that will include replicating the program at 15 new sites.

2010 Development GrantPriority: Use of DataTotal Federal Funding: $3,649,580Areas served: 18 LEAs in CaliforniaStudents served to date: 448 (target: 400 by grant end)Participating seniors receiving their diplomas to date: 19 (target: 21 by end of year two of the grant)

17

The New Teacher Project is implementing an effectiveness-based teacher preparation program which draws on research-based Teaching Fellows Programs and Practitioner Teacher Programs to provide high-need districts with a teacher pipeline that spans activities from recruitment to certification and only certifies teachers who have demonstrated effectiveness in the classroom. The project uses unique “effectiveness screens” at key points in the certification process to determine whether participants are: (1) showing potential to demonstrate effectiveness, and (2) actually demonstrating effectiveness. Successful participants are certified as “Teachers of Record.”


2010 Validation GrantPriority: Teacher and Principal EffectivenessTotal Federal Funding: $20,829,095Areas served: 7 LEAs in LA, MD, NC, IL, TX, TN, and DCStudents served to date: 66,144 (target: 253,000-337,000 by grant end)Teachers of Record to date: 567 (target: 2,450-3,300 by grant end)

18


New Visions for Public Schools’ i3 project provides training and intensive support for teachers to create mathematics curricula aligned to the Common Core standards, monitor student progress through research-based formative assessments and formative assessment lessons, and improve their instructional strategies through participation in inquiry teams. Empowered by formative assessment data and collaborative instructional resources, teachers have been able to expand their reach resulting in increased student achievement. New Visions states that i3 enabled them to increase the capacity of their organization.

2011 Validation GrantPriority: Standards and AssessmentsTotal Federal Funding: $12,887,007Areas served: 30 schools in New York CityStudents served to date: 1465 (target: 62,650 by grant end)Schools recruited to date: 14

19


With its i3 grant, the KIPP Foundation is training 1,000 future leaders over its five-year grant period. Of the 1,000 leaders, 250 are principals, 60 of whom will be outside of the KIPP network of public charter schools, that will open a new school or assume leadership of an existing school during the grant period. The remaining 750 leaders being trained will use the training to begin their paths to school leadership. Additionally, the i3 grant has enabled KIPP to create a fellowship program that fosters the sharing and dissemination of best practices across multiple school types (including charter and traditional public schools).

2010 Scale-up GrantPriority: Teacher and Principal EffectivenessTotal Federal Funding: $50,000,000Areas served: 20 states (AR, CA, CO, FL, GA, IL, IN, LA, MA, MD, MN, MO, NC, NJ, NY, OH, OK, PA, TN, TX) and DCStudents served to date: 42,960 (target: 50,000 by grant end)Leaders trained to date: 447 (target: 1,000 by grant end)

20

i3 Grantees Continue to Improve Project Implementation with Differentiated Supporti3 staff work with grantees to help them evolve and improve their projects. To support this work, the Department provides grantees support that is specific to the issues they are confronting. For example, the Department provides technical assistance to i3 grantees that begins with an assessment of each grantee’s:

Greatest strengths Challenges to implementation Capacity to set clear goals Leadership effectiveness Level of stakeholder support Capacity to monitor their own progress Capacity to communicate effectively with external stakeholders

21

Results of i3 Grantee Needs Assessment Common areas of need:

Working in implementation sites Improving project team capacity Communicating and managing

relationships with multiple partners Using project management tools Working around unforeseen obstacles,

such as district budget cuts, school closures, and transitioning to Common Core State Standards

Highest level of need across projects: Scaling and sustaining projects

Strategy Funding sources Business planning

Sharing information Next steps:

One-on-one support for grantees with high priority needs Development of communities of practice Development of materials and tools to increase capacity

22

Issues Affecting 2010 and 2011 Grantees Who Did Not Make Substantial Progress

Common issues exist among the few grantees that did not make substantial progress in their first or second year of their projects.The common issues identified include:

Implementation delays Leadership challenges Low enrollment and lack of participant “buy-in,” likely due to problems

clearly communicating program strategy Evaluation risks (e.g., data collection issues)

Despite implementation delays, of the 2010 cohort grantees that did not make substantial progress in year one, the majority still managed to make progress toward their goals in year two.

23

3. Discovering “What Works” in Education Reform

24

i3 Projects Will Generate Rigorous Education Research

i3 grantees are required to conduct an independent evaluation of their projects. Grantees have a range of needs in this area, and the Department provides evaluation technical assistance to meet these needs by helping grantees to:

Conduct high-quality research that answers questions that provide new and useful results to inform the education field

Conduct research that is likely to meet the rigorous What Works Clearinghouse (WWC) evidence standards

Maintain evaluation rigor throughout the implementation of their i3 project

Troubleshoot any issues with local evaluators Report findings in support of the national evaluation of i3

25


A primary goal of the i3 program is to ensure that projects funded under the i3 program contribute significantly to improving the information available to practitioners and policymakers about which practices work, for which types of students, and in what contexts.A key piece of this learning is our supporting project evaluations with the potential to meet What Works Clearinghouse (WWC) Evidence Standards (with or without reservations).

Development Validation Scale-up

2010Cohort Size 30 15 4

# with Potential to Meet WWC standards 22 15 4

2011Cohort Size 17 5 1

# with Potential to Meet WWC standards 13 5 1

Total 35 20 5

26


Nearly all of the i3 grantees are on track to produce evidence about the effectiveness of their projects. Of the 2010 i3 cohort:

98% of the grantees are meeting the i3 performance measure that assesses whether the grantees’ project evaluations are designed to provide evidence of effectiveness at scale (for the Scale-up grants), effectiveness (for the Validation grants), or promise (for the Development grants).

92% of the grantees are meeting the i3 performance measure that assesses whether the grantees’ project evaluations are designed to provide high-quality implementation data and performance feedback.

27

The Department anticipates the release of the report on the national evaluation of the i3 program in 2016 (following the completion of the 2010 i3 cohort).

The goals of the report are to provide clear statements about the effectiveness or promise of the i3 funded interventions on students, assessments of the strength of the evidence generated by the evaluations, and summary statements about the degree to which the interventions were implemented with fidelity.

28

National Evaluation of the i3 Program

Common Evaluation Challenges A few i3 grants experienced fragmented or low recruitment as well as

attrition. Grantees addressed this challenge by better coordinating their communication with partners to facilitate recruitment of new sites or participants.

Some grantees experienced challenges implementing the i3-supported intervention with fidelity while maintaining a rigorous evaluation design. The evaluation technical assistance aims to address this challenge by having the grantees and their evaluators develop logic models and fidelity matrices early in their projects.

Current regulations and law regarding the length of the project periods and availability of funds for the 2010 i3 grants as well as the timing of the availability of states’ assessment data present challenges for grantees completing their independent project evaluations during the project period. The i3 program staff is working with individual grantees to mitigate this issue.

29

Looking Forward

30

Improved i3 Program Design for 2013In 2013, the Department redesigned key aspects of the i3 program to increase the program’s impact. Revised requirements better reflect program expectations by:

Clarifying that projects must be designed to improve student achievement of high-need students and serve students in grades K-12

Tightening focus of grantee evaluation on impact

Revised evidence standards address prior areas of confusion by: Creating two levels of evidence for Development grants Providing specific conditions under which a study would meet the strong or moderate

evidence of effectiveness levels Explicitly defining terms used to describe the different levels of evidence

31

Structure of priorities remains similar but priority language is more focused to: Include many of the same broad priority areas (e.g., teacher and principal effectiveness) Maintain flexibility to select different priorities for each grant competition Create specific sub-parts to reflect needs in the field Strengthen the rural priority

Selection criteria remain similar but new factors are added to reflect lessons learned from previous competitions.

Process for applicants is modified to secure and demonstrate evidence of required private-sector match (e.g., allows flexibility to expand the timeframe for grantees to secure the match).

Improved i3 Program Design for 2013

32

Appendix

33

Number and Funding by Grant Type, Year

Development Validation Scale-up Total

2010 30 15 4 49

2011 17 5 1 23

2012 12 8 0 20

Total 59 28 5 92

Development Validation Scale-up Total

2010 $140 $311 $195 $646

2011 $50 $73 $25 $148

2012 $34 $109 $0 $143

Total $224 $493 $220 $937

Number by Grant Type, Year

Funding by Grant Type, Year (in millions)

34

i3 Scale-up and Validation Grantees ASSET Inc. Children's Literacy Initiative Council for Opportunity in Education George Mason University Harvard College Jobs for the Future, Inc. Johns Hopkins University KIPP Foundation (Scale-up) LEED Sacramento National Math and Science Initiative National Writing Project New Leaders, Inc. New Schools for New Orleans New Teacher Center New Visions For Public Schools, Inc Niswonger Foundation North Carolina New Schools Project

Old Dominion University Research Foundation (Scale-up) Parents as Teachers School District No. 1/Denver, CO Smithsonian Institution Success for All Foundation (Scale-up) Teach For America (Scale-up) Texas A&M University The New Teacher Project, Inc. The Ohio State University (Scale-up) University of Alaska Statewide Office of K-12 Outreach University of Minnesota University of Missouri Utah State University University of Wisconsin WestEd (one in 2010 and one in 2012)

35

i3 Development Grantees Advancement Through Opportunity and

Knowledge Alliance for College-Ready Public Schools American Federation of Teachers Educational

Foundation AppleTree Institute Aspire Public Schools AVID Center Baltimore City Public Schools Bay State Reading Institute Beaverton School District Bellevue School District Berea College Board of NYC/School of One Boston Plan for Excellence in the Public Schools

Foundation Boston Public Schools

Boys & Girls Clubs of Greater Milwaukee California Education Round Table California Association for Bilingual Education California League of Middle Schools Central Falls School District Citizen Schools, Inc. Clark County School District Columbia College Chicago Corona-Norco Unified School District Del Norte Unified School District Education Connection Erikson Institute Exploratorium Forsyth County Schools Fresno County Office of Education IDEA Public Schools

36

i3 Development Grantees (cont.) Intercultural Development Research

Association Internationals Network For Public Schools Iredell-Statesville Schools Jefferson County Board of Education Kentucky Valley Educational Cooperative KnowledgeWorks Los Angeles Unified School District Montgomery County Public Schools National Board for Professional Teaching

Standards National Forum to Accelerate Middle-Grades

Reform New York City Board of Education-InnovateNYC New York City Board of Education-District 7 New York Hall of Science

Oakland Unified School District Ounce of Prevention Fund Plymouth Public Schools Saint Vrain Valley School District School Board of Miami-Dade County Search Institute Success for All Foundation Take Stock in Children Inc. Temple University Texas Tech University The Achievement Network LTD The College Board The Metropolitan Education Commission The Studio in a School Association, Inc. Virginia Advanced Study Strategies, Inc. WestEd

37

Number of Grants by Priority, TypePriority Development Validation Scale-up TotalLow Performing Schools 12 6 2 20

Parents and Family 4 0 0 4

Rural Achievement 6 2 0 8

Standards and Assessments 14 7 0 21

STEM 6 3 1 10Teachers and Principals 10 8 2 20

Use of Data 7 2 0 9Total 59 28 5 92

38

*Note that the Rural Achievement and STEM priorities were used only in 2011 and 2012 competitions (for all three types of grants) and the Parent and Family Engagement priority was used only in the 2012 Development competition. Also note that the Use of Data priority was used only in the 2010 competition.

Funding Breakdown by Priority, Type (in millions)

Priority Development Validation Scale-up TotalLow Performing Schools $48 $118 $95 $261

Parents and Family $10 $0 $0 $10

Rural Achievement $18 $30 $0 $48

Standards and Assessments $59 $122 $0 $181

STEM $17 $35 $25 $77Teachers and Principals $39 $155 $100 $294

Use of Data $33 $33 $0 $66Total $224 $493 $220 $937

39

*Note that the Rural Achievement and STEM priorities were used only in 2011 and 2012 competitions (for all three types of grants) and the Parent and Family Engagement priority was used only in the 2012 Development competition. Also note that the Use of Data priority was used only in the 2010 competition.

Definitions

Students served is defined as number of students impacted directly or indirectly and does not include control group students.

High-need student is defined as a student at risk of educational failure or otherwise in need of special assistance and support, such as students who are living in poverty, who attend high-minority schools (as defined in this notice), who are far below grade level, who have left school before receiving a regular high school diploma, who are at risk of not graduating with a diploma on time, who are homeless, who are in foster care, who have been incarcerated, who have disabilities, or who are English learners.

40

Documents

130726 i3 portfolio review