The “Secrets” to Securing IES Funding: Some Lessons Learned

as an IES Standing Panel Member

Geoffrey D. BormanProfessor, Educational Leadership and Policy

Analysis, Educational Psychology, and Educational Policy Studies

Types of Grants

• Goal 1: Problem Identification

• Goal 2: Development

• Goal 3: Efficacy Trials

• Goal 4: Scale-up Effectiveness Trials

• Goal 5: Measurement

How are Proposals Rated?

• Multiple panels to reflect the various IES funding priorities and competitions

• Each reviewer from the panel is assigned about 8-10 studies for review

• 2-3 reviewers present critiques of each proposal• The remainder of the panel (including the chair)

adds comments and questions• Some stronger consensus is usually reached

among the 2-3 primary reviewers• All panel members then rate the application

What is Rated?

• Significance of the problem (1-7)• Research Plan (1-7)• Personnel (1-7)• Resources (1-7)• Budget (1-7)• Overall score (5-1)• Funding enthusiasm• Each application evaluated on its own merit

relative to an ideal

Overall Rating Scale

Overall Score

1.0–1.5 Outstanding1.6–2.0 Excellent

2.1–2.5 Very Good2.6–3.0 Good3.1–4.0 Fair4.1–5.0 Poor

Provide Strong Theory and/or Empirical Evidence

• In establishing the significance of the project, theory is important

• Practical contributions, though, are also key

• The chain of empirical evidence must be established and the place of your project in that chain needs to be established and justified

Be Obsessively Specific Concerning Methodological Details • If random assignment is proposed, describe the

actual process• Describe quasi-experimental methods in even

more detail• Describe the analytical models• Describe threats to internal validity• Address external validity• Do a technically correct power analysis• Choose good measures and defend them (IES

also likes high-stakes tests)

Other Details

• It is good to have an expert on experimental design (or the design of choice) on the proposal

• But panels usually do not place their trust in experts—they evaluate the quality and completeness of the design

• Achievement is usually the main outcome of interest—”policy-relevant” local or NCLB outcomes often favored unless specific skill measures for a domain are desired

• Growing interest in measuring the nature of the treatment and counterfactual to help explain null results or provide some non-experimental understandings of mechanisms driving positive results

Summary

• The rationale and significance must be strong• But, the research plan generally wins or loses it• Applications are not blinded, so the general

reputational quality of the institution and researchers influence personnel and resource ratings a great deal

• Panels sometimes like “bargain” studies, but do not generally dwell on budget details

• If your proposal is a revise and resubmit, address the original reviewers’ concerns and describe how you are responding