Embed Size (px)
Citation preview
1 | © 2013
How to Write a Successful IRGP Application
Andrew Flood, Ph.D.Jill Dreyfus, Ph.D.
Research and Sponsored ProgramsDesign and Analytics
2 | © 2013
I. Introduction to the IRGP
II. The Research Question
III. Design & Methods
IV. Analytic Plan
V. The Scoring System
Outline of Today’s Presentation
3 | © 2013
“The Internal Research Grant Program (IRGP) is designed to improve investigator access to intramural support for research projects that are likely to lead to external publication and further research.”
Purpose of the IRGP
4 | © 2013
“The Internal Research Grant Program (IRGP) is designed to improve investigator access to intramural support for research projects that are likely to lead to external publication and further research.”
Implications?
Purpose of the IRGP
5 | © 2013
1. Studies that will lead to impactful publication.
2. Studies that open up a new line of research.
3. Studies that do both are best of all.
What Kinds of Studies Will Receive Favorable Reviews?
6 | © 2013
Publications and
Foundation for Future Research
The Holy Grail of the IRGP
7 | © 2013
• Research related to the health or health care of children
• The E&R Committee will give priority to applications meeting one or more of the following characteristics:1. Collaborations with external institutions and researchers
2. Proposals from junior investigators
3. Projects within one of Children’s Cornerstones
4. Projects with other funding support so Children’s funding augments or extends a study of recognized merit
5. Proposals from established investigators seeking bridge funding
E&R Committee Funding Priorities
8 | © 2013
• Studies in areas of health care delivery and patient safety are strongly encouraged.
• Quality improvement studies may also be considered, provided they may be applied and/or published/presented beyond Children’s.
Additional Priority Considerations
9 | © 2013
Scale of IRGP Studies
• IRGP budget is $50,000. Award is for 2 years.
• Pilot studies:1. Feasibility
2. Preliminary data
• Small populations with new data collection
Power?
• Large studies using existing data
Available data?
10 | © 2013
Impact
The Key Issue
11 | © 2013
Formulating an Impactful Question
12 | © 2013
• What do we know?
• What don’t we know?
Start with the Existing Literature
13 | © 2013
1. A general statement of the problem
2. A numbered set of specific aims for the research
3. Hypotheses
Elements of a Good Research Question
14 | © 2013
• The gap in knowledge, and/or the unresolved question that the study is seeking to address.
General Statement of the Problem
15 | © 2013
• A numbered set of specific aims for the research.
• Identify the 2-4 specific pieces of information this study will provide as a means of helping to fill in the knowledge gap identified in step 1 (general statement of the problem).
Specific Aims
16 | © 2013
• Each specific aim should come with one or two hypotheses.
• A hypothesis is a statement about what the investigator expects his or her data will show if the theory of disease or medicine that he or she is using to explain and justify the hypothesis is true.
• A good hypothesis will be specific and will have directionality.
Hypotheses
17 | © 2013
• What we know: Poor nutrition is known to be a major contributor to impaired growth and cognitive achievement in children.
• What we don’t know: ?
Example: Nutrition and Growth in Impoverished Children
18 | © 2013
• What we know: Poor nutrition is known to be a major contributor to impaired growth and cognitive achievement in children.
• What we don’t know: The specific aspect of nutrition (lack of protein or lack of total kcals) responsible for these developmental deficits is unknown.
Example: Nutrition and Growth in Impoverished Children
19 | © 2013
To address this question we will conduct a feeding study using a factorial design with the following specific aims:
1.Identify the unique role of supplemental kcals in childhood growth
2.Identify the unique role of protein in childhood growth.
3.Determine the differential roles of protein and kcals in childhood growth.
Specific Aims
20 | © 2013
To address this question we will conduct a feeding study using a factorial design with the following specific aims:
1.Identify the unique role of supplemental kcals in childhood growth
Hypothesis: Children given kcal supplements will grow better than those given placebo
Hypotheses
21 | © 2013
To address this question we will conduct a feeding study using a factorial design with the following specific aims:
2.Identify the unique role of protein in childhood growth.
Hypothesis: Children given protein-enriched supplements will grow better than those given placebo.
Hypotheses
22 | © 2013
To address this question we will conduct a feeding study using a factorial design with the following specific aims:
3.Determine the differential roles of protein and kcals in childhood growth.
Hypothesis: Children given protein-enriched supplements will show improved growth beyond that seen in children given supplements with equal kcals but a standard amount of protein.
Hypotheses
23 | © 2013
(poorly constructed OR incompletely thought out) Research Question →
(difficult to write AND ineffective) Research Plans
Fancy Math
24 | © 2013
(well constructed AND well thought out) Research Question →
(easy to write AND effective) Research Plans
Fancy Math
25 | © 2013
• Research Questions and Specific Aims
• Background and Significance
Introduction
26 | © 2013
Background and Significance
27 | © 2013
• Sets the study in context
• Gives its rationale.
• From this description it should be easy to construct a logical argument as to why the proposed research question makes sense and is important to answer.
Background and Significance
28 | © 2013
• Significance of the study if it were to achieve its aims.
• Take into consideration the study question’s relevance to − the goals of the institution
− the meaningful advancement of knowledge
− the practice of medicine.
• How important is the difference this study makes?
Background and Significance
29 | © 2013
Study Design and Methods
30 | © 2013
• The study design is the approach that is taken to answer the research question.
• Examples of study designs include the following: − ecological
− case study
− case series
− cross-sectional
− case-control
− cohort
− clinical trial
Study Design
31 | © 2013
• The study design is the approach that is taken to answer the research question.
• Examples of study designs include the following: − ecological
− case study
− case series
− cross-sectional
− case-control
− cohort
− clinical trial
Study Design
32 | © 2013
• Subjects− Where do they come from?
• Relevant exposure− How it will be assessed?
• Comparison− How will the comparison be set up through the design?
• Follow-up if there is any
• Outcomes
Study Design Elements
33 | © 2013
A word about “Comparison”
Bedevere: What also floats in water? Villagers: Bread!
Apples! Uh, very small rocks! Cider!
Arthur: A duck! Crowd: Oooh.
34 | © 2013
• This is the key concept in any study
• All designs are explicitly for the purpose of setting up an appropriate comparison – what comparison?
− Frequency of an outcome
− 2 populations
− Differing on 1 factor
A word about “Comparison”
35 | © 2013
Methods
36 | © 2013
Methods - Study Population
“We’re French. Why do you think we have these outrageous accents?”
37 | © 2013
Methods - Study Population
• Describe in detail 1. Who the subjects are and
2. How the recruitment will happen
38 | © 2013
Methods - Study Population
• Inclusion Criteria− Objective in writing inclusion criteria: Identify the
precise subjects who will be best suited to provide the data that will answer the research question
− Important considerations include (but not limited to):
age
sex
disease status
demographics
language requirements
co-morbidities
39 | © 2013
Methods - Study Population
• Exclusion Criteria− Describe any characteristics that will lead to a subject
being excluded from participation in the study
− Avoid redundancy
− It is not essential to have both inclusion and exclusion criteria
40 | © 2013
Methods - Study Population
• Recruitment− Describe how you will identify and recruit potential
patients
• Enrollment− Include a description of the consent process,
specifying at what point this will occur. (In a randomized trial, consent must occur before the randomization.)
41 | © 2013
Methods – Study Protocol
42 | © 2013
Methods – Study Protocol
• This section should contain an outline of what each subject will be asked to do as a participant in the study once enrolled
• Provide sufficient detail to make it clear what you are doing to obtain your measurements
• But not every detail is important, report only the essential ones
• Give reviewers confidence that what you are planning to do is feasible and gets the right data
43 | © 2013
Methods – Study Protocol
• What measurements will be done at baseline? Using what methods?
• If using an existing database, what database? Where did it come from? Who is in it? Etc.
• What intervention, if any will be done? What will the subject have to do to complete the intervention?
• How will the investigator assess compliance?
44 | © 2013
Methods – Study Protocol
• What are the follow-up procedures? What will the subject have to do during the follow-up period? How will the investigator track the subjects?
• What will subjects have to do at the conclusion of follow up? What is the exit protocol for the study subjects, if relevant (typically useful in clinical trials)?
• How will the investigator assess endpoints? What are the end points? How are they measured?
45 | © 2013
Methods – Study Protocol
• Will any questionnaires be used to collect data from subjects? How valid are these? Will these be self reports or interviews? Who will manage questionnaire responses and how?
• What laboratory procedures, if any, will be used? How valid are these? Who will do them?
• What data management procedures are planned?
46 | © 2013
Methods – Time Frame
• Include a timeline− Outline all the steps in the study from planning to
analysis and publication
− Give reassurance to reviewers that this is feasible with respect to 2-year duration of IRGP’s
47 | © 2013
Analysis Plan
48 | © 2013
• Purpose: Describe how you will determine if there was an effect in your study.− Goal: convince reviewers that you know what you are
doing
• Tied to directly to your aims and hypotheses− How will you test for differences between groups?
− How will you test your hypotheses?
Analytic Approach
49 | © 2013
• Analytic approach is related to your study design− Experimental vs. observational design
� Observational design: consider confounding variables
− Outcome?
� Continuous or categorical
− Are measures repeated?
− Missing data?
• Include sample size and power calculations
Analytic Approach
50 | © 2013
• Sufficient power: are differences between groups− Clinically meaningful?
− Statistically different from chance?
• If power is too low, likely will not see statistically significant results
• For fixed sample size: need effect size calculation− Minimum effect difference that the study will be able
to detect between groups
• Provide sample or effect size calculations for primary aim and hypotheses
Sample Size and Power
51 | © 2013
• Probability that the result was different from chance − Not the result or primary finding
• Have equal regard for effect size and statistical significance
• Always report the magnitude of the effect, e.g.:− Difference in means
− Proportions in each group
− OR or RR
− Risk difference
P-values
52 | © 2013
1. Describe relevant comparisons− Include descriptions of the exposure and outcome
for each analysis
− These follow directly from the aims and hypotheses.
Steps for Writing the Analytic Approach Section
53 | © 2013
1. Describe relevant comparisons− Include descriptions of the exposure and outcome
for each analysis
− These follow directly from the aims and hypotheses.
2. Specify statistical test(s)− Tests are a function of the data and the types of
comparisons
− Enlist the assistance of Design and Analytics group
Steps for Writing the Analytic Approach Section
54 | © 2013
1. Describe relevant comparisons− Include descriptions of the exposure and outcome
for each analysis
− These follow directly from the aims and hypotheses.
2. Specify statistical test(s)− Tests are a function of the data and the types of
comparisons
− Enlist the assistance of Design and Analytics group
3. Sample size and power description
Steps for Writing the Analytic Approach Section
55 | © 2013
The Scoring System
56 | © 2013
• Patterned after NIH system
• Four areas scored, weighted for importance:− Significance = 40%
− Innovation = 25%
− Approach = 25%
− Investigators = 10%
• Each area scored 1 (best) to 9 (worst)
• Overall impact score = average from all voting members
The Scoring System
57 | © 2013
• Why is this study important?− Does the project address an important problem?
• How will the findings change medical practice?− How will scientific knowledge, technical capability, or
clinical practice be improved?
• Format – 1 page (from NIH document)− Introduce problem
− Provide additional background, if needed
− Emphasize significance of idea
− Emphasize significance in broader concept
Significance: 40%
58 | © 2013
• Why is your approach better than what was done before?− Does the project challenge and seek to shift current
research or clinical practice paradigms?
− Novel in one field of research, or in a broad sense?
− Is a refinement or improvement of concepts, approaches, methods, interventions proposed?
• Format – 1-2 paragraphs, labeled “Innovation”
Innovation: 25%
59 | © 2013
• Convince reviewers about your idea− Are the strategy, methods, and analyses well-reasoned
and appropriate to accomplish the specific aims?
− Are the potential problems and alternative strategies presented?
• For early stage projects− Will the strategy establish feasibility and will risks be
managed?
• For clinical research− Plans for protection of human subjects, inclusion of
both sexes, and children justified?
Approach: 25%
60 | © 2013
• Are the PIs, collaborators, other researchers well suited for the project?
• Sufficient expertise for:− Content area
− Design and analysis
− Required technical or lab work
• Financial: is the budget reasonable for meeting project goals?
Investigator(s): 10%
61 | © 2013
• Encouraged to submit a proposal.
• Reviewers blinded to new investigator status during scoring.
• Once overall rankings established, reviewers may move new Investigator proposals up 2 places.
New Investigators
62 | © 2013
We Are Here to Help – Please Ask Us!
“Who are you who are so wise in the ways of Science?”
63 | © 2013
• Study Inception Meetings
− Study objectives and requirements
− Design
− Analysis plans
− Budgeting
− IRB approvals
− Use of lab, pharmacy, and radiology
− Resources available to investigators at Children’s
We Are Here to Help – Please Ask Us!
64 | © 2013
Thank you!
