The Campbell Collaboration www.campbellcollaboration.orgThe Campbell Collaboration www.campbellcollaboration.org
Education Panel SessionComments and Points for Discussion
Sandra Jo WilsonVanderbilt University
Editor, Campbell Collaboration Education Coordinating Group
The Campbell Collaboration www.campbellcollaboration.org
Truancy Interventions• Focused on students already exhibiting
attendance problems.• Broad inclusion criteria in terms of study
designs.• Statistically significant effects on attendance
outcomes.– Also, significant variability in those effects.– And, some question about the clinical
significance.
The Campbell Collaboration www.campbellcollaboration.org
The Campbell Collaboration www.campbellcollaboration.org
What moderates the effects of truancy interventions?• A variety of moderators was examined, including
publication type and research design, participant characteristics, and intervention characteristics.
• No moderator was significantly associated with larger or smaller treatment effects.
• Small sample makes it difficult to detect significant moderators.
• But, interesting that different modalities had similar effects, and that participant characteristics were not influential.
The Campbell Collaboration www.campbellcollaboration.org
Volunteer Tutoring• Update of a previous review• Intervention scope is similar to the attendance
intervention review• Participant scope is broader• Broader array of outcomes• Inclusion regarding research designs more stringent• Statistically significant affects on most of the
achievement outcomes– Comprehension and mathematics not significant
The Campbell Collaboration www.campbellcollaboration.org
Updating a Previous Review
• Including newly available literature and broadening the geographic scope
• This should give us greater opportunities for examining variability
• New innovations in meta-analysis methodology make updating reviews attractive from a methodological standpoint, in addition to the obvious policy relevance.
The Campbell Collaboration www.campbellcollaboration.org
Discussion Points• What challenges did you face in narrowing down
your question?• What challenges did you face in gathering
necessary data from source studies?• What areas of support would make it easier for
you to do a review like this in the future?• What were the difficult choices you had to make
during the study collection process or the review process?
The Campbell Collaboration www.campbellcollaboration.org
Scoping a Review• The research question
–Are you interested in whether a particular program is effective vs. are you interested in what best improves some problem?
–What research designs would be the best for answering such a question? • What designs are actually used.
The Campbell Collaboration www.campbellcollaboration.org
Other Scoping Considerations• Your constituents
– Your funding agency– The users
• Budget
The Campbell Collaboration www.campbellcollaboration.org
Challenges in the Conduct of a Review• Definitional and language challenges
– Has implications for your search, for the scope, for how you combine (or not) outcomes in a meta-analysis, and for moderator analysis• What is truancy, how is it different from school refusal
or attendance problems?• Programs with different labels but the same content?
• Missing or vaguely reported data in primary studies
The Campbell Collaboration www.campbellcollaboration.org
Support and Training for Conducting a Review• Creating the ideal review team• Content and methods experts• Making contact with other groups who may
have access to different resources– Can help with the search and with
definitional issues
The Campbell Collaboration www.campbellcollaboration.org
Other Discussion Points?• Reporting relevant and useful reviews
– How do we create useful systematic reviews?• Questions for our review authors?• Other topics?
The Campbell Collaboration www.campbellcollaboration.orgThe Campbell Collaboration www.campbellcollaboration.org
Thank you to Brandy and SarahAnd our audience!
Sandra Jo Wilsonemail: [email protected]