The research challenges of large scale RCTs with volunteers and volunteering organisations

Facilitator: Dr Matt RyanPresenter: Professor Peter John

RCTs, trials, randomised evaluations, experiments• Randomly allocate something to a group and compare with a group

that does not get it• Needs to be staged and for certain things to be in place –

randomisation, and tying of outcomes to random allocation• See diagram• Unbiased estimate of a programme• Considered to be the gold standard• They are hard to do - worth it, and lots of leaning and interactions

with researchers to the benefit of all

Figure 7.1 Simple Random Allocation Design

                                                                                                                                                                                                                                                                                                                                      

Simple Random Allocation Design

Key distinguishing features of RCTs

• Research needs to deliver or have delivered an intervention at the same time as research

• So more activities to carry out as well as measurement• Need for effective coordination of activities• Real world constraints need to be anticipated • Need to ensure procedures follow special conditions of a trial –

attrition, selection etc.

Interactive design: The Glass Bead Game• Step 1: Assessment of fit for a RCT• Step 2: Recruitment of partner(s) • Step 3: Selection of the research site and subjects• Step 4: Determination of the treatments• Step 5: Specification of control condition• Step 6: Calculation of sample size• Step 7: Agreement on procedure for random allocation• Step 8: Collection of data on the subjects• Step 9: The preparation of research plans• Step 10: Assessment of ethical principles

Threats to experiments

• Problem 1: Loss of subjects • Problem 2: Failed randomization• Problem 3: Treatment not given or modified• Problem 3: Interference between treatment and control• Problem 4: Unavoidable confounds• Problem 5: Poor quality data• Problem 6: Loss of cooperation of partners• Problem 7: Unexpected logistical challenges and expenses

Volunteering experiments

• We are the first of these (bar one)• Sector has had a lot of attention but with giving e.g. experiments on

matching (see BIT), e.g. on JustGiving• Volunteering is different – more lumpy, differentiated, hard to match

demand and supply• Need to recruit volunteering organisations – really enthusiastic and

interested in the methods – issue was the practical constraints• RCTS need:

• enough numbers• good data and its management• those who understand experiments

Some reactions to the offer:no fear of white coats!

Politicophobia?

• Concerns about mentioning politicians, especially with governing boards

Data protection and management

• Need to ensure transfer of the data is legal, sensitive – university example

• Managing data

12

Recruitment to the project, volunteering• High attrition to staying in the study • Just getting volunteers to volunteer!

Measuring outcomes

• Hard to find records of the volunteers, information not routinely kept or easily found

Summing up

• It is possible to get the experiments to work, but new grounds – so needed our great partners to make it happen

• Might be easier now if this knowledge disseminates• Also experience generally is growing about RCTs >> helps volunteering

experiments• Can we make it easier, get better quality data