Talk to UsApproach to Evaluation

Lindsay Wager1st April 2014

2

• Questionnaires written by PhD student (member of Outreach team).

• Pupils - pre involvement questionnaire- post visit questionnaire- focus groups- celebration event questionnaire

• Teachers - pre involvement questionnaire- interviews- post project questionnaire

• Researchers - pre involvement questionnaire- interviews- post project questionnaire

Year 1

Purpose: to evidence impact of project in terms of intended outcomes and to inform practice regarding content, structure, level etc. of programmes.

3

  School Students Teachers Researchers Senior Managers in Schools

Main aims and objectives

- Understanding of purpose of research at University

- Understanding about the research topic relating to the sub-project

- Understanding about intentions to study at HE and engage in future research

- Understanding of purpose of research at University

- Confidence in leading their own research projects within school

- Views on students’ interest in attending HE, engagement with research and researchers

- Views on benefits for students, school and personal being involved in project

- Challenges and barriers of involvement

- Perceived impact of project on students, school and personal

- Views on working with school students- View on personal barriers and concerns

with engagement to project- Views on impact of interactions to

students and teachers- Personal impact of interactions - Views on how student and teacher

feedback is influencing development/review of activities

- Views on benefits for school, teachers and students being involved

- Perceived impact of project on students, school and teachers involved

Methodologies - Questionnaires to all participating students (pre and post interactions)

- Focus group of 5-6 randomly selected students from each participating school

- Questionnaires to all lead contact teachers from each school (pre and post interaction)

- Interview with each contact teacher

- Interview with teachers who withdraw from project

- Questionnaires to all researchers- Focus group - Exit interviews for any researchers who

withdraw from project

- Interviews (towards end of project 2017)

Timeline of evaluation interaction

- Pre-interaction questionnaire to be completed before first interaction with researchers

- Post-event interaction questionnaire

- Focus group session up to 2months after the first interaction

- End of year/full interaction celebration event questionnaire to be completed at end of year celebration event

- Pre-interaction questionnaire to be completed on agreeing to be part of project

- Interview session up to 2 months after first interaction

- Post-interaction questionnaire to be completed at end of year celebration event

- Exit interview should school withdraw from project

- Pre-interaction questionnaire- Focus group session up to 2 months

after first interaction- Post-interaction questionnaire to be

completed at end of year celebration event

- Exit interview should researcher withdraw from project

- Initial interview to coincide with first interaction/school signing up to project

- Final interview to take place after last interaction in year 3 of project

4

• Focus of questionnaires/ nature of questions• Lost PhD student• Programmes didn’t follow planned structure due to planning process.• logistics involved with paper questionnaires• Pupil/ parental consent• Difficulty getting data from researchers• Celebration event questionnaire

• Useful data on impact of project on academic aspirations and enjoyment of science at school.

• Useful feedback and positive comments from pupils.• Focus groups particularly useful in lieu of questionnaires.

Issues encountered

Successes

5

Original pupil questionnaire New pupil questionnaire

• Have you visited the University of Southampton before? If yes, please describe the activity.

• Does anyone in your family have a degree? If yes, please state who?

• How interested are you in attending university?

• How do you feel about learning science at school?

• How interested are you in studying science subjects after your GCSEs?

• How would you rate your confidence in leading your own scientific research in school?

• Do you think scientific research is used by companies or the general public?

• What do you hope to find out during the visit today?

• Have you visited the University of Southampton with your school before?

• What do you think the term ‘research’ means?

• Why do you think universities carry out research?

• Do you think the scientific research that happens at the University of Southampton is directly relevant to your everyday life? Please explain your answer.

• How confident are you at carrying out scientific investigations at school?

Use focus groups to unpick ideas they have related to themselves as researchers.

6

• Questions to focus on information required• Questionnaires and consent forms online (will still need parental consent forms for pupils).

• Nature of questions? (focus for researcher and teacher questionnaires).• Parental consent?• Long term impact?• Senior management?• Getting full data sets (particularly researchers).

Moving forward…

Feedback…