2014 annual evaluation report for the Teach First NZ ... second annual evaluation report of the Teach First NZ programme pilot delivered in partnership with the ... Appendix 6: Cohort

2014 Annual Evaluation Report for the Teach First NZ programme pilot delivered in partnership with

the University of Auckland Jenny Whatman, Jo McDonald

and Liesje Stevens NZCER

ISBN: 978-0-478-16584-5 (print)

ISBN: 978-0-478-16585-2 (web)

RMR-1073

© Ministry of Education, New Zealand 2015

Research reports are available on the Ministry of Education’s website Education Counts: www.educationcounts.govt.nz/publications. Opinions expressed in this report are those of the authors and do not necessarily coincide with those of the Ministry of Education.

Teach First 2014 Annual Report i

2014 Annual Evaluation Report for the

Teach First NZ Programme Pilot Delivered in Partnership

with the University of Auckland

Jenny Whatman, Jo MacDonald, and Liesje Stevens

ii Teach First 2014 Annual Report

New Zealand Council for Educational Research

P O Box 3237

Wellington

New Zealand

Teach First 2014 Annual Report iii

Acknowledgements

This second annual evaluation report of the Teach First NZ programme pilot delivered in partnership with the University of Auckland is funded by the Ministry of Education, the University of Auckland, and Teach First NZ.

The New Zealand Council for Educational Research acknowledges Ministry of Education personnel for their ongoing involvement in the evaluation.

We also acknowledge staff from the Teach First NZ partnership who enabled very easy access to schools and readily provided essential resources and information.

Our peer reviewer Andrew McConney from Murdoch University, Perth, Western Australia, provided valuable feedback. We also acknowledge the input of the Evaluation Reference Group, whose members comprise representatives from partner schools, the New Zealand Post Primary Teachers Association, the Secondary Principals’ Association of New Zealand, the New Zealand School Trustees Association, and the New Zealand Teachers Council.

As evaluators we were made very welcome in schools and by the Teach First NZ partnership. We appreciated their interest and the time they gave.

iv Teach First 2014 Annual Report

Teach First 2014 Annual Report v

Table of Contents

Acknowledgement .................................................................................................................. iii

Glossary of abbreviations ..................................................................................................... ix

Executive Summary ................................................................................................................ 1

Background ................................................................................................................................ 1 Data collection ........................................................................................................................... 2 How well (effectively and efficiently) has the programme been implemented? ......................... 2

The participants ..................................................................................................................... 3 The programme: Teach First NZ and Faculty components ................................................... 4 The programme: School components .................................................................................... 5 Viability of programme ........................................................................................................... 5

To what extent has the programme achieved its overall outcomes and objectives? ................. 6 Effectiveness of participants’ teaching ................................................................................... 6 Student engagement.............................................................................................................. 6 Support by participants for pastoral life of school .................................................................. 6 Leadership Development Strand ........................................................................................... 7 Programme impact on quality of teaching and learning in participating schools and the status of teaching ................................................................................................................... 7

Methodology ............................................................................................................................ 9

Purpose and scope of the evaluation ........................................................................................ 9 Data collection and analysis ...................................................................................................... 9

Evaluation question 1: How well (effectively and efficiently) has the programme been implemented? ............................................................................................................... 13

The 2014 cohort ....................................................................................................................... 13 Who the programme attracted (Cohort 14 only) ...................................................................... 15 The selection process (Cohort 14 only) ................................................................................... 16 Summer Initial Intensive (Cohort 14 only) ................................................................................ 18

Most useful aspects ............................................................................................................. 18 Improvements ...................................................................................................................... 19 Preparation for teaching....................................................................................................... 19

Retention rates ........................................................................................................................ 20 Cohort 13 ............................................................................................................................. 20 Cohort 14 ............................................................................................................................. 20

Programme factors .................................................................................................................. 21

vi Teach First 2014 Annual Report

Cohort 13 ............................................................................................................................. 21 Cohort 14 ............................................................................................................................. 22

Support for participants from the Teach First NZ partnership ................................................. 23 Cohort 13 ............................................................................................................................. 23 Cohort 14 ............................................................................................................................. 23

Support for participants from host schools (Cohort 13 and 14) ............................................... 24 Cohort 14 participants’ orientation to school ....................................................................... 24 Quality of school support for participants ............................................................................ 25 Observations of others’ teaching (Cohort 13 and 14) .......................................................... 27

Host schools preparation for, and support in, their roles (Cohort 14) ...................................... 27 Teach First NZ support for, and communication with, host school ...................................... 27

Affiliate schools (Cohort 13) .................................................................................................... 28 Mentor teachers ....................................................................................................................... 29

Cohort 13 ............................................................................................................................. 30 Cohort 14 ............................................................................................................................. 30 Observation of participants .................................................................................................. 31

Learning area specialists and visiting teaching specialists...................................................... 32 Cohort 13 ............................................................................................................................. 32 Cohort 14 ............................................................................................................................. 32

Programme responsiveness .................................................................................................... 33 Viability .................................................................................................................................... 34

Cohort 13 ............................................................................................................................. 34 Cohort 14 ............................................................................................................................. 34

Evaluation question 2: To what extent has the programme achieved its overall outcomes and objectives? .................................................................................................... 37

Effectiveness of participants’ teaching .................................................................................... 37 Cohort 13 ............................................................................................................................. 38 Cohort 14 ............................................................................................................................. 38 Relationships with students ................................................................................................. 40 Student engagement: Results from the Me and My Class survey ....................................... 40

Support by participants for pastoral life of school .................................................................... 42 Leadership Development Strand ............................................................................................. 43

Cohort 13 ............................................................................................................................. 43 Cohort 14 ............................................................................................................................. 44

Programme impact on quality of teaching and learning in participating schools and the status of teaching .................................................................................................................... 44

Discussion .............................................................................................................................. 47

How well (effectively and efficiently) has the programme been implemented? ....................... 47 The participants ................................................................................................................... 47 The programme ................................................................................................................... 48 The schools ......................................................................................................................... 49

To what extent has the programme achieved its overall outcomes and objectives? ............... 50

Teach First 2014 Annual Report vii

Programme aspects ................................................................................................................. 54 Purpose and scope of the evaluation ...................................................................................... 57 Key evaluation questions ......................................................................................................... 57 Evaluation approach ................................................................................................................ 58

Document analysis............................................................................................................... 64 Cohort 13 surveys ................................................................................................................ 64 Cohort 14 interviews ............................................................................................................ 65 Interviews with Teach First NZ and University of Auckland Faculty .................................... 65 Me and My Class Survey administration.............................................................................. 66

Analysis of data ....................................................................................................................... 67 Documentation ..................................................................................................................... 67 Cohort 13 surveys ................................................................................................................ 67 Cohort 14 interviews ............................................................................................................ 67 Interviews with Teach First NZ and University of Auckland Faculty .................................... 67 Me and My Class survey...................................................................................................... 68 Indicative subquestions........................................................................................................ 70

Tables

Table 1 Data collection for Cohorts 13 and 14 .............................................................................. 2 Table 2 Online survey responses 2014 for Cohort 13 ................................................................. 10 Table 3 Interviewees August–October 2014 for Cohort 14 ......................................................... 10 Table 4 Me and My Class survey data ........................................................................................ 11 Table 5 Participant placements 2013 and 2014 .......................................................................... 14 Table 6 Cohort 13 destinations in 2015 ....................................................................................... 35 Table 7 Evaluation question 1, evaluation criteria and data sources .......................................... 59 Table 8 Evaluation question 2 and intended data sources .......................................................... 62 Table 9 Online survey responses 2014 for Cohort 13 ................................................................. 65 Table 10 Interviewees August–October 2014 for Cohort 14 ......................................................... 66 Table 11 Me and My Class Survey data........................................................................................ 68 Table 12 Larger differences between Cohort 13 and 14 Year 9 classes....................................... 69 Table 13 Larger differences between Cohort 13 and 14 Year 10 classes..................................... 69 Table 14 Ministry of Education indicative evaluation questions .................................................... 70

viii Teach First 2014 Annual Report

Appendices

Appendix 1: Programme purpose ............................................................................................... 53 Appendix 2: Methodology ........................................................................................................... 57 Appendix 3: Me and My Class data ............................................................................................ 69 Appendix 4: Ministry indicative evaluation questions ................................................................. 70 Appendix 5: Cohort 13 survey questions .................................................................................... 72 Appendix 6: Cohort 14 interview questions .............................................................................. 103 Appendix 7: Interview Schedule – Partners ............................................................................. 108 Appendix 8: Me and My Class Survey...................................................................................... 109

Teach First 2014 Annual Report ix

Glossary of abbreviations

Affiliate Schools Schools that support Teach First NZ and provide “away practicums” in Year 2 for participants in schools in another context (mid to high decile)

Cohort 13 Participants who began the programme in 2013

Cohort 14 Participants who began the programme in 2014

FTE Full-time equivalent

GTS Graduating teacher standards

HoD Head of department

ITE Initial teacher education

LAT Limited authority to teach

LAS Learning area specialist (provides Faculty curriculum expertise)

NCEA National Certificate of Educational Achievement

PB4L Positive Behaviour for Learning

PLD Professional learning and development

PRT Provisionally registered teacher

RTC Registered teacher criteria

SII Summer Initial Intensive

SOAR School Observation and Reflection tool

VTS Visiting teaching specialist

x Teach First 2014 Annual Report

Teach First 2014 Annual Report 1

Executive Summary

This 2014 evaluation confirms that, in general, the Teach First NZ programme has been implemented effectively and efficiently, and that it has benefited rather than suffered from doubling in size. Feedback from 2013 has provided the base from which to improve the programme in 2014. Teach First NZ and the University of Auckland’s Faculty of Education continue to find ways to strengthen the programme and to ensure it is well known and well supported. Participants are very strong ambassadors for the programme, including the mission of reducing educational inequalities, and for secondary teaching in New Zealand. Almost all participants have achieved highly, have supported their students to do well, including in NCEA, and intend to stay in teaching at least in the short term.

Key success elements of the Teach First NZ programme are:

• the robust selection process resulting in high-calibre participants

• the responsiveness of the programme, in part made possible because of the small size of the programme, but also a consequence of the robust partnership between Teach First NZ and the Faculty of Education (hereafter referred to as the Teach First NZ partnership)

• effective support and mentoring for participants from the school and partnership personnel

• immersion in the classroom, coupled with opportunities for participants’ critical reflection on themselves and their teaching.

The successful implementation of all these elements is critical for an employment-based programme. While there is variability in the way these elements play out, particularly in relation to mentoring and in-school support, in combination they provide a powerful platform for this model of an alternative pathway into secondary teaching.

Background This report is the second annual evaluation report for the Teach First NZ pilot programme.1 The first report focused on the programme’s first year, Cohort 13.2 This second report focuses on Year 2 for Cohort 13 and Year 1 for Cohort 14. The main purpose of the 4-year evaluation of the programme is to understand how well the programme has been implemented, and to what extent it has achieved its objectives. The key evaluation questions are:

1. How well (effectively and efficiently) has the programme been implemented?

2. To what extent has the programme achieved its overall outcomes and objectives?

The Teach First NZ pilot programme is an alternative field-based initial teacher education (ITE) programme.3 The pilot phase operates between 2013 and 2016 with three annual intakes of up to 20 participants. The third and final intake is in

1 Hereafter known as “the programme”. 2 The report: 2013 Annual Evaluation Report for the Teach First NZ Pilot Programme Delivered in Partnership with the

University of Auckland can be found at http://www.educationcounts.govt.nz/publications/schooling/146589

3 A full description of the Teach First NZ pilot programme can be found in Appendix 1.

http://www.educationcounts.govt.nz/publications/schooling/146589

2 Teach First 2014 Annual Report

2015. The programme is administered by Teach First NZ in partnership with the Faculty of Education of the University of Auckland (the Faculty). Some aspects of the programme are funded by the Ministry of Education (the Ministry). After a summer intensive, participants teach in schools serving low socioeconomic communities on a 0.6 loading for 2 years on a limited authority to teach (LAT). After successful completion of 2 years’ teaching, participants are provisionally registered.

Data collection The evaluation data for the second year of the Teach First NZ pilot programme were collected differently for the two cohorts, as shown in Table 1.

Table 1 Data collection for Cohorts 13 and 14

Cohort 13 Online surveys: participants, mentors, co-ordinators, principals

Online surveys: learning area specialists

Cohort 14 Site visits and interviews in 13 schools: participants, their mentors, coordinators

Principals new to the programme in 2014

Summer initial intensive in December and January

Me and My Class survey (Year 9 or 10) with comparison class

Interviews with Teach First NZ and Faculty staff

Mentor meeting and coordinator meeting

How well (effectively and efficiently) has the programme been implemented? We looked at a number of indicators to assess the extent to which the programme had been effectively implemented, including: the participants themselves (their recruitment and selection, and their intentions after the pilot programme); programme factors (such as the Summer Initial Intensive (SII)), termly clinics and workshops, Faculty papers, and support provided by the Teach First NZ partnership); and school factors (support for the participant by the school, in particular by their mentor). In 2014 we also evaluated the viability4 of the pilot programme.

In general, the programme has been effectively and efficiently implemented for Cohorts 13 and 14. Feedback from internal and external evaluations in 2013 led to changes made to the taught programme between 2013 and 2014, which included:

• improvements to recruitment and selection to better cater for Māori applicants

• adjustments to the SII to better prepare Cohort 14 for the realities of beginning teaching

• subtle and valued changes to workshops, clinics, and the noho marae

4 We considered that the programme would be viable if the Teach First NZ model can successfully adjust to larger numbers of participants within agreed funding arrangements and if schools are able to support the programme when they self-fund participants’ salaries from 2014.


• a change in focus and responsibility for staff employed by the Faculty with the differentiation between learning area specialists (LASs) and contracted visiting teaching specialists (VTSs).

Seven new schools joined the programme in 2014. Changes to school-based components of the programme have improved communication between the Teach First NZ partnership and schools, and expanded the role for host school co-ordinators. Additional elements potentially impacting most on Cohort 13 have been the ongoing development of the Leadership Development Strand and the 3-week “away practicum”. Participants and schools reported positively on the leadership strand and on Cohort 13’s independent inquiry. Since only four participants had completed their “away practicum” before the end of November 2014, we have not been able to include evaluative data about this element in the 2014 report.

The participants As in 2013, the programme was successful in attracting high-quality applicants. Twenty participants (7.3 percent of those who filled in the online application) were selected for the 2014 cohort. Almost a third of the 2014 intake are Māori or Pasifika (seven compared with four in 2013) and 37 percent are male (seven compared with four in 2013). Seven5 majored in English, two in te reo Māori, five in mathematics, and five in chemistry or physics (compared with one teaching chemistry in 2013). Fourteen have postgraduate qualifications or doctorates (compared with six in 2013). The evaluation team talked to different groups of people and, as a result, perceived Cohort 13 participants to be high calibre.

Retention rates The retention rate of participants has exceeded expectations. A very pleasing outcome of the programme is that all but one of the original 2013 cohort of 16 were still in the programme in November 2014, and all but two will be teaching in New Zealand schools in 2015. Four of the 15 participants have been retained as provisionally registered teachers (PRTs) for 2015 by their host school and most of the others have secured teaching jobs in Auckland, albeit not all in schools serving low socioeconomic communities. Some host schools told us that they would have liked to have offered participants a job but could not employ additional staff because they were in a falling roll situation. One will be living overseas and plans to teach, and one is travelling overseas but will return to a job in an Auckland school in 2016.

All but one of Cohort 14 were still in the programme in November 2014 and all expect to complete their 2 years of teaching. At this stage four-fifths of each of Cohort 13 and 14 participants expect to stay in teaching beyond their 2-year placement, as they work towards full teacher registration.

Teach First NZ partnership support for participants The majority of Cohort 13 participants considered that they were very well supported by the Teach First NZ partnership. They also reported being very connected to the other participants in their cohort, and many of them said they also felt connected to Cohort 14 participants. We did not specifically ask Cohort 14 participants about how well supported they were by Teach First NZ. All Cohort 14 participants said that they felt part of the Teach First NZ community, although it was clear that this meant different things to different people.

Mentor support to participants Mentor support to participants was varied, although most participants in both cohorts were positive about their mentor and in-school support. Mentoring is a critical component of the programme, and ineffective mentoring can impact

5 One English participant left the programme in the first term.


negatively on participants’ learning and therefore on the overall quality of the programme. The coordinator role has been strengthened in 2014, in part to monitor or enhance mentor support. Participants’ exposure to school-based professional development and learning (PLD) varied considerably, as did their opportunities to observe their mentor teaching.

Selecting the right mentor for each participant has not always been easy, and schools appear to be weighing up a number of factors before assigning a mentor to a participant. About half of the 2014 mentors for Cohort 13 had not been the participant’s mentor in 2013, and several mentors for both cohorts are not teaching the same subjects as the participant. As in 2013, the frequency of observations, and mentors’ individual approaches to them, varied as did the extent and nature of the feedback given.

The programme: Teach First NZ and Faculty components Selection As was reported for Cohort 13, the selection process was identified as a key strength of the programme. Participants reported that they found the selection process “rigorous”, “robust” and “intense and full on—good preparation for the SII and for teaching”. Others interviewed (principals, co-ordinators and mentors) also saw the selection process as a strength as it meant high-calibre people who were most likely to succeed were selected. The recruitment and selection team worked closely with the Faculty to improve the process to better meet the needs of Māori applicants.

Programme elements Participants reported that they felt well prepared by the SII, as did Cohort 13 in the previous year. The Teach First NZ Partnership made several adjustments to the SII based on feedback from the Cohort 13 experience. There were more opportunities for Cohort 14 to practise administrative aspects of teaching, and the final week of the SII focused on being “school-ready”. Participants considered that Teach First NZ and the Faculty had been very responsive to their feedback and noted that small beneficial changes had been made to the SII and the ongoing professional development clinics (1 and ½ days 4 times a year). Cohort 14 participants reported that the SII strengthened their cultural understandings and they endorsed a strong sense of connectedness with their cohort. The residential aspect continues to be a challenging—and valuable and rewarding—aspect of the SII.

For the programme to be successful, it needs to: have a common clear vision of effective teaching which permeates course work and practice in schools; have a strong core curriculum; apply case methods and teacher research; allow participants to confront and rethink assumptions about learners; be judged by the participants as intellectually challenging and practically relevant. Participants commented on the relevance of their university papers and assignments as a strength of the programme. For Cohort 13,—the Teach First NZ Leadership Project and a Faculty professional inquiry project—were considered to be very useful for their own learning and often seen to add value to the school. Participants were also positive about the clinics and noho marae held during the year, acknowledging the valuable time with curriculum specialists and an opportunity to reconnect and “reinvigorate” with other participants. Most participants valued the support from, and approachability of, Teach First NZ personnel and Faculty LASs/VTSs.

Mentor teacher preparation Mentors were generally pleased with the training they had received, and with their role. Some found greater value than others in the support and training provided by the Faculty. Their perception of value usually depended on what other mentoring PLD they had undertaken, and their mentoring experience. Many mentors commented on their own learning from enacting the role, or discussions with Faculty specialists, or the initial training and how it had energised them as a teacher.


The programme: School components Support for participants from host schools All Cohort 13 participants told us that they felt fully accepted as a staff member at their school. Cohort 14 reported positively on the initial orientation that schools provided. Nearly all Cohort 14 participants were very positive about how the school and staff had responded to them, with other staff treating them as if they were PRTs.

Preparation and support for host schools’ roles The vast majority of mentors, co-ordinators and principals perceived the liaison between the Teach First NZ partnership and themselves to be effective. Some co-ordinators who had been involved since 2013 suggested liaison had improved from 2013 to 2014 and referred to the responsiveness of Teach First NZ personnel, saying they are “very open to feedback and adjusting to meet our needs” or “Teach First NZ took on board our suggestions”. The role of co-ordinator continues to be refined and plays out differently in different schools.

Viability of programme We considered that the programme would be viable if the Teach First NZ model could successfully adjust to larger numbers of participants within agreed funding arrangements, and if schools are able to support the programme when they self-fund participants’ salaries from 2014. There does not appear to have been an issue for 2014 schools in funding Cohort 14 (who, unlike Cohort 13, are not supernumerary). In 2014, one decile 5, two decile 4, and two decile 3 schools were added6, all schools with large rolls and high proportions of Māori and Pasifika students. While the vast majority of principals in the pilot programme to date would hire further Teach First NZ participants if they could, school personnel identify ongoing challenges, often in relation to funding and the expectations of schools.

Placing all participants in well-functioning departments with mentors who are able to support participants’ learning effectively has been a challenge for the programme in a small number of schools and in curriculum areas other than English for a small number of participants. Other issues identified cannot be addressed solely by Teach First NZ adapting the model and relate to the fit of the programme within New Zealand’s existing ITE system. Schools that are small, or have falling rolls, or both, may experience funding or staffing barriers to taking on Teach First NZ participants. These schools are also more likely to be serving low socioeconomic communities. This factor could also affect the programme’s goal of placing its participants in schools serving low-income communities and into schools which have high numbers of Māori and Pasifika students.

Two New Zealand Teachers Council requirements have been challenges for the programme, a number of Cohort 13 participants, and some schools. Participants in their second year were required to do a 3-week practicum in an “affiliate” school. However, for most participants, the Teach First NZ’s original intentions for the practicum (that the “away practicum” would also include a teacher exchange programme) were not realised. Commitments in their own schools meant most participants were unable to complete the practicum until the end of the school year and had limited opportunities to be involved in their affiliate school. Participants were also required to complete 2 years of teaching with limited authority to teach (LAT) before they gain provisional registration. All new teachers are provisionally registered for at least 2 years.

6 At the end of 2014, the Ministry of Education released new decile ratings for all schools in New Zealand. Decile ratings for four host schools changed. Table 5 on page 6 shows these changes. Throughout the report we refer to the decile ratings that were available to us during the evaluation. The 2015 evaluation will use the new decile ratings.


Just over half of Cohort 13 have been employed in mid-high decile schools in 2015. If this pattern becomes typical, it may deter schools from participating in Teach First NZ in future because they might perceive that other schools will benefit from their work supporting participants to learn to teach. However, almost all schools reported the benefits of taking part in the programme, irrespective of where the participants then went on to teach.

To what extent has the programme achieved its overall outcomes and objectives? The second evaluation question asks to what extent the programme achieved its overall outcomes and objectives. To answer this question, the evaluation focused on: the effectiveness of participants’ teaching, their level of support for the pastoral life of the school, the leadership development strand of the programme, the ongoing involvement and/or retention of participants, programme impact on quality of teaching and learning in participating schools, status of teaching, and enablers and barriers to success.

Effectiveness of participants’ teaching Participants were valued for their very strong subject knowledge, their facility with digital technologies, and for their perseverance and “can do” attitude. Everyone, including participants, thought that they would meet the graduating teacher standards (GTS). Everyone surveyed about Cohort 13 considered that participants had made considerable progress and were more confident, had stronger relationships with students, and were playing a stronger role in their department and often in the school than they had done in their first year.

Student engagement The Me and My Class survey suggests that student engagement was stronger for Cohort 13 than Cohort 14, although these data should be treated with caution. This outcome would be expected given that Cohort 13 has had a year longer teaching in their schools. According to school personnel we talked to or surveyed, almost all first-year participants compared favourably with student teachers and PRTs by September. Participants were usually regarded as PRTs by other staff. Being treated as a PRT was difficult for some Cohort 14 participants at the beginning of the year, as mentors and other staff did not always realise how much support they needed with lesson planning and behaviour management. For Cohort 13, all co-ordinators and principals considered that students responded either very positively or positively to the participants. For Cohort 14, although a few co-ordinators and mentors felt that participants’ relationships with students were still developing, many commented on the impact participants were having with students, academically and in a pastoral role. Overall, students’ responses to participants were described as positive, or “what you’d expect from any teacher”.

Support by participants for pastoral life of school Participants reported being involved in a variety of aspects of school life, with participation in revision for NCEA and scholarship and homework centres being most common. While this could be seen to be an extension of the academic rather than pastoral life of a school, it allowed participants to further develop relationships with students from their own classes and throughout the school. Smaller numbers of participants were also involved in sports coaching, Education Outside the Classroom (EOTC), Polyfest, kapa haka, other cultural events, and Positive Behaviour for Learning (PB4L) School-Wide work.


Leadership Development Strand The Leadership Development Strand is a key element of the programme. The aim is to produce teachers who will, in the long term, provide “a network of leaders in education and across all fields, who are committed to addressing educational inequality”. All but two of the Cohort 13 participants thought they had been able to take on leadership roles in 2014, and this was confirmed by other survey respondents. Participants considered that support from other school personnel and the Teach First NZ Leadership Project had enabled their contribution to leadership. The Leadership Project was seen by a number of participants and other staff members as having a significant impact on the school. Cohort 14 participants in many schools were taking a form class or serving as academic mentor for a small group of students. Others were showing leadership in their department.

Programme impact on quality of teaching and learning in participating schools and the status of teaching All Cohort 13 co-ordinators and principals considered that participants had had either a “high positive impact” or “some positive impact” on the teaching and learning in the school. Many school personnel in Cohort 14 schools noted the contribution participants made to their department and in staff meetings and their willingness to share new ideas. Three-quarters of Cohort 13 participants thought Teach First NZ had a positive impact on the perceived status of teaching as a competitive profession. A number of Cohort 14 participants said they were now more aware of the status of teaching and of how important and demanding a profession it is. There was some indication that Cohort 14 had actively encouraged friends and university colleagues to apply for the programme. Cohort 14 included more Māori, Pasifika, and male applicants than Cohort 13.



Methodology

Purpose and scope of the evaluation This evaluation’s7 key questions are:

1. How well (effectively and efficiently) has the programme been implemented? 2. To what extent has the programme achieved its overall outcomes and objectives?

The evaluation is designed to provide timely and relevant information to:

• support decision making about ongoing implementation of the programme

• develop a deeper understanding of alternative pathways into the teaching profession, particularly pathways for talented potential teachers

• generate learning about effective approaches to aspects of ITE

• understand the effectiveness of the programme in terms of its stated immediate, medium-, and long-term aims.

At this point of the evaluation, that is, halfway through the pilot programme, evaluation question 1 has a formative as well as a summative purpose.

We developed evaluation criteria and possible data sources to address both key evaluation questions. This approach was endorsed by the Teach First NZ and the Faculty partnership. The criteria are set out in Appendix 2. Interview and survey questions were based on these criteria.

Data collection and analysis We collected data from four main sources: document analysis; interviews with participants, key personnel in Cohort 14 schools and with Teach First NZ and the Faculty; an NZCER online survey (Me and My Class) for students; and online surveys for Cohort 13 participants, their mentors, co-ordinators and principals. We were provided with copies of:

• information about participants, and contact details of personnel involved

• recruitment and selection material, including Assessment Centre protocols

• programme description and course outlines, including assessments

• material provided by the Teach First NZ partnership to schools.

During site visits with Cohort 14, we also had access to:

• participants’ portfolio material—material towards meeting the GTS and the registered teacher criteria (RTC) (online and hard copy)

7 Full details of the methodology can be found in Appendix 2.


• written observations and feedback from school-based mentors and Faculty-based VTSs

• participants’ folders containing lesson plans, copies of student work, assessments, and assessment results.

Surveys for Cohort 13 respondents were emailed in October. The surveys are attached as Appendix 5. Table 1 outlines survey response rates for Cohort 13.

Table 2 Online survey responses 2014 for Cohort 13 Respondent Number returned Percentage

Participant 13 87

Principal 3 60

Co-ordinator 9 100

Mentor 15 (3 mentors mentor two participants; 3 participants have 2 mentors; some mentors are also co-ordinators)

87

LAS 5 100

Total 46

Because of the small sample size, survey data was analysed according to frequencies and we did not do further analysis (for example, cross-tab analysis). Each respondent group answered many of the same questions, and we constructed tables that showed these comparative responses.

Cohort 14 interviews were conducted at the 13 schools between 25 July and 9 September, with interviewers spending one half to one full day in each school. The interview schedules are attached as Appendix 6. In October 2014 we interviewed five Teach First NZ staff and three Faculty staff. The interview schedule is attached as Appendix 7. All together we conducted interviews with 64 people (see Table 4). We had a very high response rate of almost 100 percent including principals.

Table 3 Interviewees August–October 2014 for Cohort 14 Interviewee Number interviewed Percentage

Participant 19 100

Principal 8 (seven schools new to the programme in 2014 plus one new principal in a school with cohort 13 and 14 participants; one interview was with a deputy principal)

89

(plus one deputy principal)

Co-ordinator 10 (two co-ordinators were also mentors and their interview were recorded there; one co-ordinator was the principal and the interview was recorded there)

100

Mentor 19 (one participant had two mentors; one mentor was not able to be interviewed) 95

Teach First NZ partnership personnel

8 100

Total 64

Interviews were recorded, but not transcribed. The interview schedules included a wide range of questions to elicit people's views and experiences of the Teach First NZ programme. Responses relevant to each theme, regardless of the question, were independently coded and cross-checked by two researchers and collated in a spreadsheet, by role. This


process made it possible to gain a reliable thematic overview of participants’ responses across the conversational style of interview and allowed for greater breadth of discussion to be included in the analysis.

Interviews with the five Teach First NZ staff and three University of Auckland staff were summarised and analysed thematically. The focus of these interviews was on changes made between 2013 and 2014 and any particular challenges that the team had met and envisaged for the future. Some of these data were directly comparable to the data collected through the Cohort 13 surveys and Cohort 14 interviews.

Me and My Class is a survey8 designed by NZCER for Years 4 to 13 students. It explores students’ perspectives on learning in their classroom. The survey was administered by NZCER’s Products and Services Assessment Team in September 2014, 8 months after Cohort 14 participants began teaching and near the end of Cohort 13’s 2-year programme. We obtained results from the Me and My Class survey for 299 Year 9 or 10 classes (out of 34 participants from 14 schools) and for 29 comparison classes.

Table 4 Me and My Class survey data Class surveyed Number of students surveyed

Cohort 13 & 14 total Year 9 32010

Comparison class Year 9 193

Cohort 13 Year 9 49

Cohort 14 Year 9 192

Cohort 13 & 14 total Year 10 31811




For the purposes of this evaluation, the aggregated and anonymised results were used to assess the student perceptions of participant teachers’ effectiveness in providing pedagogically rich learning opportunities and in engaging students in their learning. We would strongly caution against too much emphasis being placed on the results of the survey in 2014. Factors to consider are that we do not have data for all participants, we have data for more than twice as many participants’ students as comparison students, and that this was the first time many students had attempted this survey and they may have been exposed to different instructions and conditions.

8 The survey is attached as Appendix 8. 9 Some participants only had results for senior classes and we did not include these in our analysis. 10 Note that many participants had results for more than one Year 9 or Year 10 class. To compare participant and comparison

classes, we randomly chose one Year 9 or one Year 10 class. This means that the total of Cohort 13 Year 9 is not the same as Cohort 13 and 14 Year 9.

11 Note that many participants had results for more than one Year 9 or Year 10 class. To compare participant and comparison classes, we randomly chose one Year 9 or one Year 10 class. This means that the total of Cohort 13 Year 10 is not the same as Cohort 13 and 14 Year 10.



Evaluation question 1: How well (effectively and efficiently) has the programme been implemented?

Key findings

The programme has been, in general, effectively and efficiently implemented for Cohorts 13 and 14. A number of changes were made to the taught programme between 2013 and 2014 based on feedback from internal and external evaluations in 2013. Seven new schools joined the programme in 2014. Changes to school-based components of the programme have improved communication between the Teach First NZ partnership and schools; and expanded the role for host school co-ordinators. In 2014, mentors and co-ordinators reported more positively on their training than in 2013. In-school support and mentoring remains variable. Additional elements potentially having most impact on Cohort 13 have been the development of the Leadership Development Strand and the 3-week “away practicum”.

This section provides an overview of the 2014 cohort and provides evidence for the evaluation criteria for evaluation question 1 (where applicable for Cohort 13 and 14), following the same order as the key findings. Evaluation question 2 is answered in the following section.

The 2014 cohort Teach First NZ reported that 7.3 percent of the people who applied for the programme were accepted for the 2014 cohort. Twenty of the 274 applicants who completed the online application form were selected to participate in the 2014 programme. (The selection process also includes a telephone interview and an assessment centre day.) The 2014 participants were placed in six deciles 1–2 schools, four decile 3, and three deciles 4–5 schools located in Northland (2), East Auckland (1), South Auckland (5), Central South Auckland (2), and West Auckland (3). Of the 16 schools participating in the programme in 2014, three are hosting Cohort 13 participants only, six have Cohort 13 and Cohort 14 participants, and seven are participating in the programme for the first time with Cohort 14 participants only (see Table 5).


Table 5 Participant placements 2013 and 2014 Participating Schools (Decile)

Cohort 13 subject placements

Cohort 14 subject placements

Total number of participants in each

school as at 2014

Mangere College (1) English

Maths

2

Okaihau College (2) Te reo Māori 1

Sir Edmund Hillary Collegiate (1) English

Chemistry

2

Alfriston College (3, revised to 2) English English

Physics

3

Papatoetoe High School (3) English

Maths

Chemistry 3

Southern Cross Campus (1) English English 2

Tamaki College (1) English

English

Maths 3

Tangaroa College (1) English

Maths

Chemistry 3

Tikipunga HS (2) Te reo Māori

English

Physics 3

Aorere College (2) English

Maths

2

Kelston BHS (4 revised to 3) English

English

2

Massey HS (5 revised to 4) English

Te reo

2

Northland College (1) Te reo Māori 1

Onehunga HS (4) English

Maths

2

One Tree Hill College (3) English

Maths

2

Waitakere College (3) Maths 1


Figure 1 Location of 2013 and 2014 participants’ schools12

Who the programme attracted (Cohort 14 only) Key findings The pilot programme had negotiated parameters. Within these boundaries, one aspect of quality we judged was the extent to which the pilot programme attracted high-quality applicants, some of whom may not otherwise have undertaken teaching at this time (particularly in schools serving lower socioeconomic communities). The Teach First NZ recruitment and selection team made adjustments for 2014 to help target and recruit higher numbers of males, Māori and Pasifika participants, and more science majors than were recruited in 2013.

As in 2013, the programme was successful in attracting high-quality applicants. Twenty participants of those who applied online (7.3 percent) were selected for the 2014 cohort. Almost a third are Māori or Pasifika (seven compared with four in 2013) and 37 percent are male (seven compared with four in 2013). Fourteen have postgraduate qualifications or doctorates (compared with six in 2013). About half of Cohort 14 had decided to become teachers but, as with Cohort 13, they were attracted to the Teach First NZ programme for a range of reasons, most commonly the programme’s mission. The financial incentive was also a factor for almost half of them.

12 PowerPoint slide supplied by Teach First NZ.

Massey HS Decile: 5

Waitakere College Decile: 3

Kelston Boy’s HS Decile: 4

Northland College Decile: 1

Tikipunga HS Decile: 2

Okaihau College Decile: 2

Tamaki HS Decile: 1

One Tree Hill HS Decile: 3

Onehunga HS Decile: 4

Southern Cross Decile: 1

Aorere College Decile: 2

Alfriston College Decile: 3

Tangaroa College Decile: 1

Papatoetoe HS Decile: 3

Sir Edmund Hillary Decile: 1

Mangere College Decile: 1

- 100% participant retention in 2013 (16/16 participants completed the year) - 95% participant retention in 2014 (34/36 participants still teaching)


As with the 2013 cohort, participants had heard about Teach First NZ through a number of avenues (some participants gave more than one response).

• Seven heard by word of mouth from family and friends.

• Six learned about Teach First NZ online or in other media either while looking for jobs, courses, and scholarships or through news articles.

• Six had it suggested by a lecturer or saw notices around campus and then attended a presentation by Teach First NZ.

Similar to the 2013 cohort, around half (nine) participants had already decided to become teachers (either now or at some point in the future) and would have applied to other teacher education programmes. A further four were unsure that they would have applied to a traditional initial teacher education (ITE) pathway. Teach First NZ appealed to these participants because it did not involve further lecture-based study, and they could “make a bigger difference”. If participants had not been accepted for the programme, eight would have taken a traditional ITE programme; most others would have stayed in a current job or taken other employment.

Twelve participants referred to the Teach First NZ mission or social justice or equity issues as a reason that they had chosen the Teach First NZ programme in preference to a 1-year graduate diploma. Eight participants said the financial incentive was a factor, and for five it was the opportunity to “learn on the job”. Other reasons given were being part of something new and cutting-edge; for personal growth and challenge; and leadership opportunities.

We talked to different groups of people and, as a result, perceived the participants to be high calibre. A quarter (seven compared with four in 2013) are Māori or Pasifika and over one-third (seven compared with four in 2013) are male. Seven are English majors, two te reo Māori, five mathematics, and five chemistry or physics (compared with one teaching chemistry in 2013). Cohort 14 participants are highly qualified, with three having PhDs and 11 having postgraduate qualifications (compared with six of Cohort 13).

While mentors did not always know participants’ personal motivations for applying for the programme, they did describe some characteristics of the participants. They frequently referred to the quality of the participants, which they put down partly to the very rigorous selection process. They described the participants as enthusiastic, energetic and with a strong sense of commitment and responsibility to their students. They saw them bringing considerable skill and experience, not only academic, but also workplace and life experience.

The selection process (Cohort 14 only) Key findings As was also reported for Cohort 13, the selection process was identified as a key strength of the programme. Participants reported that they found the selection process “rigorous”, “robust” and “intense and full on—good preparation for the SII and for teaching”. Others interviewed (principals, co-ordinators and mentors) also saw the selection process as a strength as it meant high-calibre people who were most likely to succeed were selected.

Participants reported that they found the selection process “rigorous”, “robust” and “intense and full on—good preparation for the SII and for teaching”. They commented on the length and number of aspects involved in the process, including telephone interviews, the assessment centre interviews, model lesson demonstration, and a group task, as well


as the occasional “informal chat over coffee”. As in 2013, the selection process was identified as a key strength of the programme.

Most, but not all, participants said that the selection process allowed them to demonstrate strengths and areas needing development. It also allowed them to get a feel for the process and confirm their own commitment to Teach First NZ as a suitable option for them. Their comments include the following.

It definitely allowed me to say my strengths, but they certainly knew how to spot the things that might be an issue.

The assessment centre was very intense and good preparation for SII and for teaching. It was an opportunity to show we could withstand the programme.

It was very comprehensive in three stages. [At the assessment centre] I was able to show a variety of skills and get a taster of what Teach First NZ was about.

The assessment centre was good—it clinched my decision to go ahead. I like these people and what they’re trying to do. They convinced me.

The few participants who disagreed felt that they didn’t get the opportunity to demonstrate teaching skills, or, as one put it, to “demonstrate strengths on [my] own terms”. One suggested that the application form was too long-winded and “it felt a bit contrived as if there was a right answer”. Nevertheless, the make-up of the cohort suggests that those selected share qualities such as resilience and a sense of commitment to make teaching work. One participant stated that “they picked widely, but well”. Another described Cohort 14 as a group of “very different people, but we share similar values”, and that she has therefore been able to make strong connections with a group of people she would otherwise not have come into contact with.

To help attract more Māori and Pasifika, as well as male applicants, suggestions for recruitment were made in the 2013 evaluation report and through Teach First NZ’s internal evaluation processes. These recruitment suggestions seem to have contributed to their increased numbers in the 2014 cohort, thus helping to fulfil Teach First NZ’s intentions.

Principals, co-ordinators and mentors also saw the selection process as a strength.

They probably choose people with the real disposition for teaching—who really want to teach—and who have a bit of life experience, and they get to select a few out of a large number of applicants so they really can choose top quality. (Cohort 14 principal)


Summer Initial Intensive (Cohort 14 only) Key findings Participants reported that they felt well prepared by the SII, as did Cohort 13 the previous year. The Teach First Partnership made several adjustments to the SII based on feedback from Cohort 13. They provided many more opportunities for Cohort 14 to practise administrative aspects of teaching and re-arranged the final week of SII to focus on helping participants become “school-ready”.

Compared with Cohort 13, fewer Cohort 14 participants said they would have liked more emphasis on behaviour and classroom management in the SII. Cohort 14 participants reported that the SII strengthened their cultural understandings and they endorsed a strong sense of connectedness with their cohort. The residential aspect continues to be a challenging, and valuable and rewarding aspect of the course.

In 2013 we asked participants several questions. How well the School Observation and Reflection (SOAR) tool prepare them for their SII? To what extent did the SII strengthen participants’ motivation to teach in low-decile schools and strengthen their understandings of the cultures of their students and how to incorporate this understanding into their teaching? How well prepared did participants feel to begin teaching? To what degree did the SII built a sense of connectedness within the cohort? Our deepening understanding of the SII, through attendance at it and feedback from Cohort 13, led us to ask different questions of Cohort 14.

In 2014 participants were asked four broad questions about the SII:

• What aspects of the SII were most useful/helpful?

• How well do you think the SII prepared you to begin teaching?

• Are there any areas where you needed more preparation?

• Now that you have been teaching for a while, what improvements to the SII would you recommend for the next intake?

Most useful aspects Participants in Cohort 14 were generally very positive about the SII. The most useful aspect of the SII was the practical application of theoretical knowledge. Consequently, the week of practicum was the most frequently mentioned aspect, along with other practical aspects including the time spent on curriculum and pedagogy. One participant summed this up:

[The] one week practicum ‘taster’ was useful, but we realised it was a teacher’s ‘dream classroom’ of students who were actually not at school but in a holiday programme. We all knew it wasn’t going to be like that in the real world, but it was nice to have that chance to practice some things—sadly not enough.

Despite sometimes being daunted by the intensity, and some initial resistance to the residential requirements, both these factors were reported as the most powerful aspects of the intensive. It gave the participants opportunities to bond as a group, to share ideas and to reflect on the experience of learning theoretical and practical aspects of teaching. Two participants described the SII:

That time was crazy! For me it was living and breathing education and pedagogical theory. If you’re going to live it, you may as well live it and breathe it.


[I] nearly wept when heard it would be halls of residence, but it was marvellous… relentless but positive.

The noho marae/kaupapa Māori experience contributed to participants’ sense of connectedness and being a part of a community.

One co-ordinator saw the SII as a particular strength of the programme, as this intense time immediately before starting in the classroom meant it was “very fresh and recent”, rather than participants having to remember topics that might have been covered at the start of a one-year traditional pathway course.

Improvements When asked about possible improvements to the SII all but one of the participants had suggestions for minor improvements. Several mentioned that they would have liked more time in school with their mentor before actually starting to teach. A few participants identified more subject specific preparation, and classroom and behaviour management, but these did not come with the same degree of urgency evident in the responses of Cohort 13 responses in 2013. A couple of participants wanted Māori educational input to be addressed as a foundational principle in the SII or more opportunities to learn about Pasifika education and interact with Pasifika students.

Preparation for teaching The majority of participants thought the SII had prepared them for teaching, and acknowledged that they covered a “lot of content in a short time”. Eleven said they felt “reasonably well-equipped” or “as prepared as they could be”. Participants were realistic about their expectations, knowing that “real” classes would be more challenging than their experience with the “holiday class” and were able to reflect on this later.

We covered all the aspects to prepare for teaching including the cultural side—understanding the inequality in NZ and learning as Māori—always surprised now that at PD sessions at school there is a feeling of “we’ve done that”.

It was a smooth transition into becoming a teacher—we were well prepared with teaching strategies and understanding psychology of the brain and why students behave the way they do, but needed a little more on subject content and teaching NCEA.

While several participants were able to identify some gaps in retrospect, they also noted that they may not have recognised the importance of them at the time. These included knowing more about NCEA, and behaviour management, including managing defiant behaviour.

Four did not feel they were well enough prepared, but acknowledged that some things could not be taught or prepared for until you start “living them out”.

I don't think it is possible to prepare. You just need to understand [that] when you get here you will flounder. That’s just the nature of the beast.

Principals, co-ordinators and mentors were not specifically asked about the SII but expressed varying views on how well prepared the participants were when they first arrived in schools. Many simply stated that the participants were “on a par” with, or “superior to” PRTs. However, although others had noticed that participants were initially less prepared than PRTs, it was their character and willingness to persevere that meant they were soon able to “catch up”. A principal commented: “At the beginning of the process they are truly challenged after just the SII. Later they’re like PRTs.” And according to a co-ordinator: “They have the drive to succeed. They’re well prepared when they come in. They’re well in the game already”.


Retention rates Key findings A successful retention rate over the 2 years of the programme for each 2-year cohort was set at 90 percent.

The retention rate for Cohorts 13 and 14 has exceeded this expectation. All but one of Cohort 13 are still in the programme in November 2014. Thirteen of the 15 will be teaching in New Zealand in 2015, four of them in their host schools. (Some other host schools would have liked to employ their participant/s but were unable to employ additional staffing because of falling rolls.)

All but one of Cohort 14 are still in the programme in November 2014 and all expect to complete their 2 years of teaching as a LAT. At this stage four-fifths of each of Cohort 13 and 14 participants expect to stay in teaching beyond their 2-year placement, as they work towards full teacher registration.

Cohort 13 Fifteen of the 16 participants in Cohort 13 were still in the programme in November 2014. Four of the 15 participants have been retained as PRTs for 2015 by their school and most of the others have secured teaching jobs in Auckland. Eight of the 13 will be in decile 1–3 schools, and five in higher decile schools. One will be living overseas and one is travelling overseas. Some other host schools told us they would have liked to employ their participant/s but were unable to employ additional staffing because of falling rolls.The majority of participants have remained in the programme because of the mission and vision of Teach First NZ, so that they can contribute to the educational achievement of culturally diverse students, and because of the high challenge of the programme. These were also important reasons for their applying for the programme. Four of them also commented that they had made a commitment and intended to honour that commitment.

Cohort 14 One participant left the programme and New Zealand at the end of term one. The remaining 19 were still in the programme in November 2014, and all expected to complete their 2 years with a limited authority to teach (LAT). They were positive about their career choice and committed to a future in teaching. Following is a range of their comments about teaching.

[It is] a field that is very rich in possibilities, in terms of career pathways.

I want to master the art of secondary teaching.

It’s the hardest thing I’ve ever done and the most rewarding.

I have an enormous emotional attachment to the school and kids.

I don’t see myself doing anything else.

Four participants of the 19 said that they may not stay in teaching beyond the 2 years. Cohort 14 participants’ reasons for possibly not continuing to teach were workload, travel (with a possible return to teaching in the future), and to undertake more study.


Programme factors For the programme to be successful, it needs to: have a common clear vision of effective teaching which permeates course work and practice in schools; have a strong core curriculum; apply case methods and teacher research; allow participants to confront and rethink assumptions about learners; be judged by the participants as intellectually challenging and practically relevant.

Key findings Participants commented on the relevance of their university papers and assignments as a strength of the programme. For Cohort 13, the Teach First NZ Leadership Project and the Faculty professional inquiry, were seen to be very useful for their own learning and often seen to add value to the school. Participants were also positive about the clinics and noho marae held during the year and most participants valued the support from and approachability of Teach First NZ personnel and Faculty LASs/VTSs. Participants considered that the Teach First NZ partnership had been very responsive to their feedback and noted that small beneficial changes had been made to the SII and the clinics.

This section focuses on those programme components that the Teach First NZ partnership controls or manages (as opposed to those aspects that the school determines, such as the participant’s timetable), and is focused around delivery of the Faculty courses and related university assignments. Note that the SII is also included in this suite of courses but has been addressed earlier. We received copies of all course booklets and assignments from the Teach First NZ partnership. Our main sources of information about the experienced delivery of these components of the programme were Cohort 13 survey data and Cohort 14 participants’ documentation and school personnel interviews.

There is a strong and fluid partnership between the Faculty and Teach First NZ in relation to programme implementation. While the Faculty programme leader developed and administers course materials and assignments for the postgraduate diploma, other aspects such as the SII, the clinics, the leadership component, and ongoing communication are very much shared responsibilities.

Cohort 13 We asked Cohort 13 participants and mentors whether they thought that the course workload had been appropriately challenging in 2014 and how it compared with 2013. Five participants considered the course workload had been highly challenging and eight considered it was challenging “to some extent”. Of the mentors who answered the question, seven mentors considered it highly challenging and five considered it challenging “to some extent”. Seven participants thought their course workload was greater in 2014 whereas five thought it was about the same and one thought it was less than in 2013. Over half the participants rated the professional inquiry as being the most useful component of their course work. Just under half of them also found visits from VTSs very useful, and over a third considered visits from LASs very useful. The online community and the portfolio were seen to be least useful with only one participant rating the portfolio as very useful and none rating the online community as very useful for their learning.

For some of Cohort 13, the Teach First NZ partnership and school expectations were similar for their inquiry, and most participants commented positively on its usefulness. The following quotation is illustrative.

The practitioner inquiry we are expected to conduct has been excellent this year in providing an avenue for me to formally inquire into the effectiveness of my practice. While writing it up is not the most loved part, the process has been invaluable.


We asked what changes might improve the programme for 2014 or in the future. Most did not identify changes but two or three commented on one or more of:

• the need for careful choice of schools and better preparation for the first term of teaching

• the “away practicum”—suggesting a pre-visit and regretting that the teacher exchange had not taken place in 2014

• better information and more careful planning of course work

• better connection to LASs and VTSs

• clarification of registration status.

Cohort 14 If I were running the programme, I’d run it the same way. (Cohort 14 participant)

We asked Cohort 14 participants which aspects of the taught programme they found most useful, and for any comments on how the programme is organised, presented, and assessed. Positive comments outweighed any comments about things that could be improved. All Cohort 14 participants commented positively on aspects of the taught programme, and many commented on multiple aspects, for example, usefulness of clinics, relevance of assignments, and time with curriculum specialists. Only a handful of improvements were identified: ensuring that more strategies are provided that are relevant for maths and science classrooms; suggestions about timing of clinics or assignments; and for two participants, making course work less onerous. When asked about weaknesses of the programme (see later in this section), participants identified a few other possible improvements. We could not discern any patterns associated with making suggestions for improving the programme, which might or might not indicate dissatisfaction and intentions for the future.

Participants were most likely to comment on the relevance of their university papers and assignments as a strength of the programme. They were also positive about the clinics and noho marae held during the year, providing valuable time with curriculum specialists and an opportunity to reconnect and “reinvigorate” with the cohort.

A number of participants also commented on the support from and approachability of Teach First NZ personnel and VTSs (including the usefulness of formal observations). Specific aspects of content that were highlighted by participants as being useful were the presentation about Pasifika learners, differentiation, scaffolding, and anything subject-specific.

Understandably, co-ordinators tended to comment on the school-based aspects of the programme, rather than that managed by the Faculty. Asking co-ordinators about the strengths of the programme elicited positive responses about the amount of classroom time participants gain over the 2 years of their study: “extended teaching practice”, “more time in the classroom as that’s where you do the learning”, “hard to beat being full-time in a school”. There was a sense that Teach First NZ participants really understood the role and the responsibility of being a teacher. One co-ordinator also explicitly referred to the level of mentoring and coaching as a strength of this way of preparing teachers.

In 2013 we reported that Cohort 13 participants said the quality of the lecturers, the connectedness of the cohort, and the ongoing feedback were the strengths of the programme. Although our conversations with Cohort 14 participants suggest they would not disagree, when asked directly about strengths of the programme Cohort 14 participants were most likely


to refer to the relevance of the course work and assignments, followed by reference to being “hands-on in the classroom”, with support from mentors and from the Teach First NZ partnership personnel.

Like principals and co-ordinators, some participants also saw the selection of candidates as a strength of the programme.

Only some Cohort 14 participants provided a response when asked about weaknesses of the programme. A few would like more time in clinics and with curriculum specialists. As noted in 2013 the programme is very dependent on school support, and how effective that support will be in practice can be unpredictable. One participant felt that participants could be challenged more, and gave an example of Māori perspectives on education.

Interviews with staff in schools in 2013 highlighted concerns about expectations placed on participants’ workload in terms of observations, whereas interviews for the 2014 cohort seem to suggest that these concerns are less significant or better managed in 2014. Participants mentioned a tiring workload during the SII, but that course work was “less onerous” during term time. Few other staff in schools specifically referred to the workload.

Support for participants from the Teach First NZ partnership We used information from participant interviews or survey questions to evaluate the extent to which Cohort 13 participants felt supported by the Teach First NZ partnership and the extent to which both cohorts felt part of the Teach First NZ community.

Key findings The majority of Cohort 13 participants considered that they were very well supported by the Teach First NZ partnership. They also reported being very connected to the other participants in their cohort and many of them said they also felt connected to Cohort 14 participants. We did not specifically ask Cohort 14 participants about how well supported they were by Teach First NZ. All Cohort 14 participants said that they felt part of the Teach First NZ community, although it was clear that this meant different things to different people.

Cohort 13 Eight of the Cohort 13 participants considered that the Teach First NZ partnership supported them very well. The majority also reported being very connected to the other participants in their cohort and seven said they felt connected to Cohort 14 participants. Twelve participants reported that belonging to the Teach First NZ community was very important or important to them. One did not think it was important. Seven considered that belonging to the Teach For All global community was important to them, although five participants did not think it was important. One thought it was very important.

Cohort 14 When someone says Teach First community I just have a smile on my face. (Cohort 14 participant)

We did not ask Cohort 14 participants specifically how well they felt supported by Teach First NZ, but we did ask to what extent they felt part of the Teach First NZ community and how important this connection was to them. All participants said that they felt part of the Teach First NZ community, although it was clear that this meant different things to different people.


Some focused on the community of participants in their cohort and referred to the bonding, and the enjoyment and support they got from meeting during the year at clinics. Two participants particularly highlighted how this differed from the (lack of) connection they had during other tertiary study. The clinics were highly valued as opportunities to reconnect as a cohort—during the term participants might be too busy to meet with other participants not in their own school. Three participants mentioned that there is a shared Facebook page where participants can connect digitally, though only one commented on using it.

Other participants responded to the question about community by referring more broadly to the “Teach First community” and referred to the Teach First NZ partnership personnel, and a few made direct mentions to their connections with Cohort 13 participants.

Yes, I have a sense of community and think that will increase as there are more cohorts and through events when the alumni programme is set up.

It’s definitely something I’m proud to be a part of. It will become a recognised brand.

I have a good relationship with all Teach First NZ staff and my cohort. Cohort 13 participants are [my] biggest support.

Support for participants from host schools (Cohort 13 and 14) We interviewed or surveyed participants, mentors, and coordinators to assess the degree to which the quality of mentoring and school support enabled participants to be successful beginning teachers.

Key findings Cohort 14 reported positively on the initial orientation that schools provided. Nearly all Cohort 14 participants were very positive about how the school and staff had responded to them, and that other staff treated them as if they were PRTs. All principals, and almost all co-ordinators and mentors, said that the staff reaction to having Cohort 14 participants was positive. All Cohort 13 participants told us that they felt fully accepted as a staff member at their school.

Mentor support to participants was varied, although most participants in both cohorts were positive about their mentor and in-school support. Mentoring is a critical component of the programme and ineffective mentoring can impact negatively on participants’ learning and, therefore, on the overall quality of the programme. The coordinator role has been strengthened in 2014, in part to monitor or enhance mentor support. Participants’ exposure to school-based professional development and learning (PLD) varied considerably as did their opportunities to observe their mentor teaching.

Cohort 14 participants’ orientation to school As was reported for the Cohort 13 participants in the 2013 evaluation report, Cohort 14 participants were positive about the initial orientation the school provided, referring to some or all of the following: being welcomed onto marae, participating in pōwhiri, touring the school, meeting other staff and participating in departmental planning. A few would have liked more orientation or time to prepare within their school. The quality of orientation to the school and early support for the participant was not dependent on whether the school was new to hosting participants in 2014.

As was reported in the 2013 evaluation report, most principals did not introduce participants as special or different and chose not to especially promote the Teach First NZ programme within the school. Consequently many staff were not


aware that the Teach First NZ participants were not regular PRTs. One principal suggested that most staff would not be aware of which new teachers were in fact from Teach First NZ.

There was a sense from principals and co-ordinators that “slipping in” and not raising awareness of Teach First NZ gave participants the opportunity to fit in as new teachers and develop relationships with staff as teachers. However, this also had the challenge of staff not understanding that participants might need extra support with planning and management in the early stages even though they have other highly developed skills and qualifications. While different schools successfully used different approaches, there may be advantages for participants in schools making all staff aware of the Teach First NZ approach and process.

Mentors, however, reported that teachers were generally well informed about Teach First NZ. Teachers’ reactions sometimes varied until they got to know the participants. Participants’ apparent confidence was sometimes misread by other staff as not needing as much support as they do, especially initially.

When asked about staff reaction to having Cohort 14 participants, all principals, and most co-ordinators and mentors said that the staff reaction to having Cohort 14 participants was positive. They used phrases such as “exciting new initiative” and “it’s only been positive—the concept and how the participant is treated”. Others reported some initially mixed responses from staff, possibly still influenced by early negativity in the media. Some reported that responses were largely neutral, with any initial resistance being low key and short-lived, especially in hard-to-staff areas or subjects. However, in two schools, co-ordinators suggested that “the jury is still out” with regard to staff views on the programme and that it could depend on what candidates do at the end of their 2 years: “If schools have put in a lot of effort and there is no long-term gain for the school, it may change the climate”. All other schools were positive about their involvement in the programme, regardless of whether or not they were able to employ a Teach First NZ participant in 2015.

Quality of school support for participants In 2013, four school-based aspects stood out as being crucial for participant success:

• the capability of the mentor and quality of the mentoring relationship

• a timetable that allowed participants and mentors to work together

• the assignment of participants to classes that were likely to respond well to them

• a well-functioning subject department.

This section focuses on participants’ experiences of support and mentoring. We discuss mentor teachers’ experiences in another section of the report.

Cohort 13 All Cohort 13 participants told us they felt fully accepted as a staff member at their school. Participants identified a number of aspects of mentor practice as very helpful to their learning. The most helpful aspects of their mentor’s support were in relation to pastoral support (eight participants selected this option) and pedagogical knowledge (six participants). The majority also found feedback on lesson planning, feedback after classroom observation, observation of other teachers, and advice on classroom management to be helpful. A majority of the participants also found their HODs, other teachers in the department, and other teachers in the school supported them as teachers. These teachers were most likely to provide pedagogical knowledge, pastoral support and advice on classroom management. In addition, data from 2014 identified a more central and vital role for co-ordinators and opportunities for leadership development afforded by the school for Cohort 13 compared to the previous year’s experience.


Participants’ access to PLD varied and some participants had clearly had much wider access to it than others. The most common PLD participants were involved in was PB4L School-Wide work (four participants) and inquiry (three participants). Individual participants also mentioned taking part in Brainwave Trust workshops, regular school-based PLD, beginning teacher PLD, subject-specific PLD run by external facilitators, Google training, Star Path, e-asTTle, Restorative Justice, and Te Kotahitanga.

We asked how well the participants’ expectations of the school in helping them learn to teach had been met in practice. This question was related to the support they had had from their mentor and other staff members. Five thought their expectations had been “very well” met, five thought they had been “well” met, and three that they were “not well” met.

Because they visit all Cohort 13 schools, LASs were able to provide valuable comment on participants’ experiences. This comment reflects LASs’ views:

Yes there is considerable variation in the level of informed, supportive consideration given to participants. Some schools, through the co-ordinator and mentor, appear to provide more ‘wrap around’ consideration for their participants. Other schools appear to take a more ‘sink or swim’ approach.

Cohort 14 Cohort 14 participants similarly described the kinds of support their mentors and other staff provided as being of a practical nature. Feedback continued to be the most important aspect of their mentor’s support. This feedback could arise from formal and informal observation, or in response to questions or challenges raised by the participant on a range of topics. These topics included planning and organisation, building relationships with students, and behaviour management.

Members of Cohort 14 also valued mentors having a strong interest in their success and strong professional and personal relationships. They appreciated ready access to and sufficient time with mentors. They felt that being in the same department was helpful, as was teaching at the same class levels, which gave the opportunity for joint planning and team teaching. Participants referred to mentors helping them find solutions to problems that participants had either asked about or that had been identified in observations. Specific things their mentor had supported them with were seating plans, behaviour management strategies, joint planning, curriculum design, and resources. Mentors also provided emotional and moral support, described by one participant as “offering empathy in moments of struggle”.

In many cases other staff also contributed to this support. Participants were encouraged to observe classes and discuss issues with other teachers. In some schools, as with Cohort 13, participants were assigned to share tutor groups with experienced teachers, and this offered extra opportunities to learn and practise aspects of pastoral care.

As with the majority of Cohort 13, most Cohort 14 participants were treated as if they were PRTs. Eight specifically mentioned being included in the PRT programme, which included appraisal towards full registration.

All participants had been involved in professional learning at school, often on a regular weekly basis. Some participants found topics were often similar to what they had covered on the SII but it was useful to have this refreshed in a school context. The range of professional learning identified by participants included Ako Panuku meetings, e-learning, raising achievement focus, Best Practice workshops, and subject specific topics, for example, senior chemistry or Year 9 literacy, te reo Māori, Learning with Digital Technologies (LwDT), or inquiry. Some participants were in PB4L-School Wide or Te Kotahitanga schools.


Observations of others’ teaching (Cohort 13 and 14) Participants are expected to observe their mentor and other teachers teaching. Many participants did not appear to observe their mentor teaching, although all reported observing other teachers. Three-quarters of them found observation of other teachers helpful.

Cohort 14 participants also did not observe their mentor teaching as often as expected, and this is sometimes put down to timetable constraints, but was reported by a few mentors as not being prioritised by the participants themselves. Those participants and mentors who team-teach or work in proximity were most likely to report that the participant observes the mentor teaching.

Host schools preparation for, and support in, their roles (Cohort 14) We asked mentors, coordinators and principals in the interviews or through the surveys, about the extent to which host schools thought the preparation and support for their roles by the partnership was high quality and how well supported they felt.

Key findings The vast majority of mentors, co-ordinators and principals perceived the liaison between the Teach First NZ partnership and themselves to be effective. Some co-ordinators who had been involved since 2013 suggested liaison had improved from 2013 to 2014 and referred to the responsiveness of the Teach First NZ partnership personnel, saying they are “very open to feedback and adjusting to meet our needs” or “Teach First NZ took on board our suggestions”. The role of co-ordinator continues to be refined and plays out differently in different schools.

Teach First NZ support for, and communication with, host school

Cohort 13 The majority of Cohort 13 principals, co-ordinators, and mentors responding to the online survey considered that the liaison between the Teach First NZ partnership and themselves had been effective or very effective in 2014. Such intermittent communication was by email.

Most mentors and co-ordinators considered that communication was about the same as last year (four thought it was better). One example of a change from 2013 to 2014 was getting a plan for the year at the first co-ordinators’ meeting; another was improved communication from VTSs so that the co-ordinator was also included in emails setting up visits.

All co-ordinators said they felt either very well or well supported by the Teach First NZ partnership, with a few mentioning that their role had consolidated since last year. One said:

As mentors become more expert, I have spent less time with participants in a supportive role and am doing more administration for all three participants, attending meetings and holding regular termly meetings with mentors and participants. (Cohort 13 co-ordinator)

Eight of the nine co-ordinators felt connected to other co-ordinators in 2014, one of them more so than last year. One did not feel connected. All co-ordinators considered that their role was valued or highly valued by mentors and their school senior management team.


Cohort 14 In the Cohort 14 schools all principals and nearly all co-ordinators were positive about the effectiveness of liaison between the school and the Teach First NZ partnership. Examples of positive comments were that communication was “excellent”, “everything runs pretty smoothly”, and “[I] have confidence in the process”.

Some co-ordinators who had been involved since 2013 suggested liaison had improved from 2013 to 2014 and referred to the responsiveness of the Teach First NZ partnership personnel, saying they are “very open to feedback and adjusting to meet our needs”, or “Teach First NZ took on board our suggestions”.

Three co-ordinators and two mentors were less positive about the liaison with the Teach First NZ partnership. One co-ordinator felt they had not seen the Teach First NZ partnership people at the school as often as they should, especially for curriculum support; another felt there were too many demands on the school and that the relationship was driven by Teach First NZ: “it's all about ‘we need this from you’.” When prompted, this co-ordinator agreed this was about the tone of communication as well as what the school was being asked to do. The co-ordinator felt the amount of compliance had increased: “A little bit of that crept in that I wasn't aware of last year".

The role of co-ordinator is intended to be the critical link between the Teach First NZ partnership and the school, but in 2014 we found this link to be variable depending on school context and other role(s) coordinators had in their school, for example, as the specialist classroom teacher (SCT) or the PRT co-ordinator. The co-ordinator role was described by some as “hands off”, but one which required/s keeping principals informed, supporting mentors, and to varying degrees in schools, supporting participants. One co-ordinator said: “My job is to make sure we use the 10 hours of mentoring well. We need to be accountable.”

At least one school had a full time co-ordinator for PRTs, beginning teachers, and those on practicum, so Teach First NZ participants were included as a natural extension of this. As in most schools, teachers take on a number of roles so it is no surprise that two co-ordinators, for the 2014 cohort, were also mentors. Co-ordinators were not directly asked about preparation or training, but it was clear that many had relevant experience from other roles in the school. They referred to information from and meetings with the Teach First NZ partnership: “Preparation was through meetings with Teach First NZ and other co-ordinators. We learnt from experiences of those that had participants last year.”

Affiliate schools (Cohort 13) Affiliate schools are those in which participants spend a 3-week “away practicum” in Year 2 of the programme. We hoped to evaluate the extent to which affiliate schools were aware of their responsibilities to the participants and their ability to provide useful learning experiences for participants. We also wanted to know if affiliate schools felt well supported by the Teach First NZ partnership and how useful the planned teacher exchange was judged by the affiliate school.

Key findings Four of the 15 participants completed their 3-week “away practicum” in the affiliate school before mid-November. Therefore, we have no evaluative data to report in 2014. The original goals of the “away practicum”, whereby an “affiliate fellow” swapped places with a participant, were unable to be realised and the Teach First NZ partnership reverted to a “bare basics” model which met the Teachers Council requirements. The difficulties experienced in making “away practicum” arrangements may raise questions about its viability and feasibility in its current required form.


The teacher exchange aspect of the “away practicum” in affiliate schools proved too hard to implement in the way it was originally intended, that is, where participants swapped jobs with a colleague at the affiliate school. Teach First NZ reported that the 3 weeks consecutive period13 was the main impediment. Some host schools were very supportive of the participant’s practicum experience, but others did not find it easy to implement. Because the host school and the participants were responsible for managing the participants’ classes while the participants were at their affiliate school, three quarters of them were unable to undertake their “away practicum” until after senior students left in November. This meant that we are unable to evaluate their experiences. Practicums were organised by the Faculty of Education. The expectations for the practicum were that it would be an opportunity to:

• engage in professional networking with colleagues in the host department

• reflect on their developing philosophy of teaching

• engage with a very different school culture (usually where the socioeconomic status of the community is very different)

• teach some lessons as and when appropriate

• talk about teaching and learning with hosts.

The four participants who completed their practicum in term 3 were well received by their affiliate schools and benefited from the experience. One was offered a position for 2015 by the affiliate school. We only had access to one participant’s documentation about the “away practicum” which outlined the intention of the practicum, the expectations on the participant and the school, and in the case of the participant, the usefulness of the experience. A Faculty member reported in January 2015 that, despite the difficulties experienced by many participants, all participants indicated that their experience had been worthwhile. While a Faculty member did not consider that the lack of “swapping” had compromised participants’ experiences, s/he expressed some disappointment that schools did not appear to yet be open to such exchanges. We do not have data about how affiliate schools benefited from the experience, or whether staff from affiliate schools took up opportunities to spend time in host schools.

Mentor teachers Through mentor and Teach First NZ partnership interviews and surveys, we reviewed the degree to which mentor teachers: were well supported by the partnership; provided regular high-quality observation, mentoring and feedback to participants; and helped participants to become part of the wider school community.

Key findings Mentors were generally pleased with the training they had had, and with their role. Some mentors reported finding the support and training from the Faculty more valuable than others and this usually depended on what other mentoring PLD and experience they had. Many of the mentors commented on their own learning from enacting the role, or discussions with Faculty specialists, or the initial training and how it had energised them as a teacher.

Selecting the right mentor for each participant has not always been easy, and schools appear to be weighing up a number of factors before assigning a mentor to a participant. About half the 2014 mentors for Cohort 13 had not been the participant’s mentor in 2013, and several mentors for both cohorts are not teaching the same subjects as the participant. As in 2013, the frequency of observations (and mentors’ individual approaches to them) varied, as did the extent and nature of the feedback given. In general, participants valued mentors’ pedagogical knowledge, help with pastoral support, feedback from observations, and help with behaviour management.

13 A New Zealand Teachers Council requirement.


Cohort 13 Finding the right mentors is crucial and that is not an easy task …. The time allocation is okay but especially in the first two terms a new participant relies very heavily [on the mentor] as they are not ready to teach classes independently and yet they have to under our staffing system. (Cohort 13 co-ordinator)

Six of the Cohort 13 mentors considered they were “very well supported” by the Faculty in 2014, with one of them stating that this support was greater than in 2013. The other six who answered the question reported being “well supported”. Interestingly, ten of the fifteen mentors had not been the mentor for the participant in 2013. Seven of them were in the same department as the participant they mentored, and eight were not.

Mentors identified a number of ways they help and support participants, with the most often mentioned being: with pedagogical knowledge and feedback after lesson observations (all respondents); advice on pastoral management and pastoral support (12 participants) and giving feedback on lesson planning (11 participants). These areas correspond with what participants found helpful in mentor support. Seven of the mentors observed the participants once a week and three, once every 2 weeks. Four mentors observed less frequently. Eleven mentors considered that they were able to support the participant “very well” every week, while three mentors thought they could support the participant “well”. Five mentors said that challenges included timetable clashes, which made observation times difficult to programme.

Ten mentors considered that their mentoring role was “highly valued” or “valued” by senior management, and eight thought it was “valued” by other teachers. Three “did not feel valued” by senior management and four “did not feel valued” by other teachers. One’s comment illustrates the value mentors saw in the role: “All teachers should have the opportunity to mentor on a regular basis. It increases your insight into more effective and evidence based teaching strategies and professional dialogue.”

Mentor training In general, most Cohort 13 mentors were happy with amount of training offered by the Faculty: Five said they were very happy and eight of them said they were happy. All had found the training useful or very useful for their work with participants; and all but one found the training very useful or useful for the work they do with other staff and with students. Two mentors said they felt very connected to other mentors (including in other schools), eight felt connected, and two said they were not connected. Seven of the mentors commented on how much they had enjoyed and learned from being a mentor. One said they would like to get feedback/feed-forward on their mentoring skills.

Cohort 14 For Cohort 14 the selection, training, and ongoing communication with the mentor were found to be critical to the success of the programme and to participants’ progress in the 2013 annual evaluation report, and this is further confirmed by the Cohort 14 data.

Selection of mentors for Cohort 14 When describing how they selected mentors for participants, most (seven) co-ordinators described starting with the department or curriculum area that they expected to have a participant in: “Chose the best match within a department from those with interest or experience”. The remaining three answered this question by talking about mentor qualities or experience, rather than their department: “Very experienced and accomplished mentor. The right mentor is important”.

This concurs with 10 out of the 19 mentors’ reporting that they were approached by either the principal or the co-ordinator and asked if they would take on the role. In two schools the mentor role was advertised, with the principal


asking for expressions of interest. The rest (seven) were ‘expected’, to varying degrees, to take on the role of mentor as part of their normal job or a logical extension of their current role, but were happy to do it. All but one reported that their attitude to the Teach First NZ programme was more positive than at the beginning of the year. This is largely due to the quality of the participants and the positive experience of the schools. One mentor commented, “this mentoring role is awesome—you should do it!”

School support for the mentor role was most often described in terms of time allowance which is timetabled, and described in all but two cases as good or very good. The two instances in which time was an issue involved a relatively small school—timetabling in small schools can often be a challenge—and one school in which the mentor held multiple roles.

Five of the 19 Cohort 14 mentors were not in the same department as their participant. However, being in the same department does not always necessarily mean teaching the same subject, for instance in one science department the mentor teaches biology, while the participant teaches physics. In such specialist areas, support for curriculum content and the role of the LAS/VTS is very important. A few mentors expressed that being in the same department, if not subject area, was crucial to the participants’ success, while others acknowledged that support for the participant not in the same department “could be done”, but required more planning.

Mentor training All mentors described the training sessions as useful PLD, and even “vital”. Most of those interviewed said they attended all or most sessions. One commented that the clarification of expectations and guidance on the frequency of observations and ways to record information was very helpful. Even if they were already familiar with the content, they noted it was “good to listen to others” and to “talk with other mentors”. Seven mentors gave an example of how the programme was contributing to their own development, by attending the training, using the readings, or learning from their participant. It was sometimes the case that the participant was highly qualified in their field and could add to the mentor’s content knowledge.

Two schools had sent additional teachers to the mentor training, as they saw this as valuable professional development.

Observation of participants I get in and watch new teachers as often as I can because I think this approach works—this makes everyone gain! (Cohort 14 mentor)

The focus on mentor observations and feedback is noted as a strength of the programme. The frequency of observations, and mentors’ individual approaches to them, varied somewhat. Some kept meticulous records of regular formal observations, some recorded a few formal observations but also frequently gave feedback on informal observations, and one claimed not to count them at all, but nevertheless had a comprehensive record of observations and the feedback given. Most reported aiming for weekly observations as recommended by the Faculty, although in reality this often turned out to be fortnightly as other activities or events intervened.

The frequency of formal observations also varied depending on the number of scheduled observations from LASs and VTSs. Some mentors’ and participants’ classrooms were close enough to allow informal observation and support for a number of classes. In almost all cases mentors reported decreasing the frequency of observations as the participants gained confidence and experience in the second term and certainly in the second year.


Learning area specialists and visiting teaching specialists For the programme to work well, specialists need to provide regular high-quality observation, mentoring, and feedback to participants. There should also be explicit links between the curriculum papers in the qualification and the feedback provided by the specialists in schools.

Key findings In 2014 the Teach First NZ partnership made a distinction between the LASs who provide curriculum support at the clinics and throughout the year, and VTSs who are curriculum specialists and who visit selected participants but do not teach on the programme. The vast majority of the participants reported that they found visits and feedback from VTSs and LASs useful and that these provided additional expertise and an outside perspective.

VTS/LASs visited each participant four times in terms 1 and 2 and three times in term 3 to observe a lesson and discuss it with the participant. Three times a year, the VTS/LAS and the mentor did a formal observation together, which was assessed against Faculty criteria and contributes towards the completion of one of the Faculty courses. Apart from the participants, it is the mentors who have most communication with the VTS/LAS during their visits. Almost all participants valued the VTS/LAS observations and feedback.

Cohort 13 All Cohort 13 participants but one thought they had been well supported by LASs and VTSs in 2014. Five Cohort 13 participants reported being “very well supported” by LASs in 2014, seven reported being “supported”, and one indicated that they were “not supported”. Similarly, six Cohort 13 participants reported being “very well supported” by VTSs in 2014, six reported being “supported”, and one indicated that they were “not supported”.

The five LASs responded to the survey. All described their relationships with participants and mentors as “very positive”. Three considered their relationship with co-ordinators as “very positive” and two as “positive”. LASs were positive about the PLD they have had, such as with professional readings, practising giving feedback, marking, and moderating.

[Faculty staff programme manager] is highly supportive to all staff involved in the Teach First NZ programme. [She] has provided highly detailed information/guidelines to all staff. My involvement is to teach, visit (observational and formal), mark assignments and attend meetings.

All LASs want to continue their relationship with the Teach First NZ programme. Their main form of communication is by email which they all considered works very well. One suggested a meeting between the co-ordinator, the mentor and the LAS as part of an initial visit to a school.

Cohort 14 Cohort 14 participants were asked about involvement with LASs and VTSs and the extent to which this was helping them to develop their expertise as a teacher.

Fifteen of the participants reported that they found visits and feedback from VTS and LASs helpful, providing them with additional expertise and an outside perspective. They also commented on the usefulness of joint observation with the mentor. They described LASs / VTSs as being “very collegial”, “good at clarifying learning intentions”, “helpful, challenging, supportive”. One participant “would like to see more of them”, commenting that “I really value their input”. Another said it was “good to have another outside perspective and different viewpoints”.


The variety of feedback and support offered by the multiple observers did not suit all the participants. While some said they would like more visits, others felt that too many perspectives complicated and fragmented the directions they could take. Another expressed concern over the formal assessment which contributes towards the Faculty programme results.

Programme responsiveness We looked for evidence that the programme was responsive to feedback from participants and participating schools.

Key findings In our 2013 evaluation report we noted that one of the strengths of the programme was its responsiveness. Responsiveness has continued to be a strength in 2014. Teach First NZ and the Faculty have actively sought feedback on how the programme could be improved from all parties involved.

Schools’ and other stakeholder perceptions of the programme have helped shaped it. Teach First NZ has been aware of these perceptions from the outset and has been very effective in working with different groups (including, for example, the PPTA) to redress negative perceptions and to enhance programme factors that are working well.

At principal, co-ordinator, and mentor meetings Teach First NZ and the Faculty have actively sought feedback on how the programme could be improved, both in 2013 and 2014 and in future years. The selection process has been adapted to provide a much stronger assessment of participants’ cultural competence and a more nuanced experience for Māori applying for the programme. The SII, noho marae and clinics have been adjusted to take account of feedback from 2013, and these changes have been well regarded by those involved.

We asked Cohort 13 participants how the programme had changed in response to their feedback. Eleven of the twelve participants who commented on this question considered that the Teach First NZ partnership had been very responsive to their feedback and that small changes had been made to clinics and the SII. A small minority of participants commented that these changes benefited Cohort 14 rather than themselves.

While there is the odd niggle that in providing more, and more timely, information to schools, Teach First NZ has sought more compliance, most co-ordinators appreciate the additional information. The additional seven schools involved bring a different dimension to the programme—partly because of their geographical locations, but also because they are mostly large urban secondary schools serving a wider range of ethnicities than those that began in 2013. The Teach First NZ partnership has responded to these differences and adapted its approach to recognise these differences. Likewise Cohort 14 brought participants with different strengths, needs and expectations. These have continued to be addressed during the year and have resulted in a major rethink about how the partnership addresses te Tiriti o Waitangi.


Viability We considered that the programme would be viable if the Teach First NZ model can successfully adjust to larger numbers of participants within agreed funding arrangements and if schools are able to support the programme when they self-fund participants’ salaries from 2014.

Key findings There does not appear to have been an issue for 2014 schools in funding Cohort 14 (who, unlike Cohort 13, are not supernumerary). In 2014, one decile 5, two decile 4, and two decile 3 schools were added, all schools with large rolls. While the vast majority of principals in the pilot programme to date would hire further Teach First NZ participants if they could, some school personnel identify ongoing challenges, often in relation to funding and the expectations of schools.

Cohort 13 We are very privileged to participate in the Teach First programme and the energy and enthusiasm the participants bring to our school has made a big difference to our community. (Cohort 13 principal)

For Cohort 13 we address this evaluation criterion more fully in the discussion chapter. The Teach First NZ partnership staff continue to demonstrate a strong commitment to their mission and values and continue to actively promote the programme and to advocate for it with government agencies. The participants have grown as ambassadors for the programme and for teaching and the alumni may have a positive impact on the programme’s viability in the future.

There does not appear to have been an issue for 2014 schools in funding Cohort 14 (who are not supernumerary as Cohort 13 are). While the vast majority of host school principals would hire further Teach First NZ participants if they could, some school personnel identify ongoing challenges—often in relation to funding.

Participants are employed as LATs for 2 years and work towards the GTS. On achievement of the GTS, they are provisionally registered for 2 years. Two registration issues could potentially affect the viability of the programme. The 3-week “away practicum” required by the Teachers Council for registration has been difficult to timetable and manage as envisaged and it has strained many participants and their schools. Spending time in an affiliate school is a valuable and important part of the programme, but many people we spoke to and surveyed said the uninterrupted 3-week period was too long. Eleven of the fifteen participants undertook their practicum at the end of the school year when senior students had left. This raises questions about the usefulness of that practicum and the extent to which it is a viable option in its current form.

While participants had known from the outset that they would be required to do 4 years before they would be eligible for full registration, a few were disappointed that this situation remained as they thought it would disadvantage them on the job market in New Zealand and internationally. Some also felt that their standing in the teaching community would be compromised. It is possible that in future the programme could have difficulty attracting top-quality applicants who may choose to do another ITE programme (for example, a Graduate Diploma or Masters programme) where they will be eligible for full registration after 3 years. Teach First NZ is discussing registration implications with the Teachers Council.

Cohort 14 Principals in 2014 also noted that they were keen to continue their involvement, or to see the programme used more widely.


Principals were more likely to comment on the gains to the school from the participants being funded as supernumerary. One principal, who had worked out the actual costs, noted that even when they were not counted as supernumerary, the cost to the school was still neutral.

Some principals felt that funding, or rather staffing, is the greatest barrier to sustaining involvement in the Teach First NZ programme. “Funding is always a barrier, but at least we get a little in the form of staffing allowance”. While schools with growing rolls that were looking for new staff found it easier to place participants and were keen to do so, those with falling or capped rolls were less able to take on extra staff. This was sometimes associated with comments about needing to ensure stability, or “balancing” staff (for example age and gender balance). Only one principal commented on issues with PPTA concerns about employment processes around LATs.

Most co-ordinators did not describe barriers to effective participation in the Teach First NZ programme. Three described in-school challenges: lack of cohesion in the participant’s department, a less supportive HOD, and timetabling challenges. One co-ordinator highlighted the workload for mentors as a barrier, saying they “can’t do the job justice”.

Even though Cohort 13 participants secured jobs very readily, there were some concerns from school staff about whether they would. Like principals, one co-ordinator was concerned that they would not be able to offer participants a job at the end of the 2 years because of roll projections. Another raised a more general concern that participants may find it hard to get jobs as they compete with those exiting a traditional ITE pathway, with the risk they are “labelled as different” by schools that have not had experience of Teach First NZ.

Mentors also generally felt positive and hopeful that the programme would be continued because of the high-calibre participants. This was seen as a particularly important factor in the success of the programme so far and would be vital for its viability.

The candidates seem to have been selected particularly well and I think that has to continue for it to work ... otherwise people will start seeing it as a lesser programme—if the wrong candidate came through this programme it could go very wrong for them and for the school. (Mentor)

Once participants have completed the two-year programme, they are free to choose if and where they will teach. The following table shows Cohort 13 destinations in 2015.

Table 6 Cohort 13 destinations in 201514 Destination Number

Retained by host school (all deciles 1 or 2) 4

Employment at another decile 1 school 2




Employment at another decile 8 school 2 (one to start in 2016 after overseas travel)

Employment decile 9 school 1

Living overseas 1

14 Note, in this table on future employment the new decile ratings apply.



Evaluation question 2: To what extent has the programme achieved its overall outcomes and objectives?

This chapter focuses on the evaluation criteria for evaluation question 2, as far as they can be addressed at this stage of the evaluation. Evaluation question 2 is an outcomes question, and many of its aspects will not be possible to answer fully until the end of the pilot programme. The second evaluation question looks at how well, and to what extent, the programme achieved its overall outcomes and objectives.

To answer this question, the evaluation focused on: the effectiveness of participants’ teaching; their level of support for the pastoral life of the school; the leadership development strand of the programme; the ongoing involvement and/or retention of participants; programme impact on quality of teaching and learning in participating schools; status of teaching; and enablers and barriers to success. To avoid unnecessary repetition, we have combined the data on programme impact of quality on teaching and the status of teaching, and reported on enablers and barriers to success in the discussion chapter.

Effectiveness of participants’ teaching To gauge the effectiveness of participants’ teaching we evaluated the extent to which participants’ students are engaged in learning (particularly Māori and Pasifika students), how well participants’ teaching aligns with the Teach First NZ model, and student achievement.

Key findings Participants are valued for their very strong subject knowledge, their facility with digital technologies, and for their perseverance and “can do” attitude. Everyone surveyed about Cohort 13 considered that participants had made considerable progress and were more confident, had stronger relationships with students, and were playing a stronger role in their department and often in the school than they had done in their first year. Everyone, including participants, thought that they would meet the GTS. For Cohort 13, all co-ordinators and principals considered that students responded very positively or positively to the participants. For Cohort 14, although a few co-ordinators and mentors felt that participants’ relationships with students were still developing, many commented on the impact participants were having with students, both academically and in a pastoral role.

This section discusses the effectiveness of participants’ teaching, from their own perspective and the perspectives of those in their schools (mentors, co-ordinators and principals). This includes views on how Teach First NZ participants compare with student teachers on practicum and with PRTs. It then discusses participants’ relationships with students, presenting student engagement results from the Me and My Class survey.


Cohort 13 Cohort 13 participants were nearing the end of their second year of teaching when they, along with principals, co-ordinators and mentors, completed surveys. We asked if participants’ confidence as a teacher had increased since 2013. Eight co-ordinators, twelve mentors, and ten participants said that it had “to a high extent”. We also asked how effective participants were in raising student achievement. Participants were more likely than mentors to rate themselves as “effective” rather than “very effective”. Four participants saw themselves as “highly effective”, whereas eleven of their mentors rated them as “highly effective”. Co-ordinators cited: “showing more initiative, more confidence in dealing with behavioural issues”; “less need for day to day support”; “their general approach to the profession—they are no longer in survival mode and have come to relish the work that they do.”

A similar pattern is repeated in response to the question: “How confident are you that participants are culturally responsive to Māori and Pasifika students in their classes?” All but one mentor perceived the participant(s) they worked with as being “very confident”, whereas half of the participants saw themselves as “very confident” and the other half rated themselves as “confident”.

Eight of the 13 Cohort 13 participants who responded considered they were functioning “very well” as a teacher in October 2014. Four thought they were functioning “well” and one, “not very well”. Their perceptions of how well they are functioning as a teacher are not related to their leaving or staying in teaching or to their ability to get a job. They appear to be more a function of participants’ self-assessment.

Some of the participants had picked up form/whānau classes or more senior classes which had added to their workload and responsibilities, although they were positive about this as it had extended their learning. Two commented that they would have been happy to have had more challenge. One reported a significant extra workload as she has taken on a class in an additional subject. Some participants commented on how their workload had changed.

The teaching workload is about the same as last year but the workload is different—last year most of the time was spent on day-by-day lesson planning whereas this year is about developing greater unit plans to try out new teaching strategies.

While I’m teaching the same number of classes, all three classes are low literacy with at risk learners. This has resulted in greater differentiation and individualised learning programmes which has increased my workload.

All co-ordinators considered that the participants were more effective or similar to other PRTs in their second year of teaching, and seven of them considered that the participants were at least similar to fully registered teachers.

They are well chosen. They have excellent instruction. They ‘benefit’ in the long term from the ‘deep end’. They are well supported. They receive excellent, regular feedback.

Cohort 14 For Cohort 14, principals’, co-ordinators’ and mentors’ interviews strongly identify that, on the whole, participants are doing the things that teachers do, or as one co-ordinator said, “being teacherly”. Principals commented that participants quickly developed a “presence” in the school, in the classroom and outside, making an impact with their energy and fresh approach, resilience and “can do” attitude.

Co-ordinators and mentors shared stories of participants having an impact at the school, department, and classroom or student level, including with parents and whānau. In every school we heard positive examples of things that participants


were doing, such as bringing strong disciplinary knowledge, “providing a safe environment”, relating well to students, and contributing to department meetings.

Cohort 14 principals believe that this programme is likely to produce quality teachers because of the quality of the participants, and the on-the-job experience, which although initially very challenging provides the experience of building relationships with students, under the guidance of a mentor. Most, but not all, co-ordinators and mentors were sure that the Teach First NZ programme would produce quality teachers at the end of the 2 years; a few said that the participants they worked with already were quality teachers. Their responses reiterated the key elements of the programme as: the robust selection process (“it will always come down to the individual who goes through the programme”), support and mentoring, and immersion in the classroom. Together, these were seen as essential components for the programme to produce quality teachers.

The main difference between Teach First NZ participants and other beginning teachers was the initial challenge they faced, described by principals as the “steep learning curve” and “truly challenged”. They suggested that “while they start behind the eight ball, they soon catch up”, and also that the participants were “more resilient”, “tougher, as in mental strength”, “mature” and that they have some “experience and common sense”. One summed up the difference between them simply as “personality”.

The five co-ordinators who made a comparison between Teach First NZ participants and student teachers on practicum all said participants were well above: “a step ahead” or “in a different league”. When asked to compare Teach First NZ participants with PRTs in their first year, responses were more mixed, with one saying “exceptional” (that is, well ahead of a PRT), three saying they compared with “a good PRT”, three saying the range was similar, and one saying the Teach First NZ participant in their school was performing below a PRT1, although improvements were now evident in Term 3.

Four noted that some staff had also been helped or supported by the participants, sometimes in specialist subject areas such as physics where the participant was highly qualified. Four noted participants’ positive impact on activities outside the classroom.

Mentors also described how Teach First NZ participants differ. The participants were described by mentors as having a variable range of competencies. Some had considerably more in-depth subject knowledge, especially in specialist areas such as physics and maths, and some came in with strong management skills, while others needed more support with management or with administration. One of the least “visible”, but possibly more important, differences is the sense of responsibility that Teach First NZ participants have for their classes.

This was reflected by the participants themselves, who said that they felt very responsible for their students and their achievement. Cohort 14 participants’ perspectives on how well their students are doing reflect their high expectations, with many commenting on things they were working on to improve to “get more learning happening”. What was noticeable was the number of participants (n=8) who explicitly and without prompting referred to the use of student achievement data to know how their students were doing. Ongoing challenges when we met them in Term 3 were less about behaviour management, and more about teaching classes with students performing at a wide range of levels and differentiating the curriculum for individual students.

We also asked Cohort 14 participants how well they thought they were functioning as a teacher at this stage in their development. Again, their high expectations were evident, but all were positive about where they were at, or at the very least, the distance they had travelled. There was a sense that some participants who had struggled at the start of the year had “turned a corner”.


As was reported in the 2013 annual evaluation report, participants and their mentors acknowledged that managing classroom behaviour and building relationships with classes in Terms 1 and 2 was a significant challenge (as is typical with many beginning teachers). Lesson planning and knowledge of the curriculum and assessment practices were also challenges highlighted by Cohort 14 participants. They commented that they “needed more time to resonate with the curriculum document” and they needed help with “not so much how to teach but what to teach” and “to understand what my subject was at secondary level”.

As would be expected, mentors were an important support for over half of the participants in addressing these challenges. Five participants also referred to support from other teachers in the school or department (e.g. the HOD), and two to support from Teach First NZ partnership personnel. As well as support from others, a number of participants referred to their own experience with trying things out, and consequent building of confidence as factors that had helped them address early challenges in becoming effective teachers.

Relationships with students

Cohort 13 For Cohort 13, all co-ordinators and principals considered that students responded very positively or positively to the participants.

Ten participants provided comments about the rewards of teaching. Most of these were about relationships with students. Three participants described teaching in the following terms:

Creativity, planning, commitment and aroha are rewarded with learning that is visible—confident and interested students and positive relationships

It’s about the small things—the positive affirmation from colleagues and students, the successes of students and the growing desire to achieve highly. Also the relationships you develop with your classes over the year.

The biggest reward for me is, and always has been, seeing my students succeed. This year I have been able to differentiate at a micro level for each student and this has seen their success increase—and for me this is what it is all about.

Cohort 14 For Cohort 14, although a few co-ordinators and mentors felt that participants’ relationships with students were still developing, many commented on the impact participants were having with students, academically and in a pastoral role. Overall, students’ responses to participants were described as positive or “what you’d expect from any teacher”.

Three co-ordinators also reported instances of good contact and relationship building with parents. One principal had been particularly impressed with an email sent by a participant to a parent and had asked to share this with other staff as an example of good practice. One participant told us how she had established relationships with Pasifika parents and talked with many of them in their first language at parent–teacher interviews. This had enabled her to explain about homework and NCEA in a more meaningful way.

Student engagement: Results from the Me and My Class survey Me and My Class is a survey designed by the New Zealand Council for Educational Research for Year 4 to Year 13 students. The items are based on research into the key competencies, which are an integral part of The New Zealand


Curriculum (Ministry of Education, 2007).15 The survey explores students’ perspectives on learning in their classroom. We chose to use the survey to help us assess one of the evaluation criteria for effectiveness of participants’ teaching: high engagement of students, especially Māori and Pasifika.

The Me and My Class survey has two sections:

• Me and My Class: The first section has 24 questions that explore students’ views on the extent to which learning opportunities in their class reflect effective teaching pedagogy.

• Me in My Class: This section contains 10 questions that explore students’ personal responses to learning in their class.

In September, 8 months after 2014 participants began teaching and near the end of Cohort 13’s programme, participants chose a Year 9 or 10 class, and schools were asked to choose a “comparison” class where the students were in the same year level and preferably in the same subject area and taught by another teacher (normally who was fully registered and often significantly more experienced).

We obtained results from the Me and My Class survey for 2916 out of 34 participants (14 schools) for a Year 9 or 10 class taught by the participant and for 29 comparison classes. For Year 9 we have the aggregated and anonymised results for 320 “participant class” learners and 193 “comparison class” learners. For Year 10 we have the aggregated and anonymised results for 318 “participant class” learners and 140 “comparison class” learners. We would strongly caution against too much emphasis being placed on the results of the survey in 2014. Factors to consider are that we do not have data for all participants, we have data for more than twice as many participants’ students as comparison students, and that this was the first time many students had attempted this survey and they may have been exposed to different instructions and conditions.

For the purposes of this evaluation, the aggregated and anonymised results are used to assess the student perceptions of participant teachers’ effectiveness in providing pedagogically rich learning opportunities and in engaging students in their learning. The overall finding is that in no items related to Me in My Class or Me and My Class was there a statistically significant difference between the participant’s class and the comparison class. These tentative results indicate that there is no disadvantage in terms of engagement for students taught by Teach First NZ participants.

For Year 9 there are very few items where participant and comparison classes differed by 5 or more percentage points in either the Me and My Class or Me in My Class sections of the survey. We could tentatively extrapolate that students in comparison classes are reporting having richer learning experiences than those in participants’ classes. Participants’ and comparison Year 9 classes are similarly engaged in their learning, although students in comparison classes report higher levels of engagement and interest.

The Year 10 data tells a different story and one where the students in participants’ classes are more likely to report higher levels of engagement in learning. With caution, we could interpret these data as suggesting that students in Year 10 participants’ classes are slightly more likely to report that they have rich learning experiences. For the two groups of students, there is little difference either in their perceptions of their engagement in class, or in how their culture is respected and valued.

15 Ministry of Education. (2007). The New Zealand curriculum. Wellington: Learning Media. 16 Some participants only had results for senior classes and we did not analyse these data as it would not have been possible to make

any comparisons.


Interestingly, participants’ Year 10 classes appear more satisfied than Year 9—the opposite was true in 2013.

Cohort 13 and Cohort 14 data differences17 In 2014 we separated out data for Cohort 13 and Cohort 14 Year 9 and 10 classes to see what differences there might be in responses. We hypothesised that with an extra year of teaching, Cohort 13 teachers might be engaging students in a wider range of pedagogical strategies and that students might be more engaged in a more experienced teacher’s classroom. The results were as we expected, but with some interesting suggestions of differences. With caution, we could interpret these data as suggesting that students in Cohort 13 classes are more likely to report that they have rich learning experiences, that they are engaged in class and that their culture is respected and valued.

For Year 9 we have the aggregated and anonymised results for 49 students taught by Cohort 13 participants and 192 students taught by Cohort 14 participants. For Year 10 we have the aggregated and anonymised results for 128 students taught by Cohort 13 participants and 167 students taught by Cohort 14 participants. Clearly, caution needs to be exercised into reading too much into these results, given the small sample sizes and the imbalance in numbers for Year 9. For Year 9 the margin of error for the difference in proportions strongly agreeing is 15.7 percent, which means that a confidence interval for the difference in proportions strongly agreeing would be 31 percentage points wide, with differences of about 16 percentage points being statistically significant. Since there were no results outside that range, this means that the differences are not statistically significant, however, larger differences for individual classes are likely to be valuable information for participants.

For Year 10 the margin of error is 11.5 percent, so any difference more than this is likely to be statistically significant. Again, there were no results that were statistically significant. We have only reported on data where there is a relatively large difference in students' strongly agreed responses—10 or more percentage points. It is possible that results could be skewed because more Cohort 14 participants are teaching mathematics and science than Cohort 13, but we have not tested this further by doing cross tabulations of participant subject strength by student responses.

Support by participants for pastoral life of school This evaluative criterion relates to the expectation that participants will contribute positively to wider school activities.

Key findings Participants reported being involved in a variety of aspects of school life allowing them to further develop relationships with students from their own classes and throughout the school. Participants said their involvement in other activities (for example, revision for NCEA and scholarship and homework centres, sports coaching, EOTC, cultural events, and PB4L School-Wide work) had also strengthened their relationships with students. Cohort 14 participants tended to support rather than lead these types of activities.

The Teach First NZ partnership advises participants to develop their teaching skills first before becoming too involved in extra activities, and to limit extra activities to homework groups or similar. Even so, participants take part in a wide range of activities within their schools.

17 See Appendix 3 for a table outlining large shifts on particular items.


Cohort 13 reported being involved in a variety of aspects of school life with revision / NCEA and scholarship (nine of 13 participants) and homework centre (six of 13 participants) being most common. Smaller numbers of participants were also involved in sports coaching, EOTC, kapa haka, cultural events, speech competitions and PB4L School-Wide work. Nine of the participants said that this involvement had strengthened their relationships with students.

Interviewees reported that Cohort 14 participants were involved in similar activities: kapa haka, Polyfest, homework or tutor groups, the school ball, and sports or cultural groups. Sometimes, rather than having a lead role, participants supported these activities with their presence, for example, watching sports or in a supporting role such as driving a minibus. This was seen as a good balance between taking too much on in their first year, but also providing them with the opportunity to develop relationships with students outside the classroom. Participants were positive about their involvement, even if it had taken valuable time. One participant summed up that the “time commitment was balanced out by relationships with the students. I have built that learning relationship [which will be useful] if I teach them next year.”

Leadership Development Strand The Leadership Development Strand is a key element of the programme. The aim is to produce teachers who will, in the long term, provide “a network of leaders in education and across all fields, who are committed to addressing educational inequality”. We assessed the extent to which participants were demonstrating leadership as outlined by Teach First NZ for years one and two of the programme.

Key findings All but two of the Cohort 13 participants took on significant leadership roles in 2014. Participants considered that support from other school personnel and the Teach First NZ Leadership Project had enabled their contribution to leadership. The Leadership Project was seen by a number of participants and other staff members as having a significant impact on the school. Cohort 14 participants in many schools were taking a form class or served as academic mentor for a small group of students. Others were showing leadership in their department.

Teach First NZ identifies that leadership development in the first year should be focused on developing leadership in the classroom. In their second year participants are encouraged to take on more leadership beyond the classroom and one of the requirements of the programme is a leadership project.

Cohort 13 All but two of the Cohort 13 participants thought they had been able to take on wider school leadership roles in 2014, and this was confirmed by other survey respondents. Mentors considered that leadership was dependent on three things:

• the participants’ own initiative and willingness

• the support, opportunities and encouragement offered by the school

• the Teach First NZ / Faculty course work.

The Leadership Project required three related sections:

1. an outcomes-focused description of the project 2. evaluation: an evaluation of progress 3. reflections: planning and managing a leadership project; leading through and with others; building relational trust.


Participants have organised model United Nations, speech competitions and poetry writing events, and parents’ evenings, and one has set up a student council and leadership training for junior students. Others have organised sports teams and outdoor education activities or cultural groups and events. Participants considered that support from other school personnel and the Teach First NZ Leadership Project had enabled them to make a contribution to school leadership. The Leadership Project was seen by a number of participants and other staff members as having a significant impact on the participant’s school. Several projects are described or referred to on Teach First NZ’s website.

Cohort 14 Cohort 14 participants were often taking a form class or were academic mentor for a small group of students. In some schools they were involved with other initiatives such as Te Kotahitanga and Starpath. Others were showing leadership in their department, discussed later in this chapter. A few participants articulated things they hoped to achieve as they developed their teaching career, such as building up a department.

Programme impact on quality of teaching and learning in participating schools and the status of teaching

We assessed the impact for schools of taking part in the programme. We also asked Cohort 13 survey respondents how much positive impact they thought Teach First NZ had had on the perceived status of teaching as a competitive profession. Our evaluation criteria included that Teach First NZ attracted growing numbers of high-calibre Māori, Pasifika, and male applicants to the programme.

Key findings All Cohort 13 co-ordinators and principals considered that participants had had either a “high positive impact” or “some positive impact” on the teaching and learning in the school. Many school personnel in Cohort 14 schools noted the contribution participants made to their department and in staff meetings and their willingness to share new ideas. Three-quarters of Cohort 13 participants thought Teach First NZ had a positive impact on the perceived status of teaching as a competitive profession. A number of Cohort 14 participants said they were now more aware of the status of teaching and of how important and demanding a profession it is. There was some indication that Cohort 14 had actively encouraged friends and university colleagues to apply for the programme. Cohort 14 included more Māori, Pasifika, and male applicants than Cohort 13.

In the survey responses in relation to Cohort 13, five of the nine co-ordinators and one principal considered that participants had had a “high positive impact” on the school and the remainder considered that they had had “some positive impact”. Most respondents thought there had been a positive impact. However, many mentors did not think that teachers in non Teach First NZ schools knew about the programme. One worried that this could affect the participants’ ability to get a job—but this has not been the case.

In interviews, many school personnel in Cohort 14 schools noted that the participant was “a positive person in the department”, and shared examples of their contributions at staff meetings and willingness to share new ideas. They commented on participants’ enthusiasm for the subject, which was particularly notable where participants had higher degrees in their curriculum area. We asked Cohort 14 participants if they felt differently about the status of teaching and teachers than they did before they joined the programme. Seven said they were now more aware of the status and of how important and demanding a job it was; the others had already held positive views on the role of teachers or said it


was not something they thought much about. At least one participant said the “ripple effect” (her words) is that she is now luring in other graduates to apply for ITE programmes.

We have reported on the demographic makeup of the cohorts elsewhere in the report, most fully in the section on “Who the programme attracted” (page 16).



Discussion

In this section of the report we provide some general observations about the Teach First NZ programme in 2013 and 2014 in relation to the evaluation questions. We describe some of the strengths of the programme and some issues with its implementation, as at November 2014. Our findings are based on documents that we had access to, survey data (for Cohort 13 participants, mentors, co-ordinators and principals), data from the NZCER Me and My Class surveys, site visits (which consisted of interviews with Cohort 14 school principals, mentors, and co-ordinators and participants), and interviews with Teach First NZ and Faculty.

How well (effectively and efficiently) has the programme been implemented? Now into its second year, the programme continues to be effectively and efficiently implemented. There were two cohorts in 2014 with 35 participants in all—15 from Cohort 13 and 20 from Cohort 14 (one of whom left in April). There have been a few staff additions and changes at Teach First NZ and some 2013 schools have new principals, co-ordinators, and a number of new mentors. Seven Cohort 14 participants joined Cohort 13 participants in their schools, and 12 of them were placed in seven schools that were new to the programme in 2014. These changes have been well managed and, for the most part, the programme has been strengthened by them.

The participants It will always come down to the individual who goes through the programme. (Cohort 14 co-ordinator)

Most staff I have spoken to believe that this is a great way to get into teaching as it is essentially a learned profession. However, they (high and low decile alike) do feel sorry for Teach First participants as they are ‘thrown into the deep end’. Ultimately, the feedback I have received says that participants of Teach First NZ need to be confident, charismatic, resilient and passionate people if they were to be successful in such an arrangement. (LAS)

The rigour of the selection process continues to be one of the major strengths of the programme, with everyone involved commenting on the high quality of participants. The very small number of Cohort 13 participants who were finding teaching more challenging than other participants made significant progress in 2014. All Cohort 13 participants were seen to be a great deal more confident, independent, and effective as teachers as they neared the end of their second year. A Teach First NZ staff member described the privilege of “seeing participants overcoming hurdles—with a nudge in the right direction they come out so much stronger”.

Teach First NZ noted the strengthening of the programme with two cohorts. Two staff commented on the differences in the two cohorts which provide valuable learning for everyone involved. A Faculty member described it as a “huge plus” to be able to create a professional learning community of participants. Cohort 14 is seen to have different expectations of the programme (noticeably in relation to how te Tiriti o Waitangi should be honoured). Some Cohort 13 participants commented that changes they had suggested to the programme were benefitting Cohort 14. The higher numbers of


science-related teachers has created an interesting dynamic in that some are the only chemistry and physics teachers in their school. That means they are taking more senior classes and that subject support outside the school may be needed.

Some LASs considered that the schools had gained “exponentially” from having energetic, enthusiastic, intelligent, committed, hardworking, and ambitious participants. Others commented on the value of the school having an increased awareness of quality mentoring. One considered that some schools had gained more than others from participating in the programme and that this was dependent on the quality of the participant. LASs stressed three things that participants benefited from: hands-on experience of learning to teach; an understanding of the education profession; and opportunities for critical reflection on themselves and their teaching.

One of the effects of the very strong selection process has been a high retention rate—only two participants (one from each year) have left the programme and both for personal reasons. All of Cohort 14 intends to stay in the programme for 2015 and the majority indicated that they intend to stay in teaching in the foreseeable future. Thirteen of the 15 Cohort 13 participants intend to go on teaching in New Zealand in 2014 and they had secured jobs by the end of November. While we have no hard data about whether principals are choosing to employ Teach First NZ alumni above other applicants, clearly they are very employable.

Where these teaching jobs are, however, provides one of the conundrums for the programme. Four participants have been retained in their original host schools. In other schools staff members told us they would have liked to retain their Teach First participants. The current reality of school rolls in Auckland in particular means that many schools serving low socioeconomic communities have falling rolls and are not in a position to take on new staff. One of the consequences of the “away practicum” in affiliate schools been that one participant accepted a job at an affiliate high-decile school. While just over half of the alumni are staying in low decile schools, some others accpeted jobs in high decile schools despite (according to a Teach First NZ staff member) “their desperate wish to stay in low decile schools”.

One of the things that became clearer to us this year is the extent to which the participants are being prepared as 21st century future-focused teachers: they are confident about opening up their classrooms and their practice to ongoing scrutiny; they are fluent users of digital technology in teaching and learning; they are skilled at critical reflection; they collect and use assessment data confidently and competently to improve learning; and, as one LAS said, they are “pro-active in their approach to implementing research driven learning programmes”. The LAS commented:

[they] are more open to critique, having learned to accept that they are observed as if they are in a ‘fishbowl’. They have had more demands placed on them and so are tuned to professional advancement through focussed hard work. (LAS)

They are also adept at talking publicly about teaching. It is early in the evaluation to be able to measure participants’ contribution to the status of teaching, but one Faculty member described as a highlight the 2013 end of year function where Cohort 13 talked to Cohort 14 about teaching. He described it as “inspiring to hear young people talk about their schools” and “transforming to have young people saying publicly that it’s a fabulous job.” For Teach First NZ a highlight has been “the participants, their growth, how they’ve changed and very quickly as well. The kinds of messaging that they provide to the [new] applicants.”

The programme Participants in Cohort 14 reported that they felt well prepared by the SII, including in relation to behaviour and classroom management. The residential aspect continues to be a challenging aspect, but one which is valuable and rewarding.


Positive comments about the programme from Cohort 13 and Cohort 14 participants far outweighed any comments about things that could be improved. On the whole participants found the taught programme and the assignments relevant, and they were positive about the clinics and noho marae held during the year. These provided valuable time with curriculum specialists and an opportunity to reconnect and “reinvigorate” with their peers in both cohorts. A number of Cohort 13 participants and others that they worked with described the positive impact of their inquiry and leadership projects on their learning and on the school.

There have been subtle rather than major changes to the programme, mostly in response to feedback from Cohort 13 participants and schools and also because in effect the programme has more than doubled in size and complexity.

With one cohort it was easy to get a sense of whanaungatanga—now with two we’ve worked on getting this sense with two cohorts and we’ve made quite a few changes—to clinics and noho—shared occasions. The noho was a pivotal moment in terms of getting our culture right—building on what we had and trying to weave it together. (Teach First NZ)

Teach First NZ told us they had strengthened their internal review practices to provide greater transparency in their practice. This has included improving their in-school support, their surveys, and their selection processes, especially as they relate to Māori applicants. They described a stronger system for following up on participants which includes built-in mechanisms for following up and record keeping, using a “concerns action log”. The development of the Leadership Development Strand, where participants work with Teach First NZ on a leadership project that draws together leadership theory and practice, has been a highlight. Addressing Māori tikanga appropriately has been a major focus and has included “engagement with Ngāti Whātua and thinking about our responsibility in this area”.

The programme genuinely attempts to understand indigeneity, the impact of children living in poverty etc.—Teach First puts this to the fore. We had to get people to buy into it but they came to recognise these things as strengths. (Faculty)

The schools Seven new schools joined the programme in 2014, with four of them being decile 4 and 5. The new Auckland schools have large rolls, including high numbers of Māori and Pasifika students. It is clear that almost all principals, co-ordinators and mentors had positive experiences of having Teach First NZ participants in their schools. They described their overall impressions in terms such as “exciting”, “very positive”, and “a refreshing approach”, and said that their schools had gained tangible benefits from being involved with the Teach First NZ programme. New young staff, especially when supernumerary, were particularly welcome in struggling departments and hard to staff areas. School staff acknowledged the new opportunities for staff PLD, especially mentors who gained “extra expertise” and the shared learning that developed out of conversations between participants, their mentors, other PRTs, and other staff. Some commented specifically on the benefits from building a relationship with the Faculty and an ITE programme.

Weaknesses of the programme that we identified in 2013 remained in 2014, albeit for a small number of participants. These issues included: poor internal communications in some schools; variable quality of mentoring; participants being placed in departments that were unable or unwilling to support them; and participants being the strongest subject specialist in the school. It was clear that some Cohort 13 participants were enjoying more wrap-around support than others.

Most principals, co-ordinators and mentors consider that the communication between the Teach First NZ partnership and the schools is clearer than in 2013 which places them in a better position to support participants. Despite this, not all schools recognise the high levels of support participants need initially in relation to lesson planning and behaviour management, nor the ongoing mentoring required. In some instances there is a “Catch 22” effect operating. The school


wants to build up or re-energise a department and employs a Teach First NZ participant without providing the necessary subject and departmental support. In some instances, schools have understood their need to provide wrap-around support and used mentors from other areas of the school and external subject support.

However, we are aware that a few Cohort 13 and Cohort 14 participants have been placed in situations that offer little support and where participants have had to negotiate better conditions for themselves. While this is not typical, it does mean that different arrangements for participants in less supportive environments may need to be provided by the Teach First NZ partnership. In our view, assigning participants to schools that are not yet able to provide effective mentoring support may compromise the quality of the programme. This is likely to affect the extent to which the pilot programme can be expanded as fully as intended, unless creative solutions are found.

Ensuring high-quality mentoring for all participants continues to be an important focus of the programme. A number of schools recognise the dual benefits of the mentoring programme and the ability to develop another teacher. Teach First NZ commented that “the schools that need to engage with us the most are often least able to—the smaller, low decile schools”. Teach First NZ considers that while every situation is different, and therefore flexibility is important, “we need to be clearer about our expectations about core parts of it—timetabling, measuring hours, the type of data that we get.” Faculty are working hard to improve the quality and accountability of mentoring by including principals and co-ordinators in communications related to mentoring.

To what extent has the programme achieved its overall outcomes and objectives?

Feedback from principals is that Teach First NZ is delivering exactly what it promised at the outset—highly qualified teachers committed to working in schools serving low decile communities (Faculty)

Our findings in 2014 are similar to 2013, with the programme being strengthened as it continues to develop. The strengths of the programme are in: the selection processes; the quality, resilience and commitment of participants; the responsiveness of Teach First NZ and University of Auckland Faculty to all feedback and their ability to make changes and improvements as required; and the willingness of schools to embrace the programme. Responsiveness is in part due to the small size of the programme, but in no small part due to the quality and commitment of staff and the strong respectful partnership between Teach First NZ and the Faculty. This was described to us by a Teach First NZ staff member as “a sense of values such as mutual respect and striving for excellence”. A LAS summed up the strengths of the programme:

The learning on the job, the calibre of the participants, and the support infrastructure for the participants which requires a true partnership between the school and Teach First to make it work.

Most principals and co-ordinators clearly believe that this programme is likely to produce quality teachers; indeed, a few said that participants already were quality teachers in Year 1 of the programme. They reiterated the key elements of the programme as the robust selection process (“it will always come down to the individual who goes through the programme”), support and mentoring, and immersion in the classroom. These were seen as essential components for the Teach First NZ programme to produce quality teachers.

Participants are seen to have acquitted themselves well, increasingly contributing to the wider life of the school over 2 years and growing in confidence and independence. Cohort 13 participants are judged to be at least similar to other


PRTs in their second year of teaching, with seven of the nine co-ordinators considering them to be as effective as a fully registered teacher.

By September in their first year of the programme, participants are considered to be “a step ahead” or “in a different league” from student teachers on practicum. They also compare favourably with PRTs, although many co-ordinators and principals commented positively on the strong PRTs that they have been able to employ in recent years.

Cohort 13 participants remain strongly committed to the mission and values of Teach First NZ and to the organisation. Almost all consider that they have continued both to be supported in their second year and to be challenged by course work and their teaching. Here are the words of two Cohort 13 participants which capture their experiences.

The mission is admirable and I believe wholeheartedly in it. I have had some very rewarding experiences of connecting with students and motivating them to achieve more than they thought possible. I truly love my students and respect their experiences, talents and ambitions...

It’s been very challenging, very imperfect. I think it could’ve been better for me if I’d been at a different school... I have learned a lot outside of my subject areas by virtue of being at my school so I’m grateful for those things. I feel I’ve been well supported in finding a job next year.

Cohort 14 participants are committed to completing the 2-year programme, and four-fifths intend to continue teaching. They are positive about the Teach First NZ model as a way to become a teacher, while acknowledging it wouldn't suit everyone. Participants consider that “the apprenticeship model is a fantastic model of teacher education”, “It’s been one hell of a ride!” and “It’s what I hoped it would be, and more”.

As with 2013, in 2014 each participant is in a unique teaching situation and reliant to a large extent on the quality of their in-school support. While schools and the Teach First NZ partnership are working hard to ensure that each participant is in an optimum learning situation, there is considerable variability in experience. That 13 of the 15 Cohort 13 participants have remained in teaching and have secured teaching jobs for 2015 is testimony to the success of the programme for that cohort. A further Cohort 13 participant has secured a teaching job to start in 2016. The third and final intake of the pilot has secured 20 participants of whom eight (40 percent) are male, five Māori (25 percent) and one Pasifika (5 percent). Three will teach mathematics, two physics, three chemistry, and four te reo Māori.

There has been considerable learning, consolidation, and change in 2014.

This year we have more than doubled size of programme—from 16 to 34.18 Never again will this happen and this has been a huge transition for us. And at the same time we’ve also been preparing for our first alumni—how do we engage and support a community of people committed to supporting education in low decile communities no matter what job they’re in (Teach First NZ)

Faculty would like to get more of their staff involved and get the programme more embedded in the general work of the Faculty. A Faculty member recognised the need to establish a rūnanga/advisory group as current Māori staff are “very thin on the ground” and “feel a weighty responsibility”. According to another Faculty member there is work to be done to “meld the mentors into a professional learning community — so that they talk about their role with pleasure. To build in them a scholarly approach to mentoring. Many of them are not approaching mentoring as professional learning”. Earlier in the report we have recounted principals, co-ordinators’ and mentors’ views about the mentor role. There

18 Cohort 13 had 16 participants in 2013 with one leaving at the end of the year.


appears to be a need for further discussion among all parties about the power of effective mentoring and the benefits for all concerned.

Teach First NZ is continuing discussions with the Ministry and the Teachers Council in the hope that the pilot can transition into an ongoing programme. We think it is highly likely that the partnership can continue to run the programme at its current capacity and in its current form, based on what we have seen. However, school profiles and roll predictions, especially in Auckland, a tight labour market, and comparative difficulty in recruiting physics and chemistry graduates, need to be considered for any ongoing development of the programme. There are issues with the “away practicum” in the affiliate schools and registration status that need to be sorted out with the Teachers Council or the new professional body EDUCANZ.

One of the key platforms on which Teach First NZ is based is that participants remain connected to and active within the education community. Their website states:

After successful completion of the two-year programme, participants will become Teach First NZ alumni. Our alumni network forms a new community of leaders: highly competent, passionate, socially-conscious people, who share a powerful common experience in schools.

Our alumni programme aims to equip these leaders with the motivation, skills and understanding to remain engaged with our mission to tackle educational inequality, from both inside the classroom and beyond.

To this end, during their two-year commitment participants receive training, support and networking opportunities designed to prepare them for leadership roles in a variety of fields ...

It will be interesting to see the development of the alumni and the extent to which Teach First NZ can influence the status of teaching and continue to provide effective teachers for schools serving low socioeconomic communities. Having participants working as PRTs in a range of schools from 2015 is likely to raise the profile of the programme within the secondary school community.

The 2014 evaluation has confirmed that in general the Teach First NZ programme is being implemented effectively and efficiently, and that it has benefited rather than suffered from doubling in size. Improvements have been made to the programme in 2014, based on feedback from 2013. The Teach First NZ partnership continues to find ways to strengthen the programme and to ensure it is well known and well supported. Participants are very strong ambassadors for the programme and for secondary teaching in New Zealand. Almost all participants have achieved highly, have supported their students to do well, including in NCEA, and intend to stay in teaching at least in the short term.


Appendix 1: Programme purpose The Teach First NZ programme was established as an initiative intended to contribute to tackling educational disparities in New Zealand.19 A core element of the initiative is the introduction of a new approach to the preparation of secondary teachers in New Zealand. The programme was developed by the Faculty of Education, University of Auckland (the Faculty), and Teach First NZ, operating in partnership. The Ministry of Education (the Ministry) is funding some aspects of the programme.

The programme is designed to respond to a longstanding need for quality teachers in schools serving low socioeconomic communities with high Māori and Pasifika enrolments, and the pilot is focused on preparing teachers to teach hard-to-staff secondary subjects in Auckland and Northland. English, chemistry, physics, mathematics, and te reo Māori were deemed to be hard-to-staff secondary subjects.

The 4-year pilot programme now sits within a wider government Quality Teaching Agenda which was announced in 2013. The New Zealand Council for Educational Research (NZCER) was contracted to evaluate the pilot programme between 2013 and 2016.

While a core part of the initiative is an alternative pathway into teaching, the aims of Teach First NZ are wider than a focus on preparing new teachers. Teach First NZ’s broader aims include building a community of programme alumni leaders who remain committed and who work to advance educational opportunities over the long term, from within schools, the wider education sector, and other sectors of influence.

The aims of the partnership (the Faculty and Teach First NZ) in developing the programme are:

• to contribute to reducing educational disparities and inequalities in New Zealand by preparing participants20 with the knowledge, skills, and dispositions necessary to become highly effective teachers in secondary schools with high Māori and Pasifika rolls serving lower socioeconomic communities, initially in Auckland and Northland

• to make teaching in schools serving low socioeconomic communities a first choice for graduates

• to develop the leadership capability of participants so that they have the potential to contribute to reducing educational disparities as alumni of the programme

• to contribute to supporting and improving the status of teaching as a profession in New Zealand by providing a rigorous alternative pathway into teaching that attracts high-calibre, talented individuals (particularly male, Māori, and Pasifika), including some who may not otherwise choose to become teachers in schools serving lower socioeconomic communities

• to work towards the programme’s vision that all young people in New Zealand achieve their full educational potential, regardless of socioeconomic background.

There are three intakes in the pilot programme with each intake undertaking field-based teacher preparation for 2 years. Each intake is restricted to a maximum of 20 participants. In this report we refer to the first intake as Cohort 13, the second as Cohort 14, and the third as Cohort 15.

19 The Teach first NZ website has more detail about the programme. www.teachfirstnz.org . 20 Throughout this report participant refers to the Teach First NZ beginning teacher. We do not use the term to refer more broadly

to the participants in the evaluation.

http://www.teachfirstnz.org/


The Ministry funded Cohort 13 participants as supernumerary (above school roll entitlement) at 0.6 for 2 years, and funded mentors at 0.2 Full-time Equivalent (FTE) throughout the pilot. The Ministry’s aims in providing funding to assist in the delivery of the programme are:

• to expand the pathways available for talented potential teachers to enter the teaching profession

• to actively promote and work towards achieving the Government’s priority to raise achievement and support delivery of the Better Public Service target of 85 percent of 18 year olds achieving the National Certificate in Educational Achievement (NCEA) level 2 by 2017

• to increase the engagement and achievement of Māori and Pasifika students in secondary schools.

• Cohorts 14 and 15 participants’ salaries are funded by the host schools with added Ministry funding to support a reduced teaching timetable for participants (a teaching load of 0.6) and 0.2 FTE per participant for mentoring.

Programme aspects For each cohort, the 2-year programme begins with selection and the Summer Initial Intensive (SII). Once participants begin teaching in their host school, their course work continues throughout the year with clinics, workshops and noho marae. In their second year, participants undertake a 3-week “away practicum” at an affiliate school. A leadership strand runs alongside the school-based and Faculty course work.

Teach First NZ has five key expectations of participants:

1. Completion of the SII (the final stage of the selection process), including:

• attending all compulsory SII sessions and events

• actively contributing to their own learning, reflection, and the learning of others

• completing all tasks, written assignments, and reflections to the required standard

• attending and participating successfully in practicum, observations, and teaching in schools

• demonstrating the potential to meet New Zealand’s Graduating Teacher Standards (GTS)

• being respectful of the SII learning process, environment, and facilities.

2. Employment in the host school. It is expected that participants remain employed full time in the same school throughout the 2 years of the programme. (While employed/paid full time they are expected to teach a 0.6 FTE timetable.)

3. Completion of the teaching qualification.

4. Completion of the Teach First NZ Leadership Development Strand.

5. Working within Teach First NZ’s core values:

• integrity/te ngākau pono

• respect/te whakaute

• partnership/te mahi ngātahi


• continuous learning/te ako mā te whakaaroaro

• resilience/te manawanui me te ū

• excellence/te kairangitanga.

Within the partnership, the University of Auckland’s Faculty has particular responsibility for teacher preparation leading to the university’s Postgraduate Diploma in Teaching (Secondary Field-based) and, through the completion of this programme, to provisional registration. The Teach First NZ team supports recruitment and selection, the design and delivery of the SII, drop-ins and clinics, and the alumni programme, and contributes to ongoing participant development and support. All aspects of the programme are planned together and its delivery is shared to enable participants to experience it as an integrated whole.

The selection process is designed to be rigorous and multifaceted. It includes an online application, a phone interview, a day-long assessment centre, and the SII. Those ultimately selected for the programme will have demonstrated:

• degree-level academic success in the subjects that they aspire to teach

• the organisational skills needed to manage teaching and challenging academic study

• commitment to the Teach First NZ vision

• commitment to redressing educational inequities

• cultural competence

• leadership capabilities (likely to benefit their schools and communities) and the interpersonal and social skills required of a leader.

Teach First NZ’s ten core selection competencies are: Achievement; Leadership; Perseverance and resilience; Influencing and motivating; Humility, respect, and empathy; Organisation and planning; Problem solving; Self-reflection; Flexibility; Commitment to the Teach First NZ mission.

Participants are enrolled in the Postgraduate Diploma in Teaching (Secondary Field-based) throughout the 2 years and 2 months of the programme. Participants must meet University of Auckland standards for all ITE graduates and meet the New Zealand Teachers Council’s (Teachers Council) GTS. A maximum of two participants in each cohort are employed in each host secondary school with high Māori and Pasifika rolls in Auckland and Northland—schools that mainly, but not exclusively, serve low socioeconomic communities. The programme aims to prioritise the placement of participants in schools with a culture of inquiry and data use. Participants are asked to make a commitment to working in their host schools for 2 years and sign a commitment agreement to signal this intention.

The programme over 2 years consists of five integrated university papers: EDPROF 700 (Interdisciplinary Pedagogy in New Zealand); EDCURSE 709 (Summer Intensive); EDCURSEC 719 (Learning Area Inquiry); EDPRAC 751 (Teaching 2013); EDPRAC 753 (Teaching 2014). Assessments include the following:

• During SII: a teacher learning log, a seminar, a case study, a reflection portfolio, and planning and assessment tasks.

• During the year: a portfolio of practice (critical reflection on self and critical analysis of the progress of a small group of learners and three formal assessments of teaching practice).

• The collection of evidence that the participant has met the GTS.

• In Year 2, an extended inquiry project.


After the initial 8-week SII residential programme in November/December/January, participants are employed in their host schools with Limited Authority to Teach (LAT) status. Class contact time for participants is limited to 0.6 FTE.

Nominated teachers in each host school are offered specialised mentor training by the Faculty before and during the programme, and each school selects a mentor who is intended to be well matched to an individual participant. Each host school has a designated member of staff who co-ordinates the mentor and participant to maximise the contribution of the participant at the host school. There is ongoing support available from the Faculty for mentors and for co-ordinators. Participants are visited and observed (to receive formative feedback and evaluation) by Visiting Teaching Specialists (VTSs) or Learning Area Specialists (LASs) on up to 15 occasions in Year 1 and up to 10 occasions in Year 2. Pastoral support is provided through the Partnership Team of Teach First NZ, which works with participants from the SII and collaborates with LASs and VTSs.

Participants are also engaged in a leadership development strand, which is closely integrated with the ITE elements of the programme. Alumni will be supported to engage with an alumni programme after the 2 years. During their 2 years participants attend four day-long clinics (March, June, August, November) and a three-day mid-year intensive (residential) in July which includes a noho marae. They are also required to spend a number of weeks in contexts other than their host school as an “away” practicum in local schools or in the home school community, for example, in early childhood services or community outreach initiatives. In the participants’ second year this consists of a 3-week practicum at a secondary school—usually arranged by the Faculty in consultation with the participant.


Appendix 2: Methodology

Purpose and scope of the evaluation This evaluation’s main purpose is to understand how well the programme has been implemented and to what extent it has achieved its objectives.

This evaluation is designed to provide timely and relevant information to:

• support decision making about ongoing implementation of the programme

• develop a deeper understanding of alternative pathways into the teaching profession, particularly pathways for talented potential teachers entering the teaching profession

• generate learning about effective approaches to aspects of ITE

• understand the effectiveness of the programme in terms of its stated immediate, medium-, and long-term aims.

More broadly the evaluation will contribute to national and international understandings of ways to prepare knowledgeable and effective teachers who are able to have positive effects on students’ motivation and engagement, and ultimately on outcomes for students.

The purpose of the evaluation is not to compare the quality or effectiveness (impact on student learning outcomes) of the graduates of the pilot programme with other “alternative” or “traditional” ITE programmes in New Zealand. Nor is the purpose of this evaluation to compare Teach First NZ with the Exemplary Models of Postgraduate ITE programmes, which began in 2013. Value for money is also not within scope for this programme evaluation.

Key evaluation questions The key evaluation questions are:

1. How well (effectively and efficiently) has the programme been implemented? 2. To what extent has the programme achieved its overall outcomes and objectives?

At this point of the evaluation, that is, halfway through the pilot programme, evaluation question 1 has a formative as well as a summative purpose. The Ministry developed a list of indicative subquestions for both questions, which we used as a starting point to confirm the evaluation scope, focus, and design. These are shown in Appendix 4. Note that these subquestions were not transferred verbatim into the evaluation plan, but were significant in guiding the approach agreed between NZCER and the Ministry.


Evaluation approach Between April and June 2013 the evaluation team developed an evaluation plan, based on our reading of the international literature on similar Teach First sister programmes (which are independent programmes but part of the global Teach For All network), interviews with key stakeholders (the Ministry, Teachers Council, PPTA, Teach First NZ partnership), and review of the available material on Teach First NZ.

We developed evaluation criteria and possible data sources to address each of the key evaluation questions. This approach was endorsed by the partnership. The first evaluation question addresses how well the programme has been implemented from the perspectives of Teach First NZ participants, host school teachers, and Faculty staff. The second evaluation question concerns impact and is addressed more fully in this report than it was in the 2013 annual evaluation report. This question will be given even greater attention in subsequent reports. Table 1 sets out the evaluation criteria for evaluation question 1.


Table 7 Evaluation question 1, evaluation criteria and data sources Key evaluation questions Specific areas for investigation Evaluation criteria Data sources

How well (effectively and efficiently) has the programme been implemented?

Who the programme attracts Programme attracts high-calibre* participants, some of whom may not otherwise have undertaken teaching (particularly in schools serving lower decile communities) at this time.

Document analysis (i.e. candidate applications (with permission); candidate academic records; interviews

*High calibre as defined by Teach First NZ.

Retention rates Retention rate for the 2 years is 90% Programme records

Summer Initial Intensive School Observation and Reflection (SOAR) prepared participants well for their Summer Initial Intensive

Summer Initial Intensive (SII) strengthened participants’ motivation to teach in schools serving low income communities

SII strengthened participants’ understandings of the cultures of their students and how to incorporate this understanding into their teaching

Participants felt well prepared to begin teaching

Intensive built a sense of “connectedness” within the cohort21

Data collected by providers during and after the intensive

Participant interviews

Programme personnel interviews

Programme factors There is a common clear vision of effective teaching

Vision permeates course work and practice in schools

Document analysis (for example, U of A programme resources and documents)

Interviews with participants, programme and school personnel

21 For Cohort 14 the focus shifted to: - Participants found the SII helpful and useful - Participants felt well prepared to begin teaching - Participants identified areas where they needed more preparation - Participants recommended improvements to the SII for the next intake.


Key evaluation questions Specific areas for investigation Evaluation criteria Data sources Strong core curriculum

Use of case methods and teacher research

Participants confront and rethink assumptions about learners.

Participants judge the course work to be intellectually challenging and practically relevant.

Support to participants from Teach First NZ programme

Participants feel that their wellbeing has been important to Teach First NZ.

Participants feel that they are part of the Teach First NZ community.

Interviews with participants

Surveys

Support to participant from host school

Host schools are well prepared for, and supported in their roles

The quality of mentoring, and school support has enabled participants to be successful beginning teachers.

Host schools provide optimal opportunities to support participants to develop their teaching expertise.

Host schools rate their preparation and support for their roles as high quality.

Host schools feel well supported by the partnership.

Document analysis (participant, school staff records, minutes, resources etc.)

Interviews with school personnel

Participant interviews

Selection of affiliate schools Affiliate schools provide useful learning experiences for participants.

Document analysis (for example, programme resources and documents)

Interviews with programme personnel, relevant staff in schools, participants

Support to affiliate schools Affiliate schools are aware of their responsibilities to the participants.

Affiliate schools feel well supported by the partnership.

The exchange is judged to be useful by the affiliate school.

Document analysis (for example, programme resources and documents)

Interviews with programme personnel, relevant staff in schools


Key evaluation questions Specific areas for investigation Evaluation criteria Data sources

Mentor teachers Mentor teachers provide regular high-quality observation, mentoring and feedback to participants.

Mentor teachers are well supported by the partnership.

Mentors help participants to become part of the wider school community.

Document analysis

Interviews with mentors


Visiting Curriculum Specialists (now called learning area specialists (LASs) and visiting teacher specialists (VTSs)

LASs and VTSs provide regular high-quality observation, mentoring, and feedback to participants.

There are explicit links between the curriculum papers in the qualification and the feedback provided by the specialists in schools.

Document analysis (for example, programme resources and documents; participant records)

Interviews with LASs and VTSs



Programme responsiveness The programme has evidence of its responsiveness to feedback from participants and participating schools.

Document analysis

Interviews with student teachers, school personnel, programme personnel

Surveys (perhaps at end of pilots)

Viability The Teach First NZ model can successfully adjust to larger numbers of participants within agreed funding arrangements.

Host schools are able to support the programme when they self-fund participants’ salaries from 2014.

Interviews with stakeholders

Surveys (perhaps at end of pilots, across all groups)

The second high-level evaluation question calls for a summative focus. Table 2 sets out the evaluation criteria for evaluation question 2.


Table 8 Evaluation question 2 and intended data sources Key evaluation Questions Specific areas for investigation Evaluation criteria Data sources

To what extent has the Teach First NZ programme achieved its overall outcomes and objectives?

Effectiveness of participants’ teaching High engagement of students, especially Māori and Pasifika

Teaching aligns with Teach First NZ model

Student achievement in classes taught by the participants is as good as or better than the achievement of other students in the same demographic.

Student engagement surveys (Year 2 for each cohort) 6Student attendance data

Document analysis (from mentor feedback to participants (from, for example mentor teachers, visiting specialists)

Classroom observations (not in budget, but highly desirable)

Interviews with mentors, curriculum specialists, other teachers in same department as the participant and principals

Student achievement data if appropriate and available.

Further exploration is required to determine the feasibility of using student achievement data.

Support by participants for pastoral life of school

Participants contribute positively to wider school activities.



Leadership Development Strand In first year participants demonstrate effective leadership of students

In second year participants successfully lead a professional practice project within the school

From year three, participants build on the leadership experience of the first 2 years.

Document analysis (feedback from lesson observations)

Examination of the participants’ projects

Interviews with Teach First NZ, school personnel.

Survey (end of Year 2 for each cohort)

Alumni survey (2015, 2016)

Ongoing involvement and/or retention of participants

Alumni stay in teaching or wider education after the programme, or continue to contribute to reducing educational inequalities through other pathways.

Programme graduates can articulate their

Retention data

Survey of alumni (2015, 2016)


Key evaluation Questions Specific areas for investigation Evaluation criteria Data sources key learnings from the 2 years as they begin their journey as alumni, and can describe how the 2-year programme has helped shaped their future paths.

Programme impact on quality of teaching and learning in participating schools

Participating schools can describe how (if) the programme has contributed to improving the quality of teaching and learning in their schools

Interviews with school personnel from participating schools

Survey of teachers from participating schools (2016)

Status of teaching Growing numbers of high-calibre Māori, Pasifika and male applicants for the programme

Programme data

Enablers and barriers to success Participant survey at end of 2 years

Alumni engagement Graduates continue to participate in alumni programme

Programme records

Alumni surveys in 2015, 2016 (two cohorts)


The key aim of the evaluation in 2014 has been to ascertain how well the Teach First NZ programme has been implemented, and what changes have taken place between 2013 and 2014. We have also been able to evaluate the effectiveness of implementation in relation to some outcomes for Cohort 13. We chose a mixed-methods design, collecting both quantitative and qualitative data.

We collected data from four main sources: document analysis; interviews with participants, key personnel in Cohort 14 schools and with the Teach First NZ partnership and the Faculty; an NZCER online survey (Me and My Class) for students; and online surveys for Cohort 13 participants, their mentors, co-ordinators, and principals. Interview and survey questions were based on the evaluation criteria. This enabled us to compare data for Cohort 13 across both years and survey and interview data related to programme implementation and school experiences across Cohorts 13 and 14. Qualitative data from interviews were used to develop deeper understanding of processes, including the factors that enhanced or hindered the effectiveness of programme implementation. Quantitative data from the Me and My Class surveys were used to measure student engagement and student perceptions of participants’ pedagogical approaches compared with students in comparison Year 9 or 10 classes. We chose a Year 9 or 10 class for comparison because all participants were teaching at least one of these, although participants also taught a range of senior classes. We attended sessions at the SII in December and January and a mentor and a co-ordinator meeting.

Document analysis We were provided with copies of:

• information about participants, and contact details of personnel involved

• recruitment and selection material, including Assessment Centre protocols

• programme description and course outlines, including assessments

• material provided by Teach First NZ to schools.

During site visits with Cohort 14, we also had access to:

• participants’ portfolio material—material towards meeting the GTS and the RTC (online and hard copy)

• written observations and feedback from school-based mentors and faculty-based VTSs

• participants’ folders containing lesson plans, copies of student work, assessments, and assessment results.

Cohort 13 surveys The evaluation plan allowed for site visits and interviews in Year 1 for each cohort and online surveys in Year 2. We had anticipated we would also do phone interviews with participants in their second year, but it was agreed with the Ministry that we could obtain sufficient data through the surveys without doing this as well. We also recognised that participants are very busy and we could find it difficult to schedule an interview and for participants to devote time to an interview and a survey.

A link to the online survey was emailed to all 15 participants, 16 mentors, 9 co-ordinators, 6 principals (three principals were new to their school and were deemed to be unable to answer questions about participant progress from 2013–14) and the five LASs responsible for Cohort 13. The surveys are provided in Appendix 5. On the advice of co-ordinators and mentors, the link went live in the middle week of the October term break and was closed on 24 October 2014. Response rates are presented in Table 3.


Table 9 Online survey responses 2014 for Cohort 13 Respondent Number returned Percentage

Participant 13 87

Principal 3 60

Co-ordinator 9 100

Mentor 15 (3 mentors mentor two participants; 3 participants have 2 mentors; some mentors are also co-ordinators)

87

LAS 5 100

Total 46

The response rate was high for all respondents except principals from whom we got a 60 percent response rate.

Cohort 14 interviews We visited the 13 schools between 25 July and 9 September, spending one half to one full day in each school. At each site we interviewed participants, mentors, and co-ordinators. We interviewed principals only if they had not been interviewed last year. Where a staff member had more than one role, for example, co-ordinator and mentor, we conducted one interview. The interview schedules are attached as Appendix 6. These were structured interviews and took place face-to-face and on site. The interviews typically lasted 30–40 minutes with mentors, co-ordinators, and principals and up to an hour with participants. Interviews were digitally recorded and the recordings filed as a check for veracity. The recordings were not transcribed.

As with surveys, interviews provide self-reports and could potentially be a design weakness because of the potential for self-reporting bias or lack of shared understanding of the intention of the questions. To mitigate this potential, we asked interviewees many of the same questions so that responses from a variety of people about aspects of the programme could be triangulated to check the validity of claims from different perspectives. This does not mean that all groups shared the same understanding of aspects of the programme.

Interviews with Teach First NZ and University of Auckland Faculty In October 2014 we interviewed five Teach First NZ staff and three Faculty staff. The interview schedule is attached as Appendix 7. These were structured face-to-face interviews. As with school personnel interviews, interviews were audio-recorded but not transcribed.

All together we conducted interviews with 64 people (see Table 4). We had a very high response rate of almost 100 percent, including principals.


Table 10 Interviewees August–October 2014 for Cohort 14 Interviewee Number interviewed Percentage

Participant 19 100

Principal 8 (seven schools new to the programme in 2014 plus one new principal in a school with cohort 13 and 14 participants; one interview was with a deputy principal

89 (plus one deputy

principal

Co-ordinator 10 (two co-ordinators were also mentors and their interview were recorded there; one co-ordinator was the principal and the interview was recorded there)

100

Mentor 19 (one participant had two mentors; one mentor was not

able to be interviewed)

95

Teach First NZ partnership personnel

8 100

Total 64

Me and My Class Survey administration Me and My Class is a survey22 designed by NZCER for Years 4 to 13 students. It explores students’ perspectives on learning in their classroom. The Me and My Class survey has two sections:

• Me and My Class: This section has 24 questions that explore students’ views on the extent to which learning opportunities in their class reflect effective teaching pedagogy.

• Me in My Class: This section contains 10 questions that explore students’ personal responses to learning in their class.

The survey was administered by NZCER’s Products and Services Assessment Team in September 2014, 8 months after Cohort 14 participants began teaching and near the end of Cohort 13’s 2-year programme. All teachers involved were sent information about the survey, instructions for its administration, and intended or potential uses of the survey data, as well as consent forms for their participation. For the evaluation, participants chose a Year 9 or 10 class, and schools were asked to choose a “comparison” class where the students were in the same year level and preferably in the same subject area and taught by another teacher (normally who was fully registered and often significantly more experienced). Year 9 and 10 classes were chosen because all participants had at least one Year 9 or 10 class.

We obtained results from the Me and My Class survey for 2923 out of 34 participants (14 schools) for a Year 9 or 10 class taught by the participant and for 29 comparison classes.

22 The survey is attached as Appendix 8. 23 Some participants only had results for senior classes and we did not include these in our analysis.


Analysis of data

Documentation The programme documentation helped us more fully understand the intentions of the pilot programme and how the different components fitted together over the 2 years.

The observation and feedback documents supplied by participants and, in some instances, by mentors or co-ordinators, were intended to provide an overall sense of the approach taken by mentors and LASs and VTSs to supporting the participant through regular and ongoing observations of the same classes. We did not analyse these documents in relation to whether the participant was making the desired progress, or whether each mentor and LAS / VTS was following a similar approach. Our aim was to ascertain the extent to which the frequent observations (once a week suggested for mentors and up to 15 times a year for VTSs) were providing the support the participants or their mentors told us they needed. We wanted to know how participants were using the online portfolio to track progress towards meeting the GTS and RTC. Some participants also shared their lesson planning, assessments and assessment results, and student work with us. Again, these documents helped us get a sense of participants’ emerging understanding of teaching, but we did not evaluate the quality of their work.

Cohort 13 surveys NZCER statisticians managed the online surveys for the 2013 cohort. A survey link was emailed to relevant principals, co-ordinators, mentors, participants, and LASs on 3 October, and the survey remained live until 24 October 2014. We received 46 survey responses—some mentors completed more than one survey and some mentors are also coordinators. In all, 40 of the 46 people involved completed the surveys. Two principals, two mentors and two participants did not complete the surveys, which had a response rate of 77 percent. While we timed the surveys to coincide with term break and term time, it is possible those who did not complete the survey were not able to make the time to do so. Because of the small sample size, survey data was analysed according to frequencies and we did not do further (for example, cross-tab) analysis. Each respondent group answered many of the same questions, and we constructed tables that showed these comparative responses.

Cohort 14 interviews Interviews were recorded but not transcribed. The interview schedules included a wide range of questions to elicit people's views and experiences of the Teach First NZ programme. While the interview schedule was clearly structured, the interviews themselves often took on a conversational feel and, as such, various themes were often referred to across different questions. Responses relevant to each theme, regardless of the question, were independently coded and cross-checked by two researchers and collated in a spreadsheet, by role. This process made it possible to gain a reliable thematic overview of participants’ responses across the conversational style of interview and allowed for greater breadth of discussion to be included in the analysis.

Interviews with Teach First NZ and University of Auckland Faculty Interviews with the five Teach First NZ staff and three University of Auckland staff were summarised and analysed thematically. The focus of these interviews was on changes made between 2013 and 2014 and any particular challenges that the team had met and envisaged for the future. Some of these data were directly comparable to the data collected through the Cohort 13 surveys and Cohort 14 interviews.


Me and My Class survey The survey is fairly new and we do not have national data for a large number of students as yet. However, it has been rigorously piloted and tested for psychometric validity and reliability. The survey explores students’ perspectives on learning in their classroom. We chose to use the survey because of its validity and reliability within a New Zealand context to help us assess one of the evaluation criteria for effectiveness of participants’ teaching: High engagement of students, especially Māori and Pasifika.

We obtained results from the Me and My Class survey for 29 out of 34 participants (14 schools) for a Year 9 or 10 class taught by the participant and for 29 comparison classes. For the purposes of this evaluation, the aggregated and anonymised results were used to assess the student perceptions of participant teachers’ effectiveness in providing pedagogically rich learning opportunities and in engaging students in their learning. We would strongly caution against too much emphasis being placed on the results of the survey in 2014. Factors to consider are that we do not have data for all participants, we have data for more than twice as many participants’ students as comparison students, and that this was the first time many students had attempted this survey and they may have been exposed to different instructions and conditions.

As we described more fully in the section on the survey administration, while the survey is referred to as Me and My Class it has two parts: Me and My Class and Me in My Class.

For Year 9 we have the aggregated and anonymised results for 320 ‘participant class’ learners and 193 ‘comparison class’ learners. For Year 10 we have the aggregated and anonymised results for 318 participant class learners and 140 comparison class learners. We also separated out Cohort 13 and 14 data. For Year 9 we have the aggregated and anonymised results for only 49 students taught by Cohort 13 participants and 192 students taught by Cohort 14 participants. For Year 10 we have the aggregated and anonymised results for 128 students taught by Cohort 13 participants and 167 students taught by Cohort 14 participants. These numbers are also summarised in Table 11.

Table 11 Me and My Class Survey data Class surveyed Number of students surveyed?

Cohort 13 & 14 Year 9 32024


Cohort 13 Year 9 49


Cohort 13 & 14 Year 10 31825







Appendix 3: Selected Me and My Class data Students in Cohort 13 classes26 were more likely than students in Cohort 14 classes to strongly agree with the following items from Me and My Class:

Table 12 Larger differences between Cohort 13 and 14 Year 9 classes Year 9 Cohort 13 percentages Cohort 14 percentages We think about how our new learning changes what we already know

33 23

We learn good ways to say what we are thinking in different situations

31 17

We talk about how we could use our learning in new situations

35 25

I have surprised myself with what I can achieve

41 31

I always want to go to class 35 24

The Year 10 data tells a similar story although both sets of classes are slightly more likely than Year 9 classes to choose “strongly agree” as their response.

Students in Cohort 13 classes were more likely than students in Cohort 14 classes to strongly agree with the following items from Me and My Class:

Table 13 Larger differences between Cohort 13 and 14 Year 10 classes Item Cohort 13 percentages Cohort 14 percentages

My teacher likes it when I ask questions and try out new things

43 23

We help each other to think about how to make our work better

37 23

We use ideas and skills from different parts of our learning to solve problems

35 19

We learn ways to work out our problems or challenges together

36 24

We talk about how we could use our learning in new situations

36 26

I always try my hardest 40 29

I have surprised myself with what I can achieve

45 33

My culture is respected and valued 44 32

26 Cohort 13 teachers have been teaching for a year longer than Cohort 14 teachers.


Appendix 4: Ministry indicative evaluation questions Table 14 Ministry of Education indicative evaluation questions

Key evaluation questions

1. How well (effectively and efficiently) has the Teach First NZ pathway been implemented?

Formative/process focus: short-term and intermediate outcomes

2. To what extent has the Teach First NZ programme achieved its overall outcomes and objectives?

Summative focus: long-term outcomes and objectives of programme

Indicative subquestions

• To what extent does the programme attract and recruit high-quality applicants?

• How effective are the Teach First NZ recruitment and selection processes?

• To what extent does the programme recruit participants who would otherwise be unlikely to enter teaching?

• Do participants finish the programme? What are retention rates during the 2-year programme?

• How well does the initial summer intensive prepare participants for working in schools?

• How relevant/useful is the content and format of the overall 2-year programme, including the initial intensive and the ongoing mini intensives and initial teacher education/professional development?

• To what extent are teaching and coursework loads appropriately challenging for participants? Is it a good balance?

• How well does the Teach First NZ Trust support participants?

• How well implemented is the host school recruitment and selection process?

• How well implemented is the affiliate school recruitment and selection process?

• To what extent to do host schools feel well informed and supported by the partnership?

• To what extent are appropriate mentors/university visiting specialists chosen?

• How relevant/useful is the training and support that mentors/university visiting specialists receive?

• How well do mentors/university visiting specialists support participants?

• To what extent does the host school environment sustain/support the effective operation of the programme?

• To what extent do the affiliate school arrangements support the overall goals of the programme?

• To what extent does the programme deliver/produce effective teacher graduates?

• To what degree do participants support pastoral life within their host schools?

• To what extent does the leadership development strand support, build on, and extend the concept of effective classroom teaching as leadership of learning?

• To what extent do alumni of the programme stay in teaching or remain in wider education after programme completion (retention and post-training recruitment outcomes), in comparison with retention data from established programmes?

• What is the programme’s contribution to improving the quality of teaching and learning outcomes in participating schools?

• To what extent do the features of the programme have a positive impact on the quality of teaching and learning for host schools and students?

• What is the value of participating in the programme for host schools?

• To what extent is the programme contributing to improving the status of teaching?

• What factors appear to be associated with successful and disappointing outcomes (e.g., enablers, barriers)?

• How engaged are graduates of the programme in Teach First NZ's alumni programme?


Key evaluation questions • How well and efficiently are implementation

issues resolved?

• How flexible has the programme been to adjusting to lessons learnt?

• Is the mix of programme elements optimal?

• What aspects of the programme could be refined to improve the overall quality of implementation?

• How sustainable27 is the implementation model, including the recruitment process?

27 Sustainability refers to whether the implementation model can successfully adjust to larger volumes of recruits, and implications for funding and resourcing. It also includes host schools’ ability to support the programme when they self-fund programme participants’ salaries from 2014.


Appendix 5: Cohort 13 survey questions
































Appendix 6: Cohort 14 interview questions

Participants: interview questions phase 2 (Cohort 14): 1.0 Recruitment

1.1 What was your degree and what teaching subject areas did you study in your TFNZ programme? 1.1.1 Degree 1.1.2 Teaching subjects in Post Grad Diploma of Teaching

1.2 How did you hear about Teach First (NZ)? 1.2.1 Print advertisement 1.2.2 Other media (specify) 1.2.3 Seminar/presentation by Teach First (NZ) 1.2.4 Word of mouth (specify) 1.2.5 Other (specify)

1.3 And why did you apply? 1.3.1 Opportunity to learn on the job (get out there quickly) 1.3.2 The “mission” of Teach First NZ (low decile, disparities in Maori/Pasifika

achievement, opportunity to ‘make a difference’ 1.3.3 The financial incentive (being paid a salary/no fees) 1.3.4 Good reputation (compared to the traditional pathways to becoming a teacher) 1.3.5 Liked being part of a new approach to becoming a teacher (specify) 1.3.6 Leadership development opportunities? 1.3.7 Postgrad level study opportunities?

1.4 Would you have applied to other teacher training at this point in time were it not for Teach First NZ? 1.4.1 Yes 1.4.2 No

1.5 What would you have done if you had not been accepted for this programme? 1.5.1 Applied for the traditional pathway to becoming a teacher 1.5.2 Further university study 1.5.3 Travel 1.5.4 Another job 1.5.5 Other (specify)

1.6 What was the reaction of your family/friends to your gaining a place on this programme? 1.6.1 Positive

1.6.1.1 Why? 1.6.2 Negative

1.6.2.1 Why? 1.7 Do you feel differently about the status of teaching and teachers than you did before you

joined the programme? 1.8 Did you think the selection process enabled you to demonstrate your strengths and areas

needing development, in terms of the Teach First NZ selection criteria?


2.0 Programme factors 2.1 Summer intensive

2.1.1 What aspects of the SII were most useful/helpful? 2.1.2 Now that you have been teaching for a while, what improvements to the summer

intensive would you recommend for the next intake? 2.1.3 How well do you think the SII prepared you to begin teaching? Are there any areas

where you needed more preparation? (What are they?) 2.2 The ongoing course-work

2.2.1 Which aspects of the taught programme do you find most useful? 2.2.1.1 Do you have any comments to make about the way the programme is

organised, presented and assessed? 2.2.1.2 What are the strengths and weaknesses of the programme?

(possible probes: responsiveness to feedback, vision, core aspects, teaching and learning approaches, amount of challenge, relevance of assignments, timing)

2.2.1.3 How are you keeping track of your development and teaching? (How do you organise documenting progress and to what end? GTS/RTC/Teach First NZ goals and aims)

2.3 To what extent do you feel part of the Teach First community? 2.3.1 How important is this connection to you? (And why?)

3.0 Your host school 3.1 How do you think the school community responded to you and the other Teach First NZ

participant? 3.2 What initial orientation (in the first days or weeks) to the school did the school provide for

you? 3.3 What classes are you teaching? (subjects and levels). 4.0 Teaching and involvement 4.1 What were the main challenges for you in your first term of teaching?

4.1.1 How have you been able to address these challenges? (who/what has helped you?) 4.2 How do you feel about how well you are functioning as a teacher at this stage in your

development? How do you think the students are doing? 4.3 What are the most useful aspects of your mentor’s support? 4.4 Have you been involved in any professional learning in the school with other teachers? If so

what is it, and has it had any impact on your practice? 4.5 Are you involved in other aspects of school life outside classroom teaching? If “yes” how are

you finding this involvement? 4.6 What involvement have you had with the LAS and the VTS. To what extent is this helping you

to develop your expertise as a teacher? 5.0 General 5.1 What is your overall perception of the Teach First NZ pilot programme so far? 5.2 Do you expect to complete the 2-year programme? 5.3 Do you expect to stay in teaching after the 2 years?


5.3.1 If YES, for how long? 5.3.2 If NO what are the reasons for leaving? 5.4 Is there anything else you would like to tell me about your experiences so far?

Host school mentors: interview questions phase 2 (cohort 14) 1.0 Background

1.1 What was your initial reaction to Teach First NZ approach to preparing teachers? 1.2 Have these views changed over time? How/why? 1.3 How did you get to be a mentor for this programme? 1.4 What did you see yourself gaining from being a mentor? (Are you able to use/adapt mentoring

approaches in other things that you do?)

2.0 Participants’ placement and support 2.1 Are you and the participant you mentor in the same subject area? How is this working for you? 2.2 How effective was the liaison between the programme personnel and you in terms of your

respective roles and expectations? 2.3 What is your view of Teach First NZ’s approach to mentoring of participants? Are you finding

that you are able to use the learning you have had from Teach First NZ in your work with your participant?

2.4 What has been the reaction of other teachers to the participants as individuals? As teachers? 2.5 How supportive is the school of your role as mentor?

3.0 Participant teaching and involvement 3.1 What do you think is the main purpose of your work with the participant? 3.2 Have you been able to attend the three professional training days and meetings? 3.3 How often have you been able to formally observe the participant teaching in the classroom? 3.4 What is your approach to doing observations? How does it work in practice? How do you

approach feedback? 3.5 Have you been able to model your teaching approach to your participant? How often? How

does this work in practice? Has the participant been able to observe other teachers teaching? 3.6 How have students responded to the participant/s so far? 3.7 Have the participant/s had a noticeable impact in any way that you’re aware of? (Prompt:

student engagement & achievement; other teachers in the school; activities outside the classroom)

3.8 Have you recently mentored a pre-service teacher? A provisionally registered teacher? If YES ask the following. If NO – omit relevant question. At this point, how do they rate in terms of the things outlined above against:

3.8.1 Student teachers on practicum? 3.8.2 Other PRTS in their first year? 3.8.3 What evidence do you have that Teach First NZ participants are having an impact on

student/s approaches to learning and their achievement? 3.9 From what you have seen so far, do you think it is likely that the Teach First NZ model will

produce quality teachers at the end of 2 years? Why/not?


4.0 General 4.1 What is your overall perception (so far) of the Teach FirstNZ initiative? 4.2 What do you see the school as having gained from participation in the programme? 4.3 Have there been any barriers to effective participation in the programme? 4.4 Is there anything else you can tell me about your experience to date?

Host School coordinator: Interview questions phase 2 (cohort 14) 1.0 School involvement

1.1 Why did your school become involved in Teach First NZ approach to teacher preparation? 1.2 What were the initial reactions of your staff to the Teach First NZ programme? 1.3 Have these views changed over time? How/why? 1.4 What appear to be the particular strengths of this way of preparing teachers? 1.5 Can you describe your role as coordinator? Did you have any training or preparation for this

role? How well prepared do you think you were to support the participant/s?

2.0 Participant placement 2.1 How many of your teachers were involved in the mentor training provided by Teach First NZ?

2.2.1 Are you aware of their reactions to this training? 2.2.2 How did you select mentors for the participants?

2.2 How effective was the liaison between the programme and the school in terms of your respective roles and expectations?

2.3 What has been/is the reaction of your staff to the participants in your school? How has your PPTA branch responded?

3.0 Participant teaching and involvement 3.1 Do you have any information about how well the participants are teaching? (Probe for sources

of this information. Has the coordinator observed any participants teaching?) 3.2 How else have the participants been involved beyond the classroom? 3.3 Have the participants had a noticeable impact in any way that you’re aware of? ) Prompts:

student engagement and achievement; other teachers in the school; the school culture; activities outside the classroom)

3.4 How have students responded to the participants so far? 3.5 Have you recently mentored a pre-service teacher? A provisionally registered teacher? If YES

ask the following. If NO – omit relevant question. At this point, how do they rate in terms of the things listed in 3.3 against

3.5.1 Student teachers on practicum? 3.5.2 Other PRTS in their first year?

3.6 From what you have seen so far, do you think it is likely that the Teach First NZ model will produce quality teachers at the end of 2 years? Why/not?

3.7 In what ways (if any) are participants different from other beginning teachers?

4.0 General 4.1 What is your overall perception (so far) of the Teach First NZ initiative?


4.2 What do you see the school as having gained from participation in the programme? 4.3 Have there been any barriers to effective participation in the programme? 4.5 Is there anything else you can tell me about your experience to date?

Host School principal: Interview questions phase 2 (cohort 14 only schools) 1.0 School involvement 1.1 Why did you choose to involve your school in Teach First NZ approach to teacher preparation? 1.2 What were the initial reactions of your staff to the Teach First NZ programme? 1.3 Have these views changed over time? How/why? 1.4 What appear to be the particular strengths of this way of preparing teachers?

2.0 Participant placement 2.1 How effective was the liaison between the programme and the school in terms of your

respective roles and expectations? 2.2 How many of your teachers were involved in the mentor training provided by Teach First NZ?

2.2.1 Are you aware of their reactions to this training? 2.2.2 How did you select mentors for the participants?

2.3 What has been/is the reaction of your staff to the participants in your school? How has your PPTA branch responded?

3.0 Participant teaching and involvement 3.1 Do you have any information about how well the participants are teaching? (Probe for sources

of this information. Has the principal observed any participants teaching?) 3.2 How else have the participants been involved beyond the classroom? 3.3 Have the participants had a noticeable impact in any way that you’re aware of? (Prompts:

student engagement and achievement; other teachers in the school; the school culture; activities outside the classroom)

3.4 How have students responded to the participants so far? 3.5 At this point, how do they rate in terms of the things listed in 3.3 against

3.5.1 Student teachers on practicum? 3.5.2 Other PRTS in their first year?

3.6 From what you have seen so far, do you think it is likely that the Teach First NZ model will produce quality teachers at the end of 2 years? Why/not?

3.7 In what ways (if any) are participants different from other beginning teachers?

4.0 General 4.1 What is your overall perception (so far) of the TFNZ/ initiative? 4.2 What do you see the school as having gained from participation in the programme? 4.3 Have there been any barriers to effective participation in the programme? 4.4 From what you have experienced so far, do you intend to continue a relationship with the

Teach First NZ programme in the future? 4.5 Is there anything else you can tell me about your experience to date?


Appendix 7: Interview Schedule—Partners

Teach First NZ partners interview questions phase 2 1.1 What is your overall perception (so far) of the Teach First NZ programme in 2014? Is this different

from 2013? If yes, in what ways? 1.2 What are the main changes you have made to the programme between 2013 and 2014? 1.3 What do you see the schools as having gained from participation in the programme? Is it different for

2013 and 2014 schools? 1.4 What do you see the participants as having gained from participation in the programme? Is it

different for 2013 and 2014 participants? 1.5 What are the strengths of the Teach First NZ approach? Have you changed your perceptions of these

strengths over time? 1.6 In what ways might the programme be strengthened? 1.7 Are there things you will change (or would like to change) for 2015? 1.8 What have been the main challenges for you in 2014? 1.9 What have the highlights been for you in 2014? 1.10 Is there anything else you want to tell me about your experience to date?


Appendix 8: Me and My Class Survey



Documents

2014 annual evaluation report for the Teach First NZ ... second annual evaluation report of the Teach First NZ programme pilot delivered in partnership with the ... Appendix 6: Cohort