Richard Thompson Associates - britishcouncil.org.np · Richard Thompson Associates 16 Conway Road London N14 7BA BRITISH COUNCIL NEPAL ENGLISH FOR TEACHING: TEACHING FOR ENGLISH PLUS

1 ET:TE+ Monitoring & Evaluation Report Nepal 2013_14

Richard Thompson Associates 16 Conway Road London N14 7BA

BRITISH COUNCIL NEPAL ENGLISH FOR TEACHING: TEACHING FOR ENGLISH PLUS

ET:TE+ TRAINING PROJECT IN CHITWAN AND LAMJUNG DISTRICTS

MONITORING & EVALUATION REPORT

January 2014

Contents

Executive summary

1. Introduction: the English for Teaching:Teaching for English Plus (ET:TE+) Project 2013

2. The Monitoring & Evaluation programme

3. Findings: Classroom observation of teachers

4. Findings: APTIS testing of teachers’ levels of competence in English

5. Findings: Interviewing teachers of English

6. Findings: Other views expressed in schools

7. Conclusions

Appendix: Sample for Monitoring & Evaluation Programme

Executive Summary

The principal purpose of the ET:TE+ training project was to change teacher behaviour in the

classroom: more personal competence and confidence for the teacher in their English and

changes to their pedagogy to facilitate interactive language learning.

Large-sample, standardised monitoring was put in place to evaluate the project at baseline,

mid project and end of project stages. Extensive classroom observation was the key element

in recording the ET:TE+ project’s impact and success: the key indicator was whether in the

classroom teacher behaviour had changed: transfer of training to the classroom in terms of

the three main areas of classroom observation: use of English, interaction and effectiveness

of teaching. The evidence is that there had been very significant change.

There have been dramatic increases in the use of English by teachers in the classroom: a

strong shift from the baseline position through to project end, as this graphic shows:


Baseline monitoring showed 83% of lessons with no more than 10% use of English and just

1% in the top two levels. By end of programme, the 83% has shrunk to 6% and the 1% has

grown to 81%.

There was a growth in student use of English in talking to the teacher and student to student

talk in English, which would most naturally occur in group or paired work situations. Such

interactive situations were almost non-existent at the start of the project; by its end close to

80% of lessons included group work. Students had gained the opportunity to use and practice

English themselves in the classroom, and when asked in focus groups confirmed that these

were daily opportunities.

There had also been very noticeable changes in measures of teacher effectiveness to produce

a more positive classroom atmosphere and learning experience: changes to pedagogy, much

more use of supplementary classroom materials, a focus to lessons with a much greater

variety of student activities.

These classroom findings constitute a powerful vindication of the ET:TE+ training project. The

trajectory strongly suggests that these will be permanent changes, a view reinforced by

qualitative data from interviews with teachers, parents and school managers.

Disaggregating the data by district shows some differences, which may come from the way

the training was delivered as well as from other district-specific factors: there was more

extensive use of classroom English by the teachers in Chitwan where lessons scored higher

for the use of pair or group work. By contrast, Lamjung teachers scored high on giving

encouragement and feedback to students.

Disaggregating by gender, from similar starting scores female teachers made greater

improvement in a range of classroom activities and pedagogical methods, including student

use of English. There was little gender difference in terms of teacher use of English.

All teachers were tested at the beginning and end of the ET:TE+ training with the

internationally-developed and standardized British Council APTIS test, an element which was

widely welcomed. Analysis showed very many teachers with low levels of English with only a

handful reaching Independent User standard. The re-test showed some gains but most

notably a great deal of churn in individual results.

Teachers in interview described their growing confidence with English and the new teaching

methods, with an urgent need to increase classroom resources in order to move to more

interactive teaching. There was great enthusiasm for the ET:TE+ training from teachers, SMC

members and district office staff.

28

55

16

1 0 4 2 13

29

52

0% 1-10% 11-25% 26-50% 51%+

Teacher Used English during the Lesson

Baseline END P


1. Introduction: the English for Teaching: Teaching for English Plus (ET:TE+)

Project 2013

English for Teaching: Teaching for English Plus (ET:TE+) 2013 was a British Council teacher

training project to support the development of communicative English language teaching in

government Primary schools in Nepal in the two districts of Chitwan and Lamjung, in

association with Nepal English Language Teachers Association (NELTA) in Chitwan and Global

Action Nepal (GAN) in Lamjung as implementing partners.

NELTA and GAN were encouraged to develop their own chosen routes to deliver the common

ET:TE+ training project. Both chose a methodology based on their past experience described

as a “holistic approach”, meaning the selection of the English teacher plus a number of

teachers from each school to better create a critical mass which could change practice across

a school more easily. These teachers may teach a range of subjects from the National

Curriculum. NELTA chose to recruit teachers from across the district and all Resource Centre

[RC] areas while GAN in Lamjung chose to run in a group of three RC areas and clusters of

schools.

Potential teacher recruits to the ET:TE+ training project were nominated by the implementing

partners and underwent an initial screening and selection process in March 2013 using the

British Council’s APTIS test. The test was conducted at central locations in each district,

supervised and administered by British Council Nepal and managed by this team. The APTIS

test has been designed and standardized to be consistent with the Common European

Framework of Reference for Languages [CEFR]. CEFR describes foreign language proficiency

at six levels: A1 and A2 (Basic User), B1 and B2 (Independent User), C1 and C2 (Proficient

User). The international ET:TE course was designed for a minimum of A2 competence levels;

for this project the course was redesigned locally to also be accessible for A1 language users.

The 300 teachers who would participate, 150 in each district, were selected by the end of

March 2013 and the project began. British Council Nepal authored a Training Manual and led

training for the trainers. In each district the teachers then received initial 6 day training in

April 2013 before returning to their schools for the start of the new school year.

143 teachers in Lamjung and 149 teachers in Chitwan actually attended the initial training

and began the ET:TE+ project. In Chitwan this initial training was on a residential basis. GAN

and NELTA then planned follow up sessions. For example in Chitwan NELTA organized two

sessions at 15 day intervals, then trainers visited as many trainees as they could in their

classrooms to provide feedback on classroom practice. Other follow up activities included

teacher’s clubs. GAN made use of local Resource Centres for their sessions. A second 6 day

training course in August 2013 used a second Training Manual and training in its use provided

by British Council Nepal. The second week was non-residential in both districts. Follow up

activities included school and inter-school competitions.

The APTIS testing process was repeated at the end of the project in November/December

2013.


2. The Monitoring & Evaluation programme

Preparation

Initial preparation and planning by Technical Advisor and Team Leader was shared as

appropriate with British Council Nepal and the ET:TE+ implementing partners. Qualified and

experienced Research Associates (RAs) took part in a five day intensive training session in

Kathmandu in January 2013. This included training by British Council Nepal to standardize

classroom observation recording. The data collection tools were piloted in two schools in

Kavre district and subsequently revised. Training sessions were repeated at the Mid Project

and End of Project stages.

Research Associates were allocated to two district-based teams to conduct school visits. Steps

were taken to ensure consistent, coherent and reliable data were collected from the field by

and across the two district-based teams.

Baseline assessment

A report on the baseline survey carried out by this team was completed for British Council

Nepal in March 2013. This Report draws on the baseline data for comparative purposes.

Monitoring & Evaluation - method

A representative sample of teachers and schools was compiled in each district, using data

collected by:

the baseline survey: teachers and schools from whom a full data set of teacher interview,

classroom observations and focus group outcomes had been collected;

British Council Nepal: APTIS test scores; and

the implementing partners: teachers actually attending training in April 2013 [a small

number were absent, and therefore deleted from the sample]

Details of the sampling are given as an appendix. Analysis of the sample against the full

ET:TE+ cohort demonstrates that the chosen sample was representative.

In Chitwan District the sample consisted of 30 schools each with an identified teacher of

English and 56 others. In Lamjung District the sample covered 24 schools with 24 identified

teachers of English [one per school] to be interviewed, with a higher number – 93 – of other

ET:TE+ teachers available for classroom observation.

Mid Programme and End of Programme monitoring

Central to the Monitoring and Evaluation Programme was unannounced school visits. The

Programme was structured around two series of monitoring visits. The first round of mid

programme monitoring visits was conducted in each district in June/July 2013. The second

round of end of programme monitoring visits to the same sample of schools and teachers

took place in November/December 2013.

At the core of each school monitoring visit was extensive classroom observation. In each

school the identified teacher of English was subject to extended classroom observation [4 or 5

lessons]. All other ET:TE+ teachers in the sample schools were observed in at least one

lesson. The classroom observations were recorded on a proforma devised by British Council

Nepal which concerned 19 measures related to three aspects of the ET:TE+ training project:

Use of English

5 measures of use of English in the classroom, by the teacher including giving instructions

and feedback, and by students speaking to the teacher and to each other in English. These

were each assessed at one of five levels from ‘0%’, completely absent, to ‘51%+’, use of

English on more than half the available opportunities.


Interaction

6 measures of the use of pair or group work, teacher encouragement of and feedback to

students, student involvement in the lesson and the balance between teacher-centred and

student-centred activities. These were each assessed at one of five levels again, this time on

a judgement scale from ’Not met’ to ‘Outstanding’

Effectiveness of teaching

There were 8 measures in the final aspect centred on teachers’ pedagogical skills, techniques

and behavior such as use of materials to supplement the lesson, correction of students and

checking students understood, again recorded on a judgment scale from ’Not met’ to

‘Outstanding’

The proforma included space for the fieldworker to record any additional comments or notes.

The monitoring team underwent standardization training from the British Council. The Team

Leader and Technical Advisor also set up a standardization process of random visits with

Research Associates for shared classroom observation to ensure reliable data recording. This

process was carried out rigorously and led to one indicator being dropped as there was still

some variety of practice within the team that was generating unreliable results. This indicator

was ‘there was an effective balance between teacher-centred and student-centred activities’.

During fieldwork, views were also sought at each school through:

Individual self-evaluation interviews with each identified teacher of English

A focus group of pupils, discussing the English lesson just observed and their

experience of English

A focus group of adult stakeholders, normally including English teacher(s),

headteacher, SMC chair, parents and other local stakeholders

Research Associates also provided short written reports on their work.

Competence in English: APTIS testing

This team managed the delivery of the APTIS testing, in March/April and in

November/December 2013. Findings are discussed below.

Data handling

The data collected by the field researchers was computerized using Microsoft Excel. Before

starting data entry, inconsistencies and errors were corrected and data were checked and

ensured in consultation with the Research Associates. The data were entered by two

experienced data entry personnel. The entered data were cleaned using the Statistical

Package for Social Sciences (SPSS 18). The data entered in MS Excel was converted into

SPSS data files. There were four separate data files in SPSS namely Classroom Observation,

Student Focus Group Discussion, Teacher Interview, and Focus Group Discussion.

Ethics

This study used the ethical guidelines of the British Educational Research Association (BERA).

Unambiguous and clearly written information in the Nepali language was provided to ensure

informed consent from all participants, with the right at any time to withdraw that consent.

All participants were provided with an explanatory sheet and their verbal consent was

recorded in field notes. Participants were assured that data would be anonymised and

confidentiality ensured. Any paper-data was kept safely and after entering data on computers

it was kept securely to prevent unauthorised access. No incentive was offered to participants

and no offers were made to meet any of their expenses.


3. Findings: Classroom observation of teachers

The findings from classroom observation are the key element in considering the impact and

success of the ET:TE+ training programme. The principal purpose of the training was to

change teacher behaviour in the classroom: more personal competence and confidence for

the teacher in their English and changes to their pedagogy to facilitate interactive language

learning.

Baseline

All the teachers who were initially considered for ET:TE+ training and who took the APTIS test

of English competence were visited at their schools and observed teaching in the classroom.

The total number of Classroom Observations recorded as part of the baseline assessment was

916. Tables in this report include the data from all these Classroom Observations whether or

not teachers went on to take part in the ET:TE+ training.

Mid Project and End of Project monitoring

There were two rounds of monitoring. All the teachers in the sample were visited at their

schools in a first round of monitoring [MID P] in June/July 2013 and observed teaching in the

classroom, a total of 570 complete lesson observations. 314 observations were completed in

Chitwan and 256 in Lamjung. Each teacher of English was observed on several occasions; the

total number of observations in this subset was 348 which was 61% of the total classroom

observations. Other teachers were observed [222 lessons] teaching other subjects through

the medium of English with some English lessons. 94% of observations were Grade 1-5

lessons.

Sample schools and teachers were visited again in the second and final round of monitoring

[END P] in November/December 2013 and observed teaching in the classroom, a total of 552

complete lesson observations. 299 observations were completed in Chitwan and 253 in

Lamjung with a similar balance between teachers. Tables in this section compare Baseline,

Mid Project [MID P] and End of Project [END P] data across the 19 measures and five levels

agreed and standardized in advance.

Classroom observations: all teachers

Table 3.1 Classroom Observations: Use of English: all teachers

0% 1-10% 11-25% 26-50% 51%+

Use of English: all teachers, as %

The teacher used English during the lesson

Baseline 28 55 16 1 0

MID P 9 30 55 6 0

END P 4 2 13 29 52

The teacher gave instructions in English

Baseline 58 40 3 0 0

MID P 12 58 27 2 0

END P 4 4 14 32 45

The teacher gave feedback in English


MID P 18 65 16 1 0

END P 5 5 21 34 35

The students spoke to the teacher in English


MID P 53 45 2 0 0

END P 12 36 39 10 3

The students spoke to each other

in English

Baseline 99 1 0 0 0

MID P 91 9 0 0 0

END P 54 31 9 4 2


This first table shows in the first three measures dramatic increases in the use of English by

teachers in the classroom. At baseline 83% of lessons were at the lowest two levels; at end of

project 81% where in the top two levels. There is a strong shift by mid project and this

change continues through to project end, by which point just a small minority of lessons show

little use of English.

The greatest change is with teacher use of English. However there is a growth too in student

use of English in talking to the teacher and in student to student talk in English from a very

low base. By the end of the project there has been a positive and significant change.

Table 3.2 Classroom Observations: Interaction: all teachers

Not

met Partially met

Met More than met

Outstanding

Interaction: all teachers as %

The teacher used pair work effectively


MID P 77 23 1 0 0

END P 40 16 22 19 3

The teacher used group work effectively

Baseline 99 1 0 0 0

MID P 63 30 7 0 0

END P 22 17 33 25 3

The teacher gave encouragement to pupils


MID P 1 63 36 0 0

END P 0 5 54 34 7

The students were involved in the lesson


MID P 0 65 35 0 0

END P 0 6 60 27 7

The teacher gave effective feedback as

necessary


MID P 1 74 25 0 0

END P 0 8 54 25 13

There has been an increase in the use of pair and group work, breaking the mold of 100%

and 99% of baseline lessons with no sign of these. At end of project almost 80% of lessons

included some group work and 60% some element of pairs work. We should note here that

the primary-age students who formed our pupil focus groups confirmed that the English

lesson just observed by our researcher was typical of the lessons they now received.

There have been very noticeable changes to produce a more positive classroom atmosphere

and learning experience, with very large gains in the other three measures to reach 90%

‘met’ or better in these measures of classroom interaction.

Table 3.3 Classroom Observations: Effectiveness of teaching: all teachers

Not

met Partially met

Met More than met

Outstanding

Effectiveness of teaching: all

teachers as %

The teacher used a variety of activities


MID P 4 79 17 0 0

END P 2 24 39 28 6

The teacher gave students time to respond


MID P 1 64 35 1 0

END P 0 7 51 33 9


The teacher used gestures to support

meaning


MID P 16 64 19 0 0

END P 1 12 43 42 2

The teacher used materials to supplement the lesson


MID P 25 48 26 0 0

END P 1 11 44 41 3

The teacher corrected students


MID P 1 72 26 1 0

END P 0 8 52 30 10

The teacher checked students understood


MID P 1 73 25 1 0

END P 0 5 52 33 10

The lesson effectively developed a skill Baseline 70 28 1 0 0

MID P 7 85 8 0 0

END P 0 13 62 23 2

The lesson effectively covered a language area


MID P 8 87 5 0 0

END P 1 11 64 22 3

These are measures of pedagogy. Practice has improved across the board: it is striking how

improvement continues from mid project to end of project monitoring, for example from

‘Partially met’ to ‘met’ or better. No. of observations: Baseline n = 916 Mid project n = 570 End of project n = 552

Classroom observations: comparing teachers of English with other teachers

This subsection compares designated teachers of English [all English teachers at baseline; at

MID P and END P, the 54 teachers one in each of the sample schools] with all other teachers

on selected measures that show noticeable differences between the two groups. Other

measures show less difference: the full series of tables are available in the Technical Report.

Table 3.4 Classroom Observations: Use of English: teachers of English

0% 1-10% 11-25% 26-50% 51%+

Use of English: teachers of English, as %


Baseline 13 64 23 0 0

MID P 0 21 70 9 0

END P 0 0 11 33 56



MID P 1 59 36 3 0

END P 0 1 13 40 46



MID P 7 72 20 1 0

END P 0 2 23 40 34


Table 3.5 Classroom Observations: Use of English: other teachers

0% 1-10% 11-25% 26-50% 51%+

Use of English: other teachers as %


Baseline 42 47 10 0 0

MID P 22 44 32 1 0

END P 8 4 14 25 48



MID P 30 56 13 0 0

END P 9 6 15 25 45



MID P 34 54 11 1 0

END P 9 8 18 27 37

Within a general picture of very much greater use of English the teachers of English are using

the language more in the classroom than their colleagues in the same schools, having begun

from a stronger baseline position. For example on the first indicator here ‘The teacher used

English during the lesson’, at the end of the programme teachers of English recorded 89% in

the top two levels, with none in the bottom two levels, 0 – 10%. ‘Other teachers’ scored

lower at 73% in the top two levels with 12% still using little or no English.

Table 3.6 Classroom Observations: Interaction: teachers of English

Not

met Partially met

Met More than met

Outstanding

Interaction: teachers of English as %


Baseline 99 1 0 0 0

MID P 73 26 1 0 0

END P 34 16 26 23 1


Baseline 99 1 0 0 0

MID P 55 36 9 0 0

END P 18 14 40 27 1

The teacher gave effective feedback as necessary


MID P 1 80 19 0 0

END P 0 9 58 23 9

Table 3.7 Classroom Observations: Interaction: other teachers

Not

met Partially met

Met More than met

Outstanding

Interaction: other teachers as %



MID P 82 17 0 0 0

END P 46 15 18 16 5


Baseline 99 1 0 0 0

MID P 76 19 5 0 0

END P 27 20 25 24 4



MID P 0 66 33 0 0

END P 0 6 49 27 18


Teachers of English are using more pair and group work but teachers in other lessons were

given higher gradings more frequently on other measures of improved classroom interaction

such as giving effective feedback. All have benefited from the project training.

Table 3.8 Classroom Observations: Effectiveness of teaching: teachers of English

Not

met Partially met

Met More than met

Outstanding

Effectiveness of teaching: teachers of English as %



MID P 1 59 39 1 0

END P 0 8 57 29 6



MID P 22 44 34 0 0

END P 0 14 49 34 2



MID P 1 75 24 0 0

END P 0 8 58 27 6



MID P 1 76 22 0 0

END P 0 5 60 28 7

Table 3.9 Classroom Observations: Effectiveness of teaching: other teachers

Not

met Partially met

Met More than met

Outstanding

Effectiveness of teaching: other teachers as %



MID P 1 71 28 1 0

END P 0 5 44 38 12



MID P 31 55 14 0 0

END P 1 8 38 47 5



MID P 3 67 29 0 0

END P 0 8 46 32 13



MID P 1 68 30 0 0

END P 0 6 43 38 13

By the end of the project the English teachers do less well than the other teachers on each of

these selected aspects of pedagogy at the higher levels of more than met/outstanding. The

‘gap’ is always more than 10%, and some outstanding practice is recorded. Perhaps teachers

were able to refer to subject content in for example Science or Social Studies to add to

materials and to generate student questioning to check understanding, and less confident to

do so in an English lesson. They are applying learning from the project in different ways.

Teachers of English: no. of observations Baseline n = 447 MID P n = 348 END P n = 279

Other teachers: no. of observations Baseline n = 469 MID P n = 222 END P n = 273


Classroom observations: comparing all teachers by District

Disaggregating the data by district enables us to see if there were differences in the degree

or nature of changes in the classroom between the two districts, and speculate about the

possible reasons. These might be attributable to differences in the way the common training

package was delivered by the implementing partners: there were different patterns of follow

up sessions with GAN in Lamjung in a smaller geographical area making much use of the

Resource Centres, while the greater spread across Chitwan District brought people together

from a wider spread of schools. The smaller number of schools in the project in Lamjung

meant larger numbers of trainees from some schools – two schools each supplied nine

teachers - than was the case in Chitwan. Finally, the trainees in Chitwan included 48 who

recorded a low APTIS level of A0 in the initial testing, with a smaller number, 38, in Lamjung

scoring A0.

Table 3.10 Classroom Observations: Use of English: CHITWAN DISTRICT

0% 1-10% 11-25% 26-50% 51%+

Use of English: all teachers, as

percentage: Chitwan


Baseline 27 55 17 0 0

MID P 5 30 59 6 0

END P 2 0 8 34 56



MID P 8 71 20 1 0

END P 2 0 13 42 42

Table 3.11 Classroom Observations: Use of English: LAMJUNG DISTRICT

0% 1-10% 11-25% 26-50% 51%+

Use of English: all teachers, as percentage: Lamjung


Baseline 28 55 15 1 0

MID P 13 30 51 5 0

END P 6 5 19 23 47



MID P 18 43 36 4 0

END P 7 8 16 21 49

Much more extensive use of classroom English by the teacher is recorded in Chitwan with

figures across the top two levels of 90% and 84% while in Lamjung figures are 70% for each

measure. Certainly in Chitwan at the time of the project there was very high interest in

schools improving the quality of their English teaching or going English Medium, following

pressure from parents who in our focus groups were very strongly supportive of their children

acquiring a high standard of English.


Table 3.12 Classroom Observations: Interaction: CHITWAN DISTRICT

Not

met Partially met

Met More than met

Outstanding

Interaction: all teachers as %: Chitwan



MID P 68 32 0 0 0

END P 6 27 37 29 0



MID P 55 42 3 0 0

END P 0 22 39 39 1



MID P 0 68 32 0 0

END P 0 5 68 26 1



MID P 0 93 7 0 0

END P 0 11 70 18 0

Table 3.13 Classroom Observations: Interaction: LAMJUNG DISTRICT

Not

met Partially met

Met More than met

Outstanding

Interaction: all teachers as %: Lamjung


Baseline 99 0 0 0 0

MID P 88 12 1 0 0

END P 78 2 4 7 6


Baseline 98 1 0 0 0

MID P 72 14 13 0 0

END P 48 11 25 9 4



MID P 2 57 41 0 0

END P 0 4 38 44 14



MID P 1 52 47 0 0

END P 0 4 34 33 29

Lessons in Chitwan score higher for the use of pair or group work, which suggests strong

teacher planning of interactions to involve students in use of English. These are two areas

where Lamjung does very poorly with 78% and 48% of classroom observations recording no

pair or group work respectively. By contrast, Lamjung teachers score significantly higher on

giving encouragement and feedback to students at the top two levels of more than

‘met/outstanding’. The figures overall serve to again demonstrate the degree of improvement

made across the board: for example in Chitwan the measure around effective feedback

scored 95% below the satisfactory level of ‘met’ at baseline, and by project end this had

dropped to just 11% partially met.


Table 3.14 Classroom Observations: Effectiveness of teaching: CHITWAN DISTRICT

Not

met Partially met

Met More than met

Outstanding

Effectiveness of teaching: all teachers as %: Chitwan

The teacher gave students time to

respond


MID P 0 66 34 0 0

END P 0 11 66 24 0



MID P 18 43 39 0 0

END P 0 10 51 38 1



MID P 1 86 13 0 0

END P 0 9 72 19 0

Table 3.15 Classroom Observations: Effectiveness of teaching: LAMJUNG

Not

met Partially met

Met More than met

Outstanding

Effectiveness of teaching: all teachers as %: Lamjung


respond


MID P 2 61 36 2 0

END P 0 2 32 45 20



MID P 34 55 11 0 0

END P 2 12 35 44 7



MID P 1 57 40 1 0

END P 0 2 27 50 21

These areas of pedagogy show a lot more very good practice in Lamjung than in Chitwan,

where the great majority are ‘met’ and thus satisfactory.

Across all the indicators we can tentatively identify differential success in the two districts of

the transfer of training into the classroom. These differences – for example more English in

the classroom in Chitwan, more improvement in some aspects of pedagogical practice in

Lamjung – should be reflected upon in defining and structuring future projects.

Chitwan: Baseline n = 476 MID P n = 314 END P n = 299 Lamjung: Baseline n = 425 MID P n = 256 END P n = 253


Classroom observations: comparing all teachers by Gender

Classroom observation data were generated from a good gender balance of teachers – the

sample group consisted of 98 female and 105 male teachers [see Appendix].

Table 3.16 Classroom Observations: Use of English: all FEMALE teachers

0% 1-10% 11-25% 26-50% 51%+

Use of English: all female teachers, as percentage


Baseline 30 51 19 1 0

MID P 12 30 52 7 0

END P 4 2 10 31 54



MID P 56 41 2 0 0

END P 12 30 43 11 3

Table 3.17 Classroom Observations: Use of English: all MALE teachers

0% 1-10% 11-25% 26-50% 51%+

Use of English: all male teachers, as percentage

The teacher used English during

the lesson

Baseline 26 60 14 0 0

MID P 6 30 59 5 0

END P 4 2 16 28 49



MID P 50 49 1 0 0

END P 13 42 34 8 3

Use of English by the teacher during the lesson shows no significant difference; there is more

student use of English to the teacher in lessons with a female teacher.

Table 3.18 Classroom Observations: Interaction: all FEMALE teachers

Not

met Partially met

Met More than met

Outstanding

Interaction: all female teachers as %



MID P 73 26 1 0 0

END P 33 17 22 24 3


Baseline 99 1 0 0 0

MID P 58 32 9 0 0

END P 17 16 33 31 3


Table 3.19 Classroom Observations: Interaction: all MALE teachers

Not

met Partially met

Met More than met

Outstanding

Interaction: all male teachers as %


Baseline 99 1 0 0 0

MID P 81 19 0 0 0

END P 46 14 22 14 3


Baseline 99 1 0 0 0

MID P 67 27 5 0 0

END P 28 17 33 19 2

Female teachers were observed using more group and pair work.

Table 3.20 Classroom Observations: Effectiveness of teaching: all FEMALE teachers

Not

met Partially met

Met More than met

Outstanding

Effectiveness of teaching: all female teachers as %



MID P 2 78 20 0 0

END P 2 22 38 33 7

The teacher used gestures to support

meaning


MID P 13 66 21 0 0

END P 2 7 43 47 3



MID P 23 45 32 0 0

END P 1 6 43 47 3



MID P 2 71 26 0 0

END P 0 6 55 30 9

Table 3.21 Classroom Observations: Effectiveness of teaching: all MALE teachers

Not

met Partially met

Met More than met

Outstanding

Effectiveness of teaching: all male teachers as %



MID P 3 81 14 0 0

END P 2 27 41 23 6

The teacher used gestures to support meaning


MID P 20 63 17 0 0

END P 1 18 43 36 2



MID P 28 51 21 0 0

END P 1 16 45 34 3



MID P 1 72 25 1 0

END P 0 11 49 29 11


Female teachers achieve higher classroom observation ratings than males on the first three of

these measures, with typically 10% fewer ‘partially met’ and 10% more ‘more than met’. The

last measure here, correction of students, is the only measure where more men than women

score ‘more than met/outstanding’.

The overall picture is that from similar starting scores female teachers have made greater

improvement in a range of classroom activities and pedagogical methods, including student

use of English, with little gender difference in terms of teacher use of English.

Male teachers Baseline n = 455 MID P n = 283 END P n = 263 Female teachers Baseline n = 454 MID P n = 287 END P n = 289

Classroom observations: the A0 teachers

As noted above the trainees in Chitwan included 48 who recorded a low APTIS level of A0 in

the initial testing, with a smaller number, 38, in Lamjung scoring A0. These teachers were

thought to be outside the level of personal English needed to benefit from the ET:TE+

training, though their scores were at the top end of A0.

A comparison was made to measure their progress against all teachers.

Table 3.22 Classroom Observations: Use of English: all A0

0% 1-10% 11-25% 26-50% 51%+

Use of English: A0 teachers

against all teachers, as %


Baseline A0 35 50 13 1 0

END P: all 4 2 13 29 52

END P A0 5 3 20 27 45


Baseline A0 62 34 3 0 0

END P all 4 4 14 32 45

END P A0 5 3 20 33 39


Baseline A0 69 28 2 0 0

END P all 5 5 21 34 35

END P A0 5 3 23 38 30


Baseline A0 82 18 0 0 0

END P all 12 36 39 10 3

END P A0 16 34 38 9 3

The students spoke to each other in English

Baseline A0 100 0 0 0 0

END P all 54 31 9 4 2

END P A0 70 15 5 5 3


Table 3.23 Classroom Observations: Interaction: all A0

Not

met Partially met

Met More than met

Outstanding

Interaction: A0 teachers against all teachers as %


Baseline A0 100

END P all 22 17 33 25 3

END P A0 27 21 32 14 4


Baseline A0 35 64 2

END P all 0 5 54 34 7

END P A0 0 3 54 31 11

It is striking that the pattern for the A0 cohort mirrors closely that of ‘all teachers’ [taken

from Table 3.1] in these typical exemplar measures. So their own English may be weak but

they could and did use it as much in the classroom as A1 or A2 teachers. We were not

attempting to evaluate the quality of the English being used which would have involved a

different evaluation process. The A0 teachers used group work too with positive classroom

behaviour.

Table 3.24 Classroom Observations: Effectiveness of teaching: all A0

Not

met Partially met

Met More than met

Outstanding

Effectiveness of teaching: all teachers as %


Baseline 63 38

End P: All 2 24 39 28 6

R2 2 32 33 23 10


respond

Baseline 35 61 4

End P: All 0 7 51 33 9

R2 0 8 44 31 17

The teacher used gestures to support meaning

Baseline 47 52 2

End P: All 1 12 43 42 2

R2 2 16 41 38 3

The teacher used materials to

supplement the lesson

Baseline 67 32 2

End P: All 1 11 44 41 3

R2 15 40 40 5


Baseline 39 53 9

End P: All 0 8 52 30 10

R2 0 7 46 31 15


Baseline 51 40 9

End P: All 0 5 52 33 10

R2 0 3 47 35 15

The lesson effectively developed a skill Baseline 68 28 3

End P: All 0 13 62 23 2

R2 0 18 53 26 3

The lesson effectively covered a language area

Baseline 80 18

End P: All 1 11 64 22 3

R2 0 12 59 24 5


In fact on all the measures of effectiveness of teaching in Table 3.24 the A0 teachers actually

did slightly better than all teachers on all but one measure at higher ‘more than

met/outstanding’ levels.

On some measures there is also a slightly larger percentage of A0 teachers than all teachers

left behind at the ‘not met’ level. This suggests that many but not all A0 teachers were

competent and enthusiastic teachers [with weaker personal levels of English] who made good

progress through the project, especially in general classroom effectiveness and in using

English as the medium of instruction in the classroom. The ET:TE+ project’s training manual,

follow up support, etc, can work effectively for teachers with a low Basic User competence in

English, if care is taken over the selection of participants.

All teachers Baseline n = 916 END P n = 552 A0 teachers Baseline n = 200 END P n = 128

A full set of all the above tables is available as part of the Technical Report.


4. Findings: APTIS testing of teachers’ levels of competence in English

Nominated teachers were tested on Saturdays in February and March 2013 by British Council

Nepal supported by this team using the British Council’s APTIS test to assess adults’ ability in

English, and again on two successive Saturdays in late November and early December 2013

at the end of the project.

Initial baseline testing

There were two rounds of initial testing in each district at central locations. A total of 443

teachers were tested. Outcomes were:

Table 4.1 All teachers’ overall APTIS scores, initial testing

A0 A1 A2 B1 and better

Total (n = 443) 230 173 31 9

52% 39% 7% 2%

English teachers (n = 84) 27 48 7 2

32% 57% 8% 2%

Non-English teachers (n = 359) 203 125 24 7

57% 35% 7% 2%

All teachers, Chitwan (n = 230) 130 90 10 0

57% 39% 4%

All teachers, Lamjung (n = 213) 100 83 21 9

47% 39% 10% 4%

Female teachers (n = 193) 99 79 11 4

51% 41% 6% 2%

Male teachers (n = 250) 131 94 20 5

52% 38% 8% 2%

There was no statistically significant difference between the performance of female and male

teachers; at a second level of analysis these gender scores remained very similar across the

two districts. Variations between scores at English medium and Nepali medium schools as a

whole or in each district separately did not reach a level of statistical significance.

End of project final testing

All those teachers who participated in the ET:TE+ training project were expected to attend to

be re-tested in November/December 2013. Every effort was made to inform and encourage

teachers to attend the test centre, through implementing partners, DEO staff, etc. 81% did

so; of the remainder perhaps half were known to be ill or had personal reasons leaving them

unable to attend, had changed schools or had dropped out of the project at a much earlier

stage. Attendance was higher in Chitwan than in Lamjung.

Table 4.2 All teachers’ overall APTIS scores, final testing


Total (n = 234) 67 137 30 0

29% 59% 13%

English teachers (n = 47) 10 31 6 0

21% 66% 13%

Non-English teachers (n = 187) 57 106 24 0

30% 57% 13%


All teachers, Chitwan (n = 128) 21 81 26 0

16% 63% 20%

All teachers, Lamjung (n = 106) 46 56 4 0

43% 53% 4%

Female teachers (n = 110) 30 64 16 0

27% 58% 15%

Male teachers (n = 124) 37 73 14 0

30% 59% 11%

In the final test no one scored at the Independent User level of B1 or better. Comparisons

drawn from Tables 4.1 and 4.2 should be treated with caution in that the baseline in Table

4.1 includes many teachers who did very poorly and did not go on to join the ET:TE+

training. However in Table 4.3 below the scores of all individuals who were tested on both

occasions are brought together and compared.

Table 4.3 Overall APTIS scores, same teachers, initial and final testing


Total, initial test (n = 234) 83 123 23 5

35% 53% 10% 2%

Total, final test (n = 234) 67 137 30 0

29% 59% 13%

These figures show a slight improvement in APTIS test scores in the cohort as a whole with

6% moving up from A0. However this masks a more significant feature which is the high level

of fluidity in the actual scoring. Only 103 teachers [44%] kept the same score. In Chitwan 51

teachers went up at least one level, 11 went down at least one level and 66 recorded the

same level. In Lamjung 23 teachers went up at least one level, 46 dropped at least one level

and 37 recorded the same level. In total 7 individuals went up two levels and 7 down two

levels. Of the 57 who dropped a level 27 were female and 30 male, and of those going up a

level 38 were female and 36 male.

There is no convincing evidence that teachers were more likely to score highly or to improve

if in an English Medium school

Commentary

The CEFR descriptor for A1 indicates a low level of basic language competence:

Can understand and use familiar everyday expressions and very basic phrases aimed at

the satisfaction of needs of a concrete type. Can introduce him/herself and others and can

ask and answer questions about personal details such as where he/she lives, people

he/she knows and things he/she has. Can interact in a simple way provided the other

person talks slowly and clearly and is prepared to help.

Initial APTIS test results showed that 39% of those tested performed at this A1 level with

52% scoring below A1 and therefore graded as A0. Only 9% scored above this basic level at

A2 or better. The teachers took two APTIS tests, the first testing Grammar and Vocabulary

and the second testing Reading. Analysis of the results suggested that it was the Reading test

that became the discriminator at the A0/A1 borderline. Eg. in Lamjung Sample 1, Candidate

53 scored 16 + 44 = B2, while Candidate 8 scored 22 + 4 = A0.

The ET:TE+ training project was developed for students performing on entry at language

levels A2/B1 but was seen in Nepal as accessible to A1 students given the project’s twin


emphases on development of personal language ability as well as development of pedagogical

skills. It was felt that some of the A0 teachers with respectable scores on the Grammar and

Vocabulary test could also benefit from the project, having been alerted to their low

performance level as a key area for development.

It is disappointing that the APTIS testing did not show much of an improvement in individual

teachers’ competence in English on these measures, however this was not expected in the

project design. There was a great deal of fluidity in scoring – only 44% of those tested

received the same grade in both test rounds. This team’s experience of the APTIS process

would be

- the test was invaluable as a filter to remove those with such low levels of English as to

be inappropriate for the ET:TE+ training

- the testing was taken very seriously by all concerned and added status, importance

and visibility to the ET:TE+ project. One district DEO told us he was very pleased than

an international standard test was being used

- written tests of Grammar and Vocabulary and Reading are not a good fit with the aim

of the training to develop and encourage speaking and listening skills and the use of

those skills in the classroom. Perhaps it would be unreasonable to expect the positive

impact of the training to show up strongly in the repeat APTIS tests


5. Findings: Interviewing teachers of English

Teachers of English involved in the ET:TE+ project were identified as a subgroup of the

training cohort. At Baseline each of those who identified as teachers of English [n = 84] was

interviewed at their school. Data was 90+% complete. In the monitoring phase, all teachers

in the sample were interviewed at mid programme and again at end of programme [n = 54].

Gender and age data on the group of English teachers showed a gender balance and a spread

of age and therefore experience. The age profile was younger in Chitwan where 45% of the

sample was aged 35 or younger [Lamjung 23%].

Table 5.1 Gender and Age data on English teachers

Chitwan [n = 47] Lamjung [n =37] Total

English

medium

Nepali

medium

English

medium

Nepali

medium

Female 18 7 16 1 42

Male 17 4 16 4 41

Age below 25 3 2 2 0 7

25-35 13 3 5 0 21

36-45 9 5 15 3 32

Over 45 7 5 4 1 17

Teachers and their own English

Baseline data

33% said that part of their own education had been in English but 67% said none of their

education had been in the medium of English. When asked how frequently they used English

outside school, 41% responded ‘daily’, 18% said ‘weekly’ and 41% said ‘less than once a

week’. But an overwhelming 88% felt it was ‘essential’ for their pupils to learn English to a

good standard.

Interviewees were asked about their own English, and their level of confidence in teaching

English.

Table 5.2 How would you describe the standard of your Speaking and Verbal Understanding

in English at the moment?

Baseline Chitwan [n = 47] Lamjung [n =37] Total

English medium

Nepali medium

English medium

Nepali medium

a good standard 7 0 0 0 9%

quite a good standard

21 6 13 3 52%

not a very good standard

7 5 16 2 37%

very weak 0 0 2 0 2%

Interviewees were asked how confident they were at this start point in their teaching of

English. 13% said they were ‘very confident’, while 47% said ‘quite confident’. 38% rated

themselves as ‘uncertain’ with just 2% choosing ‘very uncertain’.


Mid programme responses

Interviewees were asked again about their own speaking and verbal understanding of English.

80% chose ‘quite a good standard’, up from 52% at baseline, and 18% chose ‘not a very

good standard’, down from 37% at baseline. 41% recorded a large or very large difference in

their skill levels in English with 48% saying their confidence to use English was now much or

very much higher.

95% felt ‘quite confident’ or ‘very confident’ to use spoken English all the time in the

classroom. 96% reported they felt ‘more confident’ or ‘very much more confident’ in teaching

English as a result of the ET:TE+ training.

End of project responses

Asked the same question again, numbers choosing ‘not a very good standard’ fell again, down

to 11% from 18% at mid project and from 37% at baseline.

Table 5.3 How would you describe the standard of your Speaking and Verbal Understanding

in English at the moment?

Baseline Mid programme End of programme

a good standard 9% 2% 15%

quite a good standard 52% 80% 74%

not a very good standard 37% 18% 11%

very weak 2% 0 0

74% also reported a large or very large difference in their own skill levels in English, up from

41% at the mid project stage, indicating the progress they felt they had made during the

period of the project.

Confidence levels to use English in the classroom remained high and in interview many

teachers spoke enthusiastically about the difference they felt in their own confidence to teach

English through active use of the language in the classroom and interactive methods.

Teachers and good practice in the classroom

The interviews with teachers of English also probed interviewees’ knowledge and

understanding of good classroom practice in two aspects: student-centred learning and

communicative language teaching. Both of these areas were to be highlighted through the

ET:TE+ training project.

Baseline responses

The expression ‘student centred teaching’ was recognized by 90% of English teachers at

interview. They selected two key characteristics but with significantly lower scores: ‘teaching

to the children’s level’ 74% and ‘interaction between teacher and students’ 71%.

74% recognized the expression ‘communicative language teaching’ but the most-reported

example in this area was ‘pupils talking together in pairs or groups’ mentioned by only 56%.

Research associates conducting the interviews reported much greater hesitancy and

uncertainty in responses. In both these items, responses indicate awareness of the principal

term but a lack of understanding of what it would mean in concrete classroom practice.

Teachers were asked about the availability to them of mentoring and/or peer support. 75%

reported mentoring from headteacher or senior teacher and 66% access to peer support.


Mid project responses

In Mid Project interviews key student-centred aspects were reinforced: ‘interaction between

teacher and students’ is now at 85% and ‘student/student’ interaction up to 81%. ‘teaching

to the children’s level’ has fallen away to 59%. With communicative language teaching 68%

selected ‘pupils talking together’ but only 31% ‘pupils given roles like reporting back’, a more

specific lesson planning technique to promote oral work. All those responding wanted to use

more resources in the classroom.

At Mid Project the great majority of these teachers reported the availability of opportunities to

access mentoring and peer support.

There was a 95% or higher approval rating for the timetable and content of the training and

100% positive feedback on the ET:TE+ trainers and the help received with resources.

Teachers were very proud of having the opportunity to participate in the ET:TE+ programme,

often describing this as the best training they’d ever had.

More than 80% said they had been able to add extra resources in English. These were

overwhelmingly ‘maps, charts and wall displays’ [38 out of 54 teachers], using some

community funding [14/54] or ‘I have bought them myself’ [22/54]. Research Associates

reported the use of homemade instructional materials and resources – flashcards, wall

pictures with English words beneath, etc. Some of these had been made jointly by teacher

and children. A lack of resources was the most quoted barrier to putting ET:TE+ learning into

practice [28/54].

Teachers remained firmly of the view that learning good English is ‘important’ [30%] or

‘essential’ [70%] for their pupils.

End of project responses

When re-interviewed at the end of the project, a very similar picture emerged of a group of

professionals who had benefited greatly from their training weeks and follow up sessions.

There was more comment about the detail of pedagogical techniques and Research

Associates recording these interviews fed back that by this end point six months later

teachers appeared to be discussing established practice rather than practice which they were

trying for the first time. This would be consistent with findings from classroom observations.

Finally the English teachers were asked what they felt they had gained from the project:

Table 5.4 What do you feel you have gained from the ET:TE+ training?

Aspect END P No.

(n = 53)

to improve my speaking and communication in English 49

to improve my method of teaching 41

to gain confidence in my teaching 39

to learn how to use resources 39

to improve my reading and writing English 36

to meet and talk with other English teachers 32

to gain promotion in a school 13

The English teachers’ choice of statements is positive in that they choose statements which

actually correspond well with the aims of the project, which has delivered in its main target

areas.


6. Findings: Other views expressed in schools

During fieldwork, views were also sought at each school through:

A focus group of pupils, discussing the English lesson just observed and their

experience of English

A focus group of adult stakeholders, normally including English teacher(s),

headteacher, SMC chair, parents and other local stakeholders

Analysis of the qualitative data from focus groups indicates:

The young primary-age children who formed our pupil focus groups confirmed that the

English lesson just observed by our researcher was typical of the lessons they now

received. They are able to use spoken English very infrequently in class; 85% said they

rarely or never talk to a classmate in English and 89% that they rarely or never talk to a

group of classmates in English, but 68% said they sometimes and 11% that they nearly

always talk to their teacher in English. Well over 90% of groups said they had enjoyed

their English lesson.

Adults in each of the 54 focus group meetings included the English teacher, the

headteacher, an SMC member and at least one parent – often groups were quite large,

even the whole SMC with each member keen to add his or her words of support. They saw

the world changing especially for the next generation and the importance of equipping

students with a good standard of English. Schools had a more selfish interest in retaining

the support of parents against private school competition and loss of pupil numbers.

100% agreed or strongly agreed that it is important for children to learn good English and

96% that the school/community will do all it can to support more effective English

teaching: 98% felt that the ET:TE+ training was having a positive effect at the school.

The views of parents were gathered through the focus group meetings and through informal

contacts reported by Research Associates. Parents continue to strongly support their pupils

being taught to a good standard in English. They talked of wanting continuing monitoring and

support for ET:TE+ teachers so they continue to improve. Parents noted the wider range of

activities taking place and reported other teachers also using English more. One comment

from a rural primary in Lamjung was that better English teaching would help retain pupils in

their village school and hence keep it open, rather than parents sending their children into

town, to government or private schooling.

Research Associates reported specific interest from some SMC chairpersons many of whom

have political connections, particularly where the school was considering converting to English

Medium. Headteachers expressed confidence that the project was impacting on teacher and

school, picking out the clear, well-delivered training, the high quality manual and the fact that

training events had taken place outside school session days which helped them

organisationally. These points were made particularly by teachers from rural areas.


7. Conclusions

1. Standards of competence in English among teachers wishing to join the project were

found to be very low even in a generally more developed district like Chitwan. The initial

APTIS tests found 90% at A0 or A1. What is locally considered ‘good English’ is in

absolute language terms not very good at all. This may reflect previous and to a large

extent current curriculum and teaching methods in Nepali schools and the teacher’s own

learning of English, as well as their limited daily access to and use of English. 41% of

English teachers said they used English themselves ‘less than once a week’, which may be

under-reported given the high status nature of English. Many people including those

active on the ground in the districts were surprised by the low scores. There are large

training needs at a basic level, ie below the ET:TE + threshold level.

2. The low levels of competence in English – well below that required to teach English to

others and to teach other subjects through the medium of English - cast doubt on the

wisdom or the ability of government schools to go ‘English Medium’ unless and until many

of their teachers have improved their own competence in English or what being an

‘English Medium school’ means in language terms. If the standard of English is inadequate

trying to teach subject content through that medium will fail, while English itself is a

much-failed subject at SLC in government schools.

3. The baseline classroom observations also showed the extremely limited extent of good

pedagogical practice actually happening before the course began. Most teachers were

familiar with expressions such as ‘child-centred learning’ but fewer could confidently give

examples of what this meant in the classroom. There was strong support from their

schools to join the project.

4. District officials, teachers and schools when first contacted were enthusiastic to improve

the quality of English learned by their pupils: a good standard of English is rated

“essential” for the next generation, and was often described as the key driver for the

spread of English Medium private schools. This enthusiasm was repeated frequently at the

end of the project, from District Education Officer to minority ethnic Chepang students in

a rural school. The local print and FM radio media treated the initial APTIS testing and the

arrival of ET:TE+ training as major local news events with live interviews in March 2013

by Kalika FM, the mid-Nepal commercial radio station based in Bharatpur, Chitwan. There

were more reporters at the final APTIS testing in Chitwan.

5. Into this very mixed context, of low standards but great positive enthusiasm, the ET:TE+

project has been extremely successful as the data in this Report demonstrate. The

enthusiasm has been maintained, new practices have been tried out and have become

mainstream for many teachers, as their students testify. Teachers have had the

confidence, and a realistic confidence, to make English the language of their classroom

and of their students. Recorded progress shows that classroom practice can be changed

and that change sustained, which is a very positive finding.

6. Key successes evidenced from classroom observation and interviews include

Transformation in the use of English by teachers in the classroom

Positive and significant progress in pupils using spoken English in the classroom, from

a very low base

Introduction and sustaining of interactive group work and pair work

Greatly improved pupil involvement in lessons, and feedback and encouragement from

teachers

The successful introduction and use of more materials in the classroom

Extending the reach of ET:TE+ training to A1 and A0 beginner-standard teachers

Stimulated, motivated teachers strongly supported by schools to improve practice


Additionally, the use of an external test, here APTIS, was valuable and valued. Its use

as a gatekeeper should be reviewed in view of the degree of ‘churn’ in scores and the

progress made by A0 teachers

7. Areas of least change within the aims of the project include

Making interactive, more student-centred classrooms the norm, eg pair and group

work and students’ talking to the teacher in English

Meeting the needs of all teachers – many have such poor levels of English as to be

outside the target group, although those scoring A0 but entering the training made

good progress

The intended focus on rural and deprived locations was not taken forward by

implementing partners, though some remote areas such as Mardi in southern Chitwan

were represented

We would like to thank all those concerned for their invaluable help and support in compiling

this report, in particular district education officers and their staffs, the two implementing

partners, PANT films and of course British Council Nepal. Most thanks must go to the teachers

and their schools for their commitment to and support of the ET:TE+ project.

On behalf of Richard Thompson Associates

Monitoring & Evaluation Team

Technical Advisor Richard Thompson

Team Leader Min Bahadur Ranabhat

Research Associates Ram Prasad Paudel

Kiran Sedhain

Subodha Babu Chiluwal

Gajendra Laudahari

Data preparation and analysis Dr Shankar Khanal

Mr Raju Manandhar


Appendix: Sample for Monitoring & Evaluation Programme

It was agreed in discussing this Monitoring & Evaluation Programme for ET:TE+ that a

sampling process would be used to identify a representative subset of teachers and schools in

each district to be monitored. This is a longitudinal study using data collected:

- as part of the baseline survey, March 2013

- in a first round of monitoring [R1], June/July 2013

- in a final round of monitoring [R2] scheduled for late 2013/early 2014

In each of the two districts a complete list of those who actually participated in the ET:TE+

training by NELTA [Chitwan] and GAN [Lamjung] in April 2013 was kindly supplied by those

organisations. Names were checked against our baseline records to ensure that baseline data

[classroom observations, teacher interviews, APTIS test scores] was in place. A sample was

then developed separately for each district that would contain a full data set and enable the

agreed collection of data under this Programme.

In Chitwan District the sample consisted of 30 schools, with 30 identified teachers of English

[one per school] for extended classroom observation and interview. NELTA is training

teachers from a total of 64 schools. The schools consisted of 8 Higher Secondary Schools, 7

Secondary Schools, 8 Lower Secondary Schools and 7 Primary Schools, with a mix of urban

and rural locations across the district. The schools contained a further 56 teachers training on

the project, teaching a variety of subjects including in some cases some English, giving a

total of 86 teachers of whom 44 were female and 42 male.

In Lamjung District the training project was concentrated by GAN in one part of the District

and in fewer schools [31 in total]. The sample therefore covered only 24 schools with 24

identified teachers of English [one per school] to be interviewed, The schools consisted of 2

Higher Secondary Schools, 6 Secondary Schools, 3 Lower Secondary Schools and 13 Primary

Schools. There were in many schools a higher number of teachers engaged in ET:TE training,

giving a full sample size of 117 teachers, of whom 54 were female and 63 male.

Baseline data for each of the schools and individual teachers in these samples was extracted

from the full baseline data set and used for later comparative purposes. The data set of the

sample was back-checked to test its representativeness. Taking the key question of use of

English in the classroom, as observed and recorded by this team, we found:

Baseline data: Classroom Observations for entire ET:TE+ cohort

0% 1-10% 11-25% 26-50% 51%+

Use of English: all teachers, as percentage (n = 916)


28 55 16 1 0


58 40 3 0 0


68 31 1 0 0


84 16 0 0 0


99 1 0 0 0


Baseline data: Classroom Observations for the subset M & E ET:TE+ sample

0% 1-10% 11-25% 26-50% 51%+

Use of English: all teachers in sample schools, as percentage (n = 548)


28 54 17 1 0


57 39 4 0 0


69 29 2 0 0


82 18 0 0 0


98 2 0 0 0

All scores are rounded to nearest %.

These tables demonstrated that the pattern of scores across the entire baseline cohort was

very closely followed by the subset of teachers and schools selected as the M & E sample.

This confirmed the representative nature of the sample in terms of baseline data and enabled

us to proceed with confidence.

Documents

Richard Thompson Associates - britishcouncil.org.np · Richard Thompson Associates 16 Conway Road London N14 7BA BRITISH COUNCIL NEPAL ENGLISH FOR TEACHING: TEACHING FOR ENGLISH PLUS