Literacy Boost

Kailali, Nepal

Year 1 Report

2010

Christabel Pinto

© 2010 Save the Children

2

I. Introduction: Context and program intervention

Save the Children's Literacy Boost (LB) program was implemented as a pilot program in Kailali, a district in the far-western region of Nepal, in April 2009, the start of the school year. The program was implemented in 16 schools and approximately 52 communities located in Bouniya and Kotatilsipur village development communities (VDCs), in partnership with Backward Society Education (BASE). Save the Children has worked in the area for seven years where the target communities are amongst the poorest in Nepal. 56.5% of adults in Nepal are literate (UNESCO, 2007). The literacy rate is much higher for men: 70.3% of men are literate compared to 43.6% of women (UNESCO, 2007). The youth literacy rate in Nepal is 79.3% (UNESCO, 2007).

The Literacy Boost program aims to strengthen children's skills in learning to read and reading to learn. Literacy Boost views reading as a complex process that integrates the use of text, letter knowledge, phonological awareness, vocabulary, fluency and comprehension. Many of these skills can be assessed quickly and easily. As a result, Literacy Boost is a program built on evidence gathered by regular assessments that track the progress and identify needs of the program's efforts to help children learn to read. The program includes both teacher training and community action interventions. Literacy Boost in Nepal has six main activities: book banks, teacher training, reading buddies, L2 to L1 literacy classes, community workshops and reading camps. The purpose of this study is to compare data that were collected at baseline and at the end of the school year, on reading and numeracy skills of second graders in the 16 schools where Literacy Boost has been implemented and in 4 control schools. Comparisons between LB schools early in the school year and these same schools late in the school year will inform conclusions on program impact. In addition, household literacy surveys and parent literacy interviews were conducted for half of the students who were assessed to further understand the impact of Literacy Boost on fostering a culture of reading. II Methodology

Baseline data were collected during the period from August 20th 2009 to September 15th 2009 and final data were collected around six months later, at the end of the school year, from March 4th to April 1st 2010. There were 20 schools in total. Of the 16 schools in which Literacy Boost has been implemented, 8 were located in Bouniya VDC and 8 were in Kotatulsipur VDC. The 4 control schools were picked randomly from adjoining VDCs for their similarity to Literacy Boost school communities: 2 schools were in Joshipur VDC, 1 school in Darakh VDC, and 1 school in Sadepani VDC.

At baseline, a team of four enumerators, bilingual in Tharu and Nepali, conducted the assessments and surveys. Two enumerators assessed the literacy and numeracy skills of 20 grade 2 students, 10 boys and 10 girls, from each school where attendance permitted this sample selection. Students were chosen through systematic random selection. Two additional enumerators visited households of a randomly selected subset of 10 of these students to conduct parent interviews and household surveys. Assessments

3

and surveys were conducted in Tharu and/or Nepali depending on the preference of the respondent.

The student assessment consisted of six parts: student background questions about the students' homes, families and lives; letter identification requiring students to identify the Nepali alphabet; Concepts About Print which asks students 10 questions on how to handle a book and interpret print; reading a passage; reading comprehension questions, and 16 numeracy questions.

At the end of the year, a team of 6 enumerators, one of whom had been an enumerator at baseline, conducted both student assessments and household visits. Out of the 368 students assessed at baseline, names and codes were available for only 272 of them. These 272 students were sought at the end of the year, so that the baseline data could be matched with end of year data, and the remaining students were sampled through systematic random selection to attain a sample size of 20 children (10 boys, 10 girls) from each school, where attendance permitted. Matched data were obtained for only 214 out of the 272 baseline students who were sought at the end of the year due to student absences. Data were collected during the exam preparation and exam-taking period. Exam days were shorter than regular school days: only 1-2 hours long depending on how much time individual students took to complete the day's exam, compared to regular 5-hour schools days, so fatigue was not thought to be a compounding variable for the 30-45 minute assessments done during the exam period. The same enumerators conducted household visits after the student assessments were completed.

At baseline and at the end of the year, photographs were taken of all 20 classrooms from which students were selected. In addition, at the end of the year, three enumerators conducted adult Tharu literacy assessments at an L2-L1 literacy center, and one enumerator conducted interviews with teachers to obtain their feedback on the impact of the LB teacher training on their teaching practice. Table 1 summarizes the type of data collected and the instruments used. Table 1: Type of data collected and instruments

Type of Data collected

Sample size at baseline

Sample size at end of year

Data collection tool

Reading/Numeracy skills of grade 2 students

368 students (291 LB, 77 Control)

376 students (296 LB, 80 Control)

Reading/Numeracy assessment

Parent/Family literacy skills, habits and

interactions with child

190 parents/guardians 199 parents/guardians

Parent literacy survey

Household literacy environment

190 households 199 households Household literacy environment checklist

Grade 2 classroom literacy environment

20 classrooms 20 classrooms Camera

Adult Literacy Skills

Not collected 15 Adult Tharu reading assessment

Teacher feedback on LB training

Not collected 9 Teacher interview

4

Characteristics of students in the study were compared across treatment groups and across time since only 214 students are common between baseline and end of year assessments (Table 2). Table 2: Characteristics and Living Situation of Students Literacy Boost

at Baseline Literacy Boost at End of Year

Comparison at Baseline

Comparison at End of Year

Gender Male Female

N=291 49% 51%

N=295 47% 53%

N=77 48% 52%

N=80 47.5 52.5

Age 8.59 (N=263, SD=1.75)

8.89 (N=260, SD=1.74)

9.18 (N=60, SD=2.01)

9.66 (N=59, SD=1.49)

Home language Tharu Nepali Achhami Doteli Nepali/Tharu Hindi Tharu/Hill lang

N=290 69.0% 27.9% 0.3% 2.1% 0.7%

0 0

N=291 67.4% 28.5%

0 2.7%

0 1%

0.7%

N=77 94% 5% 1% 0 0 0 0

N=79 81%

15.2% 1.3%

0 2.5%

0 0

Type of roof Mud Tile Tin Thatch Concrete Tile&Tin Other combinations

N=291 59.5% 15.1% 10.0% 2.7% 10.3% 2.3%

N=291 50.5% 19.9% 13.1% 6.5% 7.2% 2.6%

N=77 73% 13% 5% 3% 5% 1%

N=79 55.7% 19% 6.3%

16.5% 2.5%

0

Type of walls Mud Brick Concrete Mud & Brick

N=291 89.0% 10.3% 0.7%

0

N=291 91.1% 6.9% 1.4% 0.7%

N=77 91% 8% 1% 0

N=79 91.1% 6.3% 2.5%

0 Type of floor Dirt Concrete

N=291 96.6% 3.4%

N=291 97.6% 2.4%

N=77 99% 1%

N=79 97.5% 2.5%

% who live in homes with Electricity Radio T.V. Books In-house toilet Land

N=291

92% 61% 37%

100% 24% 96%

N=290

91.4% 56.9% 32.8% 99.7% 23.4% 95.2%

N=77

85% 62% 27%

100% 13% 96%

N=79

72.2% 64.6% 26.6% 100% 11.4% 91.1%

The children in the sample were 8 to 10 years old and most claimed to speak

Tharu in their homes. The children who did not speak Tharu at home largely spoke Nepali. Fewer than 3% spoke other languages. Most of the children lived in homes that had tile roofs, mud walls and dirt floors. The communities from which students were selected are largely agrarian as evidenced by the large proportion of students, more than

5

90%, whose families have land. Nearly all students lived in homes with books but the question did not distinguish between different types of books so students could have been referring to textbooks. Students in the sample generally came from homes with electricity, around 60% had radios, and at least a quarter of the comparison group and a third of the LB group had TVs. There was a greater proportion of students in the comparison group who were Tharu-speaking compared to the Literacy Boost group at both baseline and at the end of the year. This is important since the relationship between language spoken at home and literacy skills and acquisition will be explored. While the characteristics of the LB group were similar at baseline and at the end of the year, the comparison group changed on a few characteristics. There were 79 students in the comparison group at the end of the year: 37 were students who had been assessed at baseline and 42 new students were sampled randomly. There was full attendance at 3 of the 4 comparison schools, since assessments were conducted on exam days, and only 3 students from the fourth school were absent so the shift in characteristics cannot be attributed to dropouts or absences. Forty-two new students were selected because not all of the names and codes of students assessed at baseline were available. In the comparison group the proportion of Tharu-speakers dropped from 94% to 81% and the proportion of Nepali-speakers increased from 5% to 15.2%. 72% of those sampled at endline were Tharu-speakers compared to 89.2% of those sampled at baseline. This result is puzzling because the last names of the students selected at endline, which denote community, would suggest that a higher proportion belong to the Tharu community than 72% and would probably speak Tharu at home. At baseline, enumerators observed that children sometimes said they were Nepali-speakers when they were clearly Tharu-speakers; enumerators corrected for this mistake in student self-reporting. Enumerators at endline were informed that this was an issue during training so that they could be consistent with baseline enumerators. Other ways in which the comparison group changed from baseline to the end of the year included the type of roof of the house the family lived in and whether families had electricity or land. The percent of students who live in homes with electricity and land decreased suggesting that the students sampled at endline were of a lower socioeconomic status and/or that the baseline students that were not included at endline were of a higher SES. However, the percent of students who lived in homes with mud roofs decreased while the percent of those who lived in homes with cement homes increased and these results would suggest the opposite: a higher socioeconomic status amongst those sampled at endline and/or a lower SES amongst baseline students who were not included at the end of the year. The data are all student-reported and children could be diffident in how they interacted with the research team and sometimes needed to be coaxed to speak, which may help to explain the inconsistencies. Children in comparison schools may have less experience talking about their lives and talking to outsiders than children in LB schools. On the other hand, it is possible that the inconsistencies are real and that the endline comparison group differs from the baseline group on a few characteristics.

6

III. Results Classroom Literacy Environment

At baseline, the physical classroom environment varied greatly amongst the 20 classrooms. Fewer than half the schools visited had classrooms with permanent structures and 25% of the classrooms had temporary structures. The remaining had semi-permanents structures (Table 3). The classroom literacy environment was quite poor. The second grade classrooms were all devoid of materials with print; there were no materials on the walls and no books in the room with the exception of the textbooks that the students brought with them. In general, any printed materials that the school owned, including LB materials created by Save the Children, were found in the principals' offices. The end of the year found the classroom literacy environment to be improved in Literacy Boost schools. While comparison schools remained unchanged, 13 out of 16 LB schools had colourful flex sheets or paintings of objects, living things and the English and Nepali numbers and alphabet. A classroom before LB intervention (August/September 2009):

The same classroom after LB intervention (March/April 2010):

7

An unexpected consequence of this improved literacy environment is that class 2 students were promoted to better classrooms since hanging flex sheets and painting required walls (Table 3). This improvement in classroom structure does not indicate an improvement in school infrastructure but merely an exchange of students; another class was demoted to the less permanent structures. Out of the 3 LB schools that did not have flex sheets or paintings, 2 did not have walls and 1 intended to paint the walls during the upcoming summer holiday. Table 3: Physical structure of Class 2 rooms Structure of room Permanent structures Semi-permanent

structures Temporary structures

Description of structure

Cement floor and walls, and with tin roof

Dirt floor, tin roof. One school had incomplete wall

No walls or two or fewer thatch walls, dirt floor

Number of schools at baseline

9 (7 LB and 2 control)

6 (5 LB, 1 control)

5 (4 LB, 1 control)

Number of schools at end of year

15 (13LB and 2 control)

2 (1LB and 1 control)

3 (2 LB and 1 control)

Progress of students in matched student sample Data for a subset of 214 students were collected at baseline and at the end of the year, 177 for Literacy Boost schools and 37 from comparison schools. Fifty-eight students from the incomplete list of 272 names of students assessed at baseline, were absent. For most of these students, the reasons for their absences were unknown by the teachers and other students. Students supplied reasons for 11 absences: one student had died from an

8

illness, 2 children were taking care of younger siblings, 2 children did not want to come to school, 2 more did not want to participate in the study, 1 child was sick, 2 had changed grade level, and 1 student was absent because his parents were traveling to Bombay. Letter identification: Figure 1: Average number of letters identified, 2009 and 2010

Both Literacy Boost and comparison school students were able to identify significantly more letters, out of a total of 36 Nepali alphabets, at the end of the year compared to the number they could identify at baseline (p=0.000). When the gains were compared, LB students made a significantly larger

gain than the comparison school students (p=0.005). Because these groups were equivalent at baseline, we can conclude that Literacy Boost supported greater letter knowledge progress during the school year. Concepts about Print Figure 2: Average CAP score

When baseline results are compared to end of year results, both LB school students and comparison students made significant improvements in scores (p=0.000). LB students made a significantly larger gain compared to comparison school students (p= 0.022). Because these groups were

equivalent at baseline, we can conclude that Literacy Boost supported greater CAP progress during the school year. Reading Fluency Literacy Boost students made significantly larger gains in reading fluency, measured by the number of words read correctly in a minute, when compared to their comparison school counterparts (Figure 3).

19.4

28.7

9.2

17.422.3

4.9

05

101520253035

Baseline Final GainAve

rage

# o

f let

ters

iden

tifie

d

Literacy BoostComparison

5.8

8

2.2

5.4

6.6

1.2

0123456789

Baseline Final GainAve

rage

CA

P sc

ore

(out

of 1

0)

Literacy Boost

Comparison

9

Figure 3: Average WPM, 2009 and 2010 On average, all students made significant gains in reading fluency whether they received the LB intervention or not (p<0.05). LB students made a gain of 5.24 words more per minute than the gain made by comparison school students (p=0.002).

Because these groups were equivalent at baseline, we can conclude that Literacy Boost supported greater reading fluency progress during the school year. A breakdown of the number of words read per minute (Figure 4) shows that more than half of LB students cannot read a single word at the end of the year even though this number dropped from nearly 80% at the start of the year. Of the LB children who are still at zero words per minute at the end of the year, 57% were female and 76.2% were non Nepali-speakers. There was only a 10% decrease in the percent of students who cannot read a single word in the comparison group. At the other end of the spectrum, a few LB students could read more than 60 words per minute while none of the comparison students could. Since the majority of students are non-readers, the primary issue amongst the students in this study is not of reading speed but of learning to read. Figure 4: Breakdown of WPM, 2009 and 2010

0

10

20

30

40

50

60

70

80

90

0 10 or fewer 11 to 20 21 to 30 31 to 40 41 to 50 51 to 60 more than60

WPM

% o

f stu

dent

s

LB BaselineLB FinalComparison BaselineComparison Final

4.6

12.6

8.1

5.2

8

2.8

0

2

4

6

8

10

12

14

Baseline Final Gain

Ave

rage

WPM

Literacy BoostComparison

10

Reading Accuracy Figure 5: Average % of words correct, 2009 and 2010

All students improved their reading accuracy significantly over the course of the academic year as measured by the percent of words read correctly (p<0.05). The gain made by LB students was significantly greater than the gain made by comparison students (p=0.027). Because these

groups were equivalent at baseline, we can conclude that Literacy Boost supported greater reading accuracy progress during the school year.

16.4

37.4

21.1

14.5

24.4

9.9

05

10152025303540

Baseline Final Gain

Ave

rage

% o

f wor

ds c

orre

ct

Literacy BoostComparison

11

Numeracy Figure 6: Average numeracy scores, 2009 and 2010

Numeracy scores are recorded as percentages. Both groups made significant improvements in numeracy. The gain made by LB students is not significantly larger than the gain made by comparison school students but the difference in gains

approached significance (p=0.051). In all sections of the test, LB students and comparison students made statistically significant improvements from baseline to the end of the year. This indicates, as one would hope would be the case, that learning in the classrooms is occurring even without the intervention. However, the LB intervention appears to foster greater learning since the gains made by LB children were always significantly greater than the gains made by the comparison group with the exception of the numeracy section of the assessment. When Literacy Boost students are compared to their comparison school counterparts at baseline, the differences in their performance on the all assessment components are not significant (Table 4). This is to be expected since the comparison schools were chosen randomly from communities that were similarity to LB school communities. When the two groups are compared at the end of the year, the differences between Literacy Boost students and comparison school students are significant for letter identification, CAP, WPM, percent of words read correctly, and numeracy. Both groups improved on reading comprehension scores but the difference between the Literacy Boost and comparison groups at the final assessment was not significant.

45.8

60.8

15

39.548.5

9

010203040506070

Baseline Final Gain

Ave

rage

sco

re (%

)

Literacy BoostComparison

12

Table 4: Matched sample comparison between baseline and end of year on all assessments Assessment Time Literacy boost Comparison Difference Letter identification

Baseline Final

19.42 28.65

17.38 22.27

2.04 (p=0.443) 6.38* (p=0.010)

Gain 9.23*** 4.89*** 4.34* (p=0.005) CAP Baseline

Final 5.81 8.03

5.38 6.62

0.43 (p=0.309) 1.41** (p=0.001)

Gain 2.23*** 1.24*** 0.98* (p=0.022) WPM (Fluency) Baseline

Final 4.54 12.64

5.19 8.03

0.65 (p=0.760) 4.62 (p=0.171)

Gain 8.08*** 2.84* 3.97** (p=0.002) % Words read correctly

Baseline Final

16.38 37.43

14.48 24.38

1.90 (p=0.760) 13.05 (p=0.092)

Gain 21.05*** 9.90* 11.15* (p=0.027) Numeracy Baseline

Final 45.80 60.80

39.53 48.48

6.27 (p=0.129) 12.32* (p=0.010)

Gain 15.00*** 8.95** 6.05 (p=0.051) Reading comprehension

Baseline Final

0.51 2.67

0.49 2.65

0.03 (p=0.897) 0.02 (p=0.939)

* Significant difference at the p<0.05 level ** Significant difference at the p<0.005 level ***Significant at p=0.000 level Because these groups were equivalent at baseline, we can conclude that Literacy Boost supported greater learning during the school year. Tracking the Zeros There was a high percentage of 0 scores at baseline for letter identification, reading fluency and reading accuracy. Figure 7: A comparison of the number of zero scores between baseline and end of year

19.9

77.7 79.4

3

58.1 59.5

19.5

83.1 84.4

18.8

70 70

0

10

20

30

40

50

60

70

80

90

Letter ID WPM % of words readcorrectly

% o

f stu

dent

s sc

orin

g ze

ro

Literacy Boost BaselineLiteracy Boost FinalComparison group BaselineComparison group Final

13

When these percentages are compared to the percentages of zero scores found at the end of the year, in both LB and comparison groups, the drop in the percentage of zeros in the Literacy Boost group is always larger than the drop in the comparison group and is particularly notable for letter identification. This suggests that Literacy Boost may have had an especially strong impact on supporting the weakest students in the group. On the other end of the spectrum, there was a large rise in the percent of students with full scores on letter identification, CAP and percent of words read correctly (Figure 8). The rise in the percentage of full scores is greater in the Literacy Boost group than in the comparison group and, once again, letter identification scores show a large difference between the treatment groups. LB also appears to have had a strong impact on improving CAP and reading scores. Figure 8: A comparison of the number of full scores between baseline and end of year

15.5

4.12.7

28.426.4

9.8

13

1.30

15

8.8

2.5

0

5

10

15

20

25

30

Letter ID CAP % of words read correctly

% o

f stu

dent

s w

ith fu

ll sc

ores

Literacy Boost BaselineLiteracy Boost FinalComparison group BaselineComparison group Final

Progress of entire sample of students

The findings for the entire sample of students mirror those of the subset of 214 students for whom baseline and end of year data are matched. From baseline to the end of the year, the Literacy boost school children made significant improvements in all parts of the assessment (p=0.000): letter identification, CAP, fluency, accuracy and numeracy. The comparison school students made significant improvements in only letter identification (p=0.036) and CAP (p=0.002). When LB school students and comparison school students are compared at baseline the differences between them are not significant for all parts of the assessment. However, at the end of the year, the two groups differ significantly in letter identification, CAP, reading comprehension and numeracy scores. Taken together, these results indicate that the LB intervention had the greatest impact on numeracy scores, letter identification and CAP.

While the LB group made statistically significant gains over the year in reading fluency and accuracy and the comparison group did not, at the end of the year, the performance of these two groups did not differ significantly on these components indicating that the LB intervention did not have the desired impact on reading fluency

14

and accuracy (Table 5). LB programming may have had an impact on reading comprehension as well since the difference between the two groups at the end of the year was significant. However it was not possible to make within group comparisons across time because the questions were administered differently at baseline and at the end of the year. It is noteworthy that comparisons between groups in the matched sample indicate different results concerning reading accuracy and comprehension; in the matched sample, LB programming did appear to have an impact on reading accuracy and did not appear to have an impact on reading comprehension. Therefore it is possible that other variables, not related to treatment group, influenced the results for the whole sample, since the endline sample differed on some key characteristics from the baseline sample for the comparison group (Table 2). Table 5: Full sample comparison between baseline and end of year on all assessments Assessment Time Literacy boost Comparison Difference Letter identification

Baseline Final

19.85 28.38

16.26 21.09

3.59 (p=0.059) 7.29*** (p=0.000)

Difference 8.53*** 4.83* CAP Baseline

Final 5.87 7.86

5.31 6.60

0.55 (p=0.070) 1.26*** (p=0.000)

Difference 2.00*** 1.29** WPM (Fluency) Baseline

Final 4.89 11.34

4.27 7.79

0.62 (p=0.686) 3.55 (p=0.061)

Difference 6.45*** 3.51 % Words read correctly

Baseline Final

18.07 34.70

14.18 25.63

3.89 (p=0.395) 9.07 (p=0.088)

Difference 16.63*** 11.45 Numeracy Baseline

Final 47.24 58.54

42.86 48.13

4.38 (p=0.138) 10.41** (p=0.002)

Difference 11.30*** 5.27 Reading comprehension

Baseline Final

0.59 2.68

0.47 2.31

0.12 (p=0.436) 0.37* (p=0.023)

* Significant difference at the p<0.05 level ** Significant difference at the p<0.005 level ***Significant difference at p=0.000 level The Effect of Mother Tongue on Literacy and Numeracy Assessments Since the majority of the communities in the area in which LB is implemented are Tharu-speaking, and the language of instruction in schools is Nepali, the literacy and numeracy scores in the full sample were compared between students whose mother tongue was Nepali and students whose mother tongue was not Nepali. Data from the LB school and control school samples were combined for this comparison because the Nepali-speaking sample in the comparison group was too small to conduct a separate analysis. Table 6: Effect of Spoken language on Literacy/Numeracy assessments in the full sample

Baseline End of Year

15

Assessment Nepali-speakers

Average score (N=85)

Non Nepali-speakers

Average score (N=282)

Nepali-speakers

Average score (N=95)

Non Nepali-speakers

Average score (N=275)

Letter identification /36

23.34** (SD=14.30)

17.76 (SD=14.74)

29.78** (SD=8.95)

25.83 (SD=12.05)

Total score for CAP /10

6.12 (SD=2.41)

5.64 (SD=2.37)

7.76 (SD=2.33)

7.53 (SD=2.36)

Words per minute

7.66* (SD=16.19)

3.90 (SD=10.15)

15.83** (SD=20.36)

8.85 (SD= 16.03)

Percentage of words correct

23.45 (SD=40.79)

15.45 (SD=33.79)

43.59* (SD=46.84)

29.38 (SD=42.55)

Reading comprehension

0.79 (SD=1.45)

0.50 (SD=1.16)

3.11** (SD=1.09)

2.44 (SD=1.33)

Numeracy score

46.55 (SD=21.84)

46.82 (SD=24.04)

56.25 (SD=27.17)

56.39 (SD=26.36)

*Significantly higher than non Nepali-speakers at p<0.05 level **Significantly higher than non Nepali speakers at p<0.005 level At baseline and at the end of the year, the Nepali-speaking students did significantly better than their non Nepali-speaking counterparts on letter identification and when comparing the number of words read correctly in one minute (Table 6). The CAP scores were not significantly different between the two groups. This could be because a child's knowledge of concepts about print may be more influenced by experience with printed materials than by the language spoken at home. Similarly, numeracy scores were not influenced by the mother tongue of the students since the test was administered in the language of preference of the child.

Nepali-speakers could read 8 percent more of the words correctly compared to non-Nepali speakers at baseline, but the variation in both language groups was high as evidenced by the large standard deviations, and the difference was not statistically significant. However, the difference in percent of words read correctly between the two groups was significant at the end of the year with a 13% difference in groups, as was the difference in reading comprehension scores. The gaps in scores between the two groups for reading fluency, accuracy and comprehension appear to have widened over the course of the year suggesting that Nepali-speaking students made greater gains than their non Nepali-speaking counterparts. The effect of LB in mitigating this difference is unclear since, as stated earlier, the number of Nepali speakers in the control group was too small to allow for a separate analysis. In addition, the analysis may be further complicated by the apparent replacement of Tharu-speakers with Nepali speakers in the comparison group at the end of the year (Table 2).

When the endline scores of non Nepali-speakers are compared between the LB group and the comparison group, the students in the LB group do significantly better at letter identification (p=0.000), CAP (p=0.000), and numeracy assessments (p=0.001). These results suggest that LB supports second-language learners in these areas and attention needs to be intensified to support second-language learners in reading fluency,

16

accuracy and comprehension. Female LB non-Nepali speakers do better than their comparison school counterparts in letter identification (p=0.005) and numeracy (p=0.040). Male LB non Nepali-speakers, on the other hand, do better in all three areas: letter identification (p=0.008), CAP (p=0.000) and numeracy (p=0.004). LB appears to support male non Nepali-speaking students more than female students in CAP.

In the matched sample (Figure 9), Nepali-speaking students performed significantly better at letter identification (p=0.000 at baseline and end of year), words read correctly per minute (p=0.020 at baseline and p=0.001 at end of year), percentage of words read correctly (p=0.018 at baseline and p=0.003 at end of year), and reading comprehension (p=0.009, p=0.000). Again, differences in CAP and numeracy scores were not significant. Matched sample gains were not significantly different with the exception of reading fluency gains which were higher for Nepali-speakers (p=0.002). Figure 9: Matched sample comparisons Between Nepali-speakers and non Nepali-speakers

25.5

31.9

6.4

16.9

26.1

9.2

0

5

10

15

20

25

30

35

Baseline Final Gain

Ave

num

ber o

f let

ters

iden

tifie

d (o

ut o

f 36)

Nepali-speakers

Non Nepali-speakers

6.2

8.2

2

5.6

7.7

2.1

0

1

2

3

4

5

6

7

8

9

Baseline Final Gain

Ave

CA

P sc

ore

(out

of 1

0)

Nepali-speakers

Non Nepali-speakers

17

9

21

12

3.3

95.7

0

5

10

15

20

25

Baseline Final Gain

Aver

age

WPM

Nepali-speakers

Non Nepali-speakers

28.1

52.3

24.2

12.4

30

17.6

0

10

20

30

40

50

60

Baseline Final GainAve

% o

f wor

ds re

ad c

orre

ctly Nepali-speakers

Non Nepali-speakers

1

3.3

2.3

0.4

2.5

2.1

0

0.5

1

1.5

2

2.5

3

3.5

Baseline Final Gain

Ave

. num

ber o

f que

stio

ns

corr

ect (

out o

f 4)

Nepali-speakers

Non Nepali-speakers

47.1

62.5

15.4

43.9

57.2

13.3

0

10

20

30

40

50

60

70

Baseline Final Gain

Ave

. num

erac

y sc

ore

(%)

Nepali-speakersNon Nepali-speakers

The Effect of Gender on Literacy and Numeracy Assessments

18

The differences in scores between male and female students on all components of the reading assessment were not significant. However, male students performed better than female students on the numeracy assessment at both baseline and final assessments (Table 7). Data from Literacy Boost schools and comparison schools were analyzed together in the matched sample because the comparison school sample of 18 females and 19 males was too small to analyze separately. Table 7: Numeracy Assessment scores of matched sample Average Male score (%) Average Female score (%) Difference Baseline 50.04 (N=103) 39.78 (N=111) 10.26 (p=0.001) Final 65.04 (N=103) 52.67 (N=110) 12.38 (p=0.001) Gain 15 12.89 2.11

Both male and female students in the matched sample made similar gains over the year: male scores improved by 15 percentage points and female scores improved by about 13 percentage points. The difference in the gains made by males and females is not significant. In the full sample as well, LB male students outperformed female students at both the baseline assessment and at the end of the year (Table 8). The differences between male and female students in the comparison group were not significant at both times. Table 8: Numeracy Assessment scores of Entire sample Average Male score (%) Average Female score (%) Difference Baseline LB

55.06 (N=142)

40.93 (N=149)

14.13*

Baseline Comparison

40.88 (N=37)

44.69 (N=40)

3.81

Final LB 65.65 (N=139)

52.30 (N=155)

13.35*

Final Comparison

52.53 (N=42)

43.26 (N=38)

9.27

*Significant p=0.000 While male students scored significantly higher than female students on the numeracy assessment, it is encouraging that both males and females made similar gains in numeracy over the course of the year and that males and females do not differ significantly on any other part of the literacy assessment.

19

Student Participation in Literacy Boost programming At the end of the year all students were asked about their favourite story and whether they had a reading buddy, or 'study friend' (Figure 10). In addition, LB children were asked how often they attended reading camps. Figure 10: Percent of students who have a favourite story and reading buddy

More children in the LB group were able to name a favourite story than children in the comparison group (p=0.000). A quarter of the LB children had heard their favourite story at

reading camps, suggesting that reading camps play a role in exposing children to reading material. More male students could name a favourite story when compared to female students (p=0.008) and this difference was significant in the LB group (p=0.013) and close to significant in the comparison group (p=0.063). This difference between genders does not appear to be related to reading camp attendance because more females in the LB group claim to attend the reading camps than male students (p=0.059).

Reading camps are well-attended; when LB children were asked how often they attended reading camps, 65% of them said that they "always" attended the camps and 14% said "never". In addition, while reading camps were slated as biweekly events, facilitators and children reported that the camps were held weekly. Significantly more LB children had a reading buddy compared to their comparison school counterparts (p=0.002). 39% of LB children who claimed to have a reading buddy said that their teacher formed their reading buddy compared to 3% of comparison school students. Again this suggests that LB programming might have something to do with the difference between treatment groups.

Similar findings were found in the matched sample in which LB students were more likely to be able to name a favourite story than their comparison school counterparts (p=0.000) and male students were more able to articulate their favourite story compared to female students (p=0.018). This finding is interesting in light of the fact that the difference between the number of male and female students attending reading camps “always” is not significant. Males could be using the Book Banks more or could feel more empowered to express their opinions than females. The issue warrants further study to determine the extent to which Literacy Boost is effective in reaching out to all children. In the matched sample, significantly more Nepali-speaking students claimed to have a reading buddy than their non Nepali-speaking counterparts (p=0.001) but this difference was not found in the full sample.

54.4

91.7

19.5

72.1

0

20

40

60

80

100

Favourite story Reading Buddy

% o

f stu

dent

s w

ho h

ave

one

Literacy BoostComparison School

20

Household Literacy Environment and Parent Surveys

Household literacy environment and parent surveys were conducted to examine the factors in a child's home environment that could have an effect on literacy skills. Only 71 out of about 200 households visited in total (Table 1) were common between baseline and the end of the year: 68 Literacy Boost households and 3 comparison households. At baseline and at the end of the year an effort was made to speak to a literate member of each household who was as close as possible to the head of household if not the head. If a literate member was not available, an illiterate member who had a senior and/or central position in the household was interviewed instead. However, it was not always possible to interview the same person both times reducing the comparability of the data. The same person was interviewed at baseline and at the end of the year in 30 out of the 71 households visited at both times: 29 Literacy Boost and 1 comparison household. In addition, because the sample of comparison households is so small, it is not possible to determine the impact of the Literacy Boost intervention on the home literacy environment over the course of the year during which the intervention took place. As a consequence, the focus of this analysis will be on comparing LB households (N=157 in 2010) to comparison group households (N=42 in 2010), and on relating household factors to literacy outcomes within the LB group. The respondent in each household was asked to answer questions about their reading and writing attitudes and habits, and their literacy-related interactions with the study child. In addition, respondents were asked to report the number of reading materials and posters/calendars with print. Enumerators then surveyed the house and counted all items with print including consumption goods and litter. CAP scores (out of 10) were often used as a measure for literacy because they showed the most variation in scores but differences between other indicators were also explored. WPM and percent of words read correctly had a large proportion of zero scores. Letter identification was truncated in the other direction in 2010 with about a third scoring 100%. Who are the respondents? In both groups, the majority of respondents, around 40%, were mothers, followed by around a quarter who were fathers (Table 9). Table 9: Relationship of respondent to study child Relationship to child LB group N=157 (%) Comparison N=42 (%) Mother 40.1 42.9 Father 24.2 28.6 Aunt/Uncle 8.3 9.5 Sibling 15.9 7.1 Grandparent 6.4 11.9 Sister-in-law 5.1 0 Parent attitudes and perceptions of literacy

21

Parent attitudes towards reading were not found to be significantly different between the two treatment groups at the end of the year. Parents were read a series of statements to which they had to agree or disagree. The differences between the percent of parents who agreed to the statements between the Literacy Boost and the comparison groups were not significant with the exception of one statement (Table 10). Significantly more comparison group parents agreed that "Knowing how to read is important for getting a good job one day" (p=0.000). In general, parents agreed on the importance of knowing how to read, on exposing children to books at a young age and on the importance of parental involvement in teaching children how to read. However, about 70% of parents thought that it is not worth teaching some children how to read even though more than 80% thought that every child should learn to read. Close to 60% of parents thought that the teacher is the only person responsible for teaching children how to read which is inconsistent with the 90% who agreed that parents and teachers should work together to teach reading. Enumerators reported that illiterate parents had great difficulty understanding the statements. They also observed that illiterate parents tended to simply agree to every statement. If this is indeed the case, it may explain some of the inconsistencies in the results.

When parents were asked what ages they thought children should be able to read and write by themselves, again the differences in responses between the treatment group did not differ significantly. The average age stated by LB parents was 7 years for reading and writing whereas the comparison group parents stated an average age of 6.5 years. Table 10: Percent of LB and Comparison group parents who agree to attitudinal statements in 2010

Statement LB % who

agree N=157

Comparison % who agree

N=42 The teacher is the only person responsible for teaching children how to read. 59 57

I do not know how to help my child learn to read. 60 57 It is important for a child to be exposed to books and other writing from a young age. 96 95

Every child should learn how to read. 81 88

Parents and teachers should work together to teach reading. 88 90

It is difficult for someone like me to help my child learn to read. 59 71

It is not worth teaching some children to learn how to read. 73 69

Parents should be involved in teaching their children how to read. 97 98

There are many benefits to knowing how to read. 98 100

I feel confident I can help my child learn to read. 95 98

Children should learn to read from their older siblings or cousins. 97 98

(Study child) likes to read. 90 93

22

Knowing how to read is necessary for getting a good job one day. 90 100*

*Significant difference p=0.000 Is there a relationship between parent attitudes and literacy? For the four indicators where respondents were split in their responses, with fewer than 80% agreeing, literacy scores were compared between those in the LB group who agreed to the statements and those who disagreed. Two indicators were found to have a significant impact on reading scores. The children of respondents who agreed that the teacher is the only person responsible for teaching children how to read had significantly lower CAP scores than the children of respondents who disagreed to the statement (p=0.039). Presumably respondents who agreed that the teacher is responsible were less involved with helping their children learn to read. Respondents who agreed that it was difficult for someone like them to help their child learn to read had children with lower CAP scores than those who disagreed and this difference approached significance (p=0.053). Children of respondents who agreed with a related statement, however, “I don’t know how to help my child learn to read”, had significantly lower literacy scores than the children of respondents who did not agree (p=0.017). These results reinforce the importance of parental involvement in fostering literacy skills. Future Literacy Boost programming can educate parents on the ways in which they can support their children's language development even if they are not literate themselves. Levels of Literacy in the home and reading habits At the end of the year, 91 respondents, or 58% of the LB treatment group claimed to be able to read a simple sentence, and one fewer, 57% could write a simple sentence. In the comparison group, 20 respondents, or 48% of the comparison group claimed to be able to read and write a simple sentence. Table 11: What do respondents read? 2010 Item Literacy boost N=90 (%) Comparison N=19 (%) Learner's books/Textbooks 58.9 47.4 Religious books 12.2 5.3 Non-religious books 17.8 5.3 Newspapers/Magazines/Pamphlets 17.7 21.1 Nothing 17.8 26.3 Other 2.2 5.3 Respondents who claimed they could read a simple sentence were asked what they read (Table 11). The small size of the comparison group in particular means that the percentages cannot be generalized to the larger population. Respondents could state more than one item. Most of the respondents claimed to read learners books in both groups. These respondents could either be reading textbooks to help their children or they could be reading for their own education; elder sibling respondents were enrolled in school or college and some adults were participants in adult education classes. Other

23

responses included: the Nepali alphabet, "things written everywhere" and writing on consumption goods. Table 12: Why do you read? 2010 Literacy boost N=65 (%) To learn/for education 78.5 To help my child 20 For fun/to pass time 6.2 Religious education 4.6 To be updated on the news 3.1 Respondents could give more than one reason when asked why they read. There were only 10 respondents in the comparison group, 6 of whom said they said they read to learn and 1 each read for the other reasons. The majority of all respondents said that they read to learn suggesting that a high value is placed on education. A fifth of the respondents said they read to help their children. Is there a relationship between literacy habits of respondents and child literacy levels? Respondents who read learners books had children who scored significantly higher on CAP than respondents who do not read learners books (p=0.015 in combined sample, p=0.050 in LB sample). Respondents could be reading learners books in order to help their children, or to improve their own education. The latter case demonstrates a high value placed on education that would indirectly help improve their children's education. Even though only a fifth of the respondents said that the reason why they read is to help their children, the children of these respondents performed better on CAP than children of other respondents (p=0.034 in combined sample, p=0.037 in LB sample). There was no difference between the CAP scores of children of respondents who read to learn and those who read for other reasons, and between respondents who were literate or not. These findings demonstrate that reading to help children and parental involvement can foster literacy skills. Future Literacy Boost programs can encourage parents to find ways to have their children read to even if they are not literate themselves, including using reading buddies, arranging for the help of other literate family members, and sending their children to reading camps. Parent Interactions with the Child and Literacy Literacy Boost group respondents did not tell stories, help with homework or read to their children any more than comparison group parents. Data were not collected on parent LB workshop attendance but future research could explore the relationship between workshop attendance and parent interactions with children and literacy. About two thirds of the respondents interacted with their children in these ways (Table 13). Only 58% of LB respondents claimed to be able to read a simple sentence and 48% of comparison group parents. However, a higher percentage of respondents in both groups said that they read to their children calling into question the reliability of this self-reported data. One explanation could be that respondents have a different definition of what it means to read,

24

and behaviours such as reading the alphabet could be viewed as reading. Another is that respondents believed that they should be reading and the results are exaggerated. The discrepancy is greater in the Literacy Boost group in which respondents may have attended workshops in which the importance of reading was explained. Table 13: LB and Comparison group respondents interaction with child and Literacy in 2010 Indicator LB % of

respondents Comparison

% of respondents

Tell stories to study child 66 67 Help study child with homework 60 55 Read to study child 67 59 When asked how they helped their child learn how to read or write, more than half of the parents said that they purchase school materials for their children (Table 14). About one fifth read to their children. A greater proportion of respondents in the Literacy Boost group claimed to help their children with homework; otherwise, the groups were similar with respect to how they assisted children in learning how to read and write. Other responses included telling stories, sending children to tuition classes, teaching maths, giving money, and creating a learning environment in the house. Data were not collected on whether parents attended Literacy Boost workshops so it is not clear how the workshops may have impacted the ways in which respondents assisted their children. Table 14: How do respondents assist children in learning how to read or write? 2010 Literacy boost %

of parents (N=94) Comparison group % of parents (N=19)

Help with homework 42.6 10.6 Read to child 23.5 20.1 Make child study, read and write 15 15.8 Purchase school materials 51.2 52.7 Create learning materials 6.4 10.5 Use alphabet cards/flash cards 4.3 5.3 Play learning games 1.1 0 Send the child to school 4.3 0 Nothing 2.1 5.3 Other 12.1 0 A minority of parents claimed to know what was going on in their children's classrooms: 17% of parents in the LB group and 14% of parents in the comparison group. A significant difference between treatment groups was noted when parents were asked if they had seen picture books for children in Nepali and Tharu (Table 15). Table 15: LB and Comparison group parents exposure to childrens' books in 2010 Indicator LB % of

parents Comparison % of parents

Seen picture book for children in Nepali 67 50* Seen picture book for children in Tharu 54 15** * Significant p=0.050

25

** Significant p=0.000 These differences could be a result of the use of LB Book Banks. The enumerators noticed a number of LB Book Bank materials in homes of students in the treatment group. In particular, the only Tharu children's materials found in homes were created for the Literacy Boost Book Banks. Respondents were not asked if they or their children preferred to read in Tharu or Nepali and this could be an area for future research. Is there a relationship between interactions with literacy in the home and children’s literacy? Respondents who claimed to assist their children in learning how to read and write by helping with homework (Table 14) had children with significantly higher CAP scores than respondents who did not (p=0.035 in combined sample). A greater proportion of Literacy Boost parents helped their children with homework, so it is possible that the Literacy Boost intervention helped to foster this behaviour; however, differences in literacy were not found in the LB sample alone. Telling stories did not result in significantly higher literacy scores than not doing so but the difference in literacy scores approached significance in the LB group (p=0.066). Reading to the child did not result in significantly higher literacy scores and neither did responses to a direct question on whether respondents helped their child with homework (Table 13). As suggested earlier, the data on the number of respondents who read to their children may not be accurate. Respondents who had seen children's picture books in Tharu had children with higher CAP scores than respondents who had not (p=0.050 in combined sample, p=0.034 in LB sample). The difference in CAP scores between children of respondents who had seen Nepali children's picture books and those who had not approached significance in the combined sample (p=0.060) but was significant in the LB sample (p=0.034). Exposure to children's books can indicate a greater use of the LB Book Banks. The Book Bank component of the Literacy Boost intervention appears to play an important role in making reading materials available, especially reading materials written in Tharu. Is there a relationship between reading camp attendance, use of reading buddies and literacy? Children in the LB group who claimed to attend reading camps "Always", had significantly higher reading comprehension scores that those who did not attend regularly or at all (p=0.009). Having a reading buddy also resulted in better reading comprehension scores but the difference only approached significance in the combined sample (p=0.064). There was no significant difference in scores for the other literacy measures suggesting that reading camps had the greatest impact on reading comprehension. For children without literate parents, it may be the only chance they get to listen to stories read to them. Children were asked to list the activities they did at the reading camps: reading, listening to stories and poems, singing songs and dancing, playing games, drawing, studying and working with numbers. When

26

asked to pick their favourite activity, 70.5% of the responses included reading and listening to stories. Reading camps can help to compensate for a lack of literate household members by fostering reading comprehension and creating a culture of literacy for children. Household Items with Print A wide variety of items were found in homes with print. The appendix has a full listing of items observed in the homes that were visited. These items are in addition to the books, newspapers, magazines, papers, posters, consumption goods, litter and other items with print. The parent who was being interviewed was asked to report the number of items with print in the house and then an enumerator counted items with print and categorized them by language: Nepali, Tharu, English and Hindi. English items were noted because it is a language taught in schools, and Hindi items were noted since Hindi uses the same script as Nepali and Tharu (Devnagri), and it is a closely related language, particularly to Tharu. In addition, one household had Braille books, and one or two items were found with Arabic and Korean writing. The only categories that are significantly different between the treatment group and comparison group are the number of reported materials with Tharu print (p=0.000) and the number of observed reading materials in Tharu (p=0.012). This finding supports the suggestion made earlier that the Literacy Boost intervention has made a significant contribution in making materials available in Tharu when almost none existed before. Table 16: Number of books, newspapers, magazines, posters reported by families 2010 LB average number of

items N=157

Comparison average number of items N=42

Reported items in Nepali 26.12 (SD=37.5) 22.98 (SD=21.6) Reported items in Tharu 3.32 (SD=7.2) 0.88* (SD=2.2) Reported items in English 5.46 (SD=6.4) 5.02 (SD=5.4) Reported items in Hindi 0.89 (SD=2.5) 0.81 (SD=2.0) *Significant p=0.000 The majority of materials that were reported by families were in Nepali and, in comparison, very few items were reported to be in Tharu (Table 16). There was a lot of variation between homes as evidenced by the high standard deviations. The number of books were not found to be significantly correlated with literacy outcomes The effect of L2-L1 classes on Adult Literacy

L2-L1 classes were designed as two-week courses to teach Tharu adults to be literate in Tharu (L1) if they are already literate in Nepali (L2) because both Nepali and Tharu use the same script: Devnagri. To assess how effective these courses are in teaching Tharu literacy skills, adults at an L2-L1 center were asked to read a short story in Tharu and answer comprehension questions about it. Due to time constraints the research team was only able to go to one adult center at which there were about 30

27

members of the class who were overwhelmingly women. The team was only able to get informed consent from 15 women; most of the women were hesitant to speak to us and many said that there was no point in doing so because they knew nothing. The results are therefore not generalizable.

The women ranged in age from 18 to 36 with an average age of 27.9. The L2-L1 classes at this center had evolved into adult literacy classes: for 8 out of the 15 women, L2-L1 classes were their first educational experience, and the number of sessions they had attended far exceeded the two-week cycle. The women had attended between 12 to 30 sessions with an average of 21.7 sessions. Of those who had received some education prior to the L2-L1 classes, 5 had attended adult education courses, one had completed class 4 and another had completed class 5.

The results ranged from the 6 women who could not read a word to the 8 who could read the entire passage. Of the 8 who could read the entire passage, 5 were educated solely by L2-L1 classes. The average number of reading comprehension questions answered correctly was 1.3 out of 4 for the 11 women who answered reading comprehension questions.

Six of the women who were assessed were also interviewed briefly about their experience in the L2-L1 classes. All six women said that the L2-L1 classes could be improved by being run regularly and/or that adult classes should be started. The women said they attended the classes to educate themselves so that they could educate their children and pay attention to their children's hygiene. The women claimed to teach what they learn in L2-L1 classes to their children and other family members at home and to their neighbours.

While the results are preliminary and not generalizable, the findings are revealing. Literacy Boost had the unintended impact of promoting Tharu literacy amongst women who had never attended school. In this particular community, the participants desired something different from what the program offered, and the program was flexible enough to morph to meet their needs. The sample is not representative of the class but the classes appear to have some impact on literacy since a third of those assessed could read the passage after only attending L2-L1 classes. A challenge with assessing adults is that adults are more likely to opt out of participating in a study than children, making it difficult to obtain a representative sample. This effect was confounded by the fact that the women had never interacted with anybody outside of their own village community before and so some were even shy of the Tharu enumerators.

Teacher views on Literacy boost trainings

Teachers in LB schools were interviewed for their impressions about the LB trainings and the impact of these trainings on their teaching practice. The research team tried to interview grade 2 teachers since they have a direct link with the children who were assessed but, when this was not possible, another teacher who had attended the trainings was interviewed. Five out of the nine teachers interviewed taught grade 2 at least for one subject. The sample of 9 teachers who were interviewed is not representative but can indicate areas of program success and areas for program improvement.

28

Only one teacher, who taught ECD, grade 6 and grade 7, had attended all 5 teacher training sessions. All teachers claimed to implement lessons learned from the training sessions in their classrooms. In general, there was great enthusiasm for the effect of using lessons from the training sessions on the students. Teachers reported that attendance had improved, students had developed the habit of doing their homework and reading, are more attentive and interested in studying and have begun to pay more attention to their personal hygiene. Teachers also allude to an increased sense of empowerment in children: "they can now talk about their homes and tell stories" and they know to ask their reading buddies for help. One teacher said that student oral skills develop quickly but not their writing skills. Three of the teachers said that the outcomes were mediocre and that students do not apply what they have learned.

It appears as though teachers have not yet completely changed their approach to teaching. The few classes that were observed in progress revealed rote-like methods. And when one teacher was asked what he thought the outcomes were of using this approach, he said, "students can now tell their names by using letters" referring to a specific activity he had described having implemented earlier in the interview. However, with only a few training sessions, it is unrealistic to expect a complete change of approach and it is promising that teachers are implementing activities and seeing positive outcomes as a result. Teachers were asked to state challenges they experienced in implementing the LB approach. The teachers mentioned challenges that are universally experienced by teachers everywhere: constraints of time, and the difficulty of differentiating teaching methods to cater to the needs of all students: "The weak students tell me to repeat the things that I teach them whereas the good students tell me not to repeat the same things. And the weak students fall behind and the good students always go ahead". The student assessments revealed that the weak students are often the Tharu-speaking students, and in the case of numeracy skills, the female students. Given that these are the students that the Literacy boost intervention hopes to have on impact on, teacher training that focuses on inclusive participatory methods are especially important.

Other challenges were more specific to the local context: student absenteeism, students coming unprepared without notebooks and pencils, a lack of teaching materials, no separate classroom for students, frequent school closings due to political problems, the large number of students who "make a loud noise", and the use of a carpet in class results in students being "mischievous" as they "sit together and just play" while classes are going on. The latter challenges faced by teachers speak to the need for teacher training sessions to address classroom management techniques and teacher views on what is good teaching. Participatory approaches to teaching involve movement and more noise on the part of students and, especially when class sizes are very large, this raises challenges for classroom management, and can contradict traditional views that might be espoused by teachers that good teachers have silent classrooms.

Not surprisingly, given the positive perception of the classes, many teachers asked for further, regular training sessions. Teachers want to "learn techniques" and show an interest in collaboration: "there must be coordination among teachers" and "it would be better if the other subject teachers were invited to attend the trainings." Teachers also asked to be supported with teaching materials such as colourful paper and glue. On the other hand, a principal of one LB school showed trunks of untouched books and teaching aids that were going unused because she claimed teachers did not know how to use them

29

and apply them to the curriculum. Since teachers are requesting teaching materials while materials are going unused, there appears to be a disconnect between what is needed and what is provided. Perhaps part of the challenge for future programming is to seek teacher input on what materials will be the most useful, and to show teachers how they can make use of the materials that are provided, especially since materials are never going to be enough for every child given the large class sizes. Teachers attribute improved learning, greater enthusiasm for education and reading, better attendance and attention to personal hygiene, and possibly a sense of empowerment in their students to the activities they have learned from the LB teacher training modules. Participatory methods of teaching lend themselves to being inclusive of marginalized students, and future LB programming can seek to provide more ongoing support and guidance to teachers in their efforts to implement more participatory teaching methods. Discussion

The Literacy Boost pilot project had an impact on fostering literacy and creating a culture of reading and learning in Kailali, Nepal in the academic year, 2009-2010. Over a time period of six months both groups of students made significant gains in literacy demonstrating that learning in the classrooms is occurring even without the intervention. However, LB students made significantly larger gains in all aspects of the literacy assessment than their comparison school counterparts (p<0.05) with the exception of the numeracy component for which the difference in gains was close to significant (p=0.051). Because these groups were equivalent at baseline, we can conclude that Literacy Boost is supporting greater progress in literacy acquisition during the school year. There was a large percentage of zero scores at baseline for letter identification, WPM and percent of words read correctly. At the end of the year, the drop in zero scores in the Literacy Boost group is always larger than the drop in the comparison group and is particularly notable for letter identification. This suggests that Literacy Boost may have had an especially strong impact on supporting the weakest students in the group. Nepali-speaking students did significantly better than their non Nepali-speaking counterparts on letter identification, WPM, percent of words read correctly and reading comprehension. The CAP scores were not significantly different between the two groups. This could be because a child's knowledge of concepts about print may be more influenced by experience with printed materials than by the language spoken at home. Similarly, numeracy scores were not influenced by the mother tongue of the students since the test was administered in the language of preference of the child. Male students performed better than female students on the numeracy assessment at both baseline and final assessments but the differences between males and females were not significant for the other components of the assessment.

Reading camps are well-attended; when LB children were asked how often they attended reading camps, 65% of them said that they "always" attended the camps and only 14% said "never". In fact, reading camps were found to be held weekly instead of biweekly as originally intended. More children in the LB group were able to name a favourite story than children in the comparison group (p=0.000). A quarter of the LB children had heard their favourite story at reading camps, suggesting that reading camps

30

play a role in exposing children to reading and stories. 70.5% of the children who attended the reading camps said that reading and listening to stories were their favourite activities. Children in the LB group who claimed to attend reading camps "Always", had significantly higher reading comprehension scores that those who did not attend regularly or at all (p=0.009). Having a reading buddy also resulted in better reading comprehension scores but the difference only approached significance in the combined sample (p=0.064). Literacy Boost has had a positive impact on the classroom literacy environment. All classrooms were devoid of print at the baseline. By the end of the year the classroom literacy environment had improved greatly in Literacy Boost schools. While comparison schools remained unchanged, 13 out of 16 LB schools had colourful flex sheets or paintings of objects, living things and the English and Nepali numbers and alphabet. An unintended benefit of sponsoring painting or the use of flex sheets is that class 2 students were moved into permanent classrooms with walls if these were available. Teachers responded positively to the impact their LB training inspired lessons were having on children. In addition to improving learning, teachers stated that children were more enthusiastic about learning, had developed an interest in learning, attendance had improved, and children were now paying more attention to their personal hygiene. Most teachers who were interviewed requested further and ongoing teacher training. In addition to having a positive impact on the learning of children, Literacy Boost may have had the unintended impact of promoting Tharu literacy among women who have never been to school. L2-L1 classes had morphed into adult literacy classes at the one center that was visited. While these results are not representative, it points to an interesting focus for future research and programming. Literacy habits and attitudes of household members, and the ways in which they interacted with the child and literacy, were related to child literacy outcomes. About half of the household members who were interviewed could read a simple sentence: 58% of the LB group and 48% of the comparison group. Respondents who agreed that they did not know how to help their child learn to read and that the teacher is the only person responsible for teaching children how to read had children with lower CAP scores than respondents who disagreed with those statements. Reading learners books, saying that they read in order to help their children, and claiming to assist their children in learning how to read and write by helping with homework, had a positive impact on child literacy, demonstrating the importance of parent involvement in fostering literacy acquisition. In addition, respondents who had seen children's picture books in Tharu or Nepali had children with higher CAP scores than the children of respondents who had not. Literacy Boost Book Banks seem to have provided Tharu reading materials where none existed before, because LB group homes had significantly more reading materials in Tharu than comparison group homes at the end of the year while this difference between groups was not found at baseline. These findings taken together, suggest that Literacy Boost has had a positive impact on fostering a culture of literacy in the area of program implementation. This impact is especially significant given the short, six month, period between baseline and end of year data collection. Children appear to be reading more and enjoying reading, and those involved in the Literacy Boost program are more skilled at reading compared to their comparison school counterparts. Teachers are beginning to change their approach

31

to teaching literacy and have noticed positive outcomes as a result, and Literacy Boost Book Banks have played an important role in providing reading materials in Tharu. In addition to the areas of program success, this pilot study revealed some areas for program modification and future research: 1. The teacher training component was well-received and teachers have noted a positive impact of their changing approach to teaching literacy on their students. However, nearly all teachers requested additional and ongoing training and the one principal who spoke to the research team showed three trunks of materials that were going unused because teachers did not know how to use them. Given the importance of the role of the teacher in literacy acquisition, and the potential for more participatory methods to be inclusive of all children regardless of community background and gender, it is worth exploring the possibility of expanding on the teacher training component of LB. In particular, based on their feedback, teachers could use guidance on how to manage large class sizes during activities and how to integrate learning materials into their curriculum. 2. Tharu-speakers lag behind their Nepali-speaking counterparts in all components of literacy except for CAP and numeracy, and female students have lower numeracy scores than males. Further thought needs to be given on how to support these groups in particular and this may be an additional focus of teacher training. 3. Regarding L2-L1 classes, further research is required to determine if the program is meeting the needs of the participants. Only one center was visited and, in this area, most participants had not attended school and were using the L2-L1 classes to become literate. Those who spoke to the research team all an adult literacy class that occurred regularly. 4. Given the findings that parental involvement helps to foster literacy skills, more research needs to be done on the effect of LB parent workshops on changing attitudes and literacy behaviours of parents. Parents who are not literate need to be taught ways in which they can help their children's literacy skill development beyond purchasing school materials. Acknowledgements The authors would like to gratefully acknowledge the work of the enumerators who worked tirelessly and cheerfully to collect all of the data in 2009 and 2010: Kamal Bdr Saud, Laik Ram Kushmi, Jagadish Rana, Anita Kathariya Chaudhary, Anita Chaudhary, Sunita Chaudhary, Shikha Chaudhary, Manika Chaudhary. In addition, the Save the Children Dhangadi office staff members were unflinching in their support and worked behind the scenes to ensure that data collection and entry went smoothly. Ishwor Khatry coordinated support for the research efforts, Jyoti Thapa organized all travel arrangements for field visits and remedied every technology hitch, and Arun Chaudhary acted as enumerator, guide and host as needed. In the Save the Children Kathmandu office, Deergha Shrestha coordinated support for the research project and facilitated field

32

visits, and Sanjana Shrestha helped with translating and initiating data collection in 2009, and featured the LB pilot project in the SC newsletter in 2010. Dharma Rana of BASE Kailali, was invaluable in facilitating logistics involved with school and household visits, and he was assisted by Krishna Rana. On the other side of the ocean, the authors would like to thank Elliott Friedlander for sharing his experience and knowledge about the LB program in Kailali, Karen Wiener for developing the household data collection tools that were adapted for the Nepal context, and Andre de Souza for sharing his expertise and offering advice on data analysis.

33

Appendix: Items found in homes Household items Water filter bucket Plastic bag Sacks Tin drum Gallon/Jerry can Welcome sign Bag Suitcase Bottle Cardboard box/ Carton Padlock Lantern Jug Blade Mat Hand fan (bena) Hand pump Thermos Granary (bhakari) Utensil Electrical items Meter box Multiplug Electrical Fan Speakers Light switch board Battery/cells Clock Radio Tape player CD player DVD player Cassettes/CDs TV Remote Mobile Torch Calculator Adaptor Mobile charger Emergency light Booster Stabilizer

Iron Phone set Bulb/tubelight Building materials Tin sheets of roof Cement bag Plywood School materials School bag Notebooks Geometry box Ruler Pens/Pencils Means of transportation Bicycle Motorbike Wheels on bullock cart Tractor Trolley Farm equipment/materials Thresher Pump set Fertilizer bag Rice fan Clothing & Personal items Belt Handkerchief Cap Tshirts Shoes Jacket Pants Nail cutter Toothbrush Razor Other Tape measure Playing cards Stamp pad Toy