Focusing on teacher quality in Pakistan: urgency for reform(draws from various recent pieces of work)

Monazza AslamSenior Research Associate ASER Pakistan

Introduction

•This talk will focus on the importance of teachers and an examination into their effectiveness in producing valuable student outcomes.

•Why? to underpin equity, efficiency and effectiveness in the delivery of teacher services in the country.

Current status of teacher quality in South Asia.

•What does teacher quality encompass?•What does it mean when we talk about teacher

quality?•Why is it important?•How does Pakistan fare in comparison to other SAR

countries?

Dig deeper…

•Look specifically at the teacher labour market;•Deployment; • teacher salaries;•accountability.

What makes an effective teacher? teacher competence•Measuring competence;•SchoolTELLS-Pakistan•Look into teacher competence linked to effective in-

service and pre-service training needs for Pakistan.

The Why of teacher quality• Universally recognised that variations in teacher

effectiveness are important determinants of differences in school quality (Hanushek and Woessmann, 2011).

• Poor quality schooling is one of the key factors documented to lie behind educational failures in the South Asia region - stems from a combination of factors but substandard teaching cited as the foremost reason contributing to poor schooling quality across the developing world.

• This is especially worrying because poor instruction and ineffective teaching reduces the demand for education which in turn reduces the pool of qualified teachers creating a vicious circle of poor quality schooling.

• Research now confirms that improving weak teaching may be the most effective means of raising school quality across the developing world (Glewwe and Kremer, 2006, p. 995).

What makes a ‘quality’ teacher?• ‘Quality’ encompasses a range of competencies and

skills. • Teacher quality in the very narrow sense of the term can be

defined as a ‘teacher’s ability to produce growth in student achievement’ (Eide et al. 2004).

• Despite initial research to the contrary, it is now recognized that schools do make a difference in determining student outcomes and a growing body of research acknowledges that teacher quality is probably the most important institutional influence on student outcomes (Goldhaber 1999, Hanushek 2002, Slater 2009).

• But research on teacher observables has also shown mixed results.

• Measuring ‘teacher quality’ is problematic – achievement production functions linking observable T characteristics to student outcomes or ‘fixed effects’ measuring teacher quality (US studies mostly).

Evidence from South Asia• Very few quality studies to date in the SA region that

measure teacher quality and effectiveness.• Ones that do (Kingdon and Teal 2008, Aslam and Kingdon

2012) find that standard resumé characteristics of teachers on which recruitment and pay policies of a large number of teachers (especially in government schools) are based, do not matter to student learning.

• Study by Aslam and Kingdon (2012) finds evidence that the teaching ‘process’ and teaching styles such as lesson planning and interactive teaching matter substantially for student learning in Pakistan.

• These considerations suggest that current criteria for recruitment and remuneration in South Asia and elsewhere that rely on certification characteristics of teachers are inefficient as they do not identify or reward the most effective teachers.

Pakistan in comparison with other countries in SAR•Are more ‘experienced’/aged teachers a

drain on public funds? Or do they represent an ‘experienced workforce’?

•Is Pakistan facing a shortage of female teachers?

•Is ineffective deployment an issue?

Pakistan is reported as a country with shortage of female teachersSource: UNESCO 2005, cited as Table 1 in ‘Managing Teachers: The Centrality of Teacher Management to Quality Education. Lessons from Developing Countries’, (CfBT, 2008).

Countries with a shortage of female teachers

% of female teachers

Countries with a shortage of

male teachers

% of female teachers

Chad 11 Sri Lanka 79

Togo 12 Botswana 80

Benin 19 Guyana 85

Guinea Bissau 20 Philippines 87

Afghanistan 24 Mongolia 93

Nepal 29 Kyrgyzstan 97

Pakistan 36 Kazakhstan 98

Pakistan Pupil Teacher Ratios from all public schools in 2008-2009Poor Teacher Deployment: Striking intra-country differences – KP vs Punjab(Source: Academy of Educational Planning and Management)

Pakistan fares reasonably well in terms of % of trained teachers in

the SAR region…

Note 1: 2008 UIS DataNote 2: Authors’ calculations from Statistics on School Education 2007-08Note 3: Authors’ calculations from Pakistan Education Statistics 2008-09Note 4: 2007 UIS data*Average for all levels.

Percentage of trained teachers: Nepal1

Bangladesh1

Bhutan1 India2 Pakistan

3

Srilanka4

Primary 66 54 91 90 97 63*

Lower secondary

52 57 90 91 98 63*

Secondary 58 50 83 89 98 63*

Upper secondary

64 42 72 93 86 63*

But inequity exists in the distribution of trained teachers by region and gender…

ource: Academy of Educational Planning and Management; Figures for 2008-2009. Note: This includes schools at all levels from primary to higher secondary. It also includes teachers at mosque schools. PTC is Primary Teaching Certificate; CT is Certificate of Training; B.Ed. is Bachelors in Education; M.Ed. is Masters in Education.

  Urban Rural Total

Professional Qualification

Male%

Female%

Male%

Female%

Male%

Female%

P.T.C. 41.3 58.7 62.3 37.7 57.8 42.C.T. 45.2 54.8 64.0 36.0 58.7 41.3

B.Ed. 49.0 51.0 65.7 34.3 60.7 39.3M.Ed. 56.8 43.2 71.6 28.4 65.9 34.1

Any other training

63.9 36.1 73.5 26.5 71.3 28.7

Untrained 57.6 42.4 63.1 36.9 62.2 37.8Not reported 64.7 35.3 82.8 17.2 79.4 20.6

Total 48.2 51.8 65.2 34.8 60.8 39.2

What do we know about teacher ‘effort’ in SA?• Teacher effort exerted while in school in many

countries of the SAR region is at a pitiably low level, as measured by very high teacher absence rates.

• The problem therefore is not even one of low quality teaching but one of no teaching at all, for a significant part of the time (World Bank, 2004).

• Teacher absence has been linked with low student outcomes in a diverse group of countries (Miller et al. 2007, Clotfelter, Ladd and Vigdor, 2006).

Introducing the SchoolTELLS-Pakistan data• Undertaken in April- May 2011 with World Bank funding. • Covered 3 districts of Faisalabad, Mianwali & Rahim Yar

Khan, from the province of Punjab. The project covered twenty villages in each of the three districts and two schools were selected from each village, comprising a total sample of 120 schools.

• The main purpose of this research study was to get reliable in-depth data to comprehensively understand the relationship between student learning levels and factors that can influence them, such as teachers’ background, children’s background & ability, classroom environment and school environments. 

• Idara-e-Taleem-o-Aagahi (ITA) or the centre for education and consciousness was the collaborating entity in Pakistan which also houses the secretariat for the South Asia Forum for Education Development (SAFED) that has undertaken the pioneering initiative of the Annual Status of Education Report (ASER) Pakistan in 2008 and 2010.

Assessing competencies• Learning levels of students from classes 3 and 5 were tested

using student tests, while the student ability or IQ was tested using the Raven’s Progressive Test.

• Student background information was collected using a student questionnaire.

• Learning levels of teachers were also measured using a teachers test. The test was designed in such a way that it tested three main competencies:

1. teachers’ understanding of the subject-matter from the primary curriculum,

2. their ability to spot student mistakes and3. their ability to explain content in an effective manner.

Unknown to the teachers and students, some of the questions in the mathematics and language tests were common for the two groups. Additional background information was also collected on issues such as teacher experience, education, affiliation etc. as well as information on political economy issues (such as union membership) and measures of teacher effort (time on task, absence etc.).

•Rich information has been gathered on more than 1500 students and more than 300 teachers across the 120 schools in Punjab.

•One caveat of note is that these data are not representative of Pakistan as a whole and the findings from this research are meant to be informative.

SchoolTELLS: Teacher absence is high and mainly unexplained• Poor motivation and a lack of accountability

(through credible sanctions and punishment) are said to be the main reasons for the high rates of absenteeism observed among teachers in developing countries.

• Data from SchoolTELLS-Pakistan from rural Punjab reveals that on the day of the visit, 11 per cent of the teachers were reportedly absent. The majority of this absence was ‘unexplained’ rather than attributable to official non-teaching duties. Illness accounted for most of the explained absences.

• The teacher absence rate in private schools, where there tend to be stricter accountability policies, is 7%. This is lower than the absence rate of 12% found in Government schools.

Teacher competence in rural Punjab…• Teachers in rural Punjab are substantially

competent? This is unlike similar data that was collected in India showing worryingly low levels of teacher competence in these same three aspects of competence (see Kingdon and Banerjee, 2009).

• For example, the average score of all teachers in Pakistan in the language test was 69.5% while in mathematics the score was 73.9%.

• Another way of looking at this: teachers did not have 100% knowledge of the very content they are supposed to be teaching their students!

High teacher competency is also not reflected in higher student learning

Multigrade teaching• Multi-grade teaching has become a common strategy to meet

MDG goals and to deal with issues of teacher shortages and absences particularly in remote rural areas in several developing countries.

• Research on the effects of Multigrade teaching on student learning to date has shown mixed results (Little 2008).

• Several studies report a disadvantage associated with multigrade teaching (example Kochar (2007) found that students in multi- grade settings in Andhra Pradesh in India performed at a lower level than their counterparts in monograde settings. Rowley (1992) found that monograde schools in Pakistan showed cognitive differences in favour of children in monograde schools. Suzuki’s (2006) observation of multi-grade settings in Nepal found that one major negative impact was that for some proportion of the day there was a group of children who were neglected/ignored with no teacher taking responsibility for their learning or directing them towards self-learning during this time.)

SchoolTELLS data reveals inconsistencies in grade groupings• In the one visit randomly made during the SchoolTELLS survey, data reveal

that 43 percent of the children in primary school were sitting in a monograde environment.

• However, it was also found that in 8 percent of the cases, all five primary grades were sitting together.

• These findings suggest considerable instability in grade-grouping configurations within the same school year, making it difficult for the teacher to prepare teaching for a given mix of classes.

• This provides one potential explanation for why despite having competent teachers; students in rural Pakistan have such low levels of learning. Much of this may stem from frequent teacher absence (as mentioned previously) which results in ad-hoc-ism about classroom organisation. T

• Persistence of multigrade settings in developing countries needs to be reflected in teaching training courses in order to prepare teachers for this eventuality.

Teacher Remuneration

•The provision of high-quality schooling requires an adequate supply of competent teachers.

•Among the factors that influence this supply, perhaps the most important are: the salary offered to teachers, relative earnings offered in alternative careers and varying non-pecuniary conditions of work (work load, job stress, status and satisfaction etc.).

Benchmarking teacher salaries

A. Compare teacher salaries to per capita GDP – how well off is the teacher in comparison to the average person in a country?

B. Compare teacher salaries to salaries of (comparable) persons in non teaching jobs;

Furnishing a measure of economic and social distance…•One of the means of benchmarking

teacher wages is to compute the ratio of teacher salaries as a proportion of GDP per capita/per capita income in a country.

•This ratio tells us how affluent the teacher is, with respect to the average person in the country.

•Thus, it furnishes a measure of the economic and social distance between the teacher and the taught.

Pakistan - Ratio of teacher salary to per capita income (Rupees), by province

Source: * Pakistan Statistical Yearbook (2007), Government of Pakistan, Statistics Division, Federal Bureau of Statistics; x Pakistan Statistical Yearbook (2007); + We identified teachers using the occupation codes in Pakistan Labour Force Survey (2008). The reported salaries are for all teachers in government and private school jobs teaching at all levels. ** Column (f) shows column (d) figures inflated to 2008 prices using the Wholesale Price Index for Pakistan reported in Pakistan Statistical Yearbook (2007).

Province     

Average monthly

household income

(2004-2005), rupees/month*

 

 

 

  

Average household size (1998 Census)x

 

 

    

Estimated monthly

per capita income

       

 Annual per

capita income

      

Annual teacher

salaries in 2008+

 

 

     

Annual per capita income in

2008 prices**

 

 

   

Teacher salary

as multiple

of per capita

income in 2008

   

(a) (b) (c) (d) (e) (f) (g)Punjab 9488 7 1355 16265 115172 22283 5.2

Sindh 10413 6.1 1707 20485 128624 28064 4.6NWFP 9395 8 1174 14093 106572 19307 5.5Balochistan 8849 6.8 1301 15616 127070 21394 5.9Pakistan 9685 6.9 1404 16844 119480 23076 5.2

How does Pakistan compare with India? Select states….

 State    

2008 per capita net domestic product at

factor cost (base=1999-2000)*

 

 

 

2008 per capita net domestic

product in current (2008)

prices×

 

 

Annual teacher

salaries in 2008+

 

 

 

 

 

Teacher salary as

multiple of per capita

income in 2008

  

Kingdon (2010)estimates of teacher

salary/per capita income 2004**

(a) (b) (c) (d) (e)

Andhra Pradesh 27362 42958 89876 2.1 2.8Assam 16272 25547 127853 5.0 -Bihar 10206 16023 187685 11.7 12.4Jharkhand 16294 25582 124290 4.9 -Gujarat - - 123862 - 2.8Jammu and Kashmir - - 103415 - 3.8Madhya Pradesh - - 157147 - 4.5Chhattisgarh 19521 30648 124383 4.1 -Maharashtra - - 107886 - 3.1Punjab 33198 52121 149073 2.9 -Rajasthan 19708 30942 166609 5.4 4.6Uttar Pradesh 12481 19595 103396 5.3 7.3West Bengal 24720 38810 108534 2.8 4.8Simple means for states 23642 37119 119540 4.2 5.1

Teacher Pay relative to other occupations(Source: Authors’ calculations from Pakistan Labour Force Survey 2000 and 2008)

  Pakistan

2000 2008Mean

Monthly Salary (in

2005 US$)

Ratio of Teacher

Salary/Salary in other

occupation

Mean Monthly

Salary (in 2005 PPP

US$)

Ratio of Teacher

Salary/Salary in other

occupationTeachers 230 - 303 -Legislators 351 0.7 384 0.8Professionals 314 0.7 360 0.8Associate Professional

253 0.9 303 1.0

Clerks 239 1.0 300 1.0Service workers/shops

221 1.0 239 1.3

Skilled Agriculture 170 1.4 234 1.3Crafts 215 1.1 242 1.3Plant/Machine operators

251 0.9 227 1.3

Elementary 172 1.3 179 1.7All non-teachers(weighted average)

 257

 0.9

 277

 1.1

SchoolTELLS evidence reaffirms this…• SchoolTELLS Pakistan data reveals that the average monthly

salary of a primary school teacher in rural Punjab is Rs. 17,000 as compared to Rs. 3800 earned by their private school counterparts.

• The salary of government schools teachers is roughly more than 6 times as much as the per capita monthly income of the average person in Punjab. This is likely to be an underestimate of the multiple as the latter figure includes the earnings of urban workers. This furnishes a measure of the economic and social distance between the teacher and the taught as it is believed that the greater the distance is, the more detrimental it can be to student learning (see Kingdon and Rawal, 2010).

• Salary increases are intended to improve the quality of public services delivered to citizens. The pay-rises are premised on higher salaries attracting better individuals into teaching and also on the idea that higher salaries motivate higher effort while in service, as per efficiency wage theory.

• However, Kingdon (2010) argues that salary increases unrelated to performance are not necessarily efficiency enhancing. This is true in a system where salaries are linked to a national pay scale as is the case for teachers as well as the bureaucracy and military etc. in Pakistan. Even more importantly, Aslam and Kingdon (2012) also show that teacher salaries in Lahore are not related to higher student learning suggesting that simply raising teachers’ salaries will not necessarily lead to an improvement in student achievement.

Aslam and Kingdon (2012)• Estimate education production functions linking

student achievement (1880 + 8th grade students across 65 government and private schools in Lahore), to student, school and teacher characteristics;

• This paper develops the idea that teachers’ classroom practices and the teaching ‘process’ may matter more to student learning than teachers’ observed résumé characteristics (such as certification, qualification and experience).

• There may also be important differences in teacher characteristics across government and private schools which may help explain the large documented public-private achievement differences often found in studies.

• This paper delves into the black-box representing ‘teaching’ to uncover the teacher characteristics and teaching practices that matter most to pupil achievement. T

• The data allow exploitation of an identification strategy that permits the matching of students’ test scores in language and mathematics to the characteristics of teachers that teach those subjects.

• Findings reveal that the standard résumé characteristics of teachers do not significantly matter to pupil achievement. Perversely, however, teachers are found to be rewarded for possessing these characteristics highlighting the highly inefficient nature of teacher pay schedules. Our findings also show that teaching ‘process’ variables matter significantly to student achievement.

• There are important differences across school-types – teachers in private schools are seen to adopt practices that enhance pupil learning.

Conclusions• There is a need to address issues pertaining to

all three aspects – equity, efficiency and effectiveness in the delivery of teaching services in the country.

• Have seen inequity in the distribution of trained teachers and ineffective deployment across the country.

• Have also seen the need to develop teacher competence through effective training.

• Multi-grade settings prevail especially in rural settings and there is a need to acknowledge and build this reality into teacher training curricula.

Address teacher remuneration issues…• The key policy implication is also that teacher

remuneration and tenure should be linked to teacher performance and effort to increase efficiency within the schooling system rather than being simply linked to a mechanical pay scale in a system where jobs for life are guaranteed without linkages to effort.

• This is critical also for rewarding better-performing teachers who work in extremely difficult conditions without the most basic materials and infrastructure and whose work beyond the call of duty engenders a love for learning and knowledge in the most arduous situations.