SUMMER INTERNSHIP PROGRAMME 2011 UNICEF INDIA

Documentation of Samiksha: Lessons Learned from a School Monitoring System Fatima Alam Ana Maria Angarita Adrienne Henck Ioanna Sikiaridi In collaboration with KIIT University, School of Rural Management Bhubaneswar

Disclaimer

The views expressed in this report are those of the authors alone and do not necessarily reflect the policies

or the views of UNICEF and/or the School of Rural Management, KIIT University, Bhubaneswar.

Contents

List of Tables, Maps and Figures 1

Acknowledgements 2

List of Acronyms 3

Executive Summary 5

Introduction 7

Background 9

Education context 9

Monitoring systems 12

Strategy and Implementation of Samiksha 15

What is Samiksha? 15

Key Samiksha stakeholders 16

Process 19

Samiksha process 19

Analysis of Samiksha process 21

Measures undertaken to improve the Samiksha process 24

Progress and Results 25

Factors enabling progress and results 28

Factors hindering progress and result 30

Lessons Learned 33

Next Steps 35

References 37

Appendices 39

Cuttack CMC Mission Cantonement Dagarapara Lalbag Dagarapara Mahanadi Vihar Mahanadi Vihar Narasinghpur Ransinghpur Allara Sridhara Siaria ACME NB Nodal Jharsuguda Jharsuguda Puranabasti Badhaimunda Harijanpanda PS Talpatia Mundadhipa PPS Barmal PS Kirmira Arda Kankaramunda PUP Arda PS Deopan Sulahi UP UPS Deopan Keonjhar Banspal Kundhei Kundhei Laxmidhar Suakathi Mining Talakainsari Harichandanpur Kaduadiha Kaduadiha Makasukhila Harichandanpur Kaliaduma Jaunliporkhari Koraput Koraput Koraput Ex-Board UPS Gopalput UPS Lauriguda Thana PS Project PS Semilliguda Semilliguda Aligam UPS Block Colony UPS Kunduli Malipungar UPS Kunduli UPS

List of Tables, Maps and Figures

Tables

Table 1: Comparison of education in Orissa and India

Table 2: Samiksha categories and indicators

Table 3: Samiksha colour-coded ranking categories

Table 4: District profiles

Table 5: Participants

Figures

Figure 1: Key Samiksha stakeholders

Figure 2: Flow of school data and feedback

Figure 3: Teacher attendance

Figure 4: District overall Samiksha score

Figure 5: Improvement of sample schools by category

Figure 6: Sample selection structure

Figure 7: Overall Samiksha scores for sample blocks

Maps

Map 1: Map of Orissa

Acknowledgements

We take this opportunity to thank UNICEF and the Knowledge Community on Children in India (KCCI)

Internship Programme for selecting us to be a part of their team and for giving us the opportunity for this

fruitful learning experience.

We also thank UNICEF Orissa and KIIT University, School of Rural Management, Bhubaneswar for their

logistical support throughout the internship and fieldwork.

Further, we would like to acknowledge the contributions of officials from the Orissa Department of

School and Mass Education (DoSME) for their assistance in procuring the data and for assisting us with

our fieldwork.

And finally, special thanks to the children, parents, teachers, headmasters, officials at the District SSA

offices, and Block and Cluster Resource Centre Coordinators (BRCCs and CRCCs) in Cuttack,

Jharsuguda, Keonjhar and Koraput for making our field visits possible and sharing their experiences with

us. Without their support, this report would not have been possible.

List of Acronyms

AIES All India Education Survey

ASER Annual Status of Education Report

BCF Block Compilation Format

BRCC Block Resource Centre Coordinator

CMC Cuttack Municipal Corporation

CRCC Cluster Resource Centre Coordinator

CWSN Children with Special Needs

DCF District Compilation Format

DEE Directorate of Elementary Education

DIS District Inspector of Schools

DISE District Information System for Education

DoSME Department of School and Mass Education

DPC District Project Coordinator

DPEP District Primary Education Programme

GoI Government of India

GoO Government of Orissa

ICF Individual Compilation Format

M&E Monitoring and Evaluation

MLE Multilingual Education Programme

MPLAD Member of Parliament Local Area Development

MTA Mother Teacher Association

NCERT National Council of Educational Research and Training

OPEPA Orissa Primary Education Programme Authority

PGCD People‟s Group for Children‟s Development

PTA Parent Teacher Association

PTC Performance Tracking Cell

PTR Pupil-Teacher Ratio

RTE Right of Children to Free and Compulsory Education Act

SC Scheduled Caste

SCF State Compilation Format

SI Sub-Inspector of Schools

SMC School Management Committee

SMU State Monitoring Unit

SSA Sarva Shiksha Abhiyan

ST Scheduled Tribe

TLM Teaching Learning Material

UEE Universal Elementary Education

UNICEF United Nations Children's Emergency Fund

VEC Village Education Committee

Executive Summary

Background: An effective school monitoring system is critical for the improvement of education.

However, evolving socio-economic and policy contexts require the constant revision of monitoring

systems. In response to the Right of Children to Free and Compulsory Education (RTE) Act and

challenges in education, the Orissa Department of School and Mass Education (DoSME) developed an

innovative school monitoring system in November 2010. The scheme, called Samiksha, involves

continuous monitoring of nearly all government elementary schools, analysis of monitor reports, and

interventions directed at improving school quality. Samiksha‟s 80 indicators, which represent the inputs

and processes for determining the outcomes in a model school, are divided into five categories: School

Environment, Curricular Program, Co-curricular Program, School-Community Link and School

Management.

Purpose of Report: The novelty of Samiksha means that there is much uncertainty around how the

system is functioning in actuality. The purpose of this report is to address these knowledge gaps by

providing documentation that examines lessons learned from the initial performance of Samiksha. The

analysis is based on semi-structured interviews conducted with Samiksha‟s key stakeholders, including

government officials and school actors, in 32 schools across four districts.

Progress: During the first six months of Samiksha‟s implementation, 29 out of 30 districts have

improved their overall score. Most stakeholders have become more aware of school improvement

processes and accountable in performing their duties and responsibilities. Additionally, Samiksha has

enhanced the systematization of the school improvement process and has allowed the DoSME to track

school progress across the five categories. The individual motivation of stakeholders has played an

important role in the effective use of available resources and the successful implementation of Samiksha.

On the other hand, although the standardized monitoring format has facilitated the collection and analysis

of data, the subjectivity of some indicators continues to present challenges to consistent and uniform

monitoring. Limited capacity of some monitors and lack of awareness among most School Management

Committee (SMC) members and some teachers jeopardizes the long-term sustainability and

institutionalization of the system. Furthermore, administrative issues such as teacher shortage and the

socio-economic conditions of the surrounding community may affect the capacity of schools to improve.

These factors explain part of the variations in schools within and across districts.

Lessons Learned: The following lessons learned apply both specifically to the implementation of

Samiksha in Orissa and more broadly to the use of monitoring systems in India and in other contexts:

A school monitoring system is an essential way forward in the implementation of the norms and

standards of the RTE Act.

Broad scope, frequency and regularity are essential features of an effective school monitoring system.

Raising the awareness and capacity of all stakeholders is critical for a successful and sustainable school

monitoring system.

An integrated school improvement and monitoring approach is crucial for an effective school

monitoring system.

Potential Application: To ensure the successful implementation of the RTE norms and standards, other

states may consider adopting a large-scale monitoring system like Samiksha. Such a system could be

beneficial to every state if adapted to the local context to account for the baseline conditions of the school

system and variable socio-economic and demographic characteristics.

Next Steps: The following next steps should guide the DoSME‟s future implementation of Samiksha:

Increase awareness about Samiksha among stakeholders

Improve capacity of monitors

Ensure proper monitoring

Develop stronger and more systematic information and feedback flows

Revise Samiksha continuously to ensure the ongoing improvement of schools

Introduction

An effective school monitoring system is critical for the improvement of education. Such a monitoring

system should involve the collection of data on what is happening in the schools. It also should be

integrated with the analysis, evaluation and feedback of the data and the design and implementation of

evidence-based interventions. In India, various school monitoring systems exist that capture different

aspects of the education system. However, evolving socio-economic and policy contexts require the

constant revision of monitoring systems to capture changing needs.

The Orissa Department of School and Mass Education (DoSME) introduced a school monitoring and

evaluation system in November 2010. The scheme, called Samiksha1, involves continuous monitoring of

nearly all government elementary2 schools, analysis of school performance data based on 80 indicators,

and the implementation of interventions directed at improving school quality. The innovativeness of

Samiksha stems from its broad scope, frequency, and regularity. No other state in India has implemented

a scheme involving such frequent and geographically widespread monitoring of schools—each month

about 5,000 monitors collect data from approximately 50,000 schools, or more than 90 per cent of all

government elementary schools in Orissa. 1 The DoSME also refers to Samiksha as Samikhya. This report uses Samiksha throughout.

2 The term „elementary‟ refers to primary (Class I-V) and upper primary (Class VI-VIII) levels.

Guided by the emphasis of the national Right of Children to Free and Compulsory Education (RTE) Act

on school quality, Samiksha‟s 80 indicators represent the inputs and processes which determine the

outcomes in a model school. The indicators are divided into five categories: School Environment,

Curricular Program, Co-curricular Program, School-Community Link and School Management. Thus,

this new system of continuous monitoring and evaluation provides policy makers and other stakeholders

with a broad assessment of the state of education in Orissa and facilitates the development of strategies

for improvement in the quality of schooling.

However, the novelty of Samiksha means that there is much uncertainty around how the system is

functioning in actuality. The purpose of this report is to address these knowledge gaps by providing

documentation that examines lessons learned from the initial performance of Samiksha.

The following questions guided our documentation of Samiksha and analysis of lessons learned:

a. What improvements have been made in the school system since the implementation of Samiksha?

b. How does the school monitoring format and process facilitate the implementation of Samiksha?

c. Does Samiksha reflect the real issues of schools?

d. Does Samiksha accurately capture the progress of schools?

e. What factors explain inter-district and intra-district school performance variation?

We visited 32 schools in four sample districts, Cuttack, Jharsuguda, Keonjhar3, and Koraput, and

conducted semi-structured interviews with Samiksha‟s main stakeholders, including government officials

and school functionaries.4 By analysing the data we constructed lessons learned based on the most salient

themes in the findings. We conclude this report by proposing recommendations for improving the school

monitoring system. 3 Keonjhar is also referred to as Kendujhar. This report uses Keonjhar throughout. 4 See Appendix I and Appendix II For detailed information on Methodology and Questionnaires.

As Samiksha is still new and our sample size is small, a large-scale evaluation of Samiksha is not

appropriate for this study. Nonetheless, this report will play a critical role in developing a strong

foundation for Samiksha through documentation of stakeholder perspectives, identification of challenges

and recommendations of appropriate and viable reforms. Additionally, because the DoSME produced

very little documentation on Samiksha, even in the planning stages, this report will serve as

documentation for the Government of Orissa (GoO) and as an instructional resource for other states

considering the adoption of school monitoring systems like Samiksha. This documentation is useful for

understanding the importance of monitoring systems in achieving RTE Act compliance and how states

address the norms and standards set by the Act. The DoSME seeks to continuously revise and strengthen

the monitoring system. This report will give stakeholders the opportunity to examine the initial

performance of Samiksha and address key concerns before problems become endemic. Finally, the

rationale behind this study is that the long-term success and sustainability of such a large-scale project

depends on building a strong foundation and support from all relevant stakeholders.

Background

To analyze Samiksha it is important to understand the state of education in India and Orissa. This section

will provide an overview of the national and state education policies and interventions. It also provides a

description of existing school monitoring systems in India.

Education context

National policy context

Education is the concurrent responsibility of the Government of India (GoI) and its states. The national

policy context has greatly influenced how the Orissa DoSME has approached the challenges of education

in the state.

Launched in 2001, the Sarva Shiksha Abhiyan (SSA), or Education for All, aims at providing eight years

of quality education to all children in the age group 6-14 by 2015. The GoI implements SSA in

partnership with state governments and through a district-level decentralized management framework.5

The specific goals of the policy are universal enrolment and retention, reduction of gender gaps in

education and increased enrolment and retention of Scheduled Caste (SC), Scheduled Tribes (ST) and

socio-economically disadvantaged minority groups. The SSA also addresses the crucial systemic issue of

teacher shortages and teacher training and education. 5 Sarva Shiksha Abhiyan (SSA), „Sarva Shiksha Abhiyan: Framework of Implementation‟, p. 2. Available at:

http://ssa.nic.in/page_portletlinks?foldername=ssa-framework

6 Sarva Shiksha Abhiyan (SSA), „Sarva Shiksha Abhiyan: Framework of Implementation‟, p. 3. Available at:

http://ssa.nic.in/page_portletlinks?foldername=ssa-framework

In 2009, the Indian Parliament enacted the RTE Act making free and compulsory education a right of

every child in the age group 6-14 years. The Act came into force on 1st April, 2010, and Orissa was

among the first states to implement it. In addition to issues of accessibility and equity, the RTE Act

emphasizes aspects of education quality such as comfortable Pupil-Teacher Ratio (PTR), curriculum

reform and improvement of student evaluation methods.6 However, the school system is faced with an

acute shortage of teachers. Moreover, the provision of adequate training to available teachers also remains

crucial in order to fulfil the “child-centred teaching” envisioned in the RTE Act.

State of education in Orissa

The State Education Policy in Orissa, developed and implemented the DoSME, reflects the national

education priorities and is based on achieving the goals of SSA. It is implemented through two agencies,

the Directorate of Elementary Education (DEE) and Orissa Primary Education Programme Authority

(OPEPA).

The Orissa DoSME conceived of Samiksha in the context of critical challenges faced by the state‟s

education system. These challenges include gender, regional, and SC/ST disparities in literacy rates and

educational achievement; language barriers for ST children and accommodating cultural diversity through

the curriculum. The state is home to 62 tribes which comprise 22.1 per cent of the population and speak

30 different languages.7 Even though overall state literacy rates have improved, reaching the India

average rates, SC/ST children lag behind with lower enrolment rates and higher drop-out rates (see Table

1) due to reasons such as cultural and language barriers. Poor retention rates also hamper the achievement

of universal elementary education (UEE), a key aim of the national SSA initiative. To address some of the

disparities, the state adopted the national Mid-Day Meal Programme and implemented a Multilingual

Education Programme (MLE). The Mid-Day Meal Programme, launched in 1995 by the Ministry of

Women and Child Development, aims at promoting regular school attendance while improving the

nutritional status of children at the primary level.8 The MLE Programme, which uses the tribal language

of the area as the medium of instruction, has been implemented in schools in tribal areas since 2006.9 It

operates under the belief that when children learn in their own mother tongue and environment, it helps

them to develop their knowledge in their own socio-cultural context. 7 Office of the Registrar General and Census Commissioner, Government of India, „Orissa: Data Highlights: The

Scheduled Tribes‟, Census of India 2001, New Delhi, 2001. Available at:

http://censusindia.gov.in/Tables_Published/SCST/dh_st_orissa.pdf

8 National University of Educational Planning and Administration (NUEPA), Government of India. „Education for

All Mid-Decade Assessment: Reaching the Unreached‟, New Delhi, 2008, pp. 26-27. Available at:

http://unesdoc.unesco.org/images/0018/001817/181775e.pdf

9 Orissa Primary Education Programme Authority (OPEPA), „Multilingual Education in Orissa: Issues and

Interventions.‟ Available at: http://www.opepa.in/MLE_in_Orissa.pdf

Teacher shortages, inadequate teacher training and the lack of a sufficient number of school facilities and

poor school infrastructure pose additional obstacles to improving education. To address remoteness and

poor communication in many rural areas, following the national RTE norm, the state should provide a

primary school within one kilometre of all habitations with a minimum population of 300 people. In case

of habitations located in hilly areas with minority population, the norm is 200 people. Furthermore, the

state should provide an upper primary school within a distance of three kilometres in every habitation

with a minimum population of 500 people.

Table 1: Comparison of education in Orissa and India

Orissa

India

Literacy Rate10 Overall 73.45 74.04

Male 82.40 82.14

Female 64.36 65.46

Drop-Out Rates (Classes

I-VIII)11

All

Categories

Overall 62.63 48.71

Boys 64.83 48.49

Girls 59.49 48.98

SC Overall 67.55 55.25

Boys 67.18 53.70

Girls 68.02 57.28

ST Overall 80.74 62.95

Boys 81.66 62.76

Girls 78.96 63.20

Percentage of Single-Teacher Primary Schools12 17.99 12.26

10 Office of the Registrar General and Census Commissioner, Government of India, „Provision Population Totals,

Orissa Series‟, Census of India 2011, New Delhi, 2011. Available at:

http://censusindia.gov.in/2011-prov-results/prov_data_products_orissa.html

11 Department of Education, Planning, Monitoring and Statistics Division, Ministry of Human Resource

Development, Government of India, „Selected Education Statistics 2005-2006: MHRD‟, New Delhi, 2008.

Available at: http://www.educationforallinindia.com/SES2005-06.pdf 12 National University of Educational Planning and Administration (NUEPA) and Department of Social Education and Literacy, Ministry of Human Resource Development,

Government of India, „Elementary Education in India: Progress Towards UEE‟, Flash Statistics DISE 2009-2010, New

Delhi, 2011. Available at:

http://www.dise.in/Downloads/Publications/Publications%202009-10/Flash%20Statistics%202009-10.pdf

Education interventions in Orissa

Taking universal enrolment, universal retention and quality education as essential for the proper

implementation of the RTE Act, the DoSME has introduced a number of interventions, as discussed

below.

Ama Vidyalaya: Ama Vidyalaya is a school beautification project which includes provisions for

improving both external and internal spaces of schools. For example, schools should construct a

boundary wall around the school and establish a Teaching Learning Material (TLM) corner inside

each classroom. All schools receive Rs 75,000 to assist with the implementation of the project. As part

of the Ama Vidyalaya, every school is required to have a student cabinet. Members of the cabinet are

responsible for various activities related to hygiene and sanitation, food, environment, education and

discipline. The student cabinet is an initiative to increase student participation in the management of

schools and enhance their leadership capabilities.

Samarthya: Samarthya is an integrated training and capacity building plan for elementary and

secondary school teachers.

School Management Committee (SMC): The RTE Act mandates the constitution of a SMC in every

school, other than unaided schools. The responsibilities of the SMC include recording the academic

progress of students, ensuring the enrolment and continued attendance of all the children, monitoring the

implementation of the Mid-Day Meal Programme and improving the school facilities and environment.

Siksha Chetana Abhiyan: Siksha Chetana Abhiyan is a massive state-wide campaign to encourage

community mobilization for the implementation of the RTE Act. Conducted from 15th to 21st April,

2011, in collaboration with the Department of Rural Development, Department of Scheduled Caste &

Scheduled Tribe Development and UNICEF, it aims to promote sanitation practices in schools through

community participation. Additionally, it includes „Pravesh Utsav' in which school premises are

decorated to welcome newly-enrolled children.

Monitoring systems

Role of monitoring systems

A well-functioning monitoring system is critical for the effective achievement of programme or, in this

case, education goals. Monitoring is:

“...a continuing function that uses systematic collection of data on specified indicators to provide

management and the main stakeholders of an ongoing development intervention with indications of the

extent of progress and achievement of objectives and progress in the use of allocated funds.”13 13 The Organization for Economic Cooperation and Development (OECD), Glossary of Key Terms in Evaluation

and Results Based Management, Paris, 2002, pp. 27-28. Available at:

http://www.oecd.org/dataoecd/29/21/2754804.pdf

For an effective monitoring process, the collection of data must be accompanied by its analysis and

evaluation. Monitoring in education also requires the establishment of standards for the effective

assessment of features such as teacher quality, curricula, student performance and other elements of the

entire education system.14 14 Tavola, H. „Educational Planning‟, Journal of Educational Studies, Volume 30, Nos. 1 and 2, 2008, p. 3.

15 World Bank, „Why is it Important to Institutionalize Government M&E System?‟ Available at:

http://go.worldbank.org/ZDHJ987I80

16 National Council of Educational Research and Training (NCERT), „Education Survey.‟ Available at:

http://www.ncert.nic.in/programmes/education_survey/index_education.html

While monitoring holds stakeholders accountable for their actions, the effectiveness of a monitoring

system also depends on the involvement of various stakeholders. Increased awareness of Monitoring and

Evaluation (M&E), and its tools, methods and techniques, helps to increase the demand for it and thereby

strengthens its functioning. Thus, the institutionalization of a monitoring system is necessary for its

success. The institutionalization of an M&E system implies that:

“…it produces monitoring information and evaluation findings which are judged valuable by key

stakeholders, when this information is used to improve government performance, and when there is

sufficient demand for the M&E function to ensure its funding and its sustainability for the foreseeable

future.” 15

Other monitoring systems in India

Policy makers often use data from a variety of monitoring systems to develop education strategies. The

three most widely used are discussed below:

All India Education Survey (AIES): Conducted by the National Council of Educational Research and

Training (NCERT), AIES enables the collection of information on the country‟s overall progress in the

area of school education. However, the time lag between the collection and the dissemination of data is a

significant limitation of this system.

AIES covers availability of schooling facilities in rural habitations; physical and educational facilities in

schools; incentive schemes and beneficiaries; medium of instruction and languages taught; enrolment

particularly of SCs, STs, girls and educationally disadvantaged minority groups; teachers and their

academic and professional qualifications; library; laboratory; ancillary staff and subject-wise enrolment at

+2 stage of education.16

District Information System for Education (DISE): DISE is a school-based statistical system initiated in

1995 to monitor the implementation of the District Primary Education Programme (DPEP). In 2001, in

light of the SSA, DISE was extended to cover the level of elementary education in the entire country.

District and School Report Cards are produced to provide information in a standardised format.

Furthermore, it has drastically improved the time lag in data dissemination from seven to eight years to

less than one year.17

17 National University of Educational Planning and Implementation (NUEPA), District Information System for

Education. Available at: http://www.dise.in/Downloads/DISE%20Flier.pdf

18 National University of Educational Planning and Implementation (NUEPA), About DISE. 2009. Available at:

http://www.dise.in/dise2001.htm

19 Annual Status of Education Report Centre, Annual Status of Education Report: Overview, New Delhi. Available

at: http://www.asercentre.org/ngo-education-india.php?p=ASER+survey

DISE covers two types of information: village level and school level. Village-level data comprises

variables related to access to educational facilities of various types, identification of habitations without

access to primary and upper primary schools based on distance norms, inventory of all types of

educational institutions including recognized and unrecognized schools in the village.

School-level information comprises core data on school location, management, rural-urban enrolment,

buildings, equipment, teachers, medium of instruction, age-grade matrix, children with special needs

(CWSN), and examination results. The school summary report, shared with the school, contains key data

on the school and a summary of indicators which are compared with the cluster, block and the district

averages.18

Annual Status of Education Report (ASER): Since 2005, Pratham, a non-governmental organization has

conducted an annual survey to assess children's ability to read simple text and do basic arithmetic.

Covering over 70,000 children, ASER is significant in highlighting the prioritization of education quality

in government policy.19

Strategy and Implementation of Samiksha

What is Samiksha?

Samiksha is a state-wide school monitoring system adopted by the Orissa DoSME in November 2011.

The purpose is to continuously monitor the progress of elementary schools, analyze school performance

data, and implement appropriate interventions to improve the state of education in Orissa. Three key

features define Samiksha:

Scope: Unlike other monitoring systems which collect data on a sample of schools, Samiksha aims to

track 100 per cent of the elementary schools in Orissa. Approximately 50,000 schools out of a total of

53,61420 schools are currently being monitored by about 5,000 monitors. The DoSME plans to expand the

monitoring to reach all schools by August 2011.

20 Department of School and Mass Education, Government of Orissa. Literacy and Mass Education: An Overview.

Available at: http://www.orissa.gov.in/schooleducation/index.htm

Frequency: The Samiksha process occurs on a monthly basis. Monitors quickly identify problem areas

and provide feedback to schools, and the DoSME receives rapid, comprehensive assessments of the state

of education in Orissa.

Regularity: As an ongoing system, Samiksha captures school performance trends over time and aids

in the development of strategies for further improvement in the quality of education.

The Samiksha monitoring format includes 80 indicators which each school should strive to achieve. The

indicators represent the inputs and processes which determine the outcomes in a model school. A team of

experts under the DoSME selected the 80 indicators drawing on pre-existing pedagogical interventions

and teacher training curricula and modules. Many of the indicators are linked to the norms and standards

of the RTE Act and to the state-wide education interventions discussed above. The quality of education

delivered in a school depends on the indicators which are divided into five categories (see Table 2).

Table 2: Samiksha categories and indicators

Category

Number of

Indicators

Example Indicators

School Environment 16 Boundary wall / fence Toilets available and properly used

Curricular Programme 30 Adequate Teaching Learning Materials (TLMs)

available & used by children and teachers

Students actively participate in learning

Co-curricular

Programme

14 Health check-up done in last month

School cabinet functional

School-Community

Link

10 VEC/SMC formed according to rule

School Management 10

Headmaster monitored the teaching of other teachers

Key Samiksha stakeholders

UNICEF: UNICEF is involved in the development of education in Orissa, including the

development and implementation of Samiksha.

Department of School and Mass Education (DoSME): The DoSME, headed by the Secretary is

the primary government agency responsible for administering education services in the state of

Orissa. The current Commissioner-cum-Secretary, Aprajita Sarangi, conceptualized and

spearheaded the development of Samiksha. The State Monitoring Unit (SMU), also known as the

Performance Tracking Cell (PTC), supports the Secretary to develop school improvement

interventions by analyzing the Samiksha school report data, producing state level compilation

reports, and providing feedback to the districts through communication channels and review

meetings.

District Office, Sarva Shiksha Abhiyan (SSA): The District SSA offices administer education

programmes and services at the district level. The District Project Coordinator (DPC), who heads

the SSA offices, is required to monitor at least five schools per month. The District SSA office

produces a monthly district-level compilation of Samiksha data that is submitted to the DoSME.

This compilation includes cluster and block school reports while the DPC‟s reports are

submitted to the SMU separately and are not included in the state compilation. Other district

officials such as members of the pedagogy cell and planning coordinators play secondary roles

with regard to Samiksha. They may assist with the compilation reports

or utilize data in the development of interventions targeted towards disadvantaged groups such as

girls, tribal children or CWSN.

District Inspector of Schools (DIS)

The DIS is required to monitor at least five schools, preferably upper primary, per month.

Sub-inspector of Schools (SI)

The SI is required to monitor at least five schools, preferable upper primary, per month.

Block Resource Centre Coordinator (BRCC)

The BRCC is required to monitor at least five schools, preferably upper primary, per month. He

produces a block-level compilation of Samiksha data and submits it to the District SSA office.

Cluster Resource Centre Coordinator (CRCC)

The CRCC is required to monitor at least ten schools per month. He consolidates school data into

a cluster-level Individual Compilation Format (ICF) and submits it to the BRCC for inclusion in

the block-level compilation. As the most ground-level official, the CRCC has the greatest

responsibility for ensuring school improvement.

Village Education Committee/School Management Committee (VEC/SMC)

The VEC is a working group of local community members that manages schools in a particular

village. In April 2011, the DoSME directed all schools to replace the VECs with SMCs. In

June 2011, most schools had made this transition. The SMCs, led by a President and constituted

of parents, teachers and community members, perform a similar function to the VEC.

Headmaster

The headmaster is a teacher who is the highest figure in a school. Along with the SMC President,

the headmaster has access to and makes decisions regarding school funds. The headmaster is

responsible for directing school improvement initiatives.

Teachers

Teachers, along with headmasters, are responsible for the functioning and improvement of

schools. Teachers receive feedback directly from the BRCCs and CRCCs and are held

accountable for the progress of schools.

Students

Students do not actively contribute to Samiksha but are passive beneficiaries of the system.

School improvements have the greatest impact on the education of students.

Community

The communities surrounding schools do not actively contribute to Samiksha but are passive

beneficiaries of the system through the overall improvement of the community through

education.

Figure 1: Key Samiksha stakeholders

Process

Samiksha process

Monitoring of schools

CRCCs, BRCCs, SIs, DISs, DPCs and state officials serve as school monitors. CRCCs are required to

spend a full day at each school while BRCCs, DISs, SIs and DPCs are required to spend one to two hours.

Monitors observe all aspects of the school including classroom transactions; interact with students and

teachers; and share observations with teachers, the headmaster, the VEC/SMC President and community

members. In consultation with teachers CRCCs record on the school monitoring format five specific

actions for the school to take up in the next month. The monitoring process functions like a dialogue on

school improvement between the monitor and teachers. Ultimately, this dialogue is the primary

mechanism for school improvement.

Monitors complete a school monitoring format in which they select „yes‟ or „no‟ to indicate the presence

or absence of each of the 80 indicators.21 Monitors sub-total the number of „yes‟s‟ for each of the five

categories to obtain a category score and then total the category scores to obtain an overall school score.

The number of teachers in position, number of teachers present, number of teachers in uniform, number

of classrooms available, and class and total enrolment and attendance figures are recorded. Additionally,

the format includes spaces for the following: 21 DPCs and DIs have a separate format that includes 30 of these 80 indicators. Their formats are sent to the SMU separately and are not included into the district compilation.

comments on each indicator;

school improvement activities by category for the next month;

progress made after previous monitoring event; and

remarks and suggestions by the CRCC including innovative TLMs and practices observed, important

issues raised by teachers, and five important activities to be taken up in the coming month.

The monitor leaves one copy of the completed school monitoring report with the school and retains a

second copy for use in the compilation reports. See Appendix III for a sample school monitoring format

and Appendix IV for a completed school report.

Compilation of data

The data compilation process follows the same schedule each month (see Figure 2).

1st-19th day: School monitors visit schools and complete monitoring reports.

20th day: The CRCC compiles his reports into an Individual Compilation Format (ICF) and submits the

ICF to the BRCC. The BRCC and SI compile their reports into ICFs. The DIS submits his ICF to the

DPC, and this ICF is compiled with the DPCs consolidation of his own school monitoring reports.

22nd day: The BRCC consolidates the ICFs from the CRCCs, SIs, and BRCCs into a block level

compilation, the BCF. The BRCC submits the BCF to the DPC.

25th day: All BCFs are consolidated into a district-level compilation, the DCF, which the DPC submits

to the SMU. The DIS/DPC compilation is sent separately to the SMU.

25th-7th days: The SMU consolidates all DCFs into a State Compilation Format (SCF) and submits it to

the Commissioner-cum-Secretary. The compilations of DPCs, DISs and other state officials are submitted

separately by the SMU to the Secretary.

State analysis and feedback

The SMU ranks the block and district scores, and plans to rank cluster scores in the future. The ranked

scores are classified according to colour-coded quartiles (see Table 3).

Table 3: Samiksha colour-coded ranking categories

Category

Colour

Percentage

A Green 76-100

B Purple 51-75

C Yellow 26-50

D Red 1-25

The SMU also conducts additional analysis to determine key problem areas within the 50 poorest

performing blocks and communicates this information through memos to the appropriate DPCs.

District meetings are held in which the DPC and BRCCs discuss Samiksha progress and

strategies for improvement. The BRCCs are responsible for working with CRCCs to identify

poor performing schools and improve their scores.

Figure 2: Flow of school data and feedback

Secretary SMU DIS SI DPC

BRCC Samiksha school report data

Feedback on school improvement

School

CRCC

Analysis of Samiksha process

In the section below we analyze the findings related to the functioning of the Samiksha process.

Although Samiksha has increased stakeholder awareness about the inputs and processes

necessary for school improvement, some key stakeholders still lack awareness.

The 80 indicators in Samiksha represent the qualities of an ideal school. As Samiksha has given

headmasters and teachers a structured format and guidelines of what they need to accomplish for

school improvement, they can now identify the problem areas in their school. In addition,

Samiksha has led to improvements in curricular and classroom activities as teachers have

become more aware of the inputs and processes they should use for classroom transactions. For

example, teachers now realize the importance of using TLMs, stories and other interactive

activities rather than traditional teacher-centred methods in order to foster child participation and

promote peer learning. Furthermore, the frequency and structure of Samiksha make headmasters

aware of the importance of maintaining more organized and

thorough records. This improvement in school management facilitates the further development of

schools. Even though Samiksha has raised awareness among school actors, most SMC members

were unaware of it. This is understandable as SMCs were only created in March 2011. However,

the lack of awareness among some teachers is problematic and remains a key obstacle to the

successful implementation of Samiksha.

The standardized school monitoring format facilitates the collection and analysis of school

data. Standardization of reporting formats is one of the most advantageous characteristics of any

monitoring system. Compared to other nationwide monitoring formats, the Samiksha reporting

format is particularly simple because it is centred on a relatively small number of indicators. The

„yes/no‟ format facilitates the collection, analysis and comparison of data from a large number

of schools. Monitors can easily collect data, identify problem areas and track the progress of

schools each month. In addition, schools benefit from the standardized format through

increased awareness of steps for improvement and through the systematization of the school

improvement process. Also, the format aids in undertaking analysis at different levels by

providing a common platform for evaluation.

The school monitoring format presents challenges to the implementation of Samiksha.

The monitoring format also has disadvantages that impact the broader Samiksha process. While

all monitors attended a one-time training before the initiation of Samiksha, the DoSME did not

provide detailed guidelines on how to judge individual indicators. Instead, monitors must rely on

their own judgment and make subjective decisions about the presence or absence of each

indicator. For example, when a school has only a partial boundary wall, some monitors will

choose „yes‟ to indicate that the boundary wall is present, while others will choose „no‟ to

indicate that is not present. In Cuttack, a CRCC explained that if there is one wall, he marks

„no‟, but if there are three or four walls, he marks „yes‟. In a similar case, a Keonjhar DIS

instructed BRCCs and CRCCs to mark „yes‟ even if only ten per cent of the boundary wall

existed.

For many indicators in the Curricular Programme category, monitors must evaluate classroom

situations to determine the presence or absence of an indicator. This kind of monitoring is

particularly subjective. How many children must raise their hand to indicate that “children ask

questions freely?” What level of participation is necessary to qualify as “children actively

participate in learning?” What constitutes “positive behaviours towards CWSN, girls, SC and ST

children”? For example, in a Koraput school the indicators „teacher knows every child by name

and social background‟, „students actively participate in learning‟, „students ask questions

freely‟ and „peer learning encouraged in classroom‟ were marked as

„yes‟ even though the CRCC indicated in the comments section that „some are‟. Instead of

making the monitoring process more objective, the „yes‟/‟no‟ format makes monitoring more

subjective and prone to variation.

The „yes‟/„no‟ format is also problematic for monitors when indicators are present but not

appropriately functional. For the indicator “toilets available and properly used”, some schools

in the sample had toilets available, but students were not able to use them because the facilities

were improperly maintained. For instance, in a Koraput school, the CRCC marked „yes‟ for

„toilets available and properly used‟ but indicated in the comments that they are „fully

damaged‟. Similarly, in another school in Koraput, the CRCC marked „yes‟ for child-friendly

elements but indicated in the comments section that they were not functional. By forcing

monitors to choose either „yes‟ or „no‟, Samiksha school reports may suggest a higher level of

improvement than has actually taken place.

Monitors use the comment spaces provided next to each indicator to elaborate on their decision

to select „yes‟ or „no‟. They also use the final sections of the format, including “Remarks and

Suggestions by the CRCC” and “Important Issues Raised by Teachers”, to more accurately

describe school issues. These qualitative components play an important role in structuring the

monitoring process as a dialogue on school improvement between CRCCs and teachers.

However, these comments are not taken into account in the school‟s overall score or in district,

block or cluster rankings. Additionally, because these sections are not included in the

compilation reports, they are not considered during the analysis. BRCCs, district officials and the

SMU may consequently have imperfect information on school-level realities.

The grassroots monitoring process is problematic.

Due to issues such as inaccessibility and remoteness of schools, and lack of CRCC motivation

and professionalism, CRCCs may not independently verify information thereby leading to

inaccurate reports. A lack of monitoring of CRCCs also aggravates the problem of

misrepresentation of information in school reports. Additionally, CRCCs have inadequate

training on Samiksha monitoring. Despite experience as teachers, CRCCs may not have a proper

understanding of some indicators and may lack capacity to make recommendations for the

improvement of those indicators. In Keonjhar there was a shortage of CRCC and BRCC

monitors. As a result, some schools are not monitored.

Some schools are double-counted in Samiksha compilation reports and analysis.

Some schools are monitored twice in a month by multiple officials, almost always a CRCC and a

BRCC. Because these officials do not coordinate their visits, two school monitoring reports are

completed and

included in the compilations reports. In this way, Samiksha is double-counting schools.

Twenty-eight per cent of the schools in the sample had incidences of multiple counting. In these

affected schools, 43 per cent of the total reports were repetitions. Most of the repetitions included

discrepancies of only two or three indicators. However, in some cases the discrepancies were

greater than ten indicators. In one case from Cuttack, the CRCC and BRCC visited the same

school only eight days apart, yet the BRCC reported an overall score of 44 out of 80, 22

indicators less than the CRCC‟s score of 66 out of 80. These discrepancies are problematic.

First, they reveal the subjectivity of Samiksha monitoring. In the Cuttack case, the BRCC

indicated that separate toilets were available for girls and courses were covered according to

scheme, while the CRCC indicated that there were no separate toilets for girls and courses were

not being covered according to scheme. Additionally, the double-counting of schools means that

the qualities of double-visited schools are being over-represented in compilation reports.

Finally, because BRCCs are more likely to visit easily accessible, and consequently better

performing, schools, Samiksha results may be upwardly skewed.

Measures undertaken to improve the Samiksha process

Despite the problems described above, the DoSME has already taken actions to improve the

Samiksha process.

Performance Appraisal System of CRCC: To assess grassroots-level monitoring, state officers

visit schools in an assigned education district and complete a school monitoring format intended

to assess the monitoring performance of CRCCs.

State-level meeting for BRCCs and CRCCs of the 50 lowest performing-blocks: To have a

clearer view of the problems faced by the 50 lowest-performing blocks, a state-level meeting was

organized for all of the BRCCs and CRCCs of these blocks. This meeting also helped to increase

the accountability of these stakeholders. The goal was to better understand the reasons behind

low performance in order for the DoSME to more effectively design and target interventions and

support these blocks.

Cluster-level analysis has been initiated in some districts: Some districts have initiated

analysis of cluster-level data to better identify problem areas.

SMC training at the district level: District-level training for SMC members has been initiated

to inform them of Samiksha.

Progress and Results

In the next section, we analyze the progress made in the education system since the implementation of

Samiksha. We also examine the factors that may enable and hinder this progress.

Samiksha has made the school improvement process more systematic.

Samiksha has created a state-wide flow of information and feedback. By bringing all stakeholders

together under one platform, Samiksha has improved communication between them. The monitoring

system has allowed the state to keep track of the progress of schools in all districts and identify

poor-performing districts as a first step to improve education state-wide. Furthermore, Samiksha has

made the work of schools more systematic. Schools now have a clear list of indicators that they need to

achieve according to a given timeline. The CRCC process of setting monthly priority targets helps

schools to be more systematic in making improvements.

Samiksha has made all stakeholders more accountable in fulfilling their responsibilities.

Officials are more accountable for school improvement through their role as monitors. Samiksha has also

alerted school-level actors of their duties and responsibilities and made them accountable by requiring

them to achieve monthly targets. According to state data, teacher attendance has improved (see Figure 3).

Interviewee responses support this and suggest that teacher punctuality has also improved. Teachers and

headmasters reported that since Samiksha teachers have realized their responsibility to motivate the

community and parents to send children to school. However, student regularity remains a challenge as

much depends on guardians and parents and the perceived importance of education.

Figure 3: Teacher Attendance22 80

85

90

95

100

Nov.

Dec.

Jan.

Feb.

Mar.

Apr.

Teacher Attendance (% )

Months (2010-2011) 22 Data from the DoSME Samiksha presentation, April 2011.

23 Overall 29 districts out of 30 have improved. Despite fluctuations, there is no difference between Koraput‟s

November 2010 and April 2011 score.

Samiksha has tracked school improvement progress.

Despite some variation, overall the Samiksha scores across the districts23 have improved. As depicted in

Figure 4, even though Jharsuguda started with the lowest score of 42 per cent, it increased by 23 per cent

in just six months, surpassing both Koraput and Keonjhar. Keonjhar has slowly improved by 8 per cent

whereas Koraput has remained stagnant achieving only 55 per cent. Despite a dip in February, Cuttack

has steadily increased its score to become one of the best-performing districts. We address some of the

reasons for these variations below.

Figure 4: District overall Samiksha score24 30

40

50

60

70

80

Nov.

Dec.

Jan.

Feb.

Mar.

Apr.

Samiksha Score (%)

Months (2010-2011) Cuttack Jharsuguda Keonjhar Koraput 24 Data from the DoSME Samiksha presentation, April 2011.

The 32 sample schools displayed progress in all five categories of Samiksha. As illustrated in Figure 5

even though Curricular Programme has increased the most by 32 per cent it remains the second lowest

category. This increase may be attributed to the increased awareness of school actors regarding inputs and

processes for teaching. However, insufficient teacher training presents challenges to further improvement

in this category. The rest of the categories have increased in the six months by around 20 to 25 per cent.

Regarding Co-curricular Programmes, some teachers in a school in Jharsuguda suggested that there

should be greater scope for both theoretical and practical training in order to be able to implement what

they learn in theory on the school ground. Community participation has strengthened with monthly PTA,

MTA and SMC meetings conducted regularly since they are now part of the 80 indicators that are

mandatory and are monitored by officials. The dip experienced in the School-Community Link category

in March may be partly explained by the fact that SMC were newly formed and were still in the process

of learning about their duties and responsibilities. It must be noted that all schools are not visited every

month. Thus, the number of schools included in each data point in Figure 5 varies by month. Given these

limitations, caution should be taken when interpreting the figure below.

Figure 5: Improvement of sample schools by category25 40

60

80

100

Nov.

Dec.

Jan.

Feb.

Mar.

Apr.

Samiksha Score (%)

Months (2010-2011) School Environment Curricular Programme Co-curricular Programme School-Community Link School Management Overall Score 25 Graph drawn using data available from the Samiksha reports of the sample size of 32 schools visited during fieldwork.

Factors Enabling Progress and Results

Individual motivation is critical for the effective management of resources.

Some schools are able to make positive change given the available resources. Motivated actors are able to

use innovative and resourceful practices to impact change in schools. Motivation levels may vary for

reasons such as community support, workload and personal characteristics. The following are some

examples:

Teachers can take initiatives for the improvement of their schools. A teacher in a rural Cuttack school

complained about the unavailability of raw materials required for making TLMs while another suggested

that TLMs should be provided to the teachers by the District SSA office. These examples illustrate the

contrast with a rural Keonjhar school where teachers use the Rs 500 TLM grant provided to all teachers

state-wide to procure raw material for making TLMs and students actively participate in making them.

According to state guidelines, school toilets are to be cleaned by students under the guidance and

supervision of a teacher. A teacher in an urban Cuttack school complained that parents would not let their

children clean toilets. As a result, the toilets remained dirty unless the headmistress washed them herself.

However, another school in the same district overcame this problem by discussing with parents the

hygiene and health issues associated with unclean toilets. It created a student cabinet, with a Sanitation

Committee comprising students appointed on monthly-rotation basis. The Sanitation Committee was

responsible for keeping toilets clean and teaching very young children how to use them.

The motivation of headmasters is critical for school improvement. In a Keonjhar school, which

consistently scored at or below 50 per cent, the headmaster had not reported to work for several months.

Furthermore, two schools located a few kilometres apart within the same cluster in Koraput had very

different Samiksha scores. Despite an indifferent CRCC, the presence of an active and motivated

headmaster may have enabled the better performing school to improve. Through his initiative, the

headmaster was able to involve the neighbouring community and the SMC in school improvement.

However, the poor performing school did not have strong support from the headmaster or other

stakeholders. In another instance in a Keonjhar school that lacked physical space for a playground, the

headmaster provided child-friendly toys as an alternative to child-friendly elements.

The role of parents and the community is also crucial in school performance. At a school in Cuttack,

tensions between the VEC President and the headmaster over funding and the joint control of the school

bank account impeded the construction of a much-needed new school building. However, in many cases

parents and community have a positive impact. For example, to overcome a lack of funds, an involved

SMC President in a Keonjhar school applied for and was allocated the Member of Parliament Local Area

Development (MPLAD) fund to construct a boundary wall. Members of the MTA in a Jharsuguda school

visited the school daily. They were involved in cooking the Mid-Day Meal and supervising construction

activities in the school. The group‟s intensive involvement in the school helped the school to more than

double its Samiksha score from November to April.

Local initiative is also significant in the performance of Samiksha. In Koraput, a local organization,

People‟s Group for Children‟s Development (PGCD), developed a monitoring initiative to track child

participation. This monitoring scheme was subsequently adopted by the District SSA office. During their

visits, CRCCs also have to interact with children and collect data on child participation activities such as

the school cabinet and child reporting. The district then analyzes the data with the aim of assessing and

increasing child participation in schools. Thus, in ensuring that certain critical indicators are properly

understood and fulfilled by schools, this initiative furthers the aim of Samiksha. For example, the

initiative monitors the use of the Idea Box by requiring monitors to check the suggestions given by

children and their implementation.

Factors Hindering Progress and Results

False progress captured in Samiksha score.

In some cases, Samiksha only measures the presence of certain indicators and does not reflect on their

usage. Thus, the achievement of an indicator may not indicate real success if the improvements are not

appropriately functional. This was most evident in the case of the Idea Box. In Koraput, teachers and

officials reported that the notion of student participation in school improvement was unfamiliar to the

students. As a result, students did not use the Idea Box. In most other instances across the four districts,

student submissions comprised stories and songs instead of ideas for school improvement. Additionally,

following Samiksha guidelines, almost all schools had placed dustbins in every classroom. However,

because of the absence of a well-functioning garbage collection service, a school in Keonjhar reported

emptying the school waste just outside the premises in the surrounding area, thereby undermining the

values of cleanliness and hygiene dustbins are meant to instil. Furthermore, according to Ama Vidyalaya

guidelines, a subject-wise Question Bank is to be available and hung on a designated wall in the

classrooms. While the Question Bank was available in most schools, it was kept in the headmaster‟s

office and not in apparent use. Again, while TLM corners were present in many classrooms, they were

often not stocked with adequate TLMs.

Samiksha indicators primarily focus on inputs.

The assumption behind Samiksha is that an improvement in inputs will lead to improved school quality.

A further assumption is that improved school quality will enhance student outcomes. However,

Samiksha includes only a few indirect measures of student achievement, such as “students master

learning outcomes as per Scheme” and „„written work done and regularly corrected.” While such

indicators capture some elements of student performance, they are difficult to objectively monitor.

Without linking objective measures of student outcomes to Samiksha it becomes difficult to accurately

measure the effectiveness of the school monitoring system.

School capacity to improve Samiksha indicators may be constrained by administrative issues.

Several issues related to teachers impact a school‟s capacity to improve. These include teacher training,

teacher shortage and multi-grade teaching as a result of classroom shortages. Other factors include limited

funding and resources. Some schools may not have the land to build child-friendly elements, and others

may not have access to water which may impact their ability to properly maintain toilets. In addition, lack

of inter-departmental coordination is problematic for the achievement of some indicators. For instance, in

schools where there are no toilets, the support of the Rural Works Department, which is responsible for

the construction of toilets, is needed. In schools where the condition of building infrastructure of hostels

is poor and deteriorated intervention from the Tribal Welfare Department is necessary. Similarly, the

support from the Health Department is needed to coordinate with the field-level staff to visit the schools

regularly and check the health of the children.

Important issues not accounted for in the Samiksha process nonetheless affect school ability to perform

well on Samiksha indicators.

In spite of the comprehensiveness of Samiksha‟s indicators, schools face issues that fall outside their

scope. For instance, schools in all four districts commonly cited teacher shortage as the most critical issue

facing their school. Teacher shortage affects both PTR and multi-grade teaching. However, schools are

not authorized to appoint teachers. Because teachers are such an important link in school improvement

and are directly involved with the implementation of many of the indicators, teacher shortage is an

obstacle to improving a school‟s Samiksha score.

Samiksha school performance is affected by the socio-economic conditions of the surrounding

community.

School performance is also dependent on factors external to the school system. Some Samiksha indicators

focus on classroom transactions which directly involve students, who cannot be separated from their

context. Poverty, illiteracy of parents and guardians, child work and seasonal migration for work are

responsible for student absenteeism, which directly and indirectly affects Samiksha scores. In Koraput

and Keonjhar, cultural issues such as the prevalence of child marriage and gender bias affect the

attendance and performance of children in school. Furthermore, language is a major issue in tribal areas,

creating a communication gap between teachers and ST students, particularly young students of classes I

and II. The presence of multiple tribal languages in an area, adversely affects the teaching and learning

process in the classroom and therefore, the motivation of children to attend school.

If students are missing school, their participation and achievement is adversely affected. For instance, in

Keonjhar students are irregular because they participate in work in the household and the fields and this

affects their availability and participation in the schools. In some areas of Jharsuguda, families migrated

for three to four months to work on brick construction. Children would move with their families and thus

their irregular attendance affected their learning outcomes. Moreover, schools in remote areas face several

additional challenges. Poor roads and communication networks impact teacher attendance and monitory

visits by officials.

Lessons Learned

The documentation of Samiksha provides lessons learned that apply both specifically to the

implementation of Samiksha in Orissa and more broadly to the use of monitoring systems in India and in

other contexts.

A school monitoring system is an essential way forward in the implementation of the norms and

standards of the RTE Act.

Having a monitoring system that incorporates the norms and standards of the RTE Act has helped Orissa

to assess the state of education and move towards improvement in the school system. As other states

implement the RTE Act, they may consider adopting a large-scale monitoring system like Samiksha.

Such a system could be beneficial to every state if adapted to the local context to take into account the

baseline conditions and socio-economic and demographic characteristics.

Broad scope, frequency and regularity are essential features of an effective school monitoring

system.

These characteristics have contributed to Samiksha‟s effectiveness as a school monitoring system. Given

that a large portion of Orissa‟s population lives in inaccessible areas and that remote schools are typically

excluded from review, an expansive monitoring system is necessary to ensure that all schools have a basic

level of prescribed inputs. The frequency and regularity of the monitoring process ensures that there is

minimal time lag in the availability and analysis of data. It also facilitates the state-wide tracking of

school progress.

Raising the awareness and capacity of all stakeholders is critical for a successful and sustainable

school monitoring system.

For a school monitoring system to be successful all stakeholders such as government officials, school

staff, parents and community members must recognize its importance and potential. This recognition will

motivate them to actively participate in the school improvement process and ensure the proper

implementation of the system. Furthermore, capacity-building of actors is critical for ensuring reliable

collection and analysis of data and for strengthening the system. Finally, stakeholder commitment and

involvement is necessary for the long-run sustainability of the system.

An integrated school improvement and monitoring approach is crucial for an effective school

monitoring system.

School improvement should take place both during the monitoring visits and after data analysis and

feedback. In addition to data collection the monitoring system should also be about creating spaces for

dialogue on school improvement between school-level actors and monitors. Monitors should collaborate

with school-level actors to prioritize problem areas, set targets and adopt actions for improvement. This

interaction between monitors and school-level actors is just as important to a monitoring system as

top-level data analysis and development of interventions.

Next Steps After analyzing the process and implementation of Samiksha, we recommend the following next steps.

These should guide the DoSME‟s further improvement of Samiksha.

Increase awareness about Samiksha among stakeholders.

Increased awareness of Samiksha among stakeholders will not only serve to enhance school performance

but also strengthen and institutionalize the monitoring system itself. However, most SMC members and

some teachers showed low levels of awareness about Samiksha. If made aware, these stakeholders can

provide an additional level of monitoring beyond the official monitors. Through continuous and informal

monitoring, these actors can serve exert extra pressure on schools. For example, if SMC members raise

questions regarding a school‟s provision of inputs required by Samiksha, headmasters and teachers may

feel more accountable.

Actions

CRCCs should conduct Samiksha information sessions at the school level with SMC, PTA and MTA

members.

Cluster-, block- or district-level workshops should be held for SMC members and teachers to address

knowledge gaps and provide further details about Samiksha‟s 80 indicators.

Improve capacity of monitors.

The DoSME should provide further training to CRCCs and BRCCs to improve their understanding of

their various Samiksha responsibilities. Monitors should understand how to judge the more subjective

indicators and how to coordinate school visits between them to ensure that all the schools are covered and

are not double-counted other than for verification purposes. Additionally, the BRCC and CRCC capacity

to support and guide schools towards the achievement of Samiksha indicators must be improved.

Actions

The DoSME should provide precise instructions for assessing the indicators. In addition, the DoSME

should develop and disseminate guidelines for the achievement of indicators.

The DoSME should provide training to BRCCs and CRCCs on how to analyze Samiksha school

reports to enable inter- and intra-cluster comparisons.

Ensure proper monitoring.

Given the remoteness of some clusters and schools, the monitoring of school monitors is required to

ensure that they are performing their duties as per Samiksha guidelines.

Actions

The DoSME should strengthen and regularize the existing system for monitoring of monitors. In the

monitoring, the DoSME should place special emphasis on remote clusters and schools as these are the

most affected.

The DoSME should also ensure that action is taken against fraudulent and inaccurate reporting.

Develop stronger and more systematic information and feedback flows.

The DoSME should improve the mechanism for channelling information down to lower levels. Improved

mechanisms should also be developed for communicating information on school issues to upper levels.

Actions

The process of block and cluster analysis already initiated in some districts should be regularized and

introduced into all districts such that clusters and schools receive feedback on their district and state-wise

ranking.

The cluster and block compilation report should be revised to include sections for qualitative school

information so that actors at the district level can be informed of school-level issues.

Feedback and information sharing meetings for BRCCs and CRCCs from the lowest performing blocks

should be regularized to facilitate communication among stakeholders.

Revise Samiksha continuously to ensure the ongoing improvement of schools.

Revisions to Samiksha indicators are necessary to ensure that school performance trends do not plateau.

These revisions will incrementally stretch each school towards the goal of becoming a model school.

Action

The DoSME should eliminate widely-achieved indicators and introduce more challenging ones to

ensure that schools continue to improve. For example, when the question bank is present in at least 95 per

cent of schools, the indicator “subject-wise question bank available” should be revised to “subject-wise

question bank properly used according to scheme”.

References Annual Status of Education Report Centre, Annual Status of Education Report: Overview, New Delhi.

Available at: http://www.asercentre.org/ngo-education-india.php?p=ASER+survey

Department of Education, Planning, Monitoring and Statistics Division, Ministry of Human Resource

Development, Government of India, „Selected Education Statistics 2005-2006: MHRD‟, New Delhi,

2008. Available at: http://www.educationforallinindia.com/SES2005-06.pdf

Department of School and Mass Education, Government of Orissa. Literacy and Mass Education: An

Overview. Available at: http://www.orissa.gov.in/schooleducation/index.htm

Koraput District Office, Government of Orissa, „Biju KBK Plan 2010-2011‟, 2010. Available at

http://koraput.nic.in/New/planning/plan.html

National Council of Educational Research and Training (NCERT), „Education Survey.‟ Available at:

http://www.ncert.nic.in/programmes/education_survey/index_education.html

National University of Educational Planning and Implementation (NUEPA), About DISE. 2009.

Available at: http://www.dise.in/dise2001.htm

National University of Educational Planning and Implementation (NUEPA), District Information System

for Education. Available at: http://www.dise.in/Downloads/DISE%20Flier.pdf

National University of Educational Planning and Administration (NUEPA), Government of India.

„Education for All Mid-Decade Assessment: Reaching the Unreached‟, New Delhi, 2008, pp. 26-27.

Available at: http://unesdoc.unesco.org/images/0018/001817/181775e.pdf

National University of Educational Planning and Administration (NUEPA) and Department of Social

Education and Literacy, Ministry of Human Resource Development, Government of India, „Elementary

Education in India: Progress Towards UEE‟, Flash Statistics DISE 2009-2010, New Delhi, 2011.

Available at:

http://www.dise.in/Downloads/Publications/Publications%202009-10/Flash%20Statistics%202009-10.pdf

Office of the Registrar General and Census Commissioner, Government of India, „Orissa: Data

Highlights: The Scheduled Tribes‟, Census of India 2001, New Delhi, 2001. Available at:

http://censusindia.gov.in/Tables_Published/SCST/dh_st_orissa.pdfhttp://censusindia.gov.in/Tables_Publis

hed/SCST/dh_st_orissa.pdf

Office of the Registrar General and Census Commissioner, Government of India, „Provision Population

Totals, Orissa Series‟, Census of India 2011, New Delhi, 2011. Available at:

http://censusindia.gov.in/2011-prov-results/prov_data_products_orissa.html

Orissa Primary Education Programme Authority (OPEPA), „Multilingual Education in Orissa: Issues and

Interventions.‟ Available at: http://www.opepa.in/MLE_in_Orissa.pdf

Sarva Shiksha Abhiyan (SSA), „Sarva Shiksha Abhiyan: Framework of Implementation.‟ Available at:

http://ssa.nic.in/page_portletlinks?foldername=ssa-framework

The Organization for Economic Cooperation and Development (OECD), Glossary of Key Terms in

Evaluation and Results Based Management, Paris, 2002, pp. 27-28. Available at:

http://www.oecd.org/dataoecd/29/21/2754804.pdf

Tavola, H. „Educational Planning‟, Journal of Educational Studies, Volume 30, Nos. 1 and 2, 2008, p. 3.

World Bank, „Why is it Important to Institutionalize Government M&E System?‟ Available at:

http://go.worldbank.org/ZDHJ987I80

Appendix I

Methodology

Site selection rationale

Sample selection structure

We selected four districts for the fieldwork and data collection. Within each district we selected two

blocks, within each block two clusters, and within each cluster two schools. Therefore, our total sample

consisted of 32 schools across 16 clusters in eight blocks. The structure of the sample for one district is

depicted in the figure below, and all districts follow the same pattern.26 District Block Cluster School

School Cluster School School Block Cluster School School Cluster School School 26 See Appendix V for individual district selection.

Figure 3: Sample selection structure

District profiles and selection rationale

We selected the four sample districts, Cuttack, Jharsuguda, Keonjhar and Koraput to represent a range of

geographic, demographic and socio-economic characteristics and Samiksha rankings facilitating cross-

district comparisons. As indicated in the map below, the districts were selected to represent the northern,

southern, eastern and western regions of Orissa as well as various levels of literacy rates (see Table 4).

Map 1: Map of Orissa

Cuttack: Cuttack is the administrative and commercial centre of Orissa. In 2010, the overall literacy rate

was 84.20 per cent against an overall literacy rate of 73.45 per cent for the state.27 Gender disparity in

literacy rates is relatively low compared to other districts (male: 90.51 per cent, female: 77.64 per cent).28

Overall, the district has consistently scored in the top five per cent for each month of Samiksha reporting.

The inclusion of Cuttack in the sample serves as an example of a well-performing district. 27 Office of the Registrar General and Census Commissioner, Government of India, „Provision Population Totals,

Orissa Series‟, Census of India 2011, New Delhi, 2011. Available at:

http://censusindia.gov.in/2011-prov-results/prov_data_products_orissa.html

28 Ibid.

Jharsuguda: This district lies in Orissa‟s industrial belt and is rapidly being urbanized with the expansion

of power, metal and cement plants. Consisting of only five blocks, this small district lies on the

north-western part of the state. While 17.07/31.34 per cent of the population is classified as SC/ST29,

literacy rates slightly exceed the state average (overall: 78.36 per cent; male/female: 86.27/70.05 per

cent).30 Jharsuguda ranked fifth out of 30 districts on the Samiksha report card from April. Both Keonjhar

and Jharsuguda represent average-performing districts that still face many challenges.

29 Office of the Registrar General and Census Commissioner, Government of India, „Provision Population Totals,

Orissa Series‟, Census of India 2011, New Delhi, 2011. Available at:

http://censusindia.gov.in/2011-prov-results/prov_data_products_orissa.html 30 Ibid.

31 Office of the Registrar General and Census Commissioner, Government of India, „District Profiles‟, Census of

India 2001, New Delhi, 2001. District Profiles. Available at:

http://www.censusindia.gov.in/PopulationFinder/District_Master.aspx?state_code=21

32 Ibid.

33 Office of the Registrar General and Census Commissioner, Government of India, „Provision Population Totals,

Orissa Series‟, Census of India 2011, New Delhi, 2011. Available at:

http://censusindia.gov.in/2011-prov-results/prov_data_products_orissa.html 34 Ibid.

35 Koraput District Office, Government of Orissa, „Biju KBK Plan 2010-2011‟, 2010. Available at:

http://koraput.nic.in/New/planning/plan.html

Keonjhar: The population of Keonjhar largely depends on agriculture with 86.36 per cent residing in rural

areas.31 Literacy rates are below the state average (overall: 69 per cent; male/female: 79.22/58.7 per cent),

though still significantly better than very poor districts such as Koraput. The district is located in

north-eastern Orissa and is home to 46 STs32 with SC/ST making up 11.62/44.5 per cent of the

population.33 In April, Keonjhar ranked 19th out of 30 districts on the Samiksha district report card. In

both Keonjhar and Koraput, tribal populations constitute a large portion of the rural population and are

disadvantaged in respect to education. Poor road networks, high poverty rates and language gaps for ST

populations pose significant challenges in these two districts.

Koraput: Koraput has the third lowest literacy rate (overall: 49.87 per cent; male/female: 61.29/38.92 per

cent) in the state and stands in stark contrast to Cuttack.34 Socially segregated and isolated tribal

communities make up for more than 50 per cent of the population, and more than 80 per cent of villages

include only SC/ST populations.35 Poor infrastructure and difficult communication throughout the district

are the main obstacle to improve education. While Koraput has consistently ranked in the bottom five per

cent for each month of Samiksha report, overall the district is meeting nearly 50 per cent of the indicators.

The inclusion of Koraput in the sample serves as an example of a poor-performing district.

Table 4: District profiles

District

Literacy

Rate36

Population37

Samiksha, April

2011

Overall

Male

Female

SC

ST

Rural

Urban

Overall

Score

Rank

Cuttack 84.20 90.51 77.64 19.08 3.57 72.61 27.38 72.46 2

Jharsuguda 78.36 86.27 70.05 17.07 31.34 63.53 36.47 65.13 5

Keonjhar 69.00 79.22 58.7 11.62 44.50 86.36 13.64 61.41 19

Koraput 49.87 61.29 38.92 13.04 49.62 83.19 16.81 54.82 28

36 Office of the Registrar General and Census Commissioner, Government of India, „Provision Population Totals,

Orissa Series‟, Census of India 2011, New Delhi, 2011. Available at:

http://censusindia.gov.in/2011-prov-results/prov_data_products_orissa.html

37 Office of the Registrar General and Census Commissioner, Government of India, „District Profiles‟, Census of

India 2001, New Delhi, 2001. District Profiles. Available at

http://www.censusindia.gov.in/PopulationFinder/District_Master.aspx?state_code=21

Block, cluster and school selection rationale

To facilitate the intra-district comparisons and to understand the reasons behind variations in Samiksha

scores we selected a relatively low performing block and a relatively high performing block within each

district (see Figure 4). Furthermore, we collected information from blocks and clusters with different

geographic and demographic characteristics. The final sample consists of a variation of rural and urban

schools as well as schools with different SC/ST populations. This sampling variation helped to examine

how the external factors affected schools performance in the achievement of Samiksha indicators.

Figure 4: Overall Samiksha scores for sample blocks38 69.49

61.62

62.70

55.00

57.36

59.11

70.27

52.81

0

10

20

30

40

50

60

70

80

Jharsuguda Kirmira CMC Narasinghpur Banspal Harichandanpur Koraput Semiliguda Jharsuguda

Sample Blocks Cuttack Keonjhar Koraput SamikshaScore (%)

38 Data from the DoSME Samiksha presentation, April 2011.

Participants

Although the primary unit of analysis was the school, stakeholders at all levels participated in the

study. Sample sizes for participant groups varied in the four districts based on availability and

relevance (see Table 5). We divided the participants into government officials and school-level

actors.

Government officials: At the state level, participants included the Secretary and SMU officials.

At the district level, participants included officials in the District SSA office such as the DPC,

Planning Coordinators, the DIS and others. BRCCs and CRCCs of the selected blocks and

clusters also participated in the study.

School-level actors: At the school level, participants included the headmasters and teaching staff,

SMC Presidents and members, students and other community members.

Table 5: Participants

Participant Type

Number of Participants

Government Officials Commissioner-cum-Secretary 1

State Monitoring Officer 1

District Project Coordinator 4

District Pedagogy Coordinator 2

District Tribal Coordinator 1

District Inspector of Schools 1

Sub-Inspector of Schools 2

Block Resource Centre Coordinator 5

Cluster Resource Centre Coordinator 13

School-level Actors39 Headmaster 29

SMC members 14

Teachers 24

MTA members 1

Students 18

Community members 1

39 Interviews conducted with groups of students, SMC members and MTA members are counted as one participant.

A group includes two or more individuals.

Procedure

The fieldwork was conducted over two weeks from June to July 2011. We documented and analyzed

lessons learned from the initial performance of Samiksha using qualitative research methods. To capture

issues at the policy, operational, and ground levels, we selected a broad range of methodological tools as

listed below:

1. Analysis of policy documents, census and human development reports, and Samiksha school reports,

and compilation reports;

2. Individual and group semi-structured interviews with government officials and school-level actors;

3. Observations of schools infrastructure and classroom transactions.

To analyze the lessons learned, we identified and examined recurring themes arising from our interviews

and observations. Furthermore, we used Samiksha school reports to supplement the findings.

Research constraints and limitations

Shortage of time was one of the major limitations to the research: Ten days of fieldwork limited the

amount of time spent at each school. This inhibited us from building trust and rapport with teachers,

children, and SMC members and affected the responses.

Small sample size: Given that Samiksha monitors approximately 50,000 schools in Orissa, 32 schools

are not an accurate representative sample.

Communication: Most interviews were conducted with translators. In the use of three languages, Oriya,

Hindi and English, some ideas may have gotten lost.

Limited process documentation on Samiksha: Limited process documentation was an additional

challenge to the initial development of the project.

Unforeseen challenges during the fieldwork: Various changes had to be made to the design of the

research. Although we had originally planned to verify selected Samiksha indicators during school visits,

this was not possible due to the shortage of time and other logistical challenges. In addition, we slightly

deviated from our original sample selection structure by visiting three instead of two clusters in two

blocks. Finally, even though the presence of government officials during school visits was necessary to

provide access to schools, it may have influenced the responses of participants.

Appendix II State Secretary Questionnaire

1. Why was there a need for Samiksha?

2. How did you develop the concept for the system? Was your idea influenced by national and state

policies, specifically RTE?

3. How does Samiksha differ from other planning, monitoring and evaluation mechanisms that are

institutionalized within the government?

4. What is your opinion of the progress of Samiksha since its inception? What improvements in the

sphere of education have come about as a result of the establishment of Samiksha?

Has Samiksha minimized inequities in the provision of educational services as per the specified

indicators?

5. What have been the main challenges from your standpoint in implementing Samiksha?

6. What changes and reforms to the system in the future will be needed to accomplish the original goal of

Samiksha?

Other State Level Officials Questionnaire

1. Why was there a need for Samiksha?

2. How does Samiksha differ from other planning, monitoring and evaluation mechanisms that are

institutionalized within the government?

3. How does the funding mechanism work? Do you divide the funds equally for each district, block and

school or is there some kind of formula?

4. Do you provide any additional funding to the low performing districts/blocks/schools to improve their

indicators?

5. Who is responsible for implementing the system? Who reviews the analyzes and reports and proposes

interventions?

6. How does the State communicate the results and suggests interventions to lower government

authorities such as at the district, block and cluster level? Do you target interventions to improve specific

indicators state-wide and if so how do you prioritize the indicators? Or, do you target interventions at the

regional level and if so how do you prioritize the regions that receive the interventions?

7. How have the various stakeholders such as district, block and cluster officials and schools responded

to the initiative?

8. Who is responsible for supervising and monitoring interventions at the district, block and school

levels?

9. What is your opinion of the progress of Samiksha since its inception? What improvements in the

sphere of education have come about as a result of the establishment of Samiksha? Has Samiksha

minimized inequities in the provision of educational services as per the specified indicators?

10. What have been the main challenges in implementing Samiksha?

11. What changes and reforms to the system in the future will be needed to accomplish the original goal

of Samiksha?

District Officials Questionnaire

Personal Information

1. What is your position?

2. How long have you been in this position?

3. What are your main responsibilities/duties in this position?

4. What would you say are the biggest challenges in education in Orissa and in your district?

5. How do you divide your time between serving different blocks/clusters/schools? Are some

blocks/clusters/schools easier to work with? Why?

Samiksha

Implementation

6. Are you involved in Samiksha? What is your specific role? Please describe in detail how you were

involved with Samiksha in the last months.

7. How often do you receive the score card of your district with individual-school information?

8. Who or which department is responsible for analysing the data collected from the field and suggesting

interventions? Who is responsible for implementing interventions and ensuring the improvement of

schools?

9. How do you use the monthly report to plan and target interventions at the block, cluster or school level

in order to improve your score?

10. Who are the stakeholders in Samiksha and what are their roles?

Impact

11. What improvements in the sphere of education in your specific district do you think have come about

as a result of the establishment of Samiksha?

12. How do you respond to state recommendations for improving school performance in your district?

13. How does the funding mechanism work in relation to school improvement? Do you divide the funds

equally for each block and school or is there some kind of formula?

14. Has there been a time in the last year that you did not have the resources you needed to perform your

duties? What resources were lacking? How were you able to get these resources?

15. Why is your district score good/average/bad? What are the challenges or obstacles to improve your

indicators? Are there factors (socio-economic, geographical, cultural) outside of the school system that

affect Samiksha indicators and your capacity to improve them?

16. Have you received any additional funding to improve the low performing blocks/schools of your

district?

17. Have you received help, other than financial assistance, on how to improve low-performing

blocks/schools?

18. Do you have any concrete examples (using a particular indicator, maybe?) of how the data you have

collected has influenced or changed the implementation of interventions and programmes to improve your

district‟s schools‟ performance?

Perceptions

19. In your opinion, is there need for a performance tracking system like Samiksha to monitor progress in

education throughout the state?

20. Do you think of Samiksha as an important and necessary system for ensuring progress towards

achievement of RTE?

21. How closely do you think Samiksha reflects the real issues on the ground?

22. Do you think Samiksha is able to objectively capture the progress in the schools in your district?

23. Were you consulted on the development of Samiksha? Have you been consulted regarding its

implementation?

24. Who do you think benefits most from Samiksha?

25. If you had to make changes for improvement in Samiksha, what would these changes be? Have you

expressed these recommendations/suggestions to your superior?

Other questions

26. Do you have any examples of Samiksha monitoring reports that you can share with us?

27. Could you suggest other officials who we should meet to get more information for our research?

28. Could you suggest specific schools or clusters we should visit to get a more holistic view of education

patterns in your district as reflected in your score cards?

Block and Cluster Officials Questionnaire

Personal Information

1. What is your position?

2. How long have you been in this position?

3. What are your main responsibilities/duties in this position?

4. What would you say are the biggest challenges in education in Orissa and in your district/block/cluster?

5. How do you divide your time between serving different clusters/schools? Are some clusters/schools

easier to work with? Why?

Samiksha

Implementation

6. Are you involved in Samiksha? What is your specific role? Please describe in detail how you were

involved with Samiksha in the last months. (If a monitor, do you feel the training you received was

sufficient? If you have doubts about how to answer indicators, is there someone you can consult?)

7. Who are the stakeholders in Samiksha and what are their roles?

8. Who or which department is responsible for analysing the data collected from the field and suggesting

interventions? Who is responsible for implementing interventions and ensuring the improvement of

schools?

9. Do you receive the score card of your district with individual-school information?

10. Do you use the monthly report to plan and target interventions at the block, cluster or school level in

order to improve your score?

Impact

11. What improvements in the sphere of education in your specific school/cluster/block/district do you

think have come about as a result of the establishment of Samiksha?

12. How do you respond to district recommendations for improving school performance in your

block/cluster?

13. How does the funding mechanism work? Do you divide the funds equally for each block/cluster and

school or is there some kind of formula?

14. Has there been a time in the last year that you didn‟t have the resources you needed to perform your

duties? What resources were lacking? How were you able to get these resources?

15. Why is your district score good/average/bad? What are the challenges/obstacles to improve your

indicators? Are there factors (socio-economic, geographical, cultural) outside of the school system that

affect Samiksha indicators and your capacity to improve them?

16. Have you received any additional funding to improve the low performing schools of your

block/cluster?

17. Have you received help, other than financial assistance, on how to improve them?

18. Do you have any concrete examples (using a particular indicator, maybe?) of how the data you have

collected has influenced or changed the implementation of interventions and programmes to improve your

district‟s schools‟ performance?

Perceptions

19. In your opinion, is there need for a performance tracking system like Samiksha to monitor progress in

education throughout the state?

20. Do you think of Samiksha as an important and necessary system for ensuring progress towards the

achievement of RTE?

21. How closely do you think Samiksha reflects the real issues on the ground?

22. Do you think Samiksha is able to objectively capture the progress in the schools in your block/cluster?

23. Were you consulted on the development of Samiksha? Have you been consulted regarding its

implementation?

24. Who do you think benefits most from Samiksha?

25. If you had to make changes for improvement in Samiksha, what would these changes be?Have you

expressed these recommendations/suggestions to your superior?

Others

26. Do you have any examples of monitoring reports that you can share with us?

27. Could you suggest other officials who we should meet to get more information for our research?

28. Could you suggest specific schools or clusters we should visit to get a more holistic view of education

patterns in your district as reflected in your score cards?

SMC Members Questionnaire

1. How many members are there in the committee? How many parents, teachers, others? How many

SC/ST parents? How many mothers?

2. How often do you meet?

3. What are the main issues discussed?

4. What is your decision-making process?

5. How do you improve your school?

6. What are your responsibilities in terms of financing?

7. Do you take initiatives for new programmes or do you follow directions from the district, block and/or

cluster officials?

Samiksha

8. Are you familiar with Samiksha? What do you know about Samiksha? How did you first learn about

it? What do you think is its main purpose? Do you feel Samiksha will enhance the improvement of your

school?

9. Was there any kind of orientation or training workshop held when Samiksha was implemented?

10. Do you receive your school‟s score report cards? If yes, how often?

11. Do you ask for data/info from Samiksha? If yes, how often and in what way do you use it?

12. In the past months, have you received any information from Samiksha reports that has been useful to

you? If yes, please describe it. Have the reports influenced the way and materials you use to teach?

13. How has Samiksha affected your duties/responsibilities? In the past months, have you received any

information from Samiksha reports that has made your job more challenging? If yes, please describe it.

14. Who is responsible for implementation and monitoring of district or state interventions?

Perceptions

15. In your opinion, what are the strengths and weaknesses of Samiksha? Where do you think it can

improve?

16. How closely do you think Samiksha reflects the real issues on the ground?

17. Do you think Samiksha is able to objectively capture the progress in the schools in your block/cluster?

18. What do you think are the main challenges/obstacles in improving your school performance and

reaching the RTE goals in your village?

Teachers Questionnaire

Personal Information

1. How many years of teaching experience do you have?

2. What is the highest level of education you have? Have you received any teacher training before starting

or during your years as a teacher?

3. What class do you teach? Is it a multi-grade class?

4. What are your main responsibilities/duties other than teaching?

5. Can you describe a typical day?

6. Are you satisfied with your job? Are you happy with your duty station? Was this your first choice? Do

you know the local dialect/ tribal language of the area you are stationed in?

Samiksha

7. Are you familiar with Samiksha? What do you know about Samiksha? How did you first learn about

it? What do you think is its main purpose? Do you feel Samiksha will enhance the improvement of your

school?

8. Was there any kind of orientation or training workshop held when Samiksha was implemented?

9. Do you receive your school‟s score report cards? If yes, how often? Do you ask for data or

information from Samiksha reports? If yes, how often and in what way do you use it?

10. In the past months, have you received any information from Samiksha reports that has been useful to

you? If yes, please describe it. Have the reports influenced the way and materials you use to teach?

11. How has Samiksha affected your duties/responsibilities? In the past months, have you received any

information from Samiksha reports that has made your job more challenging? If yes, please describe it.

Perceptions

12. In your opinion, what are the strengths and weaknesses of Samiksha? Where do you think it can

improve?

13. Do you think Samiksha is able to objectively capture the progress in the schools in your block/cluster?

14. How closely do you think Samiksha reflects the real issues on the ground?

Students Questionnaire

1. Why do you come to school?

2. What do you like about school?

3. What do you not like about school?

4. If you do not sometimes come to school, why?

5. What changes do you think will make school better for you?

6. How far do you live from school? How to you get to school?

7. Do you have to work after school?

Appendix III Sample Samiksha school monitoring format SAMIKSHA: ____________________________________________________________________________________

SCHOOL MONITORING SCHEDULE (For CRCC / BRCC / SI of Schools) MS-I

Instructions For CRCC Observe at least 10 schools in a month. Spend the whole day in the school, observe at least two classroom transactions and take a demonstration class. Interact with teachers and a few students. Share your observations with teachers in a cordial manner. Decide the activities to be taken up in the next month in consultation with teachers. Preferably, share your observations with the VEC President / Community Members Prepare two copies of the report: (i) one for the school, (ii) one for your office record. Compile all the monitoring outcomes of your visits in the prescribed proforma(ICF-1) before the 20thday of the month and submit the same at BRC on the next day. For BRCC / SI of Schools Observe at least 5(five) schools, preferably Upper Primary/Elementary schools in a month. Spend at least two hours in the school on the day of visit and observe at least one classroom transaction. Interact with teachers and a few students. Share your observations with teachers in a cordial manner. Decide the activities to be taken up in the next month in consultation with the teachers. Prepare two copies of this report: (i) one for the school, (ii) one for your office record. Complete your school monitoring by the 20th day of every month. Compile all the reports of your visits in the prescribed format (ICF-2) in duplicate and place one copy at the BRC by 22nd day of the month. BRCC and S.I.s of Schools of the block are to consolidate all the reports of compilation of all CRCCs, S.I.s of Schools and the BRCC in the prescribed proforma (BCF-7) in duplicate and submit one copy to the DPC of the District by 22nd day of the month.

SCHOOL MONITORING SCHEDULE Date of visit: …………………..….. Arrival: …………..… Departure: ………..…. Duration: ………………….…… hours

School Category: A / B / C / D

________________________________________________________________________________________

Summary Statement (Fill up the table after you have completed the monitoring activity) Dimension School

Environment (16)

Curricular Programme (30)

Co-curricular Programme (14)

School-Community Link (10)

School Management (10)

Total (80)

No. of “Yes” responses

Percentage

___________________________________________________________________________________________ Name of the School: …………………………………………………….. Range of Classes: I-V/I-VII/I-VIII/VI-VII/VI-VIII Cluster: ……………………………………………….… Block: …………………………………… District: …………………..……………….. No. of Teachers In position

No. of Teachers Present throughout the day

No. of teachers with uniform

No. of classrooms available

Class I II III IV V VI VII VIII Total

Enrolment

Attendance

Percentage

A. SCHOOL ENVIRONMENT Indicators Status Comments, if

any One/Two activities for the next month

Yes No Boundary wall / Fence

School compound is clean Garden Kitchen garden Drinking water Toilets available and properly used Separate toilet for girls

Information Board properly filled Child-friendly elements (slides, swings, etc.)

Ramp and Rail Classrooms named Classrooms are clean Dustbin in each classroom Children show cleanliness habits Textbooks available with every child Idea Box B: CURRICULAR PROGRAMME Features Status Comments, if any Two/ Three activities for the next

month

Yes No Teacher knows every child by name and social

background Adequate classroom /seating space Language used by teachers understood by children

Courses covered according to scheme Lesson notes maintained and used in classroom transactions

Students mastered learning outcomes as per scheme

Ground level blackboard available and used in Class I

Adequate TLMs available and used by children and teachers

TLM corner in each class Student-prepared TLMs available in the school Subject-wise Activity bank available Wall activities used for learning Textbook Practice exercises done by children Written work done and regularly corrected Students actively participate in learning Seating arrangement done as per activity Students ask questions freely Peer learning encouraged in classroom Children take up and complete project work Children’s products displayed Subject-wise Question Bank available Unit test done according to scheme

Unit Test results recorded and shared with students

Unit test results shared with parents Unit test results used to improve student performance

Progress Card regularly maintained Teaching provision to cater to the learning needs of CWSN and low achievers

Indication of positive behaviors towards CWSN, girls, SC and ST children

Library books issued to students during last month

Reference materials (handbook, manual etc) for teachers available in school .

C: CO-CURRICULAR PROGRAMME Features Status Comments, if any One/Two activities for the next month

Yes No Prayer held in time

News of the Day read in the prayer class All teachers present in the prayer class Health check up done in the last month School cabinet functional

MeenaManch / Meena Cabinet functional Meena cabinet activities recorded Physical Educational activities / Sports done as specified in time table

Work experience activities (handicrafts, clay modeling, cleaning etc.) done in school regularly.

Debates, essay, quiz, seminar/guest talk done monthly

Bi-monthly Art education facilities (dance, song, drawing etc.)

Science/ Math exhibitions done in the school during the last year

Club activities (reading club, nature club etc) conducted in school

Co-curricular performance reflected in progress card

D: SCHOOL-COMMUNITY LINK Features Status Comments, if

any One activity for the next month

Yes No VEC / SMC formed according to rule

VEC / SMC meeting held last month Resolutions of VEC / SMC meetings implemented

Activities done as per School Development Plan by VEC/ SMC

PTA meeting held during last quarter MTA meeting held last month Children’s attendance and performance discussed in last MTA / VEC / SMC meeting

VEC / SMC / Community members discuss children’s learning and performance

VEC / SMC / Community members participate in co-curricular activities

VEC / SMC monitors the utilization of grants

E: SCHOOL MANAGEMENT Features Status Comments, if

any One/Two activities for the next month

Yes No MDM served regularly

Annual Calendar of activities available and displayed

Activities done according to Annual Calendar

Head-teacher monitored the teaching of other teachers

Log Book properly maintained NTB register maintained TLM stock register maintained General Stock Register Cash Book for SIG, TLM, R & M SIG, TLM & R & M Grants properly utilized F. DEMONSTRATION LESSON BY CRCC Features Status Comments, if

any Brief description of the topic and the innovative strategy followed

Yes No CRCC took a demonstration class

Teachers were present in the demonstration class

Follow up discussion with teachers after demonstration class

G. PROGRESS MADE AFTER PREVIOUS MONITORING EVENT Monitoring done during last two months

Sl no Monitoring personnel Date Suggested activities in the last monitoring Progress made

1.

2.

3.

4.

H: REMARKS AND SUGGESTIONS BY THE CRCC Did you discuss your observations with the teachers? Yes / No Did you discuss your observations with the VEC members? Yes / No Any innovative TLMs and pedagogic practices observed. Yes/No

Innovative TLMs and practices observed Key Initiator(s)

Important issues raised by teachers:

Issues Suggestions by CRCC

1.

2.

3.

Five important activities to be taken up in the coming month

1.

2.

3.

4.

5.

Signature of all Teachers: Signature of the CRCC / BRCC / SI of Schools Date:

Appendix II Completed School Monitoring Format

Appendix V Individual District Selection

Cuttack

Jharsuguda

Keonjhar

Koraput