View
215
Download
2
Category
Preview:
Citation preview
EXECUTIVE SUMMARY
Student achievement inLatin America and the Caribbean
Student achievement inLatin America and the Caribbean
Results of the Second Regional Comparativeand Explanatory Study (SERCE)
organización de las naciones unidas para la educación, la ciencia y la cultura
united nations educational, scientific and cultural organization
organisation des nations unies pour l’éducation, la science et la culture
Regional Bureau for Education in Latin América and the Caribbean
Published by the Regional Bureau for Education in Latin America Latina and the Caribbean OREALC/UNESCO Santiago.
LLECE TeamHéctor Valdés (Coordinator), Ernesto Treviño, Carmen Gloria Acevedo, Mauricio Castro, Sandra Carrillo, Roy Costilla, Daniel Bogoya, Carlos Pardo.
Thematic AreasBeatriz Macedo, Liliana Bronzina and Ana Atorresi.
Administrative StaffSilvia Ortiz
Our special thanks go to Rosa Blanco and Ana Luiza Machado, a.i. Director and former Director of OREALC/UNESCO Santiago, respectively, and to all LLECE members who collaborated to make SERCE possible. We are particularly indebted to Javier Murillo and Marcela Román for their help in drafting the preliminary versions of this report.
Design and Lay-outAna María Baraona, Ximena Milosevic, Julia Salazar and Alejandro Urbán
English TranslationErnesto Leigh
The members of the working team are responsible for the contents of this report. The opinions expressed herein are theirs alone and not necessarily those of UNESCO.
The place names and maps used in this publication do not imply on the part of UNESCO any opinion or position in regard to the legal status of countries, cities, territories, or zones; nor regarding their authorities or the drawing of their borders. This publication may be reproduced in its entirety or in part provided that explicit reference always be made to the source.
ISBN: 978-956-8302-94-8
Santiago, Chile. June, 2008
Table of contents
PRESENTATION 7
SECOND REGIONAL COMPARATIVE AND EXPLANATORY STUDY 8
STUDENT ACHIEVEMENT 12
FACTORS ASSOCIATED WITH ACHIEVEMENT 45
FINAL REFLECTIONS 47
Executive summary 5
UNESCO has been called upon to generate, through its mandated fields of action, conditions
that guarantee individuals and communities the benefits of a genuine peace and opportunity
for development. Considering that poverty and inequality in the region continue to pose an im-
portant threat to the dignity and safety of the population, the international community should
adopt a humanised vision of development based on respect for human rights, intercultural
dialogue and the pursuit of justice. In the field of education, UNESCO has embraced three major
objectives, namely: promoting education as a fundamental human right; furthering educational
quality and innovation; and generating knowledge to inform educational policy-making.
In recent years, the Latin American and Caribbean countries have made important inroads
in the region in terms of expanding compulsory education and increasing the system’s cover-
age, designing new curricula, improving the provision of didactic materials and strengthening
school infrastructure, all of this accompanied by substantial investments in teacher training
initiatives. Nevertheless, quality of education and its equitable distribution across social groups
still remains an unresolved issue.
UNESCO’s Regional Bureau for Education in Latin America and the Caribbean proposes, from
a human-rights approach, a concept of quality that integrates five dimensions: relevance, fos-
tering learning that takes into account the developmental needs of individuals and societies;
pertinence, the need for education to be meaningful for people of different social and cultural
strata; equity, giving to all persons the aid and support that will guarantee equal opportunity to
access and complete their education, and fully develop their potential; efficacy, ensuring that
relevance, pertinence and equity-related goals translate into concrete actions; and efficiency,
the proper assignation and use of resources in the quest of the proposed objectives.
One of the central activities of the Regional Bureau is generating and disseminating
knowledge to inform decision-making on initiatives that promote educational policies and
practices aimed at strengthening the quality of education in the various countries. Within this
framework, the Latin American Laboratory for Assessment of the Quality of Education (LLECE),
founded in Mexico City in 1994, and under the co-ordination of OREALC/UNESCO Santiago,
represents a regional network of education evaluation systems committed to provide technical
support to the countries of the region. The LLECE launched its First Regional Comparative and
Explanatory Study (PERCE) during the 1995 - 1997 period releasing its results in December
1998. Subsequently, seven countries participated in a qualitative research study of the high-
performing schools identified in the First Study. The major findings of SERCE’s Second Regional
Comparative and Explanatory Study (SERCE, 2002-2008), are presented in this report. We hope
they will facilitate decision-making and foster the implementation of educational policies and
practices that make possible a faster and more assertive transition to quality education with-
out exclusion in the region.
Rosa Blanco
a.i. Director, Regional Bureau for Education in Latin
America and the Caribbean UNESCO Santiago
Executive summary 7
Presentation
Improving the quality of education is still the major challenge confronted by the education
systems of Latin America and the Caribbean. Governments strive to implement policies
that foster quality education, ensure it is made available to all, and equitably distributed,
in an attempt to break away from the social determinism that keeps the lower income
sectors –and the minorities within them– at a permanent disadvantage.
The information generated by evaluations on educational quality of national education
systems has allowed the technical and political authorities to review and analyse what is
being taught, how is being taught and, obviously, what are primary school girls and boys
learning in Latin American and Caribbean schools.
In late 2002, member countries of UNESCO’s Latin American Laboratory for Assessment
of the Quality of Education (LLECE) led by the OREALC Santiago Office, launched the Second
Regional Comparative and Explanatory Study (SERCE) which, drawing on the experience and
lessons learnt in a first such study (PERCE, 1998), took relevant steps aimed at expanding the
analysis so as to include a higher number of countries, grades and areas in its evaluations.
The main objective of the SERCE is to gather valid, accurate, and reliable data on what
are primary students actually learning, as well as relevant information on associated fac-
tors. The extent to which these results are discussed and integrated into educational and
social actions/policies aimed at enhancing and strengthening the quality of public educa-
tion in participating countries, will provide a measure of its success.
This document summarizes SERCE’s process and application, its findings and results. It
offers an outline of its purposes, the conceptual perspective used in evaluating performance
in the areas of Primary Education Mathematics, Reading and Science of students who during
the period 2005 /20061 attended third and sixth grades, major factors associated with these
results, and their implications and recommendations to social and educational policies.
The SERCE is the result of the effort and commitment of many teams, organisations
and national and regional authorities. We owe a special debt of gratitude to the persons
who headed the Organisation during the various stages of this Study, namely, Ana Luiza
Machado former OREALC/UNESCO Director and Rosa Blanco a. i. Director of the Organisa-
tion; to the World Bank, the Inter-American Development Bank, and Ford Foundation,
major donors, for their support in each one of the phases; to National LLECE Coordinators;
and to the national delegates and their teams. Our most sincere thanks to the principals,
teachers, fathers and mothers, boys and girls of participating schools, without whose
collaboration and commitment this research would have not been possible. These are
ultimately the main actors and beneficiaries of SERCE’s findings.
1 Based on the school calendar of the surveyed countries.
Student achievement in Latin America and the Caribbean8
The Second Regional Comparative and Explanatory Study (SERCE) represents the most
important and ambitious student performance evaluation project ever launched in Latin
America and the Caribbean. Under the direction and coordination of the Laboratory for
Assessment of the Quality of Education (LLECE), this Study forms part of UNESCO Region-
al Bureau for Education in Latin America and the Caribbean (OREALC/UNESCO Santiago)
global actions aimed at guaranteeing the right to quality education to all Latin American
and Caribbean students.
Its objective is to give insight into the learning acquired by Latin American and
Caribbean Third and Sixth Grade Primary Students in the areas of Mathematics, Language
(Reading and Writing) and Natural Science during their school trajectory.
Second Regional Comparative and Explanatory Study
Executive summary 9
In addition to identifying what girls and boys have learned, the results obtained are
analysed and explained keyed to factors related to students, classrooms, and schools, plac-
ing special emphasis on those factors that may be changed through the implementation of
relevant programmes and policies.
SERCE represents a collective effort on the part of participating countries, duly ar-
ticulated by a central LLECE team, and supported by a Technical Advisory Committee and
panels of experts in every area. The Study was launched in February 2004 and its main
stages will extend through the second semester of 2008.
GRAph 1 ENTITIES PARTICIPATING IN SERCE
Sixteen countries and the Mexican State of Nuevo Leon are taking part in this survey.
Third and Sixth Grade Primary Students of all participating countries were evaluated in
Mathematics and Language, while sixth graders of nine countries and the State of Nuevo
Leon were evaluated in Natural Science. A total of 3.065 schools – encompassing 4.627
Student achievement in Latin America and the Caribbean10
Third Grade classrooms and 4.227 Sixth Grade classrooms – were surveyed. This represents
a total of 100.752 Third Grade and 95.288 Sixth Grade Primary school students2. This
sample is representative of approximately eleven million third graders and ten million
sixth graders in the region.
In terms of evaluating performance and associated factors, SERCE uses a set of instru-
ments specially designed for this purpose.
Each of the evaluated students took the Mathematics, Reading and Science tests on
different days and was allotted a time consistent with the nature of the tests.
Contextual, socio-demographic, family and personal data, in addition to information as-
sociated with school processes and dynamics, were captured through the direct administra-
tion of questionnaires to students, teachers, principals, and parents of the sampled schools.
The objectives of each of these instruments are summarised in the following table.
TAbLE 1 SYNTHESIS OF SERCE’S DATA COLLECTION INSTRUMENTS
Actor Instrument Objective
Students Student Questionnaire
Inquire about the family and socio-cultural environment, classroom dynamics and interaction, degree of satisfaction with the school, classmates and teachers, among other topics.
Teachers Teacher Questionnaire
Inquire about socio-demographic aspects, professional training, labour conditions, teaching experience and degree of satisfaction with the school, among other topics.
Questionnaire on teaching practices
Look into pedagogical practices at the corresponding grade and area, such as time management, availability of educational resources, expectations teacher form of their students, types of activities, curricular implementation, evaluation strategies, among other topics.
Principals Questionnaire for Principals
Capture data relative to personals traits, professional profile and trajectory, management model adopted, expectations, degree of satisfaction with the school and co-workers, in addition to other aspects of school life.
Questionnaire on School Characteristics
Collect information on school location, equipment and infrastructure.
Parents Family Questionnaire
Inquire about the socio-demographic characteristics of the family, the availability of services and physical amenities in the home, involvement in and support of the educational process of their children, degree of satisfaction with the school, among other aspects.
2 Student sample data correspond to the total number of students who took at least one of the tests. This total differs from the total number of students evaluated in each area.
Executive summary 11
The Study shows student performance results from two different perspectives.
• On the one hand, it presents mean scores of students’ and their variability by
country, areas and grades. Also, the relationship between average scores, national
per capita income and Gini Index, for each country.
• On the other, it shows results based on student distribution at each national level
of performance. This information gives a clear idea of the percentage of students
who have similar performance profiles in each country.
The First Report on SERCE’s Results includes a progress report on achievement-associ-
ated factors that provides a preliminary insight into the variables that have an impact on
student learning.
Student achievement in Latin America and the Caribbean12
Evaluation of Learning - Approaches
In order to evaluate student performance, SERCE used tests based on curricular elements
known to be common to the region, fashioned after the life-skills approach propounded
by UNESCO.
The creation of a common and consensuated curricular framework for Latin America
and the Caribbean implied reviewing, systematising and analysing contents prescribed by
the curricula for the different areas to be evaluated in the region, in order to determine
Student achievement
Executive summary 13
which conceptual domains were common to the primary education students of participat-
ing countries3.
The identification of common contents, the approaches used by participating countries
to evaluate their students’ performance, and the organisation of this performance, were
the criteria guiding the curricular analysis on which the elaboration of tests was based.
For its part, the life-skills approach establishes the abilities, principles, values and
attitudes that Latin American students should learn to develop in order to ensure their
full and active participation in society, both as actors and citizens. This means, dealing
with situations, making decisions based on available information, solving problems, and
supportting their points of view, among others.
Designing tests that inspired on a common curricular framework also place emphasis
on life-skills, challenges education to go beyond academic success by offering students
learning spaces that promote and ensure a better quality of personal and social life.
The tests administered by SERCE evaluate not only the knowledge acquired by Third
and Sixth Grade primary education students, but also how these students use – or are
capable of using – such knowledge to understand and interpret the world under various
daily-life circumstances and contexts.
The questions administered during the tests were distributed into six different book-
lets, thus ensuring coverage of all domains contained in the test reference framework.
The inclusion of open-ended questions, allowed students to construct their own re-
sponses and, based on that construction, the strategies used by the students to respond
could be inferred. This type of question also provided insight into the degree to which the
students have acquired attitudes, values and procedures, and developed their own ways
of thinking.
The questions asked vary substantially in terms of how the information is presented:
• Some questions present information as written texts, whereas in others, the infor-
mation is contained in tables, narratives, graphs or drawings.
• Content is also presented in everyday contexts that are familiar to the students, as
a way of highlighting the functionality and usefulness of this learning.
In order to determine what do Latin American and Caribbean students know, two di-
mensions were conceived: conceptual domains or area-specific knowledge, and cognitive
processes, understood as the mental operations students use to establish relationships
with and among objects, situations and phenomena.
3 Guatemala joined the study after the curricular analysis had been completed. Therefore, there is no guarantee that Guatemala’s curricular content will fully agree with the contents selected for the tests.
Student achievement in Latin America and the Caribbean14
TAbLE 2 CONCEPTUAL DOMAINS AND PROCESSES INVOLVED IN EACH SERCE TEST
Area Conceptual Domains processes
Mathematics Numerical • Recognition of elements and objects • Solution of simple problems • Solution of complex problems
Geometrical
Measurement-based
Information handling skills
Variational
Reading Length of the tested text
• General processes • Processes related to specific texts • Metalinguistic processes
Text type and genre
Natural science Health and living beings • Concept recognition • Interpretation and application of concepts• Problem solving
Earth and environment
Matter and energy
The following are some examples of the items used:
Executive summary 15
Level I - 3rd Grade Mathematics
Example 1. Books sold per month
Summary card for example 1Grade 3rd GradePerformance level IDomain Information handling skillsProcess Recognition of objects and elementsRequired action /task Interpreting direct information presented in a
bar graph Key A: January Difficulty level 412.02Percentage of correct responses 75.64%Percentage of responses involving distractors B: 9.01%
C: 6.16%D: 6.02%
Percentage of non-valid responses 3.17%
Student achievement in Latin America and the Caribbean16
Level IV - 3rd Grade Mathematics
Example 2. Number sequence
Summary card for example 2Grade 3rd Grade Performance level IVDomain VariationalProcess Solving complex problemsRequired action /task Recognise an additive numerical sequence rule
by its definitionKey C: 300 units were added each timeDifficulty level 629.06Percentage of correct responses 30.45%Percentage of responses involving distractors A: 24.95%
B: 21.19%D: 15.98%
Percentage of non-valid responses 7.43%
Executive summary 17
Level II - 6th Grade Reading
Example 3. The perfect horse
Summary card for example 3Grade 6th GradePerformance level IIDomain Length: Complete text
Type of text and genre: Narrative; introduction-climax-resolution
Process General: Identifying secondary informationSpecific: Identifying “voices” in the narrative Metalinguistic: None
Required action /task Recognising a character’s attribute based on the saying of a third party
Key B: smartDifficulty level 436.69Percentage of correct responses 74.95%Percentage of responses involving distractors
A: 5.45%C: 6.09%D: 11.28%
Percentage of non-valid responses 2.23%
Student achievement in Latin America and the Caribbean18
Level IV- 6th Grade Reading
Example 4. Title and passages of a narrative
Summary card for example 4Grade 6th GradePerformance level IVDomain Length: A relatively lengthy text.
Type of text and genre: Explanatory/narrative: legend Process General: Associating a synthesis with that synthesised
Specific: Identifying which part of the narrative text is synthesised in the titleMetalinguistic: Knowing the meaning of “title” and the different names of the passages
Required action /task Identifying which part of the text is synthesised in the title, distinguishing them from other passages through the use of metalanguage
Key B: The conflictDifficulty level 599.623Percentage of correct responses 35.77%Percentage of responses involving distractors
A: 22.99%C: 18.67%D: 18.95%
Percentage of non-valid responses 3.62%
Executive summary 19
Level II - 6th Grade Science
Example 5. Balanced breakfast
Summary card for example 5Grade 6th GradePerformance level IIDomain Health and living beingsProcess Recognising and applying concepts Required action /task The student should be able to recognise the concepts
involved and apply them to a familiar and daily situation Key A: Fruit, milk and breadDifficulty level 495.60Percentage of correct responses 56.18%Percentage of responses involving distractors
B: 31.29%C: 5.73%D: 5.77%
Percentage of non-valid responses 1.03%
Student achievement in Latin America and the Caribbean20
Level IV - 6th Grade Science
Example 6. The Moon
Summary card for example 6Grade 6th GradePerformance level IVDomain Earth and environment Process Problem Solving Required action /task Handling concepts related to the force of gravity and their
correct application to solve the problem at handKey D: there is little gravitional forceDifficulty level 822.45Percentage of correct responses 18.37%Percentage of responses involving distractors
A: 30.39%B: 13.93%C: 33.97%
Percentage of non-valid responses 3.34%
Executive summary 21
presentation of ResultsResults are presented, by grade and area, as follows:
• Average scores and variability for each of the countries, based on an arbitrary scale
with a mean of 500 and standard deviation of 100. This is meaningless in terms of
approving/not approving grade.
• Performance levels which classify students on the basis of what they are capable
of doing.
• Comparisons of students in urban and rural contexts, and gender-based analysis.
• Relationship between learning results, per capita gross domestic product of each
country and income distribution, using the Gini Index.
Learning in Third Grade
MathematicsMathematics results for Third Grade students reveal significant differences among
countries. Countries situated at the high and low ends of the performance scale are sepa-
rated from each other by more than 250 points, equivalent to more than 2.5 standard
deviations. However, a comparison between the second and next to last countries reveals
a difference of approximately one standard deviation. This implies that there is greater
homogeneity among countries occupying mid-positions.
Based on a global analysis of results, countries may be classified in five groups accord-
ing to their difference with the countries’ average:
• Countries that exhibit mean scores in Mathematics, significantly higher than the
regional average (more than one standard deviation). This, however, is only true
for Cuba.
• Countries that exhibit average scores higher than the regional average (but less
than one standard deviation): Chile, Costa Rica, Mexico and Uruguay, and the
Mexican State of Nuevo Leon.
• Countries matching the regional average, that is, cases where no statistically sig-
nificant differences are evident. This group is comprised of Argentina, Brazil and
Colombia.
• Countries that exhibit mean scores in Third Grade Mathematics lower than the
regional average (less than one standard deviation): Guatemala, Ecuador, El Salva-
dor, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic4*.
4* Significant differences (5% error) based on a t test for median comparison.
Student achievement in Latin America and the Caribbean22
At the regional level, the difference in performance in Third Grade Mathematics between
10th and 90th percentiles is 241 points, with extreme values of 165 points (Nicaragua) and
341 points (Cuba).
GRAph 2 MEAN AND VARIABILITY OF THIRD GRADE MATHEMATICS SCORES IN EACH
SURVEYED COUNTRY
LAC total: Latin American and Caribbean countries’ total.CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05).Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the dis-tance between these two points, the greater the students’ performance variability.The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.
Along with Cuba, Paraguay and Brazil exhibit the greatest differences between their
10th and 90th percentiles, with 258 and 245 points, respectively.
For their part, Panama, El Salvador and Guatemala exhibit the smallest differences
(fluctuating around 180 points) between their 10th and 90th percentiles.
Executive summary 23
TAbLE 3 DESCRIPTION OF THIRD GRADE MATHEMATICS PERFORMANCE LEVEL AND
PERCENTAGE OF STUDENTS OCCUPYING EACH LEVEL
LevelCut-off
score% Students Description
IV
621.68
11.23% • Students recognise a numerical sequence rule and identify it. • Students solve multiplication problems with one unknown
or problems which require the use of equivalences between commonly used measures of length.
• Students identify an element on a bi-dimensional plane and the properties of the sides of a square or rectangle in order to solve a problem.
III
558.54
14.30% • Students solve multiplication problems or problems which require the use of an addition equation or two separate operations.
• Students solve addition problems involving measurement units and their equivalences or problems which require using common fractions.
• Students must identify the graphic or addition numerical se-quence rule being used in order to continue it.
• Students identify the elements of unusual geometrical shapes and interpret different types of graphs in order to retrieve information and solve problems that involve operating with the data.
II
489.01
28.26% • Students recognise the organisation of the decimal-positional nu-meral system and identify the constituent elements of geometrical shapes.
• Students identify a trajectory on a plane and the most suitable measurement unit or instrument, in order to measure a known object’s attribute.
• Students interpret tables and charts in order to obtain informa-tion and compare data.
• Students solve addition or multiplication problems involving proportional relationships, using natural numbers.
I
391.50
36.03% • Students recognise the relationship between natural numbers and common bi-dimensional geometric shapes in simple drawings.
• Students locate relative positions of an object in a spatial repre-sentation.
• Students interpret tables and graphs in order to obtain direct information.
Below I 10.19% • Students at this level have not been able to acquire the abilities required in Level I.
As shown in Table 3, 10.2% of all students are not capable of completing the tasks de-
signed for the lowest level. This group of girls and boys – which total over a million for all the
countries surveyed – demands urgent and appropriate help given their low levels of learning.
Table 4 shows student distribution by performance levels for each participating country.
Cuba exhibits the highest performance levels, with 54.36% of its students occupying
Level IV.
In Chile, Costa Rica, Mexico, Uruguay and Nuevo Leon, over a third of their students
occupy Levels III and IV.
Student achievement in Latin America and the Caribbean24
In Brazil and Argentina one fourth of their students occupy Levels III and IV. In the
rest of the countries less than one fourth of the surveyed students placed at these levels.
In the case of the Dominican Republic, 41.28% of the country’s students are below
Level I, a figure that is considerably lower (between 14% and 16%) for students of Ecua-
dor, Panama, Paraguay and Peru.
TAbLE 4 PERCENTAGE OF THIRD GRADE STUDENTS BY PERFORMANCE LEVEL IN
MATHEMATICS IN EACH SURVEYED COUNTRY
Country below I I II III IVArgentina 10.46 32.77 31.13 15.17 10.47Brazil 10.32 36.55 26.74 14.32 12.07Chile 5.10 27.90 33.60 19.37 14.02Colombia 8.57 38.60 33.19 12.97 6.67Costa Rica 2.62 24.44 37.00 22.30 13.65Cuba 1.09 10.19 16.95 17.41 54.36Ecuador 14.34 45.48 28.12 7.91 4.14El Salvador 10.31 45.00 31.80 9.25 3.64Guatemala 17.34 50.06 25.07 5.46 2.08Mexico 5.15 28.85 30.70 19.71 15.59Nicaragua 12.10 47.95 30.50 7.49 1.97Panama 15.98 49.69 25.15 6.42 2.75Paraguay 15.87 37.88 25.50 11.56 9.20Peru 15.24 45.42 25.95 8.61 4.77Dominican Rep. 41.28 49.27 8.49 0.84 0.13Uruguay 5.78 25.95 30.03 19.29 18.95Nuevo Leon 2.34 18.45 31.69 24.41 23.11
Total 10.19 36.03 28.26 14.30 11.23
Note: Below I students are those who cannot attain level I.
School location is also responsible for the differences in student performance levels
observed in the region. Table 5 shows that Latin American and Caribbean girls and boys
attending rural schools perform at lower levels when compared to their counterparts at-
tending urban schools54. In this sense, Peru, Brazil, and Mexico exhibit the largest urban-
rural gaps. Cuba, Nicaragua, and Paraguay do not reveal statistically significant differences
in the averages obtained by urban and rural students.
4 The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean represent a rough measure that, given the various definitions of rural-ity, should be taken with caution.
Executive summary 25
TAbLE 5 AVERAGE SCORE DIFFERENCES BETWEEN URBAN AND RURAL SCHOOLS, AND
BY GENDER. THIRD GRADE MATHEMATICS
Country Urban/ Rural Difference Girl/ boy DifferenceArgentina 40.09* -1.42Brazil 62.17* 1.91Chile 33.29* -13.37*Colombia 26.29* -8.26*Costa Rica 29.25* -10.80*Cuba 7.79 4.47Ecuador 20.70* 0.55El Salvador 39.92* -10.90*Guatemala 40.07* -6.98*Mexico 43.01* 0.09Nicaragua -1.15 -12.72*Panama 22.41* 5.53Paraguay 17.91 2.31Peru 69.88* -9.20*Dominican Rep. 17.60* 12.66*Uruguay 31.72* 0.28Nuevo Leon 28.68* -3.92
Total - -1.25
* Significant at a 5% confidence level.
In terms of gender, at the regional level, median scores for Third Grade Mathematics
do not reveal significant differences. However, this overall result conceals important dif-
ferences among countries:
• In Argentina, Brazil, Cuba, Ecuador, Mexico, Panama, Paraguay, Uruguay and the
Mexican State of Nuevo Leon, gender-based differences are not significant.
• Chile, Colombia, Costa Rica, El Salvador, Guatemala, Nicaragua and Peru exhibit
significant differences which would seem to indicate that boys outperform girls in
Mathematics.
• Exceptionally, in the Dominican Republic the opposite is true.
Analyses of the existing relationship between performance and gross national product
and income distribution reveal interesting differences among countries.
There is a correlation between student average score in Third Grade Mathematics and
their national per capita GDP. In fact, this economic indicator accounts for 28.37% of the
countries’ average performance variance.
The relationship between results and the Gini Index –as income distribution indica-
tor– is equally significant, although inverse. In other words, the greater the inequality
Student achievement in Latin America and the Caribbean26
the lower the results obtained in Third Grade Mathematics. The Gini Index can account for
17.06% of the countries’ average performance variance in Mathematics.
ReadingAs with Mathematics, important differences are evident among participating countries.
Thus, the difference between the countries with highest and lowest performances is 2.3
standard deviations, that is to say, about 230 points. However, the difference between the
second and next to last country is 1.15 standard deviations, which reveals a somewhat
more homogeneous distribution of results.
Based on the high disparity and internal dispersion of national averages, five groups of
countries were identified relative to the average performance of their students.
• Countries where average performance is markedly higher than the median of SERCE
participants by more than one standard deviation. This case is illustrated by
Cuba.
• Countries where average performance is higher than the average of SERCE partici-
pants by less than one standard deviation. This group is comprised of Argentina,
Chile, Colombia, Costa Rica, Mexico, Uruguay, and the Mexican State of Nuevo
Leon.
• Countries where performance exhibits a mean score statistically identical to the
regional average: Brazil and El Salvador.
• Countries where performance exhibits a lower score than the average of SERCE
participants of less than one standard deviation. This is the case of Ecuador, Gua-
temala, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic6*.
6* Significant differences (5% error) based on a t test for median comparison.
Executive summary 27
GRAph 3 MEAN AND VARIABILITY OF THIRD GRADE READING SCORES IN EACH
SURVEYED COUNTRY
LAC total: Latin American and Caribbean countries’ total.CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05).Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the dis-tance between these two points, the greater the students’ performance variability.The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.
In terms of Third Grade Reading tests, performance differences between percentile
10th and percentile 90th students in each country fluctuate between 208 and 242 points,
with the exceptions of Cuba and Nicaragua.
• Cuba shows the greatest dispersion of results, since the distance between students
of the percentiles under comparison is 295 points. However, the lower performing
Cuban students obtain scores that are similar to the countries’ average.
• For its part, Nicaragua shows a low dispersion of results with differences between
students at both extremes that scarcely exceed 183 points.
• Guatemala, Peru and El Salvador, show a moderate dispersion with differences
between extremes fluctuating in the 208 - 220 point range.
• The twelve remaining countries reveal differences between their first and last
deciles in the 224 and 241 point range.
Student achievement in Latin America and the Caribbean28
TAbLE 6 DESCRIPTION OF READING PERFORMANCE LEVELS OF THIRD GRADE STUDENTS
LevelCohort score
% students Description
IV
637.49
8.41% • Integrate and generalise information given in a paragraph or in the verbal codes and graph;
• Replace non-explicit information; • Read the text identifying new information; • Translate from one code to another (from numeric to verbal, and
verbal to graphic)
III
552.14
21.63% • Locate information discriminating it from adjacent information; • Interpret reformulations that synthesise several data; • Infer information based on knowledge about the world; • Discriminate, based on the text, the meaning of words that have
several other meanings.
II
461.32
37.74% • Locate information in a brief text that must no be distinguished from other conceptually similar information;
• Discriminate words with a single meaning; • Recognise simple sentence reformulations; • Recognise redundancies between graphic and verbal codes
I
367.36
25.51% • Locate information with a single meaning, in a prominent part of the text, repeated literally or synonymously, and isolated from other information.
Below I 6.71% • Students at this level have not been able to acquire the abilities required in Level I.
In terms of reading achievement, 6.7% of the total number of Third Grade Primary
Education students in the region scored below Level I. This means that students failed
to locate information, with a single meaning, which is repeated in the text and isolated
from other information. Table 7 shows performance levels by country, and confirms the
fact that:
• 44.3% of Third Grade Cuban students scored the highest in Reading, followed by
students of Nuevo Leon (18.4%), Costa Rica (18.2%), and Chile (17.8%).
• 31.4% of the students of Dominican Republic scored below Level I, similarly to
more than 14% of the ones of Ecuador and Guatemala, and approximately 11% of
those of Panama and Paraguay.
In terms of Reading, rural school students participating in SERCE obtained lower scores
than their counterparts attending urban schools75. This is shown in Table 8 where the differences
in the results obtained by urban school students versus rural school students are described.
5 The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean represent a rough measure that, given the various definitions of rurality, should be taken with caution.
Executive summary 29
TAbLE 7 PERCENTAGE OF THIRD GRADE STUDENTS BY READING PERFORMANCE LEVEL
IN EACH SURVEYED COUNTRY
Country below I I II III IVArgentina 6.26 22.01 39.73 23.63 8.37Brazil 6.29 25.25 39.84 21.54 7.07Chile 1.60 9.97 34.46 36.22 17.76Colombia 4.94 23.61 41.78 21.16 8.52Costa Rica 1.46 10.40 34.20 35.73 18.22Cuba 0.56 6.48 21.09 27.61 44.27Ecuador 14.62 37.47 34.20 11.61 2.10El Salvador 5.34 29.05 41.05 19.15 5.40Guatemala 14.37 43.18 32.04 8.51 1.91Mexico 3.65 19.64 37.09 27.52 12.09Nicaragua 6.95 37.29 43.38 10.69 1.70Panama 11.21 37.24 35.29 12.35 3.91Paraguay 11.47 37.85 32.27 12.92 5.49Peru 9.24 36.18 35.79 15.13 3.65Dominican Rep. 31.38 46.73 18.04 3.29 0.56Uruguay 4.69 19.96 39.02 24.94 11.39Nuevo Leon 1.70 12.71 34.82 32.40 18.38
Total 6.71 25.51 37.74 21.63 8.41
Significant differences in Reading results obtained by Third Grade students attending
urban and rural schools are evident in Latin America and the Caribbean.
• Peru exhibits the greatest differences –over 79 points– in terms of rural versus
urban school results. The country is followed by Guatemala, Brazil and Mexico with
differences that fluctuate between 62 and 64 points.
• Cuba and the Dominican Republic show the smallest differences between rural and
urban schools – 16 and 19 points, respectively.
Reading results also reveal marked differences in gender. Overall, among SERCE partici-
pants, girls obtained the highest Third Grade Reading scores. In fact, girls outperformed
boys by an average of 12.7 points.
• Argentina, Brazil, Cuba, Mexico, Panama, Paraguay, the Dominican Republic, Uru-
guay and the Mexican State of Nuevo Leon, show significant differences between
boys and girls in terms of Reading scores.
• The rest of the countries show no statistically significant differences when making
gender-based comparisons.
Reading performance of Third Grade Primary Education students shows a direct correla-
tion with the gross national product of each country. In particular, differences in national
per capita GDP, account for one third of the variability observed in national performance
averages.
Student achievement in Latin America and the Caribbean30
The greater the income distribution inequality, the lower the average Reading perfor-
mance observed among Third Grade students. The Gini Index, for its part, accounts for
12.6% of the variability detected in the national performance median.
TAbLE 8 DIFFERENCE IN AVERAGE SCORES BETWEEN URBAN AND RURAL SCHOOLS, BY
GENDER. THIRD GRADE READING
Country Urban/ Rural Difference Girl/ boy Difference Argentina 34.53* 17.74Brazil 62.67* 18.57Chile 34.68* 2.46Colombia 50.92* 4.64Costa Rica 41.24* 4.69Cuba 15.94* 13.34Ecuador 42.83* 9.02El Salvador 57.29* 1.39Guatemala 64.07* 1.67Mexico 62.47* 13.20Nicaragua 29.42* 1.77Panama 54.70* 14.94Paraguay 36.45* 15.69Peru 79.30* 0.76Dominican Rep. 19.45* 13.05Uruguay 28.56* 12.73Nuevo Leon 37.24* 9.05
Total - 12.74
* Stands for 5% confidence level.
Executive summary 31
Learning in Sixth Grade
MathematicsThe analysis of Sixth Grade Mathematics mean scores shows marked differences. The
difference between the average scores of highest and lowest performing countries (Cuba
and the Dominican Republic, respectively) reaches 220 points, that is to say, more than 2
standard deviations. However, the difference between the second and next to last coun-
tries is 1.26 standard deviations.
GRAph 4 MEAN AND VARIABILITY OF SIXTH GRADE MATHEMATICS MEDIAN SCORES IN
EACH SURVEYED COUNTRY
LAC total: Latin American and Caribbean countries’ total.CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05).Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the dis-tance between these two points, the greater the students’ performance variability.The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.
Student achievement in Latin America and the Caribbean32
Based on an overall analysis of results, countries may be divided into four groups,
relative to their difference with the countries’ average:
• Countries where Mathematics Sixth Grade students exhibit a higher average per-
formance than the regional average, at one standard deviation above this average.
Cuba, with an average of 637 points, is part of this first group.
• Countries that exhibit mean scores above the regional average, but less than one
standard deviation. Uruguay, the Mexican State of Nuevo Leon, Argentina, Chile,
Costa Rica and Mexico form part of this group.
• Countries with average performances equal to the average of all participating
countries, in other words, where no statistically significant differences between
these two average values are evident. Brazil, Colombia and Peru belong in this
group.
• Countries that exhibit mean scores below the countries’ average (less than one
standard deviation): Ecuador, El Salvador, Guatemala, Nicaragua, Panama, Para-
guay and the Dominican Republic8*.
An analysis of student performance variability can shed light on the inequality of
education. Within the region, the difference between average scores in the 10th and 90th
percentiles represents 242.6 points. Disaggregating by countries, we find differences be-
tween the 10th and 90th percentiles that fluctuate between 182 and 385 points. On this
basis, four groups of nations can be established:
• In the Dominican Republic, Nicaragua, El Salvador, Panama and Guatemala, the
range of dispersion between these percentiles is less than 200 points.
• In Colombia, Paraguay, Brazil, Costa Rica, Argentina, Ecuador, Chile and the Mexican
State of Nuevo Leon, variability between the 10th and 90th percentiles fluctuates
between 200 and 250.
• Mexico, Peru and Uruguay exhibit a performance dispersion range above 250 points
but below 300 points.
• Cuba’s internal variability exceeds 300 points.
8* Significant differences (5% error) based on a t test for median comparison.
Executive summary 33
TAbLE 9 DESCRIPTION OF MATHEMATICS PERFORMANCE LEVELS OF SIXTH GRADE STUDENTS
LevelCut-off
Score% students Description
IV
624.60
11.44% • Students find averages and do calculations using the four basic operations in the filed of natural numbers.
• Students identify paralleIism and perpendicularity in a real situation and the graphic images of a percentage.
• Students solve problems involving properties of angles, triangles and quadrilaterals as part of different shapes, or involving operations with two decimal number
• Students solve problems involving fractions.• Students make generalisations in order to continue a complex
graphic sequence pattern.
III
514.41
32.35% • Students compare fractions, use the concept of percentages when analysing information and solving problems that require this type of calculation.
• Students identify parallelism and perpendicularity on a plane, as well as bodies and their elements without the benefit of graphic support.
• Students solve problems that require interpreting the constituent elements of a division or equivalent measures.
• Students recognise central angles and commonly used geometrical shapes, such as circles, and resort to their properties for solving problems.
• Students solve problems involving areas and perimeters of triangles and quadrilaterals.
• Students make generalisations in order to continue a graphic sequence or find the numerical sequence rule that applies to a rela-tively complex pattern.
II
413.58
40.82% • Students analyse and identify the structure of the positional decimal number system, estimate weight (mass) expressing it in units con-sistent with the attribute being measured.
• Students recognise commonly used geometrical shapes and their properties in order to solve problems.
• Students interpret, compare and work with information presented through various graphic images.
• Students identify the regularity of a simple pattern sequence.• Students solve addition problems in different numerical fields
(natural numbers, decimals) including commonly used fractions or equivalent measures.
• Students solve multiplication or division problems, or two natural number operations, or operations that include direct proportionality relations.
I
309.64
13.91% • Students arrange natural numbers (up to 5 digits) and decimals (up to thousands) in sequence.
• Students recognise common geometrical shapes and the unit consis-tent with the attribute being measured.
• Students interpret information presented in graphic images in order to compare it and change it to a different form of representation.
• Students solve problems involving a single addition using natural numbers.
Below I 1.48% • Students at this level have not been able to acquire the abilities required in Level I.
Student achievement in Latin America and the Caribbean34
Table 10, shows that nearly 75% of Cuban and Uruguayan students are placed in Levels
III and IV, exhibiting the highest performances for Mathematics achievement. More than
50% of Sixth Grade students in Nuevo Leon, Costa Rica, Mexico and Chile, attained the
highest performance levels in Mathematics. On the other hand, between 50% and 60%
of all surveyed students in Argentina, Brazil, Peru, Colombia and Paraguay, performed at
Levels I and II. This is also true for more than 70% of the surveyed students in Ecuador,
El Salvador, Guatemala, Nicaragua, Panama and the Dominican Republic.
TAbLE 10 PERCENTAGE OF SIXTH GRADE STUDENTS BY MATHEMATICS PERFORMANCE
LEVEL IN EACH SURVEYED COUNTRY
Country below I I II III IVArgentina 1.53 11.89 37.99 36.26 12.34Brazil 1.46 14.00 44.09 31.65 8.80Chile 1.40 9.84 37.85 37.39 13.52Colombia 1.02 13.29 47.64 32.60 5.46Costa Rica 0.09 4.55 32.71 43.70 18.95Cuba 0.19 4.43 17.93 26.33 51.13Ecuador 4.24 24.86 45.15 21.41 4.34El Salvador 1.95 19.18 51.61 23.81 3.45Guatemala 2.78 24.94 50.80 19.52 1.96Mexico 0.51 8.38 32.41 39.10 19.60Nicaragua 2.25 23.88 52.69 19.41 1.76Panama 3.32 27.16 49.55 17.64 2.33Paraguay 3.85 21.00 46.50 23.91 4.74Peru 2.41 19.58 39.82 28.90 9.29Dominican Rep. 5.69 41.79 45.43 6.85 0.24Uruguay 0.67 4.26 22.36 40.41 32.31Nuevo Leon 0.34 6.29 29.35 40.66 23.36
Total 1.48 13.91 40.82 32.35 11.44
As shown in Table 11, Sixth Grade rural school students participating in SERCE obtain
lower scores in Mathematics than their counterparts attending urban schools.
Executive summary 35
TAbLE 11 DIFFERENCE IN AVERAGE SCORES BETWEEN URBAN AND RURAL SCHOOLS, BY
GENDER. SIXTH GRADE MATHEMATICS
Country Urban/ Rural Difference Girl/ boy Difference Argentina 40.21* -5.79Brazil 42.74* -10.02*Chile 36.51* -6.84*Colombia 29.03* -14.53*Costa Rica 23.34* -20.67*Cuba 4.98 8.24*Ecuador 42.81* 0.29El Salvador 44.76* -9.48*Guatemala 38.39* -6.91*Mexico 51.42* 6.35Nicaragua 10.24* -10.16*Panama 37.33* 2.81Paraguay 31.18* -0.59Peru 87.03* -18.94*Dominican Rep. 9.01* 0.96Uruguay 52.45* 0.18Nuevo Leon 35.79* 0.27
Total - -6.17*
* Stands for 5% confidence level
Latin American and Caribbean Sixth Grade students attending urban schools outper-
form rural school students in Mathematics.
• In terms of urban versus rural school results, Peru shows the largest gaps exceed-
ing an 87 point difference, on average. Uruguay and Mexico follow with differences
in the neighbourhood of 52 points.
• By contrast, Cuba and the Dominican Republic show the smallest differences be-
tween rural and urban schools (5 and 9 points, respectively).
A SERCE’s gender-based analysis reveals that, at the regional level, boys score some 6
points higher than girls in Sixth Grade Reading tests. Furthermore, based on other impor-
tant differences detected among countries, three groups can be established:
• A first group is comprised of Argentina, Ecuador, Mexico, Panama, Paraguay, the
Dominican Republic, Uruguay, and the Mexican State of Nuevo Leon, where no
statistically significant differences in terms of performance of girls and boys were
detected.
• A second group includes Cuba, where girls obtained significantly higher scores
than boys.
Student achievement in Latin America and the Caribbean36
• Finally, a group of countries where average performance is skewed in favour of
boys. These countries are: Brazil, Chile, Colombia, Costa Rica, El Salvador, Guate-
mala, Nicaragua and Peru.
On the other hand, national per capita income is strongly associated with student
performance in Mathematics. Differences in GDP account for 41% of the average score
variability observed in Sixth Grade Mathematics tests.
These results reveal that, in any one country, the greater the inequality the lower its
average performance. Similarly, differences in the Gini Index among countries account for
32% of the average score variance.
ReadingAn overall analysis of average Sixth Grade Reading scores and their distribution pro-
vides insight into the inequalities within and across countries. The difference between
countries located at both extremes is 1.75 standard deviations. However, between the
second and next to last countries this difference is reduced to only 1.16 standard devia-
tions.
Based on students’ average performance countries may be classified in five groups:
1. Countries where students’ scores exceed the average of the countries participat-
ing in SERCE (less than one standard deviation). Cuba, Costa Rica, Brazil, Chile,
Colombia, Mexico, Uruguay and the Mexican State of Nuevo Leon are part of this
group.
2. Countries where students’ mean scores are equal to the regional average. Argentina
illustrates the only case.
3. Countries where students’ scores are lower than SERCE’s regional average, and less
than one standard deviation: Ecuador, El Salvador, Guatemala, Nicaragua, Panama,
Paraguay, Peru and the Dominican Republic9*.
9* Significant differences (5% error) based on a t test for median comparison.
Executive summary 37
GRAph 5 MEAN AND VARIABILITY OF READING SCORES OBTAINED BY SIXTH GRADE
STUDENTS IN EACH SURVEYED COUNTRY
LAC total: Latin American and Caribbean countries’ total.CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05).Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the dis-tance between these two points, the greater the students’ performance variability.The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.
Looking at the variability across student performance facilitates an analysis of the
learning inequalities that characterise each country. Performance differences between
students in the 10th and 90th percentiles of the various countries fluctuate in the 182
to 294 point range (244 points at the regional level). In El Salvador, Nicaragua and the
Dominican Republic, students in these percentiles are separated by less than 200 points.
This is not the case of Argentina, Brazil, Chile, Colombia, Costa Rica, Ecuador, Guatemala,
Mexico, Panama, Paraguay, Peru, Uruguay and the Mexican State of Nuevo Leon, where the
difference between the 10th and 90th percentiles fall in the 206 to 259 point range. Lastly,
Cuba exhibits the greatest difference in scores, with a 294 point gap between 10th and
90th percentiles.
Student achievement in Latin America and the Caribbean38
TAbLE 12 DESCRIPTION OF READING PERFORMANCE LEVELS OF SIXTH GRADE STUDENTS
LevelCut-off
Score% students Description
IV
593.59
20.30% • Integrate, rank and generalise information distributed across the text;
• Establish equivalences among more than two codes (verbal, numerical and graphic);
• Reinstate implicit information associated with the entire text; • Recognise the possible meanings of technical terms or figurative
language; • Distinguish various tenses and nuances (certainty, doubt) used in a
text
III
513.66
26.79% • Locate information and separate it from other near-by information; • Interpret reformulations and synthesis; • Integrate data distributed across a paragraph; • Reinstate implicit information in the paragraph; • Re-read in search of specific data; • Identify a single meaning in words that have several meanings; • Recognise the meaning of parts of words (affixes) using the text as a
reference
II
424.54
35.46% • Locate information in the middle of a text that must be distinguished from a different piece of information found in a different segment;
• Identify words with a single meaning
I
299.59
16.51% • Locate information with a single meaning in a prominent or central part of the text (beginning or end), that is repeated literally or synonimously and is isolated from other information.
Below I 0.93% • Students at this level have not been able to acquire the abilities required in Level I.
Table 13 shows how students are distributed in each of the performance levels, by
country. In terms of Reading achievement, 50% of Cuba’s Sixth Grade students can be
found at Level IV, followed by Costa Rica with slightly over a third of its students occupy-
ing this level.
For their part, the percentage of students performing at Level IV in Uruguay, Chile, the
Mexican State of Nuevo Leon, Mexico and Brazil, fluctuate between 20% and 30%.
At the other extreme, 47.8% of the Dominican Republic’s Sixth Grade students per-
formed at Level I, followed by Ecuador, Guatemala, Panama and Paraguay with slightly over
one third of their students at this level.
Executive summary 39
TAbLE 13 PERCENTAGE OF SIXTH GRADE STUDENTS BY READING PERFORMANCE LEVEL
IN EACH SURVEYED COUNTRY
Country below I I II III IVArgentina 1.78 17.93 35.59 25.48 19.22Brazil 0.57 14.85 34.65 27.47 22.46Chile 0.30 8.02 30.06 32.37 29.26Colombia 0.39 13.17 38.25 30.40 17.80Costa Rica 0.22 5.00 23.45 36.73 34.59Cuba 0.30 5.26 19.57 24.20 50.68Ecuador 4.47 33.69 39.48 16.63 5.73El Salvador 0.95 21.49 44.02 23.99 9.54Guatemala 2.86 33.06 43.36 15.73 4.99Mexico 0.23 12.23 33.40 29.75 24.39Nicaragua 1.02 22.08 50.58 21.10 5.22Panama 1.95 28.97 38.76 20.77 9.55Paraguay 3.90 33.46 36.81 18.60 7.23Peru 2.24 24.08 41.65 22.57 9.46Dominican Rep. 4.08 47.84 37.50 9.19 1.39Uruguay 0.47 9.60 30.80 29.68 29.45Nuevo Leon 0.21 9.12 29.99 32.37 28.31
Total 0.93 16.51 35.46 26.79 20.30
Data on performance by school type reveal marked differences between the learning
acquired by students in urban and rural areas6 10as shown in Table 14.
Latin American and Caribbean Sixth Grade students attending urban schools outper-
form rural school students in Reading.
• Cuba is the only country that does not show significant performance differences
between urban and rural school students.
• In terms of school location, Nicaragua and the Dominican Republic show the small-
est differences - 21 and 24 points, respectively.
• By contrast, Peru shows the greatest differences –around 80 points– between
urban and rural school students, followed by Mexico, Panama and Paraguay with
differences approaching 57 points.
6 The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean represent a rough measure that, given the various definitions of rurality, should be taken with caution.
Student achievement in Latin America and the Caribbean40
TAbLE 14 DIFFERENCE IN AVERAGE SCORES BETWEEN URBAN AND RURAL SCHOOLS, BY
GENDER. SIXTH GRADE READING
Country Urban/ Rural Difference Girl/ boy Difference Argentina 43.55* 11.05*Brazil 49.35* 15.69*Chile 35.66* 6.89*Colombia 41.74* -4.43Costa Rica 34.37* -0.75Cuba 12.75 15.21*Ecuador 46.22* 6.39El Salvador 54.31* -0.19Guatemala 53.75* -2.44Mexico 57.71* 13.32*Nicaragua 21.42* -0.61Panama 56.67* 15.89*Paraguay 56.32* 11.14*Peru 78.96* -1.87Dominican Rep. 23.75* 15.09*Uruguay 49.10* 19.64*Nuevo Leon 39.23* 7.98
Total - 10.44*
* Stands for 5% confidence level
A gender-based analysis reveals that in Latin America and the Caribbean Sixth Grade
girls outperform boys in Reading. The regional gap between genders is 10.4 points.
Girls also obtain significantly higher scores in Argentina, Brazil, Chile, Cuba, Mexico,
Panama, Paraguay, the Dominican Republic and Uruguay.
Per capita GDP bears a direct correlation with students’ average learning. Differences
in national wealth account for 44.4% of the variation detected in Sixth Grade Reading
national averages.
The greater the Gini Index the lower the Reading average performance among Sixth
Grade students. Differences in the Gini Index account for 11% of the variation observed
across national Reading averages.
Natural ScienceThe Natural Science test was administered to Sixth grade Primary Education students
exclusively, with the participation of only 10 national entities: Argentina, Colombia, Cuba,
El Salvador, Panama, Paraguay, Peru, the Dominican Republic, Uruguay, and Nuevo Leon.
The difference separating countries located at the upper and lower ends of the perfor-
mance scale was calculated at 2.35 standard deviations. However, the difference between
the second highest and next to last country is only 0.68 standard deviations, which im-
Executive summary 41
plies that there is greater homogeneity among countries occupying mid-positions in the
distribution.
Overall, both national averages and distribution of scores show differences in each
country. Relative to performance in Science, four groups can be identified:
• The first group is made up of countries with mean scores markedly higher than
the regional average (more than one standard deviation, that is, over 650 points).
Cuba is the only case.
• The second group consists of countries with scores higher than the Latin American
and Caribbean average (less than one standard deviation): Uruguay and the Mexi-
can State of Nuevo Leon.
• Colombia is the only country in this third group, characterised by a national mean
that does not show significant differences versus the regional media.
• Countries that exhibit lower scores than the Latin American and Caribbean average
(less than one standard deviation) are part of a fourth group. These countries are:
Argentina, El Salvador, Panama, Paraguay, Peru and the Dominican Republic11*.
GRAph 6 MEAN AND VARIABILITY OF SCIENCE SCORES OBTAINED BY SIXTH GRADE
STUDENTS IN EACH SURVEYED COUNTRY
LAC total: Latin American and Caribbean countries’ total.CILL: Confidence interval lower limit (a = 0.05). CIUL: Confidence interval upper limit (a = 0.05). Bars depict results obtained by 80% of the students between the 10th and 90th percentiles in each country. That is to say, the far-right segment of each bar represents the scores of students in the 90th percentile and the left those of students in the 10th percentile. The greater the dis-tance between these two points, the greater the students’ performance variability.The white vertical line running through the centre of each bar identifies the mean, while the confidence interval is shown as a dark area around it. The width of this darkened area illustrates its possible values.
11* Significant differences (5% error) based on a t test for median comparison.
Student achievement in Latin America and the Caribbean42
The differences detected in learning results are reflected in the students’ scores disper-
sion. Three scenarios characterise the region:
• In most countries the distance separating the 10th from the 90th percentiles fluctu-
ates between 200 and 230 points. This is the case of Argentina, Colombia, Panama,
Paraguay, Peru, Uruguay and the Mexican State of Nuevo Leon.
• El Salvador and the Dominican Republic, with less than 200 points separating the
10th and 90th percentiles, exhibit the smallest dispersion of results.
• Cuba, in addition to showing the highest average score, also shows the greatest dis-
persion of results – 386 points between students in the 10th and 90th percentiles.
TAbLE 15 DESCRIPTION OF SCIENCE PERFORMANCE LEVELS OF SIXTH GRADE STUDENTS
LevelCut-off
score % students Description
IV
704.75
2.46% • At this level, students use and transfer scientific knowledge, which requires a high degree of formalisation and abstraction, to diverse types of situations.
• Students are capable of identifying the scientific knowledge involved in the problem at hand. These problems are more formally stated and may relate to aspects, dimensions or analyses that may be detached from the immediate setting.
III
590.29
11.40% • At this level, students explain everyday situations on the basis of scientific evidence; use simple descriptive models to interpret natural phenomena, and draw conclusions from the description of experimen-tal activities.
II
472.06
42.24% • At this level, students apply school-acquired scientific knowledge: compare, organise and interpret information presented in various formats (tables, charts, graphs, pictures); identify causality relations and classify living beings according to a given criterion.
• In connection with Level I, it should be noted that these students are capable of accessing information presented in different formats, which requires the use of much more complex skills.
I
351.31
38.72% • At this level, students relate scientific knowledge to daily situations that are of common occurrence in their context.
• Students are capable of explaining their immediate world based on their own experiences and observations, and establish a simple and lineal relation with previously acquired scientific knowledge.
• Students describe concrete and simple events involving cognitive processes such as remembering, evoking and identifying.
Below I 5.18% • Students at this level have not been able to acquire the abilities required in Level I.
Executive summary 43
Data on Science performance levels provide the grounds for grouping countries around
three possible scenarios:
• In Cuba, 65% of its students perform at Levels III and IV.
• In Colombia, Uruguay and the Mexican State of Nuevo Leon, practically half their
students perform at Level II.
• In Argentina, El Salvador, Panama, Paraguay, Peru and the Dominican Republic,
over 40% of their students perform at Level I or below.
TAbLE 16 PERCENTAGE OF SIXTH GRADE STUDENTS BY SCIENCE PERFORMANCE LEVEL
IN EACH SURVEYED COUNTRY
Country below I I II III IVArgentina 5.32 37.73 43.04 12.73 1.17Colombia 2.62 31.68 51.09 13.59 1.02Cuba 0.26 8.78 25.92 30.31 34.73El Salvador 3.78 44.73 42.55 8.23 0.71Panama 6.34 44.60 39.89 8.40 0.77Paraguay 7.20 46.18 38.11 7.52 0.99Peru 6.97 46.93 39.36 6.37 0.36Dominican Rp. 14.29 62.82 21.50 1.37 0.03Uruguay 1.69 22.76 48.47 24.01 3.06Nuevo Leon 2.59 30.98 47.78 16.38 2.28
Total 5.18 38.72 42.24 11.40 2.46
In terms of Science, students attending urban schools outperform their rural school
counterparts712.
Peru exhibits the greatest difference –in excess of 57 points– in Science performance
between urban and rural schools. El Salvador and Panama follow with an approximate 40
point difference.
Located at the opposite extreme is Cuba where no significant performance differences
between urban and rural school students are evident. For its part, the Dominican Republic
shows minimal differences that fluctuate around 11 points.
7 The definition of “rural area” is not exactly comparable among countries. The identification of rural schools was based on the definition provided by each country. Consequently, totals for Latin America and the Caribbean represent a rough measure that, given the various definitions of rurality, should be taken with caution.
Student achievement in Latin America and the Caribbean44
TAbLE 17 DIFFERENCE IN AVERAGE SCORES BETWEEN URBAN AND RURAL SCHOOLS, BY
GENDER. SIXTH GRADE SCIENCE
Country Urban/ Rural Difference Girl/boy Difference Argentina 19.74* -5.06Colombia 22.83* -18.93*Cuba 11.36 7.41El Salvador 41.91* -10.16*Panama 38.27* 1.26Paraguay 30.23* 1.88Peru 56.18* -16.12*Dominican Rep. 11.14* -0.65Uruguay 29.28* -4.44Nuevo Leon 26.65* -12.77*
Total - -11.52*
* Significant (5% confidence level)
Gender-based comparisons in the region reveal that boys have a marked advantage
over girls, obtaining average scores that are 11.5 points higher.
• In Colombia, El Salvador, Peru and the Mexican State of Nuevo Leon, boys’ Science
scores are significantly higher than girls’.
• By contrast, in Argentina, Cuba, Panama, Paraguay, the Dominican Republic and
Uruguay, no statistically significant differences between girls and boys were de-
tected.
Student performance and the internal production of a country are directly related.
National per capita GDP accounts for 11.57% of the variations observed in Science per-
formance.
Data seem to indicate that there is an inverse relationship between the learning of
Science and income distribution inequalities. In fact, the Gini Index accounts for 30.68%
of the national mean variances observed in Science performance.
Executive summary 45
It will probably come as a ray of hope to all educational systems, that through the
study of associated factors, SERCE has been able to corroborate the fact that schools are
in a position to contribute importantly to student performance. While the socioeconomic
dimension has a strong influence on performance, school-related variables can help sig-
nificantly to reduce the learning inequalities associated with social inequity.
In line with PERCE’s conclusions, the school climate variable was confirmed to have
the greatest impact on student performance. It follows that, in order to promote learning
among students, it is essential to provide a welcoming and warm environment based on
mutual respect.
Factors associated with achievement
foto pendiente
UNE
SCO/
G.Te
aldi
Student achievement in Latin America and the Caribbean46
Collectively, the school resources variable also contributes to performance. While it is
entirely possible that variables such as school infrastructure, basic services, the number
of books in the school library, and the work experience of teachers, can only make modest
individual contributions, as a whole, they can help substantially to encourage learning.
The clear message behind this assertion is that resources are necessary elements to drive
performance.
School segregation based on the socioeconomic and/or cultural status of the student is
the second most important variable that explains performance. Segregation seems to have
a stronger impact on Reading than on Mathematics or Science. And, while this is not an
education-related variable per se, any progress in this area will translate into important
advances in students’ learning.
Executive summary 47
Quality education must be seen as a right of all girls and boys. Attaining it represents
a solid base for sustainable development, democratic progress and social equality. The
SERCE embodies joint efforts undertaken by the Latin American and Caribbean countries
and OREALC/UNESCO, aimed at enhancing educational opportunities for all students and,
ultimately, promoting development in the region.
The evaluations conducted within the framework of this Study, attempt to provide an
analysis of what students learn, the inequalities that affect learning, and the factors that
determine differential achievement.
Final reflections
foto pendiente
UNE
SCO/
D.Ro
ger
Student achievement in Latin America and the Caribbean48
On Primary Education student learning
In terms of academic performance, quality education is expected to lead to high levels
of learning among all students, without exclusions of any kind. From SERCE’s perspective,
equity is transversal since it focuses on social conditions that prevent from fully exercising
the right to education, and on the way schools ensure a balanced provision of learning
opportunities to their students.
Significant differences in the quality of student learning are evident in the region. This can be observed across all areas and grades, as reflected by the dispersion of re-
sults within countries, and by the gaps in scores detected among participating countries.
Thus, in connection with Third Grade education, the differences observed between
the highest and lowest performing countries exceed 230 points, both in Reading and
Mathematics. In terms of Sixth Grade, the differences although somewhat smaller, still
exceed two standard deviations in Science and Mathematics, and rise to 174.5 points in
Reading.
This diversity affecting quality of learning can also be presented graphically by divid-
ing participating countries into four groups, on the basis of their average test results.
TAbLE 18 COMPARISON OF THIRD GRADE SCHOOL RESULTS
Difference relative to the regional mean Mathematics Reading
Higher than the mean– more than one standard deviation
Cuba Cuba
Higher than the mean– less than one standard deviation
Chile, Costa Rica, Mexico, Uruguay and Nuevo Leon
Argentina, Chile, Colombia, Costa Rica, Mexico, Uruguay and Nuevo Leon
Identical to the regional mean Argentina, Brazil and Colombia Brazil and El SalvadorLower than the mean– less than one standard deviation
Guatemala, Ecuador, El Salvador, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic
Ecuador, Guatemala, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic
Executive summary 49
TAbLE 19 COMPARISON OF SIXTH GRADE SCHOOL RESULTS
Difference relative to the regional mean Matemática Reading Science
Higher than the mean– more than one standard deviation
Cuba Cuba
Higher than the mean– less than one standard deviation
Argentina, Chile, Costa Rica, Mexico, Uruguay and Nuevo Leon
Costa Rica, Cuba, Brazil, Chile, Colombia, Mexico, Uruguay and Nuevo Leon
Uruguay and Nuevo Leon
Identical to the regional media
Brazil, Colombia and Peru
Argentina Colombia
Lower than the mean– less than one standard deviation
Ecuador, El Salvador, Guatemala, Nicaragua, Panama, Paraguay and the Dominican Republic
Ecuador, El Salvador, Guatemala, Nicaragua, Panama, Paraguay, Peru and the Dominican Republic
Argentina, El Salvador, Panama, Paraguay, Peru and the Dominican Republic
It should be noted that in countries occupying the second and next to last position in
the distribution scale, in practically all cases, mean scores differences are slightly above
one standard deviation. This would point to a greater homogeneity among countries oc-
cupying mid-positions on the performance scale. Science constitutes a special case, since
here standard deviations between the upper and lower extremes rise to 2.35 points, while
intermediate results show a standard deviation of 0.68, indicative of greater homogeneity
in this segment, and a substantial difference versus the extremes.
This diversity within countries is also made evident when comparing differences be-
tween students in the 10th and 90th percentiles. On this basis, four country categories may
be established, both for Third and Sixth Grade Primary Education students, namely:
1) Countries where the dispersion range between highest and lowest performance
levels is less than 200 points;
2) Countries that exhibit variability between 10th and 90th percentiles in the 200 - 250
point range;
3) Countries with a performance dispersion range of more than 250 points but less
than 300 points, and
4) Countries that exhibit an internal variability in excess of 300 points
In connection with scores obtained by Third Grade students, differences fluctuate
between 165 and 341 points in Mathematics, and between 183 and 296 in Reading. Cuba,
Uruguay and Paraguay exhibit the highest internal dispersions in Mathematics and Read-
ing, while Nicaragua shows the lowest.
Student achievement in Latin America and the Caribbean50
TAbLE 20 COMPARISON OF SCHOOL RESULTS DISPERSION FOR THIRD GRADE STUDENTS,
BY COUNTRY
Difference between 90th and 10th percentiles
Mathematics Reading
Less than 200 points Colombia, Ecuador, the Dominican Rep., Guatemala, El Salvador, Panama and Nicaragua
Nicaragua
Between 200 and 250 points
Brazil, Uruguay, Argentina, Mexico, Chile, Costa Rica, Peru and the Mexican State of Nuevo Leon
Paraguay, Mexico, Uruguay, Argen-tina, Brazil, the Dominican Rep., Costa Rica, Chile, Colombia, Panama, Ecuador, El Salvador, Peru, Guatemala and the Mexican State of Nuevo Leon
Between 251 and 299 points
Paraguay Cuba
300 and over Cuba
In connection with Sixth Grade students, average performance differences between
students in the 10th and 90th percentiles fluctuate between 182 and 385 points in Math-
ematics, and 176 and 387 points in the case of Science. Once again, Cuba shows the high-
est dispersion in all three areas, while the Dominican Republic exhibits the lowest internal
dispersion in the aforementioned areas and grade.
TAbLE 21 COMPARISON OF SCHOOL RESULTS DISPERSION FOR SIXTH GRADE STUDENTS,
BY COUNTRY
Difference between 90th and 10th percentiles
Mathematics Reading Science
Less than 200 points The Dominican Rep., Nicaragua, El Salvador, Panama and Guatemala
El Salvador, Nicaragua and the Dominican Rep.
The Dominican Rep., and El Salvador
Between 200 and 250 points
Colombia, Paraguay, Brazil; Costa Rica, Ar-gentina; Ecuador, Chile, and the Mexican State of Nuevo Leon
Uruguay, Mexico, Brazil, Chile, Paraguay, Costa Rica, Peru, Panama, Ecuador, Guatemala, Co-lombia, and the Mexican State of Nuevo Leon
Argentina, Colombia, Uruguay and the Mexican State of Nuevo Leon
Between 251 and 299 points
Mexico, Peru and Uruguay
Argentina and Cuba Cuba
300 and over Cuba
While Cuba shows the highest dispersion and the Dominican Republic the lowest, these
findings should be carefully interpreted. On the one hand, scores obtained by the lower
Executive summary 51
performing Cuban students are similar to those of the average Latin American and Carib-
bean students. This fact places lower performing Cuban students much farther ahead than
the rest of the region’s student population.
On the other hand, the Dominican Republic exhibits the lowest results of the surveyed
countries while the minimal score dispersion would seem to indicate that the results
obtained by these students are generally low. In short, the results yielded by these two
countries illustrate that, on the one hand, high general performance and high variability
are not mutually exclusive and, on the other, that there are cases where results may be
equally distributed but nevertheless learning levels remain low.
In qualitative terms, this diversity affecting the quality of learning of Latin American
and Caribbean students is reflected in the distribution of performance levels. Analyses
based on performance levels give an in-depth view of what students are capable of doing
in each of the surveyed grades and areas. SERCE classifies students’ achievements into four
performance levels (I through IV) of increasing complexity. Each level is made up of a set
of tasks the students will be tested on. In order to solve these tasks students must master
specific contents and apply distinct cognitive process.
An ideal distribution would show that most students perform at the higher levels.
However, results do not generally conform to this pattern and while over 20% of the
students in the region do, in fact, perform at the higher levels in practically all areas and
grades (except in Science, where the percentage drops to 13.8%), there is an important
number of students who are unable to perform beyond Level I: more than 40% in Third
Grade Mathematics and Science; 32% in Third Grade Reading; and more than 15% in Sixth
Grade Mathematics and Reading. The implication is that these students can only attempt
the tasks that SERCE has defined as having the lowest levels of complexity.
For instance, while more than 40% of Cuban students perform at the highest level, in
every area and grade, there are other countries where approximately 50% of their students
perform at or below Level I, in every area and grade. These results give insight into the
region’s learning gaps and underline the importance of going beyond average scores when
discussing educational quality, in order to identify and understand what students know
and are capable of accomplishing. These findings shed light on the challenges that must
be surmounted to enhance the quality of instruction imparted in Primary Education, and
highlight the serious learning inequities that persist in the region.
The equitable distribution of learning across different social strata remains a pending task
a) The economic conditions of countries, particularly income generation and dis-tribution, have a bearing on primary Education student learning.
In order to explain this assertion, SERCE has analysed the existing link between stu-
dent average performance, per capita Gross Domestic Product, and the Gini Index for each
Student achievement in Latin America and the Caribbean52
country. Due to the unavailability of relevant data, Cuba and the Mexican State of Nuevo
Leon have not been included in this analysis.
Data confirm the existence of a positive correlation between the average scores of
a given country and its per capita GDP. However, many countries obtain results beyond
what their internal production would have predicted, which indicates that while resourc-
es are important they are not the only factors that determine student performance.
Country by country analysis of average performance versus the Gini Index, shows an
equally significant but inverse relation. In other words, the higher the income distribu-
tion inequality the lower the average student performance exhibited by Latin American
and Caribbean students.
b) Student gender has an impact on SERCE’s results Consistent with other studies on gender-based student performance, the present
Study corroborates differences in most countries favouring girls in Reading and boys in
Mathematics. Exceptions can be found in the Dominican Republic and Cuba where girls
outperform boys in Third Grade and Sixth Grade Mathematics. In terms of Science, four
participating countries show differences skewed in favour of boys, while in the remain-
ing six countries no significant gender-based differences are evident.
c) School location influences student achievement Within the region, the location of schools is also responsible for generating differ-
ences in student performance. In Latin America and the Caribbean, rural school boys
and girls show lower levels of performance when compared to their urban school coun-
terparts.
These inequalities become sharper in some countries. The greatest differences in
performance favoring urban school students –in both areas and grades surveyed– can
be found in Peru, while the smallest differences attributable to the geographic location
of schools were evident in the Dominican Republic and Cuba. In terms of Science, the
greatest inequalities related to location are found in El Salvador and Panama. Converse-
ly, Peru and the Dominican Republic, show the smallest differences.
An analysis of student distribution by performance levels corroborates the existence
of these gaps. There are clear differences, both at the regional level and within the
countries, relative to the percentage of students occupying each of these performance
levels that depend almost exclusively on whether the student is attending an urban or
rural school. Moreover, in urban schools, performance distribution seems to have shifted
to the next upper level, vis-à-vis rural schools. As a result, the percentage of students
performing at Levels II, II and IV is systematically higher in urban schools, while at the
lower levels (I and below I) there is a larger percentage of rural students represented.
Executive summary 53
The school does make a difference
In what undoubtedly constitutes an encouraging message to all education systems, SERCE
has been able to corroborate through its study of associated factors that schools can, in
fact, contribute importantly to student performance. While socio-economic factors are
known to have a significant effect on performance, school-related variables can have a
substantial impact on reducing the learning inequalities associated with social dispari-
ties.
In line with PERCE’s findings, school climate was found to be the single most impor-
tant variable conditioning student performance. Hence, generating a friendly and positive
environment based on mutual respect becomes an essential strategy to foster student
learning.
As a whole, school resources variables also contribute positively to student performance.
While the contributions made by school infrastructure, availability of basic services, num-
ber of books comprising the school library, and the teaching experience of educators is,
at best, modest when taken individually, collectively these variables represent a valuable
help to student learning. The key message derived from this finding is that resources are
indeed necessary to drive performance.
School segregation based on the students’ socio-economic and cultural status is the
second most important performance conditioning variable. Segregation has been shown to
have a stronger impact on Reading as opposed to Mathematics and Science, and although
this is not an educational variable per se, any efforts aimed at reducing it will greatly
influence student learning and achievement.
Clearly, the Second Regional Comparative and Explanatory Study conducted by the
Latin American Laboratory for Assessment of the Quality of Education, has contributed
important information and knowledge to inform the decision-making process in matters
concerning social and educational policies in Latin America and the Caribbean. Each of
the countries participating in the Study must now retrieve the main lessons derived from
this important inquiry.
Recommended