2011 Annual National Assessment

Basic Education Portfolio Committee

16 August 2011

PRESENTATION OUTLINE

• Introduction

• Purpose of ANA

• Limitations

• Learner performance results (verification and universal ANA)

• Trends in learner performance.

• Qualitative Analysis

• Next Steps

• Conclusion

IntroductionANA is an assessment tool supporting, Action Plan to 2014, Towards the Realisation of Schooling 2025.

A key problem in the past has been the absence of a standardised measurement of the quality of teaching and learning below Grade 12 except in samples of schools and learners. ANA is a large-scale adaptation of the sample-based Systemic Evaluation programme, which was run in 2001, 2004 and 2007.

IntroductionFirst Three Goals of Action Plan to 2014:-

• Increase the number of learners in Grade 3 who by the end of the year have mastered the minimum Language and Numeracy competencies for Grade 3.

• Increase the number of learners in Grade 6 who by the end of the year have mastered the minimum Language and Mathematics competencies for Grade 6

• Increase number of learners in Grd 9 who, by end of year, have mastered the minimum Language and Maths competencies.

Introduction

Baseline and targets are calculated as “percentage of learners achieving a specific level of performance” in each case.

Baseline (2009) Target (2014)

Grd. 3 Lit. 48 60Grd. 3 Num 43 60Grd. 6 Lang 37 60Grd. 6 Math 19 60Grd. 9 Establish in 2012 60

Introduction• Envisaged that ANA will become a cornerstone of quality

improvements in South African schools, providing important information on learning and its context.

• Experience in other countries have shown that it takes up to 5 years for a programme of this nature to build up sufficient credibility and to reach the required level of impact on school practices.

• The credibility is determined by:- The assessment instrument- The administration of the assessment- The marking and moderation of assessment- The analysis

Purpose of ANA• To provide credible and reliable information for monitoring

progress in learner performance towards achieving the national goals of “improving the quality of basic education”.

• Key expected effects:- Serve as a diagnostic tool identifying areas of strength

and weakness in teaching and learning Expose teachers to better assessment practices. Provide districts with information to target schools in need

of assistance. Encourage schools to celebrate outstanding performance. Empower parents by providing them information about the

education of their children.

Limitations• This was the first time an assessment was administered on this

scale. • Comparability with previous assessments, in particular the

2004 and 2007 Systemic Evaluation, is not perfect with respect to the tests as a whole.

• Problems with the formulation and translation of certain questions have been noted and underline the importance of strengthening the test design processes in future years.

• Although all teachers were expected to follow specific procedures in administering the tests, it would appear that in some instances procedures were not followed as strictly as they were supposed to be and that could have affected learner scores either way.

Limitations

• Marking may not have been consistent across schools.

• Systems for collecting the data from schools, capturing and cleaning, proved to be inadequate for the magnitude of this study and disparate where they existed, resulting in some data not being available on time for analysis.

Analysis of verification ANA results• The data cover 19 470 Grade 3 learners and 19 397 Grade 6

learners in 827 and 840 schools respectively.

• Test scores converted to percentages are used in the analysis.

Maximum possible mark per test

 Grade Learning Area Total Mark

3 Literacy 30

3 Numeracy 40

6 Language 50

6 Mathematics 75

Percentage of learners functioning at levels of performance

Learners have been placed in specific levels of performance according to the percentage scores they obtained out of the total for each test.

Level Description Score range•Level 1: Not achieved: Score Less than 35% •Level 2: Partially achieved: At least 35% but less than

50%•Level 3: Achieved: At least 50% but less than

70%•Level 4: Outstanding: At least 70%

Learner performance results (VA)Grade 3 mean score (%) Grade 6 mean score (%)

Literacy Numeracy Language Maths

EC 39 35 29 29

FS 37 26 23 28

GP 35 30 35 37

KZN 39 31 29 32

LIM 30 20 21 25

MPU 27 19 20 25

NC 28 21 27 28

NW 30 21 22 26

WC 43 36 40 41

NATIONAL 35 28 28 30

Gender comparisons in performance (% learners)

Level 1 Level 2 Level 3 Level 4

Grade 3 Literacy

Boy 62 14 17 8

Girl 48 17 22 13

Grade 3 Numeracy

Boy 70 16 10 4

Girl 67 17 11 5

13

Gender comparisons in performance (% learners)

Level 1 Level 2 Level 3 Level 4

Grade 6 Language

Boy 75 13 10 2

Girl 63 16 15 5

Grade 6 Mathematics

Boy 69 19 9 3

Girl 63 23 10 4

14

MarkingExcept for literacy in grade 3, there were no significant differences at national level between scores allocated by teachers and those allocated independently after remarking of scripts (VA).

There were however, individual schools where teachers tended to either over- or under-mark learners’ work.

This has important implications for strengthening teachers capacity in assessment.

Learner performance at specific levelsThe percentage of learners performing at specific levels:

In all four of the ANA tests examined, over half of all learners in the country perform at a level that indicates that they have clearly not achieved the competencies specified in the curriculum.

In Grade 6, the results indicate that around 70% of learners fall into this category. At the top end, very few learners are able to achieve outstanding results. For instance, only 3% of learners in Grade 6 mathematics can be considered outstanding.

Learner performance at specific levels

Performance in Grade 3 literacy by level

Learner performance at specific levels

Performance in Grade 3 numeracy by level

Learner performance at specific levels

Performance in Grade 6 language by level

Learner performance at specific levels

Performance in Grade 6 maths by level

Percent “Struggling” and “Promising” schools by quintile

Grade & Subj.

Q 1 Q 2 Q 3 Q 4 Q 5

S P S P S P S P S P

3 Lit 17 45 16 42 12 52 7 58 9 66

3 Num 38 19 35 11 34 25 26 30 10 41

6 Lang 46 13 56 9 32 16 18 35 3 62

6 Math 45 13 33 11 26 10 17 24 8 60

Universal ANA Results

Learners achieving acceptable performance (Levels 2, 3 & 4) in Literacy and Numeracy

GradeTotal Literacy/Language Total Numeracy/Mathematics

Captured Number % Captured Number %

1 425 626 341 725 80.3 431 489 388 552 90.0

2 382 677 285 173 74.5 385 273 320 038 83.1

3 380 687 212 264 55.8 381 309 158 624 41.6

4 388 406 174 664 45.0 388 703 126 450 32.5

5 372 954 137 253 36.8 376 549 116 764 31.0

6 359 758 152 243 42.3 360 081 130 037 36.1

Total2 310 108 1 303 322 56.4 2 323 404 1 240 465 53.4

Universal ANA

Mean scores of learners in Literacy and Numeracy

Grade Literacy/Language Numeracy/Mathematics

159.4 63.0

251.6 54.7

342.0 32.7

433.5 28.4

528.6 28.3

633.1 31.1

Universal ANA

Distribution of learners per achievement levels

Trends in ANA results Universal ANA (2011)• Preliminary findings show that learners in Grade 1 to Grade 5 performed

progressively less well in both Literacy and Numeracy from the foundation phase to the intermediate phase. Results for Grade 6 are marginally better than for Grade 5.

• Notably, results for Grades 1 and 2 in both Literacy and Numeracy are very promising.

• Factors contributing to relatively good results in Grades 1 and 2: Expansion of Grade R, introduction of standardised learning material for Grade R, introduction of Foundations for Learning, interventions to improve Literacy and Numeracy in the Foundation Phase, among others.

Trends in ANA results (cont’d)

Universal ANA (2011)• The lower performance in Grade 3 compared to Grade 2 may be

attributed to the fact that the above interventions were not present to scale in time to have an impact on this Grade.

• The lower performance in Grade 4 compared to Grade 3 may be attributed largely to the challenge in the transition from mother-tongue instruction to the Language of Learning and Teaching that is introduced from Grade 4. This issue has been clearly identified and addressed in the Curriculum Revision process.

Trends in ANA resultsVerification ANA• Overall mean learner performance in Literacy in Grade 6 is lower than

in Grade 3. • Grade 3 Numeracy:-

- Learners’ mean performance in numeracy was lower compared to performance in Literacy.- More learners perform at Level 1 (not achieved) compared to performance in Literacy.

• Grade 3 Literacy:-

- 53% of Grade 3 learners perform at Level 1 (not achieved) whilst 11% perform at Level 4.

Trends in ANA resultsGrade 6 Language: - 70% of Grade 6 learners perform at Level 1 (not

achieved) - 3 percent of grade 6 learners perform at “outstanding”

level.Grade 6 Mathematics:

- Learners’ performance in Mathematics is similar to that of the Language.

Generally, girls tended to perform better than boys in all grades.

28

Qualitative analysis

Lack of generic skills:- Hand writing: Too many learners across the system have not

yet developed the basic skill of writing properly and legibly. Difficult to establish if they know anything in the tested skills and knowledge and doubtful if they can make sense of what they have written

Inability to read and follow written instructions: - Many learners provided responses that suggested that they could not read questions with understanding.

Acute lack of basic word-treasure (vocabulary) to express oneself in writing, especially from grade 4 upwards.

Qualitative analysis

The rate at which learning occurs is low.

Learners in the higher grades (grades 4, 5 and 6) display learning deficiencies in basic literacy and numeracy skills.

Numeracy

• Inability to handle basic numeracy operations - subtraction, multiplication and division that involve whole numbers.

• Conceptions of fractions, e.g. comparing fractions according to size and arranging them from the smallest to the highest or vice versa.

• Misconceptions become more pronounced when learners have to operate mixed numbers, i.e. where there are whole numbers and fractions in the same operation, such as in subtracting “three-and-one- quarter” from “five-and-one-eighth”

Numeracy

• Inability to translate problems that are given in words and to write them in ways that enable them to solve the problem using mathematical techniques. For example, inability to calculate the amount of change a shopper receives from a specified amount tendered after paying for a number of items whose individual prices are given.

Literacy• General lack in basic literacy skills including basic grammar,

correct spelling of frequently used words, proper use of language forms (e.g. correct use of prepositions, plural forms, tense, words that mean opposites of given words, etc.).

• Lack of vocabulary, arising out of inadequate or infrequent practice in the important skill of “reading”?

• Poor comprehension skills, thus only simple questions that require them to extract information directly from given text or give short one-word answers are attempted. Unable to respond to questions that demand complex skills of inferential reading (“reading between the lines”),

Literacy

• Can’t respond to questions that ask: “Why?” (Reasoning); “What do you think?” (Inferential reading); “State in your own words.” etc.

• Inability to write own text from given prompts (e.g. given picture and asked to write what one thinks about what’s in the picture) or transform a given text into another form that requires basic comprehension.

Next StepsLearner performance is influenced, inter alia, by the following factors:-

•Availability of appropriate learning and teaching support materials•Teacher competencies in relevant content areas•Time on task by both learners and teachers•Language of learning and teaching.•General school functionality

35

Next Steps• Foundations for Learning Campaign provides directives,

support and materials to improve learner performance in reading, writing and numeracy in all schools. This year DBE distributed 15 000 Foundations for Learning packs for use by teachers from Grade R to 3.

• The Curriculum and Assessment Policy Statements (CAPS) builds on the Foundations for Learning and provide more time for languages and mathematics (or literacy and numeracy). This will be supported with guided teacher development and appropriate readers and workbooks.

36

Next Steps• Development of “banks” of good quality exemplars of

assessment in each of literacy and numeracy.• Reduction in the number of subjects in the Intermediate

Phase (implementation will begin in Grades 4-6 in 2013.) This will address the big leap between the Foundation Phase (Grades 1-3) and the Intermediate Phase (Grades 4-6). Learners jump from three subjects to eight

• From 2012, the language chosen by the learner as a Language of Learning and Teaching from Grade 4 shall be taught as a subject, from Grade 1, and not from Grade 3. This will address the switch in many schools from home language to English from Grade 4.

• 37

Next Steps• DBE is working towards universal access to Grade R, by

2014. Between 2007 and 2010 there was an increase in the number of schools offering Grade R, from 12 480 schools to 16 020 and an increase in the number of learners in Grade R, from 487 222 to 707 203

• Provision learner-friendly literacy and numeracy workbooks for every learner in grades 1-6. This year government has printed workbooks for 6 million learners. In 2012, workbooks will also be distributed to learners in Grades 7-9. Workbooks are available in the appropriate language of learning and teaching for every learner.

38

Next Steps• Strengthening teacher development programmes. The focus

is on more targeted, subject-specific teacher education and development that will improve teacher content knowledge.

• Strengthening the campaign to attract young people to the teaching profession by providing incentives to attract top students into the teaching profession (e.g. the Fundza-Lushaka Bursary Scheme).

39

Next Steps•Improving accountability in the system which is anchored

around the following four pillars:-• Strengthening the culture of performance management.

• Simplifying and aligning the evaluation instruments that measure performance standards for educators;

• Strengthening monitoring and support for educators; • Strengthening reporting at all levels.• The National Education Evaluation and Development Unit (NEEDU) has been established. This Unit will help in evaluating the education system and identifying critical factors that inhibit or advance school improvement and make focused recommendations for redressing the problem areas that undermine school improvement.

40

Next Steps• Principals and deputy principals will enter into performance

contracts with clear performance targets. Performance will be measured in terms of the academic performance of the school and against the quality of management of the school.

• Support for schools, will be strengthened through the development of job descriptions for circuit managers and subject advisors and the development of performance agreements to be signed by these officials with clear targets and deliverables.

• DBE plans through the Accelerated Schools Infrastructure Development Initiative, to make all schools, rural and urban, to function optimally by 2014.

41

Next Steps• Promoting education as a societal enterprise rather than an

exclusive responsibility of DBE. The Quality of Learning and Teaching Campaign (QLTC) which provides a forum for all role-players, e.g. parents, teacher formations, learner formations, DBE, and others, is one initiative in this collective responsibility strategy.

• Ensuring that school management is led by competent staff who are continually supported by equally competent officials at district level so that schools are functional and provide a conducive environment for learning and teaching.

42

Conclusion

The 2011 ANA will serve as the baseline which the DBE will use to measure future progress.

All South Africans, parents, educators, the private sector, the children themselves are called to action, to work together in support of basic education and the future of South Africa’s children.

43