40
AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS DEVELOPMENT PROGRAMMES IN KATLEHONG, SOUTH AFRICA Jacqueline Moodley, Lauren Stuart and Thapelo Mqehe

AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS DEVELOPMENT

PROGRAMMES IN KATLEHONG, SOUTH AFRICAJacqueline Moodley, Lauren Stuart and Thapelo Mqehe

Page 2: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

2 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Copyright © 2016: The authors, the Centre for Social Development in Africa, the University of Johannesburg. Short extracts from this publication may be produced unaltered without authorisation on condition that the source is acknowledged. For rights of reproduction or translation, application should be made either to the Centre for Social Development in Africa, the University of

Johannesburg, or Liberty.

The Centre for Social Development in Africa (CSDA), at the University of Johannesburg, was established in 2003 and is dedicated to basic, applied and strategic research in social development and developmental welfare. The CSDA aims to positively influence development issues

in the Southern African region through contributing to debates on social policy, improvements in service delivery, and the expansion of knowledge through cutting-edge research.

CSDA, University of Johannesburg, PO Box 524, Auckland Park, 2006 [email protected] | www.uj.ac.za/csda

Liberty has a 40 year legacy of involvement in social investment which was pioneered by its Founder, Sir Donald Gordon, whose belief is that everyone should have the opportunity to grow their wealth and leave a proud legacy for their family.

Through its Corporate Social Investment, Liberty is funding the programmes being evaluated in Katlehong.

This study was a collaborative initiative between Liberty and the CSDA, University of Johannesburg. It must however be noted that the views expressed and information contained in this report are not necessarily those of or endorsed by

Liberty, which can accept no responsibility for such views or information, or for any reliance placed on them.

Editing: Ellen Joubert Layout: Ebrahim Karim, University of Johannesburg Graphics Department

Printing: 4 Color Print ISBN: 978-0-86970-790-6

Page 3: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

3An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

ContentsAcknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2. Education in South Africa . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.1 Supporting education from ‘cradle to career’ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 Aims and objectives of the research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.3 Structure of the report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3. Research Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 Sampling strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.2 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.3 Ethical considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.4 Limitations of the research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.1 Future Thinking Foundation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4.2 Kutlwanong Centre for Maths, Science and Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4.3 ORT SA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.4 Mindset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.5 New Leaders Foundation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.6 Foundation for Integrated Entrepreneur Leadership Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

5. Liberty’s investment in Katlehong . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

6. References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

List of TablesTable 1: Fieldwork activities for Future Thinking Foundation 2013-2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Table 2: 2013 – Grade 3 Maths and isiZulu marks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Table 3: 2014 – Grade 3 Maths and isiZulu marks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Table 4: 2013 – Grade 11 Maths and Science marks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Table 5: 2014 – Grade 12 Maths and Science marks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

List of FiguresFigure 1: Letters to Liberty from Future Thinking Foundation learners . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Figure 2: Letters to Liberty from Kutlwanong learners. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Figure 3: Perceptions of the extent to which New Leaders Foundation impacted on skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Figure 4: Coaching, support and problem solving for curriculum delivery in schools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

Figure 5: Creating communities of practice and professional learning groups to improve curriculum delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

Figure 6: Management and leverage of quality partnerships to improve curriculum delivery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

Figure 7: Technical skills of FIELD participants at baseline (2013) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Figure 8: Technical skills of FIELD participants post training (2014) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Figure 9: Entrepreneur Efficacy scale post training (2014) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

Figure 10: Social skills of FIELD participants at baseline (2013). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Figure 11: Social skills of FIELD participants post training (2014). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

Figure 12: Perceived chances of success over time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

Page 4: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

4 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

AcknowledgementsThe most important people to thank are those who participated in our research. To the beautiful children, thank you for allowing us to get to know you, and thank you to their parents who agreed to allow them to speak to us. To the educators, principals, and future entrepreneurs, may your warmth and passion continue to inspire those whose lives you touch. Without your collaboration this study would not have been possible.

To the Ekurhuleni South Department of Basic Education, thank you for being part of our research, as well as facilitating our access to the schools. We are most appreciative to Leonard Moloi and Audrey Hutton for distributing the parental consent forms to learners, and to Leonard for taking the time to drive us into Katlehong to meet the educators.

Thank you to the programme staff for all their assistance, and for providing us with the necessary information to complete this research. The following people were particularly helpful:

• Dylan Dulany and Melissa Combrink from the Future Thinking Foundation.• Tumelo Mabitsela, Khanghela Collen Mkhomazi and Madeleine Lekhula at the Kutlwanong Centre for Maths, Science and Technology.• Mpho Matlala at ORT SA.• Roith Rajpal, Tebalo Tsatsi and Grant Watt at Mindset. • Giles Gillett, Adele Ivy and Stacey Shaw-Taylor at the New Leader Foundation.• Stephen Read, Sicelo Buthelezi and Mothijoa Lebese at the Foundation for Integrated Entrepreneur Development. Special thanks to Prof Leila Patel, Prof Eleanor Ross, Tessa Hochfeld, Innocentia Kgaphola, Marium Mayet, Selby Khoza and Monde Oliphant at the CSDA, for their input at various stages of the research.

Thanks to Liberty’s Lynn Husselmann, Ayanda Sema, Ntswaki Thoka, Fatima Collins, Lucia Mongola, Joe Mwase and Shona Young for their support in ensuring that the research was conducted effortlessly. Thank you to Liberty for funding this research.

Executive SummaryIn 2013, Liberty contracted the Centre for Social Development in Africa (CSDA) at the University of Johannesburg to conduct a three year independent evaluation of the education programmes they are funding in Katlehong, which is a township situated south-east of Johannesburg. Katlehong falls under the Ekurhuleni South Department of Basic Education and the word Katlehong means “place of success”, however, the area is one of the most poorly served areas in Gauteng, and is characterised by high levels of unemployment, despite being an industrial area (StatsSA, 2011). Katlehong has just over 400 000 inhabitants, of whom 98% are Black African (StatsSA, 2011). Liberty’s educational investments in this area are in line with the goals of the National Development Plan (NDP: National Planning Commission, 2013) to improve education and skills development for black people who live in poverty. The programmes supported by Liberty which were selected for evaluation in 2013/2014 were the Future Thinking Foundation; Kutlwanong Centre for Maths, Science and Technology; the Foundation for Integrated Entrepreneur Development (FIELD) and the New Leaders Foundation. In 2015, the evaluations for FIELD and the New Leaders Foundation were discontinued and substituted by ORT SA and Mindset. These represent primary school, high school and post school programmes. Liberty’s vision for investing in these programmes was to ensure that they were able to support education from ‘cradle to career’. The advantage of Liberty taking an interest in programmes at these different levels, and concentrating them in an area like Katlehong, is that all phases of development in education are reached for a more tangible contribution to making improvements in South Africa’s education system.The effects of Liberty’s investment in Katlehong are positive, as evidenced by the findings of the evaluation. However, recommendations are made for each programme in order to enhance Liberty’s footprint in the area.

The evaluation consisted of rapid appraisals which allowed for immediate feedback on progress that the programmes were making, as well as an impact evaluation, which enabled the identification of programme elements which contributed to changes in participant knowledge and skills. The data collection methods consisted of both quantitative and qualitative research techniques. In addition, all stakeholders including the facilitators of the programmes, district officials, educators and learners participated in the research. This approach allowed for a more comprehensive evaluation of the programmes. Over the three years, a total of 95 beneficiaries participated in individual interviews or focus groups. In addition, a total of 636 learner assessments were analysed for Future Thinking Foundation and Kutlwanong in order to determine if the programmes were making an impact on learner performance.

The research carried out consisted of interviews with learners, educators and employees at the Ekurhuleni South Department of Basic Education. In addition, where necessary, an analysis of learner marks was conducted. The results revealed that:

• The Future Thinking Foundation has contributed significantly to an increase in Maths and isiZulu marks of foundation phase learners in the area. The increases were 10.2% and 7.1%, respectively in 2013. In 2014, the learner marks increased by 17.9% and 9% for Maths and isiZulu, respectively. Educators have access to materials and resources which assist greatly in classroom teaching, yet they have difficulties in incorporating the material into that prescribed by the Department of Basic Education.

• Kutlwanong has made a significant impact on Maths and Science marks for learners in Grades 11 and 12. In 2013, there was a marked improvement of 29.3% for Mathematics and 27.6% for Science. In 2014, again, increases of 23.1% for Mathematics and 20.9% for Science were recorded. This programme does experience some difficulties, specifically in relation to laboratory equipment.

Page 5: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

5An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

• For ORT SA, a relatively new programme in the area, educators find the training invaluable. One of the biggest challenges for this programme, is the co-ordination of educator training.

• Mindset’s introduction of technology into the classroom, has made a positive contribution to new ways of teaching, by use of online videos as a classroom supplement. Resources such as laptops and projectors, as well as inadequate technical support inhibit the use of this programme.

• New Leaders Foundation has succeeded in instilling processes into the education system, which has resulted in district officials feeling more valued and supported in their day-to-day activities. The programme should however, set clearer short-term goals and assessment criteria so that these can be measured over time, given that the long-term goal to increase learner performance seems distant.

• While FIELD was able to increase beneficiary knowledge in terms of opening a business, issues of certification and selection criteria are areas that need to be addressed.

To ensure that Liberty’s investment in Katlehong is enhanced, the evaluation suggests that greater synergies are built. First, these could exist between the programmes themselves, second, between the programmes, Department of Basic Education and the schools, and third, synergies could be explored between Liberty and other bodies investing in education in South Africa. More effective collaborations, as well as improved co-ordination of the programmes in Katlehong would increase the impact of the programmes within the education system, and result in more positive results for teacher development and learners attending schools in Katlehong. In addition, greater knowledge sharing about the features of programmes which work well (in terms of the programme design, support for programmes and quality in delivery) could be valuable in determining which types of interventions could yield positive results for education in South Africa.

1. IntroductionFor almost 40 years, Liberty has focused on supporting education across South Africa because they believe that their CSI initiatives should contribute to building a thriving economy that is vital to their collective destiny, and which will light the way to financial prosperity for generations to come. As a result, the Liberty Group contributes over R30m each year to community development activities across South Africa.

Liberty works with registered project partners, and their educational involvement is focused on four key areas. By focusing on these specific areas, Liberty is able to support a lifetime of learning within the communities in which they operate.

1. Foundation phase learning• Developing and producing educational material.• Providing direct support to underprivileged primary schools.• Focusing on numeracy and literacy.

2. High school learning• Providing video and interactive media support to learners and teachers.• Providing extra tuition in Mathematics and Science, as well as teaching support for teachers.• Partnering with leadership programmes to identify influential learners and grow their leadership skills.

3. Basic financial literacy• Investing in financial literacy training within the workplaces of its customers and the communities in which they operate.

4. Development of education professionals• Supporting training programmes which aim to enhance the standard of education in the country by focusing on teacher development.

The aim of this research is to evaluate Liberty’s investment in Katlehong, South Africa. Liberty selected six programmes for this evaluation. These programmes include Future Thinking Foundation; Kutlwanong Centre for Maths, Science and Technology; ORT SA; Mindset; the New Leaders Foundation; and FIELD. Apart from the Future Thinking Foundation, Liberty is not the sole funder of these programmes. In addition, Future Thinking Foundation; Kutlwanong Centre for Maths, Science and Technology; ORT SA; and Mindset run in-school programmes with learners and educators. The New Leaders Foundation is based at the Ekurhuleni South Department of Basic Education and FIELD beneficiaries are adults who are no longer in school.

Page 6: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

6 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

2. Education in South Africa Education is the most powerful weapon we can use to change the world (Nelson Mandela)

In 1953, the Bantu Education Act in South Africa ensured that non-Whites1 in South Africa received limited access to education on the basis of their race. The education they received served to inhibit their employment potential and they were deprived of social, political and economic rights (Ocampo, 2004). In 1994, the South African government implemented many policy reforms to amend inequalities in education which were present under the apartheid regime (DPME, 2014). The policies were aimed at improving access to schooling, school management, equity in education funding, and revamped curricula and assessments. Primarily, these pro-poor policies and the introduction of no-fee schools, benefitted learners who would have otherwise not been able to access education due to poverty.

While strides have been made to improve the education system, the South African education crisis is well documented in literature and the Presidency acknowledges that more time is required for the legacy of apartheid education to be fully addressed, and for apartheid patterns of poor school performance to be removed (DPME, 2014). The critical issues in education include poor performance of learners in schools, structural inequality, and lack of parental engagement in their children’s education, problems with language, teacher absenteeism and implementation of curricula.

Much academic and public debate has centred on the apparent and sustained poor performance of the South African Education System. Based on various independent assessments, Spaull (2013) indicated that it is evident that most South African learners “cannot read, write and compute at grade appropriate levels” (p3). In recent literature, a welcome transition to ensure that all children attend school but also that they receive meaningful access to quality education is documented (Dieltiens and Motala, 2012). Poor quality education denies people access to employment. In addition, the transition from school to work and tertiary education for pupils is a challenge given the poor quality of Technical, Vocational Education and Training (TVET) colleges as well as the high attrition rate of tertiary education (van Broekhuizen, 2013). South African research on education has described educator quality, resources and language of teaching and learning as some of the major factors inhibiting quality education of people living in poverty in South Africa (Spaull, 2014). In many schools across the country, there is a low coverage of the curriculum (Spaull, 2014), and a lack of qualified and skilled educators still remains a challenge (Republic of South Africa, 2013).

Shepherd (2011) found that student and family characteristics also contribute to learner performance. What this means is that parent education and employment and students’ own efforts were the most influential factors of performance in Black African schools. In terms of parent education, an inhibiting factor is that most parents from previously disadvantaged backgrounds are unacquainted with quality education and are unable to guide their children through the education system effectively (Amandla Development, no date). Furthermore, educator qualifications and extended teaching time were influential in student performance, however, these were less significant given that a lack of enabling conditions are present in Black African schools. These conditions are inclusive of a lack of effective leadership in schools and lack of parental involvement (Shepherd, 2011). In regard to school leadership, Taylor (2011) found that the aforementioned in addition to adequate curriculum planning as well as learning and teaching support material and low educator absenteeism were positive factors for learner performance. In addition, better curriculum coverage and improved educator development were required.

In terms of infrastructure, Gibberd (2007, p.1) states that “while some schools have excellent infrastructure, others lack basic services such as water and sanitation”. Furthermore, when learner attendance and performance was analysed, the South African Institute of Race Relations indicated that there was a decrease in the number of people aged 20 and older who did not receive any schooling in February 2015 (SAIRR, 2015). In addition, the report indicated an increase in the number of students who received a matric qualification, and the proportion of people that entered post-school education. While these statistics seem promising, the reality is that “if ten children enroll in grade 1 in any given year, one can expect five of them to reach matric, three to pass, and at most, only one to pass Maths with 50%” (SAIRR, 2015, p1).

It is clear that several broad challenges in the South African education system are present and these include poor performance of learners in schools, structural inequality, and lack of parental engagement in their children’s education, problems with language, teacher absenteeism and implementation of curricula.

2.1 Supporting education from ‘cradle to career’While recognising the challenges in education in South Africa, it is important to consider the work that has already been done to counteract these challenges, while being cognisant of the history behind these inequalities. To do this, we need to review the limited existing evaluations of education programmes in South Africa. The areas covered are Foundation Phase leaner materials, extra-curricular Maths and Science, educator development, access to information and communications technology as a classroom addition, school managerial support and entrepreneurial development.

The effects of workbooks and materials on student performance in the Foundation Phase

Numerous sources have pointed to the fact that a lack of workbooks in schools contributes to poor education outcomes (Fleisch, Taylor, du Toit and Sapire 2010; Glewwe, Kremer and Moulin, 2007). The fact that many schools do not have libraries, intensifies the importance of workbooks. In addition, the Foundation Phase is regarded as the building blocks for learner acquisition in numeracy and literacy. It has been argued that if a learner

1 Non-whites include Indian/Asians, Blacks and Coloureds. In South Africa, the term Coloured is a label for people of mixed ethnic origin. The government requires all organizations to report on the number of people registered in terms of four official categories so that it can monitor transformation.

Page 7: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

7An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

fails to master numeracy and literacy at this stage, it is very hard to rectify this later on in their school career. Fleisch et al. (2010) indicate that in order for children to perform well at school in developed countries, they require materials that are “pedagogically sound, culturally appropriate and durable” (p.2). In addition educators using this material need to be trained and learners need to be able to take material home. In contrast, research from developing countries showed that the introduction of textbooks to students have not necessarily led to better performance. Glewwe et al. (2007) demonstrated that while strong students showed significant improvements in performance with the use of textbooks, this was not true for average or weak students.

In conducting a randomised control trial of standard textbooks versus the Primary Mathematics Research Project workbooks which promote higher order mathematical processes in South Africa (Gauteng), Fleisch et al (2010) found that learning improvements were not significantly different between standard workbooks and customised workbooks. This finding revealed the need for evaluations of workbooks provided outside of the schooling syllabus, and a study of perceptions of how these workbooks fare in comparison to material supplied by the Department of Basic Education. There is also a dearth of literature on the effects of different kinds of learner materials over and above textbooks, in learner performance, which should also be explored. For this reason, the Future Thinking Foundation was evaluated.

The effects of extra-curricular Maths and Science classes on student performance

In 2006, Makgato and Mji identified problems in “teaching strategies, content knowledge, motivation, laboratory use and non-completion of the syllabus content over a year” (p262) as some of the factors associated with poor student performance in Maths and Science. In 2009, Mogari, Coetzee and Maritz found that students greatly benefitted from extra-curricular tuition, particularly those dominated by revision of past examination papers. The results were due to this method of tuition being associated with the least cost. In addition, Makgato and Mji (2006) indicate that parents are keen to ensure that their children receive this kind of supplementary education. The reason behind this is that programmes are able to equip children to acquire better results, and parents trust that good results are required in order to enter science-related careers which are financially rewarding and can break the cycle of poverty. Given that the Kutlwanong Centre for Maths, Science and Technology conducts extra Maths and Science classes for high school students over weekends, it was evaluated.

Educator development programmes

With the growing evidence indicating educator quality as a significant contributor to poor learner performance (Spaull, 2014; Bisschoff and Hendricks, 2009), a focus on educator development programmes is welcome. Mestry, Bisschoff and Hendricks (2009) found that educator development programmes are only effective when there is a “coherent and integrated professional development plan” that stems from educators’ commitment to learner success. The results further indicated that educators need to grow in knowledge, skills, values and attitudes to be able to manage classes. ORT SA invests in educator development programmes in Katlehong.

Page 8: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

8 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Access to online lessons as a classroom addition

With the rapid evolution of the IT world, it is also important to understand if online interventions are useful for children in schools. For underprivileged schools, which lack basic services like water and sanitation, technology is a rare addition. Nevertheless, e-learning programmes have been found to be useful and motivational for learners who are given the opportunity to use such platforms (Millham and Thakur, 2014). Some of the challenges experienced in South Africa however, were that some learners were not computer literate and sometimes, issues of internet connectivity hamper student advancement. Despite this, Bester and Brand (2013) found that learners who were exposed to technology within the classroom were likely to develop better concentration, and have higher achievements than those who were not exposed to technology. Mindset has an online presence in schools in Katlehong and was therefore evaluated in this study.

School managerial support and the influence on educator performance

Research conducted by Joseph (2004) indicated that effective school management is essential to school functioning. In order to achieve the goal of quality teaching, Joseph describes the need for effective teamwork and democratic management. For principals of schools, administrative tasks can deter them from focusing on teaching. The results of Joseph’s evaluation showed that for previously disadvantaged schools, the absence of effective school management as well as a lack of physical resources were factors which contributed to poor learner performance. NLF aims to build school managerial support, which is why it was included in the evaluation.

Entrepreneurial development

Botha, Nieman and van Vuuren conducted an evaluation of a women’s entrepreneurship programme in 2006. Their findings showed that respondents had improved management and operational skills in business, as well as increased confidence in running a business on completion of the programme. Entrepreneurship development is seen as a key mechanism which can contribute to job creation and poverty alleviation in South Africa. There is a need however to improve curriculum development and entrepreneurship education. These are the recommendations made by Isaacs, Visser, Friedrich and Brijlal (2007) who also indicate the need to emphasize this type of education in areas where unemployment levels are high. FIELD was evaluated on the basis of its investment in entrepreneurship development.

2.2 Aims and objectives of the research The overall aim of the research was to develop and implement a three year evaluation framework for selected education and skills development initiatives in order to determine how Liberty’s investments have shaped educational, skills and financial literacy outcomes in Katlehong.

In order to develop and implement the evaluation system for the programmes the following objectives were formulated:1. Development of evaluation assessment tools for the identified programmes running in Katlehong.2. Capturing and analysis of initial interviews conducted with all participants.3. Conducting follow-up evaluations interviews for the programmes.4. Development of an overview of the influence of Liberty’s investment in Katlehong.5. Ascertaining whether beneficiary perceptions of Liberty have been impacted by the programmes being run in Katlehong.

2.3 Structure of the reportThis introduction on education in South Africa is followed by the overall research methods used in the study (part 3). The report then discusses each programme in relation to the specific methodologies, results and recommendations in part 4. Finally, some general commentary about Liberty’s investment in Katlehong is contained in part 5.

Page 9: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

9An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

3. Research methodsThe evaluation process made use of a mixed methods research design. Mixed methods research is defined as a technique which combines quantitative and qualitative research methods into a single study (Delport and Fouche, 2011). This method is useful as it provides a more comprehensive analysis which is achieved by integrating approaches which each technique by itself would inadequately address (Creswell and Plano Clark, 2011; Delport and Fouche, 2011). In order to conduct the evaluation, the CSDA designed quantitative and qualitative research instruments. The evaluation consisted of rapid appraisals which allowed for immediate feedback on the progress of the programmes for reporting to Liberty. In addition, an impact evaluation enabled the identification of programme elements which contributed to changes in participant knowledge and skills (World Bank, 2004). To measure impact, the quantitative component enabled performance tracking over time where possible. This highlighted improvements in skills development which were potential outcomes of the programmes. The qualitative interview schedules and focus group guides were designed to capture beneficiary perceptions of the social impact of the programmes, the elements that may have contributed to the impact, perceptions of the benefits of the programmes and insights into the changes in perceptions of Liberty. In-depth interviews are an informal and less structured means for conducting evaluations (World Bank, 2004). In order to enhance the validity and reliability, all research instruments were pre-tested before they were administered to the target groups. More detail on the specific methodology for each programme appears in part 4.

3.1 Sampling strategyFor each of the programmes, the research team requested lists of individuals who attended the trainings provided. Using these lists, systematic sampling was conducted to select participants for the qualitative interviews. Systematic sampling is a form of probability sampling which is the selection of participants based on a form of random procedure (Strydom, 2011). The methods of sampling used in the quantitative components, as well as other relevant data such as the participating schools, are explained in part 4.

3.2 Data analysisData collected from the qualitative interviews were analysed by thematic analysis using Atlas Ti. Atlas Ti is a data management software system that allowed the research team to conduct the qualitative analysis electronically. Thematic analysis refers to the identification of patterns within the data (Braun and Clark, 2006).

Analysis of the quantitative data was conducted using the Statistical Package for Social Sciences (SPSS). The analysis made use of descriptive techniques (frequencies) and inferential statistics (Chi-Square tests) to analyse differences in marks at baseline and post-programme (Kranzler, 2003).

3.3 Ethical considerationsThis research was conducted with a range of participants, including minors, principals and educators in schools, officials at the Ekurhuleni South Department of Basic Education and adult recipients of the education and skills development programmes.

The research obtained ethical clearance from the University of Johannesburg’s Research Ethics Committee. In addition, where minors were interviewed the research team obtained permission from the Ekurhuleni South Department of Basic Education to conduct interviews with the minors in schools, during the school semester. Parental consent was received for minors, with the assistance of the schools and the Ekurhuleni South Department of Basic Education. Only learners who returned signed parental consent forms were allowed to participate in the research, provided that they also provided us with their verbal assent. Furthermore, where necessary, interviews with learners were conducted in isiZulu, which is the dominant language in Katlehong. Schools were assured that participation in this research would not affect the continuation of the programmes in any way.

For all participants, the purpose of the research was explained, and informed consent was obtained. Responses were treated as confidential, and the participants were ensured anonymity in this report. Furthermore, all participants were allowed to exit the research at any point, without repercussions. There were no specific ethical dilemmas raised during the course of the research.

3.4 Limitations of the research A limitation of this study pertains to the fact that the programmes were already running prior to the start of the evaluation. This meant that baseline data collection was conducted after the introduction of the programmes into the schools.

Furthermore the qualitative components of the research were conducted with small numbers of participants. This meant that conclusions drawn from these findings cannot be generalised to the programmes if they are run in areas outside of Katlehong. Given that qualitative research is not intended to be generalisable to the broader population, its value is in the fact that the interviews allowed us to explore stakeholder perceptions of the programmes in Katlehong.

Despite these challenges, this study still demonstrates effects of the programmes. The research was able to outline improvements in schools based on the fact that these improvements were greater with increased length of exposure to the programmes. In addition, the research encompassed input from all stakeholders, including the facilitators of the programmes, district officials, educators and learners. The results presented are therefore comprehensive which strengthens the conclusions drawn from the evaluation.

Page 10: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

10 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

4. Results The report now discusses each programme in relation to a brief programme description, methodologies specifi c to the programme, the results of the evaluation and recommendations made for the programmes. There were six programmes in total: Future Thinking Foundation (4.1); Kutlwanong Centre for Maths, Science and Technology (4.2); ORT SA (4.3); Mindset (4.4); the New Leaders Foundation (4.5) and FIELD (4.6).

4.1 Future Thinking Foundation

4.1.1 Programme description

The Future Thinking Foundation encourages the use of educational material in schools to promote improved scores in Mathematics and isiZulu. In order to achieve this goal, the Future Thinking Foundation developed a comprehensive set of resources, designed to support educators, parents and children. In addition, they provide training on the use of the material. The range of resources is called “Learn like Lerato”, and includes a series of activity books, as well as paints, crayons, beanbags, number charts and other educational material. Learn like Lerato is based on the Curriculum and Assessment Policy Statement (CAPS), and is available in English, isiZulu, isiXhosa and Northern Sotho. Learn like Lerato covers Grades 1, 2 and 3 and is incorporated into lessons at school. The programme is run in Gauteng, KwaZulu-Natal and the Western Cape.

4.1.2 Research methodology

The Future Thinking Foundation was evaluated from 2013 to 2015. For the quantitative component of the evaluation, all Grade 3 learner marks were analysed in order to determine if the average marks of the learners were improving after exposure to the Grade 3 material. A total of 139 learner assessments were analysed in 2013 and 259 were assessed in 2014. Assessments were not analysed in 2015, due to the end point assessments not being completed at the conclusion of the evaluation report. Table 1 below shows the structure of the fi eldwork over the three years.

For the qualitative component of the evaluation, three focus groups with 25 randomly selected Grade 3 learners were conducted in 2013. These focus groups were conducted in Nthuthuko, Thulisa and Thokoza Primary Schools. In 2015, 15 Grade 3 learners in the same schools were interviewed in order to determine how diff erent learners perceived the programme, and whether they felt that the material had helped them with Maths and isiZulu. In addition, seven foundation phase educators were interviewed in 2013 and 2014 in order to gain an understanding of educator perceptions of the programme in terms of training and the usefulness of the materials.

Table 1 provides a summary of the fi eldwork activities that were conducted with learners and educators, including an analysis of the learner marks.

Table 1: Fieldwork activities for Future Thinking Foundation 2013-2015

Activity 2013 2014 2015

Learner interviews

Educator interviews

Analysis of marks

4.1.3 Quantitative results

Table 2: 2013 – Grade 3 Maths and isiZulu marks (N=139)

Subject Mean marks at baseline % Mean marks at year end %

Mathematics 49.4 59.6

isiZulu 62.8 69.8

Table 2 shows that in 2013, there was a signifi cant increase in Mathematics scores following the delivery of the material at schools (increase of 10.2%; p<0.05). Although there was also an increase in isiZulu scores, this increase was not statistically signifi cant (increase of 7.1%).

complete foundation phase solutions

Page 11: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

11An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Table 3: 2014 – Grade 3 Maths and isiZulu marks (N=259)

Subject Mean marks at baseline % Mean marks at year end %

Mathematics 53.9 71.8

isiZulu 61.6 70.6

In 2014, significant increases in results were seen for both Mathematics and isiZulu scores (an increase of 17.9% and 9%, respectively; p<0.05). This means that the material provided by the Future Thinking Foundation seems to have more significant impacts on Mathematics and isiZulu scores, as educators are becoming more comfortable with the material over time. These results are set out in Table 3.

4.1.4 Qualitative findings

The findings for Future Thinking Foundation include learner and educator perceptions about the programme from 2013 to 2015. In addition, some learners wrote letters to Liberty, to indicate how they felt about the investment that Liberty made.

4.1.4.1 Educator perceptions of Future Thinking Foundation

Most educators had positive impressions about the learning outcomes associated with using the “Learn like Lerato” kit in 2013. They felt that learners’ concentration, fine and gross motor skills and creativity had improved. In addition, Mathematics and isiZulu had also improved.

“So when you introduce your material to learners, they pay attention because they want to know what is it that they’re going to do on that day”. – Future Thinking Foundation educator

“When you use the [Future Thinking Foundation material], you notice an improvement with fine motor plus hand-eye co-ordination”. – Future Thinking Foundation educator

“In terms of addition, subtraction, multiplication and division, I can say these apparatus helped to improve their grades”. – Future Thinking Foundation educator

While many educators expressed positive views of the “Learn like Lerato” kit, unfortunately due to the lack of educator training by Future Thinking Foundation, kits were not used as well as they could have been in 2013. This resulted in negative views being expressed by the educators, in relation to the programme and funder. We attribute this finding to educators being under the mistaken impression that Liberty was meant to provide training. As a result, the material was not being used effectively by the educators:

“We need more training because if you have more knowledge, and know everything about the kit, you will do right with the learners”. – Future Thinking Foundation educator

“When I came back I got the report that Liberty are coming for the training, we must wait for them so that we can use the material, but nothing has been done”. – Future Thinking Foundation educator

In 2014, the Future Thinking Foundation paid particular attention to the training of educators on the use of the material provided. As a result, the educators were appreciative of the “Learn like Lerato” kits and were extremely grateful that Liberty was investing in education. The kits provided educators with extra support, particularly when trying to think of simpler and more understandable ways of demonstrating concepts to the learners. Educators showed great enthusiasm for the programme and stated confidently that “Learn like Lerato” aided remarkably in teaching:

“They improved this programme. They taught us how to use the kits, and showed us how to use the old and new material”. – Future Thinking Foundation educator

“It’s clear. We don’t spend too long explaining the examples…it has changed our ways of teaching…and it has motivated us to be happy with our profession which is normally taken for granted”. – Future Thinking Foundation educator

Educators expressed the view that the kits helped learners with number operations and that the workbooks made use of correct Zulu terminology and rich grammar, which in their opinions, did not exist in government workbooks:

Page 12: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

12 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

“The kits helped learners with number operations and is using grammatically correct Zulu terminology”. – Future Thinking Foundation educator

“The [Learn like Lerato] workbooks are more straightforward and rich in language. The Department of Basic Education books uses words that are not correct. Learn like Lerato uses correct grammar and is richer”. – Future Thinking Foundation educator

The educators indicated how they perceived the usefulness of the material supplied by Future Thinking Foundation:

“The peg boards and bean bags were the most versatile and useable as I can use them for Maths, English and Life skills”. – Future Thinking Foundation educator

“The multiplication chart is very useful. It is colourful and clear so it is easier for them to know their tables. CD’s improve listening skills; block boxes with shapes, and there is also painting”. – Future Thinking Foundation educator

Given that educators were motivated by the kit, it was recommended that “Learn like Lerato” be implemented across South Africa and be extended to grades above the Foundation Phase, especially where learners have difficulties in reading and writing. There is also a need to integrate “Learn like Lerato” with CAPS material so that both materials can be implemented simultaneously and successfully into classes:

“Maybe one thing I would like to see is that it should be across South Africa. No child should be denied the opportunity to interact with Learn like Lerato”. – Future Thinking Foundation educator

“We are running two programmes concurrently. It is hard for us…we need to integrate the two systems [Future Thinking Foundation and CAPS]”. – Future Thinking Foundation educator

4.1.4.2 Learner perceptions of Future Thinking Foundation

In 2013, learners expressed very positive reactions to the programme - with many stating that they were “excited” about using the materials in class. In addition, learners felt that the programme had helped them with Mathematics and isiZulu.

“It’s more exciting now in class, now that Lerato is here”. – Future Thinking Foundation Grade 3 learner

“It was boring before Lerato came. Now that she is here, class is fun”. – Future Thinking Foundation Grade 3 learner

“Lerato helps us to count using our minds not calculators”. – Future Thinking Foundation Grade 3 learner

“I’m really happy that Lerato is here, she helps me with reading isiZulu and I like her stories”. – Future Thinking Foundation Grade 3 learner

In addition, learners indicated that their concentration had improved since the inception of the programme:

“I used to lose concentration but now I can concentrate and focus”. – Future Thinking Foundation Grade 3 learner

Learners also specified that having the material in classrooms made them more enthusiastic about school:

“I used to be absent in school but now that Lerato is here, I’m no longer absent from school”. – Future Thinking Foundation Grade 3 learner

“I don’t want to be absent from school anymore”. – Future Thinking Foundation Grade 3 learner

Overall, learners felt that the programme was useful, and made learning easier for them. In addition, the learners identified with Lerato as a source of support:

“I love Lerato. She made things easy for me that were difficult. Open my brains”. – Future Thinking Foundation Grade 3 learner

Page 13: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

13An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

In 2015, learners indicated that the programme contributed to an increase in their marks for Maths. They also felt that they learnt how to read and write using the programme:

“In maths I had 96; I love maths”. – Future Thinking Foundation Grade 3 learner

“For maths I got 76, then my mark went up and I got 95”. – Future Thinking Foundation Grade 3 learner

“I learnt how to read and write using the spelling tests and letter blocks. I also like the number blocks because it taught me how to add and multiply”. – Future Thinking Foundation Grade 3 learner

The learners felt that as school became more difficult, the programme made it easier for them to pay attention in class. In addition, the programme helped learners with other subjects such as English, given that the Maths workbooks were printed in English:

“[My attention has increased] because from Grade 0-2 we deal with simple things. In Grade 3-4 you have to focus and pay attention to your teacher”. – Future Thinking Foundation Grade 3 learner

“English is the most difficult subject and Lerato helps me a lot”. – Future Thinking Foundation Grade 3 learner

The quantitative results were consistent with the qualitative findings in highlighting the value of the “Learn like Lerato” kit in schools. The qualitative findings specifically provided an indication of what learners and educators appreciated about the programme.

4.1.4.3 Perceptions of Liberty as a funder

Learners were asked to express their feelings around having Liberty sponsor Future Thinking Foundation. These letters are from the learners themselves to Liberty:

Figure 1: Letters to Liberty from Future Thinking Foundation learners

Page 14: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

14 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

4.1.5 Conclusions and recommendations

The following conclusions can be made regarding Future Thinking Foundation: • Future Thinking Foundation has provided materials to schools that have assisted in increasing marks for foundation phase Maths and isiZulu. The

increases were 10.2% and 7.1%, respectively in 2013. In 2014, the learner marks increased by 17.9% and 9% for Maths and isiZulu, respectively. • Although it was not intended, the material also assists learners with improvements in English.• Learners feel motivated and enthusiastic about school as a result of the programme. • Educators were inspired by the programme, and felt that the material was of a better quality than material supplied by the Department of Basic

Education.• Educators indicated difficulties in incorporating both materials into lesson plans. • Liberty’s investment in Future Thinking Foundation was viewed as positive.

The Future Thinking Foundation programme is a well-run programme that is having the desired results on learners. This was more evident after the difficulties around teacher training were resolved. The only recommendation which arises from the evaluation is for Future Thinking Foundation material to be more adequately integrated into the school curriculum. This could result in more optimal use of the material in schools in Katlehong.

4.2 Kutlwanong Centre for Maths, Science and Technology

4.2.1 Programme description

The Kutlwanong Centre for Maths, Science and Technology offers extra Mathematics and Physical Science education to Grade 10-12 learners in previously disadvantaged communities. The programme runs in the Eastern Cape, Western Cape, Free State, Mpumalanga, KwaZulu-Natal and Gauteng. Kutlwanong uses educators from the schools in Katlehong to provide extra-curricular classes to learners over weekends. Kutlwanong identifies Maths and Science as key catalysts for students to enter globally recognised fields such as finance and science related professions. Hence, its motivation is to facilitate access to these subjects, removing obstacles and perceptions that these fields are difficult to enter into, and to empower learners to become the leaders of tomorrow. Kutlwanong serves many schools in Katlehong and classes are run by locally based educators from these different schools. Classes are held in Katlehong at the Ponego Secondary School, where educators and learners from the surrounding schools congregate for the classes.

4.2.2 Research methodology

Kutlwanong was evaluated from 2013-2015. In 2013, Grade 11 learners on the programme were interviewed. These learners were tracked in Grade 12 and again post-matric in order to determine if the programme had any impact on learners once they left the schooling environment. For the quantitative component of the evaluation, all learner marks were analysed in order to determine if the average marks of the learners were improving over time. A total of 119 learner assessments were analysed, tracked and reanalysed over two years.

For the qualitative component of the evaluation, a total of 15 randomly selected learners participated in seven focus groups over the three years. These learners were from many schools in Katlehong including: Phumulani Secondary School, Ponego Secondary School, Katlehong Technical High School, Fumana Comprehensive School, amongst others. In addition, three educators were interviewed in 2013, in order to understand educator perceptions of the programme.

4.2.3 Quantitative results

Table 4: 2013 – Grade 11 Maths and Science marks (N=119)

Subject Mean marks at baseline % Mean marks at year end %

Mathematics 16.7 46.1

Science 19.2 46.8

In 2013, there were statistically significant increases (p<0.05) in Mathematics and Science scores over the year. The results revealed a marked improvement of 29.3% for Mathematics and 27.6% for Science.

Page 15: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

15An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Table 5: 2014 – Grade 12 Maths and Science marks (N=119)

Subject Mean marks at baseline % Mean marks at year end %

Mathematics 54.6 77.7

Science 52.3 73.2

In 2014, again, there were statistically significant increases (p<0.05) in Mathematics and Science scores over the year. The results revealed a marked improvement of 23.1% for Mathematics and 20.9% for Science. The final scores for these subjects were on the upper end of the scale, and displayed massive improvements in the students, compared to the beginning of 2013. This showed that over time, Kutlwanong was hugely beneficial to this cohort of students.

4.2.4 Qualitative findings

The findings for Kutlwanong include learner and educator perceptions about the programme from 2013 to 2015. Educators were interviewed in 2013 only.

4.2.4.1 Educator perceptions of Kutlwanong

In 2013, educators involved in Kutlwanong said that they noticed an improvement in the discipline, performance and team-work of the students, as a result of the programme:

“When we started learners were not disciplined but as time goes on we emphasize discipline…they start actually changing their behaviour and study method”. – Kutlwanong educator

“You realise that those who are in the programme are doing much better than those who are not when you start comparing”. – Kutlwanong educator

“Previously they worked as individuals but when they got on the programme, they work as a team…and that improves their performance”. – Kutlwanong educator

Educators also stated that the interaction they had with colleagues from other schools assisted them in improving their teaching methods:

“We interact with other educators and we actually learn a lot from each other…their approach is different”. – Kutlwanong educator

“Your teaching improves, you can see as an educator where to polish yourself”. – Kutlwanong educator

Some of the challenges identified by the educators however were a lack of laboratory equipment, clean teaching facilities and under-appreciation for educators:

“We don’t have laboratories, like the material for experiments. All in all, we lack in resources”. – Kutlwanong educator

“Sometimes we find [the class] dirty, and it also affects our lessons as learners have to clean first”. – Kutlwanong educator

“For teachers, there is little appreciation”. – Kutlwanong educator

4.2.4.2 Learner perceptions of Kutlwanong

In 2013, the overall perception of Grade 11 students was that the programme has helped them achieve a significant improvement in their marks for Maths and Science:

“It has really helped me a lot in terms of my academic performance. It was like taking performance enhancers, like steroids”. – Kutlwanong 2013 Grade 11 participant

“I would say that I got high marks [before] but it was just above 40%, but in the beginning of this year I started improving to 65% and in September I got 85%. I am improving”. – Kutlwanong 2013 Grade 11 participant

Page 16: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

16 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

In addition, some students also reported that they were more confident personally and academically as a result of being a part of the programme. In addition, being a part of Kutlwanong motivated learners to do well:

“At first I thought school was uncool…now I really enjoy studying and it has really boosted my confidence about my studies”. – Kutlwanong 2013 Grade 11 participant

“I’m hungry for education, I know what I want to be and I focus on what I have to do”. – Kutlwanong 2013 Grade 11 participant

“Before you come here you know you have to be prepared… you have competition, and most learners are intelligent”. – Kutlwanong 2013 Grade 11 participant

While some students were able to participate in extra-mural activities, their participation has diminished due to their involvement in Kutlwanong. Despite this, being a part of the programme helped them to form new friendships with their peers at Kutlwanong and increased motivation to improve their performance. Some students also reported that they no longer interacted with peers who did not prioritise their studies:

“I’m also playing netball. My teacher told me that he’ll cancel me because every time they call me, I’m too busy with Kutlwanong”. – Kutlwanong 2013 Grade 11 participant

“I’ve met positive friends that have positive attitudes”. – Kutlwanong 2013 Grade 11 participant

“I have friends at church who understand and motivate me to study…and keep me inspired”. – Kutlwanong 2013 Grade 11 participant

“Friends from outside the programme, I can say I’ve lost them…I’ve met new friends here who are positive towards life”. – Kutlwanong 2013 Grade 11 participant

Although the educators indicated that they provided career guidance for students in 2013, the students identified the lack of career counselling as a shortcoming of the programme. This contradiction seemed to be dealt with in 2014 when students identified the importance of the career guidance they received from the programme which indicates that the career guidance provided has improved in quality:

“There was this time when we had professional career people here, they told us the things that they do and it was a motivation for me. We have the mentality that if you are from the township, it is difficult to be successful but they made it look easy. It was beneficial”. – Kutlwanong 2014 Grade 12 participant

“The importance of applying early at University. I’ve seen people suffer this year because of not applying. When I got here this year, during the career expo, they told us we have to apply early…so it helped me”. – Kutlwanong 2014 Grade 12 participant

Learners continued to indicate that the Kutlwanong programme had given them confidence in their academic work and helped them become focused and goal orientated. Examination preparedness, increased ability to complete homework and manage time, as well as a drive to be academically superior are some aspects which learners attributed to Kutlwanong. Learners described the level of work they do at Kutlwanong as being tough and challenging, therefore enabling them to be more prepared for their upcoming matric examinations.

“There are times when you get lazy…you think that other might be working…and you don’t want to get lower marks. So it’s motivation to always do something”. - Kutlwanong 2014 Grade 12 participant

“Being here has made me become more prepared and now I don’t expect the basic questions, now I want the tough and challenging questions that make you sweat so I am more prepared”. - Kutlwanong 2014 Grade 12 participant

“Maybe it’s better because we being part of such a project, we feel so much pressure and we learn how to manage pressure…if you can survive here, you can surely survive out there”. - Kutlwanong 2014 Grade 12 participant

Although the Kutlwanong programme is labour intensive, the motivation and commitment from the learners is overwhelming. They expressed a great sense of appreciation for being a part of the programme:

“I feel honored being here because it is not available to everyone”. - Kutlwanong 2014 Grade 12 participant

“I feel privileged”. - Kutlwanong 2014 Grade 12 participant

Page 17: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

17An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

The learners found the educators inspiring and they indicated that they teach in ways that make the students want to learn. One of the learners was so inspired by the educators that she wanted to be a science teacher herself. Others listed engineering, forensic psychology and commerce as areas of interest for further study. Many of the students had already been accepted into some of these courses.

“The programme has been beneficial because it helped me get a mentor”. - Kutlwanong 2014 Grade 12 participant

Top achievers surprised at full marksSOUTH AFRICA – by Bianca Bothma

Kabelo Choma of Phumlani Secondary School in Katlehong achieved 100% for mathematics in the 2014 matric examinations. Photo: eNCA/Bianca Bothma

Photo reprinted with permission from Kabelo Choma

JOHANNESBURG – Tshegofatso Mokwena of Reitumetse Secondary School in the Tshwane district was shocked when she found out she had no wrong answers in her accounting examination.

“I never knew it was possible to get 100 percent for accounting,” she said.

Her school was the top school in Gauteng and achieved the most Bachelor passes. The 18-year-old got six distinctions and she was one of a number of students honoured at an awards ceremony on Tuesday in Parktown by the Gauteng Education Department. Gauteng was the top performing province nationally with a 2014 pass rate of 84.7 percent. Four out of the five top performing students nationally are from Gauteng. Kabelo Choma says study groups with classmates are a major part of what helped him remember complicated math formulas and ultimately achieve 100 percent in the 2014 national senior certificate examination in mathematics. The 18-year-old from Katlehong went to Phumlani Secondary School and was the top matriculant in his school.

“I worked with friends in study groups, that helped me a lot,” said Choma.

Life sciences teacher at the school, Sipho Moloi, says the school achieved a 93 percent pass rate and 243 learners wrote the exam. Overcrowding along with an old building, no science and computer facilities are some of the challenges facing learners.

“We have an average of 45 learners per class. Poverty is a big issue in this community, many learners come to school hungry and substance abuse is also a problem in the area,” said Moloi.

Despite these challenges, Choma managed to obtain seven distinctions. Extra classes on weekends at Kutlwanong Centre for Maths, Science and Technology also helped a great deal, said Choma.

Aspiring to one day become a mathematician, Choma is planning on studying mathematical sciences at the University of the Witwatersrand in 2015. He says his mother who does temporary work has been incredibly supportive.

“When we did all-nighters, she made sure I always had food,” he said.

Michael Bila of Ponego Secondary school also in Katlehong achieved 100 percent in mathematics and physical science.

“I was really surprised, I keep waiting for someone to tell me it was a joke,” he said.

Bila is the top physical science student in the country, and also attributes his success to study groups with friends and extra classes on weekends at Kutlwanong. He is planning to study actuarial science in 2015.

Article reprinted with permission from Biance Bothma. eNews Channel Africa: www.eNCA.com

Page 18: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

18 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

4.2.4.3 Post-school performance

The learners were also contacted in 2015 to find out how they were coping outside of school. Many of the students had been accepted into Universities and were studying further at the University of Pretoria, University of the Witwatersrand and the Walter Sisulu University. The learners indicated that this was a huge achievement for them:

“I am studying at one of the best Universities in Africa because of Kutlwanong”. - Kutlwanong 2015 beneficiary

The learners indicated that Kutlwanong had prepared them for University, as it taught them self-discipline. For some learners however, it was more difficult to cope:

“Kutlwanong gave us tests regularly which means we had to study…just like University does…because I was part of Kutlwanong, I am comfortable working under those conditions”. - Kutlwanong 2015 beneficiary

“Yoh! Freedom is sucking my academic potential. It’s hard to cope. My priorities are changing at an increasing rate”. - Kutlwanong 2015 beneficiary

Overall, learners still felt connected to Kutlwanong, which helped them focus and get through the pressures at University:

“They’ve formed a Kutlwanong Society for University students. We were welcomed by former Kutlwanong students”. - Kutlwanong 2015 beneficiary

“I’ve met a lot of people at Kutlwanong whom I go to University with…it’s like a big family”. - Kutlwanong 2015 beneficiary

Page 19: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

19An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

4.2.4.4 Perceptions of Liberty as a funder

Learners were asked to express their feelings around having Liberty sponsor Kutlwanong. These letters are from the learners themselves to Liberty:

Figure 2: Letters to Liberty from Kutlwanong learners

4.2.5 Conclusions and recommendations

The following conclusions can be made regarding Kutlwanong Centre for Maths, Science and Technology: • The programme has resulted in increased academic performance (in Maths and Science) for learners in Katlehong. • Learner confidence has also been improved as a result of being a part of the programme. • The data reveals that the programme instils positive lifestyle and behavioural changes for learners. • Learners developed new friendships with peers at Kutlwanong, which was a source of support and increased motivation to do well at school. • Educators indicated that the differences in terms of academic performance, as well as discipline were stark when comparing students who

were a part of the programme to those who were not. • Educators felt that more could be done to support them when teaching students on the programme (specifically in relation to equipment). • The programme makes use of local educators and venues for classes, and highlights a method of including local infrastructure and resources to

improve learner outcomes.• Learners indicated that due to the complex nature of the material presented by Kutlwanong, they were able to prepare with ease for school

examinations.

Page 20: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

20 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

• Learners found that the career expos held by Kutlwanong were a vital part of being accepted into University.• The majority of the learners attributed their success at University to the discipline that they had established at Kutlwanong.

The Kutlwanong programme is a well-run programme that is having the desired results on learners. The recommendations which arise from the evaluation is for Kutlwanong to make use of the Mindset teaching aids (see part 4.4), which will assist with visual representations of experiments. This could assist in making the content more appealing to students, especially in an area like Katlehong, where resources such as laboratories are limited. In addition, support for Kutlwanong clubs at Universities could enable the clubs to provide greater support to Kutlwanong alumni in order to ensure that their transition from school to University is smoother.

4.3 ORT SA

4.3.1 Programme description

The Liberty – ORT SA Mathematics Programme commenced in 2015 and was implemented in Gauteng, in primary and secondary schools. The programme consisted of content training workshops and classroom based support for educators, delivered by ORT SA. The training workshops were conducted for educator development which is vital to sustained impact on teaching and learning. Educators received training on a weekly basis in primary schools on Saturdays, once per month in secondary schools. Classroom based support entailed lesson observation, co-teaching with educators as well as providing constructive feedback. The programme targeted Grade 4 as the entry point for the Intermediate Phase, and Grade 8 as the entry point in the Secondary schools. Training content was created by ORT SA and is in line with the CAPS. ORT SA training takes place in Gauteng and KwaZulu-Natal.

4.3.2 Research methodology

ORT SA was evaluated in 2015. To conduct the evaluation, in-depth interviews were conducted with educators in Katlehong who had attended the ORT SA training. A total of six interviews were conducted, with randomly selected educators from the following schools: Nthuthuko and Thulisa Primary Schools and Phumulani Secondary School. The interviews were conducted in order to understand some of the successes of implementing the programme, as well as some of the challenges experienced.

4.3.3 Qualitative findings

The results for ORT SA are presented separately for primary schools and secondary schools. This was done to allow challenges and successes to be identified at each level.

4.3.4.1 Primary school educator perceptions of ORT SA

The overall feeling around the ORT SA programme in primary schools was that it is beneficial to educators, and it is in line with the school curriculum which makes it indispensable. This was particularly true for educators who were not teaching at the level that they were trained to teach:

“There is a huge change to me…I can even feel for myself when I am explaining to the learners, I am at the right level”. – ORT SA participant

“What we do this week is what we have to teach next week…you can prepare”. – ORT SA participant

“My qualification said I must teach secondary school…now I have to drop low…I am more confident with that since I joined ORT”. – ORT SA participant

“The last time I learnt Maths was in Grade 9, so since I was in this programme, I began to understand concepts of Maths [so I could teach them]”. – ORT SA participant

Educators also liked the fact that different schools host the workshops, and so they get to interact with colleagues. In addition, they felt that the training workshops were well designed:

Page 21: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

21An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

“I think the structure is good...All schools are hosting”. – ORT SA participant

“I think it [training] was fruitful and beneficial…it made things more practical”. – ORT SA participant

“The training was excellent. We were benefitting a lot”. – ORT SA participant

Even though educators did not have criticisms of the content of the training workshops, they felt that time constraints were a challenge in relation to the trainings:

“Just the time. I don’t know when they can have it. Weekends are busy. After school we are tired”. – ORT SA participant

“At times we can’t go to ORT because we have to attend district workshops. The time doesn’t allow both”. – ORT SA participant

When asked about the classroom observations, educators expressed mixed feelings about the organisation and usefulness of the trainings:

“[The trainer] watched me and guided me, because I told [the trainer] I am from the senior phase [and am now teaching primary school]. [The trainer] guided me very well”. – ORT SA participant

“They can plan better and be here when they can observe a class”. – ORT SA participant

The following opinions were expressed in relation to the material needed:

“Working materials…sometimes you need it to demonstrate to the class, but the school does not have any”. – ORT SA participant

The major challenges in relation to ORT SA were therefore inclusive of time constraints in attending the training workshops, planning in relation to classroom observations and a lack of sufficient learning materials to use in classes.

4.3.4.2 Secondary school educator perceptions of ORT SA

For secondary school educators, the programme has been beneficial, and created an environment where educators get to learn:

“We work as a group, we present – you see your mistakes. And then you get other methods from different people”. – ORT SA participant

Some educators, however, felt that the material was not in line with the syllabus they are teaching at the school. This raises concerns given that the ORT SA material was meant to be in line with the school syllabus:

“They had Maths content which we did not cover in our syllabus at school. We took a little bit but at least I have knowledge about it”. – ORT SA participant

Nevertheless, educators felt that the programme had made a difference in their approach to teaching and positively impacted their confidence:

“I am more confident, even when I present the lesson”. – ORT SA participant

“Wow! The new strategies and new information. You see it and say okay let me change”. – ORT SA participant

Some of the challenges experienced by high school educators were the large student to educator ratio, as well as insufficient material to incorporate into lessons. In addition, feedback from classroom observations should be left with the educator so they remain conscious of areas that they need to improve in. These aspects affected the implementation of the programme in the classroom setting:

Page 22: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

22 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

“I would change the ratio and make it lesser so that the educator could focus on all learners”. – ORT SA participant

“They don’t give us materials, we just go to the workshop and write a test and then discuss other things. Maybe they can give us material for the learners”. – ORT SA participant

“For the observation, it was useful…but I can’t remember now. He took the report so I don’t have a copy”. – ORT SA participant

The high school educators only indicated attending one training session conducted by ORT SA. For this reason, they thought that it was too soon to comment on whether the skills they are learning at ORT SA are impacting the learners in any way. They said the following when asked if they would like to attend more trainings:

“We need more frequent sessions but programmes clash. They could give us documents, and maybe come to us during lunch. Rather than waiting to run a training with five schools”. – ORT SA participant

“I hope this interview will touch them to come back”. – ORT SA participant

4.3.4.3 Perceptions of Liberty as a funder

Participants expressed the following perceptions when they were asked how they felt about Liberty’s investment in ORT SA:

“This is helpful…especially to learners, educators and South Africans…Maths educators do need training. A lot of training. If I can tell you…some educators are afraid of Maths so they just throw the new educators to Maths…you start to feel the pressure”. – ORT SA participant

“I think it’s good because we are building the future of these little ones, and if we invest in them, our country is going to be somewhere. Knowledge is a good investment, I am one of those Liberty fans”. – ORT SA participant

4.3.5 Conclusions and recommendations

The following conclusions can be made regarding ORT SA: • ORT SA has made positive changes to educator knowledge and skills in Katlehong. Educators feel more prepared for their classes, and feel

more confident as a result of the trainings. • Educators felt that time challenges, large students to educator ratios and a lack of teaching materials threatened the effectiveness of the

programme. • Secondary school educators felt that it was too soon to gauge if ORT SA has made any impact on the learner performance.• Liberty’s investment in the ORT SA programme was viewed as extremely positive, given the challenges in training Maths and Science educators

in South Africa.

The following recommendations are made in order to strengthen ORT SA:• Trainings for high schools educators could be more frequent. In addition, supplementary material at both primary and high school levels which

educators can use to demonstrate the content of the programme practically to students would be useful.• ORT SA could include a module on how to manage large student to educator ratios, in the programme. • ORT should provide educators with copies of the feedback reports, immediately after classroom observations, so that educators remain

cognisant of the areas they need to improve in. • The classroom observation sessions need to be planned so that they can occur when classes are being taught, and this requires improved

communication between the schools and ORT SA.

Page 23: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

23An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

4.4 Mindset

4.4.1 Programme description

Mindset is an organisation that designs and implements education solutions for the formal education and health sectors. The solutions for education take the form of developing educational content by in-house content experts, and distributing this content via a range of multiple touch points, including Mindset’s own broadcasting infrastructure which has a national reach. The educational programme follows the focus and pace of CAPS aligned education, by broadcasting lessons that reinforce key topics through an expert educator in a live studio setting. Viewers are able to engage with the show presenters and also access content following the broadcasts. In addition, lessons can be accessed online for revision purposes.

4.4.2 Research methodology

Mindset was evaluated in 2015. To conduct the evaluation, in-depth interviews were conducted with educators in Katlehong who had attended the training on how to use the Mindset modems which were installed in schools. These modems were installed so that Maths and Science educators could access supplementary information in order to assist them in teaching learners who were not part of the Kutlwanong Centre for Maths, Science and Technology. A total of four interviews were conducted, with randomly selected educators from Eketsang and Phumulani Secondary Schools. The interviews were conducted in order to understand some of the successes of implementing the programme in Katlehong, as well as some of the challenges experienced.

4.4.3 Qualitative findings

The qualitative results highlighted the fact that Mindset filled an important space in the education system, by including technology into schools. This came with challenges that need to be addressed in order to make Mindset’s impact greater.

4.4.3.1 Perceptions of Mindset

Overall, educators felt that Mindset was a positive programme, given the evolution of the education system in South Africa:

“Education has gone smart”. – Mindset participant

“The paradigm has shifted, it has changed”. – Mindset participant

“I like what the government has done with the introduction of technology”. – Mindset participant

As a result of the shift in teaching methods, from traditional teaching methods towards e-learning, some educators expressed challenges with adjusting their teaching styles:

“There is no way I am going to be confident about the old teaching methods that I learnt”. – Mindset participant

“Teaching is hard as it has changed from what it used to be”. – Mindset participant

The personal challenges were elevated by difficulties experienced with the Mindset equipment itself, and participants expressed dissatisfaction with the range of the Wi-Fi, training that they received and the manner in which Mindset deals with technical issues:

“I am far away from the device…it is not reachable on that side”. – Mindset participant

“I was trained on e-learning but nothing on the hardware”. – Mindset participant

“They [Mindset technicians] don’t even bother to come. If they do, they don’t know what they are fixing because the problem remains there…I don’t know whether they are hiring shady people but I don’t understand”. – Mindset participant

Page 24: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

24 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

In addition to these challenges, resources for both the learners, and themselves were described as an inhibitor to broader use of the programme in Katlehong schools:

“When you are in these schools, you have poor learners. Learners who cannot even afford to buy a packet of chips so at least if you get a learner to a computer lab, it’s a big step”. – Mindset participant

“You see not everyone can access it…only one or two have a tablet or smartphone”. – Mindset participant

“I have to download the material on my own laptop, and we have one projector in school. So if someone else uses it [the projector], I cannot use the material”. – Mindset participant

Educators had mixed views about the impact of Mindset on student attendance and performance. When questioned about these aspects, this is what they said:

“I think it [Mindset] has sparked their interest”. – Mindset participant

“It never works so how can we answer that”. – Mindset participant

“It’s too early to say…but the fact that we have it is positive to me”. – Mindset participant

Another challenge experienced by educators was inaccuracies in the Mindset material:

“The material is fine though I pick up some errors. I had to correct it immediately in class and show them the textbook for the correct material…For children who use it at home, they wouldn’t know there is a problem”. – Mindset participant

Despite these challenges, Mindset has provided educators in Katlehong with a useful resource that serves to supplement the theory which is taught to students:

“I teach and then I bring in the videos. I use them as emphasis. Most of them have mini experiments and demonstrations in between so it makes more sense to the children…especially if I don’t have chemicals”. – Mindset participant

“I like that you can see real situations…the visuals”. – Mindset participant

4.4.3.2 Perceptions of Liberty as a funder

When participants were asked about their perceptions of Liberty’s investment in Mindset, they relayed positive messages:

“That was a very good idea, a brilliant idea”. – Mindset participant

“I believe Liberty has started a journey that is meant to benefit every South African child”. – Mindset participant

4.4.4 Conclusions and recommendations

The following conclusions can be made regarding Mindset: • Mindset is a useful tool which is used to supplement classroom education for Maths and Science in Katlehong.• Educators who use Mindset value the importance of integrating technology into education.• Some of the material provided by Mindset contains certain inaccuracies. • Resources (in terms of laptops and tablets) are scarce in areas such as Katlehong, and hence a limited number of students are able to access the

material. • Technicians employed by Mindset do not adequately assist when there are problems with the devices. The following recommendations are made in order to strengthen Mindset, and ensure that the students who are not part of the Kutlwanong Centre for Maths, Science and Technology benefit:• More in-depth training should be conducted with educators so that they are able to intervene if there are problems with the Mindset modem. • Material provided by Mindset should be revised by content experts before it is broadcast, so that the material is accurate. • Investments in the Mindset devices should come with adequate technical support, irrespective of whether additional support is sponsored or if

the schools have to contact a call centre. This seems to be a major shortfall of Mindset currently and needs to be improved. A possible solution could be that the media centre teachers in schools be trained so that they are able to assist with technical support on-site.

Page 25: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

25An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

4.5 New Leaders Foundation

4.5.1 Programme description

The objective of the New Leaders Foundation is to implement systemic change in the education sector. Their work is based on the premise that in order to achieve quality results, quality relationships between the Department of Basic Education and schools need to be present. The programme runs in Gauteng, Limpopo and the Free State and aims to build relationships within the Departments of Basic Education. The Ekurhuleni South Department of Basic Education supports and oversees schools in Katlehong to ensure coverage of school curricula. The relationships that New Leaders Foundation would like to build are based on 20% compliance and 80% support. These concepts refer to the tasks that need to be carried out within the education system (compliance), and the support that goes into achieving these tasks. This is to counteract the methods of management used under apartheid, which were 100% compliance oriented. In order to achieve their goals, the New Leaders Foundation facilitated training and coaching through a New Leaders ‘immersed consultant’ (facilitator) based at the Ekurhuleni South Department of Basic Education. The immersed consultant was involved in formal training with course materials written by New Leaders Foundation where theories of support were imparted to district officials (employees at the Department of Basic Education). The district officials, who interact with principals of schools, are then meant to use their learnings from the programme in their work with principals, who are in turn to impart this knowledge to educators. One-on-one on-site coaching sessions took place where the immersed consultants provided feedback to district officials based on their interactions in meetings and other professional activities. The tasks carried out by the programme are conducted with the long-term vision to improve school management, quality of teaching, learner attendance and learner performance. New Leaders Foundation began in the Ekurhuleni South District Department of Basic Education in 2012 and has a total of five modules which were included in the programme.

4.5.2 Research methodology

The New Leaders Foundation was evaluated in 2014. To conduct the evaluation, in-depth interviews were conducted with one of the facilitators of the programme, district officials at the Department of Basic Education at the Ekurhuleni South District, and principals at schools in Katlehong. A total of 10 interviews were conducted. Selected questions on the perceptions of skills gained by participants were probed using scales, to visually represent differences in the participants’ ability before and after the inception of the New Leaders Foundation programme. Given the difficulties with understanding the goals, activities, assessment criteria, and intended outcomes of the programme, the scales were created in consultation with New Leaders Foundation.

4.5.3 Quantitative results

When participants were asked if they felt that the New Leaders Foundation had impacted on their skills in relation to problem solving, coaching skills and curriculum delivery amongst other activities, most participants indicated that they thought the programme helped expand their skills to a great extent (see Figure 3).

Figure 3: Perceptions of the extent to which New Leaders Foundation impacted on skills

1 Poor 2 Fair 3 Good 4 Very Good 5 Excellent0

1

2

3

4

5

Coach, support and problm solve curriculum delivery in schools

Before NLF After NLF

0 1 2 3 4 5 6 7

Curriculum delivery and management process

Coaching skills

How to apply change management knowledge andskills in curriculum delivery

Understand group dynamics

Opportunity to engage in reflective practice

Joint problem solving

Clearly articulated interactions

Skills

To a great extent To some extent Not at all

Page 26: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

26 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

In addition, when questioned about how New Leaders Foundation was influential in (1) coaching, support and problem solving in terms of curriculum delivery at schools; (2) creating communities of practice and professional learning groups to improve curriculum delivery; and (3) management and leverage of quality partnerships to improve curriculum delivery, the participants indicated that they felt better equipped after the New Leaders Foundation course (see Figures 4, 5 and 6 below).

Figure 4: Coaching, support and problem solving for curriculum delivery in schools

Figure 5: Creating communities of practice and professional learning groups to improve curriculum delivery

Figure 6: Management and leverage of quality partnerships to improve curriculum delivery

4.5.4 Qualitative findings

The qualitative findings take into account how participants understood the New Leaders Foundation, as well as commentary on the inception of the programme by facilitators and participants of the programme. In addition participants’ views of the investment that Liberty has made in New Leaders Foundation was explored.

4.5.4.1 Facilitator perceptions of New Leaders Foundation

Given that New Leaders Foundation had been running for two years at the time of the interviews, the facilitator felt that the programme was useful in the sense that it was able to build relationships and provide objective feedback to the Department of Basic Education:

1 Poor 2 Fair 3 Good 4 Very Good 5 Excellent0

1

2

3

4

5

Coach, support and problm solve curriculum delivery in schools

Before NLF After NLF

0 1 2 3 4 5 6 7

Curriculum delivery and management process

Coaching skills

How to apply change management knowledge andskills in curriculum delivery

Understand group dynamics

Opportunity to engage in reflective practice

Joint problem solving

Clearly articulated interactions

Skills

To a great extent To some extent Not at all

0

1

2

3

4

5

6

1 Poor 2 Fair 3 Good 4 Very Good 5 Excellent

Create communities of practice and professional learning groups to improve curriculum delivery

Before NLF After NLF

0

1

2

3

4

5

6

Manage and lever quality partnerships to improve curriculum delivery

Before NLF After NLF

1 Poor 2 Fair 3 Good 4 Very Good 5 Excellent

0

1

2

3

4

5

6

1 Poor 2 Fair 3 Good 4 Very Good 5 Excellent

Create communities of practice and professional learning groups to improve curriculum delivery

Before NLF After NLF

0

1

2

3

4

5

6

Manage and lever quality partnerships to improve curriculum delivery

Before NLF After NLF

1 Poor 2 Fair 3 Good 4 Very Good 5 Excellent

Page 27: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

27An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

“Our activities at the beginning of the journey was really to build relationships…be present in meetings and give feedback as the voice from the outside”. – New Leaders Foundation Facilitator

In addition, the facilitator believed that the programme was able to influence the Department of Basic Education to work better in teams:

“Looking at the goal of the individual…helping them to work towards that…getting them to start influencing in a much wider space, breaking down silos, being effective as well as…working on the team dynamics”. – New Leaders Foundation Facilitator

The facilitator felt that 50% of the district officials who participated in the programme were able to implement its teachings. In addition, the facilitator described the following categories, based on the ability of the participants to absorb the programme:

“There are the leapers…somebody who has absorbed. The tradition holders, who just don’t change; the bridge builders…who change over a period of time; and the gatekeepers…those who resist change”. – New Leaders Foundation Facilitator

Some of the reflections on how the programme could have been implemented differently were that (1) there was no internal owner of the programme; (2) that the timeframes between the activities and measured outcomes of the activities was too long; and (3) that the programme set long-term goals without considering some of the short-term goals that needed to be met first:

“The other thing is to have shorter timeframes…so we can jump in much quicker to push…for deliverables”. – New Leaders Foundation Facilitator

“So the short term goal is to become more supportive and that feeds into a bigger goal. It may look like failure if we say our goal is to improver learner results now…it doesn’t work like that in two years”. – New Leaders Foundation Facilitator

4.5.4.2 Participant perceptions of New Leaders Foundation

Overall, participants seemed to have a clear idea of what the New Leaders Foundation programme was intended to do:

“The [first] goal is to improve learner achievement. The second goal is to improve team work”. – New Leaders Foundation Participant

“To make us communicate better with the schools, to achieve the results we desire as an organisation for learner achievement”. – New Leaders Foundation Participant

“That we should be working collaboratively as a team”. – New Leaders Foundation Participant

Some of the positive observations around the programme were in relation to the fact that district officials felt more supported, and in turn, they are able to offer more support to colleagues and principals in schools. In addition, district officials were able to manage schools in a more compassionate manner:

“We support each other…listen to everybody”. – New Leaders Foundation Participant

“Basically the programme taught me to learn to listen to what principals have to say, and then formulate programmes together…so it formed a partnership”. – New Leaders Foundation Participant

“We are motivated…we have grown and work together…I gain more when we are in [the district officials] development and support”. – New Leaders Foundation Participant (Principal)

Despite providing positive feedback around the programme, the principals interviewed were not actually aware of the New Leaders Foundation. This was despite the fact that principals were invited to the Department to provide input on their interactions with district officials. The aim of the enquiry was to assess if the ideologies being taught by the programme were being implemented:

“We were not told why we were there…but we were told they were developing [the district official] and they want to know if the skills they were putting into [the district official] were functional to us and to our schools”. – New Leaders Foundation Participant (Principal)

Page 28: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

28 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Some of the challenges around New Leaders Foundation pertained to the time requirements to complete the programme, and the actual environment in which the programme was being implemented. In addition, concerns about the facilitator were expressed:

“What I dislike is the fact that to achieve this…it requires a great deal of time”. – New Leaders Foundation Participant

“I think that in a different context, the programme could be useful but in a context that is poisoned with political undertones, my observation is that none of the people who are attending the programme are implementing it”. – New Leaders Foundation Participant

“You still get many schools saying that they don’t get support”. – New Leaders Foundation Participant

“[The facilitator] has not been able to remain objective…Instead [the facilitator] would play people against each other, therefore preventing people from working together”. – New Leaders Foundation Participant

When participants were asked what they thought could be done differently by New Leaders Foundation, the following responses were recorded:

“Different personnel must run the programme”. – New Leaders Foundation Participant

“This project must also be put in schools…because we want to mould the nation”. – New Leaders Foundation Participant

4.5.4.3 Perceptions of Liberty as a funder

Participants expressed the following views about the programme and Liberty’s investment in it:

“People who could have been exposed to New Leaders are interacting with people who are not exposed. After a particular time the teachings will be lost”. – New Leaders Foundation Participant

“Everything being done focuses on improving classroom dynamics…Liberty is spending money well…realising the return on investment will only be seen much more long term”. – New Leaders Foundation Participant

“If Liberty continues to invest on the future of our children, the clearly there will be wonders and improvements”. – New Leaders Foundation Participant

4.5.5 Conclusions and recommendations

The following conclusions can be made regarding New Leaders Foundation: • It was extremely difficult for the independent assessors to grasp the goals, activities, assessment criteria and intended outcomes of the New

Leaders Foundation.• Participants of the New Leaders Foundation appeared to have a clear understanding of what the programme was intended to do in the

Ekurhuleni South Department of Basic Education.• Participants indicated that the programme contributed to improved problem solving, group work and coaching skills, to name a few. • Principals at the schools in Katlehong, however, were not aware of the New Leaders Foundation, even though they had been asked to rate their

managers for the programme. • The programme has been reported to have made a difference in terms of district officials feeling more supported and motivated in their jobs,

however, this sentiment was not shared by all the district officials.• The programme has indicated that it has long-term goals of increased learner performance, however, the short-term goals and assessment

criteria of the programme are not identified. • While the programme identifies that some of the district officials are ‘gatekeepers’, it does not identify a strategy to prevent the opinions of the

gatekeepers from impacting negatively on the programme outcomes. • While New Leaders Foundation identifies the political undertones within the Department of Education, there is no consideration for how to

overcome these. • While New Leaders Foundation would prefer shorter time-frames between activities of the programme, the participants indicated difficulties in

being a part of the programme given the time constraints and competing demands at the Department. • The implementation of the programme was reported to have been compromised due to the immersed consultant being unable to remain

objective. • Given that schools are still reporting that there is insufficient support from the Department of Basic Education, there seem to be difficulties in

filtering the programme into schools.

Page 29: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

29An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

The following recommendations are made in order to strengthen New Leaders Foundation:• New Leaders Foundation ideally should set clearly articulated, short-term goals and assessment criteria in order to monitor progress at

school level.• Principals need to be made aware of the programme so that they understand that the concepts they learn are related to the programme.• New Leaders Foundation should think through strategies of how to interact with participants who are sceptical of the programme. • Facilitators of the programme need to be able to remain objective whilst delivering the programme.• Greater efforts need to be made in order to ensure that the knowledge and skills acquired from the New Leaders Foundation are filtered down

into schools. • New Leaders Foundation could consider repackaging their material so that independent assessors are able to understand their goals, activities

and anticipated outcomes.

After evaluation of the New Leaders Foundation was discontinued, the programme was accepted as part of the education curriculum at the University of the Free State, with a total of six modules. There was also an interest from the National Department of Basic Education to run the programme in all the provinces, however more funding was required for this, so the implementation has been delayed. New Leaders Foundation is still running in areas in Katlehong in Gauteng, as well as in Limpopo and the Free State. Liberty ceased funding the programme in 2015 due to New Leaders’ inability to strategise on how to overcome the political undertones in the education system, and prevent differing opinions of the programme from affecting the intended outcomes of the programme. Despite these challenges, there may be merit to this programme in areas where it is still being carried out.

Page 30: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

30 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

4.6 Foundation for Integrated Entrepreneur Leadership Development (FIELD)

4.6.1 Programme description

The Foundation for Integrated Entrepreneur Leadership Development believes that it can play a signifi cant role in ensuring that small- and micro-business owners develop their skills to respond to South Africa’s rapidly changing business environment. The FIELD entrepreneurship development programme aims to aff ord youth the opportunity to gain knowledge about business skills and analytical techniques, think critically and challenge assumptions, and be exposed to new experiences which can enhance their entrepreneurial journeys. FIELD created and delivered its own learning materials on entrepreneurship via facilitators in Katlehong. In 2013, FIELD was present in Gauteng. It also ran a bookkeeping course in 2013, which was eliminated from the programme in 2014 due to a lack of interest.

4.6.2 Research methodology

FIELD was evaluated from 2013 to 2014. While the programme was originally envisaged to target youth, and make use of psychometric testing for selection of participants, this was not carried out in practice. This was due to diffi culties with roll-out of the project, as a result of delays in funding transfers from Liberty. The results of the psychometric tests were not used in the selection process, and some participants were outside of the offi cial defi nition of youth (i.e. 18-24 years old).

In 2013, the programme inception year, individual interviews were conducted with 10 participants on the entrepreneur development programme, two participants on the bookkeeping learnership as well as two facilitators of the programme. Participants were also asked to complete scales relating to their perceived chances of success, and their social and technical skills. The 2013 FIELD graduates were interviewed again in 2014, in order for us to track progression since the end of 2013. The scales were re-administered, in order to measure a change in perceptions of skills since the beginning of the programme.

4.6.3 Quantitative results

Figure 7: Technical skills of FIELD participants at baseline (2013)

In terms of self-reported technical skills, the majority of participants were of the view that their planning, self-organisation and collaboration skills were good in 2013. However, software skills still required further training.

0 1 2 3 4 5 6 7 8 9

Self organisational skills

Team work/Collaboration skills

Presentation skills

Communication skills

Planning

Human Resource/Labour law

Drafting budgets and business plans

Software skills

Computer skills

Technical Skills of Participants 2013

Excellent Good Average Poor Very poor

0 1 2 3 4 5 6 7 8

Technical Skills of Participants 2014

Excellent Good Average Poor Very poor

Self organisational skills

Team work/Collaboration skills

Presentation skills

Communication skills

Planning

Human Resource/Labour law

Drafting budgets and business plans

Software skills

Computer skills

Page 31: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

31An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Figure 8: Technical skills of FIELD participants post training (2014)

Participants reported that they benefitted from FIELD in terms of collaboration, planning, self-organisation and communication skills, and many more participants indicated that the skills mentioned in Figures 7 and 8 were excellent after the programme. However, software skills, computer skills, budgeting and business planning skills still required further training. This finding was confirmed by the results of the entrepreneur efficacy scale, which was administered in 2014 (post training), where some participants still felt ill-equipped in terms of fund raising, executing a business plan, and managing budgets.

Figure 9: Entrepreneur Efficacy scale post training (2014)

0 1 2 3 4 5 6 7 8 9

Self organisational skills

Team work/Collaboration skills

Presentation skills

Communication skills

Planning

Human Resource/Labour law

Drafting budgets and business plans

Software skills

Computer skills

Technical Skills of Participants 2013

Excellent Good Average Poor Very poor

0 1 2 3 4 5 6 7 8

Technical Skills of Participants 2014

Excellent Good Average Poor Very poor

Self organisational skills

Team work/Collaboration skills

Presentation skills

Communication skills

Planning

Human Resource/Labour law

Drafting budgets and business plans

Software skills

Computer skills

0 1 2 3 4 5 6 7 8

Create a timeline for starting a business

Sell a product or service

Create a market plan for a service or product

Raise money to fund a business

Manage a budget for a business

Create an innovative product or service

Create a plan for executing a business

Entrepreneur Efficacy Scale

Complete confidence Much confidence Moderate confidence

Very little confidence No confidence

Page 32: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

32 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

The changes in social skills as a results of FIELD training were much more apparent, as seen in Figures 10 and 11 below, where the majority of participants felt that their skills on the indicators analysed were either good or excellent after the FIELD training.

Figure 10: Social skills of FIELD participants at baseline (2013)

Figure 11: Social skills of FIELD participants post training (2014)

Finally, when perceived chances of success were analysed, participants were more likely to believe that they had a good chance of success after the programme (see Figure 12 below). Some participants noted that they would only be able to truly gauge the success of their budgeting and business plans once they had received their start-up funds. Even though participants were clearly disappointed at not being able to start their businesses already, several were still going ahead with their business plans and others remained positive and hopeful that they would still be able to bring growth and prosperity to their families and the community of Katlehong.

0 2 4 6 8 10 12

Social Skills of Participants 2013

Excellent Good Average Poor Very poor

0 1 2 3 4 5 6 7 8 9

Social Skills of Participants 2014

Leadership

Risk taking

Discipline

Decision making

Understanding others

Positive self-esteem

Confidence

Listening

Leadership

Risk taking

Discipline

Decision making

Understanding others

Positive self-esteem

Confidence

Listening

Excellent Good Average Poor Very poor

0 2 4 6 8 10 12

Social Skills of Participants 2013

Excellent Good Average Poor Very poor

0 1 2 3 4 5 6 7 8 9

Social Skills of Participants 2014

Leadership

Risk taking

Discipline

Decision making

Understanding others

Positive self-esteem

Confidence

Listening

Leadership

Risk taking

Discipline

Decision making

Understanding others

Positive self-esteem

Confidence

Listening

Excellent Good Average Poor Very poor

Page 33: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

33An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Figure 12: Perceived chances of success over time

4.6.4 Qualitative findings

The qualitative findings take into account the facilitator and participant perceptions on the inception of the FIELD programme, as well as their perceptions of Liberty as a funder.

4.6.4.1 Facilitator perceptions of FIELD

In 2013, the facilitators of FIELD indicated that they enjoyed the lecturing experience, had grown as facilitators and felt a sense of achievement when students were able to implement the skills they were taught:

“My understanding was improved, and my confidence, my understanding was sharpened, I had to exude a lot of confidence, in terms of business and when you are nervous you make the learners nervous”. - FIELD Facilitator

“[It is] very encouraging from where we are sitting, it just says to us that the message is getting through and is being applied”. - FIELD Facilitator

Although there were issues with curriculum standardisation initially, they were overcome successfully as the course progressed:

“The things that I don’t like about the modules is that in the beginning phase there’s too many changes”. - FIELD Facilitator

Very Poor Poor Average Good Excellent

Perceived chance of success 2014

Perceived chance of success 2013

Very Poor Poor Average Good Excellent

9

8

7

6

5

4

3

2

1

0

9

8

7

6

5

4

3

2

1

0 Very Poor Poor Average Good Excellent

Perceived chance of success 2014

Perceived chance of success 2013

Very Poor Poor Average Good Excellent

9

8

7

6

5

4

3

2

1

0

9

8

7

6

5

4

3

2

1

0

Page 34: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

34 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Furthermore, one of the FIELD facilitators indicated that there could be better management of resources provided for the classes, specifically with regard to stationery supply and catering:

“Maybe let me be specific, for us we knew that students would need stationery and we bought that… but once on the ground we suddenly realized that they don’t need one exam pad but two and that is more money because [we also] said we will offer breakfast and lunch… You find that you do have the resources and you run with it for a certain while and they do run out and suddenly you need to come with a different plan for it”. - FIELD Facilitator

Due to the varied age range of participants (18-50 years), facilitators also indicated that navigating group dynamics was difficult at times:

“[The older participant] feels uncomfortable and he knows that this is an age matter that makes it more difficult with this particular subject and with his general knowledge he should know this, and he still comes forward to the group and say, you know guys with this one I have a problem and that kind of openness helps us to help him”. - FIELD Facilitator

On a personal level, the facilitators revealed that being a part of FIELD has been empowering:

“On a personal level, I have grown so much as a person…I have taken that with me into my family life and personal life”. – FIELD Facilitator

4.6.4.2 Participant perceptions of FIELD

The two participants selected for the bookkeeping learnership in 2013 were not clear on the nature and content of the learnership before they joined. Neither of the two Katlehong students intended pursuing careers in bookkeeping. They had chosen to pursue other degrees at the Universities where they were accepted in 2014. Nonetheless, the FIELD programme was beneficial as a source of encouragement to the learners:

“Since I was attending this course, we have been to many seminars, that’s where I was exposed to…you know you have to aim high in life, don’t aim low. I also learnt about the importance of my surrounding…I don’t hang out with people who limit me anymore”. – Bookkeeping Participant

For the participants on the entrepreneur development programme, 2013 interviews revealed that participants were extremely complimentary about the FIELD programme, expressing the view that they had learnt many skills. In terms of the content of the programme, participants felt that the modules were extremely useful:

“I know a lot of things in business, [FIELD] has taught me economics, labour law, relations, business planning, marketing, advertising and finance”. – FIELD Participant

“Being in FIELD has taught me about business, how to handle business and what to expect in a business”. – FIELD Participant

One of the challenges, as indicated by participants, was that they would benefit from visual aids:

“There must be something just to upgrade the programme, I think they should have some TV or projector, so that we can see things practically”. – FIELD Participant

Additional challenges experienced by the participants were the lack of resources in order to conduct research required for the course. The fact that FIELD did not have equipment such as computers available, was considered a constraining factor to the success of the programme:

“I don’t even have a PC, so I’m responsible for my own research, and going to any internet café”. – FIELD Participant

“So far they’re doing good. The only obstacle is they don’t have enough equipment, computers”. – FIELD Participant

Page 35: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

35An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

Participant perceptions of the facilitators of FIELD were also investigated, as this is crucial to the success of the programme. This is what the participants said about the facilitators:

“They are willing to help…they don’t get annoyed if you ask the same questions, they make sure you understand”. – FIELD Participant

“They are humble to us. They are not harsh…They are good at teaching the material”. – FIELD Participant

At the time of the follow-up interviews in 2014, the majority of participants were either waiting for funding to kick-start their business ventures or were already running their businesses, and waiting for further funding to enhance their businesses. Participants were still extremely complimentary of the modules offered by FIELD. Many believed that the course had affected them deeply by inspiring them to pursue entrepreneurship.

“FIELD has helped us a lot, especially when it comes to things like Labour Law, economics…motivation”. – FIELD Participant

“I can’t complain about the modules because until today, I still go back to my files. It helps me every time I encounter a problem. Practical and theory go hand in hand”. – FIELD Participant

A major challenge experienced in 2014 was in relation to funding for businesses. Participants were offered assistance by FIELD, to submit proposals to funders. Due to FIELD being the middle-man, and funders’ delay in responses to the participants, many were upset:

“It’s better for courses not to say anything about funding. Just to be given knowledge. If you promised and don’t deliver, it’s a case on its own”. – FIELD Participant

“The biggest difficulty is accessing finance”. – FIELD Participant

“I understand there is a procedure to gain finance…but sometimes what makes us have a problem is when we are not communicated as to what that procedure is”. – FIELD Participant

In contrast, for participants who took a different approach, and did not depend on FIELD entirely to access finance, funding did not seem to be an issue:

“On my side, nothing has affected me much as I wasn’t only depending on [FIELD] as I managed to get another source”. – FIELD Participant

Other challenges experienced by the participants in 2014, was the fact that they did not receive the results of their psychometric testing, or their certificates on completion of the programme. Some participants reported receiving certificates that indicated participation, and did not state whether the course was accredited by the South African Services Sector Education and Training Authority (SSETA). After the interviews, FIELD shared results of the psychometric testing with participants, and they received their certificates which had been delayed due to corrections being made (in relation to misspelled names).

4.6.4.3 Perceptions of Liberty as a funder

Participants and facilitators were asked to express opinions about the fact that Liberty has invested in FIELD. This is what the participants said:

“I would thank them because if it wasn’t for them, I would be nothing because they are teaching us the basics of business and entrepreneurship. I would say ‘big up’ to Liberty”. - FIELD Participant

“Thank you! It is powerful what they did, propelling an entrepreneurial spirit. That is how you grow the economy. It was a good investment”. - FIELD Participant

“I want to ask a favor from Liberty, if there is networking...they should let us know…Can we possibly negotiate a smaller loans with Liberty”? - FIELD Participant

Facilitators of FIELD shared the following views when they were asked how they felt about Liberty’s sponsorship of the programme:

“Thank you, not only has the course changed the participants, but it has changed me as a facilitator”. – FIELD Facilitator

“I’m very grateful…and I would like to see them take a bit more ownership”. – FIELD Facilitator

Page 36: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

36 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

4.6.6 Conclusions and recommendations

The following conclusions can be made regarding FIELD: • The programme adequately transfers knowledge to learners regarding many aspects of business, including economics, labour relations,

marketing and advertising, to name a few. • In an area as poorly resourced as Katlehong, additional support such as visual equipment and computers are needed.• FIELD has resulted in reported improvements in technical skills and social skills for both participants and facilitators of the programme. • Two of FIELD’s major shortcomings were that the selection of participants was not based on the psychometric test results, and the age group

that the programme was intended for was not the age group that it was delivered to. This meant that those who attended classes did not necessarily have competencies at the required level for successful entrepreneurship, and this in combination with age, made class dynamics difficult in some instances.

• FIELD’s assurance to participants that they would receive funding for their business ideas was another shortcoming. Participants felt dissatisfied with FIELD when funders delayed responding to the business proposals.

The following recommendations are made in order to strengthen FIELD:• Investments should be made in terms of visual equipment and computers where the programme is run in resource poor areas. • Participant selection needs to be stringent, in order to select participants who have competencies at the required level that are needed for

successful entrepreneurship.• FIELD needs to clarify its role, and the purpose of the programme to participants upfront. In this way participants will not have false

expectations of the programme, and be disappointed when certain expectations are not met. • FIELD is encouraged to confirm accreditation of the programme with SSETA.• Participants noted that for Liberty to invest in a programme such as FIELD, they should want to see the beneficiaries succeed. This requires

further investment from Liberty, and a greater visibility in the community.

After the evaluation of FIELD was discontinued, the programme closed down for a year to undergo reconstruction. Liberty ceased funding the programme in 2015 due to a decision to invest more within the schooling system itself.

FIELD aims to make better use of the online platform, with the goal to incubate participants’ business plans rather than to develop them. Participants will be required to participate in online tutorials, and can quit the programme at any stage. FIELD is also working to make better use of psychometric testing for selection of participants.

Page 37: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

37An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

5. Liberty’s investment in Katlehong Liberty’s contribution and strategy to enhance educational outcomes in Katlehong is commendable. Through the various programmes they fund, they have improved the knowledge and skills of learners and educators from the foundation phase (Grade R – Grade 3) through to Grade 10 – Grade 12, with some positive effects being carried through to further education and training. Different intervention strategies were used in the programme and some conclusions and recommendations may be drawn. These are discussed below.

The effects of workbooks and materials on student performance in the Foundation Phase

The Future Thinking Foundation has successfully contributed to an increase in learner performance for Foundation Phase Maths and isiZulu in Katlehong. In 2013 a significant increase in Maths and isiZulu marks of 10.2% and 7.1%, respectively were recorded. In 2014, the learner marks increased by 17.9% and 9% for Maths and isiZulu, respectively. In addition, due to some of the material being written in English, learners found that their English had also improved. The material was also perceived to have promoted enthusiasm for learners and educators. Adequate training offered by the Future Thinking Foundation on the use of the material was crucial to the success of the material. Yet, incorporating material from Future Thinking Foundation with the material from the Department of Basic Education proved challenging.

The effects of extra-curricular Maths and Science classes on student performance

Academic performance in Maths and Science for high schools learners in Katlehong had significantly increased through the Kutlwanong programme. Kutlwanong has made a significant impact on Maths and Science marks for learners in Grades 11 and 12. In 2013, there was a marked improvement of 29.3% for Mathematics and 27.6% for Science. In 2014, again, increases of 23.1% for Mathematics and 20.9% for Science were recorded. In addition, soft skills such as confidence, discipline and behavioural changes were positively impacted as a result of high school learners being a part of the programme. Educators on the programme felt that they could be better supported with laboratory equipment to ensure that they were able to provide better tuition. Given that resources of this kind are limited, the evaluation suggests greater synergies between Mindset and Kutlwanong to counteract the effects of limited resources.

Educator development programmes

ORT SA has made positive changes to educator skills and knowledge in Katlehong. As a result of the trainings, educators felt more prepared and confident in classes. The lack of teaching material, however, seemed to be perceived as a threat to the effectiveness of the programme. In addition, more frequent trainings, guidance on how to manage large numbers of students, and improved communication between the programme and schools were suggested in order to strengthen the programme.

Access to online lessons as a classroom addition

Mindset modems were recognised as a useful tool to supplement teaching in the classroom. In addition, educators acknowledged the importance of integrating technology into education. Concerns with this programme were the inaccuracies in teaching material, limited resources (such as laptops) which inhibit the use of the materials, and inadequate technical support to ensure that the modems are always optimally functional.

Page 38: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

38 An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

School managerial support and the influence on educator performance

The New Leaders Foundation appeared to have contributed to officials at the Ekurhuleni South Department of Basic Education, as well as school principals, feeling more supported in their jobs. Some employees indicated improved problem solving skills as a result of the programme. The shortcomings of this programme included the inability to identify strategies to ensure that all participants understood the value of the programme. In addition, facilitator objectiveness was viewed as central to the success of the programme, and in this case, objectiveness seemed to be compromised. The evaluation suggests that there is room for programme improvement, particularly in regard to clear goals, activities and objectives of the programme. Greater efforts also need to be made in order to ensure that the programme is able to filter into schools in Katlehong.

Entrepreneurial development

Overall, FIELD was successful in transferring entrepreneurial knowledge to participants of the programme. Despite this, resource constraints for equipment such as computers were identified. Selection criteria for the programme were deficient in 2013, which resulted in some participants lacking competencies that would guarantee successful entrepreneurship. The evaluation therefore recommended more stringent selection criteria, as well as accreditation of the programme with SSETA.

Overall, Liberty’s investment in Katlehong provides a hub of additional training and support for learners and staff in the education system. In order to ensure that the investment results in longer term, scalable impact, the evaluation suggests that greater synergies be constructed. This is elaborated on below.

In terms of scalability, Future Thinking Foundation and Kutlwanong have design features that could be scaled up for higher impacts. These programmes provide high quality curriculum and extra-curricular inputs. These programmes also have clearly defined assessment criteria and outputs which have positive impacts on educational outcomes. Given that we had large numbers to assess for these programmes, we would expect to see the same positive results if these programmes were expanded to other areas with the same living conditions as Katlehong. For the remainder of the programmes, better co-ordination and synergies could improve outcomes. These are commented on next.

First, synergies could be found between the programmes themselves. For instance, in the foundation phase, we suggest that ORT SA collaborate with Future Thinking Foundation to address the challenge around supplementary material for educators. In the senior phase and further education and training, we make the suggestions that Kutlwanong’s need for laboratory equipment should be complemented by the online experiments provided by Mindset. While we acknowledge that Mindset is a national programme, it could make use of teachers from Kutlwanong as content experts to review lessons before they are placed online. Similarly, ORT SA could draw on Kutlwanong’s model which is to source locally based educators to run the programme in Katlehong. This sort of strategy could alleviate some of the constraints experienced by the programmes.

Second, and in light of the repeated reference to time constraints throughout the evaluation, we suggest that greater effort should be made for co-ordination between the programmes, Department of Basic Education and the schools. Improved planning for activities relating to all three bodies could lessen the burden of competing demands for educators when they are provided with opportunities to attend training. More effective collaborations, as well as improved co-ordination of the programmes in Katlehong would increase the impact of the programmes within the education system, and result in more positive results for learners attending schools in Katlehong.

Finally, we suggest that greater synergies be explored between Liberty and other bodies investing in education in South Africa, including other Corporate Social Investment divisions. We believe that Liberty’s strategy provides valuable insights into what type of programmes are working well, and how they are making positive changes in the education system. In addition, the strategy provides insights into what kind of programmes require alteration. Overall this could ensure that investments in the education sector are well targeted, and have a greater long-term positive impact on educational outcomes in South Africa.

Page 39: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

39An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa

6. References

Amandla Development (no date). NGO’s, Education, and Opportunity in South Africa: An Analysis of Macro and Industry Trends and Capacity Need.

Bester, G. and Brand, L. (2013). The effect of technology on learner attention and achievement in the classroom. South African Journal of Education, 33(2), 1-15.

Botha, M., Nieman, G. and van Vuuren, J. (2006). Enhancing female entrepreneurship by enabling access to skills. Entrepreneurship Management, 2, 479-493.

Braun, V., and Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77-101.

Creswell, J. W., and Plano Clark, V. L. (2011). Designing and conducting mixed methods research. Thousand Oaks, CA: SAGE.

Delport, C.S.L., and Fouche, C.B. (2011). Mixed methods research. In A.S. de Vos, H. Strydom, C.B. Fouche and C.S.L. Delport (Eds.), Research at grassroots for the social sciences and human service professions (4th ed., pp. 433-448). Pretoria: Van Schaik Publishers.

Dieltiens, V. and Motala, S. (2012). The quintile ranking system in South African education. Presentation to workshop on Transparency in the Targeting and Management of Pro-Poor Educational Incentives, Brasilia, 5–6 November.

Fleisch B, Taylor N, Herholdt R, and Sapire I. (2011). Evaluation of Back to Basics mathematics workbooks: a randomised control trial of the Primary Mathematics Research Project. South African Journal of Education, 31(4),488-504.

Gibberd, J. (2007). South Africa’s school infrastructure performance indicator system. OECD Publishing.

Glewwe, P., Kremer, M. and Moulin, S. (2007). Many children left behind? Textbooks and test scores in Kenya. National Bureau of Economic Research Working Paper 13300.

Isaacs, E., Visser, K., Friedrich, C. and Brijlal, P. (2007). Entrepreneurship education and training at the Further Education and Training (FET) level in South Africa. South African Journal of Education, 27(4), 613–629.

Joseph, V. K. (2004). Management strategies to improve the academic performance of previously disadvantaged secondary schools in the Grade 12 Examinations. Unpublished Thesis, Rand Afrikaans University.

Kranzler, J. (2003). Statistics for the Terrified. Englefield Cliffs: Prentice Hall.

Makgato, M and Mji, A. (2006). Factors associated with high school learners’ poor performance: a spotlight on mathematics and physical science. South African Journal of Education, 26(2),177-319.

Mestry, R., Bischoff, T., and Hendricks, I. (2009). Perceptions of teachers on the benefits of teacher development programmes in one province of South Africa. South African Journal of Education, 29, 475–490.

Millham, R., and Thakur, S. (2014). Evaluating the effectiveness of eLearning and core skills interventions for South African secondary school learners. Proceedings of the e-Skills for Knowledge Production and Innovation Conference 2014. Cape Town, South Africa, 211-215.

Mogari D, Coetzee H, and Maritz R. (2009). Investigating the Status of Supplementary Tuition in the Teaching and Learning of Mathematics. Pythagoras, 69, 36-45.

National Planning Commission (2013). National Development Plan. Pretoria: National Planning Commission.

Ocampo, M. L. (2004). A Brief History of Educational Inequality from Apartheid to Present. http://web.stanford.edu/~jbaugh/saw/Lizet_Education_Inequity.html

Republic of South Africa. Department of Performance Management and Evaluation (DPME) (2014). 20 Year Review South Africa. Pretoria: Government Printers.

Republic of South Africa. Department of Basic Education (DBE) (2013). School Realities. Pretoria: Government Printers.

South African Institute of Race Relations (SAIRR) (2015). Fast facts, No 2/2015, Issue 282. Johannesburg: SAIRR.

Shepherd, D.L. (2011). Constraints to school effectiveness: What prevents poor schools from delivering results? Stellenbosch Economics Working Paper

Spaull, N. (2014). Accountability in South African Education. Ch4 in Transformation Audit 2013: Confronting Exclusion, 51-65, Institute for Justice and Reconciliation. Cape Town.

Spaull, N. (2013). South Africa’s Education Crisis: the Quality of Education in South Africa. Centre for Development and Enterprise.

Statistics South Africa (2014). General Household Survey: Statistical Release P0318. Pretoria: Statistics South Africa.

Statistics South Africa (2011). Census 2011. Pretoria: Statistics South Africa.

Strydom, H. (2011). Sampling in the quantitative paradigm. In A.S. de Vos, H. Strydom, C.B. Fouche and C.S.L. Delport (Eds.), Research at grassroots for the social sciences and human service professions (eth ed., pp. 222-235). Pretoria: Van Schaik Publishers.

Taylor, S. (2011). The vital role of good school management in improving primary school outcomes. Policy Brief. Stellenbosch: Stellenbosch University.

Van Broekhuizen, H. (2013). Transitions from school to work and tertiary education. In Spaull, N. South Africa’s Education Crisis: the Quality of Education in South Africa. Centre for Development and Enterprise.

World Bank (2004). Monitoring and Evaluation: Some Tools, Methods and Approaches. Washington, D.C.: World Bank.

Page 40: AN EVALUATION OF LIBERTY FUNDED EDUCATION AND SKILLS ... · An Evaluation of Liberty Funded Education and Skills Development Programmes in Katlehong, South Africa 5 • For ORT SA,

LibertyLiberty Group LimitedRegistration number 57/02788/081 Ameshoff Street, BraamfonteinPO Box 10499, Johannesburg 2000

DisclaimerThe information contained in this document does not constitute advice by Liberty. Any legal, technical, or product information contained in this document is subject to change from time to time. This document is a summary of the features of the product as at the time of publication. If there are any discrepancies betweenthis document and the contractual terms and conditions or, where applicable any fund rules, the latter will prevail.

Any recommendations made must take into consideration your specic needs and unique circumstances.

Liberty Group Ltd is an Authorised Financial Services Provider in terms of the FAIS Act (no 2409).

©Liberty Group Ltd 2016. All rights reserved.