Evaluating Education for Sustainability - A reflection after 10 years AES 2011, Sydney Larraine J Larri

Evaluating Education for Sustainability -

A reflection after 10 years

AES 2011, SydneyLarraine J Larri

1. What is education for sustainability and how did I get involved in evaluating it?

2. From Environmental Education to Learnscapes

3. From Learnscapes to AuSSI – 10 years on from Pilot to Mature program

4. Reforms in pedagogy and environmental education – understanding the challenges and implications for the evaluator

Productive pedagogies and EfS

Systemic adoption of EfS

Implications for evaluation

5. The contribution of evaluation to EfS

6. The contribution of EfS to evaluation

7. Overview

1. What is education for sustainability and how did I get involved in evaluating it?

education for sustainability (EfS).

EfS is a big idea that tries to rethink the way we learn, live and work together while respecting the capacity of the Earth’s natural systems to support life. Josephine Lang (2007)

Why do people care for the environment?Altruism? Commitment to collective action ? Can we care for the environment and at the same time be materialistic and individualistic?Which societal values are the drivers for environmental sustainability? How can economic development be sustainable and profitable?

© L Larri, Renshaw-Hitchen & Assocs P/LPhoto: Simon O'Dwyer 25 Jan 2007 The Age Mighty Murray

Learnscapes Critical Success Factors

2000

2003/05AuSSI Pilots NSW & VicPlus ‘Comparative Assessment’

2006Waste Wise WA & future link with AuSSI WA

2008AuSSI-Tas Pilot ModelSchool & Systems levels

2008Global Communities for SustainabilityPilot - Impact on student learning

2007, 2008AuSSI facilitationNational Partnership Workplan - M&E framework

2009/10CarbonKids PilotThe Science of Climate Change and link to AuSSI

Global Communities for SustainabilityPhase 2 - Expansion

AuSSI-ACTAfter 4 years

AuSSI-TasAfter 2.5 years

Scenarios for the future of Australia

1999

2. From Environmental Education to Learnscapes


The Australian Sustainable Schools Initiative (AuSSI) is a partnership of the Australian Government, and the States and Territories that seeks to support schools and their communities to become sustainable.

The AuSSI involves participants in a whole-of-school approach, to explore through real-life learning experiences, improvements in a school’s management of resources and facilities including energy, waste, water, biodiversity, landscape design, products and materials. It also addresses associated social and financial issues. The Initiative’s vision is for all Australian schools and their communities to be sustainable

a sophisticated and holistic program that sees the school as a model system for the

broader community to achieve environmental sustainability


program logic matrix (PL) was used to focus and plan the evaluations.

a participatory process involving a range of stakeholders to develop the outcomes hierarchy, and contribute to fleshing out the other details in the matrix.

Capacity building in evaluation


4.1 Productive pedagogies and EfS

how reforms in pedagogy and the adoption of EfS have been linked

Gore and colleagues (2001) developed what they called, the ‘Productive Pedagogy Framework’ to guide teachers in the construction of teaching and learning programs

Intellectual qualityHigher-order thinkingDeep knowledgeDeep understandingSubstantive conversationKnowledge as problematicMetalanguage

Relevance (connectedness)Connectedness to the worldProblem-based curriculumKnowledge integrationBackground knowledge

Social supportStudent controlStudent supportEngagementSelf-regulationExplicit criteria

Recognition of differenceCultural knowledgeInclusivityNarrativeGroup identityCitizenship

Authentic Pedagogy - USAProductive Pedagogies - QueenslandEssential Learnings – SA, Vic, TasQuality Teaching & Learning - NSW


4.2 Systemic adoption of EfS

Ongoing tension between achieving environmental management targets as well as integrating pedagogical outcomes

educationists core business is to achieve student outcomes and environmentalists whose concern is achieving behaviour change towards environmental sustainability.

convincing schools and their communities as well as stakeholders that environmental education has a legitimate place in the school curriculum, equally that it should be a

school’s objective to become a role-model for sustainable practices in the community.

United Nations Decade of Education for Sustainable Development (2005-2014, DESD, UNESCO).

1999 Adelaide Declaration on National Goals for Schools in the Twenty-First Century - included Goal 17

Melbourne Declaration on Educational Goals for Young Australians (December 2008)

April 2009, National Action Plan for Education for Sustainability, Living Sustainably. The plan aims to, equip all Australians with the knowledge and skills required to live sustainably. It is Australia’s contribution to the DESD

inclusion of ecological sustainability in the definition of ‘contemporary science’; and ‘sustainability’ as a cross-curriculum perspective in the new national Australian science

curriculum K–12


4.3 Implications for the evaluator

Understanding the context ... appreciation of :

pedagogical approaches involved in EfS and challenges for teachers in planning how to integrate it into the school’s curriculum.

the challenge to roles of school leaders who are primarily educationists and have an expanded role of managing resources and the school environment.

adoption of a whole school approach – systems thinking.

5. The contribution of evaluation to EfS Program logicParticipatory evaluation

7. The local community manages it environment in ecologically sustainable ways

6. The school environment is effectively managed for teaching and learning in ways that are ecologically sustainable

5. The school and local community implement their Learnscapes Project Management Plan

4. The school Learnscapes Committee devises their Project Management Plan

3. The school establishes a Learnscapes Committee to manage environmental improvements through a curriculum response

1a. A school committee is established to manage the school’s external

environment

1b. The school is aware of environmental education

2. The school is aware of Learnscapes as a way of combining environmental improvements with teaching and learning

Learnscapes hierarchy of intended outcomes




participatory evaluation

evaluation approaches that fit with the aspirations of EfS programs.

Social connectedness is at the heart of sustainability.

Embedded in the AuSSI model are structures dependent on whole school participation that values multiple perspectives in decision-making, in particular student involvement and strong links with community partners.

© L Larri, Renshaw-Hitchen & Assocs P/L

1

© L Larri, Renshaw-Hitchen & Assocs P/L

2008AuSSI-Tas Pilot ModelSchool & Systems levels

CarbonKids, Global Communities for Sustainability –

Some benefits using Social Software in evaluation

The nature of EfS programs: • Social connectedness is at the heart of

sustainability. • The technologies being used have the dual ability

to gather evidence and enhance networking across Australia and support cross-cultural, intergenerational, collaborative dialogue.

• Using online technologies is arguably more environmentally responsible than travelling across Australia to conduct case studies and interviews.

Data quality:

Quantitative and qualitative survey data: qualitative data is provided directly by the participants. It functions both as data and as online content.

Engaging school systems in the evaluation:

Technology literacy - Information and communication technology (ICT) skills of participants are increased and this is attractive to schools looking for appropriate real life opportunities for students and teachers. Involving the participants in the evaluation process:

Participant use of aggregated data of baseline surveys, also ‘on completion’ data fed back to individual schools and for all schools. Space to seek feedback and critical reflection comment about the evaluation. Evaluator able to post notices and comments.

Value for money:

With limited budgets, the technology provides excellent affordability, reliability and ease of use. Reduces travel costs and has carbon offset benefits


Rich opportunities for topics that stretch evaluation & research

the development of students to become environmental citizens, take action and be leaders

the nature and dynamics of school leadership in enabling change;

determining the degree to which behaviours and attitudes have changed in relation to resource management and usage (i.e. water, waste, energy) and care for the environment (i.e. biodiversity)

researching the ways in which student well-being has been affected by involvement in activities related to EfS


2: The nature and dynamics of school leadership in enabling changeData collection instruments and some findings from the AuSSI-ACT evaluation

Question in Teacher Survey (AuSSI-ACT)How important has the School Leadership Team been in contributing to ‘successful’ behavioural change towards and adopting EfS?Items for rating:have been a driving force in motivating staff to adopt of EfShave supported and encouraged the EfS / Environmental Leadership Teamhave been involved in the development of the school's vision for EfShave promoted the vision of sustainability that the school developed e.g.they talk about it often, they are role modelstake action to remove barriers, and empower people to change, they are 'enablers'they generate enthusiasm about achievements and successesactively help the EfS leadership to build on achievements and keep goingthey promote ways to embed EfS into school systems and processes e.g. purchasing, curriculum plan etcWhat AuSSI-ACT Principals considered as Key Leadership Ingredients Excerpt of from focus group Lead by example, be a role model, show that the program is valued by leadership and that it is a priority; communicate on many levels why the school is doing the Program, how it is progressing, and its achievements.Aim for student ownership and efficacy - involve students in ways that increase their agency.AuSSI-ACT Teachers comments from surveys and focus groupPrincipal support is imperative – the inspiration from the top is critical: - supportive in time/finances – total belief in the vision of being sustainable.


Box 3: Items used in teacher survey to gain an impression of shifts in student behaviours

Water: From using water indiscriminately and behaving as though it was abundant, to using

water sparingly and valuing it as a precious resource. Waste:

From creating waste without knowing or caring what happens to it to seeing it as able to be avoided, reduced, or a resource for reuse or recycling.Energy:

From using energy indiscriminately to using renewable and non-renewable energy in ways that reduce energy consumption, lessen environmental impact and extend existing energy resources.Biodiversity:

From not thinking about what impact we have on the environment to considering our actions and the effects they may have on biodiversity.Global environmental citizenship:

From being concerned only about oneself to being concerned for local, national and global communities.Environmental sustainability leadership:

From not being able to express concern about sustainability to recognising and taking responsibility for the changes required to develop a more sustainable society.


:Dimensions of well-being used in AuSSI-ACT and AuSSI-Tas evaluations(based on Fraillon, J, 2004, ACER).

The scenario of establishing and maintaining a school worm farm was used to assist teacher responses. Whilst this was not necessarily a good fit in all examples, it was hoped that it would act as a pointer towards a general direction for each

dimension of well-being.Intrapersonal: a student's sense of self and their capacity to function in the school community.

Interpersonal: a students’ appraisal of their social circumstances and their capacity to function in the school community.

Autonomy : showing independence, actions and/or values are fully endorsed

Emotional regulation: the degree to which the student's emotional responses are appropriate to the events surrounding

Resilience: the capacity to manage, recover and move on from critically challenging events

Self-efficacy: the degree to which the student believes they are effective

Self-esteem: the way students feel about themselves Spirituality: a positive sense of meaning and purpose in life Curiosity: an intrinsic desire to learn more Engagement: engagement with the learning process and the

school community Mastery orientation: the desire to complete tasks to the best of

one's ability

Communicative efficacy: using communicative skills in context to achieve a purpose Empathy: responding to another person with the same emotion and being able to understand another person's perspective Acceptance: having respect, tolerance, trust to be able to understand society and the qualities of othersConnectedness: meaningful linkage with a wide range of people


Two questionnaires were devised - one for teachers and the other for students (Year 4 or above). To date, the instruments have been used in the ACT and Tasmania. The Tasmanian data represent a significant sample.

The respondent numbers were:• ACT: 27 teachers (representing 27 schools i.e. 24% of the participating schools) and 43 students (from Years 4, 5 and 6)• Tasmania: 50 teachers (representing 40 schools i.e. 53% of the participating schools) and 240 students (from Years 4 to 10 in 48 schools)

teachers were able to use the instrument to identify significant positive impacts of EfS for their student’s well-being across most dimensions – they also gave examples of observed behaviours that supported their assessments.

Students agreed that, ‘doing things for the environment made them feel better’ – from the ACT, 34 (or 87%); and from Tasmania, 143 (79%) in Tasmania. They also ranked a series of statements that linked with the dimensions used in the teacher survey.

I feel like a plant with water.I feel like I'm doing better things for Australia.

I feel happier and more safe.I feel really proud of myself because I know that I’m doing the right thing.

It makes me feel happy inside that I can change the world in one small step. I feel like I’m eating a cake full of smiley faces and rainbows!!!!!

7. Overview

each State/Territory has been able to contextualise the AuSSI model for their policy and schooling systems in a way that is achieving the structural shifts towards the AuSSI vision

For all Australian schools and their communities to be sustainable.

After 4 or so years ... a critical mass of schools that understand the model and how its component elements can be structured to effectively achieve systemic behavioural change towards sustainability.

The challenge ... ensuring that this strong foundation becomes self-sustaining within each of the States/Territories.

My advice ... focus more sharply on what works best in each context and strive for greater efficient and effective for all. Avoid dissipating the limited energy and resources of schools and AuSSI project officers on time consuming activities with little return on investment in either student outcomes in EfS or school recognition (e.g. writing unsuccessful grant/award applications; expanding curriculum integration across too many areas so that practice is 'thinly' spread rather than deepening and consolidating what is now known to work well for 'sustainable' EfS). http://www.environment.gov.au/education/AuSSI/what-is-AuSSI/vision.html

7. Overview

What is it that I really believe? Sustainability underpins our school’s core values – care and respect for one another. Through the sustainable

schools initiative as a staff we have become more aware of the incredible value of educating for sustainability – how it will equip our students for

life. It engages students through their hands, hearts and minds. It encourages them to problem solve, care, reflect and research. It’s about

kids making connections and shifting the control and ownership of learning from the teachers to the students. The teachers act as learning

facilitators – providing big picture opportunities – authentic, personalised learning. Creating partnerships with the community.

Vicki Hales, Principal of Latrobe Primary School Quote from material provided by AuSSI-Tas

Documents

Evaluating Education for Sustainability - A reflection after 10 years AES 2011, Sydney Larraine J Larri