Establishing a multi-dimensional quality framework for assessing work-based learning

Kathy Henschke, Beverley Jackling, Friederika Kaider, Joan Richardson

Acknowledgements: Mark Tolson, Lynn Yu

RMIT University©

What is Work Based Learning?

• Industry-based learning (IBL), work-based learning (WBL) and work-integrated learning (WIL):

–university programs that bring together universities and work organisations to create new learning opportunities for students in workplaces. Provide students with professional and context-based, experiential learning opportunities

• At RMIT WIL includes co-operative education (Co-op), practicum, internships, industry-based projects, simulated work activities

RMIT University©

Our Research

• Why–RMIT WIL Policy

• Who was Involved– Academic Development Unit; Schools of Accounting & Law; &

Business Information Technology

• Scope of Investigation–Undergraduate programs across discipline schools in the College

of Business; 6-12 month placement

• How–Literature reviews; document analysis of course guides; student,

academic and industry questionnaires; academic focus groups

RMIT University©

RMIT WIL Policy

Principle 1

The key feature of a WIL experience in RMIT programs is assessed professional or vocational work in a work context in which feedback from clients and others from industry and community is integral to the experience.

Can be simulated.

This 'learning by doing' critically involves the experience and assessment of ‘doing’ in a context which reflects a realistic work situation along with work relevant interactions

RMIT University©

RMIT WIL Policy

Principle 2

All RMIT programs shall set a goal of integrating effective WIL activities, particularly the WIL assessments of Principle 1 above, through the curriculum of the program as a whole, as part of all students’ experiences.

Principle 3 By 2010: All advanced diplomas and associate degrees shall have at

least one core course where WIL activities of Principle 1 above are the predominant assessments; all higher education bachelor awards will have one or more core courses totalling at least 24 credit points of WIL; and all graduate diploma and master by coursework awards will have one or more core courses totalling at least 12 credit points of WIL

RMIT University©

Findings - Literature Review

• Types of Learning (Bloom, 1984)

• How to deliver/promote learning - Pedagogical WBL Model (Brodie & Irving, 2007)

• How adults learn influenced by personal, interpersonal, institutional, social and historical factors (Foley, 2004). Complexity of WBL (Smith and Sadler-Smith, 2006 )

• Collaboration between stakeholders (student, university & industry) to manage diversity (McNamara, 2008)

RMIT University©

Findings – Analysis of Course Guides

Examination of assessments in Co-op WIL programs in theCollege of Business (20,000+ students):

• Predominant assessments were a business proposal and a business report (ranging from 3000-7000 words)

• Little evidence of:

–assessments specifically measuring objectives outlined in course guides

–guided reflective exercises

• However, innovative WIL learning activities and related assessments were being developed in some programs

RMIT University©

Findings - Survey

Student satisfaction with types of WIL assessment

–Employer Evaluation 3.29

–Project proposal 3.16

–Workplace tasks 3.16

–Project report 3.13

Least satisfied with:

–Formal oral presentation 2.51

–Diary/Journal/Log 2.33

RMIT University©

Findings - Survey

Skill Mean Std Deviation

Analytic skills 3.10 .709

Problem Solving skills 3.10 .709

Written communication 3.08 .829

Plan and manage work 3.07 .797

Initiative and enterprise skills 3.05 .749

Helped workplace preparation 3.02 .974

Team skills 2.95 .815

Application of theories/concepts 2.90 .810

Benefits of work placements on student generic skill development

RMIT University©

How we made sense of our findings

1. What is the teaching, learning & assessment practice at RMIT?

RMIT T & L

Practice

Activities designed to enable students to acquire the capabilities

outlined in the objectives

Learning Objectives

OutcomesLearning Activities

Provides a rich learning environmentCrosses discipline boundaries

l

discipline-specific capabilities and graduates attributes/ employability skills

assessments employed to measure & provide evidence that students have acquired

capabilities outlined in objectives

RMIT University©

How we made sense of our findings

2. What is the pedagogical underpinning for RMIT’s learning and teaching practices?

• Bloom’s Taxonomy : three domains of educational activities

• Affective domain includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasms, motivations, and attitudes

Bloom’sTaxonomy

growth in feelings or emotional areas (Attitude)

manual or physical skills (Skills) mental skills (Knowledge)

affective

cognitivepsychomotor

RMIT University©

How we made sense of our findings

3a. What does the literature say about WBL?

• Brodie & Irving (2007) propose a ‘inter-relationship and inter-dependency between understanding learning, critical thinking and capability building within a WBL context’ (p11)

• Workplaces provide a rich learning environment

• Cross discipline boundaries

PedagogicalWBL Model

Students learning how they learn – approaches, theories, validity,

applicability, to make most of learning opportunities

Students reflecting critically on their learning: applying models, establishing validity, applicability, appropriateness

Self audit, target settings: interpersonal & transferable capability, ‘work’ based –

technical/discipline/subject specific

Learning

CapabilitiesCritical reflection

RMIT University©

How we made sense of our findings

3b. What does the literature say about WBL

• The Action Learning Cycle. An action learning model encourages the learner, the organisation and the program facilitators to reflect, review, and modify assumptions

and actions (Garratt, 1997; Weinstein, 1999)

RMIT University©

How we made sense of our findings

4. Recognising the complexity of workplace learning

• Influenced by a highly complex set of variables loosely grouped across three overlapping dimensions: the contextual dimension, the social dimension and the learning dimension (Smith & Smith-Sadler, 2006).

• One size does not fit all!

Complexityof

WBL

nature, type & size of companyStrategies, policies, structures, processes; learning orientation

assumptions, expectations & concerns held by the stakeholders

towards learning

Learning

SocialContextual

formal, non-formal, informal & incidentalOccurs consciously/unconsciously

Individual learning styles

RMIT University©

Criteria for the CCARDS Assessment Framework

C ONTEXTUAL Does the assessment tool provide for the integration of workplace information?

C APABILITY DRIVEN Does the assessment tool specifically measure the discipline-specific capabilities and graduate attributes / employability skills articulated in the course objectives?

A CTION-BASED LEARNING

Does the assessment tool promote the improvement cycle of action based learning?

R ELATIONSHIP COLLABORATION

Does the assessment tool factor in feedback from workplace players? How do all the stakeholders collaborate and relate?

D EVELOPMENTAL Is the assessment tool developmental in nature? Does it specifically provide for formative feedback and assessment?

S TUDENT–CENTRED Does the assessment tool recognise the self-directed learning and increased responsibility that the student takes in learning in the workplace?

RMIT University©

Case Study: Using CCARDS Framework

C ONTEXTUALDoes the assessment tool provide for the integration of workplace information?

eg organisational analysis

C APABILITY DRIVENDoes the assessment tool specifically measure the discipline-specific capabilities and graduate attributes / employability skills articulated in the course objectives?

eg innovation, problem solving, business communications

A CTION-BASED LEARNING

Does the assessment tool promote the improvement cycle?

eg project plans and report on achievements

R ELATIONSHIP COLLABORATION

Does the assessment tool factor in feedback from workplace players? How do all the stakeholders relate?

eg negotiated partnerships between students, workplace supervisor & academic supervisor

D EVELOPMENTALIs the assessment tool developmental in nature?

eg formative feedback on learning activities

S TUDENT–CENTRED Does the assessment tool recognise the self-directed learning and increased responsibility by the student?

eg reflective journals

RMIT University©

D AR

CC

S

Assessment CCARDS