Adopting an e-Portfolio as an assessment tool:Investigating options, issues and future

possibilities

Dr Robyn Smyth Ms Catherine Studdert

Academic Developer JMP Project Manager - Occupational Therapy

School of Rural Medicine School of Health Sciences

Faculty of the Professions Faculty of Health

University of New England The University of Newcastle

Dr Graeme Horton Professor Ian Symonds Mr Brett Griffin

Senior Lecturer in Medical Education Dean of the JMP IT Project Officer

and General Practice Professor of ObstetricsSchool of Medicine & Public Health School of Medicine & Public Health School of Medicine & Public Health

Faculty of Health Faculty of Health Faculty of Health

The University of Newcastle The University of Newcastle The University of Newcastle

Cross-institutional teaching of medical students – rural/regional footprint

200 students per year over wide geographical base

Assessment Committee working party investigating potential use of e-Portfolio

Chose Buzetto More & Alade’s 2008 Pentagonal e-Portfolio Model as theoretical framework

Approach taken: pedagogy to drive choice of technology

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

2Context

Joint Medical

Program

Where are we going?

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

3

Our Needs

Recording of valuable educational experiences for later reference by students

Interoperability and secure storage of sensitive information

Tracking of clinical experiences, especially those which trace the path to developing professionalism

Assessment of student learning and competencies

Provision of useful feedback to students

Provision of a tool which encourages students to reflect on their experiences and learning, which can form the basis of further dialogue with mentors.

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

4

Our Needs

RecordingStoring

TrackingAssessmentFeedback

Reflections

e-Portfolio: student-centred

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

5

17 October 2011 ePortfolios Australia Conference

2011│www.newcastle.edu.au/jmp

6

Current IT Architecture

e-Portfolios

Cohesive electronic repositories Used for recording, storing and tracking students’ work,

assessment items, feedback and evidence of clinical experience and competencies

Pedagogically and technologically progressive Student-centred approach Have the ability to adapt to their needs whilst learning in clinical

and non-clinical settings, and rural and urban locations. Promote reflective learning and planning Enhance communication Create a record of a student’s work and experience to support

their professional careers and life-long learning

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

7e-Portfolio

Student-Centred

Approach

Scholarly Approach

Our investigations have been guided by our conceptual framework reflective of Buzzetto-More & Alade’s (2008) Pentagonal e-Portfolio Model:

I. Identify Needs (purpose, performance indicators and learning outcomes)

II. Determination, Assessment & Budget (select systems features, assess existing technology, determine type, determine budget)

III. Select & Strategic Planning (select system, develop implementation plan)

IV. Development (build into curriculum, develop rubrics, assessment, allocate resources)

V. Implement & Grow Towards Continuous Improvement (prepare students/staff, pilot, testing, evaluation, implementation)

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

8Theory

Buzzetto-More & Alade (2008)

ANZAME09 poster

Our Investigation

From niche medical suites to general e-Portfolio software– None fit all our needs, but some are candidates

Anecdotal reports that e-Portfolios are desired by our students– Flexible– Shows progress– Supported

Literature Review of e-Portfolio in Medical Programs– Remember staff;– Introduce early in program where possible– Fit to purpose

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

9

Our Investigation

Available Software

Peer Experiences

17 October 2011 ePortfolios Australia Conference

2011│www.newcastle.edu.au/jmp

10Our dream IT Architecture

2010 Pilot - Overview

Small group of Newcastle-based student volunteers in Year 4 Joint Medical Program

Replaced paper-based portfolio assessment component

Training and support mechanisms provided

Volunteers participated in two focus group sessions

Could at any time elect to submit the paper-based portfolio

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

11

2010 Pilot e-Portfolio

Using Mahara based foliospaces.com

Using free accounts (50MB space)

Students using Google Docs for documents (free 1GB space)

Students had to present Work Integrated Learning (WIL) content against course competencies whether in paper or electronic

Students had freedom in media types they could present their competencies within their e-portfolios

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

12

The e-Portfolio Student Profile

• Student accesstheir e-Portfolio‘Views’

• Communicate withAcademic Mentors

or Co-ordinators

• Access to the support group

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

13

The e-Portfolio e-Portfolio Section View

• ‘View’ part of overalle-Portfolio

• Specific topic (view on each Rotation topic)

• Student use any tool to record clinical experiences (against competency list)

• De-identification ofpatient data

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

14

The e-Portfolio Support Mechanism

• ‘Groups’ in foliospaces– Forums– Files– Pages member list

• Restricted to volunteers,academics and supportstaff only

• Students can help eachother with online tools

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

15

2010 Pilot – Informing 2011 Trial

Students engaged and reflected more often on their clinical experiences (than they thought they might otherwise have done)

Described Mahara as easy to use

Would loved to have had exemplars to peruse before building their own e-Portfolios

Targeted support: self-help vodcasts, 10-step e-Portfolio setup, etc.

Would like to use e-Portfolios, particularly in early years of the program

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

16

Next project…

Full cohort of Newcastle-based Year 4 JMP students in course

– Greater use of feedback to mentor students directly in e-Portfolio

– Train more academic markers and mentors

– Greater use of online support snippets, forms and podcasts

– Use of Elluminate for one-on-one support and application walk-throughs while on placement

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

17

17 October 2011

e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

18

Theorising, Practising, Adapting..…

THANK YOU

17 October 2011 e-Portfolios Australia Conference 2011│www.newcastle.edu.au/jmp

19