Promoting a deep approach to

professional learning in the field

Development and evaluation of electronic personalised learning

G Finnerty, M A Volante, L Rockingham, J MacLaren and M O’Driscoll

Centre for Research in Nursing and Midwifery Education

Division of Health and Social Care Faculty of Health and Medical Sciences

June 2008

2

Acknowledgements

We would like to thank:

All the students who participated in the project from Cohorts 1 and 2 from the Mentorship module.

The four students who gave of their time to speak to us about their mentor experience.

Colleagues in the Mentorship teaching team.

Colleagues from the University of Surrey e-learning units and in particular Dr Steve Rowett whose resourcefulness made the project possible.

Members of the Steering Group.

Dr Andrew Comrie and Professor Pam Smith for advice, guidance and support throughout the project.

This project was funded by the University of Surrey Fund for the Strategic Development of Learning and Teaching.

3

CONTENTS

1.0 Summary..........................................................................................................5

2.0 Introduction.......................................................................................................6

3.0 Background......................................................................................................7

4.0 Module design, development and delivery for personalised learning plans……………………………………………………………………………………...…8

5.0 Literature Review...........................................................................................12

6.0 Project approach and evaluation....................................................................18

7.0 Findings..........................................................................................................22

8.0 Discussion......................................................................................................42

9.0 Conclusions and recommendations...............................................................45

APPENDIX ONE........................................................................................................48

4

Glossary of terms

E-tivities Electronic activities

ICT Technology

KSF Knowledge and Skills Framework

PDA Personal Digital Assistant

SPT Specialist Practice Teacher

VLE Virtual Learning Environment

5

1.0 Summary Aim of project

To develop an e-portfolio to manage student progress and assist with authentic assessment on the Mentorship module (FHMS).

Objectives

To evaluate student experience of peer-supported reflective learning using electronic portfolios

To increase staff understanding and potential of e-portfolios as a way to engage students in meaningful learning.

Evaluation Methods

Nominal group technique and e-learning questionnaire involving e-voting by students

Semi-structured interviews with module tutors in FHMS and staff from e-learning unit

Biographical narrative interviews with a sample of students following successful completion of the Mentorship Module.

Key Findings

Mentor students managed their learning and progress through use of the Virtual Learning Environment (U-Learn). Support was presented through multiple strands, for example, handbooks, lectures, peer engagement, a help desk, IT services and the E-Learning Unit.

Mentor students actively engaged in the reflection process to inform their learning and, importantly, their teaching and assessment.

Tutors felt they had ownership of the module, which they described as organic and ‘living’.

Learners described a deeper understanding of teaching styles, theories and new ways of performing in challenging mentoring roles. The blended learning approach was generally viewed as a positive innovation to inform mentoring as an art and a lifelong skill.

Personalised learning within dynamic e-portfolio is a product of the learner’s own history, their future aspirations and how they engage with the design of learning activities, including the use of e-technologies.

A route map which addresses learning strategies and content is part of the scaffolding necessary for learners to organise information into meaningful learning outcomes. This is demonstrated by the building of an e-portfolio of personalised learning products.

Capturing student biographies is an effective approach to illuminate understanding the modification of personal learning schemata.

6

2.0 Introduction This report presents the outcomes of an education development and evaluation of technology supported personalised learning undertaken during 2005 – 2007. The work was funded by the University of Surrey Fund for the Strategic Development of Learning and Teaching. The aim of the project was to: develop an e-portfolio to manage student progress and assist with authentic assessment. The project used the continuing professional development ‘Mentorship module’ (30 credits level 3) to pilot the development. The specific module context provided the project with a secondary aim of seeking to provide an effective and efficient learning experience that will enhance the supervision of the student experience in practice settings recognising the wider system in which mentoring and personalised learning plans are located.

Specifically the project had the following objectives

use e-portfolios to further develop the student experience of peer-supported reflective learning

support learner progress and assessment through the development of an e-portfolio as a tool for capturing evidence of learning outcomes and for ongoing professional development

enhance the supervision experience of mentor students and their learners on professional placements

increase staff understanding and capacity of e-portfolios as a way to engage students in a meaningful learning experience.

The aims and objectives were addressed using an action research design in which two members of the module teaching team worked with an education developer, and researcher to enquire into technology-supported personalised learning. The two teachers worked closely with an e-learning technologist and the other members of the module teaching team to design the planned e-tivities for students to build their e-portfolio. The action research design intended to facilitate staff engagement with the pedagogical practices necessary for the design and support of student centred e-learning. A programme of staff development for teacher moderated on-line discussion was put in place. A literature review on e-portfolios was undertaken and the e-tivities supporting personalised learning plans were evaluated using questionnaires and semi-structured interviews with staff. To understand the learner mentor experience within the wider context of the professional learning in the field, narrative interviews were undertaken after the completion of the module. This report summarises the outcomes of the development and evaluation as a basis for conversation and discussions with colleagues on e-developments for professional learning.

Section 3 provides contextual background to professional learning in the field.

Section 4 outlines the design, development and delivery of the e-learning education interventions to support personalised learning and the building of an ‘e-portfolio’.

Section 5 presents a review of selected relevant literature.

Section 6 descibes the project approach and methods for evaluation.

Section 7 presents findings using three perspectives. Firstly, the student perspective with findings from the standard module evaluation questionnaire, the e-learning evaluation tool and classroom field observations. Secondly, the teaching staff

7

perspective with findings from the semi-structured interview and finally presenting the findings from learner mentor biographies using narrative interview.

Section 8 draws together the key issues emerging from the evaluation of electronic personalised learning plans.

Section 9 provides some conclusions and offers recommendations for action for consideration by relevant teaching and learning communities of the University.

3.0 BackgroundThe project used the mentor preparation module as a case for enquiry into the development of e-learning by means of an ‘electronic portfolio’ that supports learners to manage their progress and assist with meaningful assessment processes. A technological advantage of e-portfolios over traditional portfolios is that they are sharable, which facilitates interaction between student and student as well as teacher and student (Cotterill et al. 2004). The flexibility of the presentation methods and the embedding of interactive elements in the building of e-portfolios have been found to deepen learning for trainee teachers and to also reduce workload pressure in comparison with producing traditional portfolios (Woodward and Nanlohy 2002). Peer sharing and interaction was made possible through the discussion tool of ULearn.

Whilst it was intended to supply the students with their own on-line learning space to capture meaningful learning and build an e-portfolio, early in the project it became clear that this was not going to be feasible with the information technology support available. Instead, following discussion with the e-learning unit and Dr Rowett (in particular), the functions of the university‘s virtual learning environment (ULearn – Web CT Vista) were used to enable students to experience the use of technology to support personal learning intentions and to show the personal process of enquiry undertaken within the overall learning strategies of the module, for example, reflective essay incorporating feedback from e-mediated peer discussion. The decision was taken during the delivery of the module that the students would be asked to submit a conventional paper based portfolio. Whilst some of the students may have wished at the end of the module to have submitted their portfolio of learning electronically, consideration also needed to be given to where the teachers had reached in their understanding of e-learning as well as organisation processes for example, the examination office staff.

Undergraduate teaching and learning in the Division of Health and Social Care is delivered through programmes designed around classroom based learning in the university and the development of clinical competence through practice learning and reflection with mentors (workplace supervisors) in the local health economy. Methods of assessment are based on engaging students in meaningful learning activities, the products of which form a portfolio of evidence of professional knowledge, understanding including learning how to learn, and clinical competence. The portfolio is used to both track student progress and to assess student achievement of practice learning outcomes at the appropriate academic level. Mentors are a crucial resource in the assessment of practice learning outcomes and clinical competence and are required by the professional body of the Nursing and Midwifery Council (NMC), to undertake mentor preparation through University accredited and NMC approved courses (NMC 2006).

The Division of Health and Social Care is contracted for continuing professional development by the local Strategic Health Authority to provide mentor preparation

8

(amongst other provision). The module is also used by other healthcare employers. Over one academic year the module is taken by approximately 400 learners. During the year 2006/07 the module ran 11 times. This project planned to use data from the first and second student cohorts of 2006/07 and the lessons learned through ongoing evaluation by the teaching team of the following cohorts has informed module design and delivery within the academic year.

4.0 Module design, development and delivery for personalised learning plans In this section the e-learning educational intervention is presented and outlines the curriculum design and development of the module; the student constituency; the university teachers and staff development for e-learning, and the specific e-tivities supporting the building of a portfolio of meaningful learning experiences.

4.1 Module development

• Blended campus and distance learning (15 credits at HE2/3)• Assessed by profile of reflective writing and mentor assessed learning

development plan (competency document) Each cohort was run by a different pair from a group of 6 module tutors

4.2 The student constituency

The majority of students who undertake the module are registered practitioners working in health services for example, nurses, midwives, operating department practitioners and ambulance service personnel. All are adults who are working and taking the module for professional development either in relation to their own personal aspirations or because of employer requirement. The range of ICT skills within health care is well known (Laurillard 2005) and a typical early posting on ULearn during the project reflects student difficulty:

“I am struggling with this way of learning, not sure it’s for me? It’s really quite stressful” .…

4.3 The staff

The Mentorship module is resourced by a core group of six teachers who have a particular interest in learning and teaching within practice settings and have been supporting the module over various lengths of time. The teachers had been involved in the working group undertaking the curriculum review and redesign of the module to meet the NMC standards for supporting learning and assessing in practice settings (NMC 2006).

The teachers were afforded a range of staff development in relation to ULearn and teacher supported peer mediated on-line group discussion. All university teachers are required to undertake the introduction to ULearn provided by the University e-learning unit in order to become a ULearn user. The project made funding specifically available for the six teachers to undertake the Oxford Brookes course peer learning on line. Four teachers took up the opportunity. Two members of the teaching team who supported cohorts 1 and 2 were key participants in the action research design for implementation of the e-portfolio development project. They were supported by an e-learning technologist from the university e-learning unit. The table overleaf summarises the structure and content of the e-tivities undertaken by the mentor students during Units 1 and 2 of the Mentorship Module

9

Table 1: Summary of e-tivities within overall module

Unit of learning

Coursework to be produced E-tivities Other Activities/ comments

Students to undertake pre-course checklist to ensure they have requisite knowledge and skills to be able to participate in the online learning environment

ULearn log-in session

Hopes, fears and expectations discussion thread

Introduction to

o module café for social interaction and discussions

o sources of help including the ‘Help-at-hand’ area

Discussion of library resources and online access of books, journals etc.

Threads throughout module

1 Learning and Teaching in Practice Level 2 & 3

For this assignment you should develop a teaching plan that you will use with your learners. You should submit this plan with a 500 word reflective critique which explores the effectiveness of your plan in the light of relevant theoretical literature, and personal and professional experience. Within this unit you should include a discussion of how you used learning theories and domains of learning to support the development of your teaching plan.

Discussion thread: Priorities of workload and the learning environment (level specific - students access the discussion at the level they are studying)

Uploading of draft assignments and receipt of feedback

Assignment Draft uploaded for review by personal tutor by end of day three of module

10

1 and 2 Collaborative online small group working to produce a seminar on their chosen topic. This combines enquiry based learning and group ULearn discussions for classroom presentation

Group specific discussions with allocated online tutor

Student led seminar presentations on day four

2 Assessing and Evaluating Learning

Level 2

Prepare a 500 word reflective discussion which identifies two types of assessment used to assess learners in your clinical practice. You should compare and contrast these methods / strategies in relation to their strengths and weaknesses, paying particular attention to the concepts of reliability and validity.

Level 3

You should prepare a 500 word reflective critical analysis of the concepts of reliability and validity in assessment in relation to your assessment of a learner in your practice area. You should draw upon a range of resources, including personal and professional experience.

Level specific discussions based around individual learning journeys (My learning journey) in preparation for part 3 reflection ‘my learning journey’.

face-to-face student feedback and evaluation of seminar presentations

11

A written reflection:

Unit Three: My Learning Journey

Level 2 & 3

This final piece of work for your profile is a 1500 word reflection on your development as a mentor. The title of this is ‘Reflecting on my journey from the beginning of the course how do I now feel about the mentorship module and my practice as a mentor?’ You should draw upon your experiences of undertaking the module, in attending, studying and mentoring learners in your practice area. You should demonstrate evidence of appropriate reading around the subject of mentorship by referring to the relevant theories you have learnt about.

Your work should have a clear reflective structure, and clearly demonstrate description, analysis, synthesis, evaluation and self-awareness, and give an idea of how you feel you may further develop your mentorship whilst fulfilling all the learning outcomes.

My Learning journey and group discussions remain open

To be submitted 1 month following completion of the module with unit 1 & 2 reflections and learning and development plan completed in practice with mentor (paper copy)

To be submitted (paper copy), 1 month following completion of the module with unit 1, 2 & 3 reflections

Learning and development plan

Competency assessment in practice Four proficiency statements must all be signed off and include comments on your practice at both the formative and summative stages.

12

5.0 Literature Review5.1 Introduction

The purpose of this review of literature is to highlight the key elements which have informed the development of electronic resources within the Mentorship module in the Division of Health and Social Care. Evaluation of these new developments has entailed reviewing and analysing electronic portfolios used for medical training and teacher training. The literature review is organised by firstly stating the purpose of e-portfolios and defining terms. How e-portfolios are structured will be examined before discussing problems which have been reported through evaluations in Australia, the United States of America (US) and the United Kingdom (UK). This review then incorporates evaluations which include views of students, teachers and educators who have used these electronic records over the last decade. The intention is to complement other work in progress involving e-portfolios and electronic tools for recording learning across the University.

5.2 Purpose of e-portfolios

The primary purpose of the e-portfolio is provision of a pedagogical tool for recording and managing learning. Garrett and Jackson (2006) suggest the electronic portfolio should be used as a tool to contextualise and embed knowledge in the workplace but Hennessy and Howes (2004) assert that, although useful as a collection of evidence of learning, the e-portfolio should not be merely a ‘transcription tool’ or ‘archive’ (Herbert, 1998). It would appear that e-portfolios are increasingly being used for job applications and promotions and also as a tool to show development of thinking (Butler, 2006). As Cambridge (2004) asserts, the electronic portfolio should represent what a person (or organisation) has learned over time through a collection of authentic and diverse evidence. Across the literature, it would appear that what has the most meaningful pedagogical influence is how learners are enabled to analyse their own evidence and take responsibility for their learning.

The e-portfolio has been shown to be a valuable tool to empower learners through use of self-evaluation and reflection, assisting with planning of work-based learning (Graham 2004). This is of particular value to mentors as a tool to assist with planning supervision and assessment of students in diverse workplace settings. Cotterill et al (2004) also emphasise the positive value of transferability of data to support life-long learning. It is this capacity for recording personal and professional growth which makes the electronic portfolio a versatile instrument in current fast-changing health environments. The social networks made possible through use of e-portfolios also has potential to promote lifelong learning (Cambridge, 2004). Additionally, the e-portfolio can assist learners to see patterns in their work over time and to process feedback from tutors and peers. Importantly, e-portfolios provide learners with the opportunity to see their progression and to visualise how all the learning and activities link together (Williams 2000).

Georgi and Crowe (1998) cite Gardner (1987) who suggested the portfolio provided:

13

“Naturalistic sources of information about how peoples around the world develop skills important to their way of life” (p189). Hamilton (2004) terms these skills ‘transcendent’ as they emerge through the e-portfolio building process and can encompass such things as heightened critical thinking and understanding diverse societies and cultures. Cambridge (2004) claims that the test of this durable learning is that it can last beyond a module or programme.

The e-portfolio is now frequently used as a tool to ‘showcase’ examples of best work, for example through video-clips or podcasts. The increasingly interactive elements provided by creative hardware, software and institutional VLEs (Sharpe et al 2004) mean that active learning is promoted through providing learners with a platform to dialogue with peers and tutors. This provides a medium to engage students in meaningful learning experiences. This is particularly relevant with a new emphasis on the ‘student voice’ (Hughes, 2006), which has only recently received attention (Kilpatrick 2007).

Discussions abound as to the extent to which e-portfolios can be used for performance-based assessment (Georgi and Crowe, 1998). This important issue will be covered in more detail later in the review.

5.3 Structure of e-portfolios

Individual, faculty and institutional models are used across the United States (Cambridge 2000). Throughout the literature, decisions regarding constructing and implementing an individual e-portfolio should include:

Agreeing the purpose of the electronic portfolio, i.e. for learning, assessment or professional development

Prescriptiveness of the guidelines for the creation of an e-portfolio How pieces of selected information should be organised, for example, around

themes selected by the student or lecturers? Input of tutors, lecturers and peers, including feedback mechanisms.

Learners are being encouraged to record their experiences using a variety of media, including mobile clinical e-portfolios, using wireless Personal Digital Assistants (Garrett and Jackson, 2006). As asserted by Laurillard (2007) mobile technologies such as pod-casting also enable continuity between contexts and provide opportunities for social construction of knowledge (Grob 2008). There is still a need, however, for provision of ‘tangible flags’ or sign posts for students. Laurillard (2005) emphasises how essential explicit learner goals are at the outset of an e-portfolio. This is because if learners focus on the operational aspects of a task, rather than its meaning or content, surface learning only is likely to ensue.

It is the learner’s choice of modes and materials selected which makes the e-portfolio such a learner-centred activity (Mason et al, 2004). The key to presentation of a successful e-portfolio is not the technological facilities used but the demonstration of reflective learning using a digital interface:

“The most effective electronic portfolios link student reflective practice with products and performances, which indicate hands-on and applied acquisition of practical skills and knowledge involving learning technologies” (Jonassen 2001, p 7).

14

Laurillard (2007) has analysed various modes of mobile technology used by students and teachers within a Conversational Framework. Her research has demonstrated how the richness of mobile learning can be enhanced and digitally facilitated due to immediacy of actually being in the practice environment. Certainly, ‘Pebble Pad’ enables inclusion of multiple social networks for learning and recording practice under one umbrella (Grob 2008, personal communication).

5.4 Medical portfolios

In the UK, e-portfolios are structured in a variety of ways, depending on the users and audiences. They have most commonly been used in medicine. These are often structured as an evaluation tool to assess performance in authentic contexts. As Butler (2006) suggests, this is imperative to record continuing learning and to keep pace with rapid advances and developments in health. Driessen et al (2005) assert that medical portfolios are unique because the focus of reflection is on actions. Problems arise, however, because the portfolio in this format is often presented merely as a ‘dossier of evidence’. Butler (2006) suggests that a misunderstanding of portfolio pedagogy is apparent across the medical literature. Haig (2007, personal communication) disputes this, suggesting that General Practitioner trainees in Scotland report that competencies are linked with the national curriculum and lecture time is subsequently reduced. Implementing a tool which records student progression across professions has been suggested (Cotterill et al 2004). A generic e-portfolio system is in use in Newcastle meaning that collaborative connections can be made, linking with other web-based electronic systems. Cotterill et al (2004) highlight the necessity for e-portfolios to be flexible and easy to customise.

5.5 Nursing portfolios

Having examined a selection of web e-portfolios used in medicine in 2005 (Cotterill et al 2004), it could be said that structures are traditionally ‘rigid’. In contrast, nursing portfolios have been criticised for lacking any ‘unified understanding’ in the literature (Spere and El-Ansari, 2004). This is, however, improving with increased use of digital formats for reflection on practice. The Royal College of Nursing is the largest European professional body and has recently been implementing an e-portfolio for supporting nurses’ continuing professional development and re-accreditation (Cambridge 2004).

5.6 Benefits of e-portfolios Help to focus student thinking, and translate theory into practice (Butler 2006) Aid management of work for engineers through use of a personal website

(Campbell and Schimdt, 2005) An instrument to enhance employability (Cambridge 2004) Can help art and design students form their artistic identity Can help individuals to manage and distribute personal digital information and

therefore contribute to knowledge management (Cambridge, 2004) Provide a tool for asynchronous discussion within a large group Can help to support diverse students and enhance undergraduate learning

(Cambridge 2004) Have potential to improve patient care (Butler, 2006) Provide opportunities to network and communicate both synchronously and

asynchronously (Greenberg 2004)

15

5.7 Problems with e-portfolios

Portfolio construction is notoriously time-consuming and reliant on staff support (Butler 2006). Extra time therefore needs to be set aside for preparation to use an electronic interface (Scholes et al. 2004). Additionally, the technological ‘novelty’ or troubleshooting may overshadow the purpose of the portfolio (Woodward and Nanlohy, 2002). As Laurillard (2005) asserts, although the aim is to promote discussion using communicative media, any pedagogical benefits depend on a healthy dialogue between student and tutor or student and student. Gomez (2004) found that being able to track and guide student learning remotely was beneficial for students on long placements. Conversely, students compiling electronic personal development profiles at Bournemouth University felt there was a ‘Big Brother’ effect which affected authenticity (pdp.bournemouth.ac.uk).

Ownership of e-portfolio data has emerged as a contentious issue. As Kilpatrick (2007) asserts, there is currently confusion as to an institution’s right to retain data and responsibilities of an institution to store information about students in this format. Woodward and Nanlohy (2002) argue that ownership needs to be clearly established by all authors using electronic and digital tools.

Problems also occur with assessing portfolios. The audience needs to be clearly established and defined (Herbert, 1998). Cayne (1997) described problems with lack of guidelines for assessment. For this reason, Webb et al. (2003) stress that marking and grading criteria need to be extremely clear and question whether portfolios are, in fact, valid forms of learning and competence. Problems with assessing professionalism in medicine have also been documented (Butler 2006). A supportive educational environment is therefore essential so that students feel comfortable revealing any weaknesses or mistakes (Pinsky and Fryer-Edwards 2004).

5.8 Views of students

Darling (2001) found that students described feeling confused and anxious if guidelines for portfolio-building were not well defined. Students appeared to benefit if examples of past portfolios were shared and if the scope, nature and value of the development portfolios was made explicit. Resentment by students was common if a balance did not exist between an emphasis on construction which was student-driven and over-prescribed guidelines. Students in Darling’s (2001) study also expressed concern with the subjectivity of evaluation of their portfolios.

Conversely, a sample of pharmacy students (n=154) in Manchester reported that the portfolio was an effective tool to summatively assess work, to support their learning and also prepared them for their future continuing professional development (CPD) (Ashcroft and Hall, 2006). E-portfolios have also been reported to be valued by learners for accessing of information from a “central” place (Hennessy and Howes, 2004).

‘Pebble Pad’ was used by students undertaking a Post-Compulsory PGCE course at Wolverhampton University, to record their progress (Kilpatrick 2007). A significant outcome noted by several students in video transcripts, was the longevity of the tool for blogging within a ‘personalised learning environment’. Daily entries to the e-portfolio were being made more than nine weeks after completion of the course. This demonstrates the power of virtual communities to sustain lifelong learning.

16

5.9.1 Views of trainee teachers

Trainee teachers in one study described how the flexibility of presentation methods and the interactive nature of building an e-portfolio, helped deepen learning and reduce workload pressure (Woodward and Nanlohy, 2002).

Attractiveness for students of graphics, movies, sound and other computer applications help to build a running multi-media record of a student’s growth and development found by student teachers (Bartlett 2000 p 296).

Ford and Olhausen (1991) surveyed the attitudes, beliefs and habits of 115 teacher education students in the US. Just 1% reported that their beliefs about assessment had changed and the assessment of their own portfolios played a critical role.

As part of teacher education, a reflective portfolio is often used to help embed theoretical principles, skills and knowledge in the classroom. Interestingly, there appears to be a dearth of research with a focus on e-portfolio implementation for trainee mentors or trainee nursing and midwifery teachers. With new Nursing and Midwifery Council standards in place (NMC 2006), nurses and midwives with a mentoring role will have to demonstrate that they have attained certain standards and criteria to ‘sign off’ students’ practice as professionally competent. It is possible that the e-portfolio will provide a central medium to display and validate this evidence.

5.9.2 Views of teachers/educator in the literature

E-portfolios have been reported to provide an immediate way to assess learning by teachers (McKenzie et al. 2002). In one study, focus groups were held with medical educators and questionnaires were issued (Hennessy and Howes, 2004). Although staff initially found the e-portfolio web interface difficult to use, some described how overall computer skills improved and benefited as a result of working with the e-portfolio.

Designing Constructivist Learning Environments (CLEs) through portfolio building

Jonassen (2001) asks; how will we know if students are engaging in meaningful learning? Technologies need to be introduced which enhance, not dilute meaningfulness in learning. To help guide learners to construct their learning, a rubric can be introduced. This is a tool or model presented to learners at the beginning of an assignment which:

“Aids the learner in refining the product….to include the desired content aspects, while still maintaining the freedom to develop a product-oriented assignment which reflects the learner’s personality, style and knowledge” (Crawford 2002: 5).

There are a variety of rubrics to suit a range of educational purposes, for example, for ‘holistic critical thinking scoring’ and as authentic assessment tools. Crawford (2002) suggests that a rubric can function as an evaluative tool, guiding a learner’s expectations for an assignment. Jonassen (2001) emphasises the need to assure that students’ learning is:

17

Active ; Constructive; Collaborative; Intentional; Complex; Contextual; Conversational and Reflective.

The following diagrammatic representation of a rubric shows how the attributes and characteristics of meaningful learning are interconnected. These characteristics can be used to aid discussion in the classroom (Jonassen 2001) or used as criteria for assessment to identify students’ progress, understanding, knowledge and skills (Williams 2000).

Figure 1

Constructive

CollaborativeActive/manipulative

Intentional

Complex

ContextualisedReflective

Conversational

Constructive

CollaborativeActive/manipulative

Intentional

Complex

ContextualisedReflective

Conversational

This flexibility and use of a constructivist paradigm obviously offers potential as a focal tool to underpin e-portfolio development. Jonassen’s selected characteristics are so relevant to personalised learning that this rubric will be used as a framework to underpin analysis of interview data from the mentor students. (Please see section 7.3, p 34).

Summary

This short review of the ever-expanding literature on electronic portfolios has covered the key pieces of work which have informed the development of research tools to help present a robust evaluation of implementation of an e-portfolio in the Mentorship module. Although problems with e-portfolios have been identified (for example, issues of ownership of student material) the literature demonstrates that benefits to learners, teachers and institutions may be profound. Key advantages cited are pedagogical benefits of mobile and electronic technologies such as immediate and site-specific communication, enabling ‘live’ channels of conversation and new styles of reflection

18

between students and their peers and even tutors. The Wolverhampton Pebble Pad project showed the potential for on-going and meaningful peer learning. It is evident that signposts and maps for students need to be clear in any constructivist learning environment. The central aim of the current study is to develop an electronic portfolio to manage student progress and assist with authentic assessment. The literature therefore supports the rationale for the project and highlights the need for teachers to engage with students using alternative platforms (such as an electronic portfolio) to create a meaningful, personalised learning experience. The study design has been partially derived from the literature and how the project objectives will be met will be described in the following section.

6.0 Project approach and evaluationThe normal module classroom evaluation process was used to evaluate the implementation of the e-tivities and teaching and assessment strategies supporting learners on the mentorship module to build a personalised learning plan and produce an e-portfolio. The standard module evaluation form was used along with the university e-learning unit feedback questionnaire. These two tools have consistently been used for all of the 11 cohorts undertaking the module, and provide teachers with data as a basis to review their actions in an action research process of teacher enquiry. The evaluation format was extended to incorporate qualitative in-depth feedback on the production and development of an e-portfolio within the professional life course of the mentorship module student. Teaching staff were also invited to participate. The intention of the evaluation is to illuminate how the supported student enquiry process used by the student builds a personalised e-portfolio for assessment of the module outcomes. The approach draws on that of illuminative evaluation to compare planned activity with student perceptions and experience of building an e-portfolio.

The main measures to identify project outcomes are:

Collation of student views of the experience of the process of building an e-portfolio to manage their progress

The level of engagement, formative input and levels of support in the process of reflective practice learning

The value of electronic portfolios for assisting with managing the clinical supervision of pre-registration students and also to manage continuing professional development

Teacher’s understanding of e-portfolios as a means to provide evidence of students’ learning and as a tool to enhance meaningful learning

Sample

The sample for the project and its evaluation consisted of:

The first two cohorts from a total of 11 cohorts undertaking the mentorship preparation module during the academic year 2006/07 (n= 62)

Teaching staff of the mentor preparation module team and the e-learning unit (n=7).

19

Ethical considerations

Professor Desombre (Chair of University Ethics Committee) was consulted in October 2006 regarding status of this evaluation for ethical approval. It was agreed that, because the participants had informed consent, including the project team’s intent to publish, a University ethics opinion was not necessary. All participants, students and staff were provided with written information sheets describing the project evaluation. These were issued by the evaluator during Day Five (Evaluation Day) for both Cohort One and Cohort Two in November, 2006. In March 2007 the evaluator collected the module results list from the module administrator, which was confirmed by the examinations office. All students who had passed the module were sent a formal letter inviting them to participate in an in-depth biographical narrative interview. Students were approached to participate in the interview after the module assessment results were in the public domain to avoid compromising student achievement of learning outcomes during module delivery. The decision was made that only those who passed the module were invited to participate so as not to intrude on retrieval processes.

Data sources and analysis

Three sources of data were used for the evaluation to understand how students were supported and experienced building an e-portfolio and their experience of the mentorship module within their professional life course. Also staff views of participating in the delivery of the mentorship module and educational e-tivities interventions were captured. The data sources were as follows:

1 Classroom evaluation: Standard module evaluation form, e-learning unit survey and classroom evaluation observation 2 Semi-structured interview with mentor preparation module tutors and e-learning unit staff3 Biographical narrative interview

These are now presented in turn.

1 Classroom evaluation

Standard module evaluation formAll students completed the self-report standard module evaluation form which uses a 3-point satisfaction scale of very satisfied, satisfied and unsatisfied. Specific questions on ICT and module teaching and assessing strategies were selected for analysis using frequency counts. Responses were analysed in greater depth using the responses to the e-learning survey.

E-learning surveyAn e-learning questionnaire was developed by the e-learning unit to evaluate student perceptions of using an electronic platform – ULearn. Responses were obtained from cohorts 1 and 2 using an electronic voting system capturing individual feedback via

20

handsets. This was carried out in the classroom as a group activity (see photograph)and therefore only aggregate frequency count data was available for each cohort.

Figure 2

Photo courtesy of SR from e-learning unit on use of electronic voting system to capture responses to e-learning questionnaire.

All students engaged in the exercise and, when giving informal feedback about the method, the majority stated that they particularly liked the immediacy and visual impact of the graphic scoring. In presenting the data analysis Cohorts 1 and 2 are amalgamated except where large differences were found across Cohorts.

Observation

The observations recorded by the evaluator took place during the classroom evaluation session following a nominal group technique exercise1. The evaluator was present during the nominal group activity for both cohorts and captured some feedback from the student discussion of Cohort 1 and the ‘closing circle’ discussion of Cohort 2. The data in ‘field note’ form of what was said during these closing discussions is complementary to the quantitative data sets of the standard module evaluation form and the e-learning questionnaire. The data were used to illuminate the quantitative findings.

1 The data from the nominal group technique were not used for evaluation of the project. The ease with which students engaged with U-Learn seemed to be largely dependent on each student’s confidence and prior experience with e-technologies. It was therefore challenging to make general statements which were inclusive of the group’s views. The statements also differed between Cohort 1 and 2.

21

2 Semi-structured interviews with tutors and personnel in the e-learning unit

During the lifetime of this project, the Mentorship module teaching team comprised seven teachers of whom five agreed to participate in the interview. The purpose was to ascertain views of setting up a new style of module and to also provide a space for them to share how they visualised the in-depth component of the evaluation taking place. The five teachers from the mentorship module teaching team were interviewed at the university using a semi-structured interview schedule. Each interview lasted approximately thirty minutes. The same questions were posed to all five respondents. One member of the e-learning unit was interviewed and provided detail about the setting up of electronic portfolios on the University VLE. The interviews were analysed using thematic content analysis (Patton 1990).

3 Biographical narrative interview (BNI)

Four students responded to the letter of invitation with an information sheet to participate in the interviews following feedback on their assessment results. The interviews were planned to take place after the completion of the module and submission of the portfolio of evidence for assessment to avoid compromising student achievement of learning outcomes during module delivery. All students were from Cohort One and had the permission of their line manager. The interviews were carried out by GF at the working sites of three out of the four respondents. This assisted in gaining contextual information about the workplace teaching and learning environments described by the students. Each interview lasted one hour and participants were given a high street store voucher for £25.

To gain a complete biographical narrative, it is important to have a defined beginning, then sequence of events, then an ending (Wengraf and Chamberlayne, 2006). The BNI method necessitates asking firstly a single question to induce narratives or stories from the respondent. The question asked of the four participants was designed to capture their story, beginning from a time that was significant for them:

We are interested in practitioners’ experiences in becoming a mentor. Please tell me about your experiences of the mentorship module from when it became important in your life and up to now. Please take all the time you need. I’ll listen first and take some notes as you are speaking and ask questions when you have finished talking.

The BNI method is structured around two sessions (Wengraf 2007). The first session involves free speaking of the participant, which is uninterrupted. The interviewee is encouraged to speak from their frame of reference rather than the interviewer’s. During the interview interaction, emphasis is on active listening by the interviewer so that cues can be recorded, in preparation for the second session. This is centred on capturing ‘particular incident narratives’ (PINs) considered to be the nearest to the actual experience when it occurred. Questions are asked in the same order as presented by the interviewee in their response to the first single question. The interview was closed by the interviewer saying, “That’s all my questions. Thanks very much. Is there anything else you would like to tell me before we end the session? PAUSE. What else are you doing for the rest of the day?”

22

The interviews were transcribed, read and analysed using thematic content analysis (Field and Morse 1985, Patton,1990). The underpinning framework used to codify the emerging themes was Jonassen’s rubric (2001) as presented on page 17. These themes were validated independently by the project manager and then confirmed with other members of the project team.

7.0 Findings

Results from the group evaluation are presented using descriptive statistics. Cohort One evaluated the module separately from Cohort Two.

7.1 Student perspective

This section provides results of the standard module evaluation and e-learning questionnaire and qualitative comments from observation of the evaluation process.

7.1.1 Standard student evaluation form

A large amount of data were available from the standard module evaluation data2. Taking up the advice from Cousin (2007) to think with the data, eight statements pertaining to assessment, student motivation and ICT were selected from the standard student evaluation. The results are presented in Figure 7.1 Levels of satisfaction with how the module supported learning.

With the exception of responses to the statement ‘there was clarity of information about assessment criteria’ for Cohort 1, in general the students were satisfied with the module. Of the seven statements, the response of two thirds of cohorts 1 and 2 were satisfied to ‘where it was used IT helped me learn’, is below that of the percentage satisfaction level for the other statements. For each of the statements Cohort 2 is considerably more satisfied because the proportion ‘very satisfied’ is much higher for cohort 2 (note this difference is not illustrated in Figure 7.1).

The difference in satisfaction levels between Cohorts 1 and 2 is both large and consistent across all the statements. We went back to the teachers to work with them and reflect on the data to consider why such a consistently large difference might have occurred (Cousin 2007). In the conversations, what emerged was that during the week between C1 and C2 starting, the teachers met and reflected on their teaching experience with C1 and made planned changes to the teaching approaches for implementation with C 2 including how ULearn was introduced. The alterations in the teaching and learning strategies in the delivery for C 2 are presented in table 7.1.

Table 2 Alterations to teaching and learning strategies for C2

2 The evaluation team are aware of the difficulties of using student satisfaction measures as a basis for quality enhancement of learning and teaching. The standard module evaluation form asks students to report their satisfaction.

23

Day 1: 15 minute presentation about ULearn prior to logging on session utlised buddy approach to classroom activities Library session compulsoryHopes and fears ice-breaker sheets collated and posted on ULearnStatus passage reading taken home

Day 2: ULearn refresher session held at 4pm and not lunchtimeDay 5: Posted hopes and fears revisited within evaluation

The design of the evaluation is not robust enough to make the claim that such changes explain the differences in cohort satisfaction levels but they can suggest explanations. It is possible that the methods and techniques used to support explanation for understanding from the student perspective enabled C 2 students to engage in meaningful learning and a process of meta-cognition which connected and extended their current schemata of understanding (Eraut 2000). The interpretation has some support from the field observational feedback data of C 2.

One student said: ”Professionally as well as personally, I think I’ve developed”

Another stated that the module had been “empowering”’

There were varied cohort responses to the statement; ‘there was clarity of information about assessment criteria’. 48% of C1 was satisfied compared to 100% of C2.

The findings from the e-learning survey are used to explore in more depth the response to a) ‘where it was used IT helped me learn’ of the standard module evaluation form and b) the student experience of e-tivities as part of a managed learning process for the purpose of supporting the students’ approach to meta-cognition.

24

Figure 3 Levels of satisfaction with how module supported learning (selected standard module evaluation form questions).

82

79

63

48

88

90

86

67

71

85

96

67

71

100

91

95

100

100

100

98

81

0 10 20 30 40 50 60 70 80 90 100

learning helped develop analytical and problem solving skills

module stimulated enthusiasm for further learning

module has improved confidence in tackling unfamiliar problems

where it was used I.T. helped me to learn

the number & nature of assessments (with submission dates) was madeclear to me

the assessment/s related to the module outcomes

there was clarity of information about assessment criteria

all

c2

c1

25

7.1.2 E-learning questionnaire

ULearn usage

The e-learning questionnaire was administered at the end of the module, after the students had been working with ULearn for 11 weeks. At the end of the module, 29% of C1 and C2 respondents rated their expertise with using computers in their studies as high or very high; just under half (45%) describe their expertise as ‘OK’. Confidence in C2 appeared to be higher than in C1; 20% of C2 rated their expertise ‘very high’ compared to 4% of C1; and 11% of C1 rate their expertise as very low compared to 0% of C2 (see table 7.2).

Figure 4 Expertise in using computers in your studies

11%

22%

44%

19%

4%

0%

20%

45%

15%

20%

6%

21%

45%

17%

12%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Very low

Low

OK

High

Very high average c1+c2

C2

C1

The learner constituency of the mentorship module live off campus and are full-time working professionals who are released by their employers to attend the university on a part-time study basis. Students had internet access through a range of computers, but work and home were the most predominant location for accessing ULearn. Both C1 (83%) and C2 (85%) experienced technical problems using ULearn and the open feedback responses on how to improve the students experience indicates technical resolution is likely to make a difference to student engagement with e-learning.

“Address ULearn technical issues regarding downloading information and parking assignments”

“Stop it crashing / freezing every 5 minutes”

The difference in cohort experience emerges in identifying and getting help to deal with the technical problems. Across the questions concerning help and support C2 consistently indicates a greater level of negative perceptions with the help, support and guidance offered compared to C1. Overall C1 found ULearn easier to use than C2.

C1 showed greater frequency of use of ULearn with 61% of C1 using ULearn once a week or more compared to 53% of C2. Although C1 are logging on more often, a much higher proportion of C2 are using ULearn for 5+ hours per week during the

26

module (21% compared to 8% in C1). C2 are having fewer but longer sessions on ULearn. The open questions on the E-learning questionnaire shows that the use of ULearn does realise some benefits.

` "Made me feel more comfortable using a PC than I was previously”

“Challenged me to learn something new”

The self managed learner

Response data from four questions from the e-learning questionnaire were used as an indicator of student management of their learning. The questions and responses are presented in the following table:

Table 3 Self managed learning.

Attitudes to statement 'Using Ulearn helped me manage my studies for this module better'

7

15

33

33

11

10

35

25

25

5

0 5 10 15 20 25 30 35 40 45 50

strongly agree

agree

not sure

disagree

strongly disagreeCohort 2

Cohort 1

Although C2 appear to hold more negative perceptions of technical support for ULearn, they hold more positive perceptions of ULearn than C1 in helping them manage their studies and in particular a perception that they would like to use ULearn for all their modules. It is possible that with C2 having a greater level of expertise in using ULearn for their studies than C1, C2 has more awareness of how ULearn can support their learning intentions. That said, both cohorts are similar in distribution of their perceptions that using ULearn has helped them understand the module. So it is not simply a matter of understanding the module content, there also appears to be a student orientation to e-learning technology emerging. From the field observation of developmental negative feedback data, C 1 predominantly identify technical and student feedback issues, even although they perceived they got the help and guidance they needed with technical difficulties more than C2.

27

Cohort one developmental negative comments

Need a prerequisite level of computer skills prior to coming on the course3

Felt unsupported if not computer literate, when accessing remotely

Can feel discouraged due to emphasis on IT

Teething problems with U-Learn eg ‘parking’ the assignments

Problems with downloading because home computer didn’t have Java

Hidden costs of going to an internet café and printing

More time needed to prepare for the presentations

Assignment guidelines need to be given earlier eg Day 2

Difficulty accessing a traditional tutorial

Prefer individual to generic feedback

Although C1 held more positive perceptions of gaining help and guidance when needed than C2, lack of preparedness for ULearn seemed to predominate in their perceptions of control of their learning. This may also explain why C2 were more highly satisfied in the standard module evaluation form data given the different approach to preparing the cohorts for ULearn and C1 and C2 perceptions on clarity of the information on assessment criteria. From the observational data both cohorts were of the view that ”Everything could/can be submitted on U-Learn”’ including being able to ”submit final work online”.

E-mediated discussion

In response to the question ‘I would be able to study equally well on this module if I had never logged into ULearn’ 65% of C2 agreed compared to 34% of C1. In similar vein only 25% of C2 disagreed with the statement compared to 41% of C1. The responses to the question ‘I felt comfortable using the ULearn discussions to learn on this module’ showed 58% of C1 agreed compared to 42% of C2. The data illuminate differences in how cohorts felt about studying using ULearn discussions. More marked is the difference in perception between C1 and C2 on the extent to which course materials that were available on ULearn helped them learn on the module, 79% of C1 agreed with the statement compared to 55% of C2. It is possible that the lack of help and guidance when needed by C2 is linked with their perceptions of the course materials including e-discussions. What is consistent across cohorts is the level of agreement on the lecturer feedback. Whilst discussion may have a place what makes the difference is the feedback from the teacher and teacher support in the discussions. The following quote is from a student’s open response e-learning questionnaire:

“Discussion forum was great. Enjoyed linking with peers and tutors”

3 It is interesting to note that at least three people pulled out because they felt their computer skills were inadequate

28

It is not e-learning technology per se that facilitates a self managed learner, rather how the overall learning experience is designed, constructed and experienced and e-learning located within that. The positive comments and also possibly the negative comments from the field observation data are supportive of such an interpretation table 7.3.

Table 4 Comments from observation of the module evaluation Cohort 1 positive comments Cohort 2 positive commentsWe had enough support from the personal tutors-with a prompt response

U-Learn was very good

Help was available for assignments Useful to split module into three sectionsEnjoyed group support Balance of days was good-liked the

patternStructure of the course was good Relevance of teaching sessions to written

workCohort 1 negative comments Cohort 2 negative commentsIt was “slow torture” to hear flexibility of writing assignments

“It’s exhausting…I had to go on the computer at midnight” when sharing access with children in the family

The observational data below from the closing circle of Cohort 2 shows the context to which e-learning on the module both contributes and creates.

Empowering

Relevant

Really friendly

Useful to students

“It changed my way of thinking towards students. I now look at everything in a different light”

“It made me more aware of mentorship-and the whole journey of learning”

“It made me reflect more on my practice and be a good role model…I didn’t realise how important it was until this course”

In summary, the findings from these data sources demonstrate how essential student support was, in terms of how the learning experience is constructed and timely access to teacher and technical input. The design of the learning activities helped them manage their learning rather than necessarily ULearn per se. E-learning is an element that offered a possibility of peer supported learning when not at the University. Whilst the cohorts’ experience is variable, what is generalisable is the power of carefully designed and executed learning activities for meaningful student learning.

7.2 Teaching staff perspective

29

A convenience sample of five (out of seven) tutors was invited to participate in a semi-structured interview. Questions centred on adaptation of the individual tutors to delivery of the mentor module using a personalised e-portfolio approach.

It is significant that some tutors were themselves using an electronic interface and U-Learn for the first time in their teaching practice. This created some anxiety as the tutors were expected to move away from the traditional structure of paper portfolios. Four themes were generated from the thematic content analysis of the six interviews. Embracing the virtual culture; Developing digital communities of learning; The organic nature of e-portfolio building; and Being an ‘invisible presence’. These are presented using interview extracts to reveal some of the realities of promoting a reflective platform for students using electronic activities blended with face to face, traditional learning and teaching approaches.

(i) Embracing the virtual culture

The following quote emerged in response to question three, regarding tutors’ feelings about which evaluation strategies would be most appropriate to capture students’ views:

“The nominal group technique is the first opportunity to be open and honest about engaging with the technology (there were people who didn’t log on at all)-and I need to be one step ahead of the students!” (MPT2)

Another tutor visualised embracing the virtual culture for both mentor preparation and ongoing mentor development:

“We need to change the mentor update culture, therefore we need to know about e-portfolios to make a web-page for them. Once we’ve broken the habit we’ll be able to put on mentor development days around a wider scope. For example, we could put activities on the web and look at inter-professional learning. They could attend a day then mentors could write a reflection. This could have potential for the KSF (Knowledge and Skills Framework)” (MTD)

However, one tutor raised concerns about embracing the virtual culture, questioning the student experience:

“The new style is a good idea but what value do the students place on it? And are the students ready for it? (MPT3)

There appears to be some hesitancy concerning whether the virtual environment was, in fact, embracing students as equal stakeholders.

“Having designed activities and tutored these students on-line, I feel we’ve got more tools in the toolbox to deal with more complex issues” (MPT2)

To prepare them for supporting the students and teaching using electronic methods, four tutors in the sample had completed, or were in the process of completing an e-tutoring course. These tutors provided rich information related to how the format of the course changed their views and approach to working within a virtual culture:

“The [e-tutoring] course was very interesting…It’s all virtual, on-line. All the emails and ‘posts’ were open…It helped me build my portfolio…I understand what the

30

students are going through to prepare them-and how working with a group on-line is managed. You only have a week to do a presentation and you’re only given a title. As for who’s going to do what etc, it’s all on-line” (MPT1)

In the above quote, the respondent articulates the benefits of openness of email postings to inform her own electronic portfolio. Barriers appeared to occur through having no face to face contact with team members prior to engaging in group work using team activities. Another tutor who had undertaken the e-tutoring course actually indicated that on-line ice breakers were enough to enable contribution to on-line group discussions:

“Ice-breakers in class and on-line gave time to get to know the group…It helped with summarising ‘posts’. I can go on-line and pick up the discussion threads…I learnt for myself… ”(MPT2)

The following quote highlights the necessity for clarity of learning outcomes if presented on-line:

“When we set up, we’ve got to be mindful of embracing everybody’s learning styles and learning needs. Also, [the e-tutoring course] wasn’t very organised because of lack of on-line outcomes. There were too many links so we wasted loads of time. I’d have liked them to be more prescriptive in week one” (MPT4)

The same tutor described the ‘disembodied’ nature of communicating on-line without having an indication of who they were communicating with, or even which gender!

“ I hated the course. I got so frustrated!...I didn’t know who I was talking to. With any e-learning they have to see who they’re talking to…I was talking to disembodied names. Some of the ‘posts’ were too long and were just tirades. Some were posing” (MPT4)

The issue of feeling that the person on-line was ‘posing’ and using long posts and messaging seemed to mitigate against meaningful dialogues occurring. The use of the word ‘tirade’ indicates that the messaging may have been antagonistic as well as lengthy.

(ii) Developing digital communities of learning

The tutors sometimes revealed how inspired they were by students’ digital capabilities:

“It’s much more about lifelong learning rather than the single end of the course…These students will be much more digitally adept than us…some have their own blogs and are even developing their own communities of practice” (MPT2)

It was recognised by one tutor, however, that accessibility was not always universal:

“..Blogs could be used but there are access issues” (MPT3)

Having an individual space was seen as progressive, in combination with on-line discussion groups:

“…it’s about making a contribution to the body of knowledge and producing something to say: ’This is who I am’. It’s also about the student being more collaborative on-line” (MPT2)

31

Some concern was expressed regarding anonymity and being ‘faceless’. It is interesting that the following quote is from the tutor who spoke of frustration at communicating with ‘disembodied’ names. This had evidently influenced her teaching style:

“…With team teaching it’s anonymous. Things will change now and we can nurture the cohort. I’ve got a good knowledge of the cohort now…They know who they’re talking to and are therefore more ready to share” (MPT4)

The presenting of group feedback by tutors had initially been set up as group presentations had been given. However, this was not always individual enough:

“We give group feedback but students find it difficult to decide what’s pertinent to them. Individual feedback is probably better” (MPT3)

The use of an electronic approach to encouraging student reflection within a digital community was seen by one tutor to have added potential to widen participation of students:

“This has been a learning curve for me. I’ll have to be quite pro-active. But it’s something I believe in. One day we may see students by video link only. I am all for embracing the concept. It allows for widening participation” (MPT4)

(iii) The organic nature of e-portfolio building

An important element in supporting students for the mentoring role involved encouraging students to engage in ‘live’ discussions and reflections on teaching and assessing in practice. The following quote provides an excellent vision from a tutor of the ‘movable’ and organic nature of learning using the electronic medium:

“I don’t see the portfolio as static but a living document. An experience at work builds into the portfolio, eg using an image, a picture or a photo, or a flow diagram depicting a train of thought to express the journey…I don’t want it to be linear, but movable, like a jigsaw, with an elusive piece at the end; a growing thing, like a plant. You can prune it. You can choose whether you water it or not. Learning expands and contracts” (MPT4)

Several tutors intimated that the virtual environment should enable self-expression of the students:

“A sustainable evaluation needs to have a focus on the self. You need to capture the feelings and barriers” (MPD)

The inclusion of both professional and personal aspects was viewed as positive:

“I[Portfolio-building] means creating an evidence base of what you’ve done and how you’ve done it-for academic life or personal life-or both” (MPT3)

Importantly, the development of the portfolio using U-Learn was seen as a useful tool by one tutor, to enable learners to visibly see their learning journey (Williams 2000):

“The e-portfolio is an authentic record of their learning, so they can see their journey-and a way of connecting experiences” (MPT2)

(iv) Being an ‘invisible presence’

32

One tutor alluded to the invisible work of checking students’ on-line engagement and contributions, or ‘lurking’ as it is known in the virtual world:

“There needs to be a culture change in thinking about ‘contact hours’. Your teaching, not administration will need to be identified. For example, care is needed with ‘lurking’. And tutors may ‘look in’ on groups they’re not allocated to” (MPT1)

The quote highlights the possible tension with guiding students without spying or observing interactions without the students’ explicit knowledge of this. There was also subtle concern that the electronic format provided access to the whole teaching team.

Interview with E-Learning Unit stakeholder:

The respondent from the E-Learning Unit at the University was asked primarily about e-learning initiatives across the University, about how students were assisted to manage their learning, and also how it was envisaged that tutors should be prepared to teach using the VLE.

(i) Preparing the tutors

The respondent provided useful insights into the benefits of tutors undertaking an e-tutoring course prior to teaching using a blended approach:

“One of the most powerful things is if tutors have studied on line themselves first. It’s powerful to understand what it’s like to be a student in the on-line environment. They can have a ‘shift’ in their head from understanding the issues. It also increases confidence”. (ELU1)

This ‘shift’ in the tutors’ heads described by the E-Learning respondent is congruent with the paradigm shift described by some of the tutors themselves.

(ii) Capturing individual student experiences

The respondent suggested that the most effective method for capturing experiences of e-learners was through use of qualitative methods and through use of voting electronically:

“There needs to be more talking to learners and one to one contact. With questionnaires, the qualitative part can be illuminating. There is now the power of e-voting as well… Straight away, they could see what everyone else thought” (ELU1)

The respondent indicated that even in questionnaires, the qualitative section was often the most revealing. This finding is relevant to literature with a focus on ‘student voice’. To fully capture the student experience, the respondent stressed the centrality of ‘individual feelings’ in the ELU, saying:

“We deal with the human side” (ELU1)

(iii) Provision of technical support

When asked about the more technical aspects for students of managing their learning through U-Learn, the respondent alluded to the importance of the interface

33

between IT services and the helpdesk. It was suggested that problems related to browsers and logging in had reduced through simplifying the technology.

It was felt to be important to ensure that learners received on-line help within 24 hours. Interestingly, a common problem for learners was reported to be management of time on-line:

“On-line learning still means giving a structure. It may be based on weekly tasks. It’s not a free-for-all. What are you expecting students to do and in what timeframe? So students know what’s required of them…Students with families are often better at managing their time” (ELU1)

The respondent emphasised the need for a thorough induction for each student undertaking the mentor preparation module, suggesting that it could ‘make or break’ a student’s experience:

“Work was done on the induction for students. More cohorts are going through therefore there has been some refining as we’ve gone through. One hour in the lab can increase confidence and knowledge about help that’s available etc. Induction can make or break a student. The induction is crucial for students, particularly those who aren’t IT literate. So even if you have an hour with them in the lab, it’s about how you make sure they leave in that hour happy, confident, positive…knowing where they can get help…The tutors [module leaders] were very good…very reassuring. And that sets the tone” (ELU1)

The above quote emphasises the value of face to face contact at the outset in order to guage reactions of students if new to e-learning.

(iv) Managing the learning journey

The respondent emphasised the importance of the tutors being explicit and expectations for the module and also conveying all available options to students:

“It’s very useful for the tutor to be explicit about the avenues available. You need to include a page, early on, that says If you’ve got a problem you can post a message on the discussion board or email. We’ve had examples where we’ve set up discussion rooms and absolutely nothing has happened in them at all!”

The respondent went on to stress the importance of providing explicit instructions and support:

“Be explicit about what the rules and expectations are. You have to model the practice I think. Bearing in mind that it’s the first time that some students have been exposed to this type of learning, I think you have to be almost over-explicit, until they kind of learn the rules for themselves” (ELU1).

To help guide the students through a module delivered through blended learning, the nature of ‘tips’ necessary to manage the experience was offered:

“Time and structure are important It’s how you design the opportunities that are there. Everyone will slightly adapt them for their own needs. A range of techniques are needed eg make sure you log on frequently. Just short but frequent logging on. Generic tips like that” (ELU1)

It is evident that students need to be equipped with a ‘range’ of strategies to assist building of their portfolios and undertaking group work and projects on-line.

34

Summary

The interviews with the teachers provided interesting insights into their approaches to implementing and developing an electronic portfolio with no template. Several tutors described how they had ‘embraced’ the concept and had moved from traditional modes of teaching to working within an e-tutoring model. The data provide evidence that the project objective concerning raised staff understanding of e-portfolios as a way to engage students meaningfully was largely met. The enthusiasm with which all the tutors used the ULearn interface to promote deeper learning for students was commendable.

7.3 Learner mentor biographies

Four mentor learners from the two successful cohorts of 67 students volunteered to participate in the biographical narrative interview. The settings were varied and reflect the range of placements offered to students at the University of Surrey. They comprised:

A district nursing setting (Melanie) A community mental health setting (Aisha) A hospice setting (Ruth) A school nursing setting (Pat) 4

Using thematic content analysis five themes were generated from the interview data:

1 aligning expectations with reality2 circumventing technological challenges3 experience of learning within a blended approach4 strategies for learning and teaching5 views of a modernised module

These themes are presented in the table below alongside five of the constructivist learning categories from Jonassen’s (2001) rubric: intentional, contextual, collaborative, reflective and constructive.

4 All names have been changed to protect identity of the respondents.

35

Content analysis theme Constructivist learning category (after Jonassen 2001)

Aligning expectations with reality

Intentional: all human behaviour is goal directed. The goals are either simple or complex. To achieve their cognitive goals and meet the module objectives, the respondents needed to actively and intentionally align their expectations with reality.

Circumventing technological challenges

Contextual: Learning is more meaningful and transferable to new situations if it is situated authentically. For this reason, the respondents needed to identify any technological obstacles to enhance learning within new contexts.

Experience of learning within a blended approach

Collaborative: Learning occurs through social support and modelling in knowledge building communities. The experience of learning within a blended approach meant the respondents had to collaborate on learning projects through on-line cafes and discussion groups.

Strategies for learning Reflective: articulation of learning from a technology-based curriculum assists in a fuller understanding and better use of the knowledge the learners have constructed in new situations. Through describing their strategies for learning, the respondents were reflecting on processes and decisions which informed their learning.

Views of a modernised module

Constructive: To make meaning, learners need to integrate their prior knowledge with new ideas so they construct their own meaning from different phenomena

Aligning expectations with reality (Intentional)

As suggested by Jonassen (2001), respondents needed to consider their biographical learning histories in order to align expectations of the mentor preparation module with the reality of the actual programme and the challenges of undertaking new tasks. The cognitive goals (including undertaking a team assignment) and the intentionality involved, were displayed in various ways across this data set: Because Melanie’s training as an enrolled nurse had not involved teaching, she described feeling as if she had ‘missed the loop’.

36

I feel I missed the loop because enrolled nurse training in 1982 didn’t sort of encompass looking after students… I never really understood it through the eyes of the student nurse. ..So it was very sort of hit and miss (Melanie p6, part 2)

Melanie’s expectations, however, differed from reality as she was unaware that the module had been re-designed:But it wasn’t until we got here that the tutor said that they’d re-designed it and within the course I think they called it U-learning? And so I was a bit worried because I haven’t got huge computer skills and I was worried that I’d be struggling on it. So ur I was quite frightened about doing it really (Melanie BNI).

Aisha vocalised performing teaching on ‘auto pilot’ prior to undertaking the mentorship module:

…when you’re in practice for a long time without studying you get used to the way you do things and you don’t necessarily, you do a lot of it on auto pilot, especially with regards to your teaching. You had like a student shadow you and I don’t think I thought about it as much as I did when I did the course… (Aisha BNI)

Aisha also described her intentional, goal-directed approach as a participant constructing her own learning:

…but I guess it’s reinforcing the fact that you’ve got a lot to learn yourself, you’re always on a learning curve and I guess its important to kind of ensure you’re kind of directing that and not just kind of going along with day to day stuff…(Aisha)

Ruth was a little more critical of the pre-course computer literacy preparation and described how her goals had not met with the module objectives:

We did have a questionnaire about computer literacy and what we knew about computers. Um Mine’s basic. I can cope-but only just. So I’d have appreciated perhaps a bit more of that and might have gone on the computer more before I started. (Ruth p1)

The following respondent (Pat) highlights the importance of a mentor to help align expectations of all involved and to assist in realistic goal setting. This is to avoid feeling ‘thrown in’ to challenging teaching situations:

And I also feel that because of [tutors’] perceptions, I was thrown in at the deep end with the student and just left to get on with her and not have the experience of an SPT [Specialist Practice Teacher] really, close to hand, to help me through the course. I was just left to get on with it. (Pat P2)

The above quote was from Pat, who described immense difficulties she had experienced when mentoring one student in practice. Pat had evidently expected far more hands-on support and therefore expressed disappointment.

Circumventing technological challenges (Contextual)

The nature of the technology-led emphasis in the new module meant that the learners needed access to software such as ‘Java’ and compatible hardware. This was not always easily obtainable. Melanie described her anxiety related to work getting ‘lost’ in the ether:

37

…I couldn’t download the Java so I couldn’t use it at work. I had to use it at (pause) home. So I had to ask my son to sort it out for me, to make my computer compatible to the University’s. So that was all a bit anxious for me but we managed to sort it out…And that was another thing because the Java wasn’t acceptable in the format that I had on my computer… [I was] worried that work had got lost in some sort of black hole (Melanie BNI3)

Technical support from the E-Learning Centre was evidently available and useful:

But as time went on-and um there was quite a lot of support from the tutors and one of the chaps that designed the um U-learning sort of computer system, I felt a lot more happy about it. (Melanie BNI1)

Other respondents also described challenges of ‘parking’ of assignments on-line.

I got a bit confused with parking assignments and retrieving them. And actually there was a fault with that because we were able to get everybody’s essays and things at one point. I think that was then rectified. Because I couldn’t park mine, my modules, um essays. It just wouldn’t park. And I found that really difficult. not a positive thing (.) from it. (Ruth)

The contextual elements were intrinsic to these students’ learning. As found in the literature, the asynchronous nature of communicating on-line was not always conducive to vibrant discussion within internet cafes:

The things about the U-learn, that has provoked a lot of agitation I think, a lot of anxiety in the beginning especially… I enjoyed that side of it, but I think it’s about anxiety if people couldn’t get onto a computer and that had an impact on our group work. So several times there was only me actually on it doing some work, myself and one other person. Pat P3

Greenberg (2004) suggests that, to avoid isolation (perceived by three out of four students) network communication tools need to be both synchronous and asynchronous. The virtual learning environment at the University proved to be a double edged sword. Pat describes the value of the VLE for group work and individual searching, but not assessment:

I found U-learn very helpful for the initial things that we did in the teaching. For instance, in our group work we chose motivation and it was very, very useful. Things were put on the U-learn site for us to go in or we did our own searches on the internet and that was very, very useful…When it came to assessment though, I don’t remember U-learn being very helpful at all really. There didn’t seem to be anything on there that was particularly helpful,… (Pat p10, part 2).

As described earlier, Pat had been influenced by a negative incident with a student and therefore seemed to view assessment as a barrier to learning using technology.

Experience of learning within a blended approach (Collaborative)

As found with the e-portfolio system used at Wolverhampton University (‘Pebble Pad’), the on-line community used the digital format as an electronic reflection and communication tool. The following quotes provide information about the actual experience of blended learning, which is rather sparse across the literature :

38

It wasn’t just about computer systems. We had to come in and have some taught sessions which I thought were really good and really up to date. And I thought there was plenty of time to get the work in. And I thought with the people on the course there was a really good mix of people. You got ..rather than having just one type of nurse you know from one area. So there was a really good mixture which kind of enriched the team.(Melanie)

Melanie provides information about how she managed her time on-line:

Because we had our little group given to us. And we all agreed to go um on-line at 8 o’clock in the evening and have a chat, like a web chat about what we were going to do this presentation on and I sat there-8 o’clock, went on line and there was nobody there ( Melanie part 2)

The above quote provides evidence of the challenges for some mature students in undertaking group work asynchronously. Aisha appeared to work in a group with a different approach to on-line discussions:

…the e-learning, I think we could have made more use of it, like my group made, because …you could discuss your group work and we utilised it quite well but that was because most of the group had access to it so people checked on it quite a lot…(Aisha)

Ruth described some of the barriers and frustrations inherent in attempting to communicate on-line:

The other thing as regards using U-Learn. You were put into groups to do a presentation. I found that extremely difficult a) because I was out of the area and b) because no-one was on line the same time as I was… And I think I sent about 30 emails and didn’t get any replies for a long time. So trying to formulate a presentation with the rest of the group was difficult…Working shifts isn’t conducive to being in contact at a certain time of day, really (Ruth).

What appeared to compensate for this, was the immediacy of tutor support from the University:

Tutors did respond to emails. I was surprised that someone could respond that quickly. Almost spontaneously really. They seemed to be on line all different times of day and night. Which is quite reassuring really. And gave you know, positive. You know, that feedback. And helpful. Answered the questions really that were asked. That was good. Yeh…I think (.) the staff were very supportive (Ruth).

Pat, who was probably more used to presenting independently in the community, described how positively her group ‘gelled’: The teaching side I really enjoyed. I enjoyed all of the group work that we did and we had to get into groups and do a presentation and I think I enjoyed that, probably because in my role I do do lots of presentations anyway…I think also in our group we were quite lucky, although it was randomly, we seemed to gel really easily and seemed to get on very well, so that was good (Pat)

Strategies for learning and teaching (Reflective)

As found in the literature (particularly that related to views of trainee teachers), quite intense preparation was required prior to students using technology for the first time

39

(Woodward and Nanlohy 2002, Scholes 2004, Butler 2006). Tensions with attempting to ‘cope’ with new technologies are evident in the following quote:

I’ve got to move on with the technology because I can’t communicate and move my knowledge forward if I don’t start using the new technology. So I have this sort of push me, pull me feeling about how I cope. Melanie, P2

Ruth described how the face to face teaching compensated for her lack of technological confidence:

The actual lectures and things were excellent. I felt really supported (.) They were taught really well I felt. Um. There was a lot of humour involved. And that’s how I learn…Um. Certainly the hand-outs and the reading lists that we got were excellent. And I gained a lot from that. I think more role play in the module would be good. (Ruth p2)

Ruth went on to share how confidence in technology to assist formal presentations had increased, adding that the blended approach used at the University had triggered using new approaches to assess students in practice:

So I’ve developed in the fact I can use computers better, despite all the hiccups. Y’know I could do a Powerpoint and even get the bits coming in from the side and all the other bits. So that’s been of value and I can use that in my teaching…And we’ve altered our approach to students to their evaluation…I’m more of a practical teacher I think, rather than a theoretical one…But I’ve certainly learnt the the research and the knowledge behind it now. I think that’s the value. (Ruth, part 2)

Aisha described how reflection was informing her own teaching style:

[I] selected the information according to what [the student is] more likely to come across rather than something that’s kind of common, so it would be useful to her… I structured it in relation to [the mental state examination] to make it more meaningful to her… (Aisha p2, part 1)

The individuality of the students was being encompassed to provide a relevant practicum for the student and to also influence her divergent thinking:

…all students are different and they’ll bring something and make you think in a different way…(Aisha P8, part 2)

Pat described how the module had influenced her approaches to the student at the outset of the mentoring relationship:

So I think it’s taught me about preparation, how to start off a programme in the beginning, how to approach your student, make her feel comfortable, make her feel part of the team, make her feel valued, those type of things really (Pat P18)

The following quote demonstrates some empathy for the students on their journey or ‘passage’:

It does make you feel about how it is to be a student as well, that sort of passage into the profession; and it does help you think about the objectivity of the assessment process as well, and also there’s no ideal assessment, I don’t think, anyway.(Pat, p21, part 2)

40

Views of a modernised module (Constructivist)

As suggested by Jonassen (2001) students need to combine their prior knowledge with new ideas, in order to construct meaning for future learning. The following quotes show some the ways the respondents in this sample constructed both meaning and new knowledge following completion of the module:But I think it was a positive, um not transformation of the course but a sort of modernisation of the course probably. Because I think that every nurse has got to be a mentor. If you’re still working with old paperwork and not moving forward..then we’re not going to keep up with up to date practice…And I think the way forward is using the internet and using the computer…to enhance your knowledge…And it’s much more cost effective..having it on the computer based than it is having it in books and journals and things stored in a library that take up a lot of room.(Melanie)

The benefits to mentors and students of a blended learning approach are suggested by Melanie in the following quote:

And I’ve had two students out with me since and I feel that I’ve known exactly what they’re supposed to be achieving …in their training. And um I feel much closer to the University if I had a problem with them… I think the student nurses have actually got quite a good deal really. They’re a lot more supported than they were a few years ago. (Melanie)

Aisha describes how the e-portfolio had become a tool to help her build significance and extract meaning from her individual reflections:

…what I’m doing towards my portfolio is, there’s more volume of it, as in I’m doing it more often than I did …there’s no point doing it for the sake of it but it has to be related to something kind of significant and meaningful. But certainly got me back into doing them.(Aisha)

She also relayed how making learning relevant and meaningful for the learner had become important in her teaching and mentoring:

..it’s making it meaningful, making it relevant and that the learner is more likely to take on that information if it is meaningful, that’s where that’s based in the theory that was taught (Aisha).

Ruth suggests that mentoring ‘officially’ means providing a package ‘properly’ for the students, combining theoretical principles and practical teaching strategies, gleaned both on-line and through face to face teaching and discussions:

The teaching methods that we learnt (.) have helped me sort of when I’m now back at work um and started to mentor officially if you like not in a supportive way but um do it properly as you should, with the students that we have here. Plus with all the staff so hopefully it will be beneficial for the future.(Ruth p2)

41

SummaryThe selected data presented here provide insights into how a blended learning approach was applied within authentic contexts of learning for four mentor trainees who undertook the module between September 2006 and March 2007. The biographical narrative interview method enabled contextual detail to be shared which may otherwise have remained hidden. The mini-narratives provided rich data which demonstrate the challenges and successes in portfolio-building using a virtual learning environment (VLE). Key findings from in-depth interviews with mentor trainees were:

The importance of technical preparation for the programme prior to using U-Learn;

The need for continuing support of teaching in practice, to avoid being ‘thrown in at the deep end with the student’;

The valuing of tutors engaging with students individually and promptly via U-Learn;

The need for web-based discussions to be planned and purposeful; A prompt to use the personalised interface for continued reflective writing and

finally; The personal and professional growth that was verbalised through the linking

of theory and practice using a virtual space.

It was a significant finding that all respondents described how the module helped shape their thinking ‘in a different way’ and helped to change their ‘approach’ to students, thereby making the learning experience more ‘meaningful’ and ‘relevant’ for the learner. The mentors were both learners and teachers when they undertook this module, which puts them in a unique position. These findings therefore contribute to current knowledge about use of constructivist learning environments to enhance the practice of teaching. The following section is a discussion which brings the various data sources from this project together.

42

8.0 DiscussionAs stated in the Background section of this report, the Mentorship Module provides a large proportion of CPD in the Division of health and Social Care (formerly the EIHMS) training approximately 400 qualified health care professionals each year to become mentors in practice. The NMC standards for mentors practice teachers and teachers (2006) place increasing responsibilities on mentors to sign off student nurses and student midwives as competent and safe to practise and qualify on the professional register. How the mentor preparation module is delivered is therefore of key importance. How the learners managed their learning using the University VLE was obviously highly individualised and the results are not intended to be generalised. However, there were some common themes and patterns which emerged across the data set.

The blended approach was found to enhance learning by some students. However, technological ‘hitches’ constrained some students, who were not familiar with web-language and who did not even own their own personal computer prior to the module commencing (for example, Melanie). It should be noted that some students attending Day One of the module had never sent an email prior to registering for the module. This is relevant with regards to Laurillard’s (2005) views about technology interrupting the actual learning process and the importance of tutors’ awareness of student’s prior experience with technology (Curry 2008). Other students (for example, Aisha) worked within a mental health therapeutic model and showed skills in constructing meaning from a more inter-actionist approach during discussions on-line and ‘postings’ throughout the module.

For students in cohorts one and two, who undertook the new Mentorship module, they were able to experience the synergy of collaborative electronic learning and many engaged enthusiastically. However, it was assumed that all students had computers at home (that were readily accessible to them). Additionally it was presumed by teaching and support staff that all students were able to access Java, both in the workplace and at home. The in-depth data from four biographical interviews indicate that, if working in community settings, ownership of this technological equipment and programmes was not uniform.

It is interesting that the induction was seen by the e-learning unit as a significant component to supporting students at the outset of the mentor preparation module, yet this was not mentioned by the tutors. There is evidence that the e-Learning unit, in collaborating with IT services, provide an umbrella of support for both students and tutors. Equal emphasis appears to have been placed on technological and ‘human’ or psychological aspects of electronic learning. The data from this study provides evidence of the need for sustained support for students throughout a module with extra provision in the early stages.

Deepening of student engagement in their learning was described by some students but this took an enormous commitment from both the teaching team and E-Learning support team. The two cohort leaders were very enthusiastic and evidently committed to supporting the students. This almost led to ‘lurking’ as it became difficult to be ‘hands off’ with such varied learning needs in each group. The e-tutoring course was not popular with all tutors but did appear to prepare the tutors for the realities of communicating to an unknown audience and feeling, at times, ‘disembodied’ during on-line discussions and chat-rooms. One of the project’s objectives was to: Increase staff understanding and capacity of e-portfolios as a way to engage students in a meaningful learning experience. The evidence from interviewing five tutors shows that this objective was certainly met. The introduction of U-Learn was seen as positive within months because tutors recognised that it

43

opened up ‘live’ channels of communication for direct engagement with the students. There is a possibility, however, that some students become overly dependent on the tutors’ virtual presence as a comfort blanket.

As asserted by Roberts (2007), there has been substantial technologising of language in the last decade. Whilst progressive in many respects, the data indicate that this can be constraining for some students new to this style of learning. This may be particularly challenging to mature learners in the health care professions. Laurillard (2007) asserts that tangible ‘flags’ are required to assist students. These could be in the form of on-line key words, reminders and links.

A common theme across the data from students collected for this project centred on learners feeling isolated. Whilst asynchronous networking and communicating is viewed as the panacea (Greenberg 1998), the lack of synchrony for discussions and group work led to feelings of isolation for a number of students. Strengthening of creative channels of communication using virtual media is evidently required, using an e-profile model, for example Gomez (2004). The e-profile system enabled students to be tracked and guided remotely. This showed benefits for student support on long placements and therefore has potential to be modified for use with students on professional placements at the University of Surrey. One challenge for mentors preparing for the role is that a range of models of mentoring operate across the NHS Trusts affiliated with the University (Finnerty et al, 2004). Additionally, the length of student placements varies considerably.

Although this disparity in placement length and clinical contexts exist, this evaluation provides evidence of the potential for University –wide e-portfolio model to be crafted, to promote meaningful, reflective learning and conversations within a personalised virtual space. The literature confirms the possibilities that can be realised to promote an enhanced learning environment for both the practice of teaching and lifelong learning. Examples from the literature and empirical data are:

Promotion of collaborative learning

Provision of new spaces for reflection and feedback

A space to build a longitudinal record of achievement (including mentor passports and records of teaching incidents)

A display of authentic (performance) assessments in the field (Mullen et al. 2002) and an exhibition of significant learning and teaching episodes through multi-media, for example, audio, text, video, photos and images.

Opportunities for career planning

Despite ongoing, exciting advances in digital portfolios, pedagogical considerations are required. As it was not possible to take a portfolio ‘off the shelf’ in this instance, the mentor students had to construct their portfolio without a template. For the future, this means either considering a rubric such as that created by Jonassen (2001) or using formalised, specific sections in the e-portfolio for journaling, submitting assignments and collaborating on line, as advocated by Lawson et al. (2004) in their e-portfolio for health professionals.

44

8.1 Reflection on the evaluation methods

Although the nominal group technique is commonly used to evaluate modules and programmes, tensions appeared to occur through agreeing on group statements which did not entirely reflect individual experience. The ease with which students engaged with U-Learn seemed to be largely dependent on each student’s confidence and prior experience with e-technologies. It was therefore challenging to make general statements which were inclusive of the group’s views.

The e-voting component of the module evaluation was introduced in a fun ‘Who Wants to be a Millionaire?’ format. The immediacy of visual results was found to be attractive, as was the novelty value and the anonymity. Informally, the students described their pleasure at being able to see the whole group’s response to each question. It is important to note that all students engaged with this part of the module evaluation. This is in contrast to the nominal group voting in which only about one third of the students actively contributed.

The interviews with tutors were relatively short but very focused. The most interesting data emerged from those tutors who had completed the e-tutoring course. One cohort leader admitted that her negativity to the ‘faceless’ activities had led her to adopt a more ‘nurturing’ presence for her cohort. The passion displayed by both cohort leaders to make the new module a success was evident and appeared to have cascaded to the other tutors in the Mentorship module team.

The biographical interview method proved useful to tease out specific elements which were found to block or enhance the four respondents’ e-learning experiences. However, the actual process of interviewing in the workplace and transcribing lengthy interviews was extremely time-intensive. For example, it took the evaluator 22 hours to transcribe two taped interviews. The benefits were that the process informed the analysis of data and meant that constructivist characteristics emerging from Jonassen’s rubric (2001) could be applied.

45

9.0 Conclusions and recommendationsThe overall conclusion for the evaluation was that there was a need to build an e-portfolio of learning products meaningful to the student, customising teaching approaches, to include e-learning strategies so as to align the world of the student with the learning task at hand. The project has shown the value of using a participatory approach to design a personalised learning space for trainee health care mentors.

Conclusions are presented below, with corresponding recommendations in boxes:

The Student ExperienceMany students made positive comments about the new module, for example, suggesting that the module design enhanced both professional and personal development and was empowering.

Recommendations Continuation plans for use of U-Learn could be put in place for self-selecting

students. Provision of ‘Pebble Pad’ has potential to promote sustainable communities of practice on-line.

The Mentor Preparation module handbook and pre-course reading need to be revised in light of the varied stages and e-learning capabilities of students embarking on the module.

Guidelines need to be presented which are commensurate with students’ computer literacy levels. This should include a glossary of ‘digital’ terms, for example: Asynchronous discussion, ‘parking’, ‘lurking’ discussion ‘threads’, etc

Some students expressed the need for extra sessions in the computer laboratory to practice skills such as logging on and downloading or uploading information.

Recommendations A time management mini-workshop could be considered, to assist learners to

manage their time on-line and maximise their productivity and outputs. This would leave more time for reflective activities to enhance learning.

More opportunities for asynchronous discussions would benefit those practitioners who work shift patterns and live a distance from the University.

Preparation for parking and retrieving assignments needs to be supported with written, verbal and on-line information to prevent student anxiety and fear of work disappearing into a ‘black hole’ (Melanie).

Students indicated that individual feedback on their work was crucial to them as adult learners, training to teach in complex practice arenas. How feedback is presented is obviously of relevance to trainee mentors who will be influenced by methods which may enhance their own teaching practice.

46

Recommendations Provision of individual feedback to students needs to be built in by tutors as it

is evidently of far more value and relevance than generic feedback following group work.

Attention to feedback mechanisms is necessary so that there is equity in provision to students and also tutor workload. This needs to be inclusive of extrinsic and intrinsic features and must also be ‘impactful’ for the learner (Gibbs 2004)

Due to lack of familiarity with the technology involved in this approach to learning and assessment, students displayed varying levels of confidence with regards to on-line discussions.

Recommendations Virtual action learning groups could be established on Day Five of the

module. This would encourage students who felt their learning styles to be compatible to self-select into peer groups to take their ‘live’ discussions forward into practice.

Traditional module evaluation methods need to be replaced with valid and reliable tools that will help build upon a body of evidence to support pedagogical development for e-supported personalised learning.

The Tutor ExperienceThe e-tutoring course was found to be generally not an enjoyable experience but extremely valuable to provide insights into realities of teaching and learning within virtual learning environments. Feeling ‘disembodied’ was found to block discussion and learning by one tutor (MPT4).

Recommendations To enhance learning and optimise on-line communications, it is evident that a

balance with face-to-face communication is needed. ‘Hidden’ time for provision of informal student support needs to be built into

future timetables for tutors who use a blended learning approach.

The cohort leaders confessed to going the extra mile by attending to extra emails from students, troubleshooting and ‘lurking’ in their own time. Provision of student log-in numbers, for example, was found to be very time-consuming but was not registered as a teaching activity.

Recommendation Use of a blended approach to present such a module at least ten times per

year evidently involves hidden time, which needs to be valued and supported.

47

The lack of a template for the e-portfolio was found to create apprehension for some tutors initially. Although one cohort leader admitted to being a ‘good spin doctor’, the tutors appreciated being able to make cohort-dependent adjustments to the module quickly.

Recommendations A rubric could be adapted specifically for use of the Mentorship module Consideration needs to be given by programme developers who wish to

develop personalised learning through e-learning technologies as to how best to structure the student learning journey, inclusive of a route map.

Summary The deepening of learning through creation of a reflective virtual portfolio has potential to:

improve the learning environments in practice and

increase employability profiles for mentors

Jonassen’s rubric (2001) for assessing constructivist learning environments provided a successful underpinning model to analyse the data. It was important to tease out what contributed to or prevented meaningful learning from taking place. It is suggested this model has potential to provide a framework to underpin evaluative comments from all stakeholders:

a) Tutors would have a tried and test framework to construct e-learning programmes using a VLE. This is ideally suited to a blended approach because the constructivist rubric has also been used successfully to guide classroom discussions (Jonassen 2001, Williams, 2000).

b) Students could use the rubric as a reflective tool to aid construction of meaning from their learning.

c) The rubric has assisted assessment of portfolios, helping reduce subjectivity and enhance inter-rater reliability.

Finally, the evaluation data have shown some significant benefits for both student mentors and their tutors through co-designing a module using a virtual learning environment such as U-Learn. It is encouraging that tutors described having “more tools in the toolbox” (MPT 2) and several students described how their learning had become more “meaningful” and relevant to inform the practice of mentoring. An essential component to deepening the learning appears to be tailored support for students to develop an authentic on-line identity and subsequently a dynamic personalised learning space.

48

APPENDIX ONEDissemination of project findings

One paper has been presented in France, at the Blackboard World

Conference, February, 2007.

A poster was presented at the Festival of Teaching and Learning, University

of Surrey, 2008.

A series of articles has been planned, using a practitioner inquiry approach

National and international conferences will be attended and abstracts for

papers submitted

Within the University, there will be involvement in the E-Learning

Practitioners Network. A lunchtime seminar is being prepared. This will help

to maintain strong links with CLD and the E-Learning Centre.

A summary of the project will be provided for the ‘Supporting E-learning’

booklet.

49

REFERENCES

Ashcroft D.M and Hall J (2006) Pharmacy students’ attitudes and views about portfolio-based learning: A questionnaire survey. Pharmacy Education, 6 (1) pp 1-5.

Butler P (2006) A Review of the Literature on Portfolios and Electronic Portfolios. http://creative commons.org/licences/by-nc—sa/2.5/. Accessed June 2007.

Campbell MI and Schmidt KJ (2005) Polaris: An undergraduate online system that encourages personal reflection and career planning. International Journal of Engineering Education. 21 (5), pp931-942.

Cambridge B (2001) Electronic Portfolios: Emerging Practices in Student, Faculty and Institutional Learning. AAHE

Cambridge B (2004) Electronic Portfolios: Why Now? Educause Live, American Association for Higher Education Accessed June 2007.

Cayne J (1997) Portfolios: A Developmental Influence? In: Research into Practice; A Reader. P. Abbott and R. Sapsford (Eds). Open University Press, Buckingham & Philadelphia.

Cotterill SJ, McDonald AM, Drummond P (2004) Managed Environments for Portfolio-based Reflection. www.eportfolios.ac.uk/FDTL4/docs/. Accessed 07.04.2005

Darling L.F. (2001) Portfolio as practice: the narratives of emerging teachers. Teaching and Teacher Education 17 pp107-121.

Driessen E, van Tartwijk J, Overeem K (2005) Conditions for successful use of reflective portfolios in undergraduate medical education. Medical Education. 39 (12) pp1230-1235.

Gardner, H (1993) Multiple intelligences: The theory in practice. New York: Basic Books, p85.

Gardner, H (1987, May) Beyond I.Q.: Education and human development Harvard Educational Review 57 (2) pp187-193.

Garrett B, Jackson C (2006) A mobile clinical e-portfolio for nursing and medical students, using wireless personal digital assistants (PDAs) Nurse Education Today (In press).

Georgi D, Crowe J (1998) Digital Portfolios: A Confluence of Portfolio Assessment and Technology. Teacher Education Quarterly, Winter, pp 1-10.

Greenberg G (2004) Extending the Portfolio Model. Educause Review, July, pp 28-36.

Haig A (2007) NES e-portfolios, NHS Education for Scotland (Poster presentation at HEA Festival of Learning, April)

50

Hennessy S, Howes AM (2004) Using ePortfolios to Assess Reflective Capabilities of Medical Students. Paper presented at ‘e-portfolio conference in La Rochelle, 2004.

Jonassen D.H. (2001) Design of constructivist learning environments http://www.coe.missouri.edu/~jonassen/courses/CLE/index.html (Accessed 2005)

Kilpatrick B (2007) A review of literature concerning e-portfolio use and implementation Draft document, pp1-25.

Laurillard D. (2005) Rethinking University Teaching. A conversational framework for the effective use of learning technologies London and New York: Routledge Falmer.

Lorenzo G, Ittelson J (2005) An Overview of E-Portfolios. Educause Learning Initiative pp 10-21.

McKenzie JF, Cleary MJ, McKenzie BL (2002) E-Portfolios: Their Creation and Use by Pre-Service Health Educators. The International Electronic Journal of Health Education (4) pp 79-83.

Mason R, Pegler C, Weller M (2004) E-portfolios: an assessment tool for on-line courses. British Journal of Educational Technology. 35 (6) pp717-727.

Scholes J, Webb C, Gray M (2001) Issues and Innovations in Nursing Education: Making portfolios work in practice. Journal of Advanced Nursing 46 (6) pp595-687.

Webb C, Endacott R, Gray M (2002) Models of portfolios. Medical Education. 36 (10) pp897-898.

Williams M (2000) Discussion and Reflection Evaluation Rubric. Department of Character Education, School of Education, San Diego.

Woodward H, Nanlohy P (2002) Digital Portfolios; Fact of Fashion? www.aare.edu.au/02pap/woo02363.html. Accessed 07.04.2005

51