FOCUSON:TRAINING

Portfolios can assist reflective practice and guidelearningJ.D.Greaves and S.K.Gupta

The Royal Victoria Hospital,Newcastle uponTyne,UK

Summary Aportfolio is a collectionofmaterials chosenbya learner to demonstratethebreadth andqualityoftheircurrentwork, andtoreflectupon andplantheir furtherprogress. Each portfolio will be unique in consisting of an individuals selection of evi-dence.The portfolio is in itself an educational tool. It allows the doctor to understandtheir current practice and to share that understanding with others. In order to benef|tfromthe process of portfolio preparation and review the doctormust be open in theirreflections and enter wholeheartedly into the exercise.�c 2003 Elsevier Ltd.Allrights reserved.

KEYWORDSeducation---medical,continuing;education---medical,graduate; learning;educationalmeasurement;self-assessment; training;anaesthesia

WHATISAPORTFOLIO?Aportfolio is a collection ofmaterials collectedby a lear-ner that, in their judgement, exemplify the breadth andquality of their performance.1 Its importance as an edu-cational tool depends on the process of selection, whichis specif|c to each individual. A goodportfoliowill show abalance between evidence of activity, evidence of out-come and reflection.The portfolio assists the learner tounderstand their progress and can be interpreted to in-terested third parties who can thereby gain insight intothe learner’s understandings. A portfolio can be pre-paredby a doctor who is in training; or by a careergradedoctor, as an aid to continuing education and profes-sional development. In the f|rst instance the portfoliomay be sharedwith the doctor’s trainers; in the latter itmaybe benef|cial to share it with a peer as part of an ap-praisal, or with management. A good portfolio willcatch the spirit of the doctor’s attitudes and willreflect their enthusiasm, flair and desire and capacity forinnovation.2

Finding a focus for learning

Training to be a specialist in any branch of medicine re-quires a doctor to learn huge amounts of information.The detail is staggering, and is the sticking point for trai-neeswho abandon their specialist trainingwhen they failtheir professional exams.Most of us will remember thatlearning for exams brought us to the point of exhaustion

and was associated with stress and depression. Manyyears later we continue to dream that we are in the ex-amination hall.Most of these dreams are unpleasant! It isunsurprising that these experiences, and in particularthe rigour of tests of factual knowledge, lead to a parti-cular learning style.There is too much to learn, and onlyexceptional individuals have the sort ofmental apparatusthat can accumulate all the information. For most medi-cal students and doctors the critical decision is notwhatto learn, but what to leave out. We become skilled atchoosing, and the enterprise of learning is organized bythe demands of the examination, rather than the needsof clinical work.These should be congruent but in prac-tice they are not.Thepostgraduate learner is also facedwith the taskof

learning workplace skills and ‘know-how’.The agenda forlearning is again clear and is marked out by the presentdemands of practice.These demands aremany and insis-tent; once again the learner is presented with a curricu-lum and need give little thought to planning learning.For an established specialist the problem is different.

Inmost areas of practice they are expert.Their def|cien-cies stem from a variety of factors:

* New developments occur.* Acceptedpractice changes.* They forget things.* Theychange their work pattern andundertake differ-ent work.

To remain competent the doctor needs specif|c habitsof mind. Important amongst these are the abilities to

ARTICLE IN PRESS

Correspondence to: JDG.E-mail: david.greaves@ncl.ac.uk

Current Anaesthesia & Critical Care (2003) 14,173--177�c 2003 Elsevier Ltd. All rights reserved.doi:10.1016/S0953-7112(03)00107-8

perceive where there is a need for learning, to plan thatlearning, to reflect upon it and to accurately assess howwell they have learned.Learning is driven by the doctor’spersonal perception of his or her needs.Theremaybe no‘off the shelf’ solutions to these needs and theymay needto plan their learning. There will also not be any ready-made tests to help them check their learning.Keeping a portfolio ofpractice canhelp the consultant

to understand their learning and can teach this processto trainees. In short it encourages professional reflec-tion. Preparing, reflecting upon and sharing a portfolioencourages the skills needed for life-long learning.

What should go in the portfolio

The learner must choose what they put in their portfo-lio. Materials will demonstrate what they have done,what the outcomes were and how they felt and reactedto these. A template is often provided, as in the case ofthe portfolio used in the UK for the annual appraisal ofconsultants.3 Users should understand that they are notconstrainedby the format. A goodportfoliowill containreflection as well as facts.

* How does the doctor feel about specif|c outcomes,critical incidents, etc.?

* How did attendance at a meeting turn out?* What are the good things and bad things aboutcurrentwork?

* Areyou satisf|edwithboth theworkof colleagues andyour professional relationships with them?

THEUSEOFAPORTFOLIOThe portfolio can be used to plan personal medical edu-cation and continuing professional development (CPD).3

It is important to draw a distinctionbetween these.CPDis the professional growth of the doctor. A consultantappointment is not a pinnacle; it is a new start.The doc-tor will, with time, become an expert in a narrowingf|eld. They will teach that expertise. They may researchin that area.They willmanage, innovate and lead. Perso-nal medical education is directed towards maintainingand developing standards of practice. In part itwill relateto keeping up to date and practicing state-of-the-artmedicine. Some of it will be directed towards profes-sional development.The portfolio fulf|ls many functions simultaneously. It

is not simply a list like the logbook, since it neither re-cords all experiencesnor is it conf|ned to recordingof ex-periences. Portfolio preparation involves the collectionof experiences, selection from this collection and reflec-tion on the selected experiences. It is believed that theseprocesses are thought-provoking exercises and hence aportfolio is seen as a tool for supporting developmentin technical, professional and personal competencies.1

The preparation of a portfolio provides an opportunityfor self-assessment.This is valued educationally becauseit helps the doctor learn form their experiences.Reflection can be upon anything from technical knowl-edge, themanagementof resources and time to personalqualities. When a portfolio is intended as a learningtool it is notmandatory for the doctor to let anyone elsesee it.The highly individual nature of a portfolio means that

the doctor should be free to decide its structure.Although the experiences described in the portfolio areembedded in the context of training, reflection on thoseexperiences can be used for monitoring personal devel-opment rather than technical achievement. It might beargued that this approach would shift the focus from ac-quiring knowledge and skills to personal developmentbut the current belief is that personal and academic un-derstanding are closely bound upwith each other.5

Portfolio preparation andreviewmaybe incorporatedinto an assessment. Freidman considers that portfoliocreation is one of a number of innovations in assessmentthat attempt to move from knowledge and skill towardsa more complete understanding of clinical behaviour incontext.6Other such reform-drivenmeasures of perfor-mance include clinical audit and simulator training. TheGeneral medical Council (GMC) has used portfoliosboth as part of their review of doctors who have beenidentif|ed as possibly unf|t to practice, and through an-nual appraisal as a part of the process of revalidation.7

Logbook

Part of the portfolio can be a logbook.Repeated experi-ence is needed to achieve and maintain mastery. If theportfolio includes activity data itmay be possible to pre-dict areas of practice that arebecomingrusty.Many con-sultants will feel that keeping overall records of theiractivity is futile. They may prefer to keep count only ofprocedures where the workload is critical to perfor-mance. Inmany areas of practice there are no ‘rules’ say-ing that the doctor must undertake a minimum numberof procedures to be an acceptable performer, but com-mon sense indicates that, for many skills and areas ofprofessional judgement, therewillbe a fall off of outcomewith diminishing use.Under such circumstances a recordof experiencemust bemaintained in a systematic way sothat the doctor can easily identify areas of weaknessand remedial measures can be taken at the appropriatetime.The f|rst round of appraisals for consultants in the UK

has been associated with an increase in the number ofconsultant anaesthetists seeking to learn, or improvetheir skills, in f|bre-optic intubation. This may indicatethe problem of not using appropriate data to decidelearning targets. If f|bre-optic intubation is felt to be acore skill then such learning is appropriate, but formany

ARTICLE IN PRESS

174 CURRENT ANAESTHESIA & CRITICAL CARE

consultants a log of their work would demonstrate thatthey have inadequate opportunity to use the techniquewhen they have learned it.Thismay lead them to employa f|bre-optic technique, with which they are not skilful,and to harm, rather thanhelp, patients.Theperception‘Icannot do that,’must bematched by the appreciations,‘Ineed to do that’, and ‘I have a use for that inmy practice’,before the decision to learn is taken.The logbook mustbe interpreted in terms of need and risk.This sort of re-flection on the pattern of practice is a legitimate use oflogbookdatawithin a portfolio.

Outcome data

The portfolio will usually include any data on clinical out-comes. It should include any other data that can be de-rived from practice, such as critical incident reports.

The opinions of others

The doctormay ask colleagues or patients to write a writ-ten commentary on the parts of their practice that theysee andunderstand.These data are thebasis of a 3601 feed-back. Cooperation with colleagues and team working arediff|cult aspects of practice, and in these areas othersmaynot seeus aswe see ourselves.When theGMCinvestigatesthe clinical competence of doctors theygain a lot of insightfrom the comments of third parties. The doctor is oftenvery surprised to f|nd what their colleagues think aboutthem. It is brave to ask---but very worthwhile.Adhoc data that indicate other people’s opinions, such

as letters of thanks and complaint are also helpful.

ASSESSMENTPortfolios are not useful for comparing the performanceof doctors.You cannot ‘mark’a portfolio; there is no passand fail. They are best suited to comparing the perfor-mance of the doctor with their personal objectives. Sucha use is intrinsically formative: the assessment is beingused to guide learning. Aproperlypreparedportfoliowillbe such an individual document that it cannot usefully becomparedwith others.The portfolio can have a numberof functions in assessment.

1. Ipsative assessment.---The process of preparing aportfolio requires the setting of standards of perfor-mance and then identifying the gaps between theseand actual performance. Data on clinical outcomemay be included.This then lets the doctor audit theirstandards in terms of any local or national f|gures thatare available.Most importantly it also allows the doc-tor to compare their present performancewith theirpast and to set targets for the future. In this way theprocess of maintaining the portfolio permits ipsative

assessments. (An ipsative assessment compares cur-rent with past performances).

2. It assesses the use of knowledge in real contexts.---Sincea portfolio is a reflection on a selected collection ofexperiences in real situations, it provides an opportu-nity to understand performance in real-life settings.By being ‘situated’ or ‘contextualized’ in this way, theportfolio provides the opportunity to assess otherpersonal andprofessional qualities, such as leadership,communication and management skills that are notpossible by traditionalmethods.

3. As a self-assessment.---The very selection of materialsfor a portfolio is an opportunity for reflective self-as-sessment. If the doctor approaches their portfolio inan open way they may get benef|cial insights abouttheir performance. Does the evidence they havecollected fully support the picture they have of them-selves as a clinician?

The dilemma of using a Portfolio for Formalor Summative Assessment

Inbothundergraduate andpostgraduate education thereis debate about the legitimacy of using a portfolio as aSummative assessment.Honest public reflection on per-formance requires that doctors do not feel threatenedby any use of that reflection. It is natural for doctors tofear that they may reveal too much about their weak-nesses and that this may affect their careers. Accre-ditation andre-accreditation are as summative as assess-ment can get! If the portfolio is used for assessment itmayundermine the doctor’s true reflection on their per-formance.This concern may cause them to boast abouttheir strengths, rather than identifying their weak-nesses. If some institutes or trainers want to use portfo-lios for formal assessment they must accept that theprocess of preparation may lose some of its educationalvalue.If the portfolio is used for assessment:

* The evidence presentedmust be valid; i.e. show whatit is intended to show.

* The evidence must be suff|cient; i.e. the quantity andvarietyof experiences andreflection on those experi-ences is detailed enough to ascertain that learning hastaken place.

* The criteria for checking validity and suff|ciencyshould be derived mainly from the actual context ofthe learning.

If it is being used for summative assessment portfolio re-view should be part of a broad review of a doctor’s prac-tice. There will be objective and third party evidenceabout this. How does the portfolio and the reflectiontherein relate to these? The portfolio, as such, will notbe assessed.What is assessed will be the performance

ARTICLE IN PRESS

PORTFOLIOS CAN ASSIST REFLECTIVE PRACTICE AND GUIDE LEARNING 175

of the doctor and their professional insight and aware-ness. In this the portfolio will be one source of evidence.

It provides ‘Autonomy’and ‘Motivation’ forlearning

Preparing a portfolio provides a formal framework forthe doctor to reflect. Such opportunities are scarce. Inorder to get any benef|t the doctor must see what theyare doing in those terms rather than seeing portfoliopreparation as an imposed task to dispose of in a per-functory way.Theymust think in terms such as:

* What do I do?* How well do I do it?* What is the evidence about how well I do it?* What do I do less well?* How can I improve these aspects ofmy practice?* Am I satisf|edwithmy current work?* How shouldmy professional role develop in future?

These questions can be answered for the benef|t ofthe individual doctor or shared with relevant colleaguesas part of an appraisal.

It is learner centred, individualized andworks as amirror

Since the design (what is to go in the portfolio), control(howmuch andwhen) and ownership (to be sharedwithwhom) all rest in the hands of the doctor, it is a truly stu-dent-centred educational tool that can cater for indivi-dual differences. The complete flexibility and freedomprovided by a portfolio allows a range of learning stylesto be used according to the preference of the learner.The portfolio builds up an image of the doctor as a pro-fessional that reveals strengths and weakness. They cansee this image like looking at their reflection in a mirror.Importantly this image is the doctor’s own creation. Touse a differentmetaphor, the hurly-burly of clinical prac-tice is like a maze.You cannot see where you are goingwhilst you are in it. A portfolio lets the doctor see downon themaze and appreciate their position.

It works as amap of growth

The portfolio provides an opportunity for doctors todocument their professional behaviour over time.Usedin this way, the portfolio acquires importance as an in-strument for recording development. This can be de-scribed asmaking a map of growth.

It helps in establishing a fruitful dialoguebetweenconsultants andspecialtymanagers

If doctors share their reflection on their professional ex-perienceswith senior peers andmedicalmanagers then a

discussion can be directed towards assisting the doctorto develop a personal development plan that suits theirambitions and the intentions of their directorate. Manydoctors have expended a lot of effort acquiring the skillsand knowledge needed to develop their career only tof|nd that their employer does not support their aspira-tions. I have personal experience of an anaesthetist inde-pendently starting a chronic pain service in their hospitalwithout management support and without funding.Thiswas not a good idea.

It widens the curriculum

By providing the opportunity to submit any activity as apart of the portfolio the doctor comes to appreciate thevarietyof options that are available to them for learning.6

Mathers et al. have compared the continuing medicaleducation of general practitioners with and without theuse of a portfolio in a cross over trial. The traditionalpostgraduate education regime showed a smaller spreadof learning activities and fewer subjects of study whencompared by the phase that included preparation of aportfolio.8

CONCLUSIONPortfolio preparation should be a positive educationalexperience.The doctor should include evidence that theybelieve demonstrates the strengths and weaknesses oftheir present practice. This process and that of subse-quent personal review, and discussion with third partieswill assist the doctor to be realistic about themselves asprofessionals. These reflections will guide their furtherdevelopment. A doctor who enters wholeheartedly intothis process will benef|t from it more than one whomerely goes through the motions without critically ap-praising themselves.Wholehearted involvement impliesopenness both in the selection ofmaterials for the port-folio and in subsequent reflection about them.For thoseresponsible for institutional standards portfolio reviewprovides a good insight into what makes an individualdoctor tick.

REFERENCES

1. Gupta S K, Greaves J D. Learning by maintaining a portfolio. In:

Greaves J D, Dodds C, Kumar C M, Mets B (eds). Clinical

Teaching: A Guide to Teaching Practical Anaesthesia. Lisse: Swets

and Zeitlinger, 2003; 83--91.

2. Brigden D. Career focus---Constructing a learning portfolio. Br

Med J 1999; 319: 2.

3. http://www.doh.gov.uk/nhsexec/consultantappraisal/ Website ac-

cessed on 14th July 2003.

ARTICLE IN PRESS

176 CURRENT ANAESTHESIA & CRITICAL CARE

5. Challis M. AMEE Medical Education Guide No. 11 (revised):

Portfolio-based learning and assessment in medical education. Med

Teach 1999; 4: 370--386.

6. Friedman BD. The role of assessment in expanding professional

horizons. Medical Teacher; 22 472--477.

7. Southgate L, Cox J, David T et al. The General Medical Council’s

Performance Procedures: peer review of performance in the

workplace. Med Educ 2001; 35(Suppl 1): 9--19.

8. Mathers N J, Challis M C, Howe A C, Field N J. Portfolios in

continuing medical education---effective and efficient? Med Educ

1999; 33(7): 521--530.

FURTHERREADING

Sackville A D. Assembling, Using a Professional Portfolio. Ormskirk:

Edge Hill University College, 1997.

Redman W. Portfolios for Development. London: Kogan Page,

1994Greaves J D, Dodds C, Kumar C M, Mets B. Clinical Teaching:

A Guide to Teaching Practical Anaesthesia. Lisse: Swets and

Zeitlinger, 2003.

Open University. A Portfolio Approach for Personal, Career

Development. Milton Keynes OU, 1992.

ARTICLE IN PRESS

PORTFOLIOS CAN ASSIST REFLECTIVE PRACTICE AND GUIDE LEARNING 177