The use of reflective and reasoned portfolios by doctorsjep_1514 182..185

Deborah C. Saltman AM MBBS(Syd) MD(NUSW) MRCGP FRACGP FAFPHM,1

Abdollah Tavabie MBBS FRCGP2 and Michael R. Kidd AM3

1Professor, Brighton and Sussex Medical School, Brighton, UK2Professor, GP Dean, Kent, Surrey and Sussex Deanery, London, UK3Professor, Executive Dean, Health Sciences, Flinders University, Adelaide, SA, Australia

Keywords

doctor, portfolio, postgraduate medicaleducation

Correspondence

Professor Deborah C SaltmanBrighton and Sussex Medical SchoolMayfield HouseFalmerBrighton BN1 9PHUKE-mail: deborah@gp.med.usyd.edu.au

Accepted for publication: 18 May 2010

doi:10.1111/j.1365-2753.2010.01514.x

AbstractRationale Reflective portfolios have been widely trailed in vocational programmes forhealth sciences education. While not raised explicitly yet, there is recognition that anessential part of a portfolio may still be missing – that is the experience of theoretical andanalytic questioning that should occur after reflection.Aims and objectives In this paper, we argue that portfolios require both reflective andreasoned components to provide a more complete range of educational experiences forpostgraduate doctors in the pursuit of higher qualifications.Methods We describe the differentiating features between the two components and con-clude with some suggestions on how both reflective and reasoned components of portfoliosmay work synergistically through a series of case studies.Results The reflective style of portfolio, which is in current use, has been widely acceptedas a major advance in postgraduate medical education at all levels. It provides an effectivevehicle for encouraging self-consideration and generalizability. It is an important adjunctwhere it functions as a personal development tool as well as a sophisticated log book. Areasoned or pedagogic element encourages more analytic activities: adding an additionalcomponent that can ground the portfolio in theory or research. The portfolio is then ableto function in a range of academic spheres and provide a theoretical platform for thewide range of non-clinical empirical work undertaken in clinical settings. The addition ofa reasoned component also increases the emphasis on the introduction of new conceptsoutside the sphere of the learner’s experience.Conclusions A portfolio containing both reflective and reasoned components is then ableto function in a range of academic spheres and provide a theoretical platform for the widerange of non-clinical empirical work undertaken in clinical settings. In an iterative mode,such a portfolio element can support a forum for questioning established concepts withina pedagogical environment – the essence of evidence-based practice.

BackgroundWhether in electronic or paper format, information in portfolioshas been categorized by its summative or formative intent. Moonsubdivides the summative versions into whether they are con-cerned with documenting learner outcomes or individual achieve-ments (developmental or career) or personal reflection [1].

The distinctions between these versions are contextually andexperientially based in that the environment and the readershiplargely determine what is written. In practice, most portfolios endup as an amalgam of the developmental and reflection versions.The Royal College of General Practitioners (RCGP) e-portfolio,the most established example in the UK, is an example of anamalgam of all three versions [2].

The expectation of an amalgam of summative and formativeparts in a workplace setting is that this style of portfolio willengage the postgraduate doctor in an adult learning cycle, flow onto their professional life and allow for an integrated and moremature approach to learning – akin to the tenets of continuousprofessional development. In medical undergraduate education,where learning is often less adult in style and more likely to beassessed in a summative way, portfolios are used to supplement theessential and more tangible components of knowledge and skillsacquisition with individual reflection. In both medical undergradu-ate and postgraduate education and training, the introduction ofportfolios has provided challenges for both learners and teachers.

In this paper, we argue that portfolios need to be more thanreflective exercises. In their current form their major strength is

Journal of Evaluation in Clinical Practice ISSN 1365-2753

© 2010 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice 18 (2012) 182–185182

that they are contextually based and owner driven. As such, theyhave the potential to be limited in educational application andfunction as merely sophisticated log books and diaries. The addi-tion of a critical analytic component to a portfolio, we argue,allows the portfolio owner to be guided to the use of analytic skillsto integrate their learning and critique and explore aspects ofclinical practice. We believe that this approach is more in keepingwith the broader educational aspirations of the future adult learner.

A critique of the reflective portfolioReflective portfolios have been widely trialled in vocational pro-grammes for health sciences education [3]. Plaza defines them as‘a collection of evidence that attests to achievement as well aspersonal and professional development through a critical analysisand reflection of its contents’. Implied in this definition is that theprocess is effectively self-directed by the learner in an introspec-tive rather than showcasing fashion. In this way, the portfolio canact as an educational ‘pressure release valve’, allowing postgradu-ate doctors to explore the integration of their learning in a self-directed way and record their development. This definitionpervades the higher educational literature where portfoliosincreasingly have been used to demonstrate personal and profes-sional development [4]. It is a view that is congruent with thetenets of Moon and self-directed learning, particularly for courseswhere curricula are quite dichotomized between the less well-defined, more-people focussed aspects and the more rigid struc-tured and often technical requirements.

A more fluid view of a reflective portfolio has been proposedby Chambers and Glassman. They see the portfolio as a movingobject changing with the learner. This type of portfolio is aworking document rather than a static reflection of a work-in-progress. Their concept is underpinned by the notion that, withinthe preparation of a portfolio, learners and their teachers havedistinct roles and both contribute to it. The learner prepares it; theteacher scopes what capabilities can be tested by the portfolio,assessing what constitutes acceptable evidence of accomplishmentof set competencies, and determining timelines for evaluation [5].In this mode, the portfolio assumes the role of both a formative anda summative instrument – as is the case with the RCGP portfolio.

It has been suggested that this method allows a real-time,authentic way of recording that is applicable across a variety ofsettings outside the teaching space [6]. E-portfolios also provide areal-time way of sharing this information. For example, postgradu-ate doctors can examine their own thoughts and actions and makebetter sense of what they already know, explore how their knowl-edge, actions and beliefs relate to other contexts (e.g. the use ofevidence) and consider whether a change in their perspective,beliefs or behaviour is needed. There is scope for other reflectiveroles to emerge within this paradigm: for example as they have innursing, where educators have found the portfolio to be a valuablediagnostic tool [7].

An e-portfolio can be seen as a type of learning record thatprovides actual evidence of achievement. Learning records areclosely related to a learning plan or curriculum and are an emerg-ing tool that is flexible enough to be used to manage learning byindividuals, teams, communities of interest and organizations.This capacity to complete the portfolio while in a workplacesetting provides authenticity.

Sometimes it is not possible to distinguish which role a portfoliofulfils – as a free flowing reflection or as a more structured ongoingreflection and eventual assessment of competencies – in whichcase the term ‘reflective’ may be usefully vague and collective. Theuse of reflective portfolios as a summative assessment mode hasbeen increasing: not only because they provide an adult learningplatform but also because they require less formal marking andprovide a more interactive professional conversation.

Concerns have been voiced about the apparent duality createdby the inclusion of a summative component. For example, a sum-mative or developmental emphasis may stifle the richness ofhonesty and reflection, which could be present in a formativeor purely reflective portfolio and introduce a social desirabilitybias in reflective statements when they are used for assessmentpurposes [8].

In the workplace training of health professionals, the relativelyconcrete demands of vocational certification require an emphasison the developmental notion of a portfolio. In these circumstancesportfolios are used to document certain achievements. This processof necessity precludes postgraduate doctor input into the determi-nation of the parameters of the portfolio [2]. The underlying prin-ciple behind this use of the reflective portfolio is to bridge thedivide between theory and practice. In reality it may be more aboutobliterating the chasms opened up in undergraduate health educa-tion between two ‘best’ paradigms: best theory and best practice.Reflective portfolios have been assumed to be credible sources ofevidence of competency as they are derived from a curriculum andare presented as a form of authentic assessment [9]. However, ithas been argued that a form of assessment that is unreliable andthus lacks validity cannot be credible [10].

There is considerable evidence that postgraduate doctors do notfind the portfolio experience as satisfactory as educators wouldhope. In one study, the participants felt uneasy about the indepen-dent learning required to construct a portfolio when measuredagainst the more concrete aspects of medical knowledge acquisi-tion [11]. Not only does this study highlight the ambivalent expe-rience of portfolios but also the perhaps idealistic expectations ofeducators about what portfolios can achieve. It comes as no sur-prise therefore that portfolios have been used to address a numberof educator-driven objectives – the main one being assessment.Inevitably, the use of a portfolio as an assessable task in an alreadypacked workload for both clinical teachers and postgraduatedoctors encourages negative responses [12,13]. The situation isexacerbated when a single body of work, such as an essay orexamination, both of which have been well-recognized routes forassessment, is replaced by a series of relatively short bodies ofworks, the assessment of which may not be easily discernable bythe postgraduate doctors.

Work Place Based Assessment (WPBA) will be a new compo-nent of mandatory testing of medical practitioners in the UK forlicensing and relicensing purposes. While in its infancy WPBA isexpected to contain portfolio evidence as a representation of prac-tice.[14] In the future, portfolio learning and assessment will likelybe an integral part of the assessment of any clinician. Within thisframework, as part of revalidation process, there is an implicitunderstanding that even these resume-style portfolios have aduality: in that they should be learner-centred although not alwayslearner-driven. It is purported that ‘the main purpose of WPBA isto help learners identify areas for improvement and if this is not

D. C. Saltman et al. Reflective and reasoned portfolios

© 2010 Blackwell Publishing Ltd 183

done, its educational validity will be lost’. Thus, in the workplacesetting portfolios are about continuous quality improvement (CQI)and not the acquisition of the underlying principles that may drivethis improvement, nor the integration of new and fundamentalprinciples of practice. As one author remarks: ‘the inclusion ofevidence should be both thoughtful and representative of profes-sional standards’ [15]. Even certifying agencies, such as the Post-graduate Medical Education and Training Board in the UK,acknowledge that ‘in most portfolio systems used in medical edu-cation in the northern hemisphere, trainees engage far less in thismechanism and philosophy than is required’. [14]

Reasoned portfolios as anecessary additionWhile not raised explicitly yet, there is recognition that an essen-tial part of a portfolio may still be missing – that is the experienceof theoretical and analytic questioning that should occur afterreflection. The key features of a reasoned section in a portfolio arethe capacity to allow the writer/owner to: demonstrate an under-standing of theory and relay that understanding; critique the theoryand identify pedagogic gaps; and provide reasoned rather thanreflective argument. They can also be used to evaluate competencein a specific conceptual area. Central is a core understanding of thepurpose behind the reasoned portfolio and what it adds to currentnotions of portfolios. We highlight the key additive benefitsthrough a series of case studies.

Case study 1

A postgraduate doctor might submit an example of their work onhow the theories behind clinical competence have shaped a teach-ing session they ran in the field. This component of a reasonedportfolio could look at the theoretical basis for the design of theintervention rather than its implementation and evaluation. Thesecond document for this portfolio entry could be a short annotatedbibliography of the clinical competence literature as it applies tocontemporary medical education. A third document may be anabstract on the portfolio entry; developed for a conference or aposter. Self-reflection about how well the doctor demonstrated anunderstanding of theory and was able to elucidate it in both theo-retical and practical ways might complete this section.

The entries in a reasoned section may also provide the ground-work for a component of CQI that incorporates the notions ofleadership and change management. To date CQI concepts ofportfolios have centred on a particular set of adult learning con-cepts: that future learning is driven by a reflective assessment ofperformance to identify areas for improvement. This may be donewith or without assistance from peers, colleagues or mentors. Acomplete CQI project requires not only problem identification, butalso a proposed solution. Whichever way it is looked upon it isessentially a closed loop process driven by the participants. It doesnot provide the opportunity for external input and the participantsremain unconsciously incompetent [16].

Continuous quality improvement projects could make excellentreasoned portfolio items for experienced doctors. The purpose ofthe entry would be to provide competency in assessing and resolv-ing a systems problem. The entry addresses the competency of

practice-based learning. A complete CQI project requires not onlyproblem identification, but also a proposed solution [17].

Case study 2

A reaccreditation candidate, as part of a revalidation process, isrequired to attempt to implement a CQI solution and evaluate theeffectiveness. The portfolio entry created by this doctor couldinclude the project description, data collected, solution anddescription of implementation, and subsequent data collected. Thereflective component of such a portfolio could include an assess-ment of practice and how it changed through the CQI project.Additionally, the implementation might provide evidence and self-reflection on communication skills, professionalism or aspects ofsystems-based practice.

A reasoned component to this portfolio, if used in vocationaltraining, could provide the opportunity for reflection on both thelongitudinal learning of a trainee across the years of on-the-jobtraining and the pedagogic components in their academic trainingprogramme – in both summative and formative ways.

Case study 3

In university medical student education, where the subject matteris often discrete and produced in half year blocks, portfolios canbridge cultural ‘gaps’ between subjects. Also where subject matteris rarely integrated across courses, let alone across disciplines,schools or faculties, a portfolio can provide the opportunity theassess whether the student, who really is the common threadacross these ingredients, has integrated the subject matter forthemselves in an adult way.

The most often cited limitation of portfolios is the reliabilitywith which they can be assessed [15,17–19]. In a reasoned port-folio the type of assessment, whether it is formative or summative,will be supported by a reasoned academic argument, which will bethe responsibility of the learner to develop and for the assessor toassess.

Herman and Winters have proposed a number of questions todetermine the reliability and validity of a portfolio [20]. The lit-erature remains inconclusive in this area and has not yet addressedthe issue of how to determine the reliability and validity of areasoned portfolio. Conventional methods of assessing reliabilityand standardization of assessment, for example standardizingraters and calibrators or moderators, would be more applicable tothe reasoned portfolio [21].

Comparing reflective andreasoned elementsThe key elements differentiating the reasoned from the reflectiveparts of a portfolio, as highlighted through the case studies, areshown in Table 1.

ConclusionIn this paper, we have argued that the reflective portfolio inits current form as used in postgraduate training requires anadditional component to enable it to function as more than just asophisticated log book. It is not surprising that distinctions

Reflective and reasoned portfolios D. C. Saltman et al.

© 2010 Blackwell Publishing Ltd184

between current types of portfolios become blurred when they areeffectively experiential and reflective documents. In order for theportfolio to achieve different learning, personal or work-relatedoutcomes, more related to the future practice of a clinician, areasoned component is also required thereby making it more of anactive interplay between theory and practice.

References1. Moon, J. A. (2006) Learning Journals: A Handbook for Reflective

Practice and Professional Development, 2nd edn. London: Routledge.2. RCGP e-Portfolio. RCGP home page. http://www.rcgp.org.uk/ (last

accessed 14 July 2010).3. Plaza, C. M., Draugalis, J. R., Slack, M. K., Skrepnek, G. H. & Sauer,

K. A. (2007) Use of reflective portfolios in health sciences education.American Journal of Pharmaceutical Education, 71 (2), 1–6.

4. Paulson, F. L., Paulson, P. R. & Meyer, C. A. (1991) What makes aportfolio a portfolio? Educational Leadership, 48 (5), 60–63.

5. Chambers, D. W. & Glassman, P. (1997) A primer on competency-based evaluation. Journal of Dental Education, 61 (8), 651–666.

6. Greenberg, G. (2004) The digital convergence: extending the portfoliomodel. EDUCAUSE Review, May/June, 28–36.

7. Mitchell, M. (1994) The views of student and teachers on the use ofportfolios as a learning tool and assessment tool in midwifery educa-tion. Nurse Education Today, 14, 38–40.

8. Schaffer, M. A., Nelson, P. & Litt, E. (2005) Using portfoliosto evaluate achievement of population-based public health nursingcompetencies in baccalaureate nursing students. Nursing EducationPerspectives, 26, 104–112.

9. Miller, M. D. & Legg, M. (1993) Alternative assessment in a high-stakes environment. Educational Measurement Issues and Practice,13, 9–15.

10. Joyce, P. (2005) A framework for portfolio development in post-graduate nursing practice. Clinical Nursing, 14, 456–463.

11. Snadden, D. & Thomas, M. (1998) The use of portfolio learning inmedical education. Medical Teacher, 20, 192–199.

12. Challis, M. (1999) AMEE Medical Education Guide No. 11 (revised):Portfolio-based learning and assessment in medical education.Medical Teacher, 21, 370–386.

13. Corbett-Perez, S. & Dorman, S. M. (1999) Electronic portfoliosenhance health instruction. The Journal of School Health, 69 (6), 247 –250.

14. Workplace Based Assessment (WPBA): A Guide for ImplementationPMETB. May 2009.

15. Pitts, J., Coles, C. & Thomas, P. (1999) Educational portfolios inthe assessment of general practice trainers; reliability of assessors.Medical Education, 33, 515–520.

16. Saltman, D. & O’Dea, N. (2004) Supervising research students inprimary health care using a leadership model. Australian FamilyPhysician, 33 (5), 373–375.

17. Jarvis, R. M., O’Sullivan, P. S., McClain, T. & Clardy, J. A. (2004)Can One Portfolio Measure the Six ACGME General Competencies?Academic Psychiatry, 28 (3), 190–197.

18. Pitts, J., Coles, C. & Thomas, P. (2001) Enhancing reliability inportfolio assessment: ‘shaping’ the portfolio. Medical Teacher, 23,351–356.

19. Pitts, J., Coles, C., Thomas, P. & Smith, F. (2002) Enhancing reliabil-ity in portfolio assessment: discussions between assessors. MedicalTeacher, 24, 197–201.

20. Herman, J. & Winters, L. (1994) Portfolio research: a slim collection.Educational Leadership, 52, 48–55.

21. LeMahieu, P. G., Gitomer, D. H. & Eresh, J. A. (1195) Portfoliosin large-scale assessment: difficult but not impossible. EducationalMeasurement Issues and Practice, 14, 11–16; 25–28.

Table 1 Differences between reflective and reasoned portfolioelements

Reflective portfolio Reasoned portfolio

Competencies for practice Analytic skills to integrate learningand review practice

Demonstration of practice Exploration of practiceExplains ‘how’ Questions ‘what’Theory provided Theory critiquedSelf-directed LedIdentification of clinical areas of

improvementIdentification of pedagogic gaps

Evaluation of capacity to achieve/improve skills

Evaluation of understanding ofconcepts and relaying thatunderstanding

Acceptance of criteria Discussion of criteriaCompliance expected Innovation encouragedFormative assessment Summative assessmentMay be collaborative Will be individualIndividual learning focus Allows cross-disciplinary content

focus

D. C. Saltman et al. Reflective and reasoned portfolios

© 2010 Blackwell Publishing Ltd 185