Protocol

How can the clinical supervision of junior doctors be delivered to develop clinical competencies while

maintaining patient safety? A BEME effectiveness review

Collaboration

Title: How can the clinical supervision of junior doctors be delivered to develop clinical competencies while maintaining patient safety? A BEME effectiveness review Lead reviewer: Dr Claire Allen Health Education England (North West Team) Email: c.allen2@nhs.net Review group members: Dr Claire Allen, MB ChB, FRCA. Health Education England (North West Team) Claire is an ST6 trainee in anaesthesia with an interest in medical education who is undertaking a Medical Education Fellowship with HEE. Dr Christopher Raistrick, MB ChB, FRCA. Health Education England (North West Team) Chris is an ST6 trainee in anaesthesia with an interest in medical education who is undertaking a Medical Education Fellowship with HEE. Dr Rebecca Wilson, BSc, MB ChB, MRCPCH, MPH. Health Education England (North West Team) Rebecca is an ST5 trainee in paediatrics with an interest in medical education who is undertaking a Medical Education Fellowship with HEE. Dr Steven Agius, BA (Hons), MA, PhD. Health Education England (North West Team) Steven is Senior Research Fellow in Medical Education and Head of Research at Health Education England (North West Team). He is responsible for developing and implementing a rolling programme of empirical research into priority areas in postgraduate medical education with a view to contributing to the international evidence base as well as informing local practice and quality processes. A specialist in qualitative research, with a PhD in Medical Education, he has published widely in the academic press, with particular interests in faculty development, education cultures and medical careers. Professor Jeremy Brown, BA (Joint Hons), PGCE, PGCert, PhD. Edge Hill University Jeremy is Professor of Clinical Education in the Postgraduate Medical Institute, Edge Hill University. His main interests are in transitions in medical careers, educational supervision and work based assessment. Jeremy leads on a number

of studies funded by Health Education England (North West Team) and is a core member of the Association for the Study of Medical Education Research Group. Professor Ben Shaw, MB ChB, MRCP(UK), FRCPCH, MD, MA (Clin Ed). Edge Hill University Ben trained in Birmingham, Edinburgh and Liverpool. He is a consultant in neonatal and respiratory paediatrics at Liverpool Women’s Hospital and the Royal Liverpool Children’s Hospital and a Professor at the Evidence–based Practice Research Centre, Edge Hill University. Previous roles include Clinical Director of the Neonatal Unit at Liverpool Women’s Hospital, R and D director at Liverpool Women’s Hospital and Associate Postgraduate Dean for Mersey Region. His research interests are neonatal respiratory disease and medical education. He is the simulation lead for Mersey Deanery School of Paediatrics, a member of the simulation steering and research group of the Royal College of Paediatrics and Director of the Mersey Neonatal Simulation Course. Miss Emma Child, MA. Edge Hill University Emma graduated from the University of Sheffield in 2012 with an MA (Pass with Distinction) in Librarianship. Since graduating she has worked in government and healthcare libraries where she has developed experience of designing search strategies for systematic reviews. Professor Paul Baker, MB ChB, FRCP, FRCGP, PG Dip. Health Education England Paul is a Consultant Geriatrician at the Royal Bolton Hospital. His interest in medical education surfaced as Royal College of Physicians Tutor in Bolton in 2002. He was later Postgraduate Clinical Tutor and ran pilots well before foundation training was properly rolled out in the UK. As chair of the STC in Geriatric Medicine, he founded the MSc in Geriatric Medicine (Salford University) with two consultant colleagues, a unique workplace-based Masters course which replaces the Deanery teaching programme for the specialty. He has a special interest in the assessment of the quality of medical training and has published in this area. Clinically he has a special interest in diabetes in the elderly and diabetic foot disease.

1. Abstract

Background

A major goal of postgraduate medical training is to develop clinically competent doctors who are able to practise independently. In order to achieve this goal, junior doctors perform their clinical duties under the supervision of senior medical staff. This clinical supervision needs to enable junior doctors to develop clinical competence and autonomy while maintaining patient safety. Clinical supervision can be provided in a number of different ways and can have varying effects on the quality of care provided to patients and on the learning, professional development and psychological wellbeing of doctors.

UK postgraduate medical training is currently undergoing a number of important changes. Improving patient safety and raising the profile of clinical supervision are two key components of these changes. Consequently, understanding how best to deliver clinical supervision in order to develop clinical competencies while maintaining patient safety is an area that deserves further examination. Objective

To carry out an effectiveness review of the existing literature in order to evaluate the evidence on the different ways of delivering clinical supervision to junior doctors and their impact on patient safety. Methods The following study selection criteria have been defined: Population: non-consultant grade / non-general practitioner postgraduate doctors. Intervention: clinical supervision and clinical oversight. Outcomes: Quality of healthcare, patient safety, patient care, clinical competence, autonomy, decision-making, professionalism, psychological wellbeing, burnout, resilience and training of supervisors. Only studies published in English language will be eligible. MeSH terms and free text terms will be used to search 7 electronic databases. This will be supplemented by hand searching of high yield journals, reviewing grey literature, reviewing reference lists of all papers and relevant reviews identified and contacting authors in the field regarding any unpublished material. Abstracts of all articles will be screened independently by two reviewers and any discrepancies checked by a third. Any articles not meeting the inclusion criteria will be discarded. Studies will be appraised for quality using the QATSSD scoring tool and for level of impact using a modified version of Kirkpatrick’s hierarchy. The findings will be collated in a narrative synthesis. A scoping search of MEDLINE generated 2597 results which was considered to be an appropriate number of articles. A review of the titles and abstracts of a test set of 200 articles demonstrated that the scoping search was robust and yielded appropriate articles.

2. Background to the topic

Introduction A major aim of postgraduate medical training in the UK is to enable junior doctors to develop the necessary clinical competencies to become independent practitioners, either as general practitioners (GPs) or specialty consultants. At present, since the reorganisation of UK postgraduate medical education in 2005, this complex transition is designed to take a minimum of five years post-qualification for those wishing to become GPs, and often upwards of eight years for those in hospital specialties (GMC, 2014). Within this timeframe, trainee doctors need to progress from being newly qualified foundation doctors requiring high levels of clinical supervision by more senior colleagues to become autonomous practitioners. Furthermore, this process needs to occur with minimal impact on patient safety (GMC, 2015).

UK postgraduate medical training is currently undergoing a number of important changes that will impact on how trainees are guided through this process. The recent independent review of postgraduate medical education, Shape of Training: Securing the future of excellent patient care (Greenaway, 2013), promises an overhaul of the current training structure and calls for an increased emphasis on personalised supervision and support to increase training opportunities and develop clinical competence. Alongside this, the General Medical Council (GMC), the UK’s medical regulatory body, is pushing ahead with its requirement for more formal recognition of the supervisory roles of senior medical practitioners and guarantees that those taking on the responsibility of supervising have the appropriate training to do so (GMC, 2012). In addition, patient safety, long a central theme of postgraduate medical training, is set to have an even more prominent role with the implementation of the GMC’s revised standards for undergraduate and postgraduate education and training in January 2016 (GMC, 2015). Consequently, in the light of these current changes, the complex question of how best to deliver the clinical supervision of junior doctors to achieve the dual goals of developing clinical competence while maintaining patient safety is an area that deserves more extensive examination. This systematic review will collate and critically appraise the existing literature on how the clinical supervision of junior doctors can be delivered so as to promote the development of clinical competencies while preventing harm to patients. This will produce evidence that can be used to inform the ongoing development of best practice standards for the clinical supervision of junior doctors at this pivotal time for postgraduate medical training. This section continues by defining the specific terms that will be used in this review and exploring the relationship between clinical supervision and patient safety. It then moves on to examine the existing literature and highlight how this review will add to our current understanding of the topic.

Definition of terms

Junior doctors In the UK, the term junior doctor is used to refer to all doctors in training who have not yet reached independent practitioner status as either GPs or speciality consultants (NPSA, 2015). As a result, the term trainee doctor will be used interchangeably with junior doctor in this review. We also recognise that at any one time there is a population of doctors, working at the same levels as trainee doctors, who are not formally training and progressing up the career scale towards independent practitioner status. While the safe clinical supervision of these doctors is vital to prevent harm to patients, their need to develop clinical autonomy is not paramount. We are not excluding non-training non-consultant / GP doctors from this review as we feel the experience of supervising this population may provide insights that can be used for the trainee workforce. However, our focus will remain on how to develop clinical autonomy in those actively progressing towards consultant or general practitioner status. Patient safety

Patient safety is a core principle within modern medical practice and is defined by the World Health Organisation (WHO) as the prevention of errors and adverse effects to patients associated with healthcare (WHO, 2015). Patient safety is also a top priority for the UK’s National Health Service (NHS), with patients told to “expect the people and organisations caring for them to make their safety the first priority” (NHS, 2014). Junior doctors will often be the first point of contact for patients accessing the NHS (NPSA, 2015). In this review, we are particularly interested in how the clinical supervision of junior doctors providing patient care while learning to become independent practitioners can be used to maintain safety and prevent adverse events. However, we are aware that clinical supervision is only one of many factors that influence patient safety, and we will be mindful of this as we progress through the review. Clinical competencies One of the hallmarks of current postgraduate medical education and training in the UK is the use of competency-based criteria to determine progression to the next stage of clinical training (DoH, 2004). Trainees have to provide evidence that they have reached a pre-defined level of clinical competency, as set out in speciality-specific curricula, before moving on to the next training level (UKFPO, 2014; GMC, 2014). Within this system, a commonly used marker for the progression of an individual’s clinical competencies is the ability of the trainee to perform clinical tasks with decreasing amounts of supervision and increasing autonomy. The ultimate aim of this process is the award of a Certificate of Completion of Training (CCT), which permits entry on to the GP or Specialist Register and confers the authorisation to practice independently (GMC, 2015a).

Clinical supervision Clinical supervision is a broad term that is used to describe a number of distinct activities in postgraduate medical education and training. On the one hand, it describes a formalised process of clinical supervision, whereby trainee doctors are allocated a named clinical supervisor who is a senior doctor “responsible for overseeing a specified trainee’s clinical work throughout a placement in a clinical or medical environment…He or she will provide constructive feedback during that placement…(and) lead on providing a review of the trainee’s clinical or medical practice throughout the placement that will contribute to the educational supervisor’s report on whether the trainee should progress to the next stage of their training” (p. 16 GMC, 2012). This role is in contrast to that of an educational supervisor who is responsible for “the overall supervision and management of a trainee’s trajectory of learning and educational progress during a placement or series of placements” (p.16 GMC, 2012), although both roles can be taken on by the same person if necessary. However, it is also recognised that junior doctors are likely to receive a large proportion of their clinical supervision from a wide range of more senior staff, and not just from their named clinical supervisor (AoME, 2010). Consequently, the term clinical supervision also relates to the day-to-day oversight of trainees in the workplace and is an activity that involves all clinicians that come into contact with trainees (NCAT UK, 2012). In addition, not only is clinical supervision carried out by different people, but the form that it takes will also vary. For example, Kennedy et al. (2007) indentify various forms of “clinical oversight”: different types of supervision that occur depending on the needs of the patient and the trainee. They describe “routine oversight”, where supervision is pre-planned, “responsive oversight”, where supervision occurs in response to trainee or patient-specific issues, “backstage oversight”, where the trainee is not directly aware of the supervision and “direct patient care”, where the supervisor moves to actively provide care for a trainee’s patient. In this review we are interested in exploring how the different types and practices of clinical supervision impact on patient safety. The relationship between clinical supervision and patient safety Improving and maintaining patient safety is a complex process. It requires a coordinated, system-wide approach that incorporates the vast array of factors that impact on the quality of care that a patient receives (WHO, 2015). One vital part of this patient safety jigsaw is the clinical abilities of the doctors providing medical care. For example, a number of studies have concluded that a greater degree of supervision of trainee doctors by more experienced staff results in improved quality of care for patients and fewer errors (Gennis and Gennis, 1993; Fallon, Wears and Tepas, 1993; Sox et al, 1998; Velmahos et al., 2001; Itani et al., 2005). Subsequently, there have been calls for increased levels of supervision of trainee doctors from government committees (Bell, 1993), professional regulators (GMC, 2005), researchers (Cottrell et al., 2002) and the media (Laurance, 2005). Currently, patient safety is such a priority in

postgraduate medical education that the new standards for the requirements for the management and delivery of undergraduate and postgraduate medical education and training, due to come into effect 1 January 2016, list patient safety as being their “first priority” (GMC, 2015). However, the relationship between patient safety and postgraduate medical training, and specifically clinical supervision, is more complex than merely protecting patients from risk of harm. Firstly, while there may be evidence that increased supervision of trainee doctors results in improved patient safety in the immediate term, there is concern about the impact that increased supervision may have upon opportunities for trainees to work independently and develop clinical autonomy (Kennedy et al., 2005). Indeed, Hinchley and Rothberg (2010) argue that “supervision assures the quality of present patient care, yet autonomy ensures the future of quality patient care”. Secondly, there is an increasing recognition that the successful production of clinically competent, safe doctors requires a more holistic view of training and supervision than has been taken previously. This change in focus is apparent in the new GMC standards for undergraduate and postgraduate medical training due to be introduced in January 2106 (GMC, 2015). While the GMC’s main patient safety focus has to date been on protecting patients from any risk posed by doctors in training, the emphasis now is on ensuring that education and training takes place where patients are safe, the care and experience of patients is good and education and training are valued (GMC, 2015). Patient safety is now seen as “inseparable from a good learning environment and culture that values and supports learners and educators” (GMC, 2015). This shift in emphasis, albeit subtle, not only highlights the pivotal role that clinical supervision plays in developing clinical competence and maintaining patient safety, but also how the practice and outcomes of clinical supervision cannot be seen in isolation from the environment in which they occur. As a result, in this review we will incorporate outcomes into our search strategy that will try to capture some of the factors that influence the creation of a healthy learning environment and the increased psychological wellbeing of trainees. It is exploring this dynamic and complex relationship between clinical supervision, clinical competence and patient safety that lies at the heart of this review. The existing literature In order to check that there is not already a relevant systematic review covering this topic, while ensuring there is enough pre-existing literature of sufficient quality on this topic to warrant such a review, we conducted a scoping search (see Section 5 for details). Within this search we identified a systematic review entitled A Systematic Review: The Effect of Clinical Supervision on Patient and Residency Education Outcomes (Farnan et al., 2012) which we have carefully reviewed to ensure our review will not be duplicating previous work. This review only searched two relevant databases and only included studies from the USA and Canada. Most studies were of a low to moderate quality. Given the current UK drive to cement the importance of clinical supervision while ensuring patient safety is the first priority of all medical training, there is a clear gap in the current literature, as no UK-based studies were included. Our aim is to make this review

more comprehensive by using a greater number of databases, making further efforts to find unpublished work and not limiting ourselves to studies from just two countries. We will cast our net wide in order to encompass as much of the existing literature on this complex topic as possible. However, we will also take into account the fact that our aim is to compile a body of evidence that will inform the ongoing development of best practice standards for the clinical supervision of junior doctors within the UK training system. This requirement will inform our critical appraisal accordingly.

3. Review question(s)/objectives, type of review and keywords

This review will be an effectiveness review. The Best Evidence Medical Education (BEME) Collaboration defines an effectiveness review as one that adds to our knowledge about how to enhance the outcomes of an educational intervention (BEME, 2015). We will provide a narrative synthesis of our findings. Further details of this are provided in Section 8. The review will add to the knowledge base on how best to provide clinical supervision in order to develop clinical competencies while maintaining patient safety. This body of evidence will help to inform the ongoing development of best practice guidelines for the clinical supervision of junior doctors within the UK training system.

The following study selection criteria have been defined: Population: Non-consultant or non-GP postgraduate doctors. Intervention: Clinical supervision or clinical oversight. Outcomes: Patient safety, patient care, quality of healthcare, clinical competence, autonomy, decision-making, burnout, resilience, wellbeing, professional practice, professionalism and training of supervisors.

The following key terms were used in the scoping search: Population:

exp INTERNSHIP AND RESIDENCY/

EDUCATION, MEDICAL, GRADUATE/

(doctor* ADJ3 (postgrad* OR post-grad* OR graduate OR train* OR junior OR foundation* OR new* OR f1 OR f2 OR "first year*")).ti,ab

(clinician* ADJ3 (postgrad* OR post-grad* OR graduate OR train* OR junior OR foundation* OR new* OR f1 OR f2 OR "first year*")).ti,ab

(physician* ADJ3 (postgrad* OR post-grad* OR graduate OR train* OR junior OR foundation* OR new* OR f1 OR f2 OR "first year*")).ti,ab

(medical ADJ3 (postgrad* OR post-grad* OR graduate OR train* OR junior OR foundation*)).ti,ab

"house officer*".ti,ab

housestaff.ti,ab

registrar*.ti,ab

registrar*.ti,ab

residency.ti,ab

intern*.ti,ab

“SAS doctor*”.ti,ab

“specialty and associate specialist”.ti,ab

“staff and associate specialist”.ti,ab

Intervention:

supervis*.ti,ab

oversight.ti,ab

Outcomes:

PATIENT SAFETY/

(patient* ADJ2 safe*).ti,ab

PATIENT CARE/

"patient care".ti,ab

QUALITY OF HEALTH CARE/

CLINICAL COMPETENCE/

PROFESSIONAL AUTONOMY

Autonomy*.ti,ab

DECISION MAKING

“Decision making”.ti,ab.

BURNOUT, PROFESSIONAL

(Burnout OR resilience OR “well being”) ti,ab

PROFESSIONAL PRACTICE

“professional practice” .ti,ab

TEACHING

FACULTY, MEDICAL

((faculty OR educator OR professional OR academic) ADJ

development).ti,ab

((faculty OR educator OR supervis*) ADJ train*).ti,ab

(supervis* ADJ (skill* OR practice)).ti,ab

4. Study selection criteria The study selection criteria for this review are set out below. Study design Both non-comparative (audit, action-based research, case series, historical, narrative, observational and survey-based) and comparative (cross-sectional research, before and after studies, time series studies, non-randomized trials, randomized controlled trials, group randomized trials, case control trials, cohort studies and meta-analysis) research designs will be considered in the review. We will be limited to English language as the review group only speaks English and does not have funding for translation. Study setting Included studies will need to be based in either a primary or secondary clinical care setting. There will be no limits placed on the geographical setting of the study. Population The population will be restricted to non-consultant or non-GP postgraduate medical doctors or their international equivalents. Undergraduate medical students, consultants and GPs (or international equivalents) will be excluded. Medical students will not be included as they do not have any clinical autonomy and as such their supervision is a very different process to that of qualified medical staff. Consultants and GPs have already gained sufficient clinical competence to become independent practitioners. Consequently, the outcome measures of this review will not apply to consultants or GPs. Intervention

Clinical supervision For the purposes of this review, clinical supervision is defined as the process of a senior overseeing the clinical activities of single person or group of persons. This supervision may refer to the supervision provided by nominated clinical supervisors, or the more informal oversight of junior doctors carried out by senior staff on a day-to-day basis. The review will encompass all styles or types of clinical supervision. Supervision for the sole purpose of educational development without relation to clinical implications will be excluded. Outcome measures

Clinical competencies As well as clinical competencies, we will also include “clinical autonomy” and “decision-making” as search terms.

Patient safety We will also include the terms “patient care” and “quality of healthcare” to try to capture as much relevant information as possible.

Other factors impacting on clinical competence and patient safety In order to capture the factors that influence the creation of a healthy learning

environment for junior doctors, we are including outcome measures that cover the psychological resilience of junior doctors and the teaching and training of clinical supervisors. We are excluding direct patient-related outcomes such as patient satisfaction or any financial gains for the patient or hospital in order to maintain a focused approach to this review.

Inclusion criteria

Study design All study designs

No time limit

limited to English language

Population Post graduate medical trainees

International equivalent of UK trainee

Supervision (intervention) Senior providing supervision for junior

Clinical supervision of trainees clinical duties toward

patient

All styles or types of clinical supervision

(see search term for complete list)

Outcome Measures Patient safety and quality of care

Psychological wellbeing/burnout/resilience

Trainee teaching, competence, professionalism,

autonomy & decision making

(see search term for complete list)

Setting of study Clinical context

Secondary or primary care

Exclusion criteria

Study design Not English language

Population Supervision of consultants and qualified general

practitioners

Supervision of medical students

Supervision (intervention) Supervision for the sole purpose of educational

development without relation to clinical

implications

Outcome Measures Patient convenience or satisfaction

Financial gain for medical facility staff or patient

Setting of study Setting without recognised formal training

5. Search sources and strategies To evaluate the existing literature base, a scoping search was conducted by an information professional in December 2015. It was run across Medline, via the HDAS interface on NHS Evidence, and covered the period from inception to the present (1946-2015). The search syntax is included in Appendix 1. The robustness of the scoping search was tested using a set of key articles, all of which were found to be included in the search results. When the full search is carried out, eligible studies will be identified by searching the electronic databases listed below. These databases will be searched using a strategy incorporating a combination of MeSH and free-text terms. This strategy will be translated into each database using the appropriate controlled vocabulary as applicable.

Medline

Embase

Cochrane Library

Health Management Information Consortium Database (HMIC)

Web of Knowledge

Education Resources Information Centre (ERIC)

Open Grey

Other sources to be searched will include:

Hand searching of high-yield journals (Medical Teacher, Medical Education, Academic Medicine, Advances in Healthcare Science Education) and conference proceedings.

Reference lists of all papers and relevant reviews identified.

Contacting authors in the field via email to see if they are aware of any unpublished material.

It is anticipated that the electronic database searches will be completed by spring 2016. Limits on search: The search will be limited to English language only as none of the review team can read any foreign languages. No other restrictions will be applied.

6. Extracting data Following the search procedure described above, a list of abstracts will be obtained. These will be screened independently by at least two members of the review team and studies not meeting the inclusion criteria outlined in Section 4 will be discarded. We will perform inter-rater reliability checks on 20% of the abstracts screened to ensure consistency. Full text versions of the remaining studies will then be obtained. In the event of a disagreement over the inclusion of an abstracted paper, the full paper will be obtained and subjected to scrutiny by a third member of the review team. Obtaining the full text of the remaining papers will allow us to confirm that the study is relevant to the review. Studies found not to be relevant at this point will be discarded. Data from each of the resulting eligible papers will be extracted using the prepared coding sheet. This has been specifically adapted for this review from a BEME sample coding sheet. In order to refine the data extraction process, we will pilot the coding sheet before the formal extraction begins. Each group member will independently code the same 20 studies using the coding sheet. We will then meet to discuss any anomalies or inconsistencies in the data extracted. We expect that this process will result in modification of the coding sheet to maximise data extraction and consistency. Once we have refined our coding sheet, the formal process of data extraction will begin. Each paper will be coded independently by at least two members of the review group. If the two reviewers are unable to reach a consensus, a third member of the group will review the paper. We will carry out inter-rater reliability checks on 20% of the full text papers to ensure consistency in our coding.

7. The coding sheet and appraisal of studies The coding sheet will include the following sections:

citation information and search method used to find the paper

a description of methods and findings of the paper - population - study design - study aims and objectives - findings

QATSSD quality assessment tool

a grading of the strength of the paper’s findings

a grading of the study’s impact using a modified version of Kirkpatrick’s hierarchy

an overall impression of the paper From our scoping search, we anticipate that the studies included in this review will be methodologically diverse. As a result, we will use Sirriyeh et al.’s (2012) QATSSD quality assessment tool (see Appendix 2) to inform our appraisal of the quality of the included studies. We have chosen this tool over the MERSQI tool, used in Farnan et al.’s (2012) review, as it can only be used for quantitative data. The QATSSD tool has been developed specifically to assess a body of research that deals with the same research question using different methodological approaches. The QATSDD comprises of 16 quality criteria; 14 of these criteria apply to qualitative studies, 14 apply to quantitative studies and all 16 apply to any mixed methods papers. Each criterion is scored from 0-3, with a clear description of what is required to obtain each score. Using the descriptions for each score, we will score each paper from 0-3 for each item on the scoring grid. This will result in a score out of a maximum of 48 (16x3) for mixed methods papers, and 42 (14x3) for qualitative or quantitative papers. We will then grade the strength of each paper’s findings using the standard grading system outlined below. The resultant QATSDD score will inform our decisions. While the QATSSD has been shown to display good reliability and validity for use in the quality assessment of methodologically diverse papers (Sirriyeh et al. 2012), there has been no agreed cut-off score for quality. Consequently, a key component of our refinement process following our pilot data extraction will be to decide how we will use the QATSDD score to grade the included papers. This process will be outlined in detail in the methodology section of our review.

Standard grading system for strength of findings Grade 1 No clear conclusions can be drawn. Not significant.

Grade 2 Results ambiguous, but there appears to be a trend. Grade 3 Conclusions can probably be based on the results. Grade 4 Results are clear and likely to be accurate. Grade 5 Results are indisputable

We will also grade the level of impact being studied in the paper using a modified version of Kirkpatrick’s (1967) hierarchy, such as the model laid out by Issenberg et al. (2005) in their BEME review of high-fidelity medical simulations: Level 1 Participation in educational experiences Level 2a Modification of attitudes/perceptions Level 2b Modification of knowledge/skills Level 3 Behavioural change Level 4a Changes to professional practice Level 4b Benefit to patients Finally, there will be free text space for the reviewer to add any additional comments regarding the overall strengths and weaknesses of the paper. This process is summarised in the following flow chart:

Electronic searches carried out

Hand searching, reviewing reference lists, contacting authors and

searching grey literature carried out

Database of studies obtained and stored in RefWorks

All abstracts screened independently by 2 members of the review team.

Inter-rater reliability checks on 20% of screened abstracts

Studies not fulfilling inclusion criteria discarded. If discrepancies, full text to be obtained and viewed

by third reviewer.

Full text of included studies to be obtained. Reviewers to meet to confirm

relevance of all included papers

Each reviewer to independently code a sample of 20 included papers to pilot coding sheet and amend coding sheet

accordingly

Each paper to be coded independently by 2 reviewers. If inconsistency, paper to be reviewed by a 3rd reviewer. Inter-rater

reliability checks on 20% of papers.

Use coding sheet to determine QATSSD scre, grade of strength, level of impact

and overall impression of paper.

Discard lowest scoring studies (expected to be strength grades 1 and 2 and impact levels 1 and 2) and commence analysis.

8. Synthesis of evidence and transfer to research and practice Synthesis of evidence We are expecting the data we collate in this review to be diverse and heterogeneous, with both descriptive and qualitative components. Consequently, we will produce a narrative synthesis of the data, which is recommended for the initial synthesis of diverse data from a wide range of sources (Centre for Reviews and Dissemination, 2009). We will follow guidance from the Economic and Social Research Council (Popay et al., 2006) on how to conduct a narrative synthesis. There are four key steps in this process: 1. developing a theoretical model 2. developing a preliminary synthesis 3. exploring relationships in the data 4. assessing the robustness of the synthesis product Transfer to research and practice This review will add to the knowledge base on how best to provide clinical supervision so as to develop clinical competencies while maintaining patient safety. The body of evidence will help to inform the ongoing development of best practice guidelines for the clinical supervision of junior doctors within the UK training system. This is particularly timely given the changes currently happening to UK postgraduate medical training and education, all of which place considerable emphasis on the importance of clinical supervision and patient safety. We also expect the review to highlight gaps in the current evidence base and provide the basis for the development for further research questions. By undertaking this systematic review, we hope to identify evidence which can contribute to setting the direction for the future as currently envisioned by the profession and its regulators.

References Academy of Medical Educators. 2010. A framework for the professional development of postgraduate medical supervisors. London, AoME. Bell BM. 1993. Supervision, not regulation of hours, is the key to improving the quality of patient care. Journal of the American Medical Association. 269:403–4. Best Evidence Medical Education Collaboration. 2015. A BEME review: definition and description. Available from http://www.bemecollaboration.org/What+is+a+BEME+Review/ (accessed 20/12/12). Centre for Research and Developement. (2009). Systematic Reviews: CRD’s guidance for undertaking systematic reviews in health care (p. 292). York: Centre for Reviews and Dissemination. Brennan et al. 1991. Incidence of adverse events and negligence in hospitalized patients - Results of the Harvard Medical Practice Study. New England Journal of Med. 324:370–76. Cottrell D et al. 2002. What is effective supervision and how does it happen? A critical incident study. Medical Education. 36:1042–49. Department of Health. 2004. Modernising Medical Careers The Next Steps: The future shape of Foundation, Specialist and General Practice Training Programmes. London, Department of Health. Fallon WF, Wears RL, Tepas JJ. 1993. Resident supervision in the operating room: does this impact on outcome? Journal of Trauma. 35(4):556-560. Farnan JM et al. 2012. A systematic review: the effect of clinical supervision on patient and residency education outcomes. Academic Medicine 87(4):428-442. General Medical Council. 2012. Recognising and improving trainers: the implementation plan. London, GMC. General Medical Council. 2014. A reference guide for postgraduate speciality training in the UK (The Gold Guide). London, GMC. General Medical Council. 2105. Promoting excellence: standards for medical education and training. London, GMC. General Medical Council. 2015a. Certificate of completion of training. Available from http://www.gmc-uk.org/doctors/24629.asp (accessed 12/12/15)

Greenaway D. 2013. The shape of training: securing the future of excellent patient care. London, GMC. Gennis VM, Gennis MA. 1993 Supervision in the outpatient clinic: effects on teaching and patient care. Journal of General Internal Medicine. 8(7):378-80. Hinchey KT, Rothberg MB. 2010. Can Residents Learn to Be Good Doctors without Harming Patients? Journal of General Internal Medicine. 25(8):760-761. Itani KM et al. 2005. Surgical resident supervision in the operating room and outcomes of care in veterans affairs hospitals. American Journal of Surgery. 190(5):725-31 Kennedy TJ et al. 2005. Progressive independence in clinical training: a tradition worth defending? Academic Medicine. 80(10 suppl):S106-11. Kennedy TJ et al. 2007. Clinical Oversight: Conceptualizing the Relationship Between Supervision and Safety. Journal of General Internal Medicine. 22(8):1080-1085. Kilminster SM, Jolly BC. 2000. Effective supervision in clinical practice settings: a literature review. Medical Education. 34:827–40. Laurance J. 2015. Doctors' basic errors are killing 1,000 patients a month. The Independent. Available from .http://www.independent.co.uk/life-style/health-and-families/health-news/doctors-basic-errors-are-killing-1000-patients-a-month-7939674.html. Accessed 6/7/12. National Association of Clinical Tutors. Proposed best practice guideline for ongoing clinical supervision. Milton Keynes, NACT. National Health Service. 2014. Patient safety in the NHS. Available from http://www.nhs.uk/NHSEngland/thenhs/patient-safety/Pages/about-patient-safety.aspx (accessed 12/12/15). National Patient Safety Organisation. 2105. Patient safety resources: junior doctor. Available from: http://www.nrls.npsa.nhs.uk/juniordoctors/. (Accessed 24/11/15. Popay J et al. 2006. Guidance on the conduct of narrative synthesis in systematic reviews. Lancaster, ESRC Research Methods Programme Spritz, N. 1991. Oversight of physicians' conduct by state licensing agencies. Lessons from New York's Libby Zion case Annals of Internal Medicine. 115(3): 219–22.

Sox CM et al. 1998 The effect of supervision of residents on quality of care in five university-affiliated emergency departments. Academic Medicine. 73(7):776 82. Temple J. 2010. Time for training: a review of the impact of the European Working Time Directive on the quality of training. Medical Education England. Available from http://hee.nhs.uk/healtheducationengland/files/2012/08/Time-for-training-report.pdf (accessed 6/7/2015) UK Foundation Programme Office. 2014. The UK Foundation Programme Curriculum. Velmahos GC et al. 2001. Around the clock attending radiology coverage is essential to avoid mistakes in the care of trauma patients. American surgeon. 67(12):1175-7. Word Health Organisation. 2015. Patient safety. Available from http://www.who.int/patientsafety/en/ (accessed 13/11/15).

Appendix 1

Initial scoping search strategy used, piloted across Medline:

1. Medline; exp INTERNSHIP AND RESIDENCY/; 37797 results. 2. Medline; EDUCATION, MEDICAL, GRADUATE/; 22997 results. 3. Medline; (doctor* ADJ3 (postgrad* OR post-grad* OR graduate OR train* OR junior OR foundation* OR new* OR F1 OR F2 OR "first year*")).ti,ab; 6836 results. 4. Medline; (clinician* ADJ3 (postgrad* OR post-grad* OR graduate OR train* OR junior OR foundation* OR new* OR F1 OR F2 OR "first year*")).ti,ab; 3548 results. 5. Medline; (physician* ADJ3 (postgrad* OR post-grad* OR graduate OR train* OR junior OR foundation* OR new* OR F1 OR F2 OR "first year*")).ti,ab; 14856 results. 6. Medline; (medical ADJ3 (postgrad* OR post-grad* OR graduate OR train* OR junior OR foundation)).ti,ab; 21533 results. 7. Medline; "house officer*".ti,ab; 1670 results. 8. Medline; housestaff.ti,ab; 680 results. 9. Medline; registrar*.ti,ab; 2698 results. 10. Medline; resident*.ti,ab; 125471 results. 11. Medline; residency.ti,ab; 18057 results. 12. Medline; intern*.ti,ab; 685689 results. 13. Medline; "SAS doctor*".ti,ab; 3 results. 14. Medline; "specialty and associate specialist".ti,ab; 2 results. 15. Medline; "staff and associate specialist".ti,ab; 5 results. 16. Medline; 1 OR 2 OR 3 OR 4 OR 5 OR 6 OR 7 OR 8 OR 9 OR 10 OR 11 OR 12 OR 13 OR 14 OR 15; 871308 results. 17. Medline; supervis*.ti,ab; 42806 results. 18. Medline; oversight.ti,ab; 3793 results. 19. Medline; 17 OR 18; 46477 results. 20. Medline; PATIENT SAFETY/; 7977 results. 21. Medline; (patient* ADJ2 safe*).ti,ab; 36197 results. 22. Medline; PATIENT CARE/; 7665 results. 23. Medline; "patient care".ti,ab; 42777 results. 24. Medline; CLINICAL COMPETENCE/; 70182 results. 25. Medline; "clinical competenc*".ti,ab; 2034 results. 26. Medline; QUALITY OF HEALTH CARE/; 60054 results. 27. Medline; PROFESSIONAL AUTONOMY/; 8719 results. 28. Medline; autonom*.ti,ab; 98031 results. 29. Medline; DECISION MAKING/; 72661 results. 30. Medline; "decision making".ti,ab; 76992 results. 31. Medline; BURNOUT, PROFESSIONAL/; 7956 results. 32. Medline; (burnout OR resilience OR "well being").ti,ab; 59124 results. 33. Medline; PROFESSIONAL PRACTICE/; 15450 results. 34. Medline; "professional practice".ti,ab; 3496 results.

35. Medline; professionalism.ti,ab; 5078 results. 36. Medline; TEACHING/; 43500 results. 37. Medline; FACULTY, MEDICAL/; 10549 results. 38. Medline; ((faculty OR educator OR professional OR academic) ADJ development).ti,ab; 7431 results. 39. Medline; ((faculty OR educator OR supervis*) ADJ train*).ti,ab; 1382 results. 40. Medline; (supervis* ADJ (skill* OR practice)).ti,ab; 369 results. 41. Medline; 20 OR 21 OR 22 OR 23 OR 24 OR 25 OR 26 OR 27 OR 28 OR 29 OR 30 OR 31 OR 32 OR 33 OR 34 OR 35 OR 36 OR 37 OR 38 OR 39 OR 40; 552451 results. 42. Medline; 16 AND 19 AND 41; 2780 results. 43. Medline; 42 [Limit to: (Language English)]; 2597 results.

Appendix 2

Coding sheet 1. Citation information Paper number:___________________ Reviewer: ____________________

Type: ☐Book ☐Interview ☐Non-peer reviewed article

☐Comment ☐Journal article ☐Official publication

☐Conf. paper ☐Lecture ☐Report

☐Editorial ☐Letter ☐Statistical data

☐Guidelines ☐News ☐Thesis

Authors: ___________________________________________________ Title: ___________________________________________________

Publication: ___________________________________________________ 2. Study description a. Population Country: ______________________________________________________ Location: ______________________________________________________ Level of doctor: _________________________________________________ Speciality: _____________________________________________________ Number of participants: ___________________________________________ b. Study design Type of study: __________________________________________________ Data collection methods: _________________________________________

Type of supervision/intervention: ___________________________________

c. Aims and objectives Stated aims and objectives: _______________________________________

d. Study findings Description of findings: ___________________________________________ ______________________________________________________________ ______________________________________________________________ 3. QATSSD Tool

Total score:

4. Strength of findings

☐Grade 1 No clear conclusions can be drawn. Not significant.

☐Grade 2 Results ambiguous, but there appears to be a trend.

☐Grade 3 Conclusions can probably be based on the results.

☐Grade 4 Results are clear and likely to be accurate.

☐Grade 5 Results are indisputable

5. Impact of intervention being studied

☐Level 1 Participation in educational experiences

☐Level 2a Modification of attitudes/perceptions

☐Level 2b Modification of knowledge/skills

☐Level 3 Behavioural change

☐Level 4a Changes to professional practice

☐Level 4b Benefit to patients 6. Overall impression of paper