1

COMP 1 2013/14

Child Health, Public Health Medicine and Evidence Based Medicine

Assessment Criteria

University of Bristol Medical School MB ChB: Year 4

2

Community Oriented Medical Practice 1

Assessment Information

Contents

Summary of Assessments ............................................................................................................ 3

Pass marks and standard setting ................................................................................................. 3

COMP1 Prize ............................................................................................................................... 4

Re-sits ....................................................................................................................................... 4-5

Assessments Blueprint ................................................................................................................. 6

Feedback ..................................................................................................................................... 7

Assessment Regulations .............................................................................................................. 7

Extenuating Circumstances .......................................................................................................... 7

Plagiarism .................................................................................................................................... 7

Appendix A - The COMP1 internal SSC………………………………………………………………………8 Appendix B - Calculations expected for the exam………………………………………………………….14

3

Assessment Information

Summary of Assessments

COMP1 learning objectives are assessed through an OSCE (Observed Structured Clinical Examination), Written Examination and SSC.

1. SSC written project 20% (combined ‘Public Health & Evidence Based Medicine’ and ‘Child Health’ - will be assessed by individuals from both disciplines)

2. OSCE (Child Health: clinical examination) 30% 3. Written examination 50%

(combined ‘Public Health & Evidence Based Medicine’ and ‘Child Health’; half of the questions will be Child Health and half Public Health & Evidence Based Medicine).

In line with medical school policy, negative marking is no longer used in the written exam.

Pass marks and standard setting

In order to achieve an overall pass, a student must achieve the pass mark or better on each of the components of the assessment. A student who fails to achieve the pass mark in one or more components of the COMP1 assessment will be required to re-sit the whole or part of COMP 1 assessment (see below) See the marking scheme (blue folders) for details of how the SSC is graded. The pass marks for the written exam and OSCE are determined each time by standard setting methods, taking into account the difficulty of the particular assessment, so that those pass marks indicate that a minimum standard of competence has been achieved. The grades on each component (A to E on the SSC, pass/fail on the OSCE and written exam) are then transferred to a common scale so that 50% or more on each assessment indicates a pass. In 2009 the rescaling of raw marks to the common scale was as presented in the table below. In February 2009, for example, the pass marks on the raw scale were 45% for the exam and 56% for the OSCE which were both rescaled to a pass mark of 50%. Rescaling the marks in this way does not affect whether a student passes or fails the assessment, and does not affect the ranking of students on any particular assessment.

Further details of assessment are available on blackboard

4

GRADE Common

Scale EXAM July09

OSCE July09

EXAM Feb09

OSCE Feb09

E FAIL <50% <43% <56% <45% <56%

D BARE PASS 50-54% 43-50% 56-59% 45-52% 56-59%

C CLEAR PASS 55-59% 51-59% 60-64% 53-59% 60-64%

B GOOD PASS 60-64% 60-68% 65-68% 60-69% 65-68%

A VERY GOOD PASS 65-74% 69-76% 69-73% 70-76% 69-72%

A* EXCELLENT PASS 75%+ >76% >73% >76% >72%

In line with medical school policy, we do NOT conduct pass/fail vivas.

COMP1 Prize On the basis of the assessment results, students with the highest aggregate marks will be eligible for a prize.

Re-sits Other than in exceptional circumstance, and by arrangement with the Director of Student Affairs, students re-taking the whole of year 4, as a first or second attempt, will retake all components of the COMP1 assessment. For students required to retake COMP1 only in September, the following table indicates the retake requirements normally expected of students failing specific components of the COMP1 assessment. The specific requirements for an individual student will be confirmed by the Faculty Exam Board in each case. Students in this situation may be required to do an external SSC in the area of COMP1.

5

WRITTEN EXAM RESULT

SSC RESULT

OSCE RESULT

Clear pass+ Bare pass Fail

Pass Clear Pass+

N/A N/A Written exam

Pass Bare pass

N/A N/A Written exam

OSCE

Pass Fail OSCE

Written exam

OSCE

Written exam

OSCE

Fail Clear Pass+

SSC SSC Written exam

SSC

Fail Bare pass

SSC

SSC

Written exam

OSCE, SSC

Fail Fail OSCE

SSC

Written exam

OSCE, SSC

Written exam

OSCE, SSC

Resit requirements for those retaking one or more COMP1 components (clinical exam, written exam, SSC) will be detailed in a letter from the exam board. A student who fails one or more components of the COMP1 assessment, and then achieves a pass on re-sitting those components as a “second attempt” will have their overall COMP1 mark capped at 50% on the common scale. A student who fails one or more components of the COMP1 assessment may be allowed to re-sit those components as a “first attempt”, if there were extenuating circumstances at the time. In this case the overall COMP1 mark will not be capped. Students must follow the Faculty procedure for having extenuating circumstances considered by the exam board (see Assessment Regulations below).

6

Assessments Blueprint *Assessment of knowledge, skills, attitude

Major curriculum outcomes/learning outcomes

*K,S,A Interaction with vertical

themes*

Written assessments (MCQ, Essay,

EMQ)

SSCs (Oral,

written)

OSCEs

PH and EBM

Formulate answerable clinical qu’s K,S EBM

Search the literature K,S EBM

Interpret statistical information K EBM

Critically appraise available evidence K EBM

Systematic reviews, Observational study design + RCTs

K EBM

Appraise diagnostic tests, screening and prognosis

K EBM

Sensitivity and specificity K EBM

Define public health and approaches to disease prevention

K EBM

Health determinants and strategies K EBM

Infectious disease epidemiology, control, and immunisation

K EBM

Qualitative research and methods K EBM

health economic evaluation and quality of life

K EBM

Genetic epidemiology K EBM

CHILD HEALTH

Normal c’hood growth developments and its common variants

K

Developmental influences on the presentation, spectrum and natural history of diseases of childhood

K

Common problems of childhood, their assessment and treatment

K

Overview of immunisations and childhood infections

K

Management of common diseases of c’hood.

K

Practical examination skills K,S,A CSS/WPC

Examine and communicate with children of different ages in an age appropriate manner.

K,S,A CSS/WPC

Take a comprehensive paediatric history K,S,A CSS/WPC

Recognise serious illness + undertake basic paediatric life support. K,S,A

Complete a simple prescription K,S,A

Medical, social, psychological effects of chronic illness on child and family. K WPC

7

Principles of prevention in child health K

Common psychological problems of childhood and their management

K

Child protection K DDD

Behaviour problems in young children K DDD

Principles: genetics and inherited disease. K

Mental Health problems in ages 5 -11 K DDD

Epidemiology of childhood disease K

Adolescents + transition to adulthood K

Aspects of international child health K

Disability K DDD

* : Vertical Themes Ethics & Law EML, Communication Skills CSS, Evidence Based Medicine EBM, Disability diversity and difference DDD, Whole Person Care WPC, Personal and interprofessional development PAID

Feedback

We invite comments on the content and lectures at any time. These comments can be submitted either verbally or by e-mail to the unit or element leads. We conduct an annual review of all lectures, and hold a focus group with a group of students to reflect on the course and make suggestions on how we can improve the course. We encourage all students to submit feedback on the Unit through the official Faculty survey on Blackboard. We reflect on this feedback and use it as a basis for improving the Unit where appropriate.

We have a monitored online discussion board where we encourage students to post any questions related to unit content. We aim to respond to questions within 48 hours. OSCE examiners will give feedback to students at the end of completion of the OSCE exam. Written feedback on the SSC is given on blackboard. Students that fail the unit are invited for a one to one discussion, including feedback on their written exam.

Assessment Regulations

You must adhere to all relevant regulations relating to the MB ChB. University Examinations regulations can be found at: http://www.bris.ac.uk/secretary/studentrulesregs/examregs.html.

Medical School rules can be found at:

http://www.bris.ac.uk/medical-school/staffstudents/rulesandpolicies/

Extenuating Circumstances: If before or during an assessment you feel that your performance has been affected by personal, family or health problems you should complete an extenuating circumstances form. For more information please see:

http://www.bris.ac.uk/medical-school/staffstudents/student/excircs/

Plagiarism: Passages quoted or closely paraphrased from other authors must be identified as quotations or paraphrases, and the sources of the quoted or paraphrased material must be acknowledged. Use of unacknowledged sources may be construed as plagiarism. More information about plagiarism can be found at: http://www.bristol.ac.uk/library/support/findinginfo/plagiarism/

8

Appendix A - The COMP1 internal SSC 2013/14

Aim: To allow students to practice and demonstrate their skills in critically appraising published

research evidence, and to appropriately use that evidence to inform a hypothetical clinical decision in

the area of child health.

Marks are gained in this assessment by demonstrating an understanding of published research

evidence, an ability to critically appraise that evidence, and an ability to use it to inform a hypothetical

clinical decision. While an understanding of the child health topic chosen will no doubt contribute to a

well-written report, there are no marks available in this assessment for demonstrating that

understanding of child health.

Summary of the rules

Please submit your clinical scenario to the Blackboard discussion board by 5pm on Friday of

Week 4 so that it can be approved. Two marks (5%) will be deducted for late submission

without prior arrangement with an element or the unit lead.

Please choose a clinical scenario, requiring a decision, in the area of child health. Two marks

(5%) will be deducted if you proceed with an unapproved clinical scenario that is not clearly in

the area of child health.

Once a SSC scenario has been submitted it cannot be changed, other than in exceptional

circumstances with the agreement of an element lead. Two marks (5%) will be deducted if the

scenario addressed in your report is clearly different to the scenario originally submitted (e.g.

clearly not the same patient).

Work must be submitted before the stated deadline, using the Blackboard submission point

provided. When submitted late, four marks (10%) will be deducted if the report is submitted

within one week of the deadline and forty marks (100%) will be deducted if the report is

submitted more than one week after the deadline.

Your written report must not be more that 3000 words in total, as determined by the MS Word

tool. This word count includes everything in your submitted document. One mark will be

deducted for every 25 words (or part thereof) in excess of the word limit up to a maximum

penalty of four marks (10%).

Any material included as an embedded image file will not be marked.

Plagiarism will be dealt with by following the University procedures.

Deadline for submission: 12 Noon on Friday of week 8.

The dates for topic and report submission will be on Blackboard for each round. Late submission will

be penalized, as described above.

Penalties for late submission may be removed by the extenuating circumstances panel. If you have

extenuating circumstances please follow the Faculty procedure for having these considered by the

exam board. Unit organisers cannot grant extensions.

The student is responsible for contacting the public health & EBM administrator WELL BEFORE THE

DEADLINE in order to resolve any difficulties submitting the SSC using the Blackboard submission

point. Late penalties will not be discounted for students who left submission to the last minute

and then encountered technical difficulties.

9

1. Format for the written report

Word count: Your written report should be no more than 3000 words. This word limit includes

everything that forms a part of your submission including the study question, Medline search strategy,

tables, appendices and references. You are required to provide a word count; however, we will check

the word count of all projects electronically. We will automatically deduct one mark for every 25 words

(or part thereof) in excess of the word limit, up to a maximum penalty of 4 marks.

Embedded image files: Please do not embed image files in your report, e.g. a screen shot of your

Medline search strategy. Any material included as an embedded image file will not be marked.

Written report: Your report should normally include the following. ALL material in your report will be

included in the word count.

A cover sheet that includes your name, short title, and the word count.

The clinical scenario and question posed. This must be in an area of Child Health

PICO formulation of the question

Your search strategy and results of the search

A flow chart describing how many references were found, how many were relevant to the

clinical question and a brief indication of why excluded references were not relevant, and how

many met quality criteria (e.g. the right study design) and brief reasons for excluded references

not being of sufficient quality.

Justification for the choice of 3 papers.

A brief description of the methods and results for each of the 3 papers. Critical appraisal of the

3 papers.

Overall synthesis of the evidence with a reasonable conclusion in response to the clinical

question

A lay summary of the evidence explaining the implications, using words that you would use to

answer the question posed by the patient, parent or other non-medical person.

References and web-links for the 3 papers that have been critically appraised, plus references

for any other papers referred to.

Remember the SSC has to be your own work. All SSC reports are checked for plagiarism. Copying

text directly from papers without acknowledgement is plagiarism and will be treated in the same way

as copying from other students. Passages quoted or closely paraphrased from other authors must be

identified as quotations or paraphrases, and the sources of the quoted or paraphrased material must

be acknowledged. Use of unacknowledged sources may be construed as plagiarism. More

information about plagiarism can be found at:

http://www.bristol.ac.uk/library/support/findinginfo/plagiarism/

Double marking: A random 10% of the reports will be double-marked to look for variance between

markers.

Feedback: Written feedback will be available on Blackboard with mark out of 40. These marks are then rescaled as indicated in this table, to accommodate the wider range of marks awarded in the SSC compared to the DOCS and written exam.

10

Table - Grading of the SSC

Grade

Marks (max 40)

Re-scaled to common marking scale range

A* 34 and above 75%+

A 28 to 33 65-74%

B 24 to 27 60-64%

C 20 to 23 55-59%

D 18 or 19 50-54%

E 17 and below <50%

Assessment criteria for SSC marking (half marks can be given as relevant)

Section

Key things to look for

Total for

section

Search

strategy and

abstracts

Appropriate formulation of a scenario, clinical question

and formulation of PICO question (2 marks)

Appropriate databases searched using reasonable

strategy with MESH and text words as required (2

marks)

Flowchart of references identified by search strategy,

subset relevant, subset of sufficient quality, and

reasons for exclusion at each stage (2 marks)

Reasonable justification for selection of papers to

appraise (2 marks)

8 marks

Key aspects

of 3 papers

Adequate description of key aspects the study (2

marks), including identification and interpretation of

the key results (2 marks)

4 marks

for each

of the 3

papers

Critical

appraisal of 3

papers

Identifies the key strengths and weaknesses of the

study, and describes how each of these aspects of

the study strengthen or weaken the conclusions that

can be drawn in addressing the PICO question (4

marks) Marks are only gained if it is explained

HOW each aspect strengthens or weakens the

conclusions that can be drawn in addressing the

clinical question.

Addressing the key strengths and weaknesses for the

particular study design (1 mark)

5 marks

for each

of the 3

papers

Synthesis of

evidence and

lay summary.

References.

Arrive at reasoned conclusions from the evidence (2

marks)

Ability to answer the clinical question, in lay terms as

used when responding to the patient. (2 marks)

Appropriate referencing & providing weblinks (1 mark)

5 marks

11

Further guidance on marking the SSC

The SSC accounts for 20% of the total COMP1 mark. It is an in-course assessment of competency in

using published empirical evidence to address clinical questions and inform decision-making.

The marking scheme is given in the “Format for the written report” document. The following is an

example of how the marks available can be allocated. These are not fixed criteria, simply a guide

aimed at increasing consistency across markers. Markers will still need to judge how well a student

has addressed each aspect of this SSC, awarding marks in light of that judgement.

PICO question Half mark each for adequate description of patient, intervention, comparison and outcome.

Search strategy Half mark each for reasonable use of text words, MeSH terms, Boolean operators, and appropriate databases / alternative sources (e.g. review reference lists).

Flow chart 1 mark for flowchart with it being clear how many abstracts were excluded at each stage. 1 mark for listing, in appropriate detail, reasons exclusion due to lack of relevance and insufficient quality.

Paper selection Need to justify choice of each paper in turn with reasons such as the hierarchy of evidence for causality, recency, precision, availability, relevance of patients and outcomes. Say half mark per paper, with the full two marks awarded if choice of all three papers is competently justified.

Key aspects of study One mark for picking out the key result(s) relevant to the study question, and presenting this with effect size, confidence interval and p-values (or noting that these are not available in the paper, with relevant summary statistics presented if possible). One mark for appropriate interpretation of the presented results. Two marks for describing the study, making it clear (for example) what patient group is being studied, how many patients are in the relevant study arms, what intervention is being compared to what, and how and when the relevant outcome variables are being measured. Similar for systematic reviews, with a brief description of how the literature search was conducted (main search terms and databases).

Strengths and weaknesses (S/W) Normally one mark for each strength or weakness with a reasonable explanation of why it affects the conclusions that can be drawn in response to the clinical question. Half mark if the explanation is correct but vague (e.g. “to control confounding”), but the total mark for this section should be the sum of the four best marks for S/Ws (e.g. eight half marks would sum to 2). Exceptionally, 2 marks can be given for a single S/W if it is beyond what would be expected of a student on the COMP1 course, although the total mark for this section is capped at 5 per paper. If the student makes no attempt at explaining why a S/W is important, or the explanation is wrong, then that S/W does not earn any marks. One mark for including the key strengths and weaknesses for a study design, e.g. allocation concealment, blinding, and completeness of follow-up (ITT) would each need to be considered for an RCT to earn this mark. This mark is to credit students who are aware of the particularly important strengths and weaknesses, with an extra mark compared to what can be earned by a correct by relatively idiosyncratic set of strengths and weaknesses.

Conclusions from the three papers

One mark for a reasonable conclusion from the three papers (even if this is at odds with what would

have been concluded from a full blown systematic review). One mark for a reasonable indication of the

level of confidence we can have in this conclusion, given an overall appraisal of the strengths and

weaknesses of the three studies. Or ALTERNATIVELY, one mark for putting the conclusion in the

context of broader concerns, such as the feasibility of implementing the intervention.

12

Lay summary

One mark if the language used is appropriate for a lay person. One mark for a practical

recommendation / advice in response to the clinical question, and suitable cautions given in light of the

quality of the evidence in the three papers.

References

One mark if both weblinks and standard citations are given for each of the three papers.

3. Frequently asked questions (FAQ’s) for the SSC

Q1. Should I get started with writing my SSC report as soon as possible? No. To help you co-ordinate your SSC work and clinical placements we strongly encourage the following timetable:

WEEKS 1, 2, 3: Sessions which help choose a clinical question to address in your SSC are timetabled in week 1, so that you can think about your question during weeks 2 and 3 in the academies. Your child health log book includes an “educational prescription” which allows you to record clinical questions that arise during your placement, and discuss these with the academy tutors as appropriate. You are advised to have two or three possible questions, in case somebody else in your block has a similar question to one of yours approved first. It will not be possible to have your question approved if you submit it before 1:30pm on Monday of Week 4, and in weeks 2 and 3 you should not do more than formulate your questions and check relevant studies are available in an initial literature search.

WEEK 4: A Blackboard discussion board will be made available at 1:30pm on Monday of Week 4 for you to submit your question. You need to do this before 5pm on Friday of Week 4. The discussion board will be checked at least once every 24 hours and questions approved. You should wait to see if your question is approved before working on it; students do occasionally need to choose a new question, usually because their original choice had already been approved for somebody else in the same block. Once you have your question approved you are encouraged to try and do your literature search and select your papers during the remainder of week 4. The EBM sessions timetabled during this week give essential information for conducting competent critical appraisal. Attendance as the SSC Group Tutorials during this week is compulsory, and will usually focus on question choice and literature searching.

WEEKS 5, 6, 7: You will need to work on your critical appraisal during these weeks, but you are responsible for identifying time slots which do not conflict with you academy commitments. If you need tutor support during this time, you need to post your queries on the Blackboard discussion board in the COMP1 area. Week 7 concludes with a long weekend in Bristol, with the chance to book a 15 minute appointment with one of the EBM tutors, and self-directed time to work on your report. If you have finished your report, then you are welcome to remain in your academy for this time, though you must come to Bristol to attend the Genetic Epidemiology sessions (usually Monday morning of week 8).

WEEK 8: The deadline for submitting your report is Friday 12 Noon of Week 8. The date this falls on will be announced on Blackboard for each block. Q 2. Once I have submitted my scenario, can I change it? No, other than in exceptional circumstances with the agreement of one of the element leads. Students are advised to do a preliminary search of the literature to make sure that sufficient relevant studies are available in the area they are considering. Whilst the scenario and question cannot be changed, it is perfectly okay to refine the question into an answerable form when expressing it in the PICO format. This may include changes in the exact intervention being considered, changes in the choice of a sensible comparison, and selection of one or two relevant outcomes.

13

Q 3. What is the total number of full papers that we have to critically appraise and what if they are systematic reviews? The total number of papers that you have to appraise in full is THREE. These three papers could be based on individual studies or be systematic reviews; however, a systematic review paper still counts for one paper only. If the individual paper that you want to review is also considered in the systematic review that is not a problem. Q 4. Should I choose three different types of studies? No; when justifying your choice of papers you will get marks for showing you selected what looked to be the best evidence available. There are two caveats to this: (i) it would rarely be justifiable to choose more than one systematic review paper; (ii) an observational study may be chosen over an RCT if, for example, you were interested in rare treatment harms.

Q 5. Who is my tutor? This SSC is different from other SSCs that you are used to in that there is no allocated tutor. You are provided with opportunities in your timetable to ask questions of the lecturers and clarify your doubts. You are expected to use these opportunities or to discuss amongst each other on the discussion board provided on Blackboard. Please view this SSC as an assignment and an opportunity to practice by yourself the skills you will be taught during the course.

Q 6. How long should my MEDLINE search strategy be and does it count towards the word count in the report? The actual search strategy can be quite long but in the report you are only expected to show a shortened version of it to demonstrate that you understand the key concepts. Typically, most search strategies can be summarised to 8-10 lines. You are expected to include this in the body of your SSC and it is included in the word count.

Q 7. Can I ask the librarians for help with writing my search strategy? No, the librarians are only able to offer general advice during their timetabled sessions. Please raise any specific problems you are having with your search strategy on Blackboard, or during one of the timetabled tutorials with the COMP1 tutors.

Q 8. What if the journal carrying my paper is not available electronically?

If the journal article is not available electronically but there is a paper copy in one of the Bristol

University / NHS libraries, you are expected to go and find the article. As a University of Bristol student

you should be able to visit these libraries and copy a paper but ring the librarian to check first. You are

not expected to obtain inter-library loans.

14

Appendix B - Calculations expected for the exam

In the exam you will be expected to be able to use all of the statistical ideas covered in the first and fourth year courses for the purposes of critical appraisal. You will also be required to do some calculations, and so you should arrive at the exam able to calculate the following.

From the first year Space for revision notes

Prevalence

Incidence – risk

Incidence – rate

Risk ratio or rate ratio

Risk difference or rate difference

Number needed to treat

Odds and odds ratio

Standard error of a single mean (given the standard deviation)

Standard error of a single proportion

95% Confidence interval for a single mean, and a single proportion.

95% Confidence interval for a difference in means, or a difference in proportions (standard errors will be provided)

Reference range for a single mean

Please turn over ...

15

New in the fourth year Space for revision notes

Incremental cost-effectiveness ratio

Net benefit statistic

Quality adjusted life year

Likelihood ratio

Positive predictive value and negative predictive value

Sensitivity and specificity

Standardized Mortality Ratio (SMR)