| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea1 |

ICD-11Chapter 23 Field Trial pilot phase

ICD-11Chapter 23 Field Trial pilot phase

Nenad KostanjsekClassifications, Terminologies, Standards (CTS)

Meeting on ICD-11 Chapter 23 Field Trial pilot phase25-26 March 2013, Seoul, Korea

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea2 |

Draft programDay 1. (Korean ICTM Working Group meeting)

Draft programDay 1. (Korean ICTM Working Group meeting)

Review & discussion of overall FT pilot phase incl. analysis plan

Review & discussion of frequency analysis and draft case summaries

Review & discussion of procedure and instruments for Study 1.Reliability and Feasibility

– Form I

– Form J

– Form E

– PI Form

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea3 |

Draft program Day 2, FT site Kyung Hee University Hospital

Draft program Day 2, FT site Kyung Hee University Hospital

FT site coordinator(s) to familiarise group of selected clinicians/coders (i.e. TM residents)

–    Familiarization with ICD Chapter 23

–    Briefing on Study 1 instruments

–    Familiarization with interviewing techniques

Coders perform Live case evaluations (which are video recorded if possible)

Coders perform CS evaluationsDiscussion and evaluation

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea4 |

ICD-11 Field TrialsICD-11 Field Trials

Basic aims– To test the “fitness of ICD-11 for multiple purposes”

• Mortality coding• Morbidity coding• Other use cases

– To ensure the comparability between ICD-10 and ICD-11– To increase consistency, identify improvement paths, and reduce errors

. Key Assessments:

• Applicability – feasibility easy to use

• Reliability - consistency gives same results in the hands of all

• Utility - added value renders useful information

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea5 |

ICD-11 Field TrialsICD-11 Field Trials

Applicability (Feasibility) – – Is the classification easy to implement in the hands of the real life users (coders, doctors etc.) ?

Utility – – What is the value of the classification to enhancing data capture and its uses?– Does it improve recognition? – Does it serve for better documentation? – Does it enable re-use? – Does it guide better diagnosis? – Does it allow better resource allocation?

Reliability – – Is the classification used in the same manner by different users? – Do two different users code the same case with the same code? – What are the sources of discrepancy? – What are the factors to improve comparability and consistency?

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea6 |

Field Trial StudiesField Trial Studies

1. Core Studies (mandatory)– Study One:

• Reliability & Feasibility– Study Two:

• Bridge Coding– Study Three:

• Basic Questions

2. Additional Studies– e.g. testing index

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea7 |

Field TrialsField Trials

KEY USES: – Mortality: cause of death coding, verbal autopsy – Morbidity: various morbidity codings – hospital discharge, DRG etc.– Other uses 

DIFFERENT SETTINGS: – Primary Care

• High-resource settings• Low-resource settings

– General Health Care• Specialty settings

– Research settings• Use in population studies - epidemiology• Use in clinical research

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea8 |

Field Trials Chapter 23Pilot Phase & Main Phase

Field Trials Chapter 23Pilot Phase & Main Phase

Pilot Phase is about testing the FT procedure and the instruments and understand what works & what does not work

Things that don’t work in the pilot phase can be put aside.

Things that work in the pilot phase can be used in the main phase

Pilot Phase is conducted in selected countries, Main Phase is implemented in multiple countires

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea9 |

Management of FT pilot phase (1) Management of FT pilot phase (1)

FT Structure– WHO– Field Trial Centres– Field Trial Sites

Roles & Responsibilities– Coder/rater– FT site coordinator & assistant– FT center coordinator & assistant

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea10 |

Management of FT pilot phase (2)Management of FT pilot phase (2)

Data management– Infrastructure

• (web-base software (multilingual) & Paper and pencil– Workflow

• Entry, check/validation/submission

Ethics review (e.g. WHO ERC, national ERC)– Check requirements– If review & clearance is needed stress that:

“in the FT we are capturing how clinician/raters represent their diagnosis in ICD- 11 Chapter 23 codes. FT do not interfere with the diagnostic procedure”

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea11 |

Study 1: Inter-rater reliabilityStudy 1: Inter-rater reliability

The Case information• Case Summaries (CS)• Live Cases (LC)• Other (e.g. Video Cases)

Coded using ICD-11 Chapter by at least two different people

Agreement rates measured

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea12 |

Case Assessments can include (simulate or real)…

Case Assessments can include (simulate or real)…

Patient history

Tongue examination

Pulse examinations

Physical examinations

| ICD-11 Ch. 23 FT pilot phase Meeting, 25-26 March 2013, Seoul, Korea13 |

Analysis Analysis

Level of agreement among multiple raters

– Who agrees on what?– Reason for disagreement

Predictors of reliability

Testing of hypothesis

Data patterns

High Agreement

Case 1, 2, 3…

Low Agreement