Three model curricula for teaching clinicians to use the ICF

Three model curricula for teaching clinicians to use the ICF

GEOFFREY M. REED1, KAREN DILFER2, LYNN F. BUFKA3, MARCIA J. SCHERER4,

PHIA KOTZE5, MALUTA TSHIVHASE6 & SUSAN L. STARK2

1International Union of Psychological Science, Madrid, Spain, 2Programme in Occupational Therapy, Washington University

School of Medicine, St. Louis, Missouri, USA, 3American Psychological Association, Washington, DC, USA, 4The Institute

for Matching Person and Technology, Webster, New York, USA, 5HWH Integrated Human Capital, Woodmead, South

Africa, and 6National Department of Health, Pretoria, South Africa

AbstractPurpose. Three systematic programmes to train health professionals to use the World Health Organization’s InternationalClassification of Functioning, Disability, and Health (ICF) are described, along with efforts to evaluate their effectiveness.Methods. The first programme was a randomized study comparing the effects of a 2-hour instructor-led programme and aself-directed learning module on ICF-related knowledge, attitudes, and coding skills among occupational therapy graduatestudents. The second programme was a series of intensive 3.5-day workshops for public sector rehabilitation professionals inSouth Africa. The third programme involved a series of internet-based teaching modules regarding the ICF for graduatestudents in rehabilitation counselling.Results. The first project found that both instructor-led and self-guided training formats were effective in improving basicICF-related knowledge, but only instructor-led training led to a significant improvement in coding skill. It also had morepositive effects on ICF-related attitudes. This approach to learning assessment was generalizable to multidisciplinary healthprofessionals in South Africa, who achieved a relatively high degree of coding accuracy after the 3.5-day workshop.Participant evaluations supported the structure, content, and length of the training. Students in the third programme alsoreported a very positive learning experience and positive views of the ICF.Conclusions. An empirical basis is important for identifying the best and most efficient training methods for particularaudiences and specific purposes. The length and format of training can be differentially related to specific training goals (i.e.,knowledge, attitudes, and coding skills). Interactive distance learning methods may help to overcome the weaknesses of self-directed training in comparison to face-to-face training.

Keywords: International Classification of Functioning, Disability, and Health (ICF), instructor-led programme, self-directedlearning module, internet-based teaching modules, rehabilitation training

Introduction

Health care settings have traditionally been con-

cerned with two types of classification. Diagnostic

classification, most commonly based on the World

Health Organization’s International Statistical Clas-

sification of Diseases and Health Related Problems

(ICD-10) [1] is used to identify the illness, injury, or

other condition that the patient has. Classification of

procedures, which in the US is most commonly

based on the Current Procedural Terminology

(CPT) codes [2] is used to identify the specific

services or treatments that the patient receives.

Together, these coding systems direct billing and

reimbursement and allow examination of such

questions as whether the treatment provided is

appropriate to the condition the patient has.

The conceptualizations of illness and health

system functioning that underlie the near-exclusive

use of these types of classifications are based on a

medical model of the acute treatment process. That

is, the primary tasks for clinicians are first to identify

correctly the pathogen or nature of the injury or other

disease process, and second to administer the

appropriate treatment for that particular condition.

This model has been appropriate historically for

Correspondence: Geoffrey M. Reed, PhD, International Union of Psychological Science, Glorieta de Bilbao, 5, 4to 4ta, 28004, Madrid, Spain.

E-mail: [email protected]

Disability and Rehabilitation, 2008; 30(12 – 13): 927 – 941

ISSN 0963-8288 print/ISSN 1464-5165 online ª 2008 Informa UK Ltd.

DOI: 10.1080/09638280701800301

Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

meeting the challenges associated with infectious

illness, where rapid identification and highly targeted

treatments are the keys to success. Information other

than the nature of the patient’s condition and the

treatments provided is largely irrelevant to the

clinical process.

However, the health care demands of the world’s

population are changing. Chronic, disabling condi-

tions, including mental disorders and injuries that

have lasting, and pervasive functional consequences

(e.g., spinal cord or traumatic brain injuries), now

account for the greatest burden on the health systems

of developed countries [3 – 5]. Even in developing

countries, the World Health Organization estimates

that by 2020 these types of conditions will account

for 78% of total disease burden and be the greatest

source of health care costs [5]. In the context of

chronic health conditions and injuries, diagnosis

alone is an inadequate conceptualization of health

status and a poor predictor of service needs, both at

the level of individual treatment planning and at the

level of population health policy. Even in the case of

infectious diseases such as AIDS, health states,

functional status, and service needs may vary

dramatically across individuals with the same diag-

nosis and over time within individuals.

Models of acute care that emphasize diagnosis as

the primary basis for clinical decision-making do not

fit these conditions well. Regardless of whether an

individual’s functional limitations are considered to

be chronic and life-long or of limited duration, it is

the level of functioning more than a diagnosis itself

that is often the best indicator of service needs and

treatment outcomes. For example, the needs of

individuals with a diagnosis of schizophrenia can

only be understood when information on functional

status is available. For some, appropriate treatment

may require extensive and costly hospitalization

while for others maintenance medication and inter-

mittent community-based services may be indicated.

In the case of multiple sclerosis, individuals with the

same diagnosis can experience vastly different

patterns of functioning, and consequently require

different treatment approaches of varying intensity.

Some individuals may have functional limitations in

mobility, self care and interpersonal relationships

while others may experience limitations in only one

of these domains. Interventions in the context of

such conditions are more accurately conceptualized

as aimed at the improvement of functioning or the

prevention, delay, or amelioration of the severity and

course of illness, rather than at the elimination of an

underlying disease process. Both for clinical deci-

sion-making and the evaluation of health services and

systems, more information is needed than simply the

diagnostic category and the nature of the service

provided.

The ICF as a clinical framework for

conceptualizing functional status

The World Health Organization’s International

Classification of Functioning, Disability, and Health

(ICF) [6] is the only viable system that can be used to

provide clinicians and health systems with the

information they need regarding functional status

in order to plan and direct treatment appropriately

[7]. In order for the ICF to be useful in making

point-of-service decisions about care, the system will

need to be implemented by health professionals.

This is true whether or not health professionals

themselves actually end up doing the physical coding

of patient data. Even if patient records are coded by

others (as is generally the case with procedures),

health professionals will need to understand the

system in order to record the information about

functional status that will be necessary and to use it

as a conceptual basis for the services they deliver [8].

The fact that health professionals have an interest

in the implementation of the ICF has been made

increasingly clear by the support that ICF has

attracted from US professional associations, includ-

ing occupational therapy, physical therapy, psychol-

ogy, recreation therapy, speech-language pathology,

and social work [9]. The ICF provides health

professionals with a powerful tool to communicate

specific nuances of function, document outcomes,

and conduct research. Furthermore, the ICF’s wide

descriptive capabilities have the potential to make

treatment more client-centred and functionally-

oriented by expanding the scope of functional

activities that can be documented, and thus ad-

dressed in therapy. This will support health profes-

sionals in providing treatments that are relevant over

the course of people’s lives.

Many of the functions classified require sophisti-

cated professional knowledge and clinical judgment

to assess reliably, particularly if such classifications

are to be used as a basis for making high-stakes

health care decisions. The ICF does not direct the

clinician in selecting the most salient aspects of

functioning to assess for a given individual or in the

selection of measures that are most relevant for a

particular clinical situation. A variety of assessment

methods are compatible with the ICF framework and

can provide information that can be used for

formulating priorities, planning treatment, and as-

sessing outcomes. Tracking clinical changes, explor-

ing outcomes of treatment, and measuring

performance over time require the use of different

methodologies in different contexts. A thorough

assessment of functioning – even in a highly targeted

area – will frequently require convergent validation

through multiple measures or observations, or in

multiple situations, in order to produce reliable ICF

928 G. M. Reed et al.

Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

coding and useful information for the health care

system. In order for the ICF to be applied reliably in

clinical settings and in research, it is important for

health professionals to develop methodologies for

translating the concepts and results of professional

assessments they already know and conduct into ICF

codes.

Training health professionals to use the ICF

Previous assessments of US health professionals’

attitudes toward the ICF have found that the

majority were unaware of it and that a major

educational effort would be required in order for

the system to be implemented in health care settings

[9]. Multidisciplinary groups of health professionals

have also indicated that the ICF itself does not

provide sufficient information for its implementation

in health care settings [10]. Despite this, there is a

lack of systematic descriptions in the literature

regarding the best ways to teach health professionals

and students to use the ICF. As we develop training

models, it is important to evaluate their effectiveness

in an empirical, criterion-oriented fashion in order to

develop a research base that will help us to identify

the best and most efficient training models.

The purpose of this article is to describe three

systematic efforts to train clinicians to use the ICF.

The first of these was a randomized study that

compared two different training methods – a 2-hour

instructor-led discussion-based programme and a

self-directed learning module – in terms of their

effects on the ICF-related knowledge, attitudes, and

coding skills of occupational therapy graduate

students. The second programme described was a

series of 3.5-day workshops sponsored by the

Department of Health of the Republic of South

Africa designed to train health professionals in public

sector rehabilitation programmes to apply the ICF in

clinical settings.

The first and second programmes are related in

that the core content of both of these training

programmes has been developed over the past

several years, particularly by the first, third, and

last authors (Reed, Bufka, and Stark), in relation-

ship to the development of the Procedural Manual

and Guide for a Standardized Application of the ICF

(Manual) intended for health professionals. The

Manual has been under development for the past

several years by the American Psychological Asso-

ciation in collaboration with World Health Organi-

zation and in association with several other

associations of health professionals, include the

American Occupational Therapy Association, the

American Physical Therapy Association, the Amer-

ican Speech Language Hearing Association, the

American Therapeutic Recreation Association, and

the National Association of Social workers [9]. The

format of the training has been strongly influenced

by Adult Learning Theory [11], which conceptua-

lizes the role of the educator as a facilitator, and

emphasizes the importance of experience and real-

life application. The training emphasizes the prac-

tical application of the ICF by health professionals

in clinical settings, and addresses the issues

previously found to be important in needs assess-

ment of health professionals [10]. The general

outline of the training is as follows:

(1) Conceptual overview of the ICF model;

(2) Anatomy of the ICF: Organization and

content of codes;

(3) Code sets;

(4) Applying codes to clinical cases;

(5) Linking clinical assessments to ICF codes;

(6) Challenges to using the ICF; and

(7) Implementing the ICF.

The first project presented lies at one end of the

spectrum of this group’s training experience in terms

of length and intensity of training, consisting of an

approximately 2-hour instructor-led training or self-

directed training module. The second programme

lies at the other end of this group’s experience,

consisting of a series of 3.5-day workshops for health

professionals in South Africa. This group has also

conducted a variety of trainings of intermediate

lengths. The trainings of different lengths do not

differ in terms of these core topics, but do vary

significantly in terms of depth and detail and the

amount of practical experience provided as part of

the training. Depending on their length, trainings

may involve: didactic instruction; discussion and

application to each clinician’s practice, including the

development of code sets; practice coding case

vignettes; practice coding more detailed clinical

information, including assessment data; and practice

coding clinicians’ own cases.

The third programme involved the development

and implementation of a series of internet-based

teaching modules regarding the ICF as a part of a

distance learning (on-line) course on assistive tech-

nology for graduate students in rehabilitation coun-

selling. ICF training was provided via on-line

lectures and slideshows, and students had the

opportunity to discuss the ICF concepts with the

professor and with each other via an on-line

discussion board.

Programme 1: Training occupational therapy

graduate students to use the ICF

As noted, this programme evaluated the impact of

two different training methods on knowledge,

Three model curricula for teaching the ICF 929

Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

attitudes, and coding skills related to the ICF among

occupational therapy graduate students using a

randomized controlled study design. Training meth-

ods included a 2-hour instructor-led discussion-

based training and a self-directed learning module

with the same content designed to be completed in

the same length of time. Based on Adult Learning

Theory [11], the preliminary hypothesis was the

instructor-led training would be more effective.

Methods

Design and participants. This study was a randomized

two-group pre- and post-test of the effectiveness of

two different methods of training about the ICF.

Participants were a convenience sample of 56

occupational therapy students at the graduate level

at Washington University Saint Louis (St. Louis,

MO, USA). The majority of participants (62.2%)

were enrolled in the masters’ programme in occupa-

tional therapy. The remainder of participants were

enrolled in a clinical doctoral programme. However,

almost all participants were first-year students

(98.2%), and the instructional content of the first

year of the masters’ programme and the first year

doctoral programme is essentially the same. All

participants were enrolled in a course entitled

‘Environments that Influence Occupational Perfor-

mance’ required as part of the occupational therapy

graduate curriculum and taught by the last author

(Stark). The mean age of participants was 23 years.

They had no previous clinical work experience as

occupational therapists or trainees.

Study participants participated in the ICF train-

ing and learning assessments during class periods.

Because the ICF training was presented as part of

coursework for graduate school curriculum, study

participants were given the choice of whether or

not to include their anonymous assessment and

survey results in the analysis for this research

study. All students agreed to participate in the

research. It had been determined that students

would not be included in the study if they reported

disabilities that prohibited the use of self-directed

learning strategies, but no student reported such a

condition.

Measures. There are no published assessments of

ICF-related knowledge and coding skills. The last

author and the second author (Stark and Dilfer)

developed a survey and learning assessment based on

the curriculum developed for the study. The first

portion of the survey consisted of information

regarding participants’ prior exposure to the ICF.

The remaining components of the survey and

learning assessment were conceptually based on

Bloom’s taxonomy of educational objectives [12],

and focused on student’s knowledge, skills, and

attitudes related to the ICF.

The knowledge portion of the learning assessment

consisted of 15 True/False and multiple choice

questions designed to measure participants’ knowl-

edge of the conceptual basis and organization of the

ICF. For example, ‘ICF qualifiers are optional’

(True/False) and ‘The ICF is based on the _____

model of disability’ (multiple choice). This section of

the learning assessment was scored as the percentage

of questions answered correctly.

The skills portion of the learning assessment

examined students’ ability to apply the ICF to the

coding of a mock case. The mock case consisted of a

case description and associated mock data from

standard measures familiar to these occupational

therapy students. Participants were provided with a

pre-determined code set in a format based on the

ICF checklist Version 2.1a, Clinician Form [13] and

asked to supply the qualifier ratings for the codes

provided. The code set consisted of 8 Body Func-

tions codes, 5 Body Structures codes, 21 Activities

and Participation codes, and 5 Environmental

Factors codes. Participants were asked to make

qualifier ratings only for the required qualifiers

(one for Body Functions, two for Body Structures,

two for Activities and Participation, and one for

Environmental Factors). This section was scored as

the percentage of agreement with a pre-determined

coding that had been established by three expert

coders (the first, second, and last authors; Reed,

Dilfer, and Stark).

The attitude portion of the learning assessment

consisted of 13 statements regarding the ICF with

which participants were asked to indicate their level

of agreement on a 5-point Likert scale. Sample items

include ‘I think the ICF is a useful tool’ and ‘I

understand how the ICF defines disability’.

Procedures. There were two lab sections in the course,

each consisting of 28 students. Based on a coin toss,

one lab section was assigned to receive the self-

directed learning module and the other to participate

in an interactive lecture presentation. The self-

directed module was developed based on previous

training curricula used to teach occupational thera-

pists and other health care professionals to use the

ICF. It consisted of a 30-page paper packet with

learning activities and questions for each topic area

(described earlier in the introduction). The self-

directed learning module gave participants a brief

overview of the history and purposes of ICF and then

provided information regarding how to use the ICF

coding system. Although students were given 2 hours

to complete the self-directed learning packet, they

spent an average of 59 minutes on it. The 2-hour

instructor-led discussion-based training consisted of


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

a lecture and discussion led by the second author

(Dilfer) covering the same information presented in

the self-directed module. During a normal 3-hour

lab section meeting, participants completed a pre-

instruction learning assessment then either partici-

pated in the 2-hour instructor-led training or the

self-directed learning module, and then completed a

post-instruction learning assessment. Students were

given 30 minutes to complete both the pre- and post-

instruction learning assessments.

All students were offered the alternative educa-

tional strategy at the completion of the study, but

none chose to exercise this option.

Results

Participants reported having a moderate level of

familiarity with the ICF prior to participating in the

training experience. A substantial number (43%;

n¼ 24) reported having read parts of the ICF before,

70% (n¼ 40) reported that the ICF had been

discussed as part of their previous education, 5%

(n¼ 3) reported that they had had used the ICF in

school assignments over the last year, and one

student reported having used it in a previous research

project.

Participants in both the instructor-led and the self-

directed learning formats demonstrated significant

increases in knowledge. Prior to the ICF training, the

instructor-led group answered an average of 66% of

the questions correctly and the self-directed group

answered an average of 71% of the questions directly

correct. (Pre-test scores between groups were not

significantly different.) Following the training, both

groups answered an average of 87% of the questions

correctly. Paired t-tests indicated that this was a

significant increase for both groups (p5 0.001).

Both formats were effective in enhancing the knowl-

edge base assessed by the learning assessment.

However, ICF coding skill improved significantly in

the instructor-led group, from an average of 36%

correct (as determined by agreement with expert

coders) to an average of 42% correct. A paired t-test

indicated that this increase was significant (p50.001).

Although the mean score for the self-directed group

increased from 30% to 37%, this difference was not

statistically significant at p¼ 0.04 when Bonferroni

adjustments were made for multiple t-tests.

A factor analysis was performed on the attitude

items, using a maximum likelihood extraction with a

varimax (orthogonal) rotation. The criteria used in

determining the factor solution included factor

complexity and the magnitude of factor loadings

(40.50), though interpretability of the data was the

final determinant of the solution selected. This

examination yielded a two-factor solution using 10

of the items.

The first factor, labelled Mastery, consisted of

eight items related to the student’s sense of mastery

over the ICF material, including ‘I know how to use

the qualifiers appropriately’ and ‘I can identify the

components of the ICF’. Item ratings for both factors

were summed to yield factor scores. The second

factor, labelled Usefulness, consisted of two items

that related to the perceived value of the ICF for

practice. These were ‘The ICF will be useful in my

career as an OT’ and ‘The ICF is useful in my career

as a student’.

Both groups showed significant increases in sense

of Mastery over the ICF material (p5 0.001), from a

mean of 22.14 to a mean of 37.46 (out of 40) for the

instructor-led group and from 24.53 to 35.25 for the

self-directed group. Additional analysis indicated

that the increase in Mastery was significantly greater

(p5 0.02) for the instructor-led group (Mean

increase¼ 15.32) than for the self-directed group

(Mean increase¼ 10.72). On the Usefulness factor,

mean scores for the instructor-led group increased

very slightly from of 7.93 pre-training to 8.18 post-

training (out of a possible score of 10). Usefulness

scores for the self-directed group decreased slightly

from 7.89 pre-training to 7.39 post-training. Neither

of these changes was significant. In examining this

finding further, the range of scores on Usefulness at

pre-training was the same for both groups (6 – 10).

Post-training, however, Usefulness scores were

significantly more variable (p5 0.02) for the self-

directed group (Range¼ 2 – 10) than for the instruc-

tor-led group (Range¼ 7 – 10). Post-training, 29%

(n¼ 8) of participants in the self-directed group had

scores on Usefulness of 6 or lower, while this was

true of none of the participants in the instructor-led

group.

Discussion

Both methods of teaching the ICF resulted in

significant learning. Thus, these data suggest that

both instructor-led and self-directed learning formats

are effective mechanisms for exposing students to

certain types of material related to the ICF. ICF-

related knowledge as assessed by True/False and

multiple choice questions improved significantly and

to the same level in both groups. In terms of

attitudes, there were increases in both groups in the

extent to which they expressed a sense of mastery

over the ICF material. However, the lecture format

resulted in significantly more learning in terms of

actual coding skill.

Two additional aspects of the data regarding ICF-

related attitudes suggest caution regarding wide

reliance on self-directed training as a primary vehicle

for training clinicians to use the ICF. First, the

instructor-led group exhibited a significantly greater


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

increase in their sense of Mastery in using the ICF

than the self-directed group. Second, although views

of the ICF’s usefulness were the same in both groups

prior to the training, following the training a sizable

proportion of the self-directed group viewed the ICF

as less useful than did any of the participants in the

instructor-led group. Students who have difficulty

when they are using self-directed materials may end

up viewing the ICF as less valuable, while students in

an instructor-led format have the opportunity to ask

for clarification. It may also be important to have a

live instructor to motivate and excite students about

the concepts and practical uses of the ICF. These

points bear consideration when designing ICF

training methods for students and clinicians.

In spite of the fact that this study suggests that an

instructor-led format produces superior participant

learning if the criterion is actual coding of clinical

material, and may also have more positive effects on

participants’ attitudes toward the ICF, there are also

drawbacks to this approach. Lectures require a

trained leader and can be both expensive and time-

consuming. A self-directed learning module, how-

ever, eliminates the need for a trainer and allows

participants to learn material at their own pace,

making this approach practical and cost-effective.

The disadvantage is that it may be difficult to learn

complicated information alone and this difficulty

may lead to more negative attitudes. It will be useful

to explore learning techniques that might be added

to self-directed training programmes to overcome

these disadvantages.

While ICF-related knowledge, at least to the

extent that it was assessed by the learning assess-

ment, seemed to improve in both groups to an

acceptable criterion level (87%), the very low scores

in the mock coding assessment suggest that using the

ICF with a high level of precision requires additional

training beyond a 1 to 2-hour learning experience. It

is also possible that the coding assessment used in

this portion of study was not sensitive enough to

capture the learning that did occur. A large potion of

this assessment required participants to interpret the

results of standard measures presented as part of the

mock case, such as scores on the Functional

Independence Measures (FIM) [14], in assigning

ICF codes. Because students have limited experience

interpreting the results of these measures, their

overall coding success may not be related solely to

their knowledge of the ICF. In future studies, case

studies should be examined to ensure that partici-

pants are presented with data based on measures

with which they are sufficiently familiar so that the

learning assessment accurately tests their ability to

apply ICF knowledge rather than their knowledge of

the measures. It would be important to repeat the

study with practicing clinicians to see if they would

have better success coding, based on their greater

clinical experience with interpreting the results of

such measures. The difference in time spent in

learning (longer for the instructor-led group) may

also have contributed to the difference in coding

performance between the groups – something that

could also be examined in future studies. It is also

possible that the instructor was somehow emphasiz-

ing in her discussion the specific areas that were most

relevant to the learning assessment, something that

could also be controlled experimentally in further

research.

Programme 2: ICF training for public sector

rehabilitation professionals in the Republic

of South Africa

Prior to 1994, the health system in South Africa was

characterized by race-based systems and facilities.

There were 14 different health departments, resulting

in extensive administrative duplication and fragmen-

tation of care. The health care sector was focused on

hospital-based care. Hospitals were assigned to racial

groups and concentrated in white areas. Thus, there

was no real commitment to delivering primary care to

the vast majority of people [15].

Today, the picture is radically different. Public

sector health care is delivered via a district-based

health system to ensure local control over services.

The 9 provinces of South Africa are divided into 53

health districts. Primary care is delivered through

more than 3,500 community-based clinics and

district hospitals, with secondary and tertiary hospi-

tals at the regional and provincial levels. Public

health care accounts for 11% of the government’s

total budget [16].

South Africa faces serious public health chal-

lenges. As of the end of 2005, it was estimated that

approximately 5.5 million people in South Africa

were living with HIV [17]. HIV/AIDS accounts for

nearly half of all deaths in South Africa and nearly 2

million South Africans have died of HIV-related

causes to date [18]. There have been increases in the

rate of tuberculosis, particularly in association with

HIV-AIDS, and treatment-resistant strains are a

growing concern [19].

South Africa continues to be characterized by a

two-tiered health care system, though to a far lesser

extent than in the past [15]. The public sector

accounts for 40% of the total health care expendi-

tures in South Africa, but provides services to 80% of

South Africa’s population of 47 million. Private

sector health care is provided to high wage earners

with employer-based health insurance. Health pro-

fessionals are concentrated in the private sector,

where rates of service are much higher. For example,

private sector drug expenditures are almost 10 times


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

higher per person in the private than in the public

sector.

South Africa struggles with other structural issues

as well [15]. At an international level, South Africa is

experiencing a ‘brain drain’ of health professionals

to wealthier countries eager to hire well-trained

English-speaking professionals at higher rates than

the South African public health system is able to pay.

Increases in funding for hospitals have been devoted

to raising salaries, but hospitals are cash-strapped

and overstretched, particularly due to the increasing

burden of HIV/AIDS and tuberculosis. South Africa

experiences high rates of poverty, unemployment,

and illiteracy that are worse in rural areas. There are

large disparities in total health care resources among

provinces, and acute shortages of health profes-

sionals in poor provinces and rural areas. South

Africa has 11 official languages, of which English is

only the fifth most common home language and is

the first language of less than 10% of the population

[20]. Non-English speaking South Africans experi-

ence higher rates of poverty, unemployment, and

illiteracy and greater problems with access to health

care, particularly to professionals who are fluent in

their own language.

It is against this backdrop that South Africa has

attempted to develop and implement equitable

health care policies consistent with the social justice

principles of its new Constitution [21]. Among these

is the 2003 Free Health Care Policy, which provides

health care to all persons with permanent disability in

one or more of the following areas: moving, getting

around, self-care, communication, seeing, hearing,

and involvement in major psychosocial life situations

such as interpersonal interaction. The policy also

provides free health care to individuals with chronic

mental health conditions or dementia and to frail

older persons. A Disability Grants programme is

available to individuals who are unable to work

because of mental or physical disability on a

temporary or permanent basis. In addition to a

monthly stipend, the programme also includes grants

for personal assistance and housing subsidies. In

2006, the Disability Grants programme provided

assistance to more than 500,000 people [21].

Rehabilitation is one of the programmes repre-

sented at the national Department of Health level

[22], where it is housed with chronic disease

programmes. While the national Department plays

a coordinating and policy-setting role, services and

staffing of rehabilitation programmes are organized

at the provincial level, and the national Department’s

direct control over provincial operations is limited.

As a result, there are major variations in staffing

patterns, structure, and scope of the rehabilitation

programmes among provinces and between urban

and rural areas.

In 2006, the Rehabilitation Programme of the

Department of Health of the Republic of South

Africa issued a request for applications to conduct

training on the use of the ICF for rehabilitation

professionals and programme managers in each of

the nine South African provinces. The reasons for

the decision to devote resources to this in the context

of the Department’s other priorities are complex and

included the following: (i) to help bring South Africa

in line with international health standard; (ii) to help

standardize and systematize rehabilitation services;

(iii) to provide a framework for record-keeping, data

collection, and data collation; (iv) to help provide a

basis for greater resource allocation for the high-cost,

high-impact, chronic disease and injuries treated

within the rehabilitation programmes; (v) to docu-

ment the capability of rehabilitation professionals to

treat these conditions; and ultimately, (vi) to provide

a basis for broader policy changes.

Description of the training

The first author (Reed) was engaged to conduct the

series of trainings. A 3.5-day curriculum was devel-

oped with the collaboration of the third and last

authors (Bufka and Stark) and the support of the

American Psychological Association based on the

general model described earlier in this article. The

workshop was intended to provide participants with

the skills to use the ICF to code clinical cases and to

assist provincial programme managers and the

national Department in designing implementation

strategies at the local and national levels. The

workshop emphasized the practical application of

the ICF, and participants had the opportunity to

practice and discuss coding of clinical cases based on

written vignettes as well as their own case examples.

Issues and problems encountered in the clinical

application of the ICF were discussed, as well as

possible solutions to these problems. Specific learn-

ing objectives for workshop participants were: (i) to

understand the conceptual organization of the ICF

and the basic mechanisms for coding clinical

encounters; (ii) to understand the conceptual issues

that hinder the use of the ICF by clinicians and the

recommended approaches for resolving those issues;

(iii) To develop strategies for selecting the most

appropriate set of ICF codes based on clinical setting

and purpose of assessment; (iv) To develop strategies

for mapping the results of clinical assessments to

appropriate ICF codes; (v) To practice applying ICF

codes to clinical case examples; and (vi) To develop

specific plans for implementation of the ICF in the

provincial health systems and the specific profes-

sional settings of workshop participants.

Workshops were conducted in all 9 South African

provinces between January and May, 2007. All


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

workshops were led by the first author (Reed), and

the first two workshops (Guateng and Limpopo

provinces) were co-led with the third author (Bufka).

A total of 167 health professionals participated in the

workshops. Participants were managers and clini-

cians in rehabilitation programmes at community

health facilities and at community, district, and

provincial hospitals. Clinical work settings of parti-

cipants ranged from general primary care to highly

specialized practice settings. Workshop participants

represented the following health professions (in

order from highest to lowest number): Physiother-

apy, Occupational Therapy, Speech-Language

Pathology/Audiology, Nursing, Optometry, Ortho-

tics, Pharmacy, Dentistry, and Social Work. Partici-

pants had very little familiarity with the ICF prior to

the workshop. Most (70%) reported that they had

either never heard of the ICF prior to the workshop

or had heard of it but didn’t know exactly what it

was. Only 6% reported that they had ever looked at

ICF codes.

Participants were told that they would be respon-

sible for training other professionals in their pro-

grammes. They were also informed that the national

Department did not have in mind a specific ICF

application that they would soon be required to

implement in their settings. Rather, the purpose of

the workshop was to think collaboratively about

whether and how the ICF might be useful in

improving their work in terms of the quality of

patient care and in facilitating the work of clinicians.

In order to fully engage this task, a relatively

intensive training was required.

In its content, the workshop covered the same

material as has been described earlier, though

obviously in much more detail and with much more

time available for practical experience. Participants

in the earlier workshops were provided with short

versions of the ICF, but had access to several copies

of the long version for use during the workshop. In

response to the consensus among participants that

the long version was important for richer clinical

applications, the long version of the ICF was

provided to all participants in the later workshops.

Participants were also provided with a number of

case examples, related articles, and sample chapters

from the Manual for health professionals described

earlier [9]. Considerable attention was paid to

developing materials appropriate for the South

African context (e.g., case examples based on clinical

material from South African rehabilitation patients).

The workshop incorporated the following core

practical activities:

. Participants were asked to define the set of

codes most relevant to their practice, usually by

discipline. Participants were asked to identify a

maximum of 40 – 60 codes. The main purpose

of this activity was to get participants to

actually examine the codes in detail.

. Participants practiced coding paper case ex-

amples. These were designed to be progres-

sively more complex, with later examples

incorporating discipline-specific professional

assessment data.

. Participants were asked to map the assessments

they most commonly used to ICF codes.

Participants had been asked to bring samples

of their assessments with them to the work-

shop, but this instruction was not conveyed

consistently, such that this activity did not

occur systematically in several workshops.

. Participants were asked to make case presenta-

tions of real patients and to develop codings for

these clinical examples.

. At the end of the series, an activity was

incorporated whereby participants were asked

to develop behavioural descriptions of the

qualifier rating anchor points (i.e., 0 – 4) for

several specific codes. This appeared to be

extremely helpful, and will be incorporated

into future intensive workshops.

. Participants were asked to design specific

possible ICF applications for their province.

. Participants were asked to develop implemen-

tation plans for the province. In some ways, the

boundaries of this task extended beyond the

workshop. This task will need to be worked out

in practice in conjunction with other facility

and provincial decision-makers and with the

national Department. Therefore, the extent to

which real implementation plans could be

developed was somewhat limited.

Participant evaluations

At the end of the workshop, all participants were

asked to complete an anonymous evaluation of their

experience. Three participants had left before the

end of the workshop for work-related or transporta-

tion reasons and so did not complete the evaluation.

The evaluation was structured according to the six

learning objectives identified earlier. For each

objective, participants were asked to indicate: (i)

the extent to which they felt the course had met the

objective; (ii) the extent to which they felt that they

personally had accomplished the learning related to

the objective; (iii) how useful they thought the

objective was to their work; and (iv) how effective

the workshop presentation related to this objective

was, including handouts and other materials. (The

language for each specific question was modified

according to the objective.) All of these questions

were rated on a 5-point anchored scale from 1 (Not


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

at all well, Not at all) to 5 (Extremely well, A great

deal, Extremely). Participants were also about

whether the length of the workshop was appropriate.

Overall, participant evaluations were extremely

positive. In 19 of the 24 areas (6 learning objec-

tives6 4 evaluation parameters), over 75% of

participants rated the course, their own learning,

the usefulness of the topic, and the effectiveness of

the presentation as either 4 or 5. There were slightly

lower ratings for the extent to which participants felt

that they personally understood strategies for resol-

ving the conceptual difficulties of the ICF (66%

rating 4 or 5) and strategies for mapping clinical

assessments to ICD codes (66%). There were also

slightly lower ratings related to the learning objective

of developing specific implementation plans (56% to

75% rating 4 or 5 on the four parameters). These

results are presented in Table I. Most participants

(72%) felt that the length of the workshop was about

right, with 17% saying that it was too long and 11%

saying that it was too short.

Pre- and post-workshop learning assessment

In addition, a pilot objective learning assessment of

ICF-related knowledge and coding skills was im-

plemented pre- and post-workshop in 4 of the 9

provinces (Western Cape, Northern Cape, Eastern

Cape, and Kwa-Zulu Natal provinces, n¼ 68). The

main purpose of this evaluation was to examine the

feasibility of this methodology as a tool to assess

more objectively the learning of multidisciplinary

health professionals in a country outside the US.

After the first several workshops had gone smoothly,

the Department of Health was approached for

permission to implement the learning assessment as

a part of the remaining workshops.

The same learning assessment instrument was

used as in Programme 1 described above, except that

the attitude questions were not administered. It was

felt that these would not be meaningful pre-work-

shop due to participants’ lack of familiarity with the

ICF and that post-workshop the questions would be

experienced as redundant with the participant

evaluations. Participants were given 30 minutes to

complete the instrument based on the amount of

time the occupational therapy graduate students had

spent completing it. However, this did not appear to

be sufficient time, and the learning assessments were

collected before many participants had finished.

Comparison of pre- and post-workshop results

indicated significant increases in basic ICF-related

knowledge (to 77%, p5 0.001). There were also

significant increases in coding skill, to higher levels

than that achieved by the much briefer training for

occupational therapy graduate student. The rate of

agreement with expert coders was 58% across Body

Functions and Body Structures. However, compar-

isons of pre- and post-workshop results could not be

completed for Activities and Participation and

Environmental Factors due to missing data for the

last sections of the instrument. Results also indicated

extremely high rates of inter-rater reliability among

the clinicians, suggesting consistency in ICF applica-

tion to case material across professionals.

Discussion

Participant evaluations generally support the struc-

ture and content of the workshop, including the

length of the training. They also suggest some areas

for possible improvement. A slightly lower propor-

tion of participants (66%) expressed confidence that

they personally understood strategies for resolving

the conceptual difficulties of the ICF and for

mapping the results of clinical assessments to ICF

codes. It will be helpful in future workshops to

develop practical exercises that focus specifically on

these two aspects of coding. For example, case

examples that focus specifically on the recommended

Table I. Percentage of South Africa Workshop Participants (n¼162) answering 4 (Substantially, Quite a bit, Quite) or 5 (Extremely well, A

great deal, Extremely) on Workshop Evaluations.

Learning objectives

Course met

objective

You

personally

Useful

to work

Presentation/

materials

Understand conceptual organization of ICF and basic

mechanisms for coding clinical encounters

94% 87% 85% 93%

Understand strategies for resolving conceptual difficulties

of the ICF for clinical coding

80% 65% 80% 83%

Develop strategies for selecting most appropriate set of codes

for setting and purpose of assessment

90% 81% 85% 92%

Develop strategies for mapping results of clinical assessments

to appropriate ICF codes

79% 66% 75% 76%

Practice applying ICF codes to clinical case examples 90% 84% 85% 92%

Develop specific plans for implementation of the ICF in

participants’ professional settings

63% 56% 75% 74%


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

resolution of conceptual difficulties may be helpful.

As noted, mapping of clinical assessments to ICF

codes had been planned as a practical activity, but

was not completed to the extent envisioned due to

logistical difficulties.

Further, participants viewed the workshop as

having accomplished the objective of developing

specific implementation plans somewhat less effec-

tively than the other learning objectives. Largely, this

simply reflects reality. As the workshop discussion

began to focus on implementation, participants

brought up concerns related to their place within

the structure of the health care institutions where

they worked and the practical difficulties of their

being able to be responsible for implementation of

the ICF in those settings. They consistently said that

coordination and policy-setting by the national

Department of Health and strong support both

nationally and at the provincial level would be critical

factors in implementation. Much of this is outside

the workshop’s ability to influence. The national

Department had anticipated that additional follow-

up activity would be necessary to meet these needs

and is currently planning and conducting a series of

follow-up activities, including province-based pilot

projects, a national advisory group on ICF imple-

mentation, specific application materials, and addi-

tional training.

Data from the pre-workshop and post-workshop

learning assessments support the feasibility and

usefulness of this approach to learning evaluation.

There appears to be considerably generalizability of a

learning assessment approach developed with occu-

pational therapy graduate students in the US to

multidisciplinary, non-US health professionals, the

majority of who were non-native speakers of English.

Clearly, health professionals can be trained to apply

ICF codes and understand the value of the ICF to

their work. The levels of agreement with expert

coders that was reached in coding the parts of the

ICF for which data could be analysed was consider-

ably higher than those achieved with a 2-hour

training for occupational therapy graduate students

in Programme 1. This is particularly noteworthy

given the almost complete lack of knowledge about

the ICF among the South African group, in contrast

to the students’ relative familiarity with the ICF

before the training. Provocatively, the high inter-rater

reliability of coding in the South African group

suggests that participants often agreed with one

another about coding, even when they did not agree

with expert coders. It is possible that differences in

training or culture relate to how certain types of case

material are interpreted. This is a theme that

warrants further exploration in future research.

Several other issues warrant specific mention.

First, the data from standard measures provided in

the mock case had been specifically designed to

reflect an occupational therapy assessment. Although

the assessment covered areas that should be generally

familiar to most rehabilitation professionals and the

assessment data were largely face valid (that is, it was

clear from reading it what was being assessed by any

one item on any of the measures used), coding scores

may have been depressed artificially by having

participants work with clinical assessment data and

codes that do not fall within their area of expertise.

Profession-specific mock cases, clinical assessment

data, and ICF coding forms would produce more

externally valid data and more useful indicators of

participant learning. Second, even if participants can

decipher item content, lack of familiarity with how an

instrument is administered (e.g., whether it reflects

performance or capacity) is likely to contribute to

coding error. Therefore, assessment data used as

a part of the learning evaluation should ideally be

both professionally and culturally familiar to the

participant.

Third, a time limit of 30 minutes had been set to

make the evaluation conditions as consistent as

possible with those of the occupational therapy

graduate students. In retrospect, this was a bad idea.

Most of the South African participants were non-

native speakers of English and, as noted, many were

working with clinical assessment data outside their

fields. Therefore, validity appears to have been

sacrificed for reliability in that many participants

were unable to complete the forms within the allotted

time. To obtain an accurate picture of learning, it

seems important to allow participants sufficient time,

which is likely to vary with population and setting.

Programme 3: A distance learning model for

teaching the ICF to rehabilitation counselling

students

This programme involved the development and

implementation of internet-based teaching modules

regarding the ICF for students as a part of a distance

learning (on-line) graduate course on assistive

technology. Students were enrolled in a fully

accredited, distance learning masters’ programme

in rehabilitation counselling at San Diego State

University (San Diego, CA, USA). The distance

learning programme is administered through the

Consortium for Distance Education in Rehabilita-

tion (CDER), a joint programme of San Diego State

University/Interwork Institute, University of North

Texas (Denton, TX, USA), and Georgia State

University (Atlanta, GA, USA). This distance learn-

ing graduate programme is geared primarily for

professionals working as vocational rehabilitation

counsellors. Most students live within the United

States (including the islands of Hawaii), but students


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

from Canada, Guam, the Northern Marianas

Islands, American Samoa, Ireland, and China have

also enrolled in the programme.

Description of the training

‘Applications of Rehabilitation Technology’ is a

required course in the programme taught by the

fourth author (Scherer), who also developed the

course content. Course content includes assistive

technology legislation, research, equipment, ser-

vices, and resources. A core focus of the course is

person-centred assessment strategies related to the

need for assistive technologies, emphasizing two

assessment models. First, the ‘Matching Person and

Technology’ assessment approach [23,24] is a series

of measures designed to assess consumer assistive

technology needs, preferences, and degree of

match. Second, the ‘Tech Points’ model [25]

provides vocational rehabilitation counsellors and

other staff with an easy to follow reference to help

determine when and how to use rehabilitation

technology with individual clients. The ICF was

introduced as part of this course, focusing on how

the ICF relates to the rehabilitation process in

general and in particular how these assessment

approaches crosswalk to the components and

domains of the ICF [26].

Five training modules were developed for use in

the course. These were:

(1) The World Health Organization’s introduc-

tory presentation on the ICF [27].

(2) The Relevance of the ICF to Disability and

Rehabilitation.

(3) Linking the ICF and the Matching Person

and Technology Model and Assistive Tech-

nology Assessment.

(4) Linking the ICF and Tech Points.

(5) The ICF and Disability and Rehabilitation

Statistics.

The ICF was introduced in this course through

on-line lectures and slideshows (including transcripts

and text versions for students using screen readers).

Students had the opportunity to discuss the concepts

via an on-line discussion board with their student

colleagues, co-instructors, and professor.

During the Winter semester of 2006 – 07, there

were 133 students enrolled in the distance learning

masters’ programme, and 71 students enrolled in the

course ‘Applications of Rehabilitation Technology’.

Some of the students had disabilities including: total

blindness; other vision impairments; deafness; being

hard-of-hearing; and partial paralysis. These stu-

dents used their own assistive technologies including

optical character recognition (OCR) screen readers,

screen magnifiers, telephone relay/video equipment

and voice-activated dictation software.

Student evaluations

After completing the five modules on the ICF,

students were asked: (i) How well the ‘Matching

Person and Technology’ and the ‘Tech Points’

frameworks were integrated with the ICF; and (ii)

Why these different frameworks related to acquiring

appropriate assistive technology were valuable to

learn. A formal evaluation was not used. Rather,

students posted their comments on the course

discussion board, in the same way that discussion

had been facilitated throughout the course.

Overall, students’ evaluations of the five modules

were very positive. Student comments included the

following:

. ‘ICF is very appropriate for rehab profes-

sionals, particularly vocational rehabilitation

counsellors, as it focuses on all aspects of a

consumer’s disability.’

. ‘Makes a great deal of sense in looking at a

person’s abilities in multiple areas and across

environments. I can see that the Matching

Person and Technology and ICF will be very

helpful in ensuring that we consider all possible

angles when selecting assistive technology.’

. ‘The ICF helps us think globally, gives us a

common language and permits us to compare

data. From an assistive technology standpoint,

that will be great because we will be speaking

the same terms and comparing the same data

around the world, which keeps getting smaller

every day.’

. ‘Using ICF is a great opportunity for the

disability community to become more solid

and to be understood and respected by both

the scientific and healthcare communities.’

Overall Discussion

These three projects differ in their methods and

target audience, but all three represent efforts to

develop systematic and ultimately testable pro-

grammes for training health professionals on the

ICF. The first project is the most research-based,

though it involved the briefest training. Perhaps the

most important contribution is that it provides a

methodology for the objective assessment of ICF-

related learning that can be applied to a variety of

different training methods. The second programme

suggests that there is considerable generalizability of

this learning assessment methodology across health

disciplines, level of experience, clinical settings, and

even countries. It is our view that the development of


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

profession-specific mock cases and the use of mock

data based on clinical assessment measures that are

both professionally and culturally familiar to training

participants will contribute to the external validity of

evaluations of ICF coding skill such as those

presented for Programmes 1 and 2 and therefore to

the usefulness of these data as an indicator of

participant learning. We strongly recommend that

other ICF training programmes for students and

health professionals begin to incorporate comparable

learning assessments. It is important that we begin to

develop an empirical basis for the selection of

training methods for particular audiences and for

specific purposes.

It is also possible to distinguish the differential

effects of training on ICF-related knowledge, coding

skill, and attitudes toward the ICF. In terms of

learning basic information about the ICF, the 2-hour

instructor-led session and the self-directed learning

packet had similar effects among occupational

therapy graduate students. These students also had

relatively high scores on ICF-related knowledge prior

to the training. It is difficult to compare the results

for the South African health professional group

directly with those for the occupational graduate

students. The South African group had more clinical

experience, but they began the workshop with much

less familiarity with the ICF, were likely less

accomplished test takers, and most of them were

not native English speakers. However, there is no

evidence that the much more intensive face-to-face

training produced superior effects on knowledge

content as assessed by True/False and multiple

choice questions, which tended to be factual, basic,

and related to the general ICF model. Therefore, if

the goal of a particular training is simply to increase

basic factual knowledge about the ICF model, a

briefer, self-directed training will be more efficient

than intensive face-to-face models and may be

equally effective.

However, if the goal of training is to influence

attitudes about the ICF, the data from Programme 1

suggests that face-to-face training may be more

effective. That is, there is often a broader purpose

to training beyond the acquisition of knowledge.

Personal contact can stimulate interest, motivation,

and a sense of ‘belonging’ to an important interna-

tional effort in a way that self-directed materials

reviewed in isolation likely cannot. As described,

although both the instructor-led training and the self-

directed training produced significant increases in

students’ experience of mastery related to the ICF,

these increases were significantly greater among

instructor-led group. In addition, a notable propor-

tion of the self-directed group expressed attitudes

regarding the usefulness of the ICF that were more

negative than they had been initially and that were

lower than those expressed by any of the participants

in the instructor-led group. These findings suggest

that the dissemination of self-directed materials as

primary mechanism for ICF-related training should

be approached cautiously and with their purpose

clearly in mind. A particular issue may be that when

individuals experience difficulty with such materials

and are not able to obtain assistance or clarification,

they may develop more negative attitudes about the

ICF as a result.

If the goal of training is to enable health profes-

sionals to code real clinical cases, these data and our

own subjective experience suggest that more inten-

sive, face-to-face training is more effective. This is

supported by the data from Programme 1 showing

that occupational therapy graduate students showed

significant increases in coding skill after a 2-hour

instructor-led training, but not after participating in a

self-directed learning programme containing the

same information. It is also supported by the level

of coding reliability achieved by the South African

participants in Programme 2, and by their evalua-

tions of the training (see Table I). However, expert

ICF trainers are few and it is expensive and

logistically difficult to travel them around the world

to conduct training programmes for clinicians. Only

the commitment of substantial government re-

sources, such as in South Africa, or other strong

institutional sponsorship can really make this possi-

ble. Intensive training programmes must address the

question of ‘Training for what?’, and strong govern-

ment or institutional support is necessary in order to

provide a meaningful context in terms of foreseeable

implementation.

The approach to training clinicians used South

Africa and the above conclusions about format and

length of training are consistent with those of a large-

scale ICF training programme in Italy developed by

the Disability Italian Network [28]. This programme

includes an 8-hour instructor-led basic training

course, the intention of which is to convey informa-

tion about the ICF and its development, basic

principles, coding structures, and relevance to

different settings and uses. The basic course does

not attempt to teach coding. A separate 3.5-day

advanced course, which assumes completion of the

basic course, is intended to teach coding and clinical

use and implementation of the ICF. The advanced

course is offered in a face-to-face format and

emphasizes working in small groups. The advanced

course is followed by a period of internet-based

supervised practice in coding of case vignettes and

participants’ own clinical cases. In keeping with our

comments about the importance of sponsorship, the

Disability Italian Network training has the strong

support of the Italian government. Several thousand

health professionals from a variety of disciplines have


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

now been trained in Italy [29], and the same training

has been offered in English to other countries

participating in the Measuring Health and Disability

in Europe (MHADIE) project [30] funded by the

European Union. No evaluation of the effectiveness

of this training programme has yet been published. A

major difference between the Disability Italian Net-

work training and the South Africa training de-

scribed in this article is that the Italian training places

considerable emphasis on the use of ICF Checklist

[13] and the WHO Disability Assessment Schedule

(WHODAS II) [31]. Although these tools were

presented as part of the South Africa training, they

were not seen by participants or by the national

Department of Health as candidate methods for

organizing the implementation of the ICF in that

system.

In spite of our preference for face-to-face training,

there are other methodologies that are worthy of

exploration, either as adjunctive methods, such as in

the Disability Italian Network training, or as alter-

natives. For example, Programme 3 presented

lectures and slides regarding the ICF in a self-

directed, internet-based format. However, partici-

pants also had the opportunity to interact with their

professor and to discuss the material with their peers

via internet discussion boards. Although their atti-

tudes were not assessed systematically, the com-

ments of these students do not seem to convey any

sense that the ICF is too complicated to be useful or

other negative attitudes on their part. Therefore,

similar interactive distance-learning methods, which

might include discussion boards, ‘chat’ formats, or

streaming video, may help to overcome the dis-

advantages of self-directed learning and bring the

outcomes of such training more in line with those of

face-to-face training. This is an empirical question,

however, and an important area for future research.

The learning assessment methodology described in

this paper can help to serve as an important tool in

such research, one that is externally valid and not

overly burdensome.

In relation to training health professionals to code

actual cases using the ICF, training needs will vary

according to the specific application envisioned. The

South Africa training was intended to provide

programme managers and directors with the knowl-

edge and skills necessary to develop ICF applications

in their own contexts. This goal requires a level of

conceptual sophistication and substantive familiarity

with the ICF system and its more than 1,400 codes

that is much different than if the goal were to train

health professionals to use the ICF checklist, which

contains less than 200 items, in a reliable fashion in a

particular clinical setting. One of the reasons that the

students in the Programme 3 may not have expressed

the view that the ICF is overly complex and unwieldy

is that it was presented to them in reference to a

specific population (people in need of assistive

technology) and in relation to two specific assess-

ment models.

These findings raise the issue of what is the most

appropriate metric for determining the competency

of health professionals to code cases using the ICF.

For example, rate of agreement with expert coders is

important to ensure comparability of ICF codes

across settings and countries. However, within a

particular setting, inter-rater reliability may be even

more important, as it provides an indicator of the

extent to which different clinicians understand and

apply the codes in the same way. Further, reliability

of ICF coding by health professionals must be

viewed in the context of the reliability of other

classification systems such as the ICD-10 and CPT.

For example, one relevant study involved an

intensive training programme to use the ICD-10

mental disorders codes for 220 psychiatrists and

psychologists from 14 European countries [32]. The

training included rating of written test cases and

development of a coding manual to avoid diagnostic

pitfalls not addressed in the World Health Organiza-

tion’s ICD-10 manual [33]. Following the training,

participants rated 13 written case histories. About

three-quarters of the participants reached a reliability

level (kappa) of at least 0.70, which was viewed as a

desirable outcome, while 6% had a reliability level of

less than 0.40. It is important to note that these

results were achieved among mental health specia-

lists who had been trained on the use of the ICD-10

mental disorders classification as an important part

of their professional training and had been using it

clinically throughout their careers. Another study

compared CPT coding of hospital emergency de-

partment records by different coding agencies, where

coding is performed by trained coders rather than by

the treating health professionals. [34] Poor agree-

ment was observed in the coding assigned to

individual charts, with kappa values of only 0.28

among the agencies. Another study with important

implications for the enterprise of training clinicians

to use the ICF found that coding reliability is

enhanced by point of service data entry as opposed

to third-party coding, and by specific training in the

use of standardized terminologies [35].

In reality, no clinician is going to sit down in front

of a patient with a copy of the ICF ‘red book’ and

look through the book to find the codes that might

apply to that person. Code sets and data capture

forms (electronic or paper) that narrow and focus the

scope of ICF coding will be necessary to end-user

clinical applications, something that was very clear in

the South African discussion. Moreover, the selec-

tion of the universe of codes to apply to a particular

patient is perhaps the major source of reduced


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

reliability in ICF coding, much more than the

determination of whether a particular impairment is

‘moderate’ or ‘severe’. Once they had learned about

the ICF, what South African clinicians reported that

they needed for ICF implementation were specific

forms and applications.

To begin to address this issue, ICF ‘Core Sets’

have been developed for acute and post-acute

settings and for 12 chronic conditions [36 – 38].

The Core Sets are intended to describe the typical

spectrum of problems in functioning of patients with

specific conditions and to guide multidisciplinary

assessments of functioning. While the Core Sets

provide a solid basis for conceptualizing functional

assessment and data collection, they still may be too

long for immediate use in many clinical settings that

are just beginning to implement systematic collection

of functional data. Also, because the Core Sets are

intended to capture the range of functional issues

associated with a particular condition, they may be

overly broad for specific applications. Therefore, a

major task for South African rehabilitation pro-

gramme managers and directors will be to develop

code sets – whether these are based on professional

discipline, clinical setting, patient condition, or some

other organizing principle – that contain information

that is important for their system, useful to the

clinicians who work there, and can be feasibly

collected in their clinical settings. When training is

highly targeted to a specific application (e.g., the use

of a specific form) a much less intensive format will

likely be required.

Training health professionals to use the ICF in

their clinical work settings is highly important to the

eventual global impact of the system. This is a critical

and exciting time to explore which training methods

are best suited for which audiences and for what

purposes. We encourage a creative, yet empirical,

approach to the development of these methods. In

our view, there is a danger inherent in prematurely

‘standardizing’ training approaches and in encoura-

ging rapid dissemination of the most expedient

methodologies. Additional research will help us to

think carefully about the goals of our training and to

be accountable for its outcomes.

Acknowledgements

The authors wish to thank Marjorie Greenberg of the

US National Centre for Health Statistics (NCHS)

for her long commitment and critical thinking

regarding this topic, some of which is reflected in

this article, and also wish to thank Marjorie, Paul

Placek, formerly of NCHS, and John Stone of the

Centre for Rehabilitation Research and Education

(CIRRIE) for helping us to connect to the global

community. The development of some of the

training materials described under Programmes 1

and 2 in this article was supported by the American

Psychological Association.

References

1. World Health Organization. International statistical classifica-

tion of diseases and related health problems: Tenth revision.

10th ed. Geneva: Author; 1992.

2. American Medical Association. Current procedural terminol-

ogy; 2007. Accessed 19 July 2007 from: http://www.ama-

assn.org/ama/pub/category/3113.html

3. World Health Organization. The World Health Report 2002:

Reducing risks to health, promoting healthy life. Geneva:

Author; 2002.

4. Murray CJL, Lopez AD. The global burden of disease (Vol.

1). Cambridge, MA: Harvard University Press; 1996.

5. World Heath Organization. Innovative care for chronic

conditions: Building blocks for action. Geneva: Author; 2002.

6. World Health Organization. The International Classification

of Functioning, Disability, and Health. Geneva: Author; 2001.

7. National Committee on Vital and Health Statistics, US

Department of Health and Human Services. Classifying and

reporting functional status; 2001. Accessed 19 July 2007 from:

http://www.ncvhs.hhs.gov/010716rp.htm

8. Rentsch HP, Bucher P, Dommen Nyffeler I, Wolf C, Hefti H,

Fluri E, et al. The implementation of the International

Classification of Functioning, Disability and Health (ICF) in

daily practice of neurorehabilitation: An interdisciplinary

project at the Kantonsspital of Lucerne, Switzerland. Disabil

Rehabil 2003;25:411 – 421.

9. Reed GM, Lux JB, Bufka LF, Trask C, Peterson DB, Stark S,

et al. Operationalizing the International Classification of

Functioning, Disability and Health (ICF) in clinical settings.

Rehabil Psychol 2005;50:122 – 131.

10. Reed GM, Saleeby P, Stark SL, Bufka LF, Threats TT,

Brandt D. Health professions’ manual for ICF: Results of

clinician field trials. Presented at the Annual Meeting of the

World Health Organization Collaborating Centres for the

Family of International Classifications, Cologne, Germany;

2003.

11. Knowles MS, Holton EF, Swanson RA. The adult learner:

The definitive classic in adult education and human resource

development. 5th ed. Houston: Butterworth-Heinemann;

1998.

12. Anderson LW, Krathwohl DR (editors). A taxonomy for

learning, teaching, and assessing: A revision of Bloom’s

taxonomy of educational objectives. New York: Longman; 2001.

13. World Health Organization. The ICF Checklist;

2003. Accessed 20 July 2007 from: http://www.who.int/

classifications/icf/site/checklist/icf-checklist.pdf

14. Wright J. The FIM(TM). The Centre for Outcome Measure-

ment in Brain Injury; 2000. Accessed 20 July 2007 from:

http://www.tbims.org/combi/FIM

15. Republic of South Africa Department of Health. White paper

for the transformation of the health system in South Africa;

1997. Accessed 26 July 2007 from: http://www.doh.gov.za/

docs/policy/white_paper/healthsys97_01.html

16. Republic of South Africa Government Communication and

Information System. Pocket guide to South Africa 2006/2007:

Health; 2007. Accessed 26 July 2007 from: http://www.gcis.

gov.za/docs/publications/pocketguide/026_health.pdf

17. Republic of South Africa Department of Health. Report:

National HIV and syphilis seroprevalence survey in South

Africa 2005; 2006. Accessed 26 July 2007 from: http://

www.doh.gov.za/docs/reports/2005/hiv.pdf


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

18. Dorrington RE, Johnson LF, Bradshaw D, Daniel T. The

demographic impact of HIV/AIDS in South Africa: National

and provincial indicators for 2006. Centre for Actuarial

Research, South African Medical Research Council and

Actuarial Society of South Africa; 2006. Accessed 26 July

2007 from: http://www.mrc.ac.za/bod/DemographicImpact

HIVIndicators.pdf

19. World Health Organization. South Africa: Country situation

analysis; 2007. Accessed 26 July 2007 from: http://www.

unaids.org/en/Regions_Countries/Countries/south_africa.asp

20. Statistics South Africa. Census 2001: Census in brief; 2003.

Accessed 26 July 2007 from: http://www.statssa.gov.za/

census01/html/CInBrief/CIB2001.pdf

21. Republic of South Africa Department of Social Development.

Discussion document: Linking social grants beneficiaries to

poverty alleviation and economic activity; 2006. Accessed 26

July 2007 from: http://www.socdev.gov.za/documents/2006/

link.pdf

22. Republic of South Africa Department of Health. The primary

health care package for South Africa—A set of norms and

standards: Rehabilitation services; 2000. Accessed 26 July

2007 from: http://www.doh.gov.za/docs/policy/norms.html#

rehabilitation

23. Institute for Matching Person & Technology. Matching

Person & Technology; 2007. Accessed 9 July 2007 from:

http://members.aol.com/IMPT97/MPT.htm

24. Scherer MJ. The Matching Person & Technology (MPT)

Model Manual and Assessments. 5th ed. [CD-ROM].

Webster, NY: The Institute for Matching Person &

Technology, Inc; 2005.

25. Pathfinder Associates. More about TechPoints; 2006. Ac-

cessed 9 July 2007 from: http://www.pathfinderassociates.net/

3_Services/services_handouts/More_about_TECHPOINTS.

pdf

26. Scherer MJ, Sax C. Cross-walking the ICF to a Measure of

Assistive Technology Predisposition and Use. 11th World

Health Organization (WHO) North American Collaborating

Centre (NACC) Conference on the International Classifica-

tion of Functioning, Disability and Health (ICF). Rochester,

MN; 2005. Accessed 20 July 2007 from: http://www.

icfconference.com/abstracts/Batch%20A/1_Scherer.doc

27. World Health Organization. ICF Powerpoint Slide Show;

2001. Accessed 20 July 2007 from: http://www.who.int/

classifications/icf/site/icftemplate.cfm?myurl¼training.html&

mytitle¼Training%20Materials

28. Leonardi M, Bickenbach J, Raggi A, Sala M, Guzzon P,

Valsecchi MR et al. Training on the International Classifica-

tion of Functioning, Disability, and Health (ICF): The ICF-

DIN Basic and the ICF-DIN Advanced Course developed by

the Disability Italian Network. J Headache Pain 2005;6:159 –

164.

29. Leanordi M, Raggi A, Ajovalasit D, Cattoni G, Pisoni C.

Education and training on ICF in Italy. Presented at the 12th

Annual North American Collaborating Centre Conference on

the ICF, Vancouver, British Columbia, Canada; 2006.

Accessed 7 August 2007 from: http://www.icfconference.

com/New%20Presentations/Tuesday%20(Ballroom)/400–500%

20-%20Plenary%20Presentation/Matilde%20Leonardi%20-%20

Plenary.pdf

30. Measuring Health and Disability in Europe; 2007. Acces-

sed 7 August 2007 from: http://www.nda.ie/cntmgmtnew.nsf/0/

AC2092609F36EE6E802570A60033F8A4?OpenDocument

31. World Health Organization. WHODAS II: Disability Assess-

ment Schedule; 2001. Accessed 7 August 2007 from: http://

www.who.int/icidh/whodas/index.html

32. Malt UF, Huyse FJ, Herzog T, Lobo A, Rijssenbeek AJMM,

and The ECLW. The ECLW collaborative study: III.

Training and reliability of ICD-10 psychiatric diagnoses in

the general hospital setting—An investigation of 220 con-

sultants from 14 European countries. J Psychosomat Res

1996;41:451 – 463.

33. World Health Organization. The ICD-10 Classification of

Mental and Behavioural Disorders: Clinical descriptions and

diagnostic guidelines. Geneva: Author; 1992.

34. Bentley PN, Wilson AG, Derwin ME, Scodellaro R, Jackson

RE. Reliability of assigning correct current procedural

terminology-4 E/M codes. Ann Emerg Med 2002;40:269 –

274.

35. Bernstein RM, Hollingworth GR, Viner G, Shearman J,

Labelle C, Thomas R. Reliability issues in coding encounters

in Primary Care using an ICPC/ICD-10-based controlled

clinical terminology. J Am Med Informatics Ass, Symposium

Suppl 1997;21:843 and D004493. Accessed 20 July 2007

from: http://www.amia.org/pubs/symposia/D004493.pdf

36. Cieza A, Ewert T, Ustun B, Chatterji S, Kostanjsek N, Stucki

G. Development of ICF Core Sets for patients with chronic

conditions. J Rehabil Med 2004; 44(Suppl.):9 – 11.

37. Biering-Sorensen F, Scheuringer M, Baumberger M,

Charlifue SW, Post MW, Montero F, Kostanjsek N, Stucki

G. Developing core sets for persons with spinal cord injuries

based on the International Classification of Functioning,

Disability and Health as a way to specify functioning. Spinal

Cord 2006;44:541 – 546.

38. Cieza A, Stucki G. The International Classification of

Functioning, Disability and Health (ICF): A basis for

multidisciplinary clinical practice. In: American College of

Rheumatology: Clinical care in rheumatic diseases, 3rd ed.

(pp. 79 – 87). Atlanta, GA: Association of Rheumatology

Health Professionals, a Division of the American College of

Rheumatology; 2006.


Dis

abil

Reh

abil

Dow

nloa

ded

from

info

rmah

ealth

care

.com

by

Lib

rary

of

Hea

lth S

ci-U

niv

of I

l on

10/2

7/14

For

pers

onal

use

onl

y.

Documents

Three model curricula for teaching clinicians to use the ICF