Three model curricula for teaching clinicians to use the ICF
GEOFFREY M. REED1, KAREN DILFER2, LYNN F. BUFKA3, MARCIA J. SCHERER4,
PHIA KOTZE5, MALUTA TSHIVHASE6 & SUSAN L. STARK2
1International Union of Psychological Science, Madrid, Spain, 2Programme in Occupational Therapy, Washington University
School of Medicine, St. Louis, Missouri, USA, 3American Psychological Association, Washington, DC, USA, 4The Institute
for Matching Person and Technology, Webster, New York, USA, 5HWH Integrated Human Capital, Woodmead, South
Africa, and 6National Department of Health, Pretoria, South Africa
AbstractPurpose. Three systematic programmes to train health professionals to use the World Health Organization’s InternationalClassification of Functioning, Disability, and Health (ICF) are described, along with efforts to evaluate their effectiveness.Methods. The first programme was a randomized study comparing the effects of a 2-hour instructor-led programme and aself-directed learning module on ICF-related knowledge, attitudes, and coding skills among occupational therapy graduatestudents. The second programme was a series of intensive 3.5-day workshops for public sector rehabilitation professionals inSouth Africa. The third programme involved a series of internet-based teaching modules regarding the ICF for graduatestudents in rehabilitation counselling.Results. The first project found that both instructor-led and self-guided training formats were effective in improving basicICF-related knowledge, but only instructor-led training led to a significant improvement in coding skill. It also had morepositive effects on ICF-related attitudes. This approach to learning assessment was generalizable to multidisciplinary healthprofessionals in South Africa, who achieved a relatively high degree of coding accuracy after the 3.5-day workshop.Participant evaluations supported the structure, content, and length of the training. Students in the third programme alsoreported a very positive learning experience and positive views of the ICF.Conclusions. An empirical basis is important for identifying the best and most efficient training methods for particularaudiences and specific purposes. The length and format of training can be differentially related to specific training goals (i.e.,knowledge, attitudes, and coding skills). Interactive distance learning methods may help to overcome the weaknesses of self-directed training in comparison to face-to-face training.
Keywords: International Classification of Functioning, Disability, and Health (ICF), instructor-led programme, self-directedlearning module, internet-based teaching modules, rehabilitation training
Introduction
Health care settings have traditionally been con-
cerned with two types of classification. Diagnostic
classification, most commonly based on the World
Health Organization’s International Statistical Clas-
sification of Diseases and Health Related Problems
(ICD-10) [1] is used to identify the illness, injury, or
other condition that the patient has. Classification of
procedures, which in the US is most commonly
based on the Current Procedural Terminology
(CPT) codes [2] is used to identify the specific
services or treatments that the patient receives.
Together, these coding systems direct billing and
reimbursement and allow examination of such
questions as whether the treatment provided is
appropriate to the condition the patient has.
The conceptualizations of illness and health
system functioning that underlie the near-exclusive
use of these types of classifications are based on a
medical model of the acute treatment process. That
is, the primary tasks for clinicians are first to identify
correctly the pathogen or nature of the injury or other
disease process, and second to administer the
appropriate treatment for that particular condition.
This model has been appropriate historically for
Correspondence: Geoffrey M. Reed, PhD, International Union of Psychological Science, Glorieta de Bilbao, 5, 4to 4ta, 28004, Madrid, Spain.
E-mail: [email protected]
Disability and Rehabilitation, 2008; 30(12 – 13): 927 – 941
ISSN 0963-8288 print/ISSN 1464-5165 online ª 2008 Informa UK Ltd.
DOI: 10.1080/09638280701800301
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
meeting the challenges associated with infectious
illness, where rapid identification and highly targeted
treatments are the keys to success. Information other
than the nature of the patient’s condition and the
treatments provided is largely irrelevant to the
clinical process.
However, the health care demands of the world’s
population are changing. Chronic, disabling condi-
tions, including mental disorders and injuries that
have lasting, and pervasive functional consequences
(e.g., spinal cord or traumatic brain injuries), now
account for the greatest burden on the health systems
of developed countries [3 – 5]. Even in developing
countries, the World Health Organization estimates
that by 2020 these types of conditions will account
for 78% of total disease burden and be the greatest
source of health care costs [5]. In the context of
chronic health conditions and injuries, diagnosis
alone is an inadequate conceptualization of health
status and a poor predictor of service needs, both at
the level of individual treatment planning and at the
level of population health policy. Even in the case of
infectious diseases such as AIDS, health states,
functional status, and service needs may vary
dramatically across individuals with the same diag-
nosis and over time within individuals.
Models of acute care that emphasize diagnosis as
the primary basis for clinical decision-making do not
fit these conditions well. Regardless of whether an
individual’s functional limitations are considered to
be chronic and life-long or of limited duration, it is
the level of functioning more than a diagnosis itself
that is often the best indicator of service needs and
treatment outcomes. For example, the needs of
individuals with a diagnosis of schizophrenia can
only be understood when information on functional
status is available. For some, appropriate treatment
may require extensive and costly hospitalization
while for others maintenance medication and inter-
mittent community-based services may be indicated.
In the case of multiple sclerosis, individuals with the
same diagnosis can experience vastly different
patterns of functioning, and consequently require
different treatment approaches of varying intensity.
Some individuals may have functional limitations in
mobility, self care and interpersonal relationships
while others may experience limitations in only one
of these domains. Interventions in the context of
such conditions are more accurately conceptualized
as aimed at the improvement of functioning or the
prevention, delay, or amelioration of the severity and
course of illness, rather than at the elimination of an
underlying disease process. Both for clinical deci-
sion-making and the evaluation of health services and
systems, more information is needed than simply the
diagnostic category and the nature of the service
provided.
The ICF as a clinical framework for
conceptualizing functional status
The World Health Organization’s International
Classification of Functioning, Disability, and Health
(ICF) [6] is the only viable system that can be used to
provide clinicians and health systems with the
information they need regarding functional status
in order to plan and direct treatment appropriately
[7]. In order for the ICF to be useful in making
point-of-service decisions about care, the system will
need to be implemented by health professionals.
This is true whether or not health professionals
themselves actually end up doing the physical coding
of patient data. Even if patient records are coded by
others (as is generally the case with procedures),
health professionals will need to understand the
system in order to record the information about
functional status that will be necessary and to use it
as a conceptual basis for the services they deliver [8].
The fact that health professionals have an interest
in the implementation of the ICF has been made
increasingly clear by the support that ICF has
attracted from US professional associations, includ-
ing occupational therapy, physical therapy, psychol-
ogy, recreation therapy, speech-language pathology,
and social work [9]. The ICF provides health
professionals with a powerful tool to communicate
specific nuances of function, document outcomes,
and conduct research. Furthermore, the ICF’s wide
descriptive capabilities have the potential to make
treatment more client-centred and functionally-
oriented by expanding the scope of functional
activities that can be documented, and thus ad-
dressed in therapy. This will support health profes-
sionals in providing treatments that are relevant over
the course of people’s lives.
Many of the functions classified require sophisti-
cated professional knowledge and clinical judgment
to assess reliably, particularly if such classifications
are to be used as a basis for making high-stakes
health care decisions. The ICF does not direct the
clinician in selecting the most salient aspects of
functioning to assess for a given individual or in the
selection of measures that are most relevant for a
particular clinical situation. A variety of assessment
methods are compatible with the ICF framework and
can provide information that can be used for
formulating priorities, planning treatment, and as-
sessing outcomes. Tracking clinical changes, explor-
ing outcomes of treatment, and measuring
performance over time require the use of different
methodologies in different contexts. A thorough
assessment of functioning – even in a highly targeted
area – will frequently require convergent validation
through multiple measures or observations, or in
multiple situations, in order to produce reliable ICF
928 G. M. Reed et al.
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
coding and useful information for the health care
system. In order for the ICF to be applied reliably in
clinical settings and in research, it is important for
health professionals to develop methodologies for
translating the concepts and results of professional
assessments they already know and conduct into ICF
codes.
Training health professionals to use the ICF
Previous assessments of US health professionals’
attitudes toward the ICF have found that the
majority were unaware of it and that a major
educational effort would be required in order for
the system to be implemented in health care settings
[9]. Multidisciplinary groups of health professionals
have also indicated that the ICF itself does not
provide sufficient information for its implementation
in health care settings [10]. Despite this, there is a
lack of systematic descriptions in the literature
regarding the best ways to teach health professionals
and students to use the ICF. As we develop training
models, it is important to evaluate their effectiveness
in an empirical, criterion-oriented fashion in order to
develop a research base that will help us to identify
the best and most efficient training models.
The purpose of this article is to describe three
systematic efforts to train clinicians to use the ICF.
The first of these was a randomized study that
compared two different training methods – a 2-hour
instructor-led discussion-based programme and a
self-directed learning module – in terms of their
effects on the ICF-related knowledge, attitudes, and
coding skills of occupational therapy graduate
students. The second programme described was a
series of 3.5-day workshops sponsored by the
Department of Health of the Republic of South
Africa designed to train health professionals in public
sector rehabilitation programmes to apply the ICF in
clinical settings.
The first and second programmes are related in
that the core content of both of these training
programmes has been developed over the past
several years, particularly by the first, third, and
last authors (Reed, Bufka, and Stark), in relation-
ship to the development of the Procedural Manual
and Guide for a Standardized Application of the ICF
(Manual) intended for health professionals. The
Manual has been under development for the past
several years by the American Psychological Asso-
ciation in collaboration with World Health Organi-
zation and in association with several other
associations of health professionals, include the
American Occupational Therapy Association, the
American Physical Therapy Association, the Amer-
ican Speech Language Hearing Association, the
American Therapeutic Recreation Association, and
the National Association of Social workers [9]. The
format of the training has been strongly influenced
by Adult Learning Theory [11], which conceptua-
lizes the role of the educator as a facilitator, and
emphasizes the importance of experience and real-
life application. The training emphasizes the prac-
tical application of the ICF by health professionals
in clinical settings, and addresses the issues
previously found to be important in needs assess-
ment of health professionals [10]. The general
outline of the training is as follows:
(1) Conceptual overview of the ICF model;
(2) Anatomy of the ICF: Organization and
content of codes;
(3) Code sets;
(4) Applying codes to clinical cases;
(5) Linking clinical assessments to ICF codes;
(6) Challenges to using the ICF; and
(7) Implementing the ICF.
The first project presented lies at one end of the
spectrum of this group’s training experience in terms
of length and intensity of training, consisting of an
approximately 2-hour instructor-led training or self-
directed training module. The second programme
lies at the other end of this group’s experience,
consisting of a series of 3.5-day workshops for health
professionals in South Africa. This group has also
conducted a variety of trainings of intermediate
lengths. The trainings of different lengths do not
differ in terms of these core topics, but do vary
significantly in terms of depth and detail and the
amount of practical experience provided as part of
the training. Depending on their length, trainings
may involve: didactic instruction; discussion and
application to each clinician’s practice, including the
development of code sets; practice coding case
vignettes; practice coding more detailed clinical
information, including assessment data; and practice
coding clinicians’ own cases.
The third programme involved the development
and implementation of a series of internet-based
teaching modules regarding the ICF as a part of a
distance learning (on-line) course on assistive tech-
nology for graduate students in rehabilitation coun-
selling. ICF training was provided via on-line
lectures and slideshows, and students had the
opportunity to discuss the ICF concepts with the
professor and with each other via an on-line
discussion board.
Programme 1: Training occupational therapy
graduate students to use the ICF
As noted, this programme evaluated the impact of
two different training methods on knowledge,
Three model curricula for teaching the ICF 929
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
attitudes, and coding skills related to the ICF among
occupational therapy graduate students using a
randomized controlled study design. Training meth-
ods included a 2-hour instructor-led discussion-
based training and a self-directed learning module
with the same content designed to be completed in
the same length of time. Based on Adult Learning
Theory [11], the preliminary hypothesis was the
instructor-led training would be more effective.
Methods
Design and participants. This study was a randomized
two-group pre- and post-test of the effectiveness of
two different methods of training about the ICF.
Participants were a convenience sample of 56
occupational therapy students at the graduate level
at Washington University Saint Louis (St. Louis,
MO, USA). The majority of participants (62.2%)
were enrolled in the masters’ programme in occupa-
tional therapy. The remainder of participants were
enrolled in a clinical doctoral programme. However,
almost all participants were first-year students
(98.2%), and the instructional content of the first
year of the masters’ programme and the first year
doctoral programme is essentially the same. All
participants were enrolled in a course entitled
‘Environments that Influence Occupational Perfor-
mance’ required as part of the occupational therapy
graduate curriculum and taught by the last author
(Stark). The mean age of participants was 23 years.
They had no previous clinical work experience as
occupational therapists or trainees.
Study participants participated in the ICF train-
ing and learning assessments during class periods.
Because the ICF training was presented as part of
coursework for graduate school curriculum, study
participants were given the choice of whether or
not to include their anonymous assessment and
survey results in the analysis for this research
study. All students agreed to participate in the
research. It had been determined that students
would not be included in the study if they reported
disabilities that prohibited the use of self-directed
learning strategies, but no student reported such a
condition.
Measures. There are no published assessments of
ICF-related knowledge and coding skills. The last
author and the second author (Stark and Dilfer)
developed a survey and learning assessment based on
the curriculum developed for the study. The first
portion of the survey consisted of information
regarding participants’ prior exposure to the ICF.
The remaining components of the survey and
learning assessment were conceptually based on
Bloom’s taxonomy of educational objectives [12],
and focused on student’s knowledge, skills, and
attitudes related to the ICF.
The knowledge portion of the learning assessment
consisted of 15 True/False and multiple choice
questions designed to measure participants’ knowl-
edge of the conceptual basis and organization of the
ICF. For example, ‘ICF qualifiers are optional’
(True/False) and ‘The ICF is based on the _____
model of disability’ (multiple choice). This section of
the learning assessment was scored as the percentage
of questions answered correctly.
The skills portion of the learning assessment
examined students’ ability to apply the ICF to the
coding of a mock case. The mock case consisted of a
case description and associated mock data from
standard measures familiar to these occupational
therapy students. Participants were provided with a
pre-determined code set in a format based on the
ICF checklist Version 2.1a, Clinician Form [13] and
asked to supply the qualifier ratings for the codes
provided. The code set consisted of 8 Body Func-
tions codes, 5 Body Structures codes, 21 Activities
and Participation codes, and 5 Environmental
Factors codes. Participants were asked to make
qualifier ratings only for the required qualifiers
(one for Body Functions, two for Body Structures,
two for Activities and Participation, and one for
Environmental Factors). This section was scored as
the percentage of agreement with a pre-determined
coding that had been established by three expert
coders (the first, second, and last authors; Reed,
Dilfer, and Stark).
The attitude portion of the learning assessment
consisted of 13 statements regarding the ICF with
which participants were asked to indicate their level
of agreement on a 5-point Likert scale. Sample items
include ‘I think the ICF is a useful tool’ and ‘I
understand how the ICF defines disability’.
Procedures. There were two lab sections in the course,
each consisting of 28 students. Based on a coin toss,
one lab section was assigned to receive the self-
directed learning module and the other to participate
in an interactive lecture presentation. The self-
directed module was developed based on previous
training curricula used to teach occupational thera-
pists and other health care professionals to use the
ICF. It consisted of a 30-page paper packet with
learning activities and questions for each topic area
(described earlier in the introduction). The self-
directed learning module gave participants a brief
overview of the history and purposes of ICF and then
provided information regarding how to use the ICF
coding system. Although students were given 2 hours
to complete the self-directed learning packet, they
spent an average of 59 minutes on it. The 2-hour
instructor-led discussion-based training consisted of
930 G. M. Reed et al.
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
a lecture and discussion led by the second author
(Dilfer) covering the same information presented in
the self-directed module. During a normal 3-hour
lab section meeting, participants completed a pre-
instruction learning assessment then either partici-
pated in the 2-hour instructor-led training or the
self-directed learning module, and then completed a
post-instruction learning assessment. Students were
given 30 minutes to complete both the pre- and post-
instruction learning assessments.
All students were offered the alternative educa-
tional strategy at the completion of the study, but
none chose to exercise this option.
Results
Participants reported having a moderate level of
familiarity with the ICF prior to participating in the
training experience. A substantial number (43%;
n¼ 24) reported having read parts of the ICF before,
70% (n¼ 40) reported that the ICF had been
discussed as part of their previous education, 5%
(n¼ 3) reported that they had had used the ICF in
school assignments over the last year, and one
student reported having used it in a previous research
project.
Participants in both the instructor-led and the self-
directed learning formats demonstrated significant
increases in knowledge. Prior to the ICF training, the
instructor-led group answered an average of 66% of
the questions correctly and the self-directed group
answered an average of 71% of the questions directly
correct. (Pre-test scores between groups were not
significantly different.) Following the training, both
groups answered an average of 87% of the questions
correctly. Paired t-tests indicated that this was a
significant increase for both groups (p5 0.001).
Both formats were effective in enhancing the knowl-
edge base assessed by the learning assessment.
However, ICF coding skill improved significantly in
the instructor-led group, from an average of 36%
correct (as determined by agreement with expert
coders) to an average of 42% correct. A paired t-test
indicated that this increase was significant (p50.001).
Although the mean score for the self-directed group
increased from 30% to 37%, this difference was not
statistically significant at p¼ 0.04 when Bonferroni
adjustments were made for multiple t-tests.
A factor analysis was performed on the attitude
items, using a maximum likelihood extraction with a
varimax (orthogonal) rotation. The criteria used in
determining the factor solution included factor
complexity and the magnitude of factor loadings
(40.50), though interpretability of the data was the
final determinant of the solution selected. This
examination yielded a two-factor solution using 10
of the items.
The first factor, labelled Mastery, consisted of
eight items related to the student’s sense of mastery
over the ICF material, including ‘I know how to use
the qualifiers appropriately’ and ‘I can identify the
components of the ICF’. Item ratings for both factors
were summed to yield factor scores. The second
factor, labelled Usefulness, consisted of two items
that related to the perceived value of the ICF for
practice. These were ‘The ICF will be useful in my
career as an OT’ and ‘The ICF is useful in my career
as a student’.
Both groups showed significant increases in sense
of Mastery over the ICF material (p5 0.001), from a
mean of 22.14 to a mean of 37.46 (out of 40) for the
instructor-led group and from 24.53 to 35.25 for the
self-directed group. Additional analysis indicated
that the increase in Mastery was significantly greater
(p5 0.02) for the instructor-led group (Mean
increase¼ 15.32) than for the self-directed group
(Mean increase¼ 10.72). On the Usefulness factor,
mean scores for the instructor-led group increased
very slightly from of 7.93 pre-training to 8.18 post-
training (out of a possible score of 10). Usefulness
scores for the self-directed group decreased slightly
from 7.89 pre-training to 7.39 post-training. Neither
of these changes was significant. In examining this
finding further, the range of scores on Usefulness at
pre-training was the same for both groups (6 – 10).
Post-training, however, Usefulness scores were
significantly more variable (p5 0.02) for the self-
directed group (Range¼ 2 – 10) than for the instruc-
tor-led group (Range¼ 7 – 10). Post-training, 29%
(n¼ 8) of participants in the self-directed group had
scores on Usefulness of 6 or lower, while this was
true of none of the participants in the instructor-led
group.
Discussion
Both methods of teaching the ICF resulted in
significant learning. Thus, these data suggest that
both instructor-led and self-directed learning formats
are effective mechanisms for exposing students to
certain types of material related to the ICF. ICF-
related knowledge as assessed by True/False and
multiple choice questions improved significantly and
to the same level in both groups. In terms of
attitudes, there were increases in both groups in the
extent to which they expressed a sense of mastery
over the ICF material. However, the lecture format
resulted in significantly more learning in terms of
actual coding skill.
Two additional aspects of the data regarding ICF-
related attitudes suggest caution regarding wide
reliance on self-directed training as a primary vehicle
for training clinicians to use the ICF. First, the
instructor-led group exhibited a significantly greater
Three model curricula for teaching the ICF 931
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
increase in their sense of Mastery in using the ICF
than the self-directed group. Second, although views
of the ICF’s usefulness were the same in both groups
prior to the training, following the training a sizable
proportion of the self-directed group viewed the ICF
as less useful than did any of the participants in the
instructor-led group. Students who have difficulty
when they are using self-directed materials may end
up viewing the ICF as less valuable, while students in
an instructor-led format have the opportunity to ask
for clarification. It may also be important to have a
live instructor to motivate and excite students about
the concepts and practical uses of the ICF. These
points bear consideration when designing ICF
training methods for students and clinicians.
In spite of the fact that this study suggests that an
instructor-led format produces superior participant
learning if the criterion is actual coding of clinical
material, and may also have more positive effects on
participants’ attitudes toward the ICF, there are also
drawbacks to this approach. Lectures require a
trained leader and can be both expensive and time-
consuming. A self-directed learning module, how-
ever, eliminates the need for a trainer and allows
participants to learn material at their own pace,
making this approach practical and cost-effective.
The disadvantage is that it may be difficult to learn
complicated information alone and this difficulty
may lead to more negative attitudes. It will be useful
to explore learning techniques that might be added
to self-directed training programmes to overcome
these disadvantages.
While ICF-related knowledge, at least to the
extent that it was assessed by the learning assess-
ment, seemed to improve in both groups to an
acceptable criterion level (87%), the very low scores
in the mock coding assessment suggest that using the
ICF with a high level of precision requires additional
training beyond a 1 to 2-hour learning experience. It
is also possible that the coding assessment used in
this portion of study was not sensitive enough to
capture the learning that did occur. A large potion of
this assessment required participants to interpret the
results of standard measures presented as part of the
mock case, such as scores on the Functional
Independence Measures (FIM) [14], in assigning
ICF codes. Because students have limited experience
interpreting the results of these measures, their
overall coding success may not be related solely to
their knowledge of the ICF. In future studies, case
studies should be examined to ensure that partici-
pants are presented with data based on measures
with which they are sufficiently familiar so that the
learning assessment accurately tests their ability to
apply ICF knowledge rather than their knowledge of
the measures. It would be important to repeat the
study with practicing clinicians to see if they would
have better success coding, based on their greater
clinical experience with interpreting the results of
such measures. The difference in time spent in
learning (longer for the instructor-led group) may
also have contributed to the difference in coding
performance between the groups – something that
could also be examined in future studies. It is also
possible that the instructor was somehow emphasiz-
ing in her discussion the specific areas that were most
relevant to the learning assessment, something that
could also be controlled experimentally in further
research.
Programme 2: ICF training for public sector
rehabilitation professionals in the Republic
of South Africa
Prior to 1994, the health system in South Africa was
characterized by race-based systems and facilities.
There were 14 different health departments, resulting
in extensive administrative duplication and fragmen-
tation of care. The health care sector was focused on
hospital-based care. Hospitals were assigned to racial
groups and concentrated in white areas. Thus, there
was no real commitment to delivering primary care to
the vast majority of people [15].
Today, the picture is radically different. Public
sector health care is delivered via a district-based
health system to ensure local control over services.
The 9 provinces of South Africa are divided into 53
health districts. Primary care is delivered through
more than 3,500 community-based clinics and
district hospitals, with secondary and tertiary hospi-
tals at the regional and provincial levels. Public
health care accounts for 11% of the government’s
total budget [16].
South Africa faces serious public health chal-
lenges. As of the end of 2005, it was estimated that
approximately 5.5 million people in South Africa
were living with HIV [17]. HIV/AIDS accounts for
nearly half of all deaths in South Africa and nearly 2
million South Africans have died of HIV-related
causes to date [18]. There have been increases in the
rate of tuberculosis, particularly in association with
HIV-AIDS, and treatment-resistant strains are a
growing concern [19].
South Africa continues to be characterized by a
two-tiered health care system, though to a far lesser
extent than in the past [15]. The public sector
accounts for 40% of the total health care expendi-
tures in South Africa, but provides services to 80% of
South Africa’s population of 47 million. Private
sector health care is provided to high wage earners
with employer-based health insurance. Health pro-
fessionals are concentrated in the private sector,
where rates of service are much higher. For example,
private sector drug expenditures are almost 10 times
932 G. M. Reed et al.
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
higher per person in the private than in the public
sector.
South Africa struggles with other structural issues
as well [15]. At an international level, South Africa is
experiencing a ‘brain drain’ of health professionals
to wealthier countries eager to hire well-trained
English-speaking professionals at higher rates than
the South African public health system is able to pay.
Increases in funding for hospitals have been devoted
to raising salaries, but hospitals are cash-strapped
and overstretched, particularly due to the increasing
burden of HIV/AIDS and tuberculosis. South Africa
experiences high rates of poverty, unemployment,
and illiteracy that are worse in rural areas. There are
large disparities in total health care resources among
provinces, and acute shortages of health profes-
sionals in poor provinces and rural areas. South
Africa has 11 official languages, of which English is
only the fifth most common home language and is
the first language of less than 10% of the population
[20]. Non-English speaking South Africans experi-
ence higher rates of poverty, unemployment, and
illiteracy and greater problems with access to health
care, particularly to professionals who are fluent in
their own language.
It is against this backdrop that South Africa has
attempted to develop and implement equitable
health care policies consistent with the social justice
principles of its new Constitution [21]. Among these
is the 2003 Free Health Care Policy, which provides
health care to all persons with permanent disability in
one or more of the following areas: moving, getting
around, self-care, communication, seeing, hearing,
and involvement in major psychosocial life situations
such as interpersonal interaction. The policy also
provides free health care to individuals with chronic
mental health conditions or dementia and to frail
older persons. A Disability Grants programme is
available to individuals who are unable to work
because of mental or physical disability on a
temporary or permanent basis. In addition to a
monthly stipend, the programme also includes grants
for personal assistance and housing subsidies. In
2006, the Disability Grants programme provided
assistance to more than 500,000 people [21].
Rehabilitation is one of the programmes repre-
sented at the national Department of Health level
[22], where it is housed with chronic disease
programmes. While the national Department plays
a coordinating and policy-setting role, services and
staffing of rehabilitation programmes are organized
at the provincial level, and the national Department’s
direct control over provincial operations is limited.
As a result, there are major variations in staffing
patterns, structure, and scope of the rehabilitation
programmes among provinces and between urban
and rural areas.
In 2006, the Rehabilitation Programme of the
Department of Health of the Republic of South
Africa issued a request for applications to conduct
training on the use of the ICF for rehabilitation
professionals and programme managers in each of
the nine South African provinces. The reasons for
the decision to devote resources to this in the context
of the Department’s other priorities are complex and
included the following: (i) to help bring South Africa
in line with international health standard; (ii) to help
standardize and systematize rehabilitation services;
(iii) to provide a framework for record-keeping, data
collection, and data collation; (iv) to help provide a
basis for greater resource allocation for the high-cost,
high-impact, chronic disease and injuries treated
within the rehabilitation programmes; (v) to docu-
ment the capability of rehabilitation professionals to
treat these conditions; and ultimately, (vi) to provide
a basis for broader policy changes.
Description of the training
The first author (Reed) was engaged to conduct the
series of trainings. A 3.5-day curriculum was devel-
oped with the collaboration of the third and last
authors (Bufka and Stark) and the support of the
American Psychological Association based on the
general model described earlier in this article. The
workshop was intended to provide participants with
the skills to use the ICF to code clinical cases and to
assist provincial programme managers and the
national Department in designing implementation
strategies at the local and national levels. The
workshop emphasized the practical application of
the ICF, and participants had the opportunity to
practice and discuss coding of clinical cases based on
written vignettes as well as their own case examples.
Issues and problems encountered in the clinical
application of the ICF were discussed, as well as
possible solutions to these problems. Specific learn-
ing objectives for workshop participants were: (i) to
understand the conceptual organization of the ICF
and the basic mechanisms for coding clinical
encounters; (ii) to understand the conceptual issues
that hinder the use of the ICF by clinicians and the
recommended approaches for resolving those issues;
(iii) To develop strategies for selecting the most
appropriate set of ICF codes based on clinical setting
and purpose of assessment; (iv) To develop strategies
for mapping the results of clinical assessments to
appropriate ICF codes; (v) To practice applying ICF
codes to clinical case examples; and (vi) To develop
specific plans for implementation of the ICF in the
provincial health systems and the specific profes-
sional settings of workshop participants.
Workshops were conducted in all 9 South African
provinces between January and May, 2007. All
Three model curricula for teaching the ICF 933
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
workshops were led by the first author (Reed), and
the first two workshops (Guateng and Limpopo
provinces) were co-led with the third author (Bufka).
A total of 167 health professionals participated in the
workshops. Participants were managers and clini-
cians in rehabilitation programmes at community
health facilities and at community, district, and
provincial hospitals. Clinical work settings of parti-
cipants ranged from general primary care to highly
specialized practice settings. Workshop participants
represented the following health professions (in
order from highest to lowest number): Physiother-
apy, Occupational Therapy, Speech-Language
Pathology/Audiology, Nursing, Optometry, Ortho-
tics, Pharmacy, Dentistry, and Social Work. Partici-
pants had very little familiarity with the ICF prior to
the workshop. Most (70%) reported that they had
either never heard of the ICF prior to the workshop
or had heard of it but didn’t know exactly what it
was. Only 6% reported that they had ever looked at
ICF codes.
Participants were told that they would be respon-
sible for training other professionals in their pro-
grammes. They were also informed that the national
Department did not have in mind a specific ICF
application that they would soon be required to
implement in their settings. Rather, the purpose of
the workshop was to think collaboratively about
whether and how the ICF might be useful in
improving their work in terms of the quality of
patient care and in facilitating the work of clinicians.
In order to fully engage this task, a relatively
intensive training was required.
In its content, the workshop covered the same
material as has been described earlier, though
obviously in much more detail and with much more
time available for practical experience. Participants
in the earlier workshops were provided with short
versions of the ICF, but had access to several copies
of the long version for use during the workshop. In
response to the consensus among participants that
the long version was important for richer clinical
applications, the long version of the ICF was
provided to all participants in the later workshops.
Participants were also provided with a number of
case examples, related articles, and sample chapters
from the Manual for health professionals described
earlier [9]. Considerable attention was paid to
developing materials appropriate for the South
African context (e.g., case examples based on clinical
material from South African rehabilitation patients).
The workshop incorporated the following core
practical activities:
. Participants were asked to define the set of
codes most relevant to their practice, usually by
discipline. Participants were asked to identify a
maximum of 40 – 60 codes. The main purpose
of this activity was to get participants to
actually examine the codes in detail.
. Participants practiced coding paper case ex-
amples. These were designed to be progres-
sively more complex, with later examples
incorporating discipline-specific professional
assessment data.
. Participants were asked to map the assessments
they most commonly used to ICF codes.
Participants had been asked to bring samples
of their assessments with them to the work-
shop, but this instruction was not conveyed
consistently, such that this activity did not
occur systematically in several workshops.
. Participants were asked to make case presenta-
tions of real patients and to develop codings for
these clinical examples.
. At the end of the series, an activity was
incorporated whereby participants were asked
to develop behavioural descriptions of the
qualifier rating anchor points (i.e., 0 – 4) for
several specific codes. This appeared to be
extremely helpful, and will be incorporated
into future intensive workshops.
. Participants were asked to design specific
possible ICF applications for their province.
. Participants were asked to develop implemen-
tation plans for the province. In some ways, the
boundaries of this task extended beyond the
workshop. This task will need to be worked out
in practice in conjunction with other facility
and provincial decision-makers and with the
national Department. Therefore, the extent to
which real implementation plans could be
developed was somewhat limited.
Participant evaluations
At the end of the workshop, all participants were
asked to complete an anonymous evaluation of their
experience. Three participants had left before the
end of the workshop for work-related or transporta-
tion reasons and so did not complete the evaluation.
The evaluation was structured according to the six
learning objectives identified earlier. For each
objective, participants were asked to indicate: (i)
the extent to which they felt the course had met the
objective; (ii) the extent to which they felt that they
personally had accomplished the learning related to
the objective; (iii) how useful they thought the
objective was to their work; and (iv) how effective
the workshop presentation related to this objective
was, including handouts and other materials. (The
language for each specific question was modified
according to the objective.) All of these questions
were rated on a 5-point anchored scale from 1 (Not
934 G. M. Reed et al.
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
at all well, Not at all) to 5 (Extremely well, A great
deal, Extremely). Participants were also about
whether the length of the workshop was appropriate.
Overall, participant evaluations were extremely
positive. In 19 of the 24 areas (6 learning objec-
tives6 4 evaluation parameters), over 75% of
participants rated the course, their own learning,
the usefulness of the topic, and the effectiveness of
the presentation as either 4 or 5. There were slightly
lower ratings for the extent to which participants felt
that they personally understood strategies for resol-
ving the conceptual difficulties of the ICF (66%
rating 4 or 5) and strategies for mapping clinical
assessments to ICD codes (66%). There were also
slightly lower ratings related to the learning objective
of developing specific implementation plans (56% to
75% rating 4 or 5 on the four parameters). These
results are presented in Table I. Most participants
(72%) felt that the length of the workshop was about
right, with 17% saying that it was too long and 11%
saying that it was too short.
Pre- and post-workshop learning assessment
In addition, a pilot objective learning assessment of
ICF-related knowledge and coding skills was im-
plemented pre- and post-workshop in 4 of the 9
provinces (Western Cape, Northern Cape, Eastern
Cape, and Kwa-Zulu Natal provinces, n¼ 68). The
main purpose of this evaluation was to examine the
feasibility of this methodology as a tool to assess
more objectively the learning of multidisciplinary
health professionals in a country outside the US.
After the first several workshops had gone smoothly,
the Department of Health was approached for
permission to implement the learning assessment as
a part of the remaining workshops.
The same learning assessment instrument was
used as in Programme 1 described above, except that
the attitude questions were not administered. It was
felt that these would not be meaningful pre-work-
shop due to participants’ lack of familiarity with the
ICF and that post-workshop the questions would be
experienced as redundant with the participant
evaluations. Participants were given 30 minutes to
complete the instrument based on the amount of
time the occupational therapy graduate students had
spent completing it. However, this did not appear to
be sufficient time, and the learning assessments were
collected before many participants had finished.
Comparison of pre- and post-workshop results
indicated significant increases in basic ICF-related
knowledge (to 77%, p5 0.001). There were also
significant increases in coding skill, to higher levels
than that achieved by the much briefer training for
occupational therapy graduate student. The rate of
agreement with expert coders was 58% across Body
Functions and Body Structures. However, compar-
isons of pre- and post-workshop results could not be
completed for Activities and Participation and
Environmental Factors due to missing data for the
last sections of the instrument. Results also indicated
extremely high rates of inter-rater reliability among
the clinicians, suggesting consistency in ICF applica-
tion to case material across professionals.
Discussion
Participant evaluations generally support the struc-
ture and content of the workshop, including the
length of the training. They also suggest some areas
for possible improvement. A slightly lower propor-
tion of participants (66%) expressed confidence that
they personally understood strategies for resolving
the conceptual difficulties of the ICF and for
mapping the results of clinical assessments to ICF
codes. It will be helpful in future workshops to
develop practical exercises that focus specifically on
these two aspects of coding. For example, case
examples that focus specifically on the recommended
Table I. Percentage of South Africa Workshop Participants (n¼162) answering 4 (Substantially, Quite a bit, Quite) or 5 (Extremely well, A
great deal, Extremely) on Workshop Evaluations.
Learning objectives
Course met
objective
You
personally
Useful
to work
Presentation/
materials
Understand conceptual organization of ICF and basic
mechanisms for coding clinical encounters
94% 87% 85% 93%
Understand strategies for resolving conceptual difficulties
of the ICF for clinical coding
80% 65% 80% 83%
Develop strategies for selecting most appropriate set of codes
for setting and purpose of assessment
90% 81% 85% 92%
Develop strategies for mapping results of clinical assessments
to appropriate ICF codes
79% 66% 75% 76%
Practice applying ICF codes to clinical case examples 90% 84% 85% 92%
Develop specific plans for implementation of the ICF in
participants’ professional settings
63% 56% 75% 74%
Three model curricula for teaching the ICF 935
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
resolution of conceptual difficulties may be helpful.
As noted, mapping of clinical assessments to ICF
codes had been planned as a practical activity, but
was not completed to the extent envisioned due to
logistical difficulties.
Further, participants viewed the workshop as
having accomplished the objective of developing
specific implementation plans somewhat less effec-
tively than the other learning objectives. Largely, this
simply reflects reality. As the workshop discussion
began to focus on implementation, participants
brought up concerns related to their place within
the structure of the health care institutions where
they worked and the practical difficulties of their
being able to be responsible for implementation of
the ICF in those settings. They consistently said that
coordination and policy-setting by the national
Department of Health and strong support both
nationally and at the provincial level would be critical
factors in implementation. Much of this is outside
the workshop’s ability to influence. The national
Department had anticipated that additional follow-
up activity would be necessary to meet these needs
and is currently planning and conducting a series of
follow-up activities, including province-based pilot
projects, a national advisory group on ICF imple-
mentation, specific application materials, and addi-
tional training.
Data from the pre-workshop and post-workshop
learning assessments support the feasibility and
usefulness of this approach to learning evaluation.
There appears to be considerably generalizability of a
learning assessment approach developed with occu-
pational therapy graduate students in the US to
multidisciplinary, non-US health professionals, the
majority of who were non-native speakers of English.
Clearly, health professionals can be trained to apply
ICF codes and understand the value of the ICF to
their work. The levels of agreement with expert
coders that was reached in coding the parts of the
ICF for which data could be analysed was consider-
ably higher than those achieved with a 2-hour
training for occupational therapy graduate students
in Programme 1. This is particularly noteworthy
given the almost complete lack of knowledge about
the ICF among the South African group, in contrast
to the students’ relative familiarity with the ICF
before the training. Provocatively, the high inter-rater
reliability of coding in the South African group
suggests that participants often agreed with one
another about coding, even when they did not agree
with expert coders. It is possible that differences in
training or culture relate to how certain types of case
material are interpreted. This is a theme that
warrants further exploration in future research.
Several other issues warrant specific mention.
First, the data from standard measures provided in
the mock case had been specifically designed to
reflect an occupational therapy assessment. Although
the assessment covered areas that should be generally
familiar to most rehabilitation professionals and the
assessment data were largely face valid (that is, it was
clear from reading it what was being assessed by any
one item on any of the measures used), coding scores
may have been depressed artificially by having
participants work with clinical assessment data and
codes that do not fall within their area of expertise.
Profession-specific mock cases, clinical assessment
data, and ICF coding forms would produce more
externally valid data and more useful indicators of
participant learning. Second, even if participants can
decipher item content, lack of familiarity with how an
instrument is administered (e.g., whether it reflects
performance or capacity) is likely to contribute to
coding error. Therefore, assessment data used as
a part of the learning evaluation should ideally be
both professionally and culturally familiar to the
participant.
Third, a time limit of 30 minutes had been set to
make the evaluation conditions as consistent as
possible with those of the occupational therapy
graduate students. In retrospect, this was a bad idea.
Most of the South African participants were non-
native speakers of English and, as noted, many were
working with clinical assessment data outside their
fields. Therefore, validity appears to have been
sacrificed for reliability in that many participants
were unable to complete the forms within the allotted
time. To obtain an accurate picture of learning, it
seems important to allow participants sufficient time,
which is likely to vary with population and setting.
Programme 3: A distance learning model for
teaching the ICF to rehabilitation counselling
students
This programme involved the development and
implementation of internet-based teaching modules
regarding the ICF for students as a part of a distance
learning (on-line) graduate course on assistive
technology. Students were enrolled in a fully
accredited, distance learning masters’ programme
in rehabilitation counselling at San Diego State
University (San Diego, CA, USA). The distance
learning programme is administered through the
Consortium for Distance Education in Rehabilita-
tion (CDER), a joint programme of San Diego State
University/Interwork Institute, University of North
Texas (Denton, TX, USA), and Georgia State
University (Atlanta, GA, USA). This distance learn-
ing graduate programme is geared primarily for
professionals working as vocational rehabilitation
counsellors. Most students live within the United
States (including the islands of Hawaii), but students
936 G. M. Reed et al.
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
from Canada, Guam, the Northern Marianas
Islands, American Samoa, Ireland, and China have
also enrolled in the programme.
Description of the training
‘Applications of Rehabilitation Technology’ is a
required course in the programme taught by the
fourth author (Scherer), who also developed the
course content. Course content includes assistive
technology legislation, research, equipment, ser-
vices, and resources. A core focus of the course is
person-centred assessment strategies related to the
need for assistive technologies, emphasizing two
assessment models. First, the ‘Matching Person and
Technology’ assessment approach [23,24] is a series
of measures designed to assess consumer assistive
technology needs, preferences, and degree of
match. Second, the ‘Tech Points’ model [25]
provides vocational rehabilitation counsellors and
other staff with an easy to follow reference to help
determine when and how to use rehabilitation
technology with individual clients. The ICF was
introduced as part of this course, focusing on how
the ICF relates to the rehabilitation process in
general and in particular how these assessment
approaches crosswalk to the components and
domains of the ICF [26].
Five training modules were developed for use in
the course. These were:
(1) The World Health Organization’s introduc-
tory presentation on the ICF [27].
(2) The Relevance of the ICF to Disability and
Rehabilitation.
(3) Linking the ICF and the Matching Person
and Technology Model and Assistive Tech-
nology Assessment.
(4) Linking the ICF and Tech Points.
(5) The ICF and Disability and Rehabilitation
Statistics.
The ICF was introduced in this course through
on-line lectures and slideshows (including transcripts
and text versions for students using screen readers).
Students had the opportunity to discuss the concepts
via an on-line discussion board with their student
colleagues, co-instructors, and professor.
During the Winter semester of 2006 – 07, there
were 133 students enrolled in the distance learning
masters’ programme, and 71 students enrolled in the
course ‘Applications of Rehabilitation Technology’.
Some of the students had disabilities including: total
blindness; other vision impairments; deafness; being
hard-of-hearing; and partial paralysis. These stu-
dents used their own assistive technologies including
optical character recognition (OCR) screen readers,
screen magnifiers, telephone relay/video equipment
and voice-activated dictation software.
Student evaluations
After completing the five modules on the ICF,
students were asked: (i) How well the ‘Matching
Person and Technology’ and the ‘Tech Points’
frameworks were integrated with the ICF; and (ii)
Why these different frameworks related to acquiring
appropriate assistive technology were valuable to
learn. A formal evaluation was not used. Rather,
students posted their comments on the course
discussion board, in the same way that discussion
had been facilitated throughout the course.
Overall, students’ evaluations of the five modules
were very positive. Student comments included the
following:
. ‘ICF is very appropriate for rehab profes-
sionals, particularly vocational rehabilitation
counsellors, as it focuses on all aspects of a
consumer’s disability.’
. ‘Makes a great deal of sense in looking at a
person’s abilities in multiple areas and across
environments. I can see that the Matching
Person and Technology and ICF will be very
helpful in ensuring that we consider all possible
angles when selecting assistive technology.’
. ‘The ICF helps us think globally, gives us a
common language and permits us to compare
data. From an assistive technology standpoint,
that will be great because we will be speaking
the same terms and comparing the same data
around the world, which keeps getting smaller
every day.’
. ‘Using ICF is a great opportunity for the
disability community to become more solid
and to be understood and respected by both
the scientific and healthcare communities.’
Overall Discussion
These three projects differ in their methods and
target audience, but all three represent efforts to
develop systematic and ultimately testable pro-
grammes for training health professionals on the
ICF. The first project is the most research-based,
though it involved the briefest training. Perhaps the
most important contribution is that it provides a
methodology for the objective assessment of ICF-
related learning that can be applied to a variety of
different training methods. The second programme
suggests that there is considerable generalizability of
this learning assessment methodology across health
disciplines, level of experience, clinical settings, and
even countries. It is our view that the development of
Three model curricula for teaching the ICF 937
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
profession-specific mock cases and the use of mock
data based on clinical assessment measures that are
both professionally and culturally familiar to training
participants will contribute to the external validity of
evaluations of ICF coding skill such as those
presented for Programmes 1 and 2 and therefore to
the usefulness of these data as an indicator of
participant learning. We strongly recommend that
other ICF training programmes for students and
health professionals begin to incorporate comparable
learning assessments. It is important that we begin to
develop an empirical basis for the selection of
training methods for particular audiences and for
specific purposes.
It is also possible to distinguish the differential
effects of training on ICF-related knowledge, coding
skill, and attitudes toward the ICF. In terms of
learning basic information about the ICF, the 2-hour
instructor-led session and the self-directed learning
packet had similar effects among occupational
therapy graduate students. These students also had
relatively high scores on ICF-related knowledge prior
to the training. It is difficult to compare the results
for the South African health professional group
directly with those for the occupational graduate
students. The South African group had more clinical
experience, but they began the workshop with much
less familiarity with the ICF, were likely less
accomplished test takers, and most of them were
not native English speakers. However, there is no
evidence that the much more intensive face-to-face
training produced superior effects on knowledge
content as assessed by True/False and multiple
choice questions, which tended to be factual, basic,
and related to the general ICF model. Therefore, if
the goal of a particular training is simply to increase
basic factual knowledge about the ICF model, a
briefer, self-directed training will be more efficient
than intensive face-to-face models and may be
equally effective.
However, if the goal of training is to influence
attitudes about the ICF, the data from Programme 1
suggests that face-to-face training may be more
effective. That is, there is often a broader purpose
to training beyond the acquisition of knowledge.
Personal contact can stimulate interest, motivation,
and a sense of ‘belonging’ to an important interna-
tional effort in a way that self-directed materials
reviewed in isolation likely cannot. As described,
although both the instructor-led training and the self-
directed training produced significant increases in
students’ experience of mastery related to the ICF,
these increases were significantly greater among
instructor-led group. In addition, a notable propor-
tion of the self-directed group expressed attitudes
regarding the usefulness of the ICF that were more
negative than they had been initially and that were
lower than those expressed by any of the participants
in the instructor-led group. These findings suggest
that the dissemination of self-directed materials as
primary mechanism for ICF-related training should
be approached cautiously and with their purpose
clearly in mind. A particular issue may be that when
individuals experience difficulty with such materials
and are not able to obtain assistance or clarification,
they may develop more negative attitudes about the
ICF as a result.
If the goal of training is to enable health profes-
sionals to code real clinical cases, these data and our
own subjective experience suggest that more inten-
sive, face-to-face training is more effective. This is
supported by the data from Programme 1 showing
that occupational therapy graduate students showed
significant increases in coding skill after a 2-hour
instructor-led training, but not after participating in a
self-directed learning programme containing the
same information. It is also supported by the level
of coding reliability achieved by the South African
participants in Programme 2, and by their evalua-
tions of the training (see Table I). However, expert
ICF trainers are few and it is expensive and
logistically difficult to travel them around the world
to conduct training programmes for clinicians. Only
the commitment of substantial government re-
sources, such as in South Africa, or other strong
institutional sponsorship can really make this possi-
ble. Intensive training programmes must address the
question of ‘Training for what?’, and strong govern-
ment or institutional support is necessary in order to
provide a meaningful context in terms of foreseeable
implementation.
The approach to training clinicians used South
Africa and the above conclusions about format and
length of training are consistent with those of a large-
scale ICF training programme in Italy developed by
the Disability Italian Network [28]. This programme
includes an 8-hour instructor-led basic training
course, the intention of which is to convey informa-
tion about the ICF and its development, basic
principles, coding structures, and relevance to
different settings and uses. The basic course does
not attempt to teach coding. A separate 3.5-day
advanced course, which assumes completion of the
basic course, is intended to teach coding and clinical
use and implementation of the ICF. The advanced
course is offered in a face-to-face format and
emphasizes working in small groups. The advanced
course is followed by a period of internet-based
supervised practice in coding of case vignettes and
participants’ own clinical cases. In keeping with our
comments about the importance of sponsorship, the
Disability Italian Network training has the strong
support of the Italian government. Several thousand
health professionals from a variety of disciplines have
938 G. M. Reed et al.
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
now been trained in Italy [29], and the same training
has been offered in English to other countries
participating in the Measuring Health and Disability
in Europe (MHADIE) project [30] funded by the
European Union. No evaluation of the effectiveness
of this training programme has yet been published. A
major difference between the Disability Italian Net-
work training and the South Africa training de-
scribed in this article is that the Italian training places
considerable emphasis on the use of ICF Checklist
[13] and the WHO Disability Assessment Schedule
(WHODAS II) [31]. Although these tools were
presented as part of the South Africa training, they
were not seen by participants or by the national
Department of Health as candidate methods for
organizing the implementation of the ICF in that
system.
In spite of our preference for face-to-face training,
there are other methodologies that are worthy of
exploration, either as adjunctive methods, such as in
the Disability Italian Network training, or as alter-
natives. For example, Programme 3 presented
lectures and slides regarding the ICF in a self-
directed, internet-based format. However, partici-
pants also had the opportunity to interact with their
professor and to discuss the material with their peers
via internet discussion boards. Although their atti-
tudes were not assessed systematically, the com-
ments of these students do not seem to convey any
sense that the ICF is too complicated to be useful or
other negative attitudes on their part. Therefore,
similar interactive distance-learning methods, which
might include discussion boards, ‘chat’ formats, or
streaming video, may help to overcome the dis-
advantages of self-directed learning and bring the
outcomes of such training more in line with those of
face-to-face training. This is an empirical question,
however, and an important area for future research.
The learning assessment methodology described in
this paper can help to serve as an important tool in
such research, one that is externally valid and not
overly burdensome.
In relation to training health professionals to code
actual cases using the ICF, training needs will vary
according to the specific application envisioned. The
South Africa training was intended to provide
programme managers and directors with the knowl-
edge and skills necessary to develop ICF applications
in their own contexts. This goal requires a level of
conceptual sophistication and substantive familiarity
with the ICF system and its more than 1,400 codes
that is much different than if the goal were to train
health professionals to use the ICF checklist, which
contains less than 200 items, in a reliable fashion in a
particular clinical setting. One of the reasons that the
students in the Programme 3 may not have expressed
the view that the ICF is overly complex and unwieldy
is that it was presented to them in reference to a
specific population (people in need of assistive
technology) and in relation to two specific assess-
ment models.
These findings raise the issue of what is the most
appropriate metric for determining the competency
of health professionals to code cases using the ICF.
For example, rate of agreement with expert coders is
important to ensure comparability of ICF codes
across settings and countries. However, within a
particular setting, inter-rater reliability may be even
more important, as it provides an indicator of the
extent to which different clinicians understand and
apply the codes in the same way. Further, reliability
of ICF coding by health professionals must be
viewed in the context of the reliability of other
classification systems such as the ICD-10 and CPT.
For example, one relevant study involved an
intensive training programme to use the ICD-10
mental disorders codes for 220 psychiatrists and
psychologists from 14 European countries [32]. The
training included rating of written test cases and
development of a coding manual to avoid diagnostic
pitfalls not addressed in the World Health Organiza-
tion’s ICD-10 manual [33]. Following the training,
participants rated 13 written case histories. About
three-quarters of the participants reached a reliability
level (kappa) of at least 0.70, which was viewed as a
desirable outcome, while 6% had a reliability level of
less than 0.40. It is important to note that these
results were achieved among mental health specia-
lists who had been trained on the use of the ICD-10
mental disorders classification as an important part
of their professional training and had been using it
clinically throughout their careers. Another study
compared CPT coding of hospital emergency de-
partment records by different coding agencies, where
coding is performed by trained coders rather than by
the treating health professionals. [34] Poor agree-
ment was observed in the coding assigned to
individual charts, with kappa values of only 0.28
among the agencies. Another study with important
implications for the enterprise of training clinicians
to use the ICF found that coding reliability is
enhanced by point of service data entry as opposed
to third-party coding, and by specific training in the
use of standardized terminologies [35].
In reality, no clinician is going to sit down in front
of a patient with a copy of the ICF ‘red book’ and
look through the book to find the codes that might
apply to that person. Code sets and data capture
forms (electronic or paper) that narrow and focus the
scope of ICF coding will be necessary to end-user
clinical applications, something that was very clear in
the South African discussion. Moreover, the selec-
tion of the universe of codes to apply to a particular
patient is perhaps the major source of reduced
Three model curricula for teaching the ICF 939
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
reliability in ICF coding, much more than the
determination of whether a particular impairment is
‘moderate’ or ‘severe’. Once they had learned about
the ICF, what South African clinicians reported that
they needed for ICF implementation were specific
forms and applications.
To begin to address this issue, ICF ‘Core Sets’
have been developed for acute and post-acute
settings and for 12 chronic conditions [36 – 38].
The Core Sets are intended to describe the typical
spectrum of problems in functioning of patients with
specific conditions and to guide multidisciplinary
assessments of functioning. While the Core Sets
provide a solid basis for conceptualizing functional
assessment and data collection, they still may be too
long for immediate use in many clinical settings that
are just beginning to implement systematic collection
of functional data. Also, because the Core Sets are
intended to capture the range of functional issues
associated with a particular condition, they may be
overly broad for specific applications. Therefore, a
major task for South African rehabilitation pro-
gramme managers and directors will be to develop
code sets – whether these are based on professional
discipline, clinical setting, patient condition, or some
other organizing principle – that contain information
that is important for their system, useful to the
clinicians who work there, and can be feasibly
collected in their clinical settings. When training is
highly targeted to a specific application (e.g., the use
of a specific form) a much less intensive format will
likely be required.
Training health professionals to use the ICF in
their clinical work settings is highly important to the
eventual global impact of the system. This is a critical
and exciting time to explore which training methods
are best suited for which audiences and for what
purposes. We encourage a creative, yet empirical,
approach to the development of these methods. In
our view, there is a danger inherent in prematurely
‘standardizing’ training approaches and in encoura-
ging rapid dissemination of the most expedient
methodologies. Additional research will help us to
think carefully about the goals of our training and to
be accountable for its outcomes.
Acknowledgements
The authors wish to thank Marjorie Greenberg of the
US National Centre for Health Statistics (NCHS)
for her long commitment and critical thinking
regarding this topic, some of which is reflected in
this article, and also wish to thank Marjorie, Paul
Placek, formerly of NCHS, and John Stone of the
Centre for Rehabilitation Research and Education
(CIRRIE) for helping us to connect to the global
community. The development of some of the
training materials described under Programmes 1
and 2 in this article was supported by the American
Psychological Association.
References
1. World Health Organization. International statistical classifica-
tion of diseases and related health problems: Tenth revision.
10th ed. Geneva: Author; 1992.
2. American Medical Association. Current procedural terminol-
ogy; 2007. Accessed 19 July 2007 from: http://www.ama-
assn.org/ama/pub/category/3113.html
3. World Health Organization. The World Health Report 2002:
Reducing risks to health, promoting healthy life. Geneva:
Author; 2002.
4. Murray CJL, Lopez AD. The global burden of disease (Vol.
1). Cambridge, MA: Harvard University Press; 1996.
5. World Heath Organization. Innovative care for chronic
conditions: Building blocks for action. Geneva: Author; 2002.
6. World Health Organization. The International Classification
of Functioning, Disability, and Health. Geneva: Author; 2001.
7. National Committee on Vital and Health Statistics, US
Department of Health and Human Services. Classifying and
reporting functional status; 2001. Accessed 19 July 2007 from:
http://www.ncvhs.hhs.gov/010716rp.htm
8. Rentsch HP, Bucher P, Dommen Nyffeler I, Wolf C, Hefti H,
Fluri E, et al. The implementation of the International
Classification of Functioning, Disability and Health (ICF) in
daily practice of neurorehabilitation: An interdisciplinary
project at the Kantonsspital of Lucerne, Switzerland. Disabil
Rehabil 2003;25:411 – 421.
9. Reed GM, Lux JB, Bufka LF, Trask C, Peterson DB, Stark S,
et al. Operationalizing the International Classification of
Functioning, Disability and Health (ICF) in clinical settings.
Rehabil Psychol 2005;50:122 – 131.
10. Reed GM, Saleeby P, Stark SL, Bufka LF, Threats TT,
Brandt D. Health professions’ manual for ICF: Results of
clinician field trials. Presented at the Annual Meeting of the
World Health Organization Collaborating Centres for the
Family of International Classifications, Cologne, Germany;
2003.
11. Knowles MS, Holton EF, Swanson RA. The adult learner:
The definitive classic in adult education and human resource
development. 5th ed. Houston: Butterworth-Heinemann;
1998.
12. Anderson LW, Krathwohl DR (editors). A taxonomy for
learning, teaching, and assessing: A revision of Bloom’s
taxonomy of educational objectives. New York: Longman; 2001.
13. World Health Organization. The ICF Checklist;
2003. Accessed 20 July 2007 from: http://www.who.int/
classifications/icf/site/checklist/icf-checklist.pdf
14. Wright J. The FIM(TM). The Centre for Outcome Measure-
ment in Brain Injury; 2000. Accessed 20 July 2007 from:
http://www.tbims.org/combi/FIM
15. Republic of South Africa Department of Health. White paper
for the transformation of the health system in South Africa;
1997. Accessed 26 July 2007 from: http://www.doh.gov.za/
docs/policy/white_paper/healthsys97_01.html
16. Republic of South Africa Government Communication and
Information System. Pocket guide to South Africa 2006/2007:
Health; 2007. Accessed 26 July 2007 from: http://www.gcis.
gov.za/docs/publications/pocketguide/026_health.pdf
17. Republic of South Africa Department of Health. Report:
National HIV and syphilis seroprevalence survey in South
Africa 2005; 2006. Accessed 26 July 2007 from: http://
www.doh.gov.za/docs/reports/2005/hiv.pdf
940 G. M. Reed et al.
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.
18. Dorrington RE, Johnson LF, Bradshaw D, Daniel T. The
demographic impact of HIV/AIDS in South Africa: National
and provincial indicators for 2006. Centre for Actuarial
Research, South African Medical Research Council and
Actuarial Society of South Africa; 2006. Accessed 26 July
2007 from: http://www.mrc.ac.za/bod/DemographicImpact
HIVIndicators.pdf
19. World Health Organization. South Africa: Country situation
analysis; 2007. Accessed 26 July 2007 from: http://www.
unaids.org/en/Regions_Countries/Countries/south_africa.asp
20. Statistics South Africa. Census 2001: Census in brief; 2003.
Accessed 26 July 2007 from: http://www.statssa.gov.za/
census01/html/CInBrief/CIB2001.pdf
21. Republic of South Africa Department of Social Development.
Discussion document: Linking social grants beneficiaries to
poverty alleviation and economic activity; 2006. Accessed 26
July 2007 from: http://www.socdev.gov.za/documents/2006/
link.pdf
22. Republic of South Africa Department of Health. The primary
health care package for South Africa—A set of norms and
standards: Rehabilitation services; 2000. Accessed 26 July
2007 from: http://www.doh.gov.za/docs/policy/norms.html#
rehabilitation
23. Institute for Matching Person & Technology. Matching
Person & Technology; 2007. Accessed 9 July 2007 from:
http://members.aol.com/IMPT97/MPT.htm
24. Scherer MJ. The Matching Person & Technology (MPT)
Model Manual and Assessments. 5th ed. [CD-ROM].
Webster, NY: The Institute for Matching Person &
Technology, Inc; 2005.
25. Pathfinder Associates. More about TechPoints; 2006. Ac-
cessed 9 July 2007 from: http://www.pathfinderassociates.net/
3_Services/services_handouts/More_about_TECHPOINTS.
26. Scherer MJ, Sax C. Cross-walking the ICF to a Measure of
Assistive Technology Predisposition and Use. 11th World
Health Organization (WHO) North American Collaborating
Centre (NACC) Conference on the International Classifica-
tion of Functioning, Disability and Health (ICF). Rochester,
MN; 2005. Accessed 20 July 2007 from: http://www.
icfconference.com/abstracts/Batch%20A/1_Scherer.doc
27. World Health Organization. ICF Powerpoint Slide Show;
2001. Accessed 20 July 2007 from: http://www.who.int/
classifications/icf/site/icftemplate.cfm?myurl¼training.html&
mytitle¼Training%20Materials
28. Leonardi M, Bickenbach J, Raggi A, Sala M, Guzzon P,
Valsecchi MR et al. Training on the International Classifica-
tion of Functioning, Disability, and Health (ICF): The ICF-
DIN Basic and the ICF-DIN Advanced Course developed by
the Disability Italian Network. J Headache Pain 2005;6:159 –
164.
29. Leanordi M, Raggi A, Ajovalasit D, Cattoni G, Pisoni C.
Education and training on ICF in Italy. Presented at the 12th
Annual North American Collaborating Centre Conference on
the ICF, Vancouver, British Columbia, Canada; 2006.
Accessed 7 August 2007 from: http://www.icfconference.
com/New%20Presentations/Tuesday%20(Ballroom)/400–500%
20-%20Plenary%20Presentation/Matilde%20Leonardi%20-%20
Plenary.pdf
30. Measuring Health and Disability in Europe; 2007. Acces-
sed 7 August 2007 from: http://www.nda.ie/cntmgmtnew.nsf/0/
AC2092609F36EE6E802570A60033F8A4?OpenDocument
31. World Health Organization. WHODAS II: Disability Assess-
ment Schedule; 2001. Accessed 7 August 2007 from: http://
www.who.int/icidh/whodas/index.html
32. Malt UF, Huyse FJ, Herzog T, Lobo A, Rijssenbeek AJMM,
and The ECLW. The ECLW collaborative study: III.
Training and reliability of ICD-10 psychiatric diagnoses in
the general hospital setting—An investigation of 220 con-
sultants from 14 European countries. J Psychosomat Res
1996;41:451 – 463.
33. World Health Organization. The ICD-10 Classification of
Mental and Behavioural Disorders: Clinical descriptions and
diagnostic guidelines. Geneva: Author; 1992.
34. Bentley PN, Wilson AG, Derwin ME, Scodellaro R, Jackson
RE. Reliability of assigning correct current procedural
terminology-4 E/M codes. Ann Emerg Med 2002;40:269 –
274.
35. Bernstein RM, Hollingworth GR, Viner G, Shearman J,
Labelle C, Thomas R. Reliability issues in coding encounters
in Primary Care using an ICPC/ICD-10-based controlled
clinical terminology. J Am Med Informatics Ass, Symposium
Suppl 1997;21:843 and D004493. Accessed 20 July 2007
from: http://www.amia.org/pubs/symposia/D004493.pdf
36. Cieza A, Ewert T, Ustun B, Chatterji S, Kostanjsek N, Stucki
G. Development of ICF Core Sets for patients with chronic
conditions. J Rehabil Med 2004; 44(Suppl.):9 – 11.
37. Biering-Sorensen F, Scheuringer M, Baumberger M,
Charlifue SW, Post MW, Montero F, Kostanjsek N, Stucki
G. Developing core sets for persons with spinal cord injuries
based on the International Classification of Functioning,
Disability and Health as a way to specify functioning. Spinal
Cord 2006;44:541 – 546.
38. Cieza A, Stucki G. The International Classification of
Functioning, Disability and Health (ICF): A basis for
multidisciplinary clinical practice. In: American College of
Rheumatology: Clinical care in rheumatic diseases, 3rd ed.
(pp. 79 – 87). Atlanta, GA: Association of Rheumatology
Health Professionals, a Division of the American College of
Rheumatology; 2006.
Three model curricula for teaching the ICF 941
Dis
abil
Reh
abil
Dow
nloa
ded
from
info
rmah
ealth
care
.com
by
Lib
rary
of
Hea
lth S
ci-U
niv
of I
l on
10/2
7/14
For
pers
onal
use
onl
y.