_ERGONOMICS_

Assessing devices from the user's perspective

I

The right choices can improve safety and productivity

BY ERIC F. SHAVER AND CURT C. BRAUN

UICK TAKE>>>

Assessing the human factors and

ergonomics of medical devices is

a critical factor in the decision-making

process. How users interact with a

product con affect their safety, perfor­

mance and satisfaction. That is why

users need to be involved in the

process and why toking them into

account initially con save hospitals

both time and money. Heuristic evalu­

ations and usability testing con help

materials managers understand how

users interact with devices. This helps

improve efficiency and ensure both

worker and patient safety.

ealth care organizations face

increasing demands in the 21st

century. Workforce shortages,

rising financial concerns, and advances in

medical technology are just a few of the

Shaver

challenges. Although

medical technology has

great potential, its

implementation is often

viewed as a mixed bless­

ing. Medical technolo­

gy that looked good

during the initial sales

presentation is likely to

create as many problems as it solves.

Many of these problems originate from

equipment that was not appropriately

1 designed for the conditions in which it

will be used or for the people who will

use it. Equipment and technology must

be incorporated into existing workflow

and patient care practices.

Failing to adequately consider the design

of equipment or technology, how it will

work within existing systems and how

physicians, nurses and other care special­

ists will interact with it can increase med­

ical errors and threaten patient safety.

Even when medical technology under­

goes an evaluation, the implementation

is all too often associated with surprises

that require changes in work methods and

processes.

To capitalize on the potential of med­

ical technology, it is incumbent on health

30 MATERIALS MANAGEMENT IN HEALTH CARE september 2008 I matmanmag.cam

r

care personnel to scrutinize needs, eval­

uate systems, assess impacts on patients

and staff, establish work procedures, offer

training and provide technical support

and maintenance. Given the number of

choices and the limited time and resources

health care personnel can devote to such

an endeavor, a way to facilitate the selec­

tion process is needed. One way makes

use of methods from the field of human

factors and ergonomics.

Working proposition Human factors and ergonomics is a

unique scientific discipline that systemat­

ically applies the knowledge of human

abilities and limitations

to the design of systems

with the goal of opti­

mizing the interaction

between people and oth­

er system elements to

enhance safety, perfor­

mance and satisfaction.

In simpler terms, human

factors and ergonomics focus on design­

ing the world to better accommodate peo­

ple. They are relevant anywhere people

work with systems, whether the systems

are social and/or technical in nature.

A health care organization is an exam­

ple of a complex system that consists of

many system elements, including people

(e.g., patients, staff, etc.), technology (e.g.,

tools, machines, products, software, etc.),

tasks, environments, organization (e.g.,

culture, policies, financial, etc. ), industry,

legal (e.g., regulations, enforcement, etc.)

and political.

By taking a holistic approach to under­

standing the interactions between people

and system elements, it is possible to iden­

tify the technological requirements that

will best fulfill users' needs, which in turn

will improve quality and safety.

The people-centered systems approach

used by human factors and ergonomics

practitioners can give health care organi­

zations a systematic method to use when

purchasing and implementing medical

technology.

It entails determining if the medical

technology conforms to basic human fac-

tors and ergonomics principles and

whether it will negatively influence the

system into which it will be integrated.

An increasing number of health care

organizations are turning to this methods

as a means of facilitating the process used

to specify, evaluate, procure and imple­

ment new technology.

Ideally, in the future, all medical tech­

nology will undergo some type of human

factors and ergonomics methods prior to

purchase and implementation in health

care organizations.

In the business world, benefits are often

judged by their return on investment

(ROI). A common way to determine the

benefit of a given initiative is by perform­

ing a cost-benefit analysis.

The results of such an analysis can guide

where an organization can best invest their

financial resources, thus maximizing their

ROI. With regard to human factors and

ergonomics methods, there is a growing

body of literature that demonstrates a

positive ROI for their implementation (see

sidebar on page 34).

For example, in the book Usability

Engineering, it was reported that a 6 per­

cent budget investment on usability has

provided returns of 200 percent to 500

percent. Specifically, for the health care

industry, the potential benefits to patients

and staff include:

INCREASED • Patient and staff safety • Productivity and work quality • Staff satisfaction and acceptance • Ease of learning and use (i.e., less

training)

DECREASED • Number of adverse events • Potential for introduction of new

system hazards • Need for modifications, "work­

arounds" and/ or shortcuts

Human factors and ergonomics prac­

titioners use a variety of methods when

matmanmag.com I september 2008 MATERIALS MANAG EMENT IN HEALTH CARE 31

_ERGONOMICS -~ __

0

0 ..c 2-u B "' ~

I

evaluating, designing or procuring tech-

1 nology. Two human factors and ergonom- 1

ics methods can be used to analyze poten­

tial state-of-the-art medical technology

tors using a list of human factors and

ergonomics principles (Heuristics) to inde­

pendently identify usability issues with a

device. 1 prior to procurement and implementa­

tion: Heuristic evaluations and usability

testing.

These methods are directly applicable

to health care organizations and can be

performed either in-house or with the

With respect--to health care organiza­

tions, the evaluators should be individu­

als who are familiar with the technology,

the potential working conditions in which

the technology will be implemented and

have an understanding of human factors

assistance of a human factors and and ergonomics.

ergonomics consultant. Ideally, a health I Heuristic evaluations have several

care organization will use both techniques, strengths and weaknesses, including:

given that they each have their own I strengths and weaknesses. STRENGTHS

• Easy to learn and use

Heuristic evaluations • Requires minimal training

Heuristic evaluations consist of evalua- • Cost effective to implement

Nielsen-Schneiderman Heuristics 1. Consistency: Users should know that similar actions lead to similar results.

2. Visibility: Users should be informed about what is going on in the system.

3. Match: Users should have a mental model that matches the state of the medical

technology.

4. Minimalist: Users should not be exposed to unnecessary information.

5. Memory: Users should not be required to memorize a lot of information to use

the medical technology.

6. Feedback: Users should be provided prompt and informative feedback.

7. Flexibility: Users should have the ability to customize or create shortcuts.

8. Message: Users should receive an informative error message.

9. Errors: Users should be insulated from error occurrence by the design of the

medical technology.

10. Closure: Users should be informed of task completions.

11. Undo: Users should have the ability to recover from errors.

11 12. Language: Users should be presented information in an understandable form.

13. Control: Users should not have the impression that the medical technology is in

control. 1ij <

i i 14. Document: Users should be provided help by the medical technology when

necessary.

32 MATERIALS MANAGEMENT IN HEALTH CARE september 2008 I matmanmag.cam

--~-~---+---!

WEAKNESS ES

• Does not discover missing functionality

• Relies on the knowledge and the experi­

ence of the evaluator

These evaluations consist of three steps:

comparing the Heuristic list against the

medical technology, identifying Heuristic

violations and determining the severity of

each violation.

For example, a set of 14 Heuristics,

coined the Nielsen-Shneiderman Heuris­

tics, were used by researchers in articles

published in a 2003 issue of the journal

of Biomedical Informatics and a 2004

issue of the International Journal of Med­

ical Informatics to evaluate the usability

of medical infusion pumps.

The Nielsen-Shneiderman Heuristics

were developed based on the combina­

tion of 10 user-interface design Heuris­

tics proposed by Nielsen and the eight

golden rules of interface design proposed

by Shneiderman (see sidebar).

The procedure consists of evaluators

independently rating each of the devices

using the Nielsen-Shneiderman Heuris­

tics and generating lists of violations.

The evaluators' lists are aggregated for

each device and redundant violations are

removed.

The compiled lists are returned to the

evaluators, where violations are assigned

a severity rating using a five-point scale

(0-not a usability problem, 1--cosmet­

ic problem only, 2-minor usability prob­

lem, 3-major usability problem, 4-

1 usability catastrophe), and an average

severity rating is computed across the eval­

uators. The results from the Heuristic eval­

uation allow the number of devices to be

narrowed down based on the lowest com­

bined rating.

Heuristic evaluations are used for three

reasons. First, they are used to identify

potential usability issues that might arise

from the medical technology itself or after

its integration into the current system,

which in turn, might lead to medical

errors.

The literature indicates that one eval­

uator will find approximately 35 percent

of the violations, while three to five eval­

uators will find approximately 60 percent

to 70 percent.

Second, they are useful when compar­

ing multiple devices. It allows the evalu­

Human Factors, along with the book

Usability Engineering, ha·s shown that

testing with five users will typically find

approximately 80 percent of the usabili­

ty problems.

Usability testing has several strengths

and weaknesses, including:

STRENGTHS • A small number of test participants

are needed to identify most problems • Identifies the severity of usability prob­

lems • Detects problems not identified by

Heuristic evaluations ators to "weed out" medical technology r----------------~----------------~

that has poor usability, thus saving the

time and cost of performing subsequent

usability tests.

Finally, the outcome of Heuristic eval­

uations can help guide training efforts by

educating health care personnel about

potential violations that routinely occur

when using the medical technology, there­

by decreasing the likelihood of medical

errors.

Usability testing Usability testing consists of having intend­

ed users (i.e., staff) perform routine tasks

with the specific medical technology in a

realistic setting. The steps required to com­

plete a usability test include:

• Designing the test • Performing a dry run and making

modifications to the test design if necessary

• Recruiting users • Conducting the test • Analyzing the results • Determining the appropriate action

to take based on the test results.

Usability testing can capture objective

and subjective outcome performance data.

The former typically includes time to per­

form individual tasks, resolutions of tasks,

and number and type of errors.

The latter includes user feedback regard­

ing strengths and weaknesses of the med­

ical technology, perceived ease of use and

observations made by the tester.

As with Heuristic evaluations, usabili­

ty testing can help guide user training by

covering violations that routinely occur

when using the medical technology.

An article published in a 1992 issue of

Modern Medical Systems keeps your clinical equipment resources ready to go by providing a variety of services 24f7/365.

• Comprehensive Clinical Engineering Programs (BIOPRO) • Environmental Testing & Trace Gas Analysis • Medical Gas Utility Management • Ultra-clean Ventilation Systems for Operating Rooms

Give us a call today at 800.736.8257 or visit us at www.modmedsys.com and experience 30 years of dependability.

matmanmag.com I september 2008 MATERIALS MANAGEMENT IN HEALTH CARE 33

Ergonomic resource books

• International Journal of Medical Informatics, 2004-Heuristic evaluation of infusion pumps: Implications for patient safety in intensive care units, 73, 771-779

• Journal of Biomedical Informatics, 2005-Human factors engineering: A tool for medical device evaluation in hospital procurement decision-making, 38, 213-219

• Journal of Biomedical Informatics, 2003-Using usability heuristics to evaluate patient safety of medical devices, 36, 23-30

• International Journal of Industrial Ergonomics, 2002-Application of usability testing to the development of medical equipment. Usability testing of a frequently used infusion pump and a new user interface for an infusion pump developed with a human factors approach, 29, 145-159

• International Journal of Industrial Ergonomics, 2006-Usability in a medical technology context assessment of methods for usability evaluation of medical equipment, 36, 345-352

• Cost-justifying usability: An update for the Internet age, 2005-Learning about ROI and cost-benefit analysis as it applies to ergonomics

• Applied Ergonomics, 2003-Determining the cost-benefits of ergonomics projects and factors that lead to their success, 34, 419-427

• Applied Ergonomics, 2008-Assessing the value of human factors initiatives, 39, 305-315

OCTOBER 16-18, 2008 • NAVY PIER • CHICAGO, IL

The 2008 HIDA MedSurg Conference & Expo"' is the industry's premier healthcare distribution

sales, networking and education event.

This is your opportunity to solidify business relationships, hear great speakers and learn new selling & management skills.

Register today: www.HIDA.org

34 MATERIALS MANAGEMENT IN HEALTH CARE september 2008 I matmanmag .com

WEAKNESSES • Can be time consuming and costly to

perform • Requires testers that are knowledge­

able with the method

In the end The people-centered systems approach of

human factors and ergonomics can give

health care organizations a systematic

method of purchasing and implementing

medical technology. Health care organi­

zations should incorporate both Heuristic

evaluations and usability testing to iden­

tify potential problems with the introduc­

tion of new medical technology into an

existing system.

Heuristic evaluations are useful for iden­

tifying potential medical technology usabil­

ity issues that might occur from the tech­

nology itself or after its integration in the

current system. However, Heuristic eval­

uations cannot capture all the potential

usability issues for new medical technol­

ogy or the level of staff acceptance and

increased work efficiency.

Thus, it is also necessary to conduct

usability testing with health care

personnel. While both methods have their

place, when used in tandem, they can have

the greatest impact and success. These

methods also help determine whether the

medical technology effectively serves the

goals of organizations (e.g., decreased

medical errors). Finally, human factors

and ergonomics methods are designed to

help health care organizations maximize

their ROI by increasing the positive and

reducing the negative impact that new

medical technology can create. MMHC

ERIC F. SHAVER, PH.D., IS A SENIOR CON­SULTANT, AND CURT C. BRAUN, PH.D., IS THE PRESIDENT, CEO AND FOUNDER OF BENCHMARK RESEARCH & SAFETY INC., MOSCOW, IDAHO. BENCHMARK PROVIDES CONSULTING AND SERVICES FOR A VARIETY OF AREAS, INCLUDING HUMAN FACTORS AND ERGONOMICS, USABILITY, SAFETY AND TRAINING.