-
Human Factors Assessment of Devices
CBEB Presentation, October, 2012 Tony Easty, PhD PEng, CCE,
Centre for Global eHealth Innovation, Toronto, Canada
-
Video Clip Medication Error
-
How did we get this information?
Recruit typical users as volunteers
High fidelity simulation of a clinical environment
Train participants before each experiment
Look for replication of problems, saturation
-
Human Factors Engineering
-
Human Factors Engineering
Aims to improve the design of devices, and systems to conform to
user needs, capabilities, and limitations (both physical and
cognitive)
Users within the system boundary
System boundary
users
patient
device
environment
-
Centre for Global eHealth Innovation Labs
15,000 square feet within University Health Network Includes
testing and observation rooms Advanced remote observation via
web-streaming and
video conference
Multi-tasking Simulation Laboratory Using theatre style props
space can be transformed into
range of real-world settings where health care is delivered
-
Centre for Global eHealth Innovation Labs
-
Why do Human Factors Evaluations?
improved adoption of technology
improved efficiency
decreased training
improved patient safety
-
Areas for Inclusion of Human Factors
Medical Device and Healthcare IT Evaluations
Clinical & technical reviews to short-list products
Heuristic evaluation Low & high fidelity usability
evaluation
Iterative User-Centered Design For use during device development
cycle
-
Our Methodologies
Ethnographic Observation and Workflow Analysis
Heuristic Analysis
Low and High Fidelity Simulation
-
Ethnographic Observation and Workflow Analysis
Conduct observational studies in the environment in which the
technology is used
Understand the limitations of the technology in the
environment
Understand the constraints the environment places on the use of
the technology
-
Heuristic Analysis
Seeking to discover
Systematic rules-based review of a technology
Typically performed by one or more trained human factors
engineers
-
Nielsen-Shneiderman Heuristics*
Consistency & standards Visibility of system state Match
between system
and world Minimalist Minimize memory load Informative feedback
Flexibility and efficiency
*Zhang et al. Using usability heuristics to evaluate patient
safety of medical devices. Journal of Biomedical
Informatics 2003;36:23-30.
Good error messages Prevent errors Clear closure Reversible
actions Use users language Users in control Help and
documentation
-
Usability Testing
Actual simulation of the use of a technology Low or high
fidelity environment
Can be performed in a real clinical environment or a lab
Scripted series of encounters with a technology, observed and
analyzed
-
Usability Testing (Low Fidelity)
-
Usability Testing (High Fidelity)
3 Anesthesia Information Management Systems tested
High fidelity testing because of complicated systems and
environment
17 anesthesiologists participated
-
High Fidelity Usability Testing
-
High Fidelity Usability Evaluation
Representative end users perform realistic tasks in simulated
real-world environments
Environmental design considers Lighting Noise Space
Scenarios take into consideration Multi-user interactions
Interruptions
-
High Fidelity Usability Evaluation
Participants are audio and video taped while verbalizing their
thoughts
Video is analyzed across participants for: Interface issues
Training implications Patient Safety Implications
-
Evaluation Criteria
Efficiency & Speed Task completion time
Task accuracy & error frequency Number of requests for help
Number of attempts to correct errors
Impact & Severity of error (patient safety)
Workload/Difficulty scale (1-7) User feedback & preferences
-
How Applicable?
Technology Medical Devices, Healthcare IT Applications
Processes How is the technology integrated into overall
workflow?
Communications Are these verbal, written, electronic?
Environment Ambient noise, lighting, interruptions, layout and
ergonomics
-
Role of Human Factors in the Product Selection Process
- Electrosurgical Generators
-
Simplified ESU Interface
-
Automatic External Defibrillators - Study Objective
To identify the safety issues of 4 leading Automatic External
Defibrillators (AEDs)
Heuristic evaluation and high fidelity usability testing (8
participants)
-
AED
-
Usability Testing Example: Opening the Case
-
Usability Testing Example: Inadvertently Turning Device Off
-
Study Outcomes
Purchase of the top AED from the usability testing
Installation of AEDs in strategic areas across UHN and MSH
-
Some General Outcomes
Reduction in errors with technologies selected with Human
Factors evaluation
Staff are empowered by Human Factors training and knowledge
Industrial Collaborators release new device to enthusiastic user
response
Education can be targeted to identified problem areas of use
-
What Can We All Do?
Observe how clinical users interact with devices (workflow
analysis)
Try your own heuristic analysis does the sequence of operations
make sense to you?
Target training on devices to address
problem areas
-
Conclusions
HF evaluation is a necessary process to help ensure a safer
patient environment when selecting medical devices or IT systems
for purchase
Educational approaches can be developed based on outcomes, and
general Human Factors training empowers staff
Incorporating HF into iterative design process results in safer,
easy to use products for the future
-
Thank you! Contact: [email protected]
