Upload
informa-australia
View
226
Download
2
Embed Size (px)
Citation preview
The role of Usability & UCD
in eHealth ProcurementAnne Miller, PhD, Assoc. Prof
Vanderbilt University Medical Center,University of Sydney,
eHealth NSW
Acknowledgements
The research reported in this presentation was funded by Vanderbilt University Medical Center,
Nashville, Tennessee.
The author acknowledges the contributions of Shilo Anders, Ray Booker, Dan France, Tiercy Forteberry, Thad
Hoffman, Peggy Joseph, Jennifer Slaughter, Marcella Woods, PhD, and Matt Weinger MD, PhD.
The views expressed in this presentation are the presenter’s own and may not reflect those of past or current employers.
e-health implementations• Include:
– Electronic health records– Practice management systems– Dispensing & laboratory
systems– Medical devices,– Personal use apps
HIT implementation outcomes
• 62% of HIT studies report positive outcomes (improvement in an aspect of care without negative effects)
• 38% of HIT studies reported mixed or negative outcomes
• Quantitative efficiency / effectiveness studies more positive outcomes
• Qualitative satisfaction studies find more negative outcomes
Buntin et al (2011) Health Affairs
Nature of HIT is changing…To now… •Isolated functions (e.g., CPOE)•Limited user group (e.g., physicians)•Defined processes (e.g., ordering)•One-to-one mapping intervention & outcomes
Next generation…•Multiple, network functions (e.g., Whatsapp)•Multi-user groups (incl. patients)•Emergent processes (image sharing & messaging)•Many-to-one intervention to outcome mapping
How can Human Factors help…
What is Human Factors?The Science and Practice of Socio-technical Systems design for efficient, effective, safe
and satisfying work.
•Science & Practice application of theory & method•‘Human’ psychology, sociology, anthropology•Business/Organization change management,
strategic directions•Technical systems engineering, physics, chemistry,
biology & domain expertise•Design interaction, interface, graphic, industrial
Usability & User Centered Design
User Centered Design (UCD)A process for achieving software quality that depends on:•Evidence-based understanding of contexts of use•The direct involvement of representative end-users•Iterative design & evaluation•Usability testing
Usability:Allows users to complete tasks in a context of use efficiently,
effectively, safely and with satisfaction
HF & Usability in HIT Procurement: Case Study
Positive Patient Identification project
Problem:VUMC had short listed two barcode blood checking products and wanted to make a purchasing decision. Could HF evaluation help to differentiate?
UCD in HIT Procurement: Case Study
Center for Research & Innovation in Systems Safety (CRISS):
Which of the two products:1.Is most likely to be accepted and used by clinicians; 2.Is easiest to learn/use; has lowest training costs3.Supports clinicians’ cognitive and work processes;4.Reduces blood administration errors, while not introducing new errors.
UCD in HIT Procurement: Case study Method:
Near-realistic multi-agent simulations: Operating Room, Emergency Department, Oncology Clinic & Paediatric Intensive Care Unit.
Participants: 22 clinicians drawn from each context
– 11 participants used product A– 11 participants used product B
Test tasks:1) Routine 2) Emergency compared to manual3) Wrong patient identification; 4) Wrong blood type identification
VUMC simulation center
UCD in HIT Procurement: Case study Results
Efficiency (Time on task): •No statistically significant differences Product A vs Product B•Manual vs Product A/B checking in emergency infusion: 15 vs 21secs
Effectiveness (# interaction errors / product / scenario): Product A clinicians 1.2 ±1.8 vs Product B clinicians 2.8 ± 4 interaction errors (p<0.05)
Safety:All wrong patient / wrong blood type errors identified. One potential error was identified.
Satisfaction ratings:No statistically significant differences in mean ratings
“It doesn’t give clear instructions—it tells you that the scan is wrong but doesn’t tell you which scan [is wrong],”
“You have to rework vitals, you write them and enter later when the screen comes up”
“I hit cancel and had to restart from the beginning.”
“It’s like using a brick - why can’t we have a (scanning) gun like the supermarkets use?”
UCD in HIT Procurement: Case study Conclusions
1.Is most likely to be accepted and used by clinicians; 2.Is easiest to learn/use; has lowest training costs;3.Supports clinicians’ cognitive and work processes;4.Reduces blood administration errors, while not introducing new errors.
“It’s like using a brick - why can’t we have a gun like the supermarkets use”
UCD in HIT Procurement: Case study Outcomes:
1. Each vendor received a confidential & comprehensive product specific report and were present during their product tests2. Results were published in ‘Transfusion’
Anders S, Miller A, et al. (2011) Blood product positive patient identification: Comparative simulation-based usability test of two commercial products. Transfusion. 51: 2311-2318
3. VUMC elected not to purchase either due to technological immaturity
– Levels of frustration experienced by staff mitigates against adoption - system likely to be abandoned
– Unlikely return on investment
Blood transfusion cross checking
Limitations• Simulated environments are not real environments• Effectiveness depends on detailed understanding of
contexts of use for representative test scenarios• Issues identified may not be remediable in current
HIT versions• May not capture issues effectively in more fully
networked HIT configurations
Healthcare without humans!
Conclusions• Human Factors & User Centered Design:
– Methods can provide purchasers and vendors with greater outcome certainty during procurement
– Fills a common gap in procurement processes – involvement by front-end clinicians doing realistic tasks
• As we move towards more networked configurations, Human Factors evaluation approaches are moving towards a more broader ‘pilot’ review approach that better fits real-world complexities
Thank you • Questions?
Nashville, Tennessee