Designing man–machine interactions for mobile clinical systems: MET triage support using Palm handhelds

European Journal of Operational Research 177 (2007) 1409–1417

www.elsevier.com/locate/ejor

Designing man–machine interactions for mobileclinical systems: MET triage support using Palm handhelds

Wojtek Michalowski a,*, Marta Kersten b, Szymon Wilk c, Roman Słowinski c

a MET Research Group, School of Management, University of Ottawa, 136 Jean-Jacques Lussier St.,

P.O. Box 450, Stn. A, Ottawa, Ont., Canada K1N 6N5b Medical Computing Laboratory, Queens University, Kingston, Canada

c Institute of Computing Science, Poznan University of Technology, Poznan, Poland

Available online 16 June 2005

Abstract

The Mobile Emergency Triage (MET) system is a clinical triage support system that aids physicians in making triagedecisions as to whether a child presenting in the Emergency Department of a hospital with a specific pain complaintshould be discharged to a family physician, needs to be admitted for further investigation/observation, or requiresan urgent specialist consult. The system�s mobile component is designed to work on handheld computers. This paperpresents our experience and discusses our original solutions with regard to designing man–machine interactions formobile clinical support systems. The specific interactions adopted in the MET design that are discussed in the paperwere created in consultation with potential end-users and tested at the Children�s Hospital of Eastern Ontario inOttawa.� 2005 Elsevier B.V. All rights reserved.

Keywords: Decision support systems; Mobile devices; Clinical decision making; Emergency triage support; Human–machine intera-ction

1. Introduction

The Mobile Emergency Triage (MET) system isa computer-based clinical support system that aidsemergency personnel in making a triage decision asto whether a given patient:

0377-2217/$ - see front matter � 2005 Elsevier B.V. All rights reservdoi:10.1016/j.ejor.2005.04.012

* Corresponding author. Tel.: +1 613 562 5800x4955.E-mail address: [email protected] (W. Mi-

chalowski).

(1) should be discharged to a family physician,(2) needs to be admitted for further observation,

or(3) requires an urgent specialist consult.

The MET system is developed in a modularway, with each module designed to evaluate a spe-cific acute pain condition. Thus, the system�s func-tionality can be easily modified by replacingexisting modules or by adding new ones.

ed.

mailto:[email protected]

1410 W. Michalowski et al. / European Journal of Operational Research 177 (2007) 1409–1417

The MET mobile component is implementedon a handheld computer, and at present we sup-port devices running Palm OS. MET users,namely medical residents and Emergency Depart-ment (ED) physicians, enter data into a handheldcomputer about a patient�s history, his/her physi-cal examination, and/or the results of tests. Thesystem has a triage function that allows a userto consult a rule-based decision model in orderto get a triage recommendation. The decisionrules constituting the model have been generatedfrom historical data (transcribed from verified pa-tient charts during a retrospective study) usingdata discovery techniques based on rough set the-ory (Michalowski et al., 2004). The triage func-tion can be invoked at any time in the triageprocess, irrespective of the amount of informationabout the patient�s condition that is currentlyavailable. The system consults rules that are mostappropriate from the point of view of availabledata. It infers from partially matched rules ifnot enough data about the patient�s condition isavailable, or there are no rules that can bematched against available information, and it in-fers from fully matched rules if enough data isavailable and appropriate rules exist. Matchingof the rules is translated into triage strength fac-tors calculated for each triage recommendation.The strength of the recommendation depends onthe amount of available data about a patient,and the generality of matched rules; the moredata is available and the more general rules arematched—the higher the strengths of the recom-mendations (Stefanowski, 1993). The triage rec-ommendation that acquired the highest strengthfactor is then labelled the ‘‘suggested triage’’,but all possible recommendations are presentedto a user.

It is worthwhile to notice that the system notonly helps in making triage decisions, but also con-tributes to improved management by allowingphysicians and residents to collect data in a struc-tured way, which by itself would lead to improvedclinical performance (Guerlain et al., 2001; Glaset al., 2002).

MET is designed to work in an ED environ-ment. Due to the significant space limitations ofmost EDs, use of a desktop computer in such

an environment is not a feasible option, andhence a handheld, that is a compact and mobiledevice, was selected. Mobility of the computingdevice is a significant advantage in terms of thesystem�s accessibility, but adds to the complexityof the design. Drawbacks such as small screen,limited display capabilities, and restrictive dataentry format place new requirements on the de-sign specifications and the quality of man–ma-chine interactions. Some design guidelines,especially those related to domain-specific anduser-centered interactions were applied whiledesigning MET (Gulliksen, 1996; Gulliksen andSandblad, 1995; Weinger, 2000). However, lackof well established and tested design principles,combined with the scarcity of research on stan-dards for mobile devices, made our work bothnovel and challenging.

The paper is organized as follows. In the nextsection we discuss the basics of man–machineinteractions in the medical domain. It is followedby a presentation of the solutions that were imple-mented in the MET system, and we give the ratio-nale behind them. The paper ends with adiscussion.

2. Man–machine interactions in the medical domain

Well-designed interactions are essential in com-puter systems developed for the medical domainbecause the cost of potential flaws can be enor-mous. Notwithstanding such a requirement, manyinterfaces in medical/clinical systems ‘‘are poorlydesigned, fail to adequately support the clinicaltasks for which they are intended, and frequentlycontribute to medical error’’ (Weinger, 2000). Pos-sible reasons for this situation include:

• lack of the ability to field test the interactionframework outside the application context(namely the operating room, the ED, theambulance),

• discontinuity between the development effortand end-user requirements.

Due to the nature of the application domain(clinical environment) and work pressure on end-

W. Michalowski et al. / European Journal of Operational Research 177 (2007) 1409–1417 1411

users, it is very difficult to develop a controlledclinical trial to evaluate the interaction frame-work. This is especially true for the ED, wherethe nature of the work and constant time pressureon the healthcare providers do not create the test-ing opportunities that are available for businessapplications. Such a situation leads to a phenom-enon that is well documented in the InformationSystems literature, namely a lack of congruencebetween design specifications and end-user expec-tations, further amplified by the fact that the anal-ysis stage of the system design is for practicalconsiderations curtailed because of the reasonsoutlined earlier. In order to address theseproblems Wiklund (1998) proposed to adhere tospecific principles while designing man–machineinteractions for medical devices. These principlescome mainly from software engineering research;thus a design framework proposed by him aimsto improve typical interactions by eliminatinginformation and control overload, eliminatingthe liberal use of colors in a manner that isinconsistent with medical conventions, and byallowing easy navigation within the system�scomponents.

The importance attached to well-designed andnon-ambiguous man–machine interactions in themedical domain intensifies the need for their do-main-specific and user-centered design (Gulliksenand Sandblad, 1995; Gulliksen et al., 1993). Thisimplies, on the one hand, an understanding of hu-man behavior in terms of task execution, reason-ing, problem solving and interpreting, and on theother hand, development of a ‘‘clean’’ interactionframework. Such a framework should allow anend-user to focus his/her attention on the tasksto be supported through enhancement of his/hercognitive abilities. This should be accomplishedby the use of domain-specific metaphors, and elim-inating end-user�s need to be concerned with theoperation of the computing device. While thereare some general frameworks enforcing such anapproach, they were not developed with a hand-held computer in mind. In the following section,we will describe how through research and exten-sive experimentation we were able to develop spe-cific interaction solutions for a mobile triagesupport system.

3. MET man–machine interactions

3.1. General design

The MET system design (presented in Fig. 1)follows the general principles of a client—serverarchitecture, with the server being responsible formanagement of the databases (database subsys-

tem), synchronization of the clients (sync subsys-

tem), and decision model calibration (model

subsystem). The mobile MET client, implementedon a handheld computer, is used for enteringinformation about a patient�s condition, and forinvoking a rule-based clinical algorithm for recom-mending the triage. The Web-based MET clientthat could be executed on a desktop computer con-nected to the Internet or Intranet, has the samefunctions as a mobile one, with the additionalcapability of managing the patients� data that iscollected on a mobile client.

The MET mobile client was developed accord-ing to object oriented principles, and its designblueprint is illustrated in Fig. 2. When designingthe system we separated out a generic solver (triage

subsystem) that could be used for various problemsfrom knowledge bases (modules) associated withspecific presentations of acute pain. Such a designnot only improves the reusability of the system�scomponents (there is no need to customize the tri-age subsystem for specific modules), but follows di-rectly from research in medical informatics, wheregeneric solvers reflect fundamental algorithms forprocessing data, and knowledge bases representspecific health-related application areas (Musen,1999).

For the purpose of this discussion, only the dia-

log and interface subsystems are of interest. Therole of the dialog subsystem is to manage man–ma-chine interactions according to the embedded prin-ciples that specify the sequence of displayingappropriate screens associated with individualsymptoms and signs, and prescribe the system�sbehavior in response to the end-user�s actions(e.g., tapping a button). The interface subsystem

is a repository of interface elements (buildingblocks) that are used by the dialog subsystem tointeract with an end-user. These blocks include:screens common to all clinical modules (e.g., login

Web-basedMET Client Model

subsystem

Syncsubsystem

MobileMET Client Database

subsystem

Modules Patients MET Server

Fig. 1. MET architecture.

Dialogsubsystem

Databasesubsystem

Triagesubsystem

Interfacesubsystem

Patients Modules

Fig. 2. Mobile MET client architecture.


screen, list of patients, etc.), general screens cus-tomized for the particular module during the run-time (e.g., history or report screen), and editors,i.e., tools used for editing the values of clinicalattributes (pictograms, check boxes, lists).

In developing the framework for man–machineinteractions in the MET mobile component, thefollowing domain and user-specific principles werefollowed:

• keeping the overall interactions transparent,simple, and clean,

• making efficient use of the display,

• eliminating data entry using a handwriting rec-ognition system (graffiti),

• contributing to the accomplishment of the taskat hand by using cognitive clues.

It is clear that the cognitive needs associatedwith a specific application domain should play aparamount role when designing interactions aimedat a particular work environment (Gulliksen et al.,1993). The MET system is designated to be used inthe ED, and any obstruction of the task at handcaused by the system would simply make it unus-able. ED medical personnel will be consulting the

Fig. 3. Main triage activities.

Fig. 4. Navigation between activities.


MET system while performing tasks under pres-sure, when the need to use a computer is often per-ceived as a hindrance. Thus, one of primaryobjectives was to develop a system that does notdeter medical residents and physicians from rou-tine patient management. Consideration of the in-tended user group, the way in which its membersperform tasks and what type of systems they areaccustomed to were very important factors whendeciding about specific solutions. The users ofthe MET system were more familiar with clinicalactivities, filling out patients� paper charts andadministrating tests, than with scrolling and navi-gating through screens, tapping buttons, and usinggraffiti. In order to facilitate their interactions withthe MET mobile client, symbols that are familiarto the end-users, such as check boxes, body picto-grams, icons with well-known cues (e.g., micro-scopes or thermometers) and labels used withinthe hospital, were incorporated as the main designfeatures. They were combined with intuitive navi-gation between the different system elements andthe elimination of the use of graffiti and textualdata entry.

3.2. Navigation

The main concept behind managing the naviga-tion between the different MET functional ele-ments was to focus on four main activitiesassociated with triage: taking a patient�s history,conducting a physical examination, evaluatingthe test results, and triaging a patient. Correspond-ingly, MET has four main screens—one for eachactivity. Selecting an icon associated with a givenactivity invokes the MET element responsible fordata gathering for that activity (see Fig. 3). Theicons are subsequently repeated (in smaller format)on every MET screen, thus enforcing a cognitiveimage of each of the activities, and allowing anend-user to move easily throughout the systemfunctions (see Fig. 4).

There are good reasons behind the use of iconsin MET, the first being that icons require lessscreen space then the usual navigational buttons,such as �next�, �back� and so forth. Also, the useof graphical symbols enhances man–machineinteractions, as suggested by Wiklund (1998).

Deriving the specific graphical icons used in theMET system was an interactive process character-ized by numerous consultations with the ED phy-sicians at the Children�s Hospital of EasternOntario. As a result of these consultations it wasdecided to use the graphical presentation of a labelstanding for the appropriate activity. These labels(Hx for History, PE for Physical Examination, Ix


for Tests, and TR for Triage recommendation) aretypical short forms used by physicians in describ-ing the routine activities on a patient�s chart.

The use of color should be consistent and notoverabundant in order to enforce the end-user fo-cus on the task to be accomplished. The METinterface relies on a few colors that are limited toicons and pictograms only. The colors enhancedwith text labels on the icons are used as visual cuesto reinforce the distinction between the four maintriage activities.

3.3. Inputting data

Data entry on a handheld device generally in-volves using a handwriting recognition system(graffiti), or the virtual pop-up keyboard. It is dif-ficult to master these data entry modalities, so wedecided to significantly simplify the input modein the MET system, and use fast shortcuts when-ever possible. In the case of attributes with a lim-ited number of possible values (i.e., those takingyes/no, present/absent, etc. values) it was achievedby presenting check boxes (see Fig. 5). Checkboxes are familiar constructs to all who have everfilled out a form, and therefore, they were selectedover pull down lists or other similar solutions. Aspecial case was made for those attributes that re-

Fig. 5. Using check boxes.

fer to a site on the human body (i.e., a site of pain,site of tenderness, etc.)—then, the check boxeswere augmented with pictograms representingappropriate body parts (i.e., abdomen) so possibleambiguities associated with the interpretation ofpresented choices are diminished. A user can iden-tify a specific location by tapping the appropriatearea on a pictogram or by checking a check box(see Fig. 6). The utilization of graphical controls(photographs, pictograms) for selected data entriesenhances the cognitive association between thetask and the MET function by presenting anend-user with an image that is very closely associ-ated with the clinical activity he/she is familiarwith.

Numeric attributes (those having a virtuallyunlimited number of possible values), such as tem-perature, or duration of pain, require differentdata entry modalities. The appropriate data is en-tered, using a simplified keypad that mimics aphone keypad (see Fig. 7); thus, such data entrytool should not present problems for inexperiencedusers.

3.4. Writing comments

Physicians are accustomed to writing commentson charts, so it is important that the MET system

Fig. 6. Using pictograms.

Fig. 7. Using a keypad.

Fig. 8. Writing comments.

Fig. 9. Presenting results.


also has such a facility. Following our earlierassumption about the cumbersome use of graffiti,we decided on an alternative comment entry for-mat. After analyzing hundreds of patients� charts,we have developed dictionaries with frequentlyused terms and keywords for almost all clinicalsymptoms and signs used in the system. The phy-sician simply needs to select an item from the dic-tionary to create a simplified comment (see Fig. 8).

3.5. Presenting results

As mentioned earlier, the system does not giveone definite suggestion, but shows all possible rec-ommendations that acquired positive strengths(see Fig. 9), and labels ‘‘suggested’’ the triage rec-ommendation with the highest strength. Thus, thesystem�s operation reflects the uncertainty that isinherent for clinical reasoning, further amplifyingthe fact that MET does not work as an oracle thatreplaces the physician, but as a helper (influenceror debiaser (Spreckelsen et al., 2000)) that triesto point out and eliminate possible biases when tri-aging a patient.

4. Discussion

As with any man–machine interaction frame-work, feedback from the intended end-users is ex-tremely important for the practical acceptance ofthe system. Throughout the design stage, medicalstudents, medical residents, ED physicians, andpaediatric specialists were consulted about alterna-tive design solutions. These consultations, com-bined with adhering to the principles of user-centered and domain-specific design, resulted inan interaction framework that is intuitive, easy


to use and supports the tasks at hand. It allows theED physicians to focus on the interactions thattake place while conducting the patient interviewor physical exam, and provides for a seamless tran-sition between the tasks of gathering and enteringdata and triaging a patient. Thus, we have devel-oped a man–machine interaction solution for amobile device that is in line with the end-user cog-nitive expectations, and minimizes the cognitiveburden associated with the change that wouldaccompany use of a computerized support toolfor ED triage decisions. Following the initial posi-tive reception of the MET system by the physi-cians, we had proceeded to evaluate MET�soperation during a regular clinical trial in the EDof a pediatric hospital. The trial lasted for 7months and ended in the winter of 2004. Duringthat period, the MET system was used by over150 different members of the ED staff (physiciansand residents). This group had diverse prior expe-rience with handheld computers, ranging fromcomplete novice to advanced users experiencedwith medical applications. Each of the ED staffmembers participated in a short orientation ses-sion after which he/she was able to operate theMET mobile client without any difficulties. Over-all, the participants were satisfied with the clearand easy-to-use interface.

The overall triage accuracy of the MET systemduring the trial was slightly higher, though not sta-tistically different, than the accuracy of ED physi-cians (72% for MET and 70% for the physicians).Though some may question the utility of a toolwith 72% accuracy, we maintain that the perfor-mance of the MET system is satisfactory, giventhat it is similar to this of the physicians them-selves, and that it is a support tool used at an earlystage of the patient�s assessment, when a smallamount of information is available. Moreover,while calculating the MET triage accuracy, weconsidered the recommendation with the higheststrength only and matched it against the gold stan-dard. In everyday use, the physicians interactingwith MET would receive the recommendationswith associated strength, and analyzing the differ-ences in strength between the recommendationsshould provide an additional piece of informationto be used while making a triage decision.

One of the interesting discoveries following theclinical evaluation of the MET system was theobservation that the individual�s acceptance of aclinical DSS in the ED was significantly influencedby the person�s comfort level with computing tech-nology in general. Thus, residents trained in emer-gency medicine that were exposed to computers asearly as primary school, perceived the handheld-based triage support as a helper in their work, aslong as it fitted the workflow. The same cannotbe said about older physicians whose comfort levelwith general computing technology was varied.

With the development of the MET system, wehave demonstrated that the creative use of existingdesign frameworks, combined with a thoroughunderstanding of the application domain, is essen-tial for the successful development of the man–machine interactions appropriate for the medicaldomain. We have shown that even complex appli-cations (much more sophisticated than standardaddress books or to-do lists available for hand-helds) can be successfully implemented on mobiledevices and introduced into practice if their designfollows the principles of simplicity enhanced withdomain-specific and user-centered guidelines.

Clean and intuitively designed interactions for aclinical application have the additional advantageof contributing to the structured collection ofinformation about a patient�s condition. Severalstudies (Glas et al., 2002; Guerlain et al., 2001)have shown that structured data collection andthe use of standardized forms to collect data aboutthe patient, improve the diagnostic performance ofER medical personnel in evaluating patients� con-ditions. This aspect should not be underestimatedwhen deciding about the format of man–machineinteractions in clinical support systems.

Acknowledgements

Research described in this paper was supportedby grants from the Natural Sciences and Engineer-ing Research Council of Canada.

The MET mobile component was developedusing AppForge MobileVB and Sybase iAnywhere.

The authors would like to thank Anne Burgessfor the English editing of the text.


References

Glas, A., Pijnenburg, B., Lijmer, J., Bogaard, K., de Roos, M.,Keeman, J., Butzelaar, R., Bossuyt, P., 2002. Comparisonof diagnostic decision rules and structured data collection inassessment of acute ankle injury. Canadian Medical Asso-ciation Journal 166 (6), 727–733.

Guerlain, S., LeBeau, K., Thompson, M., Donnelly, C.,McClelland, H., Syverud, S., Calland, J., 2001. The effectof a standardized data collection form on the examinationand diagnosis of patients with abdominal pain. In: Pro-ceedings of the Human Factors and Ergonomics Society45th Annual Meeting, Minneapolis, MN, pp. 1284–1289.

Gulliksen, J., 1996. Designing of efficient human–computerinterfaces in working life by means of a domain specificapproach. Graduate Thesis, Uppsala University.

Gulliksen, J., Sandblad, B., 1995. Domain specific design ofuser interfaces. International Journal of Human–ComputerInteraction 7 (1), 135–151.

Gulliksen, J., Johnson, M., Lind, M., Nygren, E., Sandblad, B.,1993. The need for new application specific interfaceelements. In: Salvendy, G., Smith, M.J. (Eds.), Human–Computer Interaction, Proceedings of the 5th International

Conference on Human–Computer Interaction, Orlando,FL, pp. 15–20.

Michalowski, W., Rubin, S., Słowinski, R., Wilk, Sz., 2004.Triage of acute abdominal pain in childhood: Clinical use ofa Palm handheld in a pediatric Emergency Department. In:Proceedings of the 37th Annual Hawaii InternationalConference on System Sciences (HICSS�04), IEEE, LosAlamitos, pp. 60161a (electronic version).

Musen, M., 1999. Scalable software architectures for decisionsupport. Methods of Information in Medicine 38, 229–238.

Spreckelsen, C., Lethen, C., Heeskens, I., Pfeil, K., Spitzer, K.,2000. The roles of an intelligent mobile decision supportsystem in the clinical workflow. In: Brandt, D., Cernetic, J.(Eds.), Automated Systems Based on Human Skill 2000(Joint Design of Technology and Organization). Pergamon/Elsevier Science, Oxford, New York, pp. 93–96.

Stefanowski, J., 1993. Classification support based on the roughsets. Foundations of Computing and Decision Sciences 18(3–4), 371–380.

Weinger, M., 2000. User-centered design: A clinician�s perspec-tive. Medical Device and Diagnostic Industry Magazine 1,80–88.

Wiklund, M., 1998. Making medical devices more user-friendly.Medical Device and Diagnostic Industry Magazine 5, 177–186.

Documents

Designing man–machine interactions for mobile clinical systems: MET triage support using Palm handhelds