Industry Proposal to Create Safe Oversight of Electronic Health Records from 1997

8/9/2019 Industry Proposal to Create Safe Oversight of Electronic Health Records from 1997

1/16

442 MILLER, GARDNER, Monitoring and Regulation of Clinical Software

JAMIAThe Practice ofInformatics

Position Paper

Recommendations forResponsible Monitoring andRegulation of Clinical

Software SystemsRANDOLPH A. MILLER, MD, REED M. GARDNER, PHD. For the American MedicalInformatics Association (AMIA), the Computer-based Patient Record Institute(CPRI), the Medical Library Association (MLA), the Association of Academic HealthScience Libraries (AAHSL), the American Health Information ManagementAssociation (AHIMA), and the American Nurses Association

A b s t r a c t In mid-1996, the FDA called for discussions on regulation of clinical softwareprograms as medical devices. In response, a consortium of organizations dedicated to improvinghealth care through information technology has developed recommendations for the responsible

regulation and monitoring of clinical software systems by users, vendors, and regulatoryagencies. Organizations assisting in development of recommendations, or endorsing theconsortium position include the American Medical Informatics Association, the Computer-basedPatient Record Institute, the Medical Library Association, the Association of Academic HealthSciences Libraries, the American Health Information Management Association, the AmericanNurses Association, the Center for Healthcare Information Management, and the AmericanCollege of Physicians. The consortium proposes four categories of clinical system risks and fourclasses of measured monitoring and regulatory actions that can be applied strategically based onthe level of risk in a given setting. The consortium recommends local oversight of clinicalsoftware systems, and adoption by healthcare information system developers of a code of good

business practices. Budgetary and other constraints limit the type and number of systems thatthe FDA can regulate effectively. FDA regulation should exempt most clinical software systemsand focus on those systems posing highest clinical risk, with limited opportunities for competenthuman intervention.

J Am Med Inform Assoc. 1997;4:442457.

Affiliations of the authors: Past President (RAM), and President(RMG) of the American Medical Informatics Association, Be-thesda, MD.

Dr. Millers efforts have been supported in part by Grants 5-G08-LM-05443 and 1-R01-LM-06226 from the National Library

of Medicine.

Correspondence and reprints: Randolph A. Miller, MD, Room436, Eskind Library, 2209 Garland Ave., Vanderbilt UniversityMedical Center, Nashville, TN 37232.E-mail: [email protected]

Received for publication: 7/3/97; accepted for publication:

7/17/97.

Health care practitioners, clinical facilities, industry,and regulatory agencies share an obligation to man-age clinical software systems responsibly, using a

common, equitable framework. In mid-1996, theUnited States Food and Drug Administration (FDA)called for new discussions on the regulation of stand-


2/16

Journal of the American Medical Informatics Association Volume 4 Number 6 Nov / Dec 1997 443

alone software programs as medical devices.1,2 In re-sponse, a consortium of health information-related or-ganizations has developed recommendations for publicand private actions to accomplish responsible monitor-ing and regulation of clinical software systems.

The consortium believes that implementation of anynew procedures for regulation of clinical software sys-tems as medical devices requires detailed prior anal-ysis of regulatory relevance to, or impact on, clinicalsoftware vendors, health care providers, and patients.Failure to carry out analyses prior to regulatory ac-tions could halt progress in an emerging new industrythat has substantial potential to improve the qualityof health care delivery. Manufacturers, users, and pa-tients cannot tolerate the delays in critical software

improvements that might result from excessive gov-ernmental review and approval processes.

All parties (including the FDA, consortium members,and organizations endorsing the consortium position)emphasize the same objectivesprotection and safetyof patients, and facilitation of approaches that im-prove health care delivery and outcomes.

The authors, in consultation with the Editors ofJAMIA and the Annals of Internal Medicine, have pre-pared a condensed version of this manuscript, moresuitable for a clinical audience, for concurrent publi-cation in the Annals of Internal Medicine (Miller RA,

Gardner RM, American Medical Informatics Associa-tion, et al. Summary Recommendations for the Re-sponsible Monitoring and Regulation of Clinical Soft-ware Systems. Forthcoming, Ann Intern Med, Vol.127, November 1997.)

Background

Benefits of Clinical Software Systems

Clinical software systems are defined here as individ-ual computer application programs, or interconnectedgroups of such programs, that are directly used in

providing health care. Clinical software systems re-quire supportive environments, including computeroperating systems and network interfaces. A growingliterature documents how clinical software systemscan improve health care delivery processes andoutcomes.4 4 3 Users employ such systems to track andmanage patient-related information, to retrieve localand general clinical information, and to apply clinicalknowledge in making patient-specific decisions. Clin-ical system usage encompasses hospital informationsystems and electronic record-keeping4 1 1; clinicaldata repositories1 2 1 3; health service-specific support(e.g., laboratory, pharmacy, and dietary systems);

decision support for diagnosis, therapy, or progno-

sis1 4 2 6; guidelines and reminders2 7 3 8; protocol man-agement3 9 4 0; telecommunication and tele-health41;signal processing (e.g., ECG interpretation sys-tems)42; image storage and analysis (e.g., picturearchival and communications systems PACS)43;advice-giving systems for patients; and other health-related applications.

To maximize benefit, providers should integrate signif-icant technological advances promptly and safely intoclinical practice. Currently, there are no widely accepted,practical standards for the evaluation, use, and moni-toring of clinical software systems. The FDA is only be-ginning to formulate a definitive policy with respect tosuch systems. In our opinion, regulation and oversightof clinical systems is both too important and too com-

plicated to be the sole responsibility of users, vendors,or regulatory agencies a combined approach is re-quired, with roles for each category of participants.

Obstacles to Evaluation and Monitoring ofClinical Software Systems

Determination of the safety of clinical software systemsis difficult because of the varied nature of clinical soft-ware, the changes that occur when a software productis integrated into a complex clinical information man-agement infrastructure, the changes to systems that oc-cur during maintenance, and the miscellaneous inter-

actions between software programs and their users.

Evaluation of Simple TurnkeyClinical Applications

It is difficult to evaluate and monitor even simple in-dependent, turnkey programssingle software prod-ucts that do not connect to or depend upon other ap-plication programs (other than an operating system).Such programs are often used as is on microcom-puters in individual practitioners offices, and onlymodified when users upgrade to the next version. Ac-cording to a recent survey, individual clinical vendorproducts number at least several thousand.45 In addi-tion, there exist thousands of health-related, Internet-

based World Wide Web resources of variable quality.

Persons evaluating the performance of a clinical soft-ware system employ numerous criteria in determin-ing whether a system merits purchase, installation,continued utilization, or approval by a regulatoryagency.4 6 5 2 The appropriateness of a given evaluationmethod varies with the stage of system develop-ment.47 While it is important during system develop-ment to evaluate system performance in isolated, ar-tificially controlled situations, evaluators of moremature systems should compare clinicians caring for

actual patients using the software to clinicians with-


3/16


out accessand not simply report system perfor-mance on a series of cases.

With respect to patient safety, approval of any sys-tems performance should be based on demonstrationof either absolute or relative benefit. For absolute ben-efit, system use should cause no measurable harm,produce outcomes at least as good as the status quo,and do so at an acceptable cost in time and money.For relative benefit, system use should demonstratenet improvement over the status quoi.e., the systemwill reduce the overall level of patient morbidity, mor-tality, or costs, even though the system itself maycause adverse effects. For example, electrocardiogram(EKG) interpretation programs have acceptable rela-tive benefit in patient care settings because they im-

prove upon many physicians ability to rapidly andeffectively interpret EKGs, even though it is widelyknown that they are not as authoritative as expertCardiologists, and may on occasion mislead healthcare providers.

Evaluation and Monitoring of Complex,Interconnected Clinical Software Systems

A software product may work well in isolation butfail when integrated with other software products orwith unsupported network interfaces. Large clinicalsites (such as tertiary referral centers) contain diverse

hardware platforms, multiple networks, and manyvendors software products. Clement J. McDonald re-cently estimated that large U.S. health centers inter-connect several dozen major clinical software systems,consisting of both vendors clinical software productsand locally developed software programs.44 A dia-gram of the HELP System, developed at LDS Hospitalin Salt Lake City, Utah,6 illustrates how complex suchsystems can become (Fig. 1). Suppose, for example,that a small to medium-sized health care institutiondecides to purchase and connect via a network a se-ries of applications for their laboratory, pharmacy, ad-mission/discharge/transfer (ADT), dietary, and cleri-

cal order processing activities. If the institutionconsiders ten different vendors that produce productsof the sort being considered, there are 100,000 differ-ent basic configurations possible. Major referral cen-ters install dozens of individual software components,each selected from more than a hundred possibleproduct configurations (Fig. 1). One hundred choicesfor each of 12 components yields a trillion trillion(1024) possible overall configurations for each largesite! Because each clinical site combines different soft-ware products in different combinations, a universalevaluation of whether or not a given product willfunction safely when embedded in a clinical environ-

ment is impractical.

Evaluation of Clinical Software Systems thatChange Over Time

A clinical application categorized as safe and effec-tive based on extensive testing at its inception might

become less than effective over time due to improperor inadequate maintenance. Both system softwarecode and the clinical data supporting system func-tions may be altered in a manner that invalidates eval-uation results for previous baseline products. Config-uration and integration of complex systems requirestesting, tuning, correction of software errors, modifi-cations based on installation environments and userfeedback, and upgrades, all of which increase localvariability over time. These changes occur on an al-most daily basis. Requiring a formal evaluation of

safety or efficacy related to each system change wouldparalyze ongoing implementation.

Users: An Important Consideration in Evaluationand Monitoring

Clinical software users include institutions, individualhealth care providers, and the general public, includ-ing patients. In analyzing causes of system-related ad-verse events, it is essential to consider end-userfactors.4 6 5 2 Systems with exemplary hardware andsoftware components can cause problems when usersdo not understand system applicability and limita-tions; when users do not understand how to inputcritical information into the system, when users can-not reliably interpret system output; or when it is dif-ficult to integrate system use into common workflowpatterns. Monitoring to detect problems often requiresaggregation of objective observations from a numberof sources and a perspective that goes beyond indi-vidual system components. Hence, raw complaintsfrom individual users need to be analyzed to deter-mine if the problem is in the software or in the usereducation program.

Past and Current FDA Regulation of ClinicalSoftware Systems

Through its mandate from Congress to safeguard thepublic, the FDA has regulated marketing and use ofmedical devices. Section 201(h) of the 1976 FederalFood, Drug, and Cosmetic Act defines a medical de-vice as any instrument, apparatus, implement, ma-chine, contrivance, implant, in vitro reagent, or othersimilar or related article, including any component,part, or accessory, which is . . . intended for use in thediagnosis of disease or other conditions, or in the cure,mitigation, treatment, or prevention of disease. . . or intended to affect the structure or any functionof the body.53 The FDA asserts that all clinical soft-

ware programs, whether associated with biomedical


4/16


F i g u r e 1 Diagram of HELP Information System at LDSHospital, Salt Lake City, Utah.6

devices or stand-alone, are contrivances, and there-fore fall within the FDAs realm of responsibility forregulating medical devices.

The FDA regulates medical devices that are: (a) com-mercial products used in patient care; (b) devices usedin the preparation or distribution of clinical biologicalmaterials (such as blood products); or (c) experimentaldevices used in research involving human subjects.Commercial vendors of specified types of medical de-vices must register as manufacturers with the FDAand list their devices as products with the FDA. Uponlisting, FDA classifies medical devices by categories.In its regulation of classified medical devices, the FDAusually takes one of three courses of action. First, theFDA can exempt specific devices, or categories ofdevices, deemed to pose little or no risk. Second, theFDA employs the so-called 510(k) process pre-mar-ket notificationfor non-exempt systems. Throughthe 510(k) process, manufacturers attempt to demon-strate that their devices are equivalent, in purpose andfunction, to low-risk (FDA Class I or Class II) devicespreviously approved by FDA (or to devices marketed

before 1976). Such devices can be cleared by FDA di-rectly. Finally, the FDA requires pre-market approval(PMA) for higher-risk (FDA Class III) products andfor products with new (unclassified) designs inventedafter 1976. Through the pre-market approval process,

a manufacturer provides evidence to the FDA that aproduct performs its stated functions safely and effec-tively. Evidence for PMAs usually is presented in theform of controlled trials. Pre-market approval is es-pecially important for those products which pose sig-nificant potential clinical risk. Exemption can takeplace in two ways: a device can be exempt from reg-istration, and thus not subject to 510(k) requirements;or, a category of listed (classified) devices may be spe-cifically exempted from certain regulatory require-ments. Whenever a non-exempt product is modifiedsubstantially (as defined by FDA guidelines), the ven-dor must re-apply to the FDA for new clearance

through the 510(k) or PMA mechanisms. The pro-cesses of 510(k) pre-market notification and pre-mar-ket approval typically take a few to many months tocomplete, and may involve numerous exchanges anditerations.

The FDA has a long history of regulating hardwaremedical devices, making it important to distinguish

between medical device-associated software and otherclinical software. Clinical software can be categorizedas stand-aloneexternal to and independent of amedical hardware device or embedded an in-tegral component of a medical hardware device. Of

note, a second connotation of stand-alone system

a single, independent turnkey systemwas pre-viously discussed. However, in the context of FDAdiscussions, stand-alone refers to independencefrom a traditional (hardware) medical device. Embed-ded software is often placed on a computer ROM(read-only-memory) chip that physically controls allor part of a hardware medical device, such as a lab-oratory auto-analyzer or a cardiac pacemaker. TheFDA regulates any embedded software program aspart of the medical hardware device itself.

In general, the FDA does not actively regulate locally

developed stand-alone software programs, whetherdeveloped by an individual or an institution, unlessspecial circumstances apply. One such circumstance islocal preparation of FDA-regulated biomaterials, suchas blood products. The FDA regulates individualstand-alone software products that are commerciallymarketed by individual manufacturers. It rarely spec-ifies or controls how independent vendors productscan be combined at a specific site, unless such prod-ucts are components of a single, larger system, suchas a PACS system. This is similar to FDA regulationof commercial pharmaceuticals, wherein the FDA reg-ulates each individual drug comprising a chemother-

apy protocol, but does not regulate multiple drug che-motherapy protocols themselves. The requirementsfor re-review are somewhat controversial, in that up-grading the network operating system of a compo-nent part of an FDA-regulated PACS system from ver-sion 3.1 to version 4.0 could potentially be viewed bythe FDA as requiring resubmission for 510(k) or PMAapproval. For this reason, the FDA has drafted andperiodically updated guidelines on when to submitfor re-approval.

In 1986, Frank Young, then director of the FDA, pro-posed a commendable plan for the oversight and reg-

ulation of clinical software.54 This plan evolved into a


5/16


1989 draft FDA policy statement that was never for-mally adopted. That draft policy has served, until re-cently, as the basis for the FDAs actions with respectto stand-alone software systems. The 1989 draft policyrecommended that the FDA exempt from regulation

both information-providing educational systems andsystems that generate patient-related advice for clini-cians in a manner that licensed practitioners (users)could easily override. During the 1990s, the FDA hasdeveloped new draft regulations and guidelines for

blood bank software systems55,56 and for PACS sys-tems.57 The FDA is developing new guidelines for te-lemedicine systems as well.58

At present, aspects of FDA regulation of clinical soft-

ware systems can be applied in an arbitrary manner.Although the FDA can initiate review of software prod-ucts brought to its attention, for the most part, theagency depends on vendors to submit their productsvoluntarily both for initial review, and review aftermodifications. The review process itself may vary, be-cause the FDA often employs a variety of different ev-aluators and consultants in reviewing similar products.Some vendors may be more likely than others to con-sider their software products as requiring FDA review.

3 Participating Organizations and

Consensus Process

Participating Organizations

The American Medical Informatics Association(AMIA) is a non-profit organization dedicated to im-proving health care through the application of infor-mation technology. The more than 3,700 members ofAMIA represent professions associated with health-care informaticsphysicians, nurses, biomedical en-gineers, computer scientists, information scientists,programmer-analysts, librarians, biomedical educa-

tors, biomedical researchers, vendor-consultants, den-tists, veterinarians, students, and a variety of otherhealth care practitioners. AMIAs goals are to promotethe development of health care informatics as a rec-ognized academic and professional discipline; to helpsolve health care problems through informatics re-search and development; and to promote diffusion ofknowledge in the discipline of health care informatics.The Computer-based Patient Record Institute (CPRI)is a non-profit membership organization committed toadvancing improvements in health care quality, costand access through routine use of information tech-nology. CPRI serves as a neutral forum for bringing

the diverse interests of all health care stakeholders to-

gether to develop common solutions. The Medical Li-brary Association (MLA) is a professional organizationrepresenting approximately 5,000 individuals and in-stitutions involved in the management and dissemi-nation of biomedical information to support patientcare, education, and research. MLA members serve so-ciety by developing new health information deliverysystems, fostering educational and research programsfor health sciences information professionals, and en-couraging an enhanced public awareness of health careissues. The Association of Academic Health SciencesLibraries (AAHSL) is composed of the directors of li-

braries of 142 accredited U.S. and Canadian medicalschools. AAHSLs goals are to promote excellence inacademic health sciences libraries and to ensure that

the next generation of health practitioners is trained ininformation-seeking skills that enhance the quality ofhealth care delivery. The American Health InformationManagement Association (AHIMA) is the professionalorganization of more than 37,000 specialists in secur-ing, analyzing, and managing patient records andhealth information. AHIMA members support qualitypatient care through advancing data accuracy, advo-cating confidentiality, and championing new informa-tion management technology. AHIMA, founded in1928, accredits educational programs at 250 collegesand universities and is the certifying agency for healthinformation managers and technicians. The American

Nurses Association (ANA) is the only full-service pro-fessional organization representing the nations 2.5 mil-lion Registered Nurses through its 53 constituent as-sociations. ANA advances the nursing profession byfostering high standards of nursing practice, promotingthe economic and general welfare of nurses in theworkplace, projecting a positive and realistic view ofnursing, and by lobbying the Congress and regulatoryagencies on health care issues affecting nurses and thepublic. ANA has been involved in healthcare infor-matics since the early 1970s and continues to activelyengage in diverse informatics activities. The AmericanCollege of Physicians (ACP) was founded in 1915 to

uphold the highest standards in medical education,practice, and research. Today, the College is the largestmedical specialty society in the world. It has more than100,000 members, including medical students as wellas practicing physicians, and it is the only society ofinternists dedicated to providing education resourcesand information resources to the entire field of internalmedicine as well as its subspecialties. The Center forHealthcare Information Management (CHIM) is a na-tional, 100-plus member association for companiessupplying information technology products and ser-vices to the healthcare industryincluding softwaresuppliers, consultants, hardware firms, network inte-

grators, medical imaging companies, publishers, and


6/16


executive recruiters. CHIM members seek to bring agreater understanding and awareness among health-care professionals as to how information technologycan be harnessed to improve the quality and cost-ef-fectiveness of patient care.

Consensus Process

In an attempt to develop a parsimonious approach tohandling proposals for changed regulations, the FDAhas conducted, during 1996 and 1997, a series of publicmeetings that have included leaders from the informa-tion technology and health care arenas, representativeprofessional organizations, and vendors. The first draftof the consensus recommendations was prepared as adiscussion document by authors RAM and RMG, as

representatives of the American Medical Informatics As-sociation (AMIA) at the first (and subsequent) publicmeetings held by the FDA. Key contributions to theevolving position paper were then made by other mem-

bers of the AMIA Public Policy Committee and theAMIA Board of Directors. After initial endorsement bythe AMIA Board of Directors, subsequent suggestionsand further revisions were contributed by the Center forHealthcare Information Management (CHIM); the Com-puter-based Patient Record Institute (CPRI), the MedicalLibrary Association, the Association of Academic HealthSciences Libraries (AAHSL), the American Health Infor-mation Management Association (AHIMA), and the

American Nurses Association (ANA). The Boards of Di-rectors (or Regents) of the following not-for-profit or-ganizations have now endorsed the recommendationsentailed in this document: the American Medical Infor-matics Association (AMIA), the Computer-based PatientRecord Institute (CPRI), the Medical Library Association(MLA), the Association of Academic Health Sciences Li-

braries (AAHSL), the American Health InformationManagement Association (AHIMA), the AmericanNurses Association (ANA), and the American Collegeof Physicians (ACP). The Center for Healthcare Infor-mation Management (CHIM) has reviewed successivedrafts of this document and contributed to its writing.

CHIM is supportive of the consortiums position, andholds the same opinions expressed in this document inall areas except for the definitions for proposed risk cat-egories of clinical software systems and classes of reg-ulations. The Appendix presents an algorithm prepared

by CHIM suggesting how the FDA might review stand-alone clinical software systems for regulatory purposes.It should be noted that, while CHIM includes in itsmembership a number of commercial vendors from thehealthcare information industry, the viewpoints of in-dividual members have not been represented in thisconsensus document, and the original and subsequentdrafts of the consortiums recommendations have been

prepared and endorsed by not-for-profit organizations.

Consortium Recommendations

The consortium recommends that users, vendors,and regulatory agencies (including the FDA) adoptthe guidelines detailed below. Detailed explanationsand justifications follow in the section after the rec-ommendations themselves. The recommendations go

beyond the scope of interventions that the FDA orother vendor groups would ordinarily undertake,since they include clinical software products that arelocally developed, shareware, non-commercial, andcommercial.

Recommendation 1: We propose four categories ofclinical software system risks and four classes of mea-sured regulatory actions as a template for clinical fa-cilities, vendors, and regulatory agencies to use in de-termining how to monitor or regulate any givenclinical software system (Tables 1A and 1B).

Recommendation 2: We recommend local oversightof clinical software systems whenever possible,through creation of Software Oversight Committees,or SOCs (Tables 2A through 2D).

Recommendation 3: Budgetary and other constraintslimit the type and number of systems that the FDAcan regulate effectively. We recommend that the FDAfocus its regulatory efforts on those systems posing

highest clinical risk that give limited opportunities forcompetent human intervention (Tables 2A2D). Themajority of clinical software systems should be ex-empt from FDA regulation. The FDA should requireproducers of exempt clinical software systems to listthem as products with the FDA for simple monitoringpurposes i.e., without having to undergo the 510(k)or PMA processes. The FDA should develop new,comprehensive standards for product labeling that areappropriate for clinical software products. The FDAshould require manufacturers of most exempt and allnon-exempt software products to adhere to labelingstandards (Tables 2A2D).

Recommendation 4: We recommend adoption byhealth care information system vendors and local soft-ware producers of a code of good business practices.The practices should include (but not be limited to)guidelines for quality manufacturing processes; stan-dardized, detailed product labeling; responsible ap-proaches to customer support; and, adoption of in-dustry-wide standards for electronic informationhandling and sharing including standards forhealth care information format, content, and trans-port.

Recommendation 5: We recommend that health in-

formation-related organizations work together with


7/16


Table 1A

Consortium Recommendations for Risk-based Categories of Clinical Software Systems

Category Description Category Description

0 Informational or generic systems that providefactual content (electronic textbooks); verifia-

bly calculate (showing all parameters used)simple quantities, such as drug dosage

based on body weight; or give simple,straightforward advice based on user-re-viewed guidelines (e.g., give potassiumsupplementation to patients receiving di-goxin who are hypokalemic). Includescontent-free generic software such as gen-eral database programs, generic spreadsheetprograms. Non-clinical systems also fall inthis category.

provide easy opportunities for practitioners toignore or override them. For example, systemsin this category might recommend a single pa-tient-specific therapy over a number of alter-native forms of therapy; recommend that thepractitioner commit to (conclude) a specific di-agnosis from a patient-specific differential di-agnosis; or guide patients in determiningwhich advanced-care directives might be ap-propriate for their situation. Systems in thiscategory might also store and transmit patient-specific instructions for life-critical care (e.g.,ventilator management or chemotherapy or-ders) to practitioners in a manner that is

1 Low-risk, patient-specific systems that performcomplex health-related functions with rela-tively low clinical risk and provide ampleopportunities for practitioners to ignore oroverride them. Such systems might non-

judgmentally suggest a number of alterna-tive forms of therapy; list a comprehensivepatient-specific differential diagnosis with-out making a conclusion; or provide an esti-mate of the patients prognosis based onmatched cases from a clinical database. Sys-tems in this category might also store andtransmit patient-specific passive data (e.g.,laboratory results or clinical reports) in amanner that is easily verified.

3

easily verified.

High-risk, patient-specific systems that pro-vide complex, health-related functions thathave high clinical impact and provide littleif any opportunity for practitioners to inter-vene in their function or to override them.For example, systems in this category mightinclude individualized chemotherapy mixingand dispensing systems, systems that auton-omously plan courses of radiation therapyfor uploading into automated equipment todeliver the therapy, and systems that moni-tor physiological parameters and automati-cally adjust settings related to therapy (for

example, a ventilator auto-pilot).2 Intermediate-risk, patient-specific systems that

provide complex, health-related functionsthat have relatively high clinical risk, but

other groups, including clinical professional associa-tions, vendor organizations, regulatory agencies, anduser communities, to advance our understanding andknowledge of approaches to regulating and monitor-ing clinical software systems.

Explanation and Justification forConsortium Recommendations

Explanation of Recommendation 1: Risk andRegulatory Categories

Software installation and maintenance must be treatedas a process, not a single event. Review of a softwareenvironment at one point in time does not guaranteesafety or efficiency at a later point in time. Decisionsabout whether to install and how to monitor clinicalsoftware systems should take into account: (a) the

clinical risks posed by software malfunction or mis-

use; (b) the extent of system autonomy from useroversight and control the inability for qualifiedusers, such as licensed practitioners, to recognizeand easily override clinically inappropriate recom-mendations (or other forms of substandard softwareperformance); (c) the pattern of distribution and de-

gree of support for the software system, including lo-cal customization; (d) the complexity and variety ofclinical software environments at installation sites; (e)evolution of systems and their environments overtime; and, (f) the ability of proposed monitors or reg-ulators to detect and correct problems in a timelymanner that protects patients. The ability of licensedclinician-users to override a system should be a con-sideration for decreased regulatory intervention. It isimportant to identify the most logical forum for sys-tem oversight. The best choice will often be local mon-itoring through SOCs (as described below), ratherthan nationally centralized data collection and moni-

toring.


8/16


Table 1B

Consortium Recommendations for Regulatory Classes

Class Description Recommendation

A Exempt from FDA regulation; local SOC monitoring op-tional

Good business practices would be applied by develop-ers, vendors, and users.

B Exempt from FDA regulation; local SOC monitoringmandatory

Commercial products in this category should adhere toproduct labeling standards, and commercial vendorsshould list products with FDA for simple identifica-tion purposes (no FDA review required). Good busi-ness practices recommended. Standardized surveys ofisolated users could be conducted on request by theFDA (or FDA-approved regional organizations) for in-dividuals who lack resources or expertise to imple-ment local monitoring and review guidelines. TheFDA would serve as a consultant to manufacturers,vendors, and SOC committees, rather than as a regu-lator.

C Simplified, expedited initial FDA approval through ver-ification that product labeling is accurate and adheresto standards; mandatory listing of products with FDAfor post-marketing surveillance; application of usualFDA 510(k) or PMA processes to any productsgenerKating concerns during post-marketing surveil-lance; local SOC monitoring mandatory.

Good business practices required. Reporting and re-cording of adverse events required with FDA collat-ing and aggregating standardized reports after localcollection and verification. FDA can require vendorswith unusually large number of unexplained adverseevent reports to undergo 510(k) or PMA to documentsafety and efficacy. In such trials, FDA should employstandard that the system demonstrates absolute orrelative benefit (as defined above). Vendors requiredto maintain lists of users of each version of each sys-tem in this class.

D Usual FDA 510(k) or PMA processes applied prior tomarketing; local SOC monitoring mandatory.

Prior to marketing, vendors must submit to FDA evidencethat system safely and efficaciously performs its statedfunctions. Good business practices required. Vendorsmust maintain lists of users of each version. Recording

of adverse events required locally and reported to FDAusing standardized forms for aggregation of data.Troubleshooting activities aggregated locally and re-ported after filtering to FDA. FDA may require addi-tional testing if the rate of adverse events is too high.

We have proposed categories of clinical software sys-tems based on patient risk and classes of regulatoryinterventions, as detailed in Tables 1A and 1B. Tables2A through 2D represent our recommendations onhow to apply the classes of regulatory actions respon-sibly to all clinical systems, based on the systems risk

categories. The recommendations in Tables 2Athrough 2D cannot be carried out solely by the FDAor by local institutions, vendors, or users. A combinedapproach with shared responsibilities is required. Theconsortium recommendations represent such an ap-proach.

Explanation of Recommendation 2: LocalMonitoring of Clinical Software Systems

Local software installation sites have the greatest abil-ity to detect software problems, analyze their impact,and develop timely solutions. It would be advanta-

geous to develop a program of institutional- and ven-

dor-level controls for the majority of clinical softwareproducts, rather than to mandate comprehensive,cumbersome, inefficient, and costly national-levelmonitoring.

Institutional Review Boards (IRBs) provide an exam-

ple of a local monitoring process that is in widespreaduse.5 9 6 1 In order to protect the human subjects of re-search, federal law has required each clinical facilityengaged in patient research have an IRB to review,approve, and monitor research protocols locally. EachIRB includes clinical experts, administrators, individ-uals familiar with research study designs and meth-ods, and persons experienced in data analysis. SomeIRBs also include laypersons as representatives of thepatient community. The IRBs review the purpose ofproposed research; the anticipated benefits to individ-uals and to the general community; the risks to sub-

jects in terms of health outcomes, pain and suffering,

and expenses; informed consent, voluntary participa-


9/16


Table 2A

Proposed Classes of Clinical Software Exemptions

and Regulation by Category of SoftwareType of

Software Description

Class A Software exempt from FDA regulation; lo-cal SOC monitoring optional. (Please re-fer to definitions of risk categories andregulatory classes in Table 1.)

Category 0 Informational or generic productsCategory 0 Non-clinical systems

Any system not directly involved in patientcare and any system not serving as anintegral component of a larger systemproviding patient carei.e., systems thatdo not play any role in suggesting diag-noses, suggesting prognoses, or suggest-ing or implementing orders or treat-ments.

Table 2B

Proposed Classes of Clinical Software Exemptionsand Regulation by Category of Software

Type of

Software DescriptionClass B Software Exempt from FDA regulation; lo-

cal SOC monitoring required. (Please re-fer to definitions of risk categories andregulatory classes in Table 1.)

All Category 1 All low-risk, patient-specific, low-impactsoftware giving adequate control to li-censed practitioner (easily overridden).

Some Category 2 Intermediate-risk, high-impact patient-spe-cific software giving adequate control tolicensed practitioner (easily overridden),including locally developed or share-ware non-commercial software systems;local SOC monitoring recommended forsuch software. EXCEPT: Category 2 sys-

tems commercially distributed that arenot modified in any way locally andthat do not serve as part of an inte-grated local system.

Some Category 3 Locally developed, non-commercial, pa-tient-specific, high-impact systems withminimal ability of practitioner to over-ride. It is mandatory, on an ethical basis,that such systems have careful local re-view and institutional oversight through

both IRBs and SOCs. Such systemsshould adhere to product labeling stan-dards and be registered for simple iden-tification purposes with FDA, eventhough non-commercial. EXCEPT: Cate-

gory 3 systems distributed commercially.

tion, and ability to withdraw from the research study;and the plans for monitoring and conduct of the re-search protocol itselfe.g., ability to detect adverseoutcomes or grossly positive benefits so that the studycan be terminated early, if indicated. In making deci-sions, the locally autonomous IRBs can take regionaldemographics, local practice patterns, patient con-cerns, and individual researchers skills and pastrecords into account much more efficiently and effec-

tively than could a centralized national agency.While IRBs per se are not well qualified to review andmonitor clinical software systems, they provide amodel for creating a multidisciplinary team with ap-propriate expertise at the local level. Local and re-gional Software Oversight Committees (SOCs) couldenlist members with expertise in health care infor-matics, clinical practice, data quality, biomedicalethics, patients perspectives, and quality improve-ment. When the complexity, diversity, and number ofclinical software and computer hardware products atan installation site reach a critical size, a local SOCshould be formed to review clinical software on anongoing basis within the institution. On a similarnote, the Joint Council for the Accreditation of Health-care Organizations (JCAHO) reviews organizationssystems and processes for improving their perfor-mance, and specifically includes information manage-ment as one of the areas reviewed. Small practitionersoffices or smaller community hospitals without ade-quate expertise could participate in regional SOCs, orpossibly request consultations from local SOCs atlarger institutions.

The SOCs would develop and follow guidelines forlocal or regional software quality maintenance similar

to the International Organization for Standardiza-

tions (ISO) 9000 guidelines62 for quality manufactur-ing processes. More than 80 countries have adoptedthe ISO 9000 for consistency in manufacturing pro-cesses. ISO 9000 requires that manufacturers produceexplicit documentation for accepted procedures for all

business activities; implement methods to prove thatprocedures are followed correctly and perform theirintended functions; conduct audits of process quality;and implement improvements or corrections whenproblems are detected. The SOCs would monitor pro-cedures by which an institution goes through opera-tional needs assessment; specification of desired sys-tem functions; selection of vendors products (or localproduct design and development); testing of productsin a realistic environment before going live; ade-

quate training of prospective users; installation andfollow-up during and after installation of softwaresystems; monitoring of users competencies and com-plaints; and the adequate provision of a help desktied to documentation procedures and methods formaking software improvements. SOCs could help en-sure that institutions build clinical enterprises to-


10/16


Table 2C

Proposed Classes of Clinical Software Exemptions

and Regulation by Category of SoftwareType of

Software Description

Class C Software subject to simplified, expeditedinitial FDA approval through verifica-tion that product labeling is accurateand adheres to standards; mandatorylisting of products with FDA for post-marketing surveillance; application ofusual FDA 510(k) or PMA processes toany products generating concerns dur-ing post-marketing surveillance; localSOC monitoring mandatory. (Please re-fer to definitions of risk categories andregulatory classes in Table 1.)

Some Category 2 Intermediate-risk, high-impact, patient-specific software giving adequate controlto licensed practitioner (easily overrid-den), commercially distributed and notmodified in any way locally.

Table 2D

Proposed Classes of Clinical Software Exemptionsand Regulation by Category of Software

Type of Software Description

Class D Software subject to FDA 510(k) and PMAapproval before marketing; Local SOCmonitoring mandatory. (Please refer todefinitions of risk categories and regula-tory classes in Table 1.)

Some Category 3 High-risk, high-impact, patient-specificsoftware giving adequate control to li-censed practitioners (not easily overrid-den), commerically distributed and notmodified significantly locally. Systemswould also adhere to product labeling

standards.

ward a reference architecture with a modular designso that components can be replaced as they are out-dated.

SOCs would work with system administrators, users,and vendors to make sure there is ongoing monitor-ing to detect adverse events, address them, and to in-

sure that the overall system performs as designed;and, that adequate attention is paid by vendors tomake sure that vendor-product related problems de-tected are corrected in a timely manner (appropriatefor the clinical risk posed by the problem). It wouldalso be important to develop ethical guidelines thatwould specify behavior for employees and SOC com-mittee members who might have special relationshipswith vendors or have software-related companies oftheir own.

Explanation of Recommendation 3: A FocusedFDA Strategy for Exemption and Regulation of

Clinical Software SystemsThe consortium recommends local oversight of clini-cal software systems through SOCs (as describedabove) whenever possible, and adoption by healthcare information system developers and vendors of acode of good business practices (see Explanation ofRecommendation 4, below). Governmental regulatorshave a legislative obligation to play a meaningful rolein assuring patient safety. To maximize benefit of FDAefforts, we believe that the agency should focus onthose commercial stand-alone systems presenting thehighest degree of clinical risk (as defined in Tables 1Aand 2A to 2D). The FDA should move to formalize

the spirit of the 1989 FDA draft policy54 in a clearly

worded new policy. The FDA should now define ex-plicitly the categories of clinical software systems thatwill be exempt from its regulation. Recently, the Cen-ter for Healthcare Information Management (CHIM)has prepared an algorithm, expressed in the form ofa flow sheet, suggesting how the FDA might go aboutclassifying stand-alone clinical software systems forregulatory purposes (see Appendix). This documenthas not been reviewed for endorsement by consor-tium members. However, the document is consistent,for the most part, with the portion of consortium rec-ommendations related to the FDA.

Other than exemption, the two actions now availableto FDA for medical software devicesthe 510(k) pro-cess and the PMA processare inappropriately cum-

bersome for all but the highest risk category of clinicalsoftware systems. However, the FDA might serve aless intrusive monitoring role by requiring producersof exempt clinical software systems to list them asproducts with the FDA for monitoring purposes,without having to undergo the 510(k) or PMA pro-cesses. By assigning a universal registration ID to eachproduct, the FDA could develop a centralized data-

base repository for collecting information on adverseclinical software events. Reporting of problems withindividual products could come to the FDA throughSOCs, or from individual users in sites without SOCs.The FDA could then collect and distribute aggregated,standardized reports of system-specific and globalproblems (including interactions).

Another beneficial role the FDA could play would beto develop, in conjunction with vendors, clinical sites,professional organizations, and users, new, compre-hensive standards for clinical software product label-ing. Labels should describe system requirements,functions, document sources of medical information,and describe limitations of use. Labeling standards for


11/16


clinical software would be different than existing FDAstandards developed primarily for hardware medicaldevices. The FDA could require manufacturers of bothexempt and non-exempt software products to adhereto the labeling standards. As noted in Table 2C, theFDA could modify its approach to certain intermedi-ate-risk commercial products by creating an expeditedprocess to verify that the products labeling conformsto FDA standards. The FDA would then assign a post-marketing surveillance ID to such products and care-fully monitor for evidence of adverse effects. If a suf-ficient number of concerns were raised duringproduct monitoring, the manufacturer would then berequired to submit full-scale 510(k) or PMA applica-tions to be permitted to continue marketing.

Explanation of Recommendation 4: Adoption ofGuidelines by Clinical Software Producersand Distributors

The health care information system industry, throughthe Center for Healthcare Information Management(CHIM), is in the process of refining its code of good

business practices. This code, in conjunction with ISO-9000, is expected to address use of sound softwaredevelopment and implementation methods appropri-ate to the level of clinical risk posed by a softwaresystem. It will also address the need for adequatetraining, for support of open industry standards for

messages and communication, for protecting patientconfidentiality, and for other relevant matters. Adher-ence to the guidelines should be strongly encouragedfor all clinical software systems, and be mandatory forhigher-risk systems. The FDA Current Good Manu-facturing Procedures (CGMPs) for individual prod-ucts may not work effectively for the complex soft-ware environments in large health care deliveryfacilities (see Fig. 1). The FDA currently requires med-ical device manufacturers to comply with ISO-9000-like regulations as part of their manufacturing pro-cess. Through SOCs, local institutions should developguidelines for the acquisition, testing, installation,training, and monitoring of the potentially complexsystems under their control, in the spirit of ISO-9000.Local SOCs should also oversee implementation andmonitoring of local guidelines, and interinstitutionalsharing of experiences.

Examples of guidelines for good business practices in-clude the following: Manufacturers should discloseexisting evaluations of software products (noting howthey relate, if at all, to the current product) for usersto review before purchasing systems. Developersshould help users to estimate how often and by whatmechanism system componentsincluding databases

or knowledge bases, as well as software codeneed

to be upgraded or replaced. Users and/or external re-viewers should determine if upgrades are of profes-sional quality. Vendors and users should verify thatupgrades are made available or distributed to all whoshould receive them. Vendors should help local usersto ensure that only users who are well qualified (i.e.,possess sufficient biomedical knowledge) and welltrained (i.e., have adequate skill in using the program)will have access to a given clinical software system.In addition to standard software version control prac-tices, developers should document sources and datesof creation/revision for biomedical knowledge em-

bedded in software programs. Manufacturers shoulddisclose any known risks and limitations associatedwith a clinical software product, and inform users of

their responsibility to detect and override faulty sys-tem recommendations. Outdated systems shouldadvertise to users that their components are poten-tially invalid e.g., through start-up screens thatforce users to acknowledge that the system is out-dated before allowing the users to proceed, or throughprominent banners displayed at all times during sys-tem usage. If warnings are not adequate for high-riskor significantly compromised outdated components,the system might simply prevent users from usingoutdated functions by removing access to the func-tions from the system. Vendors, as well as regulators,should provide standardized forms and convenient

avenues for problem reporting. Manufacturers shouldensure that notification procedures are in force forhigher-risk product categories to ensure users areaware of product alerts, recalls, and upgrades. Man-ufacturers should continue to utilize guidelines thatprotect patients and users through the expedited, ef-ficient testing of new system components and up-grades. Last but not least, manufacturers shouldadopt and adhere to, whenever possible, national orinternational standards for data representation andexchange.63

Explanation of Recommendation 5:Collaboration to Evolve Clinical SoftwareMonitoring and Regulation

Institutions with installed advanced health care infor-mation systems should implement SOCs and sharetheir experiences and recommendations with others.The expanded consortium, consisting of clinical pro-fessional associations, vendor organizations, regula-tory agencies, and user communities, should evolvethe guidelines contained in recommendations 1through 4 as they gain increasing experience withapproaches to monitoring and regulating clinicalsoftware systems. Focus groups, the peer-reviewed

literature, regional and national conferences, Internet-


12/16


based information resources, and other means shouldbe used to disseminate the information.

Summary

Clinical software systems are ubiquitous. A growingliterature documents the benefits of such systems forimproved health care delivery. While no reports sug-gest that many patients are being harmed by stand-alone clinical software systems, concerns for patientsafety must be addressed. The consortium recom-mends local oversight of clinical software systemsthrough Software Oversight Committees (SOCs), andadoption by health care information system vendorsof a code of good business practices. Budgetary andother constraints limit the type and number of sys-tems that the FDA can regulate effectively. Most clin-ical software systems should be exempted from FDAregulation. FDA regulation should focus on patient-specific commercial software systems that pose highclinical risk (e.g., directly control life support systemsor directly administer potentially dangerous thera-pies) that are not modified locally (i.e., are not underlocal programming control) and which offer little orno opportunity for practitioner intervention. For suchclosed loop systems, based on the degree of riskposed, we recommend the traditional FDA 510(k) no-tification process and full-scale pre-market approval.

During pre-market trials for such narrowly defined,high-risk, closed loop systems, it must be demon-strated that the systems cause no harm, or, alterna-tively, that the automated systems improve upon im-perfect baseline (manual) systems at a tolerable(non-zero) level of risk and at an affordable cost.

We have provided recommendations on how to de-velop and realize processes for responsible monitor-ing and regulation of clinical software systems. Ourgoal is to encourage a coordinated effort to safeguardpatients, users, and institutions as clinical systems areimplemented to improve clinical care processes.

This document represents the opinions of the authors and ofparticipating consortium organizations, and it does not repre-sent positions of the FDA or of other governmental or regula-tory agencies. The authors thank Harold M. Schoolman, MD,for his insightful comments during the drafting and revision ofthis manuscript. The American Thoracic Society, the Associationof Operating Room Nurses, the American Association of Oc-cupational Health Nurses, the National Association of SchoolNurses, the Society of Gastroenterology Nurses and Associates,and the American Radiological Nurses Association have sentletters of support.

References

1. Federal Register. July 15, 1996. 61:36886 7.

2. Food and Drug Administration Center for Devices and Ra-

diological Health. Announcements and minutes of meetingson future regulation of stand-alone clinical software sys-tems. 1996 1997. FDA Web site: http://www.fda.gov/cdrh

3. AMIA News. J Am Med Inform Assoc. 1994;1:91 2.4. Institute of Medicine. Dick RS, Steen EB (eds). The com-

puter-based patient record: an essential technology forhealth care. Washington DC: National Academy Press. 1991.

5. Ball MJ, Collen MF (eds). Aspects of the Computer-basedPatient Record: Computers in Health Care Series. NewYork: Springer-Verlag, 1992.

6. Kuperman GJ, Gardner RM, Pryor TA. HELP: A DynamicHospital Information System. New York: Springer-Verlag,1991.

7. McDonald CJ, Tierney WM, Overhage JM, Martin DK, Wil-son GA. The Regenstrief Medical Record System: 20 yearsof experience in hospitals, clinics, and neighborhood healthcenters. MD Comput. 1992;9:20617.

8. Barnett GO. The application of computer-based medical rec-ord systems in ambulatory care. N Engl J Med. 1984;310:164350.

9. McDonald CJ, Tierney WM. Computer-stored medicalrecords: their future role in medical practice. JAMA. 1988;259:343340.

10. Garrett LE Jr, Hammond WE, Stead WW. The effects ofcomputerized medical records on provider efficiency andquality of care. Meth Inform Med. 1986;25:1517.

11. Rector AL, Glowinski AJ, Nowlan WA, Rossi-Mori A. Med-ical-concept models and medical records: an approach

based on GALEN and PEN&PAD. J Am Med Inform Assoc.1995;2:1935.

12. Yount RJ, Vries JK, Councill CD. The Medical Archival Sys-tem: an Information Retrieval System based on DistributedParallel Processing. Information Processing & Management.1991;27:37989.

13. Vries JK, Yount RJ, Singh J: Total integration of health centerinformation through distributed parallel processing. HIMSSProceedings. 1994;4:24152.

14. Warner HR, Toronto AF, Veasey LG, Stephenson RA. Math-ematical approach to medical diagnosis. JAMA. 1961;177:7581.

15. Gorry GA, Barnett GO: Experience with a model of sequen-tial diagnosis. Computers & Biomedical Research. 1968;1:490507.

16. Bleich HL. Computer evaluation of acid-base disorders. JClin Invest. 1969;48:168996.

17. Horrocks JC, McCann AP, Staniland JR, Leaper DJ, de Dom-bal FT: Computer-aided diagnosis: description of an adapt-able system, and operational experience with 2,034 cases.Br Med J. 1972;2:59.

18. Pauker SG, Gorry GA, Kassirer JP, Schwartz WB: Towardthe simulation of clinical cognition: taking a present illness

by computer. Am J Med. 1976;60:98196.19. Shortliffe EH. Computer-Based Medical Consultations: MY-

CIN Artificial Intelligence Series. New York: Elsevier Com-puter Science Library, 1976.

20. Yu VL, Fagan LM, Wraith SM, et al: Antimicrobial selectionby computer: a blinded evaluation by infectious disease ex-perts. JAMA. 1979;242:127982.

21. Blois MS. Judgement and computers. N Engl J Med. 1980;303:1927.

22. Miller RA, Pople HE Jr, Myers JD: Internist-1: an experi-mental computer-based diagnostic consultant for generalinternal medicine. N Engl J Med. 1982;307:46876.

23. Barnett GO, Cimino JJ, Hupp JA, Hoffer EP: DXplain: anevolving diagnostic decision-support system. JAMA. 1987;

258:6774.


13/16


24. Warner HR, Haug P, Bouhaddou O, et al. ILIAD as anexpert consultant to teach differential diagnosis. ProcTwelfth Ann Symp Comput Appl Med Care. New York:

IEEE Computer Society Press, 1987;3716.25. Bankowitz RA, McNeil MA, Challinor SM, Parker RC, Ka-

poor WN, Miller RA: A computer-assisted medical diag-nostic consultation service. Implementation and prospectiveevaluation of a prototype. Ann Intern Med. 1989;110:82432.

26. Miller RA. Medical diagnostic decision support systems past, present, and future. J Am Med Inform Assoc., 1994;1:827.

27. McDonald CJ. Protocol-based computer reminders, thequality of care and the non-perfectability of man. N Engl JMedicine. 1976;295:13515.

28. McDonald CJ, Wilson GA, McCabe GP Jr. Physician re-sponse to computer reminders. JAMA. 1980;244:157981.

29. Tierney WM, Hui SL, McDonald CJ. Delayed feedback ofphysician performance versus immediate reminders to per-form preventive care: effects on physician compliance. Med-ical Care. 1986;24:65966.

30. Tierney WM, McDonald CJ, Martin DK, Rogers MP. Com-puterized display of past test results: effect on outpatienttesting. Ann Intern Med. 1987;107:56974.

31. Tierney WM, McDonald CJ, Hui SL, Martin DK. Computerpredictions of abnormal test results: effects on outpatienttesting. JAMA. 1988;259:11948.

32. McDonald CJ. Computers. JAMA. 1989;261:2834 6.33. Tierney WM, Miller ME, McDonald CJ. The effect on test

ordering of informing physicians of the charges for outpa-tient diagnostic tests. N Engl J Med. 1990;322:1499504.

34. McDonald CJ, Overhage JM. Guidelines you can follow andcan trust: an ideal and an example. JAMA. 1994;271:8723.

35. Evans RS, Larsen RA, Burke JP, et al. Computer surveillance

of hospital-acquired infections and antibiotic use. JAMA.1986;256:100711.

36. Pestotnik SL, Evans RS, Burke JP, Gardner RM, Classen DC.Therapeutic antibiotic monitoring: surveillance using acomputerized expert system. Am J Med. 1990;88:438.

37. Gardner RM, Golubjatnikov OK, Laub RM, Jacobson JT,Evans RS. Computer-critiqued blood ordering using theHELP system. Computers & Biomedical Research. 1990;23:51428.

38. Classen DC, Evans RS, Pestotnik SL, Horn SD, Menlove RL,Burke JP. The timing of prophylactic administration of an-tibiotics and the risk of surgical-wound infection. N Engl JMed. 1992;326:2816.

39. Hickam DH, Shortliffe EH, Bischoff MB, Scott AC, JacobsCD. The treatment advice of a computer-based cancer che-motherapy protocol advisor. Ann Intern Med. 1985;103:92836.

40. Musen MA, Carlson RW, Fagan LM, Deresinski SC, Shor-tliffe EH. T-HELPER: automated support for community-

based clinical research. Proc Annu Symp Comput Appl MedCare. 1992;71923.

41. Institute of Medicine. Telemedicine: A Guide to AssessingTelecommunications in Health Care. Washington, DC. Na-tional Academy Press, 1996.

42. Willems JL, Abreu-Lima C, Arnaud P, et al. The diagnosticperformance of computer programs for the interpretation ofelectrocardiograms. N Engl J Med. 1991;325:176773.

43. Becker SH, Arenson RL. Costs and benefits of picture ar-chiving and communication systems. J Am Med Inform As-soc. 1994;1:36171.

44. McDonald CJ. The barriers to electronic medical record sys-

tems and how to overcome them. JAMIA. 1997;4:22232.

45. Slack W (ed). Medical hardware and software buyersguide. MD Comput. 1996;13:485576.

46. Friedman CP, Wyatt JC. Evaluation methods in medical in-

formatics. New York: Springer-Verlag, 1997.47. Stead WW, Haynes RB, Fuller SL, et al. Designing medical

informatics research and library-resource projects to in-crease what is learned. J Am Med Inform Assoc. 1994;1:2834.

48. Tierney WM, Overhage JM, McDonald CM. A plea for con-trolled trials in medical informatics. J Am Med Inform As-soc. 1994;1:353 4.

49. Schoolman HM. Obligations of the expert system builder:meeting the needs of the user. MD Comput. 1991;8:31621.

50. East TD, Morris AH, Wallace CJ, et al. A strategy for de-velopment of computerized critical care decision support

systems. Intl J Clin Monit Comput. 1992;8:2639.51. Miller RA. Why the standard view is standard: people, not

machines, understand patients problems. J Med Philos.1990;15:58191.

52. Miller RA, Schaffner KF, Meisel A. Ethical and legal issuesrelated to the use of computer programs in clinical medi-cine. Ann Intern Med. 1985;102:52936.

53. Public Law 94-295. Medical Device Amendments to theFederal Food, Drug, and Cosmetic Act (passed on May 28,1976).

54. Young FE. Validation of medical software: present policy ofthe Food and Drug Administration. Ann Intern Med. 1987;106:62829.

55. FDA Center for Biologics Evaluation and Research, Officeof Blood Research and Review, Office of Compliance, Officeof Regulatory Affairs. Draft Guideline for the Validation ofBlood Establishment of Computer Systems. Issued Septem-

ber 20, 1994. Available from FDA WWW Site: http://www.fda.gov

56. FDA Center for Biologics Evaluation and Research, Officeof Blood Research and Review. Draft Reviewer Guidancefor a Pre-Market Notification Submission for Blood Estab-lishment Computer Software. Issued April 12, 1996. Avail-able from FDA WWW Site: http://www.fda.gov

57. FDA Center for Devices and Radiological Health. Guidancefor the Content and Review of 510(K) Notifications for

Picture Archiving and Communications Systems (PACS)and Related Devices, Issued August 1, 1993. Availablefrom FDA WWW Site: http://www.fda.gov//cdrh/ode/odeot416.html

58. FDA Center for Devices and Radiological Health. Tele-medicine Related Activities. Issued July 11, 1996. Availablefrom FDA WWW Site: http://www.fda.gov//cdrh/tele-

med.html

59. National Commission for the Protection of Human Subjectsof Biomedical and Behavioral Research. Ethical principlesand guidelines for the protection of human subjects of re-search (The Belmont Report). Federal Register, Document79-12065, April 17, 1979.

60. Protection of Human Subjects. Code of Federal Regulations,

Title 45, Part 46, June 18, 1991.61. FDA. FDA information sheets for IRBs and clinical investi-

gators. Issued October 1, 1995. Available from FDA WWWSite: http://www.fda.gov

62. Organization for International Standards. ISO 9000 NewsService, 1996 97. Documentation available from: http://

www.iso.ch63. AMIA Board of Directors. Standards for medical identifiers,

codes, and messages needed to create an efficient computer-

stored record. J Am Med Inform Assoc. 1994;1:17.


14/16


APPENDIX

Appendix Flow diagram prepared by Center for Healthcare Information Management (CHIM) Suggesting Possible

Decision Algorithm for FDA to Use in its Regulation of Stand-alone Clinical Software Systems as Medical Devices.

Yes

Yes

No

No

Yes

No

No

Yes

Question 1. Is the "stand-alone" software a medical device?

1.1 Is the software sold(or otherwise commercially

provided) to clinicalpractitioners and/or

institutions?

1.2 Is the softwareembedded in themedical device?

1.3 Is the softwarean accessory to a

medical device?

1.4 Is the softwareused in the process

of delivering careto a patient?

Proceed toQuestion 2.

Used in financial management Used in administrative management Used in clinical practice management Used in library functions Used in professional education

Stop, not a medical device

Refer to FDA Guidance onaccessory software

Refer to FDA Guidance onembedded software

Stop, not a medical device

Appendix continued on next page.


15/16


No

Yes

Yes

Yes

Yes

No

No

No

No

Question 2. What are the risks or hazards associated with and the level of concern for the "stand-alonesoftware"?

2.2 Is the softwaredeveloped by practitionerfor noncommercial use

in own practice?

2.3 Is software's data usedin immediate decisions that

could cause death or seriousinjury without competent

user review?

May represent higher level ofconcernSee FDA guidance.

May represent higher level ofconcernSee FDA guidance.

Exempt from active regulation

Exempt from active regulation2.1 Is the

software a generalpurpose article?

2.4 Does failure of softwaredirectly caused death or

serious injury?

Are there factors that canbe used to reduce hazard?

Yes

Proceed to 2.5(next page)


16/16


Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

YesSoftware provides or replaces existing software applications with

well understood functions and no history of adverse events.

Re-evaluate.

2.5 Checklist to establishsuitability for exemptionfrom active regulation

Re-examine.

Yes

Software automates well-understood manual/paperbased processes.

Software simply manages data from clinicians.

Software allows intended/competent user intervention.

Software allows competent user organization to evaluate and test.

Software assures back-end integrity.

System failure does not cause software to add risk to patient.

Software automates difficult manual/paper based functions.

Software monitors events and alerts but takes no action.

2.6 Is softwarecharacterized by oneor more indicators?

2.8 Software is exemptfrom active regulation.

2.7 Will complete, accuratelabeling provide user withadequate information to

use safely?

No

Yes

Yes

Documents

Industry Proposal to Create Safe Oversight of Electronic Health Records from 1997