Embed Size (px)
Citation preview
Soft
ware
an
d S
yste
ms D
ivis
ion
IHE-PCD , HL7 HC Dev WG,ISO/IEEE 11073,
and NIST
Medical Device Communication andIHE-PCD Cycle 4 Test Strategy
IHE-PCD, HL7, ISO/IEEE Joint WG Meetings (@ Atlanta, GA)
23 September 2009
Soft
ware
an
d S
yste
ms D
ivis
ion
Medical Device Test EffortNIST Team Members
• John Garguilo ([email protected],
301-975-5248)• Sandra Martinez ([email protected],
301-975-3579)
• Maria Cherkaoui ([email protected]
Guest Researcher)• *Jing Gao ([email protected] Guest Researcher)
* Jing joined our project in May ’09
Project Web site: www.nist.gov/medicaldevices
Soft
ware
an
d S
yste
ms D
ivis
ion
NIST MDC Testing Topics
• IHE-PCD Test Strategy– NIST Framework for Testing Distributed Healthcare
Applications– IHE-PCD Testing Framework Instance– Cycle 4 Pre-connectathon (and virtual-
connectathon)– Test Plan– Support both Conformance & Interoperability
• RTM Terminology Management System Update
• NIST ISO/IEEE 11073 Tooling– ICSGenerator, ValidatePDU
• NIST ISO/IEEE 11073 Standards Development Work– MDC X73 XML Schema (X73-P10202)
Soft
ware
an
d S
yste
ms D
ivis
ion
Test Environments• Instance Testing
– Conformance (e.g., against HL7 2.x or CDA)• Implementation conforms to Spec. on which it is
based• IHE Model: ~Virtual and Pre-Connectathon
• Isolated System Testing– Includes Instance Testing Activities– Protocol Conformance – Functional Behavior Conformance
• Features and Operational behavior correspond to Specs.
• IHE Model: ~Virtual and Pre-Connectathon
• Peer-to-Peer System Testing– Includes Isolated System Testing Activities– Interoperability Testing
• Testing complete application environment • May include interacting w/ Database, using Network
Communications, or interacting w/ other hardware, apps, or systems if appropriate
• IHE Model: ~Connectathon
Soft
ware
an
d S
yste
ms D
ivis
ion
Test Environment Message ValidationNIST V2 Testing Tools: IHE-PCD
• Validation of IHE-PCD message(s) and corresponding HL7 Profile(s)
• Syntax and Semantic Content Validation– Against HL7 V2 message (e.g., PCD-01)
• Message structure (e.g., MSH,PID,PV1,OBR,NTE,{{OBX},OBX,OBX,OBX,…})
– Against HL7 profile• (Msg_type^Event_type^ e.g., ORU^R01^…)
– Against HL7 and/or user provided tables• Example of user provided table is RTM for Ref_IDs,
Units, etc.– Against ‘validation context’, including specific
values• Defined in XML (e.g., specific test case values)
Soft
ware
an
d S
yste
ms D
ivis
ion
• Validation against ‘failure types’: – VERSION*: The version in the message and in the profile should match. – MESSAGE_STRUCTURE_ID*: The message type (MSH.9 element) in the
profile and in the message should match. – MESSAGE_STRUCTURE: The message should have a valid message
structure (correct usage, correct cardinality, and correct element name). – USAGE: R elements should be present; X elements should not be
present in the message. – CARDINALITY: Elements should be present at least the minimum times
and at most the maximum times specified in the profile. It should also take into account the usage of the element (X element with a minimum of 4 should not be present in the message).
– LENGTH: The value of the element should have a length equal or less than the value specified in the profile.
– DATATYPE: For the datatype NM, DT, DTM, SI and TM, the value of the element should match the regular expression defined in the standard.
– DATA: The value of the element should match a constant specified in the profile, a value set specified in a table, a value or a regular expression specified in the message validation context.
– MESSAGE_VALIDATION_CONTEXT*: This is a user input error when the location specified in the message validation context can't be found in the message.
– TABLE_NOT_FOUND*: This is a user input when a table can't be found in the table files (TableProfileDocument).
– AMBIGUOUS_PROFILE*: The profile should not be ambiguous.
NIST V2 Testing Tools and ServicesTesting Validation Types
Soft
ware
an
d S
yste
ms D
ivis
ion
Test Artifacts• Conformance Profiles
• HL7 Tables• Validation Context Files• Generation Context Files
HL7 V2Message
Validation
Services Test Management
TestHarness(Java Code)
HL7 V2 MessageValidation Criteria
ResultsHL7 V2
MessageValidation
Report User
Instance Testing HL7 V2 Message Validation
Test Execution
Web Application
ClientReport
HL7 V2 Message
Conformance Testing: Showing that messages are incompliance w/ HL7 V2.x and constraints specified in IHE-PCD framework documents.
Soft
ware
an
d S
yste
ms D
ivis
ion
Test Artifacts• Conformance Profiles
• HL7 Tables• Validation Context Files• Generation Context Files
Services Test Management
TestHarness(Java Code)
HL7 V2 MessageRTM
Validation Criteria
ResultsRTM
MessageValidation
ReportUser
Instance System Testing IHE-PCD RTM Validation
Test Execution
Web Application
ClientReport
HL7 V2 Message
HL7 V2Message
Validation
RTMMessage
Validation
IHE PCDDOC
Test Agent
Conformance Testing: Showing that messages are incompliance w/ Harmonized-Rosetta terminology
SecurityTime
Soft
ware
an
d S
yste
ms D
ivis
ion
System Under Test
IHE-PCDDOC
System
Future: Isolated System Testing IHE-PCD Application Functional Behavior Testing
Test Artifacts• Conformance Profiles• HL7 Tables• Validation Context Files• Generation Context Files
IHE-PCDDOC
Test Agent
HL7 V2Message
Generation
IHE-PCDDOR
Test Agent
HL7 V2Message
Validation
Services Test Management
TestHarness(Java Code)
Router
IHE-PCD DECDOC
Test Scenario
ResultsMessageValidationReports
Vendor
Test Execution
Web Application
Client
ReportRTM
MessageValidation
Conformance Testing: May include HL7 message syntax validation, RTM, and/or application functional behavior testing
SecurityTime
Soft
ware
an
d S
yste
ms D
ivis
ion
Test Artifacts• Conformance Profiles
• HL7 Tables• Validation Context Files• Generation Context Files
Services Test Management
TestHarness(Java Code)
HL7 V2 MessageRTM
Validation Criteria
ResultsRTM
MessageValidation
Report
User
Future: Isolated System Testing IHE-PCD RTM Validation
Test Execution
Web Application
ClientReport
HL7 V2 Message
HL7 V2Message
Validation
RTMMessage
Validation
IHE PCDDOC
Test Agent
Conformance Testing: Showing that messages are incompliance w/ Harmonized-Rosetta terminology
SecurityTime
System Under Test
IHE-PCDDOR
System
HL7 V2 Message
Soft
ware
an
d S
yste
ms D
ivis
ionFuture: Peer-to-Peer System Testing
IHE-PCD Application Functional Behavior Testing
Test Artifacts• Conformance Profiles• HL7 Tables• Validation Context Files• Generation Context Files
IHE-PCDDOC
Test Agent
HL7 V2Message
Generation
IHE-PCDDOR
Test Agent
HL7 V2Message
Validation
Services Test Management
TestHarness(Java Code)
Router/Logger/Proxy
IHE-PCD DECDOC
Test Scenario
ResultsMessageValidationReports
Vendor
Test Execution
Web Application
Client
ReportRTM
MessageValidation
Conformance Testing: May include HL7 message syntax validation, RTM, and/or application functional behavior testing Interoperability Testing: system(s) (functional) behavior
SecurityTimeSystem Under Test
IHE-PCDDOC
System
System Under Test
IHE-PCDDOC
System
Soft
ware
an
d S
yste
ms D
ivis
ion
HL7 Message Validation StudyExample PIV (PCD-03)
Soft
ware
an
d S
yste
ms D
ivis
ion
PIV-PCD-03 Test Case 60101Validation using the IHE TF/Supplements
• MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001
• PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L• ORC|RE|12345|||||||||||||||||N60101• RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|
[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC
• RXR|IV||IVP• OBX|1||
69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun
Soft
ware
an
d S
yste
ms D
ivis
ion
PIV-PCD-03 Test Case 60101Validation Using HL7 Profile Tables
• MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001
• PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L• ORC|RE|12345|||||||||||||||||N60101• RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|
[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC
• RXR|IV||IVP• OBX|1||
69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun
Soft
ware
an
d S
yste
ms D
ivis
ion
PIV-PCD-03 Test Case 60101hRTM Validation
• MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001
• PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L• ORC|RE|12345|||||||||||||||||N60101• RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|
[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC
• RXR|IV||IVP• OBX|1||
69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun
Soft
ware
an
d S
yste
ms D
ivis
ion
PIV-PCD-03 Test Case 60101Use/Test Case Specific Validation
• MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001
• PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L• ORC|RE|12345|||||||||||||||||N60101• RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||1000|
[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC
• RXR|IV||IVP• OBX|1||
69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun
Soft
ware
an
d S
yste
ms D
ivis
ion
PIV-PCD-03 Test Case 60101Overall Validation
• MSH|^~\&|BBRAUN_TEST_IOP^0012210000000001^EUI-64|IOP|PAT_DEVICE_BBRAUN^0012210000000001^EUI-64|BBRAUN|20090112090938-0600||RGV^O15^RGV_O15|1|P|2.5||||||ASCII|EN^English^ISO659||IHE_PCD_PIV_001
• PID|||60101AB^^^AA1^PI||Bains^Allen^^^^^L• ORC|RE|12345|||||||||||||||||N60101• RXG|1|||1^Heparin|500||mL^mL^UCUM||||||||
1000|[iU]/h^[iU]/h^UCUM|25000|267616^MDC_DIM_X_INTL_UNIT^MDC|||||500|263762^MDC_DIM_MILLI_L^MDC
• RXR|IV||IVP• OBX|1||
69986^MDC_DEV_PUMP_INFUS_VMD^MDC|||||||||||||||^^A26672^BBraun
Soft
ware
an
d S
yste
ms D
ivis
ion
PIV-PCD-03 Test Case 60101Validation Coverage
• Coverage % = Number of Values in Message / Number of Values Checked
• IHE Supplement Validation 43% • Validation using Profile Tables 24% • Test Case Specific Validation 19%• Overall Validation 74%• Overall Validation without profile 62%
• Note: remaining 24% (from overall validation percentage) is validated at the message structure level via HL7; these include message segments that are not of primary interest to the IHE-PCD. If this level of validation is considered, then nearly 100% validation coverage can be expected.
Soft
ware
an
d S
yste
ms D
ivis
ion
Message Validation ContextTF/Supplement Validation
• DEC-PCD-01-ValidationContext.xml• SPD-PCD-02-ValidationContext.xml• PIV-PCD-03-ValidationContext.xml• ACM-PCD-04-ValidationContext.xml• IDCO-PCD-09-ValidationContext.xml
Soft
ware
an
d S
yste
ms D
ivis
ion
Message Validation ContexthRTM Validation
• DEC-PCD-01-ValidationContext.xml• PIV-PCD-03-ValidationContext.xml• ACM-PCD-04-ValidationContext.xml
• Needed: IDCO-PCD-09ValidationContext.xml
Soft
ware
an
d S
yste
ms D
ivis
ion
Message Validation ContextTest Case Specific Validation
• PIV(for PCD-03)– 60101-ValidationContext.xml– 60102-ValidationContext.xml– 60103-ValidationContext.xml– 60104-ValidationContext.xml– 60105-ValidationContext.xml– 60106-ValidationContext.xml– 60107-ValidationContext.xml– 60108-ValidationContext.xml– 60109-1-ValidationContext.xml– 60109-2-ValidationContext.xml– 60109-3-ValidationContext.xml– 60109-4-ValidationContext.xml– 60110-ValidationContext.xml
Soft
ware
an
d S
yste
ms D
ivis
ion
IHE-PCD ’09-’10 Pre-Connectathon
NIST Support (see recommendation document: "NIST Support for IHE-2009v1.docx")
• Primary Goal:Provide tools to test conformance to HL7 and IHE-PCD Integration Profiles such as DEC, DEC-SPD, ACM, PIV, IDCO, and RTM
• Instance Testing – System Under Test (SUT) Test Agent (TA)– HL7 syntax validation (basic validation)– IHE-PCD profile syntax– IHE-PCD profile semantic validation– RTM validation– Test case specific validation
• Develop Test Plan across IHE-PCD Integration Profiles– Test Scenarios– Interaction diagrams– Pre-conditions– Validation criteria – all test-able requirements (assertions)– Test cases– Validation context file(s)
Soft
ware
an
d S
yste
ms D
ivis
ion
IHE-PCD ’09-’10 Pre-ConnectathonContinued
Proposed conformance testing for ’09-’10 Pre-Connectathon
• All DOR messages (PCD-01) sent during DOR testing.• All IOP messages (PCD-03) sent during IOP testing.• All IDCO-DOR messages (PCD-09) sent during IDCO
testing.• All DOF messages (PCD-01) sent during DOF testing.• All DOC messages (PCD-02) sent during DOF testing.• All AR messages (PCD-04) sent during AR testing.
Soft
ware
an
d S
yste
ms D
ivis
ion
Show NIST V2 Tools (via Web)
Demonstration NIST HL7 Web• Upload HL7 IHE-PCD ORU^RO1 profile• Upload HL7 IHE-PCD context validation file (XML)• Upload HL7 IHE-PCD message (ER7 format)• Validate• Generate and review resulting ‘Report’
– Review errors
Soft
ware
an
d S
yste
ms D
ivis
ion
RTM Terminology Management System Update
Soft
ware
an
d S
yste
ms D
ivis
ion
RTM Mgmt System Changes(from last WG mtgs in May)
Features Completed• Enhanced lookup of REFIDs (description of
REFID’s) and provided an interface to propose new terms
• Implemented an interface for browsing terms in X73 Nomenclature
Soft
ware
an
d S
yste
ms D
ivis
ion
RTMMS In progress and next steps…
Features based on the Rosetta Supplement• Incorporate “Enumerations” management
capability(ies)• Add ranking capabilities of implementation
priority of valid terms in the Rosetta table• Implement Rosetta validation against hRTM • Add ability to edit hRTM table and save
changes• Implement hRTM version control and
management system
Soft
ware
an
d S
yste
ms D
ivis
ion
RTMMSFuture New Enhancements
Features based on the new/recent enhancements to HRTM
• Assign dimensions to the units• Implement Rosetta rules• Implement generation of hRTM based on Rosetta
rules• Generate comment table based on the
Recommended Rosetta Comment Submittal Format• Implement RCH table
User oriented features• Regular Expression based column filtering• Implement “change trailing” capabilities
– To identify occurred changes, time they were made, users who made them…
• Enhance registration process• Incorporate enhanced X73 Nomenclature database
Soft
ware
an
d S
yste
ms D
ivis
ion
NIST ISO/IEEE 11073 ToolingUpdate
Soft
ware
an
d S
yste
ms D
ivis
ion
X73 NIST Tooling…• ICSGenerator
– User Interface Work• Met w/ Todd Cooper, captured issues, made changes• Updated issues list – used as basis for task list
– Integrating database into ICSGenerator…• Lite-weight database identified• To include hRTM…
– Initial discussions• X73-compliant Device Containment • Generation of partial OBX segments
• On horizon: MDC Semantic Database• Terminology• Nomenclature• Medical Device Profiles and other Test Artifacts
Soft
ware
an
d S
yste
ms D
ivis
ion
ISO/IEEE 11073 Standards WorkUpdate
Soft
ware
an
d S
yste
ms D
ivis
ion
ISO/IEEE 11073 Standards Work
• MDC X73 XML Schema (X73-P10202)• Verification and validation of XML Schema
continues via tool usage• Support for Personal Health Device Working
Group– X73-20601– 104xx device specializations
• Continue to validate via tooling prior to ballot• Report errors for correction prior to balloting
Soft
ware
an
d S
yste
ms D
ivis
ion
Thank YOU!
• Discussion– Propose “ORU^R01^ORU^Rxx”
naming convention for IHE-PCD HL7 profiles
– xx denotes profile specific for Integration profile/device class (use of DEC/PCD-01)
• Questions?
Thanks for your attention
Please visit us at:http://www.nist.gov/medicaldevices
Soft
ware
an
d S
yste
ms D
ivis
ion
Extra Slides
Soft
ware
an
d S
yste
ms D
ivis
ion
NIST HIT Testing InfrastructureA Framework for Building Test
Systems
TestArtifacts
Time
AggregatedService
Test Agent
Generation
Security
EvaluationAgent
Test Data
Validation
Report
OtherServices
SpecializedServices
Services Test Management
TestHarness
Router/Logger/ProxySystem Under Test
TestDescription
Results
Facilitator
Monitor
User
Test Execution
System Under Test
IHE-PCDDOC
System
