-
Certification and Conformance Approaches for Canadas EHRElaine
Sawatsky, Senior Management Consultant, Privacy & Security,
Canada
-
Conformance: Canadas EHRInfoway Conformance Services in
Canada
Supporting the Implementation of pan-Canadian Standards
Provincial/Territorial work
On the ground experiences
-
Conformance: Canadas EHRSharing information requires a quality
assessment of products to ensure interoperability, to reduce risk
to patients, purchasers and the system. Standards support sharing
data and bringing it togetherImproves comparability and
harmonisation between products.Certification: a process of
validation of products in a given context against specific
validation criteria.Certification criteria are even regarding the
same functionality up to a certain degree similar but also
different between jurisdictions in:The way they are formulated
& validatedThe importance given to each of themCertification
should be unambiguous, fair and trustworthy
-
Conformance Services Vision
To support the successful implementation of pan-Canadian
Standards within projects through the provision of conformance
testing artefacts and tooling.To improve the consistent
implementation of pan-Canadian StandardsTo reduce the effort
required to implement pan-Canadian StandardsTo reduce the cost of
implementing pan-Canadian StandardsStrategic Intent of Conformance
Services:Ensure that there is a national service to support the
vendor conformance and certification of iEHR, public health and EMR
solutions.
-
SK EHR Blueprint and Concepts
Provincial Infostructure
Point-of-Service (POS) Infostructure
Registries Data & Services
Data Warehouse
EHR Data & Services
Ancillary Data & Services
Client Registry (SCI)
Provider Registry (PRS)
Location Registry
LRS
Outbreak Management
Shared Health Record
Drug InformationPIP
Diagnostic Reports & Imaging PACS
Lab RepositorySLRR
PHS Reporting
Health Information
Surgical Care Registry
POS Infostructure will continue to evolve as future needs are
identified
BusinessRules
MessageStructures
EHRIndex
CANADA
Modified: October 2, 2006
HIAL
User Registry
SecurityManagement Data
PrivacyData
Consent
Integration
Security
Organizational Infrastructure
Access Systems
Long Term Care (MDS)
SK Cancer Agency
Clinical Management Service (Varian)
Prov Lab
Lab (LIMS)
Registry(Varian)
Regional Systems
Drug (WinPharm, RxTFC, PCSI)
Lab (TriWin, SCC)
Clinical Information System (SCM)
Regional ADT(WinCIS, Enovation)
Health Records (WinRecs, 3M)
Home Care(Procura)
Surgical (SIS)
Transcription (WinCIS, HealthSuite,...)
TerminologyServices
Index
Normalization
Orchestration
Radiology (RIS)
Public HealthImmunizationPublic Health Surveillance
RHA Integration Hub (Cloverleaf)
Primary Health CareChronic Disease Mgmt. (CDM)
Clinical Viewer (SCM)Provincial Medication ProfileProvincial
PACSProvincial Lab Results
Vendor PoS
Community PharmaciesMedication Profile
Clinical EMRsPrimary Health Care (PHC)LABMedication
ProfileShared Health Record (SHR)
eHealth Portal
EHR DesktopLab Results (SLRR)Medication Profile (PIP)DI Image
& ReportsShared Health Record (SHR)Public Health Surveillance
(PHS)Client Id. (SCI)Health Insurance (PHRS)Provider Id. (PRS)Wait
List (Surgical Care)
Individual Viewers
Client ID Viewer (SCI)
Health Insurance Viewer (PHRS)
Medication Profile (PIP)
Diagnostic Imaging Viewer (PACS)
...
V3.1
-
The need: Bridging The GapThe challenge is to bridge the gap
that exists between the health informatician (modeler) and the
clinical integration specialist (solution developer). EHR in
OperationHealth InformaticsModelerIntegration SpecialistModeling
CultureDelivering SpecificationsProject and Product CultureDelivers
Production SolutionsStandards DevelopmentStandards Implementation,
Conformance & MaintenanceTooling*
-
Conformance: Canadas EHR
-
Conformance: Canadas EHRApproachThere is recognition that there
is a need for conformance testing to ensure alignmentThe approach
is founded on international standards
Prototype & Learn conformance testing is not well understood
yetBuild Methods & Processes with stakeholder input Build
tooling & leverage : Standards HL7 v3, LOINC, SNOMED,
XDSIToolingComponent model integration is key!Sharing of testing
artefacts huge value to stakeholders
-
Conformance: Canadas EHR
Key Messages
There is recognition that we all need to do end to end and
integrated testingWe are committed to leveraging work of others We
are making good progress in developing processes and tooling to
assist our jurisdictions in implementing Conformant solutions
-
Conformance ContinuumConformance includes:Message content and
structureVocabulary and terminologyInteroperability profiles and
message sequencingSending and receiving applications and
intermediaries
Conformance Testing includes:Functionality of applicationBasic
privacy and security services related to the functionalityWorkflow
(to an optimal level determined by conformance continuum)*
-
Our ApproachFirst, develop Profiles for all pan-Canadian
standardsDevelop a process & mechanisms linking the
pan-Canadian Profiles through to conformance test Develop, deploy,
continuously improve quick-hit toolsProvide business and technical
support for Investment Projects to set up and execute conformance
testingDocument & refine requirements for long-term
solutionHarvest artefacts for sharing with other
jurisdictionsRationalize jurisdiction artefacts to create
pan-Canadian versionsEstablish a pan-Canadian Working Group of
early participant jurisdictions and their vendor partners to
provide guidance and assistance in the development of Conformance
Testing artefacts
-
EvaluationAdoptionStrategiesTools
andTemplatesStandardsKnowledgeManagementLiaisonPromotion &
CommunicationLicensesChange ManagementStandards Selection
StrategyStable for UseFinal ApprovalOur Approach
-
Conformance: Canadas EHRBenefits:Financial savings through
reduced duplication of efforts in test authoringProvinces leverage
other provinces Business RulesFinancial savings though tools based
test message developmentProvinces use Infoway tools and other
provinces messagesBreadth of testing improved through increased
quantity and quality of test casesProvinces are committed to
sharing test artefacts.
-
Conformance: Canadas EHRBenefits:Depth of testing improved
through consistent test rules and toolingProvinces use of tooling
has improved UAT testing Too early to accrue benefitsQuality and
consistency of conformance testing improved between Investment
Projects Too early to accrue benefits
Improved consistency in Investment Project implementations of
pan-Canadian standards Too early to accrue benefits
-
BenefitsProviding Implementation support for the development of
Test Authoring and Test Execution will enable jurisdictions to
leverage and share Conformance Profiles and testing artefacts,
leading to:Improved quality of testingFinancial savings through
reduced duplication of effortsBreadth of testing improved through
increased quantity and quality of test cases Improved quality and
consistency of conformance testing Financial savings though tools
based test message developmentDepth of testing improved through
consistent test rules and toolingImproved consistency in
implementations of pan-Canadian standards
-
Conformance: Canadas EHRConformance Services OverviewMain
elements influencing content of test criteria:Legal, national,
cultural (business and social)Sophistication of the industry
areaEnvironmentCertification is the processConformance is the
resultMust be good enough, not best or everyone will failArea
subject matter experts what will the medical professional need?
-
Stakeholder EngagementGovernanceCollaboration on Test Authoring
componentsCollaboration on supporting tools development and
implementationRegistry of vendor systems & certification status
Registry of implementation projectsOngoing liaison/communication
with jurisdictions and vendorsPromotion and Education around
Conformance ServicesOverview of Conformance Services *
-
Strategic Direction Analysis ProjectProject Objective:Develop a
preferred option for the future delivery of Conformance Services.
Define and identify existing conformance services models;Develop a
shared vision for the delivery of Conformance Services for the
Canadian healthcare sectorDevelop conformance services delivery
options for the Canadian healthcare sector; and detail the
preferred option. Defined Strategic Intent of Conformance
Services:Ensure that there is a national service to support the
vendor conformance and certification of iEHR, public health and EMR
solutions. *
RFP ReleaseFebruary 15 08Project StartMarch 15 08Draft
recommendationsJune 30 08Final Recommendations AvailableAugust 30
08
-
Investment Project Support Overview Project Objective:Provide
immediate Standards Implementation Support for Test Authoring &
Test Execution to key Infoway Investment projectsProject Approach:
Develop a process & mechanisms linking the pan-Canadian
Profiles through to conformance testDevelop, deploy, continuously
improve quick-hit toolsRollout to qualified projects (if proof of
concept successful)Harvest artefacts for sharing with other
jurisdictionsProvide lessons learned and input into CS Strategic
Direction ProjectProvide transition and support to recommended
Strategic Direction*
Project Kick-offNovember 1 07Proof of concept (2-3
projects)March 31 08Proposed Project Extension:Limited rollout (4-6
projects)September 08
-
Investment Project Support ActivitiesThe specific services that
Infoway is offering to assist projectsAssistance in developing test
strategiesSort out overlap between conformance testing vs. UAT
Education on development of business rules & test
casesTechnical education on development of message instances and
test dataSharing of testing artefactsQuality review of artefacts
developed by investment projectsFull set of tools to assist getting
to Conformance Testing Assistance to jurisdiction in rationalizing
and integrating their testing tools with Infoway conformance
toolsHarvesting of testing artefacts for sharing with
jurisdictionsSharing of Lessons LearnedFacilitating
inter-jurisdiction collaboration on conformance testing *
-
*Engagement Process to ConformanceTest case Examples
HarvestedartefactsEnd to EndTest AuthoringExamplesJurisdiction
Test ArtefactsToolsEnd to EndTest ExecutionExamplesJurisdiction
Test Authoring ArtefactsJurisdiction still needto complete, but
ready to perform ConformanceTestingOngoing Harvesting
-
*Conformance Constructs End-to-End
ProcessJurisdictionInputBestPractice
-
Conformance: Canadas EHRMajor Tooling ComponentsBusiness Rules
Manager- Maintains Business RulesConformance Profile Manager -
browse pan-Canadian Conformance ProfilesImplementation Conformance
Profile Manager - browse profiles and create/maintain ICPsStatic
Model Constraint Tool - captures changes to models, and
traceability informationInstance Editor - creates Conformance Test
Scenario Files, Message Fragments and Parameter Files as XML
output. Test Case Author - Maintains Test CasesTest Authoring
Manager - The main user interfaceTest Execution Manager - links the
Test Authoring Manager to control the test executionTest Authoring
Reports Manager - run reports
-
Conformance: Canadas EHRConformance Test Execution
ScenariosThere are different ways to perform Conformance Test
Execution. This process uses the pCCT Test Execution manager and /
or the HL7 Test Harness to allow the user to test both POS and iEHR
systems as real or simulated applications under test.
Scenario 001 - Real POS with Simulated iEHR Application Scenario
002 - Simulated POS with Simulated iEHR Application Scenario 003 -
Simulated POS with Real iEHR Application Scenario 004 - Real POS
with Real iEHR Application
-
Lessons Learned Considerations for AB
Testing artefact reuse is seen to be a big winOne province has
100 business rules & 300 test cases target is 500-700 test
casesAnother target will be significantly higher (~2,500 test
cases)Others will leverage oand likely add moreToolset are not all
or nothingOne province using blend of Infoway tools and home
grownUses Infoway tools and processes first to perform testing then
conformance testing of PoSOne province using Infoway tools for go
forward & PoS testingSpecific initiatives will likely use
blend
Infoway is investing in tools so jurisdictions dont have to!
-
Conformance: Canadas EHRTooling VisionCreate a suite of
integrated Conformance Test tools based on Open Source principles
and accepted pan-Canadian standards that allows jurisdictions, and
vendors to perform conformance testingIncludes:Well documented
processes and proceduresA repository of structured conformance
profiles, test cases, test data and business rules that can be
reusedA suite of re-usable conformance testing solution components
able to easily exchange structured data between each component and
projectsAn Enhanced Web Based Presence to allow remote
administration, self registration, shared resources management,
self testing / monitoring and publication of results
-
Lessons Learned Considerations for ABMany jurisdiction projects
have left UAT & Conformance Testing until very late in their
project lifecycleThere is not a clear understanding of the
continuum of conformance testing Profile Business rules Test cases
Messaging Conformance testTraceability is key to conformance
testProfile Business rules Test cases Messaging Conformance
testUsing Profiles to help them define user rolesWhat does a
physician do vs. project focusTools can and are providing immediate
benefit to jurisdiction projects for publishingConsolidated view of
all business rulesPublication of specification for vendor*
-
Conformance ProfilesPan-CanadianConformance Profile
(pCCP)ImplementationConformance Profile
(ICP)pCCPExtensionConstraintExample:BC ICP #84#84BC
ExtensionsBCConstraintLikelyInitial StateDesiredState
-
Conformance: Canadas EHRTooling for Test Authoring Components
(TAC)Use prototyping methodology to conduct staged development Test
with jurisdictional review and input Document processes,
requirements and changesGarner feedback and experience to inform
the refinement of detailed specifications for long-term open source
tooling development Address immediate jurisdictional needs &
provide tangible benefitsAll test artefacts exportableWork with
jurisdictions to rationalize and integrate tooling Align test
cases, business rules across jurisdictionsPromote early sharing and
leveraging of work by jurisdictions
-
Conformance: Canadas EHRTooling for Test ExecutionLeverage an
existing open source test harness (previous Infoway investment),
adding ability for jurisdictions to easily configure to reflect
local technology solutions such as transport and securityLeverage
Eclipse Instance Editor for message and vocabulary creation Use
data exported from test authoring toolsKey is to ensure tools
export into a format that can be consumed by HL7 Test HarnessInvest
in documenting processes, requirements, integrationRationalize and
integrate toolsDevelop guidelines for local Test Execution
prototyping environments including a reference
implementationDevelop processes to ensure data integrity between
different groups of users during prototyping phase
-
Conformance: Canadas EHRIts not easy:In order to get to a
conformance test, you need to start with the Profile, know what the
business rules are and create test data and message instances that
will then test a scenario
Once you have done this for every business and technical
scenario, only then can you think about performing a conformance
test
By our simple calculations, this will be thousands and thousands
of test cases!
-
0 Step 0 Engagement1 Step 1 Workshop2 Step 2 Activities3 Step 3
Workshop4 Step 4 Activities5 Step 5
WorkshopPlannedScheduled/UnderwayCompleted
0,1004,50,1,2,3Panorama00,100,10*13-Mar-08Conformance Services
Activity in Canada
-
Conformance: Canadas EHRProvinces have done functional
Certification for EMRs specifically. Other certification means
defining differences in function, scope, perspective (e.g. Social
interaction between users, staff and patients), Legal requirements
are different between provinces, policies are different: Infoway
cannot set policy but provinces must Provinces must work in
collaboration with Health Authorities and professional bodies.
Conformance criteria will be different from one business
environment to another.
-
Conformance: Canadas EHRWork is based on existing provincial
modelsEstablish business rules to support policy set by Ministries
of Health, College of Physicians and Surgeons and College of
PharmacistsPublish compliance specifications that include all the
transactions needed at the point-of-service (e.g. pharmacy claims,
pharmacy prescriptions, Client Registry, Provider Registry, Common,
Lab, Immunization, etc.)Provide support to vendors during their
developmentConduct compliance testing of vendor software once they
pass a pre-testSupport change agents (chain stores, vendors, etc.)
during their training and deployment activities
-
Conformance: Canadas EHRPOS Software Compliance PrinciplesPOS
systemsMust enhance patient safety & Supportclinician
workflowMust be secure and ensure privacy of patient medical
recordsMust support interoperability standardsAre integral in
creating and maintaining quality data for use throughout the health
care systemPOS vendors must be able to differentiate their
productsCompliance documentsAre referred to in some provinces
legislationCommunicate what rules need to be followed, not how the
rules are implemented in vendor softwareProvide one place to house
Ministry policy, College rules and application capabilitiesProvide
a level playing field for all vendors regardless of size
-
Conformance: Canadas EHRPOS Conformance SpecificationsDefine
Business rulesBased on existing business rules from compliance
documents and input from Clinical Working GroupIdentify items that
need further clarificationDevelop Compliance specificationsBased on
existing compliance documents and published message specifications
from InfowayBuilds on Infoway Conformance Profile
definitionsLeverage, where possible, work from other
jurisdictionsMaterials include:POS Software Vendor Compliance
specifications;Vendor registration and development support
processes;Compliance testing process;Process for approving vendors
with compliant softwareStakeholders included in joint review of
material prior to publishing
-
Conformance: Canadas EHRCompliance Lessons LearnedIt takes time
for the vendors to develop code to the level that warrants
compliance testingVendors may present software that was not stable
enough to warrant compliance testingVendors can require multiple
attempts at compliance testingThorough compliance testing takes
time, especially on the very first test of new software or a major
release changeGood rules are needed on what constitutes an initial
test and a follow-up test for each version of the vendor
softwareGood rules are needed on when penalty and charges are
applied to the test
-
Conformance: Canadas EHRSample tests were provided for
vendorsFocus needed to ensure that the vendors could prove they ran
the tests and the tests worked before scheduling a compliance
testingNeed mechanisms to ensure compliance team provided
consistent responses to all vendors throughout the compliance
support processGood rules are needed around mandatory training that
a vendor must provide e.g. training plans, training outcomes,
etc.Good test facilities are needed for vendors and for compliance
testingMultiple training environments are needed for each
vendorGood tools are needed for the compliance testers that assist
them in developing the test cases and support them during the
testing
-
Conformance: Canadas EHRClinical guidance was critical during
vendor development and compliance testing to clarify options and
workflowRequires ongoing clinical involvementSpecifications,
clinical guidance and support needs to include detailsCompliance
team requires good communication and negotiation skills and need to
be able to stand their ground under pressureCompliance
documentation needs to be targeted to specific audiences and also
clearly identify elements common across audiencesA tool that
enables easy linking and/or packaging of modules is highly
desirable. Documents that are difficult to maintain dont get
maintainedBusiness context for the compliance material is just as
important as the rules themselves
-
Conformance: Canadas EHRConformance includes:Message content and
structureVocabulary and terminologyInteroperability profiles and
message sequencingSending and receiving applications and
intermediaries
Conformance Testing includes:Functionality of applicationBasic
privacy and security services related to the functionalityWorkflow
(to an optimal level determined by conformance continuum)
-
Contact
Standards Collaborative Information Desk Bureau d'information de
l'Unit collaborative de normalisation dInforoute 1-877-595-3417 or
416-595-3417 [email protected] or
[email protected]
[email protected] and Security, SME
-
Holding slide
*I will not spend a lot of time on this, just to point out that
within each of these boxes, tubes and lines there is a need for
decision points that have rules and consequences. When integrating
information from multiple sources it will be necessary to normalize
the data in such a way as not to change the meaning but to reflect
it in standardized format. These require Business/clinical input
and agreement. Those who create the data and those who are
consumers of the data will need to agree on this common platform
for presentation of the data.Within the context of this slide, our
project is focused on trying to bridge the gap!Our project is
focused on working with jurisdictions to understand how to bridge
the gap********Introduction Workshop on project, approach and tools
why we always do things the way we do things (Done for SK, NL Dec
12th)Two day Workshop Test Authoring Component (TAC) FocusedHow to
create a test case end to end Business process focusedHow to use
Quick-hit tools to create testing artefacts note: you dont get the
tools unless there is commitment on both sides to
success!Jurisdiction Test Authoring Development (time boxed to
15-20 days develop for breadth and depth, not 100%)Jurisdiction
development of test casesInfoway business and technical
supportRegular bi-weekly checkpoint meetings/progress
reviewScheduling of Test Execution Component (TEC) WorkshopTwo day
Workshop TEC FocusedHow to import TAC artefacts into PEI Test
HarnessHow to use PEI Test Harness to execute a conformance
testJurisdiction TEC set up & testing (time boxed to 15-20 days
develop for breadth and depth, not 100%)Jurisdiction set up; import
of test cases into TECJurisdiction set up of vendor test
environment & testingInfoway business and technical
supportRegular bi-weekly checkpoint meetings/progress
reviewJurisdiction Pilot testing of vendorJurisdiction execution of
pilot of conformance testing with vendorInfoway business and
technical support (3-5 days)
Rationale for tooling approachTools to demonstrate and refine
requirements quicklyLeverage existing tools (PEI HL7 Test Harness,
Eclipse Instance Editor)Meets immediate needs of implementations
Content easily imported, exported & migrated Allows integration
with jurisdiction UAT, or other automated testing toolsInform
long-term tools development approach & requirements
*Conformance ProfilesThe initial set of pCCPs have been
developed by Infoways Subject Matter Experts (SMEs) based on the
Conformance Framework Principles as well as knowledge of existing
jurisdictional development projects pCCPs are defined at the level
of building blocks to facilitate the bundling or grouping of one or
more Conformance Profiles in an application pCCPs to be reviewed
with jurisdictions and through SCWGs over the next weeks/month
*
