1 EHR-Q TN Final Review Brussels, March 30 th 2012

1

EHR-QTN

Final Review

Brussels, March 30th 2012

2Brussels, March 30, 2012

Introduction of consortium and experts

3

Agenda

The partners

EHR-QTN: why was it needed?

Overview of the Network Activities

Investments done during the lifetime of the project

Validation of the EuroRec Statements

EHR Market Overview: main conclusions

Roadmap towards a sustainable pan-European certification of EHR systems

Recommendations from the Roadmap

Quality Labelling and Certification Procedures: the quality of the process

Conclusions and what brings the future?

Questions

Brussels, April 8, 2011


5

EHR-QTN : Why certification? Why the project?

Brussels, March 30, 2012

6

Reasons for “Certification”

eHealth and more specifically Electronic Health Record systems have an enormous potential to improve quality, accessibility and efficiency of care, provided they are:

reliable, trustworthy and of sufficient quality;sharable and interoperable;used appropriately.

Quality labelling and certification through professional third party assessment offers best chances for a comparable and reliable quality documentation of those systems. Brussels, March 30, 2012

7

Some EHR quality issues

1. Patients are too important to just suppose that EHR systems are trustworthy.

2. Patient data should not be locked into one system or application.

3. Patients essential data should be made available anywhere anytime to health professionals authorised to access them.

4. Patient has the right to request confidentiality of some data to be handled while taking full responsibility for that option.

5. Patients’ data accesses should be audit-trailed.


The quality challenge

Myers et al*. show that adverse events are mainly resulting from:

• missing or incorrect data;• data displayed for the wrong patient;• chaos during system downtime;• system unavailable for use.

Examples of reported incidents in healthcare where a medical information system was the cause or a significant factor: http://iig.umit.at/efmi/badinformatics.htm

*Myers DB, Jones SL, Sittig DF, Review of Reported Clinical Information System Adverse Events in US Food and Drug Administration Databases, Applied Clinical Informatics 2011; 2: 63–74.

http://iig.umit.at/efmi/badinformatics.htm

9

Why was such project needed?

Too many authorities not aware of the quality needs and related problems.Even more authorities not effectively involved in quality labelling and certification.Lack of awareness about what exists.Existing initiatives not comparable with each other.Cross-border quality assessment almost unexisting.


10

Overview of the Network Activities


Dissemination & AwarenessWorkshopsAnnual EuroRec ConferencesProject DataConsortium MeetingsDeliverables

11

Workshops

In total 76 workshopsAt least one per countryAgenda of the workshops:

Year 1: Communications from the Commission, Seal Level 1, RepositoryYear 2: Validation of the EuroRec StatementsYear 3: Validation of procedures & roadmap for the countries with most chance of making progress


12

Overview Workshops

• Listed in the three Annual Reports and the Final Report

• Reported in deliverables D1.4, D2.2, D3.2, D4-I, D4-II, D6.1a, D6.1b and D6.4


Q14/09

Q27/09

Q310/9

Q41/10

Q54/10

Q67/10

Q710/10

Q81/11

Q94/11

Q107/11

Q1110/11

Q121/12

Tot.

National Workshops Planned - 6 6 7 7 7 6 7 7 6 7 6 72

National Workshops Reported / Announced

4 4 13 4 2 8 10 4 2 1 24 76

13

Workshops: questions and suggestions

Resulted in valuable comments and questions regarding:

Quality labelling and certification as such: need, quality & professionalism of the process;The content of quality assessment;The role of the (national) health authorities;The use and appropriateness of the EuroRec Descriptive Statements;The importance of functional testing.


14

A Norwegian statement…

A recent Norwegian statement is an important one and based on a large experience in certifying “messages” (certifying all kind of standards based data exchange).

The Norwegian Ministry of Health and Care Services stated that “EHR Quality will be difficult to reach unless certification of the EHR systems is made mandatory”.


15

EuroRec Conferences


EHR-QTN EuroRec (Annual) Conferences

1EuroRec Conference 2009, Sarajevo, Bosnia & Herzegovina

August 30-31, 2009

2EuroRec Session at WoHIT 2010, Barcelona, Spain

March 17, 2010

3 EuroRec Conference 2010, Tallinn, Estonia June 18, 2010

4EuroRec Session at WoHIT 2011, Budapest, Hungary

May, 12, 2011

5Regional Conference on EHR Systems Quality Labelling and Certification, Belgrade, Serbia

November 21-22, 2011

16

Consortium Meetings


EHR-QTN Consortium Meetings

1 Prague, Czech Republic, Kick-Off February 17-18, 2009

2 Sarajevo, Bosnia-Herzegovina August 31, 2009

3 Dublin, Ireland November 17-18, 2009

4 Tallinn, Estonia June 17, 2010

5 Sofia, Bulgaria December 9-10, 2010

6 Lisbon, Portugal June 17-18, 2011

7 Nicosia, Cyprus December 2, 2011

17

Deliverables


TABLE 1. DELIVERABLESDel. no.

Deliverable name WP. Lead participant Nature (i) Dissemination level (ii)

Due delivery date

Deli-vered Actual / Forecast delivery date Comments

D1.1 Kick-Off Meeting 1 EuroRec E C 2 Yes 05.11.2009D1.2 Documentation for the workshops 1 EuroRec R C 2 Yes 05.11.2009

D1.3 EHR-QTN website 1 EuroRec R P 2 Yes 05.11.2009

http://ehrqtn.eurorec.org/http://www.eurorec.org

D1.4Report on Workshops 1 - 6 1 EuroRec E/R C 6 Yes 05.11.2009 Workshops 1 to 3

reported

D2.1Documentation for the second set of National Workshops

2 EuroRec R C 9 Yes 18.02.2010

D2.2Report on National Workshops 7 - 19

7 to 19

2 EuroRec E C 12 Yes 26.02.2010 Workshops 4 to 21 reported

D2.3Report on the Annual EuroRec Conference 2 EuroRec E C 12 Yes 15.02.2010

D2.4 First Annual Project Report 2 EuroRec R P 12 Yes 03.03.2010

D3.1

EHR Market Overview – Suppliers 3 EuroRec R P 15 Yes 30.12.2010

D3.1EHR Market Overview - Stakeholders 3 EuroRec R P 15 Yes 01.12.2010

D3.2Report on the National Workshops 3 EuroRec E C 18 Yes 04.11.2010 Workshops 22 to 27

reported

D4.1Report on the National Workshops (2 parts) 4 EuroRec E C 24 Yes 07.03.2011

09.03.2011

Workshops 28 to 44 reported

D4.2Report on the Second Annual EuroRec Conf. 4 EuroRec E C 21 Yes 16.03.2011

D4.3 Second Annual Project Report 4 EuroRec R P 24 Yes 16.03.2011

D4.4Validation of EuroRec Statements and EuroRec Use Tools

4 EuroRec R C 22 Yes 15.03.2011

D4.5Documentation for the third set of National Workshops

4 EuroRec R C 22 Yes 24.03.2011

D5.1 Report on the National Workshops 5 EuroRec E C 30 No Merged with Deliverable D6.1

D5.2Roadmap towards Sustainable Pan-European Certification of EHR systems

5 EuroRec R P 30 Yes 11.11.2011

D6.1Report on the National Workshops (2 parts) 6 EuroRec E C 36 Yes 08.02.2012 Workshops 45/76

reported

D6.2

Report on the validation of the quality labelling and certification procedures and scenarios

6 EuroRec R P 36 Yes 14.02.2012

D6.3Translated EuroRec Statements Report (2 parts)

6 EuroRec R C 33 Yes 05.03.2012

D6.4Report on the Third Annual EuroRec Conference

6 EuroRec E C 36 Yes 30.12.2011

D6.5 Third Annual Report 6 EuroRec R P 36 Yes 05.03.2012D6.6 Final report 6 EuroRec R P 36 Yes 10.03.2012

18

Investments during the lifetime of the project


19

Investments

Improving functionality of the Repository.Definition of EuroRec Seal Level 1 and 2.Translations into 21 languages.Validation workshops.Setting-up effective quality labelling.


20

Functional Tools Investments

Extensionof the repository: secondary use of EHR data and lab reporting related issues.Adding content-related functionality:

definition sectioncomments and interpretationnational variants

Adding maintenance functions to manage e.g. modifications in a multi-lingual environment


21

EuroRec SealsEuroRec Seal Level 2 defined during the lifetime of the projectBasic sets of quality criteria

Addressing reliability, trustworthiness, authentication, access management and basic functionalityComparable across borders and domains

Example of a “market driven” approachStandardisation of the procedure (see further)Products out of 7 different countries certified


22

Industrial testimonies

“EuroRec Seal greatly helped to improve SW products.”

“Developers got clear guidelines about key features that are often neglected by end users.”

“Customers got additional assurance of software quality.”

“The Seal offers increased odds at foreign markets.”

“The EuroRec approach is very useful for new software (modules), new application design… giving ‘new ideas’.”


23

Translations: 12.379 in total


24

Validation of the EuroRec Repository and of the Descriptive Statements


25

Validation of the statements

Validation doneWhen translating the original statementsDuring the workshopsBy the “clients”, software suppliers

Validation of the following aspects:FormulationContentTechnical correctnessImportance (for application quality)Feasibility

Reported in deliverable D4.4


26

Setting-up effective quality labelling


27

Effective Quality Labelling

Documentation free available.Seal request forms on the web.Procedure validated against the applicable standards.Partners involved (not only ProRec centres).


28

EHR Market Overview : main conclusions



30

EHR Market Overview - Summary

24 National market Overviews using a common templateTwo Deliverables

Del. D3.1 Part I: Suppliers and Supplier Organisations: 1.005 supplier / product IDDel. D3.2 Part II: Authorities and important stakeholders: 663 addresses

Detailed presentation: previous review


31

EHR Market: some considerationsVery fragmented as expected

May endanger quality of applications, though never proven.Not the privilege of the suppliers: also large number of “important stakeholders”.

There is no one single nor homogeneous provision of healthcare in Europe, neither within one country

Each profession needs a “different” application.Using the same application in several countries does not work.

There is some market “concentration”Concentration of ownershipNo concentration of applications, even when the same name is used in different countriesBrussels, March 30, 2012

32

One of the project conclusions

The only approach that may work seems to be to increase harmonisation within diversity, offering more and more “similar” (not identical), functions based on the same basic functional and quality specifications.




Roadmap towards a sustainable pan-European certification of

EHR systems

35

Deliv. D5.2 :Table of Content

1. Potential of the EHR: Why? Possible benefits? Barriers to adoption? Requirements?

2. Importance of Quality Labelling & Certification

3. Stakeholders and their role

4. State of Practice in Europe

5. Prerequisites for (sustainable) Quality Labelling & Certification

6. Main strategies regarding quality labelling and certification

7. Roadmap for certification at National Level

8. Cross-border Certification Roadmap

9. Cost of Certification & Business Definition

10. Risks and Remedies

11. EHR-QTN Recommendations


36

Introductory statement

“Realising the potential health added and economic value linked to using an EHR system is not obvious. It requires huge investments and a professional development environment. It requires a permanent focus on “quality” at functional quality level and thus also regarding “interoperability”.Quality should be documented in an objective, comparable and trustworthy way.”


37

Verification versus Validation

Verification = technical correctness of the software application or component of an application. Verification attempts to answer the question “is the software built right (rightly)?” => medical device directive ?Validation = compliance of the application to the consumer’s / user’s functional expectations: is the application offering what it is expected to do? Validation attempts to answer the question “is the right software built?” => procurement and functional validation !


38

Five areas for quality labelling and certification

Data exchange facilities (incl. IOP)Functional (incl. some aspects of IOP)Administrative and billing facilitiesUse related measurements and validationSoftware development quality (out of scope, not specific for EHR systems)

=> Different expertise , different organisationsDublin, November 17-18

39

Scope of EHR QL & Certification

Different expertise => Different organisations.Our focus for Deliverable D6.2 is on

Functional testing, including some aspects of interoperabilityData exchange (message production and integration)

We will address how to “cooperate” later on.

Dublin, November 17-18

40

The use of EHR systems

Consortium listed the top 5 good reasons to generalise the use of EHR systems.A small literature survey proofs quantifiable profits.There are nevertheless still barriers to EHR adoption:

by Healthcare Professionalsby IT providersrelated to political and organisational factors


41

Not all EHR systems are good enoughSelecting the most appropriate application from the correct vendor is a real challenge => importance of assessing the systems’ quality.Comprehensive and correct use is another important factor =>

Importance of training the usersImportance of assessing the usersMotivation for incentives for the users.


42

Impact of certificationConsortium listed the top 5 good reasons to adopt country wide EHR certification:• Assure compliance to national rules and

standards.• Increase quality of the products through

coherent and pre-tested functionality.• Leverage exchange of health (care)

related data and interoperability of systems.

• Improve patient safety in care.• Have a reliable data source for

secondary use.Brussels, March 30, 2012

43

Prerequisite

“ If quality labelling and certification of EHR systems is to become generalised, then it needs endorsement at the highest competent levels e.g. by the EU Commission, the responsible Member States Ministries, the Healthcare Providers Organisations and the specialised industry.”


44

Stakeholders & Functional Diagram


Certification body

CAB

InitiatorIAF

Product

Supplier Authority

Accreditation body

Certificate

Label

ISO/IEC 17020 =>

<= ISO/IEC 17011

ISO/IEC 17025 =>

45

Kind of Quality Assessments

“Authority driven” versus “Market Driven”

Independent organisation / Industrial organisationPublic initiative / Supplier initiative

Third party assessment versus self-assessmentComprehensive versus ModularNational / Regional versus Cross-BorderSystem functionality versus “Interoperability”Generic versus Domain/Target Specific (LIS,…)


Procedure and kind of attestation

Certification procedure Attestation

granted

Third party assessment by a CAB being a public authority or an organisation granted power by a public authority either by law or by regulation.

Certificate

Third party assessment by a CAB on requirements issued by an organisation not empowered by law or by regulation.

Quality label

Self-assessment with an external audit. Conformity assessment is done by the supplier and documented to a third party, being a public entity, a professional organisation or an industry federation.

No “attestation” but a Quality Mark on

the product is allowed

Self-assessment by vendor who performed testing on his own products and affirms that they conform to a given set of requirements.

Declaration of quality

most

su

itab

le

pro

ced

ure

47

Actual “National” Certification


• Existing “national” certification

• Foreseen within 1-2 years

• Considered

48

Actual “cross border” quality labelling

Not “authority driven”There is no such an authorityNo formal recognition of certificates across borders

Three “private” initiativesEuroRec: independent, focus on EHR systems (functional and exchange as function)

I.H.E.: industry driven, focus on testing the exchange and the technical interoperabilityContinua Health Alliance: industry driven, focus on devices content portability


49

One Destination, Two Itineraries

Different and complementary approaches but always phased in a similar way:

Setting the framework & the decision to go for itPre-assessment: organisational contextAssessment: test related activitiesGranting label or certificate


Roadmap for Certification: national level

Setting the Framework

•Convince the authorities to take the decision to go for quality labelling and certification

•Create the legal framework

•Accredit a certification body

•Create advisory platforms with users and suppliers

•Select the standards to be used

Pre-assessment Phas

e

•Select and prioritise functions / processes to be assessed

•Negotiate feasibility with suppliers and users

•Formulate verifiable quality criteria

•Create appropriate documentation regarding each of the criteria and the procedure

•Manage Q & A and "interpretations"

•Select CAB (Conformity Assessment Body)

Assessment Phas

e

•Define "test population"

•Define "test scenario's"

•Development of assessment tools

•Evaluate test scenario's on feasibility and appropriateness

•Install logistics (inscription, financial aspects,..)

•Assess the applications

•Document conformity or non conformity to the criteria

Granting & Maintaining the

Label /

Certificate

•Decide on granting or not granting the label or certificate

•Document motivation and archive assessment documentation

•Enable "appeal" and "retesting" sessions

•Distribute the "incentives"

•Update / upgrade of the criteria for future use

51

Conditions enabling “national certification”

Consortium listed the top 5 enablers for a country wide certification:• Stimulate the use of certified EHR systems by creating

incentives (€).• Create a legal framework enabling to define quality

criteria for the EHR.• Initiate a cooperative platform involving all stakeholders

to define domain / profession specific quality criteria for the different EHR settings (GP, secondary care, …).

• Stimulate the use of certified EHR systems by offering services (e.g. simplification of administrative procedures).

• Initiate a cooperative platform involving all stakeholders to define overall quality criteria for the EHR.


Cross-border Certification Roadmap

Setting the Framework

•Create or recognise an institute or organisation to grant labels to EHR applications

•Create contractual context governing the quality labelling platform and processes

•Create advisory platforms with users and suppliers to gather domain expertise

•Elaborate and validate internal quality procedures, compliant with standards (ISO 17020)

Pre-assessmen

t Phas

e

•Select and prioritise functions / processes to be assessed (set of criteria / IHE profiles) applicable across Europe

•Consider feasibility with suppliers and users

•Create appropriate documentation regarding each of the criteria and the testing procedure

•Manage Q & A and "interpretations"

•Select CAB (Conformity Assessment Body)

Assessme

nt Phas

e

•Install logistics (inscription, financial aspects,..)

•Define "test scenario's"

•Develop assessment tools

•Evaluate test scenario's on feasability and appropriatness

•Assess the applications

•Document comformity or non confirmity to the individual criteria or to the profile

Granting & Maintaining the

Label /

Certificate

•Decide on granting or not granting the label or certificate

•Document motivation and archive assessment documentation

•Enable "appeal" and "restesting" sessions

53

Roll-out of Quality Labelling & Certification

Mandatory

Voluntary

Incentivised


54

Mandatory CertificationAdvantages

Less important to consider feasibility. Less efforts required for consensus building with users and suppliers.Easier to implement a long term strategy for authority and for suppliers.

ProblemsOnly possible when use of EHR is made mandatory.Authority needs to have the means of its ambition.Defensive suppliers contesting ‘interpretations’ of the criteria.May reduce competition and innovation.


55

Voluntary quality labelling & certificationAdvantages

Based on a consensus between different stakeholders.Less “contestation”.Shared cost has more chance to be accepted.

ProblemsSlower take-up and still non-certified products on the market. Risk that the actual results will falter.Long and fastidious negotiations regarding requests from users and/or authorities.Long and stepwise approach.Weakest partners may hamper progress.


56

Incentivised model

Best of two worlds… empowerment of the user.Advantages

Involvement of all stakeholdersWin-win for all stakeholders

ProblemsImportant involvement of authorities granting incentivesImportant pre-assessment efforts.


57

Functionality & Interoperability


58

Cooperation: eHealth Quality Institute

To improve quality and efficiency of the processesTo align / link requirements formulation and documentationTo have full documentation availableTo centralise information on “certified” applications and componentsTo invest in toolsTo favour trans-European certification through recognitionTo increase independence from providers and users


59

European Dimension ?

Healthcare remains national competence.Dual approach only realistic:

Extended National CertificationIncremental upgrade of cross-border initiatives

Recognition of national certificatesEuropean “incentives” for using certified systems


60

Recommendations from the Roadmap


1. Business aspects2. Main Risks3. Recommendations4. Conclusions

61

Business DefinitionDefining and rolling-out a / ONE common business opportunity is very hypothetic:

Different approaches (authority driven versus market driven).Different options and content (criteria) in the different countries.Market driven initiatives each on a different track.Impossible to receive information from the commercial initiatives.

Market potential for EuroRec and IHE described in Del. D5.2


62

Budget for maintaining infrastructure


Budget for completion of the translation of the actual repository

Number of languages

Cost per language

Total

Completing actual sets of translations 20 15.000 300.000

New Languages 5 20.000 100.000

Coordination and Consistency Management 25 5.000 125.000

525.000

Supporting integration of national specific variants

Defining the national variants and translating them 30 5.000 150.000


225.000

Annual Maintenance Costs

Number of languages

Cost per language

Total

Maintenance of the translated statements 25 5.000 125.000


187.500

Expanding the repository to be more comprehensive (3.000 statements)

Central Repository Maintenance and Expansion 225.000

Translating to the different languages 25 15.000 375.000

600.000

63

Sustainability of national certification

Create a relation of trust and partnership between authorities, users and industry. Increase economic viability of the suppliers by supporting the use of certified EHR systems and stimulating defragmentation of the market. Costs need to be shared between health authorities and the industry, industry e.g. paying only the effective testing / assessment.


64

Sustainability of national certification

Develop and support a common “language” to describe functional requirements in a comparable way: national repository of reusable statements publicly available. Functional assessment and “interoperability” assessment should be offered by a recognised Health IT Quality Institute.


65

Main Risks identified

1. Lack of any decision to go for quality labelling and certification.

2. Insufficient resources to invest in certification bodies, CABs and in favouring the use of certified EHR systems.

3. Market fragmentation due to national / regional healthcare delivery systems, regulations and the multi-professional and multi-lingual European reality => limited resources.


66

Main risks

4. Nationally defined functional and data exchange related criteria to be avoided, when possible.

5. Actual cross-border health-IT is dominated by solution provider for “technical” departments and services. Insufficient to guarantee quality expressed in reliability, trustworthiness and appropriateness of the content.

6. Balkanisation and commercialisation.


67

Recommendations• Legal and regulatory framework

• Create and harmonise the legal and regulatory framework stimulating national or regional authorities to enforce the use of quality labelled and certified applications.

• Clarify the role of Directive 2007/47/EC regarding software development aspects, EHR functional aspects and Data-Exchange related issues

• Involvement of stakeholders• Certification bodies should be accredited and compliant to

international standards, more precisely ISO 17020.• Favour cooperation between all service providers active in

different areas of quality labelling and certification of EHR systems: administrative data exchange, clinical data exchange and system functionality.

• Create an advisory platform involving all stakeholders to agree on content and feasibility of requirements.


68

Recommendations (2)• Technical Framework

• It is highly recommended to strengthen the European scale pioneering initiatives (EuroRec / I.H.E) in order to keep certification on the agenda.

• Invest in maintenance and expansion of the actual descriptive statements and profiles towards more completeness and towards including more “domain- or profession-specific sets”.

• Address the issue of personnel shortage in health informatics in general and more specifically in health informatics quality assessment.


69

Recommendations (3)

• Quality labelling and certification process

Third party assessment is the most suitable procedure for the still immature market of EHR systemsIt is strongly recommended to start “small”, to evaluate effectiveness and to increase focus step by stepThe incentivised model seems the most promising, surely for self-employed healthcare professionals


70

Recommendations: cross-borderQuality labels and certificates are primarily national. Strengthen national certification in order to improve average quality and to enable in a second step the Trans-European harmonisation, improving comparability and portability of content. The EuroRec statements facilitate cross-border and cross-domain functional certification of EHR systems. It would be a pity not to use these opportunities. The IHE profiles are in principle country-independent and are used as European Profiles for data-exchange.


71

Cross-border recommendationsPromote equivalence of certificates across Europe by validating at European level both the functional descriptive statements of EuroRec and the IHE profiles. Enforce portability of quality assessment labels and certificates across the Union. Consider the possibility to create a cross-border “Register of Quality Labelled or Certified Clinical Software”, offering information about the products (complete EHR systems as well a software modules) and documentation about the certification process.


72

Quality Labelling and Certification Procedures: the quality of the process


73

Deliverable D6.2

Documents quality requirements / applicable standards to quality labelling and certification.Describes the distinct roles of the initiator / authority, the certification body, the accreditation body and the conformity assessment body.Describes in detail the assessment and the granting phase as described in the roadmap deliverable.


74

Applicable standards


Stakeholder Applicable ISO/IEC StandardAccreditation Body ISO/IEC 17011:2004

ISO 9000:2000ISO/IEC 17000:2004VIM:1993

Certification Body ISO/IEC Guide 65Conformity Assessment Body ISO/IEC 17025:2005

[1] General requirements for accreditation bodies accrediting conformity assessment bodies (ISO/IEC 17011:2004)- Quality management systems – Fundamentals and vocabulary (9000:2000)[2] Conformity assessment – Vocabulary and general principles (ISO/IEC 17000:200)[3] International vocabulary of basic and general terms in metrology (VIM:1993)[4] General requirements for bodies operating product certification systems (ISO/IEC Guide 65)[5] General requirements for the competence of testing and calibration laboratories (ISO/IEC 17025:2005)

75

A reminder…


Certification body

CAB

InitiatorIAF

Product

Supplier Authority

Accreditation body

Certificate

Label

ISO/IEC 17020 =>

<= ISO/IEC 17011

ISO/IEC 17025 =>

76

Accreditation Bodies

Standard (17011:2004) addressesLegal responsibilityStructureImpartialityConfidentialityLiability and FinancingAccreditation activitiesGeneral procedures (application, …)Management of non-conformitiesInternal audits…Human resources

Describes the responsibilities of accreditation bodies, certification bodies and conformity assessment bodies


77

Certification Bodies

ISO/IEC Guide 65 outlines the following aspectsGeneral provisionsOrganisational and operational aspectsSubcontractingQuality system in placeProcedural aspects about granting, maintaining, extending, suspending and withdrawing a certificateInternal auditsDocumentationRecord keepingConfidentialitySpecific quality criteria regarding the personnel


78

Certification Bodies (2)

Other topics addressed in the ISO GuideWay to deal with changes in certification requirementsAppealsProcedures for complaints, appeals and disputesProcedure for the certification applicationHow to prepare the evaluationThe evaluation itselfDecision making on the certificationSurveillance aspectsUse of licenses, certificates and other marks


79

CAB – Conformity Assessment Bodies

ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratoriesAddresses 5 major topics

Scope of the standardNormative referencesTerms and definitionsManagement requirementsTechnical requirements


80

Actual Status

Accreditation is “national” => cross-border activities are formally excludedmain issues still to be addressed:• Independence• Split between CB and CAB

National initiativesAccreditation body seems to be active in FranceBelgium, Serbia, Ireland… clear split CB/CABOther countries: authority defining scope, testing and granting the certificate


81

Assessment & Granting Phase


82

Assessment Phase

Test Bed (environment, test population, test patient)Test Scenarios and Test ScriptsAssessment toolsEvaluation feasibility & appropriatenessLogistics, mainly the service agreementAssess the applicationDocumenting conformity & non- conformity to the criteria


83

Granting Phase

Decision (by the certification body)MotivationArchive assessment documentationAppeal and RetestingValidity of Label or Certificate (version…)Withdrawal of Label or CertificatePreparing the future


84

Documenting actual certification

The deliverable documents the authority driven national certification procedure(s) in

AustriaBelgiumDenmarkFranceIreland PortugalSerbia


85

For each country we documentedWho initiates / initiated the process of certification?Legal or Regulatory context, if anyDomain addressed by the national certificationIs the certification compulsory, optional and/or incentivisedInvolvement of different stakeholdersNational scenario(s)Effectively granting the CertificateAppeal proceduresPossible contractual commitments related to the Quality Labelling and Certification


86

Documenting actual quality labelling

The deliverable documents the market driven cross-border quality labelling procedure(s) as implemented by:

EuroRecI.H.E.Continua Health Alliance


87

For each initiative we documented

Who initiates / initiated the process of certification?Legal of Regulatory context, if anyDomains addressed by the quality labelIs there any incentive favouring the quality labelling?Involvement of different stakeholdersImplemented scenario(s)Effectively granting the Quality LabelAppeal proceduresPossible contractual commitments related to Quality Label or Certificate


88

Global Comparison

Global comparison made by the University of Victoria between:

The Certification Commission for Healthcare Information Technology (CCHIT) – United StatesDepartment of Health and Human Services (HHS) – United StatesISO/IEC 15408 – The Common Criteria for Information Technology Security EvaluationThe European Institute for Health Records (EuroRec)Health on the Net foundation (HON)Canada Health Infoway (Infoway)The Australian National E-Health Transition Authority (NEHTA)


89

What has been compared?

• Service category• Quality objectives(accountability, identifying

purpose, consent, limiting collection, limiting use, disclosure, retention, accuracy, safeguards, openness,

individual access, challenging compliance) • Focus of conformance assessment• Certification body (administration, independencde,

impratiality, integrity, quality system, accreditation, complaints & appeal)

• Methods and procedures

• Certificate (assessment details, expiration, re-certification, warranty..)


90

EuroRec : Certification Body


91

HHS: Certificate related


92

Recommendations (about the process)


•Discretion and Confidentiality

•Impartiality

•Openness

•Distinct roles involved organisations

Independence

•Initial Documentation

•Rules of Evaluation

•Testing Documentation

Documentation of the process

93

Recommendations (about the process)


•Involvement of all stakeholders

•Distinguish generic and domain specific

•Consider national / regional variants

Content to be validated /

tested

•Precise unambiguously the version of the SW

•Limit the validity to intended user group(s)

•Limit validity to region or country (if applic.)

Limitations of Certificate or

Label

•Pay attention to effective use to realise full added value

Effective Use

94

What could the future bring?


95

Certification

Authority driven certification will progress progressively:

because quality only now becomes an issuebecause the authorities want to influence the products

There is a risk that these efforts aren’t

coordinated and comparablere-usable outside country of certification


96

Quality Labelling & Quality Mark

Border-independent and on initiative of the supplierQuality Mark for products:

established practice and related businessnot independent but it works

Quality Labelprogress slowly (free to candidate for it)(never) comprehensiveSMEs are a problem


97

Main risk

Americanisation:One single countryRules issued endorsed by the authoritiesResources available to “influence” the market


98

Conclusions

The project clearly contributed to increased awareness about QL&C amongst specialised industry, health professionals and health authorities.Certification is in the lift (> 10 countries)

In principle national and authority drivenStill fragmented and not (very) comparableCovers functionality and data exchange


99

Conclusions

Market driven QL is the current approach for cross-border quality assessment. This should be supported, independent and freely accessible.Market driven QL services are offered mainly by three organisations addressing different aspects :

• Functional quality of the (professional) applications

• Ability to exchange (emit and receive) content• Portability (and safety) of device outcome


100

Conclusions

Functional certification should be comparable and documented in the same way across borders by using an EHR functional descriptive language. Comparable national certification leads towards an increased harmonisation of the applications and facilitates interoperability.One-stop shopping regarding both data exchange and functionality quality labelling could boost the quality labelling and certification activities.



Thank You...