Establishing stability of WHO Prequalification · Technical Guidance Series (TGS) Establishing stability of an in vitro diagnostic for WHO Prequalification TGS–2 Draft for public

Technical Guidance Series (TGS)

Establishing stability of an in vitro diagnostic for

WHO Prequalification TGS–2

Draft for public comment 14 December 2015

Working document 14 December 2015

© World Health Organization 2015 All rights reserved. Publications of the World Health Organization can be obtained from WHO Press, World Health Organization, 20 Avenue Appia, 1211 Geneva 27, Switzerland (tel.: +41 22 791 3264; fax: +41 22 791 4857; e-mail: [email protected]). Requests for permission to reproduce or translate WHO publications – whether for sale or for non-commercial distribution – should be addressed to WHO Press, at the above address (fax: +41 22 791 4806; e-mail: [email protected]).

The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement.

The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.

All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use.

Contact: Irena Prat, EMP Prequalification Team Diagnostics

WHO - 20 Avenue Appia - 1211 Geneva 27 Switzerland

Working document 14 December 2015

mailto:[email protected]

Establishing stability of an in vitro diagnostic for WHO Prequalification TGS–2

Contents

1 Introduction ....................................................................................... 5 1

2 Definitions and abbreviations ........................................................... 6 2

3 WHO prequalification requirements ............................................... 10 3

4 Basic principles for stability testing ................................................. 11 4

5 Shelf-life studies .............................................................................. 15 5

6 Component stability studies ............................................................ 16 6

7 Stability during transport ................................................................ 19 7

8 In-use stability studies ..................................................................... 21 8

9 Production lots used in stability studies .......................................... 22 9

10 Stability plan .................................................................................... 24 10

11 Stability report ................................................................................. 29 11

12 Changes to a Prequalified IVD ......................................................... 30 12

13 Authors and acknowledgements ..................................................... 32 13

14 References ....................................................................................... 33 14

Appendix 1: Example stability protocols .............................................. 35 15

Appendix 2: Specimens for the stability testing panel .......................... 43 16

Appendix 3: Summary table of standards relevant for stability 17 studies ................................................................................................... 47 18

Working document 14 December 2015 3

WHO Prequalification of IVDs

WHO Prequalification Programme: IVD Technical Guidance Series

The World Health Organization (WHO) Prequalification Programme is coordinated through the Department of Essential Medicines and Health Products. The aim of WHO prequalification of in vitro diagnostics (IVDs) is to promote and facilitate access to safe, appropriate and affordable in vitro diagnostics of good quality in an equitable manner. Focus is placed on in vitro diagnostics, known commonly as IVDs, for priority diseases and their suitability for use in resource-limited settings. The WHO Prequalification Programme undertakes a comprehensive assessment of individual IVDs through a standardized procedure aligned with international best regulatory practice. In addition, the WHO Prequalification Programme undertakes post-qualification activities for IVDs to ensure the ongoing compliance with prequalification requirements.

The findings of the WHO Prequalification of IVDs Programme are used to provide independent technical information on safety, quality and performance of in vitro diagnostics, principally to other United Nations (UN) agencies but also to WHO Member States and other interested organizations. The WHO prequalification status, in conjunction with other procurement criteria, is used by UN agencies, WHO Member States and other interested organizations to guide their procurement of in vitro diagnostics.

In vitro diagnostics (IVDs) prequalified by WHO are expected to be accurate, reliable and be able to perform as intended for the lifetime of the IVD under conditions likely to be experienced by a typical user in a resource-limited Member State. The countries using WHO-prequalified IVDs often have minimal regulatory requirements. In addition, the users of IVDs in these countries have different needs resulting in different requirements. For instance, the IVDs are often used in patients with a different disease profile, by health care workers without extensive training in laboratory techniques, in harsh environmental conditions and in settings without extensive pre- and post-test services. Therefore, the requirements of the WHO Prequalification Programme may be different to the requirements of the users and regulatory authority in the country of manufacture.

The Technical Guidance Series was developed following a consultation, held on 10-13 March 2015 in Geneva, Switzerland attended by experts from national regulatory authorities, national reference laboratories and WHO prequalification dossier assessors and inspectors. The guidance series is a result of the efforts of this and other international working groups.

This guidance is intended for manufacturers interested in WHO prequalification of their IVD. It applies in principle to all IVDs that are eligible for WHO prequalification for use in WHO Member States. It should be read in conjunction with relevant international and national standards and guidance.

The TGS guidance documents are freely available on the WHO web site.


Use of prequalified IVDs

Prequalification requirements

About the Technical Guidance Series

Audience and scope

4 Working document 14 December 2015


1 Introduction 19

1.1 Key concepts 20

Stability is the ability of an IVD reagent to maintain its performance characteristics 21 over a defined time interval [1,2]. The purpose of stability studies is to verify the 22 time interval and the storage conditions over which stable performance 23 characteristics of an IVD can be claimed. 24

1.2 Rationale of stability studies 25

The stability of an IVD is fundamental for its reliable performance over a defined 26 period of time. It is a regulatory requirement for the manufacturer to provide 27 objective, scientifically sound evidence to support all claims made regarding the 28 stability of an IVD. In addition a manufacturer can use stability studies to show 29 that all lots manufactured during the commercial life of the IVD will meet 30 predetermined user needs (inputs). 31

1.3 Purpose of this document 32

The purpose of this document is to provide IVD manufacturers with guidance on 33 possible approaches to determine stability. More specifically it describes the 34 requirements for WHO prequalification in terms of stability1. 35

1.4 Limitations of this guidance 36

This guidance document should not be taken as a prescriptive checklist of what 37 should be performed, but as a guide on how to improve processes and generate 38 the evidence needed to ensure a comprehensive, systematic procedure with an 39 appropriate risk management plan. 40

The examples included throughout the document apply to the principles outlined 41 in this document only. Manufacturers must still perform their own product-42 specific risk assessment for each of their IVDs. 43

It is possible that, depending on the particular categorization of the product and 44 on the particular jurisdiction that additional requirements may apply. These 45 regulatory and legal issues are specific for each regulatory authority and are 46 beyond the scope of this document. 47

1 See Instructions for Compilation of a Product Dossier, Prequalification of Diagnostics, PQDx_018 v3 27.08.2014, Section 7.2, Stability (excluding specimen stability).


http://www.who.int/diagnostics_laboratory/evaluations/100506_pqdx_018_dossier_instructns_v1.pdf


2 Definitions and abbreviations 48

49

2.1 Definitions 50

The definitions given below apply to the terms used in this document. They may have different 51 meaning in other contexts. 52

Accelerated stability evaluation: Study designed to increase the rate of chemical and/or physical 53 degradation, or change, of an IVD reagent by using stress environmental 54 conditions to predict shelf-life. 55

NOTE: The design of an accelerated stability evaluation can include extreme 56 conditions of temperature, humidity, light or vibration. 57 Source: [1], definition 3.1 58

Acceptance criteria: A defined set of conditions that must be met to establish the performance of a 59 system. 60 Source: [2] 61

Numerical limits, ranges, or other suitable measures for acceptance of the results 62 of analytical procedures. 63 Source: [3] 64

Accuracy of measurement: Closeness of the agreement between the result of a measurement and a 65 true value of the measurand. 66

NOTE 1: Accuracy of measurement is related to both trueness of measurement 67 and precision of measurement. 68

NOTE 2: Accuracy cannot be given a numerical value in terms of the measurand, 69 only descriptions such as 'sufficient' or 'insufficient' for a stated purpose. 70 Source: [4], definition 3.1 71

Arrhenius plot: Mathematical function that describes the approximate relationship between the 72 rate constant of a chemical reaction and the temperature and energy of activation. 73 Source: [2] 74

Batch/Lot: Defined amount of material that is uniform in its properties and has been 75 produced in one process or series of processes. 76 Source: [5], definition 3.5 77

Component: Part of a finished, packaged and labelled IVD medical device. 78 Source: [5], definition 3.12 79

NOTE 1: Typical kit components include antibody solutions, buffer solutions, 80 calibrators and/or control materials. Source: [5]. 81

Constituent Raw materials used to make a component. 82 Source: WHO 83 84 Design input: The physical and performance requirements of an IVD that are used as a basis for 85

IVD design. 86 Source: [6], definition (f) 87

Drift: Characteristic slow change of a metrological value from a measuring instrument. 88 Source: [7] 89



Environmental factors: Variables that might affect the performance or efficacy of IVD reagents e.g. 90 temperature, airflow, humidity, light. 91 Source: [2] 92

WHO note: For WHO purposes, this also includes dust and micro-organisms. 93

Evidence: Information which can be proved true, based on facts obtained through 94 observation, measurement, test or other means 95 Source: Modified from [8], definition 3.8.1 96

Instructions for Use (IFU): Information supplied by the manufacturer to enable the safe and proper 97 use of an IVD 98

NOTE: Includes the directions supplied by the manufacturer for the use, 99 maintenance, troubleshooting and disposal of an IVD, as well as warnings and 100 precautions. 101 Source: [5], definition 3.30 102

In vitro diagnostic (IVD): A medical device, whether used alone or in combination, intended by the 103 manufacturer for the in vitro examination of specimens derived from the human 104 body solely or principally to provide information for diagnostic, monitoring or 105 compatibility purposes. 106

NOTE 1: IVDs include reagents, calibrators, control materials, specimen 107 receptacles, software, and related instruments or apparatus or other articles and 108 are used, for example, for the following test purposes: diagnosis, aid to diagnosis, 109 screening, monitoring, predisposition, prognosis, prediction, determination of 110 physiological status. 111 NOTE 2: In some jurisdictions, certain IVDs may be covered by other regulations. 112 Source: [9] 113

IVD reagent: Chemical, biological or immunological components, solutions, or preparations 114 intended by the manufacturer to be used as an IVD 115 Source: [5], definition 3.28 116

WHO note: This document uses the terms IVD and IVD reagent interchangeably. 117

118

Metrological traceability: Property of the result of a measurement or the value of a standard 119 whereby it can be related to stated references, usually national or international 120 standards, through an unbroken chain of comparisons all having stated 121 uncertainties. 122

NOTE 1: Each comparison is affected by a (reference) measurement procedure 123 defined in a calibration transfer protocol. 124 Source: [4] 125

Performance claim: Specification of a performance characteristic of an IVD as documented in the 126 information supplied by the manufacturer 127

NOTE 1: This can be based upon prospective performance studies, available 128 performance data or studies published in the scientific literature. 129 “Information supplied by the manufacturer” includes but is not limited to: statements in 130 the IFU, in the dossier supplied to WHO and /or other regulatory authorities, in advertising, 131 on the internet. 132

Referred to simply as “claim” or” claimed” in this document 133

Source: [5], definition 3.51 134



135 Real-time stability evaluation: Study designed to establish or verify the shelf-life of the IVD reagent 136

when exposed to the conditions specified by the manufacturer 137 NOTE 1: Conditions that can affect stability of an IVD reagent include temperature, 138

transport conditions, vibration, light, humidity. 139 Source: [1], definition 3.8 140

Risk management: The systematic application of management policies, procedures and practices to 141 the tasks of analysing, evaluating, controlling and monitoring risk. 142 Source: [10] 143

Risk management plan: For the particular IVD being considered, the manufacturer shall establish 144 and document a risk management plan in accordance with the risk management 145 process. 146 Source: [10], para 3.4 147

Shelf-life: Period of time until the expiry date, during which an IVD reagent, in its original 148 packaging, maintains its stability under the storage conditions specified by the 149 manufacturer 150

NOTE 1: Stability and expiry date are related concepts. 151 152

Source: [5], definition 3.66 153

WHO NOTE: In this document “Labelled life” is considered as the time up to the expiry date 154 printed on the label of an IVD or a component of the IVD. 155

Stability: The ability of an IVD reagent to maintain its performance characteristics within 156 the limits specified by the manufacturer 157

NOTE 1: Stability applies to 158 - IVD reagents, calibrators, and controls, when stored, transported and used in the 159 conditions specified by the manufacturer 160 - reconstituted lyophilized materials, working solutions, and materials removed 161 from sealed containers, when prepared, used and stored according to the 162 manufacturer’s instructions for use 163 - and measuring instrument or measuring systems after calibration. 164

NOTE 2 to entry: Stability of an IVD reagent or measuring system is normally 165 quantified with respect to time: 166

- in terms of the duration of a time interval over which a metrological property 167 changes by a stated amount, 168 - in terms of the change of a property over a stated time interval. 169 Source: [1], definition 3.10 170

In-use stability: Duration of time over which the performance of an IVD reagent within its 171 expiration date remains within specified limits after opening the container system 172 supplied by the manufacturer, and put into use under standard operation 173 conditions (e.g. storage on the instrument) 174 For the purpose of this guidance, WHO considers that it also includes the number of times 175 the reagents can be removed and returned to the storage condition without impact on 176 test kit performance. It shall reflect the routine conditions of use e.g. On-board stability, 177 reconstitution, and open-vial/bottle stability. 178

Source: [2] 179

Stability monitoring: Real-time stability testing at certain points in time during shelf-life (or in-use) 180 to assure that an IVD reagent performs within specified claims. 181 Source: [2] 182



NOTE: A continuing stability monitoring program (ongoing stability monitoring) 183 is required to verify that the stability claim is maintained over the commercial life 184 of the product. Data on stability should be obtained at end of shelf life 185 (Reference [1] para. 4.1) and additionally at half assigned-life to so that if 186 problems occur they can be dealt with in a timely fashion. 187

WHO note: WHO expect that file samples are used for each component and lot of 188 product. 189

190

Trueness of measurement: Closeness of agreement between the average values obtained from a 191 large series of results of measurements and a true value. 192 Source: [4], definition 3.33 193

2.2 Abbreviations 194

CE Conformité Européenne (European Conformity) CV Coefficient of variation

CD4 Cluster of differentiation 4 CLSI Clinical and Laboratory Standards Institute EIA Enzyme-linked immunoassay GHTF Global Harmonization Task Force HBsAg Hepatitis B surface antigen

HBV Hepatitis B virus HCV Hepatitis C virus HIV Human immunodeficiency virus IFU Instructions for Use

IgM Immunoglobulin M IMDRF International Medical Devices Regulators Forum ISO International Organization for Standardization IVD In vitro diagnostic

NAT Nucleic acid test OD Optical density PEI Paul Ehrlich Institute QA Quality assurance QC Quality control

QMS Quality management system RDT Rapid diagnostic test RPM Revolutions per minute R&D Research and development

TP Treponema pallidum USP U.S. Pharmacopoeial convention

195



3 WHO prequalification requirements 196

WHO requires that reports of studies used in establishing the stability claims for 197 the product are submitted as part of the prequalification application2. In the 198 submission, manufacturers should describe the rationale, the study methods, the 199 stability monitoring program followed and the testing algorithms used, with 200 references to the relevant standard operating procedures. The information 201 provided should demonstrate the link to the predetermined user requirements 202 and product development. 203

3.1 Manufacturer responsibility 204

It is a manufacturer’s responsibility to ensure that all claims made regarding the 205 stability of the IVD performance are supported by objective, scientifically-sound 206 evidence. 207

3.2 Standards 208

WHO recommends the following standards for the use in establishment of 209 stability claims: ISO 23640:2013 [1], CLSI EP25-A [2] and ASTM:D4169-14 [11]. 210

It is recommended that manufacturers be familiar with these standards and 211 consider them when designing and planning their stability studies. 212

3.3 Suitability for use in Member States 213

The stability studies submitted to WHO should accurately reflect the expected 214 environmental conditions and the normal usage conditions/methods encountered 215 by the users in WHO Member States, such as: 216

• Extremes of temperature for in-use conditions and during transportation 217 • Extremes of humidity encountered during in-use conditions, transportation 218

and storage 219 • Dust 220 • Light, both the amount required for accurate testing/results interpretation and 221

any affects that light may have on the IVD functionality 222 • Micro-organisms 223

3.4 Meeting customer requirements 224

By undertaking well-designed stability studies including periodic verification 225 activities, the manufacturer can demonstrate that the product meets input (i.e. 226 customer) requirements, as required by ISO 13485:2003 (see [12] under 7.2, 227 Customer-related processes). Meeting predetermined user expectations, not 228 merely evaluating the capability of an IVD, is a fundamental aspect of 229 development of IVDs (see [6] definition (f) and [12] para 7.3.4). It is a proactive 230 means for the manufacturer to prevent quality problems at lot release and in the 231 post-production and marketing phase. 232

2 PQDx_049 Product dossier checklist and PQDx_018 Instructions for compilation of a product dossier Both available on the WHO Prequalification of in vitro diagnostics website http://www.who.int/diagnostics_laboratory/evaluations/en/


http://www.who.int/diagnostics_laboratory/evaluations/en/


4 Basic principles for stability testing 233

4.1 Critical characteristics of the IVD 234

A well-designed stability study must generate evidence of stability of each of the 235 critical constituents of the IVD (risk-evaluated critical constituents), each of the 236 claimed analytes, and any particular level of performance including precision, 237 sensitivity and specificity of the kit. 238

Examples: 239

1) A hepatitis C virus (HCV) assay containing the critical constituents related to 240 detection of NS3 or core proteins should have the stability of all such constituents 241 proven. 242

2) For an assay designed to detect both IgG and IgM by use of protein A and protein 243 L, the stability of both protein A and protein L should be proven. 244

3) For CD4, all the antibodies involved (e.g. anti-CD3 and anti-CD4) must be shown 245 to be stable. 246

4) For an IVD claimed to detect particular seroconversion specimens, or genotypes, 247 or to have specified precision at particular analyte concentrations, or a particular 248 specificity, each of these claims must be proven over the stated shelf-life. 249

4.2 Finalised product presentation 250

During stability testing, all IVD components (including the device, calibrator 251 and/or control material, etc.) must be made and tested to the finalised 252 manufacturing documentation and in the finalised packaging including intended 253 labels and containers. All presentations (e.g. different buffer volumes used for 254 different kit sizes) must be used during stability testing. 255

4.3 Environmental conditions 256

The study should subject the IVD to a combination of conditions which define the 257 limits of stability for all lots made during its commercial life. The combinations of 258 conditions, durations of exposure and the number of lots to be used will be driven 259 by a manufacturer’s risk assessment for the IVD and data from R&D. The risk 260 assessment should take into account at least: 261

• the variability of the constituent materials (identifying the most important 262 sources of variability); 263

• the nature of the users’ environments; and 264 • extreme conditions potentially occurring during transportation to those 265

users (see also 3.3). 266

Boundary conditions for stability studies should reflect realistic extreme 267 conditions that are consistent with the design input requirements for the IVD. The 268 consequent stability studies will prove the IVD capable of meeting performance 269 requirements at the end of its stated shelf-life, after transport to the users. 270



4.4 Minimum number of lots 271

Similarly to clinical performance validation, the design of stability studies should 272 take into consideration lot-to-lot variation, with a risk assessment to identify the 273 most important sources of variability. Lot variability is caused usually more by the 274 biological reagents than by the actual manufacturing process. Although existing 275 standards recommend the use of one lot for certain stability studies, the impact of 276 lot-to-lot variability must be taken into consideration and use of additional lots 277 may be necessary. To ensure the potential of lot-to-lot variability is addressed, 278 optimally lots containing different batches of critical constituents such as 279 nitrocellulose membranes, recombinant antigens, peptides, nucleic acids and 280 enzymes used in nucleic acid testing (NAT), etc. that are as different as possible 281 should be used. 282

Example: For NAT assays, it is critical to use unique enzyme lots for stability 283 studies. Other components including primer, probe and buffer can also be 284 affected by the manufacturing process (purity, pH, DNase & RNase 285 contamination, etc.). For these, different lots are also highly desired that 286 represent both material and process variability. 287

4.5 Assessment of liquid components 288

It is standard best practice in stability studies to ensure that liquid components 289 are in contact with all the parts of their container – vial, sachet or bottle, such as 290 the stopper, the seal and the body of the container. This is sometimes called 291 “inverted container stability” but is probably best studied by ensuring all 292 containers are on their sides and disturbed by movement during the stability 293 study. This aspect needs particular attention for in-use stability studies of those 294 components that are diluted or reconstituted from freeze-dried before use. 295

4.6 Specimens for the stability testing panel 296

The specimens used in the stability testing panels must reflect all the performance 297 claims related to the IVD. The specimen types most likely to be utilised with an 298 IVD in WHO Member States should be considered and included in the specimen 299 panels used throughout the stability studies (see Appendix 2). If a variety of 300 specimen types (e.g. serum, plasma, whole blood, saliva) is claimed as being 301 suitable for use in the instructions for use (IFU), the stability plan must be 302 designed to provide evidence that the IVD will maintain each of the claims (e.g. 303 sensitivity, specificity, proportion of valid runs, precision) for each of the specimen 304 types for the whole of the claimed shelf-life including transport to the final users. 305 Evidence should be statistically valid (see section 10).The stability testing panel 306 must be validated accordingly and rejection and replacement criteria should be 307 established. Regulatory requirements may also dictate the addition of panel 308 members. 309

A stored validated stability testing panel is not always feasible. For example, this is 310 often the case for assays requiring fresh and/or whole blood specimens e.g. CD4, 311 assays to detect RNA. When replacing panel members, the accuracy of results 312 generated with the replacement material must be confirmed using an appropriate 313 reference comparator method. Replacement criteria for unstable panel members 314 will include the duration for which a critical member will give valid results. 315



4.7 Validation of stability testing panel 316

The validation of the stability testing panel members used is critical. Stability 317 testing panel members themselves must be stable, and they must monitor 318 parameters that are useful to control the component involved. 319

4.8 Assigning criteria to panel members 320

Stability testing panel members are chosen deliberately to ensure each member 321 has an attribute pertinent to the intended use. As with lot release testing, the goal 322 of stability testing is to ensure that the test method appropriately monitors the 323 functionality of the antigens, epitopes, and antibodies that are relevant to the 324 intended use at the end of the assigned (shelf/in use) life. 325

For instance, the intended use claim may be that early seroconversion specimens 326 are detected. To show that this claim is true at the end of the product’s life, a very 327 early seroconversion specimen is included in the stability panel. This specimen 328 may be a weakly reactive IgM specimen. 329

An expected value is then assigned to each panel member and this is used to 330 assign the acceptance criteria for that panel member. The value for each member 331 is assigned in a measurable manner relevant to the outputs of the particular 332 methodology. For instance, the acceptance criteria for each panel member may 333 be assigned in terms of sample-to-cut-off ratio, cycle time (CT) values, and band 334 intensity measured semi-quantitatively/quantitatively. 335

In the example of a weakly reactive IgM seroconversion specimen, the specimen 336 at the start of shelf life may have a reading score on an RDT of 1+ out of 4, 337 assigned by using a semiquantitative value based on band intensity. The 338 acceptance criteria may be that all reactive specimens remain reactive, and all 339 non-reactive specimens do not react in the assay. 340

As such, panel members must be chosen that not only will be relevant to 341 demonstrate the intended use, but have values that will appropriately detect and 342 therefore monitor any deleterious effects of storage. A strong positive specimen, 343 which has a 4+ out of 4 semi-quantitative reading value, may remain giving this 344 reading despite decay in the assay, whereas a specimen with a reading of 1+ out 345 of 4 (with an assigned acceptance criteria of remaining positive) is more likely to 346 give an indication of the ongoing stability of the assay. 347

Thus it is essential to know that where a panel member meets acceptance criteria, 348 this is a true reflection of the stability of the product and not due to the inability 349 of the specimen result to reflect this change. 350

4.9 Time points 351

A simple study design requires three testing intervals: 352 • an initial baseline test and 353 • a test at the time point beyond the claimed stability limit 354 • and one point in between. 355 However, this is a high risk approach that has the potential for wastage of time 356 and resources. If the IVD does not meet the acceptance criteria at the end of 357 testing there is little information about the deterioration of the component or IVD 358 (or lack of deterioration) in the interim period. 359



A more effective approach is to test at predetermined time point intervals. The 360 manufacturer should decide on practical intermediate test points. The number 361 and length of testing intervals should be determined in advance and form part of 362 the stability plan/protocol. This planning will help to understand the resources 363 required to execute the experiment. 364

Testing of all panel members is not required at all test/time points. However 365 testing with all panel members is required at the initial, the second last and the 366 last test/time point of any of the study specimen types. The manufacturer should 367 decide on practical intermediate test points at which a smaller minimal number of 368 panel members are tested. There should be a documented rationale for the 369 choice of the panels used at the intermediate test points (e.g. representative 370 members, specimens that are close to the medical decision points and at the 371 extremes of the assay range tested). 372

4.9.1 Duration of testing 373 Testing conducted in stability studies should extend beyond the shelf-life 374 determined from the user needs. The shelf-life should be assigned based on a risk 375 assessment of the lot-to-lot variability in signal change at the end of shelf life. At a 376 minimum, testing should extend at least one time point (one testing interval) 377 beyond the determined user requirement. This provides a safeguard in the event 378 of unexpected IVD failure at the end of the testing period, in which event 379 extrapolation from an earlier time point would not be considered acceptable. 380

381

It is recommended to utilize standardized units of measure for the entire study 382 (e.g. Unopened kit shelf life are always measured in months; opened kit /reagent 383 stability in days or weeks ) 384



5 Shelf-life studies 385

5.1 Requirements for determination of shelf life 386

The stated shelf-life of an IVD must be based on real-time experimental results. 387 Accelerated stability studies are usually not sufficient to support a claimed shelf-388 life. They might be used in situations where experience already exists with similar 389 products (see para 4.1 in Reference [1]) or when the stability of very similar 390 products is already known (see para 7.3.1 in Reference [2]). 391

WHO prequalification requirements: If at the time of dossier submission 392 the real-time study outcome is not available, accelerated studies might 393 be considered. The manufacturer must justify why the accelerated study 394 is acceptable as supportive evidence until real-time experimental results 395 become available. In these cases, the results of real-time stability studies 396 will be requested as a condition of WHO prequalification. The shelf-life of 397 the IVD could be extended as the real-time data accumulates. 398

5.1.1 Real-time stability studies 399 Real-time stability is determined using storage temperatures derived from user 400 requirements, over a period longer than the required life of the IVD. 401

Where a range of storage temperature is claimed (e.g. “Store at 4–40°C”), WHO 402 expects the studies will provide evidence for stability over the whole of the 403 temperature range for at least the length of the claimed shelf-life. Exceptionally, 404 where claimed stability is restricted to a limited range e.g. “Store at 2-8°C”, it is 405 acceptable that stability studies are conducted at a single temperature within this 406 range. 407

A sequential approach should be used [2], in which IVDs are first submitted to 408 stresses simulating transport before they are placed into a shelf-life or in-use 409 study. This approach best simulates the real-life situation, where products will 410 first be transported to the end-user and then stored under the recommended 411 conditions before use, possibly almost until the end of their labelled shelf-life. 412

5.1.2 Accelerated stability studies 413 Accelerated stability studies are designed to predict the shelf-life of an IVD from 414 the increased rates of chemical and/or physical degradation caused by extreme 415 environmental conditions (e.g. elevated temperature at higher humidity). 416

If the Arrhenius equation is used to calculate the expected life at temperatures 417 other than those actually used, then the parameters of the equation must be 418 derived from the data and not assumed [2]. 419

Accelerated stability studies provide results in a relatively short time. However the 420 results of these studies are made using assumptions about the degradation of 421 reagents and IVD components that may not reflect their performance under 422 normal conditions of storage and use. 423



6 Component stability studies 424

6.1 General principles 425

6.1.1 Testing on final specifications 426 Component stability studies, including antimicrobial and desiccant studies, must 427 be performed using components made according to finalized and approved 428 manufacturing specifications – ideally to validated manufacturing scale – on 429 qualified manufacturing equipment and meeting finalized and approved in-430 process quality control (QC) specifications. 431

6.1.2 Considering component stability 432 Sometimes components of IVDs are prepared in bulk and stored before being 433 used in several different lots of a completed IVD. The design input documentation 434 should define how long components are likely to be stored before use. With that 435 information, component stability studies should be planned to give evidence that 436 component labelled lives will not restrict IVD labelled lives: an IVD cannot have a 437 labelled life beyond that of any of its dependent components. 438

Shelf-lives of components manufactured in bulk and used in several different lots 439 of an IVD can be verified as for the IVD itself – three lots of the component as a 440 minimum for shelf-life studies and, depending on documented risk assessment 441 related to variability, one or more lots subsequent to change. The evaluated lots 442 of the component must differ in batches of critical constituents but, again subject 443 to documented risk assessment, may all be tested in their final presentation with 444 a single set of the other components which will be used together to constitute the 445 IVD. 446

Examples of stored components: Wash solutions and substrates for 447 enzyme immunoassay (EIA), amplification reagents for nucleic acid 448 testing, calibrators for quantitative tests, manufactured and stored in 449 their final labelled vials ready to be put into a kit 450

Component stability can be assessed from the functionality of the lot and also by 451 factors related to the component itself, such as turbidity, colour change, microbial 452 contamination and pH of liquid components changes over time. 453

Depending on the IVD and the conditions it is subjected to it may be necessary to 454 distinguish between turbidity that arises from heat/cold denaturation and 455 turbidity that arises from microbial contamination. 456

6.1.3 Considering constituent stability 457 The plan should also consider whether components made from new constituents 458 (antigens, recombinant antigens, enzymes, antibodies, membranes) will have the 459 same lives as components made from stored raw materials. Although this aspect 460 is difficult to study, some evidence should be provided supporting the use of 461 stored constituents, as well as a plan to evaluate lot-to-lot variance from different 462 critical constituents. 463

The choice of the reagents to be used to measure the performance of the 464 constituent under study (either materials of proven shelf-life or freshly made) 465 needs substantial consideration. 466



Examples of stored constituent: Purified recombinant antigens and 467 monoclonal antibodies stored in aliquots ready for dilution and striping 468 onto RDT membranes or other supports 469

6.2 Stability of control materials 470

Assay specific control materials provided by the manufacturer are to show that an 471 IVD has performed as intended during use. The manufacturer must be able to 472 demonstrate that the loss of signal of a control does not occur at a different rate 473 from the loss of signal from a validated stability testing panel member or from 474 genuine, critical specimens; otherwise a failed IVD might be regarded as still 475 functional. Thus the stability of the control material must accurately reflect the 476 stability of the assay. A control that is more stable than the IVD and other 477 components, or incorrectly set values for the control material, must be avoided 478 [13]. 479

Example: It is frequently seen in dossiers relating to IVDs submitted to 480 the WHO Prequalification Programme that a positive run control will 481 produce a signal of >2.0 optical density (OD) in a freshly manufactured 482 lot, and the IFU will state that an OD > 0.8 for the same control qualifies 483 a run. Thus the IVD could have lost more than half its activity and still 484 appear functional, although some critical specimens are shown in the 485 dossier to have very weak signals on freshly made IVDs. This is not 486 considered appropriate unless data can be provided that demonstrate 487 that the critical specimens will still be detected at the end of shelf life. 488

6.3 Antimicrobial stability and efficacy 489

6.3.1 Rationale 490 IVDs are used in areas which are not necessarily clean and sterile, and 491 antimicrobials are not stable under some circumstances. Bacterial and fungal 492 organisms relevant to the environment of use should be identified in the design 493 input risk assessment, and antimicrobial preservatives should be chosen to avoid 494 contamination of the product. The manufacturer must obtain evidence that the 495 antimicrobial preservative and concentration chosen is stable and effective 496 against the micro-organisms of concern throughout the claimed shelf-life and in-497 use shelf life. 498

6.3.2 Study conditions 499 The studies should reflect expected in-use conditions in opened containers: clean, 500 particle-free laboratories do not usually reflect universal user environment. 501

See Reference [14] Sections <51> ,<61> and <62>; and Reference [15] Appendix XI 502 for suggested methods. Examples of bacterial groups to consider are spore-503 forming bacteria, fungus, indigenous bacteria, bacteria found in the environment 504 of the country of manufacture, as well as use of a negative control. Specific 505 examples outlined in References [14] and [15] include Aspergillus niger, Bacillus 506 subtilis, Candida albicans, Escherichia coli, Salmonella species, Pseudomonas 507 aeruginosa, Clostridium sporogenes and Staphylococcus aureus. 508



6.3.3 Time of testing 509 Antimicrobial preservative effectiveness, as measured by the viable microbial 510 species load present in kit components, should be demonstrated during 511 development, during scale-up, and throughout the shelf-life. 512

Testing for antimicrobial preservative content should normally be performed at 513 release. Under certain circumstances, in-process testing may suffice in lieu of 514 release testing. The acceptance criteria for in-process testing should remain part 515 of the specification [16]. 516

6.4 Desiccant functionality 517

Desiccants affect the stability of the entire IVD. Stability studies must show that 518 the desiccant will support the product over the whole claimed shelf-life within the 519 predetermined extremes of transport, storage and in-use conditions. 520

WHO Note: 521

1) WHO recommends that a self-indicator (a humidity indicator that changes colour 522 upon saturation) be part of the desiccant design. However, WHO strongly 523 recommends against the use of cobalt dichloride, the most commonly used 524 humidity indicator, as it is a carcinogenic substance. 525

2) Sachets are preferable over tablets, since labelling such as “Do not eat” is more 526 visible. There have been anecdotal reports of desiccants in a tablet formulation 527 being mistaken for antimalarial medicine. 528



7 Stability during transport 529

7.1 Rationale 530

Transport stability studies evaluate the tolerance of an IVD to the kinds of 531 environmental conditions (e.g. temperature, humidity, dust) and physical 532 conditions (inversion, vibration, physical handling, stacking) to which it is likely to 533 be subjected in the time between shipping from the manufacturer to its final user. 534 They should provide evidence that there will be no impact on the IVD 535 performance over the whole of its stated shelf-life after recommended 536 transportation methods for the IVD. The manufacturer should assess the potential 537 impact of multiple factors and justify and document whether or not to include 538 them in the evaluation. 539

WHO expects that a transportation challenge should precede the real-time 540 determination of shelf-life This serves to determine that transportation conditions 541 do not reduce the shelf-life of the IVD (see also 5.1) . 542

In some cases it might be acceptable to test the product only over the transport 543 simulation duration, without a subsequent long-term study under normal storage 544 conditions. If that is done, shelf-life must be established under specified storage 545 conditions along with a stringent, evidence-based risk assessment of the 546 probabilities of extreme transport stress affecting the performance at the end of 547 the claimed life (see para. 4.2.3 in Reference [2]). 548

7.2 Challenge conditions 549

Determination of the stability during transportation of an IVD should take into 550 consideration the local routes, transport means and transit used to supply the IVD, 551 usually defined in the design input risk assessment. It is not necessary to test the 552 IVDs to the point where it is no longer usable, but merely to validate the window 553 of transport conditions within which the IVD will retain its claimed performance to 554 the end of its stated shelf-life. However, knowledge of the possible limitations of 555 an IVD and at what point the IVD becomes unusable is useful to a manufacturer 556 when trouble shooting post-market problems. WHO expects the manufacturer to 557 consider that the product might continue to be subjected to sub-optimal storage 558 conditions at the end-user. 559

Example: While a static challenge of 45°C for 3 days might represent 560 conditions seen during actual transport of a IVD, a more stringent 561 challenge of cyclical higher and low temperatures (including freezing) for 562 a longer period of time and under vibration might better cover a ‘worst 563 case scenario’ of shipment, storage and subsequent transportation to 564 the end-user. 565

7.3 Number of lots 566

Where transport stability studies are incorporated into studies to establish shelf 567 life, as recommended in this guidance, a minimum of three lots of the IVD should 568 be used. For transport studies alone, at least one lot of the IVD can be used, 569 however, as with shelf life studies, more may be required depending on lot 570 variability (see 9.1). 571



7.4 Multiple stress test sequences 572

Mere proof of performance after actual shipment is generally insufficient 573 evidence of stability under all conditions and with the hazards of delays. Multiple 574 stress test sequences are typically needed to address the range of transport 575 conditions used for global product delivery, including some extreme conditions 576 expected to be evaluated according to relevant guidance [11]. 577

Appropriate sequences may be developed on the basis of data from actual 578 product transport studies. Testing multiple stress sequences allow a manufacturer 579 to identify the most cost- and/or resource effective transport conditions from a 580 set of alternatives while ensuring adequate product stability protection 581 (Reference [2] para 4.2.3). 582

WHO requirement: Environmental conditions investigated as part of a 583 stability study must reflect those likely to be encountered in resource-584 limited Member States. Temperatures at some airport tarmacs in Sub-585 Saharan Africa can exceed 40°C while temperatures encountered during 586 air transport fall below 0°C. Significant delays can be encountered at all 587 times and especially during wet season transport to remote health 588 centres. 589

7.5 Physical conditions 590

Physical handing can be both manual and mechanical. The relevant user and 591 commercial factors should be identified as part of the design input risk 592 assessment and the packaging and shipping methods developed accordingly. 593 Reference [11] defines a number of factors to be considered, and their evaluation: 594 drop, impact, compression, vibration, repetitive shock, longitudinal shock, cyclic 595 exposure, vacuum, impact, inversion; along with the size, weight, and composition 596 of the packaging. 597

7.6 Simulated versus actual challenge 598

An actual shipping challenge can be used to verify the conditions found in the 599 simulated transportation challenges. However it should only replace a simulated 600 shipping challenge when there is an appropriate risk evaluation and with 601 experience and data already actively collected from similar products and 602 documented in detail (for example it is insufficient to note “no complaints”). 603

In the R&D phase, actual data from shipping can be used to define the conditions 604 needed for an appropriate simulation of extremes. However in the post-605 production phase actual shipping challenges often do not explore the full range of 606 shipping conditions that could be encountered, including extreme values. 607



8 In-use stability studies 608

8.1 Rationale 609

In-use stability of an IVD is the period of time over which components retain 610 adequate performance, after transport to the users, once they are opened, 611 reconstituted and/or diluted and exposed to the environmental conditions in 612 which they will be used. 613

If a range of conditions for use is stated in the IFU (e.g. use at 15–40°C) evidence 614 should be provided to prove the stability over that range with all the specimen 615 types (e.g. serum, whole blood, oral fluid) claimed. It is considered best practice 616 that the manufacturer extends the stability range by 5°C at the lower and upper 617 end of the proposed acceptable range on the labelling for all components to 618 ensure that that the claimed stability ranges is acceptable. 619

8.2 Conditions of use 620

Determination of the in-use stability of an IVD and/or its components should 621 reflect routine conditions of use of the IVD. Freeze-thaw stability should be 622 considered to address reagents which are exposed to multiple freeze-thaw during 623 use. 624

WHO recommendation: In-use stability studies must take into account 625 environmental conditions and usage conditions encountered by WHO 626 users and Member States, such as exposure to extreme temperatures, 627 humidity, dust, light and micro-organisms. 628

8.3 Multiple in-use stability claims 629

Depending on the way in which the IVD is used it may be necessary to have 630 several in-use stability claims. In situations where multiple stability claims are 631 made, a manufacturer must provide evidence from testing that investigates 632 routine use supporting each of the claims. 633

Examples: 634

1) A reagent may have a stated period of stability once it has been placed on-board 635 an instrument and another period of stability once it is in active use (i.e. during 636 actual use/testing). 637

2) Multiple use reagents (e.g. buffers) may repeatedly be exposed to high 638 temperatures during the day while in use and exposed to lower temperatures 639 when not in use and stored in the refrigerator. The actual use of the multiple use 640 reagent – squeezing of bottles, exposure of the lid and tip to working surfaces, 641 hands, exposure to dust and light – also affect stability. Stability studies should 642 take into account all of these factors. 643



9 Production lots used in stability studies 644

Comments in this section apply equally to all types of studies described in this 645 guidance. 646

9.1 Considering variability 647

As noted in 11.3_Consider_variability, planning for stability studies must take into 648 consideration all possible sources of variation within and between manufactured 649 lots. For most IVDs it is likely that differences between batches of the biological 650 reagents will cause the most variance. Factors to consider include apparently 651 minor, technically-uncontrollable differences in culture and purification for 652 recombinant antigens and antibodies; synthesis and purification for primers, 653 probes and peptides; undocumented production changes of an outsourced buffer 654 component and the lot of nitrocellulose membrane used in lateral-flow IVDs. 655

At a minimum, lots chosen for stability studies should be different in the critical 656 constituents, e.g. different purification and/or culture batches for all recombinant 657 antigens and monoclonal antibodies. If pilot or small scale lots are chosen, special 658 attention must be paid to the potential for variability (see also 11.3). However, 659 the sources of variation will depend on the particular process, product and 660 component, and should be identified during product development risk analyses. 661

Use of different batches of critical components ensures that the stability evidence 662 obtained is more likely to be representative of long-term manufacture. Any 663 variability found can be taken into consideration when assessing the outcome of 664 the studies against the design input requirements and when making claims. This 665 minimizes user problems and hence complaints. 666

9.2 Testing the final configuration 667

Shelf-life, in-use and transport stability must be determined for the finalized, 668 approved product in terms of: 669 • manufacturing specifications; 670 • release-to-market QA criteria; 671 • packaging and labelling; and 672 • validated manufacturing scale on qualified manufacturing equipment. 673 Testing methods should be as included in the IFU of the commercialized IVD. 674

WHO prequalification requirements: It is important that it can be 675 established that the stability studies were conducted on the IVD as 676 submitted to WHO for prequalification. Even changes perceived as small 677 (e.g. change in production scale, bulk container materials, supplier of a 678 critical biological, change in vial stopper) can have unexpected effects on 679 stability and other performance characteristics. After such changes, a 680 stability plan and study is needed again. Manufacturers should have 681 change control procedures in place compliant with ISO 13485:2003 [12]. 682 Stability studies undertaken in the R&D phase of the product lifecycle 683 are important to understand how to design the product so it will meet 684 the final stability requirements in the input documentation. However, 685 these studies are not sufficient for submission to WHO since they might 686 not reflect the final design and manufacture of the IVD. 687



9.2.1 Exceptions 688 If any of the above criteria are not met (for example if “pilot lots” or small scale 689 lots are used, or if the IFU is not finalized), strong evidence must be provided that 690 the evaluated materials will perform exactly the same as the final commercial 691 product. 692

WHO prequalification requirements: In some exceptional circumstances, 693 where it is not possible to sample from actual production lots, samples 694 from pre-production or development lots might be used. If this is the case, 695 manufacturers should justify why production lots were not used, and 696 they should provide robust evidence that the lots chosen are expected to 697 behave identically to the production lots. Data concerning lot-to-lot 698 variability must still be submitted. Although WHO will consider the 699 available evidence on its merits, this preliminary information must be 700 followed by stability claims conducted on production lots. A post-701 prequalification commitment may be required to amend this situation 702 when the manufacturer is able to produce fully qualified production lots. 703

9.3 Number of lots required for testing 704

Existing guidance [1,2] recommends that three product lots at a minimum must 705 be used to verify shelf-life; in-use claims require testing on a minimum of one lot. 706 The actual minimum number of lots to be used should be determined by a 707 stringent risk assessment based on evidence of variability obtained during R&D, 708 (see section 9.1). However, the minimum will never be less than three lots for 709 shelf life verification. 710

Note from WHO prequalification experience: It is not acceptable to 711 sample IVDs from a single production lot but label them so that they 712 appear to have been taken from three separately manufactured 713 production lots. This is true for all performance evaluation and 714 regulatory submission purposes. This aspect will be investigated during 715 an onsite inspection by WHO. Non-compliance with this requirement 716 may result in a critical non-conformity grading. 717

9.4 Components of lots required for testing 718

Existing guidance [1,2] requires that stability work is performed using materials in 719 their final packaging, with intended labelling. If there is more than one variant of 720 the IVD (e.g. pack size differences, CE marked and non-CE marked) any potential 721 effects on performance, including stability, must be assessed. 722

In particular, if different reagent-container sizes are used in packs intended for 723 different numbers of use, stability evidence should be obtained on all variants, 724 even if the contents of the containers are identical. 725

Once component shelf-lives are assigned use relatively fresh components and 726 components which have progressed into their assigned shelf-life in the different 727 production lots used in the establishment of the product shelf-life. 728



10 Stability plan 729

Stability studies should be well designed, scientifically sound, well implemented, 730 well recorded and able to deliver meaningful conclusions about IVD performance. 731 This will minimize the time and resources taken by the manufacturer to generate 732 appropriate evidence and by the regulatory authority to assess it. 733

It is good practice to prepare, within the mechanisms of a quality management 734 system (QMS), a plan for the investigation of each aspect of IVD stability. A well-735 developed study plan, with clearly defined objectives, responsibilities and 736 pass/fail criteria should be developed, reviewed and internally approved in 737 advance of testing. The plan should be associated with the design input 738 requirements. 739

It is essential that the study plan takes into account the intended use of the 740 product to ensure that these elements are covered within the stability studies. 741 The results of the stability studies support the claims in the instructions for use. 742

Careful forward planning will make a significant contribution to ensuring that 743 sufficient resources are made available, effective experiments are performed and 744 both experimental results and associated documentation are recorded in an 745 appropriate manner. 746

10.1 Responsibilities 747

The study plan should outline responsibilities and applicable training for said 748 responsibilities of all staff involved in the study. The R&D department is usually 749 responsible for set up of the study and testing of newly developed IVDs, 750 monitoring, and any equipment validation if required, and for the documentation 751 of the testing plan and sample selection. 752

The R&D department should nominate a responsible person for investigating 753 failures. The QA department should nominate a responsible person for conducting 754 risk assessments if the IVD fails to meet the requirements of the design inputs. 755 Performance evaluation is not to characterize an IVD but to show that it meets (or 756 exceeds) predetermined qualities. 757

10.2 Preparing the testing plan 758

A complete, detailed description should be prepared that fully documents 759 everything to be done and the expected outcomes. Authorization of the plan 760 should be obtained internally in advance of starting work. The plan should include 761 the following details. 762

• Qualification and training of technical staff performing the work 763 • Biohazard issues identified with reagents 764 • The instrumentation, including storage facilities or rooms, validation, 765

calibration, monitoring, servicing 766 • The batch numbers of kits to be used with justification for any 767

manufacturing anomalies or excursions from documented procedures 768 • The expected life of the kit from the input documentation 769 • Any proposal, with justification to launch a kit with a life based on 770

accelerated data, or to launch with a shorter life than in the input 771



documentation while awaiting the conclusion of real-time testing 772 documented. 773

• The documented nature and extent of in-use testing 774 • The justification for the choice of lots and components taking into account 775

lot-to-lot variation and the critical characteristics 776 • The number of units (cassettes, bottles, tablets, etc.) of each component 777

to be collected and stored under each condition 778 • The nature of the stability testing panel to be used, justifying each panel 779

member’s inclusion and defining the volume and characterization of the 780 bulk specimen to be used and the aliquot size and number to be stored 781 for the testing (See section 10.5) 782

• The expected criteria for each stability testing panel member at the 783 beginning and end of the product’s proposed shelf-life 784

• The statistical methods to be used for data analysis 785 • Graphs (paper or electronic) to visualize the performance of each stability 786

testing panel member over the course of testing 787 • Methods of approval and justification of any deviations from the plan 788

10.3 Product storage 789

A sufficient number of product components from the identified lots should be 790 reserved and stored separately to ensure that the study will be completed with 791 identified products. Sufficient volumes should be retained to allow for the 792 predetermined invalid rate. 793

10.4 Documentation 794

The plan should make reference to a study report which will be used to 795 summarize interim, and ultimately, final study findings and conclusions. The study 796 plan, the testing protocol and study report and all associated documentation 797 (worksheets, etc.) should be controlled within the manufacturer’s QMS. At the 798 end of the study, the manufacturer should be able to confirm that design input 799 requirements have been met. 800

Any changes in method must be recorded and undergo risk assessment. It should 801 refer to the development of a detailed and valid testing protocol which includes 802 all information and material relevant to testing. 803

10.5 Statistical methods 804

Statistical methods are used to support stability claims by providing estimates of 805 the probability of results being as stated. For example: prior to the stability 806 studies on an EIA it has been documented that if a stability testing panel member 807 has at least a particular optical density (OD) then that device will meet a particular 808 claim. Given the results of the stability study using that stability testing panel 809 member and showing the variability within and between lots of the IVD, the 810 probability of future similar production of the device meeting claims at the 811 assigned life can be estimated. The derivation of valid criteria and the probability 812 of maintenance of all claims can be estimated by appropriate statistical methods. 813

There is a wealth of information on the statistical methods used in R&D of IVD 814 from both ISO [17, 18, 19] and CLSI [2, 20, 21, 22]. Most of these methods apply 815



only to quantitative assays but information on statistic methods for qualitative 816 assays is available also in Reference [23]. 817

A fundamental problem is that of how many replicates should be used at each 818 time point and from how many different production lots to produce acceptable 819 overall probability estimates of the likelihood of all future production of similar 820 devices and lots meeting claims (and hence user input requirements) at the end of 821 the assigned life. There are two aspects to this – what is “acceptable” and “how 822 many replicates?” “Acceptability” is a decision critical to quality and must be 823 decided in advance from the user requirements – for example 80% confidence 824 that 95% of all lots will meet the claims. This is in fact a tolerance interval as 825 described in ISO 16269-6:2014 [19]. “How many replicates” can then be derived 826 from the tolerance interval required but advice from a professional statistician is 827 strongly advised – after defining the quality critical requirement but before 828 beginning any experimental work. 829

The statistical methods to be used will be documented in the plans and protocols 830 of any stability study and consideration given to treatment of unexpected and 831 atypical results. In general all results must be used unless there is a documented 832 physical reason (e.g. known operator error, too little volume, incorrect timing, use 833 of an unqualified instrument such as lacking maintenance or calibration) why a 834 result can be ignored – but even then that result must be recorded and included 835 in the report of the stability work. 836

10.6 Stability testing protocol 837

As part of an approved study plan for the determination of IVD stability, a detailed 838 testing protocol should be prepared (examples of stability protocols are provided 839 in Appendix 1: 840 Example stability protocols) including the following as a minimum, as appropriate. 841

• QMS identifiers (e.g. experiment name, document references, etc.) that 842 allow traceability to both the overarching study plan and to subsequently 843 generated records/documents such as result worksheets 844

• The name(s) of operator(s) 845 • The dates and times when the experiment was performed 846 • Signatures of the operator and supervisor(s) 847 • The objectives of the study (i.e. determination of shelf-life, determination 848

of in-use stability of a component, etc.) 849 • The name and lot number of the IVD and/or components being 850

investigated 851 • How the components will be sampled from the production department 852 • Stability testing panel members and their characterization to be used, 853

including valid test methods which reflect the IFU claims 854 • The experimental method that will be used for testing. This must follow 855

the finalized testing method from the IFU. It must describe clearly how 856 the experiment was performed in terms of: 857 − required storage and/or challenge conditions; 858 − the duration of storage/challenge; 859 − the schedule of testing intervals (see Reference [2] section 4.3); 860 − the stability testing panel; and 861 − the numbers of replicate tests performed for each stability testing 862

panel member. 863



• How and where results are to be recorded 864 • Acceptance criteria 865 • How aberrant, discordant or invalid results will be dealt with 866 • How storage/challenge conditions are to be applied 867

Example: For determination of stability during transportation it should be 868 made clear that each IVD will be subjected to a sequence of stated 869 temperatures. 870

• How actual storage/challenge conditions are recorded 871 Example: Recording of temperature not as “room temperature” but as an 872 actual numerical value obtained from calibrated instrumentation 873 874

Note: It can be unclear to a regulatory or WHO reviewer from a general 875 statement such as “… Sample buffer was stored at the required 876 temperature and tested each month…” whether (1) the bottles of sample 877 buffer were stored open at the required temperature for the entire testing 878 period, or (2) the bottles were stored capped and refrigerated, and only 879 reopened briefly at the required temperature at each schedule test point. 880

881

10.7 Reading and recording results 882

10.7.1 Avoiding reader bias 883 It is good practice to use approaches to make the reading more objective, such as 884 a scoring system. For IVDs where a subjective element forms part of the result, e.g. 885 reading the intensity of an RDT band within a specified time frame, the results 886 should always be reviewed by a first and second reader to avoid operator bias. 887 Both readers must be blinded to the expected results; the second reader must be 888 blinded to the first reader’s results. If a validated band intensity scoring tool is to 889 be included in the final RDT kit, this should be used to record results. 890

10.7.2 Recording actual individual results 891 The results of a test, not only the test interpretation, should be recorded. An 892 interpretation on its own has insufficient resolving power to allow degradation of 893 a signal over time to be observed. 894

Some IVDs, e.g. line-blots, may require particular band patterns to allow an 895 interpretation to be reached, and several different patterns may yield the same 896 final result. Recording only the final interpretation of a test specimen may cause 897 the failure of particular bands to go unnoticed while allowing the IVD to otherwise 898 “pass”. Photographic records of qualitative tests are recommended, as 899 appropriate. 900

This is particularly important when testing a panel of like specimens, e.g. “20 HIV 901 antibody positive specimens” for which the acceptance criterion is “all 20 902 specimens must be positive”. It is not sufficient to simply record “all 20 positive” 903 or “pass” without first recording the individual test result directly from the IVD for 904 each specimen in the panel. 905

Example 1: For most enzyme-linked immunoassays (EIAs) if the sample-906 to-cut-off ratio is > 1 then the result is interpreted as “positive” or 907 “reactive”. In this case three pieces of information should be recorded: (1) 908



the numerical value of the assay sample-to-cut-off ratio, (2) the 909 numerical value of the signal for the specimen and (3) the final 910 interpretation. 911

Example 2: Some rapid diagnostic tests (RDTs) may stipulate that the 912 strength of test band is not correlated with the strength of antibody titre. 913 Nevertheless, the following should be recorded: (1) the intensity of 914 observed patterns according to a predetermined, validated intensity 915 scoring system with as fine a gradation as possible, and (2) the final 916 result interpretation. 917

Example 3: A qualitative NAT assay may report “positive” and “negative” 918 for a particular analyte, but the underlying decisional parameter is often 919 quantitative (e.g., a PCR signal-based cycle number). The quantitative 920 parameter should be recorded. 921

922

10.7.3 Retention of records 923 WHO encourages retention of photographic records, machine printouts, 924 electronic data or physical retention of membranes from opened cassettes, as 925 appropriate. Records should be retained for the period of time equivalent to the 926 commercial lifetime of the IVD but not less than two years. (Modified from 927 Reference [12] para 4.2.4) 928

10.8 Degradation vs deterioration 929

Testing at more than two time points can be important to avoid confusion 930 between imprecision and stability. For example, if the end testing shows 10% 931 decrease, one may not judge if the difference was due to imprecision or 932 degradation. If tested one or more times in between are used, fluctuation caused 933 by imprecision can be distinguished from drift due to instability. This can be 934 ameliorated by increasing the number of replicates and runs. 935

All studies should support precisely defined periods of in-use stability claims. 936

Example: An RDT test cassette – may be labelled “Use immediately on 937 opening”. In such cases it is still necessary to determine the interval (one 938 hour, one day, etc.) over which the IVD performance remains stable after 939 the component is opened. 940

10.9 Testing schedule 941

Testing intervals should be selected to detect any trending activity over the 942 testing period. Concurrent testing of separate types of components may be 943 approached with different intervals. For example, it may be appropriate to test an 944 IVD test cassette against a stability testing panel on a monthly or quarterly basis. 945

10.9.1 Acceptance criteria for results 946 The acceptance criteria to establish what is acceptable or not acceptable should 947 be defined according to the stability testing panel criteria for both qualitative and 948 quantitative test methods. Results from failed (invalid) test runs must not be used 949 in the determination of the stability claim. However the invalid results should also 950 be recorded. 951



11 Stability report 952

11.1 General 953

After testing has been completed, the findings should be summarized in a stability 954 study report. The report should clearly identify the IVD that was tested, the 955 objectives of the study, the conditions under which the IVD was tested and 956 conclusions that were drawn from findings. The report should be traceable to the 957 study plan, testing protocol and user needs. It should make clear references to 958 other supporting documentation (e.g. result worksheets). 959

11.2 Link to claims 960

The results and conclusions of stability studies presented in the study report must 961 support the claims of IVD stability reported in the IFU and elsewhere in the 962 dossier. 963

11.3 Consider variability 964

An overall stability claim (whether for shelf-life, in-use stability, or stability during 965 transportation) must be based on the expected stability when taking into account 966 inter-lot variability. 967

Example: The manufacturer should evaluate the variability between the 968 different lots studied (see 9.1) and assume that any differences in shelf-969 life are inherent to the manufacturing process. The claimed life should be 970 calculated so that a known and stated proportion of all lots 971 (usually >95%) will meet the claimed shelf-life. Frequently more than 972 three lots are needed to obtain a realistic idea of the variability of the 973 results. 974

11.4 IVD stability versus component stability 975

A claim of stability for an IVD as a whole must not exceed any individual 976 component stability. 977

Example: For an IVD claimed to detect HIV-1 and HIV-2 antibodies – if 978 detection of HIV-1 antibodies is stable to 24 months but that of HIV-2 to 979 only 18 months, then the shelf-life must be based on the shorter time. 980



12 Changes to a Prequalified IVD 981

12.1 Dealing with change 982

Any critical or major modification to a prequalified IVD or to its process of 983 manufacturing will require provision of direct evidence of stability. An appropriate 984 risk analysis and an accelerated stability study comparing the original product and 985 the modified product for usability, performance and lot-to-lot variation may serve 986 to assess the impact of the changes to a product formulation or manufacture. It 987 would be necessary to validate the stability of the modified IVD in at least one lot 988 of the IVD (subject to risk analysis) in order to demonstrate equivalence between 989 the original and modified IVDs. More lots may be appropriate depending on the 990 product nature, variability of components and failure risk . (Reference [2] section 991 7.1.2). WHO expects results of accelerated testing to be confirmed by real-time 992 studies. 993

If there are different presentations, the stability of each one must be assured (see 994 also 9.4). 995

The following examples seek to illustrate the scope for considering the 996 performance evidence from one IVD as support for performance in another: 997

Examples: 998

1) For an HIV RDT which uses an identical cassette and physical components of a 999 manufacturer’s existing, fully validated HCV RDT, the reagent formulations are 1000 different (antigen/antibodies, buffers, conjugates, etc.). Evidence of stability of 1001 the HCV RDT would not suffice for the HIV RDT. 1002 Even if the manufacturer claims that both IVDs have been sold in a number of 1003 countries for several years and no adverse feedback has been reported, this 1004 would not constitute evidence in support of the stability of either IVD. 1005

2) For an HIV RDT that has been fully validated for detection of HIV-1 antibodies; a 1006 new product is developed which includes detection of HIV-2 antibodies. The 1007 stability of any sample buffers that are identical between the two IVDs would 1008 probably not need to be validated. However, other components (conjugates, 1009 antigens, antibodies) that are different between the two IVDs would need to be 1010 tested; it would not be sufficient to assume that HIV-1 reagents will have the 1011 same stability in the new IVD. A modification of this nature is likely to require 1012 substantial validation of stability. 1013

3) An HIV RDT IVD previously intended for testing serum/plasma has added to it a 1014 claim for detection of HIV-1 in whole blood. The only substantive design change 1015 associated with the new claim is the addition of a small pad of some suitable 1016 material near the sample port which acts as filter for whole blood specimens. 1017 Depending on the nature of the material it may be reasonable to argue that the 1018 material would not be expected to age; that it is not, in any practical sense, 1019 chemically labile. Consequently, shelf-life and in-use stability may not necessarily 1020 need to be retested in full. However, stability during transportation may need to 1021 be determined to provide confidence that the modification is able to withstand 1022 likely shipping conditions (e.g. that the extra square of filter paper doesn’t 1023 dislodge when packages are jostled and bumped in transit). 1024 1025



4) Based on an HIV RDT that has been fully validated for detection of HIV-1 1026 antibodies, a new IVD is developed which includes detection of antibodies to 1027 Treponema pallidum (TP). Detection of TP specific antibodies occurs on a 1028 completely separate membrane (and associated architecture) to that of HIV 1029 antibody detection. Additional handling steps may have an impact on the 1030 stability of the HIV-1 antibodies and it may be required to retest. It may be 1031 necessary to review evidence of stability during transportation to ensure that 1032 new components are not affected by transport (for example a new packaging 1033 concept is used). 1034 If a new machine is used for striping of the HIV-1/TP IVD, 1035 validation of the new machine (installation qualification, operational 1036 qualification and performance qualification) would be required to show that the 1037 stability studies are still valid. 1038 If the IVD is designed in a way that HIV and TP detection occurs 1039 either on the same membrane and/or using most of the same architecture (and 1040 assuming that sample buffers are identical between IVDs) it is likely that this new 1041 IVD would need to be fully validated. 1042

1043

It should be noted that these observations pertain specifically to IVD stability. 1044 Other aspects of IVD performance should still be validated as appropriate. 1045

1046



13 Authors and acknowledgements 1047

The document TGS-2 Establishing stability of an in vitro diagnostic for WHO 1048 prequalification was prepared by Julian Duncan (consultant, London, UK), Sue 1049 Best, Susie Jane Braniff, Mark Lanigan (National Serology Reference Laboratory, 1050 Australia), Deirdre Healy (WHO/HIS/EMP) and Robyn Meurant (WHO/HIS/EMP). 1051 WHO wishes to thank Sally Hojvat (consultant, USA), Luc Kestens (ITM), Debbie 1052 Lepine (Health Canada), Luann Ochs, CLSI and members of the CLSI Consensus 1053 Committe, the ISO TC212 WG3 committee and Monika Zweygarth (editor) for 1054 their contributions. This document was developed as part of the Bill and Melinda 1055 Gates Foundation Umbrella grant. This document was produced under the 1056 coordination and supervision of Robyn Meurant and Irena Prat (WHO/HIS/EMP), 1057 Geneva, Switzerland. 1058

The draft guidance has been posted on the WHO website for public consultation 1059 from the 14 December 2015 to 31 January 2016. 1060



14 References 1061

1062

1 ISO 23640:2011. In vitro diagnostic medical IVDs - Evaluation of stability of in vitro diagnostic reagents. Geneva, Switzerland: International Organization for Standardization; 2011.

2 CLSI. Evaluation of Stability of In Vitro Diagnostic Reagents; Approved Guideline. CLSI document EP25-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2009.

3 ICH Harmonised Tripartite Guideline Specifications: Test Procedures and Acceptance Criteria for Biotechnological/Biological Products Q6B. International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use. Current Step 4 version. 10 March 1999.

4 ISO 17511:2003. In vitro diagnostic medical IVDs – Measurement of quantities in biological samples – Metrological traceability of values assigned to calibrators and control materials. Geneva, Switzerland: International Organization for Standardization; 2003.

5 ISO 18113-1:2009. In vitro diagnostic medical IVDs – Information supplied by the manufacturer (labelling) – Part 1: Terms, definitions and general requirements. Geneva, Switzerland: International Organization for Standardization; 2009.

6 United States CFR - Code of Federal Regulations Title 21. Sec. 820.3 Definitions.

7 ISO/IEC Guide 99:2007. International vocabulary of metrology -- Basic and general concepts and associated terms (VIM). Geneva, Switzerland: International Organization for Standardization; 1993.

8 ISO 9000:2005. Quality management systems – Fundamentals and vocabulary. Geneva, Switzerland: International Organization for Standardization; 2005.

9 GHTF/SC/N4:2012 (Edition 2). Glossary and Definitions of Terms Used in GHTF Documents. Global Harmonization Task Force (GHTF) Steering Committee; 2012.

10 ISO 14971:2007. Medical IVDs – Application of risk management to medical IVDs. International Organization for Standardization; Geneva, Switzerland: 2007.

11 American Society for Testing and Materials (ASTM). ASTM D4169-14. Standard Practice for Performance Testing of Shipping Containers and Systems. ASTM International, West Conshohocken, PA; 2014.

12 ISO 13485:2003. Medical IVDs – Quality management systems – Requirements for regulatory purposes. Geneva, Switzerland: International Organization for Standardization; 2003.


https://www.iso.org/obp/ui/%23iso:std:iso:23640:ed-1:v1:en

http://shop.clsi.org/method-evaluation-documents/EP25.html

http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Quality/Q6B/Step4/Q6B_Guideline.pdf

http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Quality/Q6B/Step4/Q6B_Guideline.pdf

http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=30716

http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=30716

https://www.iso.org/obp/ui/%23iso:std:iso:18113:-1:ed-1:v1:en

https://www.iso.org/obp/ui/%23iso:std:iso:18113:-1:ed-1:v1:en

http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/cfrsearch.cfm?fr=820.3

http://www.iso.org/iso/home/store/catalogue_ics/catalogue_detail_ics.htm?csnumber=45324

http://www.iso.org/iso/home/store/catalogue_ics/catalogue_detail_ics.htm?csnumber=45324

http://www.iso.org/iso/home/store/catalogue_tc/catalogue_detail.htm?csnumber=45481

http://www.imdrf.org/docs/ghtf/final/steering-committee/procedural-docs/ghtf-sc-n4-2012-definitions-of-terms-121109.pdf

https://www.iso.org/obp/ui/%23iso:std:iso:14971:ed-2:v2:en

http://www.astm.org/Standards/D4169.htm

http://www.astm.org/Standards/D4169.htm

http://www.iso.org/iso/catalogue_detail?csnumber=36786


13 ISO 15198:2004. Clinical laboratory medicine – In vitro diagnostic medical IVDs – Validation of user quality control procedures by the manufacturer. Geneva, Switzerland: International Organization for Standardization; 2004.

14 United States Pharmacopeia and National Formulary (USP 31-NF 26). Rockville, MD, United States: Pharmacopeia Convention; 2008.

15 Pharmacopoeia of the People’s Republic of China. English edition. Beijing, China: The State Pharmacopoeia Commission of the People's Republic of China; 2000.

16 ICH Harmonised Tripartite Guideline. Specifications: Test procedures and acceptance criteria for new drug substances and new drug products: chemical substances. Q6A. International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use; 1999.

17 ISO 5725-1,2,3,4,6:1994, ISO 5725-5:1998 Accuracy (trueness and precision) of measurement methods and results- Parts 1-6. Geneva, Switzerland: International Organization for Standardization; 1994 and 1998

18 ISO 3534-1,2:2006, ISO 3534-3:2013. Statistics -- Vocabulary and symbols – Part 1-3. Geneva, Switzerland: International Organization for Standardization; 2006 and 2013

19 ISO 16269-4:2010, ISO 16269-6:2014, ISO 16269-7:2001, ISO 16269-8:2004. Statistical interpretation of data. Geneva, Switzerland: International Organization for Standardization; 2001, 2004, 2010, 2014.

20 CLSI. Evaluation of the Linearity of Quantitative Measurement Procedures: A Statistical Approach; Approved Guideline. CLSI document EP06-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2003.

21 CLSI. Interference Testing in Clinical Chemistry; Approved Guideline - Second Edition. CLSI document EP07-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2005.

22 CLSI. Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures; Approved Guideline - Second Edition. CLSI document EP17-A2. Wayne, PA: Clinical and Laboratory Standards Institute; 2012.

23 Valcárcel, M., Cárdenas, S., Barceló, D. et al. Metrology of qualitative chemical analysis, KI-NA-20-605-EN-C, ISBN 92-894-5194-7; 2002 Available free of charge from: http://bookshop.europa.eu/en/metrology-of-qualitative-chemical-analysis-pbKINA20605/

24 CLSI. Evaluation of Precision of Quantitative Measurement Procedures; Approved Guideline—Third Edition. CLSI document EP05-A3. Wayne, PA: Clinical and Laboratory Standards Institute; 2014.




http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Quality/Q6A/Step4/Q6Astep4.pdf

http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Quality/Q6A/Step4/Q6Astep4.pdf

http://www.iso.org/iso/home/search.htm?qt=5725&sort=rel&type=simple&published=on










http://bookshop.europa.eu/en/metrology-of-qualitative-chemical-analysis-pbKINA20605/

http://bookshop.europa.eu/en/metrology-of-qualitative-chemical-analysis-pbKINA20605/



Appendix 1: 1063

Example stability protocols 1064

1065

This appendix contains examples for a wholly fictitious IVD, illustrating the kinds 1066 of experimental design to determine the following: 1067

1. Stability of whole kit during transport 1068 2. Stability of whole kits during shelf-life, and 1069 3. In-use stability of whole kits including reagents 1070

The information provided in these examples should not be taken as a checklist of 1071 sufficient conditions, but should be used as a guide on possible approaches to 1072 generate evidence of a standard sufficient to satisfy the requirements of the WHO 1073 Prequalification Programme. Additional examples can be found in the WHO 1074 sample CD4 dossier available on the WHO prequalification website. 1075

It is recommended that transportation stress studies are undertaken prior to the 1076 shelf-life studies. 1077

Description of fictitious IVD 1078

The fictitious IVD used for the purpose of the examples is a RDT for the detection 1079 of antibodies to HIV-1, HIV-2 and Treponema pallidum in serum, plasma and 1080 whole blood. It is recommended that the kit is stored at 8–40°C, but components 1081 of the kit must be used at 15–30°C. The product is supplied as a kit with each test 1082 cassette sealed in a foil pouch (with desiccant). The pouch must be brought to 15–1083 30°C. Once opened, it is recommended that the cassette is used immediately. 1084

The IVD includes a bottle of specimen buffer/diluent for use with all three 1085 specimen types. The specimen buffer is expected to have similar stability as the 1086 test cassette in its unopened form. The stability of the opened bottle of specimen 1087 buffer is determined below (see Example 3: In-use stability protocol). 1088

The manufacturer of this product proposes to determine the stability of its product 1089 and has written a stability plan. As part of this plan a preliminary determination of 1090 accelerated stability has been conducted at several extremes of temperature and 1091 suggests that the IVD would be stable to an equivalent of 12 months following 1092 manufacture. The plan now calls for the development of real-time stability 1093 protocols that will form the basis of subsequent testing of the IVD. 1094

Preliminary work has shown that the variability between lots is minimal so that 1095 three independent lots (no critical constituents in common) will suffice to enable a 1096 reasonable estimation of shelf-life taking lot variation into account. 1097


http://www.who.int/diagnostics_laboratory/evaluations/140314_simu_poc_cd4_dossier_web.pdf?ua=1



Example 1: Evaluation of stability during transportation 1098

Objective 1099 To determine the stability during transportation of the HIV RDT in real-time using 1100 simulated shipping conditions and to generate stressed components to be used in 1101 real-time shelf-life studies as proposed in Stability Study Plan XZY00001. 1102

Preparation 1103 Acquire sufficient numbers of kits from three independent production lots using a 1104 predetermined sampling protocol (e.g. random, first X kits in first box, every 100th 1105 kit, etc.). Allow at least 10% for unexpected requirements and re-testing 1106

Note 1: To provide security against unforeseen events, duplicate tests should be 1107 performed as a minimum. Testing in triplicate as a minimum provides a level of 1108 statistical confidence in the observed test result. 1109

Testing will be conducted at 0, 3, 6, 9, 12 and 13 months. 1110

Note 2: Testing beyond 13 months will allow an understanding of when, in real-1111 time, the IVD is likely to ‘fail’ and may allow an extension of the proposed shelf-life. 1112

Note 3: For determination of shelf-life a fresh bottle of specimen buffer must be 1113 opened at each testing point – although there may be circumstances in which 1114 multiple sampling could be taken from the same bottle after it has been opened. 1115 Acquire sufficient volume of each stability testing panel member for the duration 1116 of the testing schedule. 1117

The protocol for these studies specifies the number of devices to be picked, the 1118 statistical sampling plan to be used and the required stability testing panel 1119 members and their volumes. 1120

In Worksheet XYZ00001 record the following: 1121

• The lot numbers from which kits were sampled 1122 • The number of kits sampled from each lot 1123 • Details (including manufacturing/lot information) for each of the kit 1124

components that will be tested as part of this protocol: 1125 Test cassette:.. 1126 Bottle of Sample Buffer:… 1127

The product kits chosen to be tested are in their final packaging including all 1128 labelling. 1129

The IVDs are stored so that the reagents are in contact with all elements of the 1130 packaging (e.g. the bottles in the product kits are stored horizontal lying flat on 1131 their sides). 1132

Kits will be divided into two groups. One group will be stored at 42±2°C, the other 1133 at 4±2°C. Kits from each group will then be subjected to the following conditions. 1134

Testing schedule: for transport simulation 1135 Condition 1, Temperature and humidity sequence: all kits will be taken through a 1136 temperature and humidity sequence consisting of: 1137

Ambient humidity (X% RH) 1138 Put at IFU storage temperature for 24±4 hours followed by 1139 30 ± 5°C for 24±4 hours followed by 1140



45 ± 5°C for 24±4 hours, followed by 1141 8 ± 5°C for 24±4 hours , followed by 1142 IFU storage temperature for 24±4 hours 1143

Followed by 1144

Desert humidity (30% RH) 1145 Put at IFU storage temperature for 24±4 hours followed by 1146 30 ± 5°C for 24±4 hours), followed by 1147 45 ± 5°C for 24±4 hours, followed by 1148 8 ± 5°C for 24±4 hours, followed by 1149 IFU storage temperature for 24±4 hours 1150

Followed by 1151

Tropical humidity (85% RH) 1152 Put at IFU storage temperature for 24±4 hours followed by 1153 30 ± 5°C for 24±4 hours), followed by 1154 45 ± 5°C for 72±4 hours, followed by 1155 8 ± 5°C for 24±4 hours, followed by 1156 IFU storage temperature for 24±4 hours 1157

Followed by 1158

Ambient humidity (X% RH) 1159 Put at IFU storage temperature for 24±4 hours followed by 1160 30 ± 5°C for 24±4 hours), followed by 1161 45 ± 5°C for 24±4 hours, followed by 1162 8 ± 5°C for 24±4 hours, followed by 1163 IFU storage temperature for 24±4 hours 1164

Note 1: It is important to make clear that the above complete sequence of 1165 temperatures will be used as opposed to separate kits held at individual 1166 temperatures. The actual temperatures, durations and the nature of the sequence, 1167 will depend on the IVD and the kinds of conditions expected to be encountered 1168 during shipping 1169

Note 2: Freezing temperatures are not considered in this example but would 1170 need to be if the kits could be exposed to freezing temperatures during transport. 1171

Note 3: If transport by air is anticipated the effect of reduced pressure must be 1172 included in this protocol (ASTM D4169 section 16) and should be for at least 10% 1173 longer than the longest anticipated flight at a pressure expected in aircraft holds. 1174

Note 4: The protocol will call for testing at least five individual devices after each 1175 stress condition with the stability panel members giving the most informative 1176 results. This will verify that the devices are sufficiently stable to progress to the 1177 next condition but that should already be certain from preliminary experiments 1178 and R&D work. 1179

Condition 2, Shaking. Each kit will be placed on a shaking table at X rpm for 1180 X hours/days at 42 ± 5°C as defined by ASTM D4169 section 12 (Reference [11]). 1181

After the simulated shipping challenge each kit will be returned to its 1182 corresponding storage temperature (42 ± 5°C or 8 ± 5°C). 1183



Testing will be conducted at 0, 3, 6, 9, 12 and 13 months. At each scheduled time 1184 point the allotted number of IVDs will be brought to room temperature and used 1185 to test each member of the panel in triplicate. 1186

Note 1: the test at 0 months will provide evidence that the device is stable under 1187 extreme (but quite likely) conditions of shipping, the testing at later time points 1188 will give evidence to support the claimed shelf-life after transport, that beyond the 1189 claimed shelf-life will provide evidence that the device is stable and not close to a 1190 failure point. 1191

Documentation for transport stress conditions 1192 In Worksheet XYZ00001 record: 1193

• The lot numbers of the IVD used to conduct the test 1194 • The Operator(s) name(s) 1195 • The dates of testing 1196 • Identifying details for each member of the stability testing panel being tested 1197 • The temperature that kits are stored at 1198 • The values of temperature and humidity for each of the challenge conditions 1199 • Instrument settings for the shaking apparatus and duration of operation 1200 • The ambient temperature and humidity during testing 1201 • Each test result as an interpretation according to the IFU 1202 • Each test result as a band intensity. Band intensity should be scored using the 1203

calibrated scale described in ProtocolZXY0001 (e.g. 0, faint/trace, +1, +2, 1204 +3 … +10) (even though the IFU does not give scores to results) 1205

• Any aberrations or deviations from the protocol, the reason for the deviation 1206 and any remedial action undertaken. Results from invalid assays must be 1207 recorded but not included in calculations of shelf-life. Apparently aberrant 1208 results, unless the underlying cause can be positively identified as not related 1209 to a problem with the device, must be included in the calculations of life. 1210

Stability testing panel 1211 See the suggestions in Appendix 2: Specimens for the stability testing panel. 1212

Acceptance criteria 1213 Each stability testing panel member should exhibit a band intensity at each time 1214 which matches its expected result. The expected result must be validated so that 1215 if the device would fail to meet claims (e.g. fail to detect critical specimens, have 1216 unacceptable performance at medical decision concentrations, have unacceptable 1217 specificity) the stability testing panel member would also fail to meet its specified 1218 result 1219

The stability after transportation of the IVD will be taken as the time point before 1220 the last time point to have met the acceptance criteria, e.g. if the IVD is stable to 1221 13 months, the stability during transportation will be deemed to be 12 months. 1222

The stability after transportation should be identical to the claimed shelf-life of 1223 the IVD, i.e. the extremes of possible conditions to which the IVD is likely to 1224 subjected during transport must not affect the shelf-life of the IVD. 1225

Calculation of results 1226 It is not the intent of this guide to provide detailed statistical instruction: that 1227 must be obtained from a professional statistician with an understanding of the 1228 requirements expressed herein. Professional statistical guidance is especially 1229



recommended when calculating confidence limits for discrete data such as 1230 readings from a graduated scale 1231

The following applies separately at each time point. 1232

The variance of the results for all replicates within and between all the lots must 1233 be calculated for each stability testing panel member. From the overall variance 1234 between lots the confidence with which future lots of the device will detect the 1235 panel member at that time point after manufacture and transport can be 1236 calculated. If the confidence of the panel member meeting its specification is less 1237 than some pre-defined value (normally 95%) then it must be deemed to have 1238 failed at that time point and the life of the device restricted accordingly. 1239

If regression analysis is used to define the time point at which a panel member 1240 would not meet its criterion then lot-to-lot variation must be included when 1241 setting the confidence limits around the regression line. However, real time data 1242 must extend beyond the claimed shelf-life so the intercept of the regression 1243 confidence limit and the expected value must be at a time longer than the claim. 1244 It is usually more appropriate to calculate as in the previous paragraph particularly 1245 if the regression cannot be proven to be linear. 1246

The stability of a device is not governed by the least stable lot that happens to 1247 have been tested – shelf-life must be supported by statistical evidence that all lots 1248 manufactured in that way will achieve the claimed life. This, of course, is true of 1249 all performance claims. 1250

Example 2: Shelf-life protocol 1251

Objective 1252 To determine the shelf-life of the HIV/TP RDT in real-time when stored at 8–40°C 1253 as proposed in Stability Study Plan XZY00001. 1254

Acquire sufficient numbers of kits from three separate production lots using a 1255 predetermined sampling protocol (e.g. random, first X kits in first box, every 100th 1256 kit, etc.). 1257

Acquire sufficient volume of each stability testing panel member for the duration 1258 of the testing schedule. Establish a method for randomising the panel members 1259 for IVD testing. Stability data of the panel members should already be established 1260 and recorded. 1261

Documentation 1262 In Worksheet XYZ00001 record the following: 1263

• The lot numbers kits were sampled from 1264 • The number of kits sampled from each lot 1265 • Details (including manufacturing/lot information) for each of the kit 1266

components that will be tested as part of this protocol: 1267 Test cassette: … 1268 Bottle of Specimen Buffer: … 1269

Note: The stability of an opened bottle of specimen buffer must be determined 1270 (see Example 3: In-use stability). For determination of shelf-life a bottle of 1271 specimen buffer should be unopened (i.e. fresh) at each testing point – there may 1272



be circumstances in which multiple sampling could be taken from the same bottle 1273 after it has been opened. 1274

Preparation 1275 The product kits chosen to be tested are in their final packaging including labelling. 1276


Kits will be divided into two groups. One group will be stored at 42 ± 5°C, the 1280 other at 8 ± 5°C. Kits from each group will then be subjected to the following 1281 conditions. 1282

Testing schedule 1283 At each scheduled time point the allotted number of IVDs will be brought to room 1284 temperature (20 ± 2°C) and used to test each member of the stability testing 1285 panel (see below) in triplicate. 1286

Note: To provide surety against unforeseen events, duplicate tests should be 1287 performed as a minimum. Testing in triplicate as a minimum provides a level of 1288 statistical confidence in the observed test result. 1289

Testing will be conducted at 0, 3, 6, 9, 12 and 13 months. 1290

Note: Testing beyond 13 months would allow an understanding of when, in real-1291 time, the IVD is likely to ‘fail’ and may allow an extension of the proposed shelf-life. 1292 It may be useful to know the “fail” point of an assay as it is the best measure of 1293 stability. However if it is obvious that the kit performance is decreasing over time, 1294 it can also be estimated visually and statistically when it will fail. 1295

Documentation 1296 In Worksheet XYZ00001 record: 1297

• The lot number(s) of the IVD(s) used to conduct the test 1298 • The Operator(s) name(s) 1299 • The dates of testing 1300 • Identifying details for each member of the stability testing panel being tested 1301 • Each test result as a band intensity. Band intensity should be scored using the 1302

calibrated scale described in Protocol ZXY0001 (e.g. 0, faint/trace, +1, +2, +3 … 1303 +10) 1304

• Each test result as an interpretation according to the IFU 1305 • Any aberrations or deviations from the protocol, the reason for the deviation 1306

and any remedial action undertaken 1307 • The temperature that kits are stored at 1308 • The ambient/room temperature during testing 1309

Stability Testing Panel 1310 See the suggestions in Appendix 2: Specimens for the stability testing panel. 1311

Acceptance Criteria 1312 Each stability testing panel member should exhibit a band intensity at each time 1313 point which matches its expected result. 1314

The shelf-life of the IVD will be taken as the time point before the last time point 1315 to have met the acceptance criteria. 1316



Example: If the IVD is stable to 13 months, the shelf-life will be deemed 1317 to be 12 months. 1318

Example 3: In-use stability protocol 1319

Objective 1320 To determine the stability of opened bottles of HIV RDT/TP Sample Buffer in real-1321 time when stored at 15–30°C as proposed in Stability Study Plan XZY00001. 1322

[In this example the manufacturer recommends that the test cassette be used 1323 immediately upon opening; this claim should also be validated in a separate 1324 experiment, such that it can be established that the IVD will still perform 1325 satisfactorily after the test cassette has been removed from its pouch and open at 1326 room temperature for 1, 2, 6, 24 hours, etc., as appropriate.] 1327

Acquire sufficient numbers of kits from one production lot using a predetermined 1328 sampling protocol (e.g. random, first X kits in first box, every 100th kit, etc.). 1329

Acquire sufficient volume of each stability testing panel member for the duration 1330 of the testing schedule. Establish a method for randomising the stability testing 1331 panel for IVD testing. 1332

In Worksheet XYZ00001 record the following: 1333 • The lot numbers kits were sampled from 1334 • The number of kits sampled from each lot 1335 • Details (including manufacturing/lot information) for each of the kit 1336

components that will be tested as part of this protocol: 1337 Test cassette: … 1338 Bottle of Sample Buffer: … 1339

Preparation 1340 Two sets of sample buffer are to be tested. One set of the component must be 1341 freshly made, the other towards the end of the assigned shelf-life of the device. 1342

The component is to be tested are in its final packaging including labelling. 1343


Half of each set will be stored at 30± 5°C, the other half at 15± 5°C. At the start of 1347 testing each bottle will be brought to room temperature (20 ± 2°C), opened, used 1348 for testing and then recapped and returned to the stated storage temperature. 1349

Note 1: It is important that the components under test are opened and used 1350 under circumstances likely to occur in users’ laboratories (i.e. not in rooms with 1351 HEPA filtered air) mimicking as far as possible genuine use. 1352

Testing schedule 1353 At each subsequent scheduled time point the allotted number of bottles will be 1354 brought to room temperature and used to test each panel member in triplicate. 1355 Testing will be conducted at 0, 1, 2, 3, 4 weeks up to the end of the claimed in-use 1356 life 1357



Documentation 1358 In Worksheet XYZ00001 record: 1359

• The lot number of the IVD used to conduct the test 1360 • The Operator(s) name(s) 1361 • The dates of testing 1362 • Identifying details for each member of the stability testing panel being tested 1363 • Each test result as a band intensity. Band intensity should be scored using the 1364

calibrated scale described in ProtocolZXY0001 (e.g. 0, faint/trace, +1, +2, +3 … 1365 +10) 1366

• Each test result as an interpretation according to the IFU 1367 • Any aberrations or deviations from the protocol, the reason for the deviation 1368

and any remedial action undertaken 1369 • The temperature at which kits are stored 1370 • The ambient temperature during testing 1371

Stability Testing Panel 1372 See the suggestions in Appendix 2: Specimens for the stability testing panel. 1373 1374

Acceptance Criteria 1375 Each stability testing panel member should exhibit a band intensity at each time 1376 which matches its expected result. 1377

The in-use stability of the sample buffer will be taken as the time point before the 1378 last time point to have met the acceptance criteria. 1379

Example: If the IVD is observed to be stable to 5 weeks, the in-use 1380 stability will be deemed to be 4 weeks. 1381



Appendix 2: Specimens for the stability testing panel 1382

Examples in this section 1383

Not all of the specimens in the examples that follow will be necessary for all IVDs, 1384 nor is the list exhaustive. Panels must be composed according to strict risk 1385 management principles, and all decisions must be documented and traceable. 1386

The minimum specimens to be included in a testing panel for the different 1387 products are outlined below. 1388

1 Nucleic acid test (NAT) 1389

If a proprietary nucleic acid preparation /extraction system is provided, the recovery must be shown to 1390 meet claims for each genotype from each of the specimen types claimed (e.g. dried blood spots, whole 1391 blood, plasma). Successful removal of inhibitory substances, if intended, must be demonstrated for 1392 appropriate specimen types. Unless potentially variable biological reagents are involved this system 1393 would be expected to be verified in manufacture and not necessarily tested at release. 1394

Specimens Remarks

Specimens to demonstrate maintenance of sensitivity and/or limit of detection, and/or accuracy, and precision

Traceability is probably required to one of the WHO international standards (e.g. 3rd HIV-1 International Standard NIBSC code: 10/152; 4th International Standard for hepatitis C virus for Nucleic Acid Amplification Techniques NIBSC code: 06/102; 3rd International Standard for HBsAg NIBSC code: 12/226). More than one genotype may be required to validate these claims: see 1st WHO International Reference Panel for HBV Genotypes for NAT-Based Assays, PEI code 5086/08. This may be required on each of the claimed specimen types.

Specimens to demonstrate specificity and validity of runs

Sufficient negative specimens should be included to ensure that the claims will be met at end of shelf life.

Specimens (or reagents) to demonstrate stability of each of the critical components of the IVD

If more than one part of the genome is to be detected, both systems must be shown to be stable. If both DNA and RNA are measured the complete system must be shown to be stable.



2 Stability testing panel for CD4 measuring IVDs 1395

Rationale 1396 CD4 measurements are quantitative, and accuracy at the clinical decision points is important. The 1397 design input should have information on the accuracy and other parameters required, and the panel 1398 must be designed to provide evidence that these parameters are maintained over the assigned-life of 1399 the reagent and measuring IVD. 1400

Parameters 1401 The stability testing panel used in stability work must be able to demonstrate the following. 1402 • Stability of all the antibodies used in the IVD (frequently anti-CD4 and anti-CD3 antibodies; any 1403

other critical components must be covered). 1404 • Accuracy and trueness of measurement maintained at the critical level (at least five specimens 1405

required) 1406 • Claimed linearity over the required range of CD4 count (at least five specimens required) 1407 • Measure drift 1408

Specimens 1409 Artificial specimens, such as stabilized blood specimens, can be used if a risk assessment based on R&D 1410 work indicates that they are effective. Fresh specimens are usually required. Measurements should be 1411 compared to an approved reference system. 1412

Examples of approaches 1413 Aged or in-use lots may be compared with a reference, e.g. a new lot. Precision studies can be 1414 performed as described in Reference [24]. 1415

More information is found in the “Sample Product Dossier for WHO Prequalification Simu POC CD4 1416 System”, which is available on the WHO Prequalification Team’s website 1417 http://www.who.int/diagnostics_laboratory/evaluations/140314_simu_poc_cd4_dossier_web.pdf?ua=1. 1418

3 Specimens to monitor tests for HIV antibodies 1419

Specimens Remarks IgM first seroconversion specimens and IgG first seroconversion specimens

Possible approaches to obtain samples : • Study the early data from commercial seroconversion panels

where the seroconversion was frequently monitored by IgM and IgG blots

• Study the responses to second and third generation assays or protein A and protein L assays (this approach is less useful).

All other parts of the HIV proteome included, e.g. reverse transcriptase (RT)

Late stage specimens – usually a high dilution set near the sample-to-cut-off ratio

This might serve to monitor any kit run control. HIV type serology is not particularly genotype dependent. It is usually not necessary to include controls for genotype detection unless risk or experiment shows that it is for a particular IVD.

HIV-2, diluted to near the sample-to-cut-off ratio

Seroconversion specimens are very rare

HIV-1 (0), if claimed Difficult specimens to monitor specificity and invalid rates

100 negatives at release subject to risk analysis and statistical analysis of the allowable (relative to the claimed) false reactive rate and invalidity rate




4 Specimens to monitor tests for antibodies for HIV-1/2 and 1420

Treponema pallidum (TP) 1421

Specimens Remarks

Specimens to detect HIV See 3 Specimens to monitor tests for HIV antibodies

Specimens to detect all the critical epitopes in the IVD, for example TpN47, TpN17 and TpN15

Note: Each of these epitopes play a role in detecting syphilis in different stages of the infection. It is necessary to have a stability testing panel member to monitor each epitope system present (and possibly each stage of infection), even if poly-fusion proteins are used. This can be avoided if the manufacturer can demonstrate that each epitope system is equally stable.

Specimens able to show that the invalidity and specificity rates do not fall outside the claims, particularly if whole blood is a claimed specimen type

Note: It would not be sufficient for WHO prequalification to extrapolate to the stability of HIV-2/TP detection by testing only HIV-1 positive specimens.

1422

5 Specimens to monitor tests for hepatitis C (HCV) virus 1423

antibodies 1424

Specimens Remarks

NS3 first seroconversion specimens and core first seroconversion specimens

Specimens to monitor any other antibodies claimed (frequently against NS5 and NS4)

Results can be obtained from line immunoassays that differentiate antibody responses to the different proteins.

A late stage dilution near the sample-to-cut-off ratio

Note: HCV serology is not particularly genotype-dependent in terms of anti-core and anti-NS3, but it is possible to make serotyping assays based on NS4 that mimic genotyping reasonably well. It is usually not necessary to include controls for genotype detection, unless risk assessment or experiment for a particular IVD show otherwise

Difficult specimens to monitor specificity and invalid rates

100 negatives at release subject to risk analysis and statistical analysis of the allowable false reactive rate and invalidity rate (relative to the claimed rates)



6 Specimens to monitor for tests for hepatitis B surface antigen 1425

(HBsAg) 1426

Specimens Remarks

Specimens to define sensitivity relative to the claim

This will almost certainly be one or more specimens with traceability to the HBsAg international standards and probably also to the ad and ay standards available from PEI. Seroconversion specimens commercially available are almost all of the adw2 serotype, different from the 3rd international standard – so claims of critical threshold specimen detection must be proven by specimens in the stability testing panel.

Specimens to monitor the maintenance of the claims of a variety of serotypes / genotypes and mutant forms

These will almost certainly be traceable to the “1st International Reference Panel for HBV genotypes for HBsAg—based assays" (WHO/BS/2011.2180).

Specimens to control against prozone effect if found or if theoretically an issue

If detection of HBsAg in the presence of anti-HBsAg is claimed (current best practise) proof of maintenance of the claim

Specimens to monitor the critical components of the IVD

If the monoclonal antibodies used have particular function or bias, such as against the ayr or adr serotypes (not controlled by the standards) or to detect mutant forms of the antigen, each must be monitored to ensure viability at end of life. These might well be the same specimens as in the previous paragraphs. If there are critical dissociation chemicals or red-cell capture or rupture agents used these must be monitored.

Difficult specimens to monitor specificity and invalid rates

100 negatives at release, subject to risk analysis and statistical analysis of the allowable (relative to the claimed) false reactive rate and invalidity rate.

1427 1428



Appendix 3: Summary table of standards relevant for stability studies 1429

Expectation Comment Standard Studies must be compliant with CLSI EP 025A and ISO 23640:2011

The minimum expected standards CLSI EP25A ISO 23640:2011

Studies must be fully documented with risk evaluations, plans and protocols prior to initiation

Risk assessment must be specific to the analyte, type of physical device and assay format, and previous manufacturing experiences, not generic nor by rote

CLSI EP25A (many section), ISO 23640:2011 Section 2 ISO 14971:2007

Studies and risk management must take into consideration conditions likely to be encountered in the geographies and healthcare settings in which the device is intended to be used

This is particularly important for transport stress where extreme conditions must be evaluated

Devices must be subjected to simulation of transport stress before being used to establish any form of stability

This is particularly important to WHO-PQ as transport will always be involved before use of a device and transport conditions cannot be guaranteed nor predicted

CLSI EP25A paragraph 4.2.3 & 5.2 [1]

Transport simulation must cover the extremes of environmental conditions ascertained during risk evaluations

It is most unlikely that actual transport will involve all extreme conditions that might occur during the marketing life of the device, nor that the conditions during actual transport can be adequately documented

CLSI EP25A Section 4.2.3

Devices used in any stability studies must be made to finalised manufacturing specifications, to final scale and in the packaging, including all labelling, in which the devices will be made available

If devices are not made to final validated and documented manufacturing scales a stringent proof that scale change will not affect any parameters of the device, nor any of the manufacturer’s claims, must be presented. Pre-production lots can only be used for stability work if these conditions are met

Good manufacturing practice (GMP) CLSI EP25A

If several presentations of the device are to be presented all aspects of stability must be shown for each

If, for example two pack sizes are to be provided, even though the contents are identical except for vial size, each pack size must be evaluated completely

CLSI EP25A

47

Expectation Comment Standard Sufficient numbers of independent lots of the device must be evaluated to enable each form of stability to be evaluated in terms of inter-lot variability

“Independent lots” means lots with different critical reagents (e.g. biological reagents prepared in different syntheses, growths or purifications; other risk-defined critical reagents from different manufactured lots, or different suppliers if applicable). CLSI EP25A and ISO 23640 specify minimum numbers of lots to be used but give no guidance to recommended numbers beyond documented risk evaluation

CLSI EP25A Section 4.4

If critical components of the device are assigned lives independently of the life of the device the various forms of stability of the device must be proven with those reagents at different stages of their lives

It must be documented that stored materials, e.g. freeze thawed biological reagents operate as expected during the whole of the assigned lives

CLSI EP25A Section 4.4

Each form of stability must be defined statistically with respect to any inter-independent lot variability, not just assigned to the minimum stability found among the lots that happened to be evaluated experimentally

If any lot-to-lot variability is found the manufacturer must provide evidence that subsequent lots will not have worse stability than that claimed

If any control material with a claim to prove the functionality of the device is provided to users that claim must be justified in stability studies in addition to any other studies

If the analytic function of the device is out of specification from any cause, including stability failure, the control material must be demonstrated to be able to alert the user to that fact

Use of accelerated stability, even to provide interim life assignments, must justified scientifically

Accelerated stability is acceptable to provide interim life if the parameters of the Arrhenius equation, or any other method used, are adequately proven and documented

CLSI EP25A Section 7.3 & Appendix B ISO 23640:2011 para 5.3.1 notes 1 & 2

1430
