Validation of Fluorescence in Situ Hybridization Using An

Validation of Fluorescence In Situ Hybridization Using anAnalyte-Specific Reagent for Detection of Abnormalities

Involving the Mixed Lineage Leukemia Gene

Debra F. Saxe, PhD; Diane L. Persons, MD; Daynna J. Wolff, PhD; Karl S. Theil, MD; for the Cytogenetics Resource Committee ofthe College of American Pathologists

N Context.—Fluorescence in situ hybridization (FISH) is amolecular cytogenetic assay that is commonly used in labora-tory medicine. Most FISH assays are not approved by the USFood and Drug Administration but instead are laboratory-developed tests that use analyte-specific reagents. AlthoughseveralguidelinesexistforvalidationofFISHassays,fewspecificexamples of FISH test validations are available in the literature.

Objective.—To provide an example of how a FISH assay,using an analyte-specific reagent probe, may be validatedin a clinical laboratory.

Design.—We describe the approach used by an individ-ual laboratory for validation of a FISH assay for mixedlineage leukemia (MLL) gene.

Results.—Specific validation data are provided illustrat-ing how initial assay performance characteristics in a FISHassay for MLL may be established.

Conclusions.—Protocols for initial validation of FISHassays may vary between laboratories. However, alllaboratories must establish several defined performancespecifications prior to implementation of FISH assays forclinical use. We describe an approach used for assessingperformance specifications and validation of an analyte-specific reagent FISH assay using probes for MLL rear-rangement in interphase nuclei.

(Arch Pathol Lab Med. 2012;136:47–52; doi: 10.5858/arpa.2010-0645-SA)

As the fields of cancer cytogenetics and molecularcytogenetics continue to expand, new tests are being

developed to identify an increasing number of chromo-some abnormalities that are important for the diagnosis,prognosis, and monitoring of neoplastic diseases. Molec-ular cytogenetic methods, such as fluorescence in situhybridization (FISH), are more sensitive than routinemetaphase chromosome analysis because they can detecta specific abnormality in both dividing and nondividingcells and have a lower threshold for detecting smallpopulations of abnormal cells.1,2 Thus, FISH testing hasbecome part of routine clinical practice in many labora-tories that deal with neoplastic specimens.

Introduction of a new FISH test requires assessment ofperformance characteristics of the assay and validation ofthe test by the laboratory. If the FISH test has beenapproved by the US Food and Drug Administration, thelaboratory must independently verify the performancecharacteristics of the assay. However, most FISH probes

used in clinical testing are not US Food and DrugAdministration–approved but instead are analyte-specificreagents. Although the technical performance of theseprobes has been evaluated by the manufacturer prior tomarketing, they must be validated as a laboratory-developed test prior to use in clinical testing. Generalguidelines on validation of FISH tests are available andprovide an excellent introduction to issues that areparticular to this methodology.3–7 Validation of FISHprobes should include the following steps: (1) familiar-ization and planning, (2) assessment of performancecharacteristics (accuracy, precision, sensitivity and spec-ificity, reference ranges, and reproducibility), and (3)implementation of the test. This article describes anexample of how to use these steps to validate FISH probesfor detecting aberrations in the mixed lineage leukemia(MLL) gene in interphase nuclei using an analyte-specificreagent manufactured by Abbott Molecular, Inc (DesPlaines, Illinois).

FAMILIARIZATION AND PLANNING

Prior to initiating laboratory studies, it is important tounderstand the biology of the target locus and its diseaseassociation(s). For example, a literature review of the MLLgene at 11q23 shows that it encodes a transcriptionalregulatory factor that is a major regulator involved inhematopoietic and embryonic development that actsthrough regulation of HOX gene expression.8 MLL is themost promiscuous gene yet observed in cancer withrearrangements involving more than 50 gene partners.Abnormalities involving MLL account for approximately

Accepted for publication December 14, 2010.From the Department of Pathology and Laboratory Medicine, Emory

School of Medicine, Atlanta, Georgia (Dr Saxe); the Department ofPathology and Laboratory Medicine, University of Kansas MedicalCenter, Kansas City (Dr Persons); the Department of Pathology andLaboratory Medicine, Medical University of South Carolina, Charleston(Dr Wolff); and the Department of Clinical Pathology, Cleveland Clinic,Cleveland, Ohio (Dr Theil).

The authors have no relevant financial interest in the products orcompanies described in this article.

Reprints: Debra F. Saxe, PhD, Department of Pathology andLaboratory Medicine, Emory University Hospital, 1364 Clifton Rd,Room 143A, Atlanta, GA 30022 (e-mail: [email protected]).

Arch Pathol Lab Med—Vol 136, January 2012 Validation of FISH Using ASR for MLL—Saxe et al 47

10% of the chromosome abnormalities observed in acuteleukemia of both lymphoid and myeloid lineages.8–11

The laboratory personnel involved in testing andscoring should become familiar with the concepts under-lying the probe labeling, testing strategy, and resultreporting. A break-apart probe strategy is particularlyappropriate for detecting MLL gene rearrangementsbecause chromosomal breakpoints occur in a relativelynarrow region of the gene regardless of translocationpartner. The dual-color analyte-specific reagent probe setavailable from Abbott Molecular consists of differentiallylabeled 59 and 39 probes that flank the common breakpointregion in the MLL locus. A chromosome translocationdisrupts the normally juxtaposed probes, creating 2widely separated signals that can be distinguished asdifferent colors. Test results are reported qualitatively aseither negative or positive for MLL rearrangement, but thenumber of negative or positive cells can also be included.

Laboratorians should be familiar with the establishedpractice and/or laboratory standards and guidelines andregulations that are applicable to the testing methodology.Helpful references include the general FISH standardsand guidelines from the American College of MedicalGenetics (www.acmg.net, accessed September 2010), theClinical Laboratory Improvement Act and Amendments,and the College of American Pathologists standardchecklist items that pertain to FISH.12–14 An approvedguideline for FISH assay verification and clinical valida-tion prepared by the National Committee for ClinicalLaboratory Standards (now Clinical and LaboratoryStandards Institute) is available15 and a second edition isanticipated.

During the familiarization phase of probe validation thelaboratory performs practice assays with the FISH probeset to assess its performance characteristics. The goal is forlaboratory personnel to gain experience with the specificvariables that affect hybridization and scoring of thebreak-apart probe. Most probes come with a packageinsert that describes a hybridization protocol optimized bythe manufacturer for the particular probe set. In addition,several manuals are available that include protocols andtroubleshooting guidance for FISH.16–18 If the laboratory isnot familiar with specific probe designs (break-apart,single fusion, double fusion, etc), several normal andabnormal specimens should be tested to gain experiencewith the reagents, techniques, protocols, fluorescencefilters, and hybridization characteristics. For laboratoriesalready familiar with the break-apart probe design andinterpretation, several specimens should be assayed withthe new probe to establish its specific hybridizationcharacteristics and to set criteria for scoring for varioussignal patterns.

The MLL dual-color break-apart probe set used here asan example is available from Abbott Molecular andconsists of a probe to the 59 proximal portion of the geneand upstream flanking sequences (350 kb) labeled withSpectrumGreen and a probe to the 39 distal portion of thegene and downstream flanking sequences (190 kb) labeledwith SpectrumOrange. The latter signal may appear morered than orange and is referred to as red (R) in thedescriptions of normal and abnormal probe signalpatterns that follow. In a normal cell, the probe signalsusually appear as 2 yellow fusion signals (FF) due tooverlap of the closely spaced 59 green and 39 redfluorescent signals on each chromosome 11 (Figure, A).

In a cell with MLL rearrangement the proximal and distalprobe sequences are physically separated (usually as aresult of chromosome translocation), and the probe signalpattern commonly appears as 1 green (G) signal (corre-sponding to the 59 MLL locus remaining on the derivativechromosome 11), 1 R signal (corresponding to the 39 MLLlocus translocated to a partner chromosome), and 1 Fsignal (corresponding to the normal chromosome 11); thispattern (Figure, B) can be designated FRG for datarecording purposes.

During the familiarization phase, normal cases may beused to optimize the conditions for hybridization, pre-hybridization and posthybridization washing, and analy-sis. All signal patterns observed in the normal samplesshould be carefully examined and recorded for futurereference. Depending upon the specific probes, a normalsignal pattern produced by dual-color break-apart probesmay appear as a yellow F signal, as separate side-by-sideRG signals with no space in between, or as separate, side-by-side RG signals spaced only a short distance apart. Thisvariability is due in part to differing degrees of DNAcondensation in normal interphase nuclei. Becomingfamiliar with the range of signal separation associatedwith a particular break-apart probe set and with thefrequency of unexpected signal patterns encountered inknown normal cells is critical to avoid false-positiveinterpretations during the next stage of the validation.Some abnormal cases known to have MLL rearrangementby either metaphase chromosome analysis or anotherlaboratory assay should also be studied during this phaseto gain familiarity with the range of abnormal signalpatterns.

Criteria for evaluating the quality of hybridization andstaining, selecting appropriate interphase cells for scoring,and recording signal pattern interpretations should bedeveloped. For example, probe signal intensities shouldbe sufficiently robust that all hybridized signals are easilyrecognized without interference from nonspecific hybrid-ization or background staining. Interphase nuclei shouldbe single, nonoverlapping, round cells or have the shapeand size appropriate for the cell type targeted in theanalysis. Data should be recorded in a standardizedfashion. After formulating these criteria, training shouldbe provided to the laboratory personnel involved inscoring cells for analysis prior to the next step of thevalidation.

ASSESSMENT OF PERFORMANCE CHARACTERISTICS

After technologists and the laboratory director arefamiliar with the MLL FISH assay and have developedcriteria for scoring normal and abnormal cells, the formalprocess for determining the performance characteristics ofthe assay begins. Prior to analysis, FISH slides areevaluated to ensure that the assay has met criteria foracceptable technical performance. Characteristics exam-ined include adequacy and consistency of signal strength(brightness), lack of background and/or cross-hybridiza-tion signals, and presence of appropriate control (internalor external) signals. Only slides that are consideredacceptable for analysis are further evaluated.

Assay performance characteristics include accuracy,precision, sensitivity, and specificity. Metaphase cells areused to localize the probe and determine its analyticalsensitivity and specificity; male cells are used to rule outcross hybridization of the probe with the Y chromosome.

48 Arch Pathol Lab Med—Vol 136, January 2012 Validation of FISH Using ASR for MLL—Saxe et al

The metaphase cells can be obtained from bone marrowfrom a pooled sample (prepared from 5 karyotypicallynormal males with similar mitotic indices) or from 5separate karyotypically normal male samples (20 cellsscored per sample). Probe localization and analyticalsensitivity and specificity can be assessed simultaneously.The discussion that follows focuses on the use of MLLFISH as a qualitative test in bone marrow specimens.

Accuracy

Whether the qualitative MLL FISH assay is consideredaccurate is determined both by the performance of theprobes and by the performance of the assay as a diagnostictest. It is important to confirm that the probes hybridize totheir expected chromosomal loci (probe sensitivity) andonly to their expected chromosomal loci (probe specific-ity) and to demonstrate positive (abnormal) results in ahigh proportion of cases known to have MLL rearrange-ment (assay sensitivity) and negative (normal) results in ahigh proportion of cases known to be negative for MLLrearrangement (assay specificity). Assays that are bothhighly sensitive and highly specific have a high degree ofaccuracy. The accuracy of the MLL FISH assay wasdetermined as follows.

Probe localization and its sensitivity and specificitywere determined by recording the location (11q23 targetversus nontarget) of MLL probe signals in 100 intactmetaphase cells. Five representative metaphase cells werecaptured for documentation of probe localization. MLLprobe signal localization to 11q23 was determined byevaluating chromosome morphology using reverse 49,6-diamidino-2-phenylindole dihydrochlorine (DAPI) imag-ing of the hybridized chromosome and/or sequentialG-banding followed by FISH.

To determine the assay sensitivity and specificity,interphase signal patterns were examined in knownnormal and abnormal specimens (see discussion under‘‘Reference Range’’ later).

Precision (Reproducibility)

Precision is defined as the closeness of agreementbetween independent test results. For qualitative FISHassays, such as the MLL rearrangement detection assaybeing described here, precision may be consideredequivalent to reproducibility.7 One approach that canbe used for determining reproducibility is to performreplicate assays of a normal specimen and a specimenwith a known proportion of abnormal cells duringseveral days. A comparison of the mean, standarddeviation, and range of results between replicatesdetermines the level of reproducibility. Alternatively,as in this example, once a laboratory has become veryfamiliar with different FISH probes, a good estimate ofprecision and/or reproducibility for qualitative inter-phase FISH assays can be provided by the analyticalsensitivity and specificity.3 In addition, an estimate of theinterobserver and intraobserver variability encounteredin scoring a FISH assay obtained while collecting data todetermine probe and assay sensitivity and specificity(described later) provides another measurement ofreproducibility.3 Assigning an acceptable level of repro-ducibility for a FISH assay can be problematic due to therelatively small number of cells analyzed in each

The probe for MLL (11q23) (Abbott Molecular, Des Plaines, Illinois) is a Locus-Specific Indicator (LSI) break-apart probe designed with the 59 portionof the gene labeled with SpectrumGreen (G) and the 39 portion of the gene labeled with SpectrumOrange (R). A, In normal diploid interphase nucleieach MLL locus appears as adjacent overlapping red and green signals or as a yellow fusion (F) signal. Two fusion (FF) signals are expected in eachnucleus, 1 corresponding to each 11q23 locus. B, When one MLL locus is rearranged due to a chromosome translocation, the 59 and 39 probes arephysically separated and appear as single red (R) and green (G) signals. The signal pattern is designated FGR, corresponding to 1 F signal on thenormal chromosome 11, 1 G signal on the 59 portion of MLL remaining on the derivative chromosome 11, and 1 R signal on the 39 portion of MLLthat is translocated to another chromosome (original magnifications 31500).

Table 1. Sensitivity and Specificity of Mixed LineageLeukemia (MLL) FISH on 5 Normal Male Bone

Marrow Specimens

Case No.

MLL Probe Signal Pattern in Metaphase Nuclei

Normal (FF at11q23)

Abnormal(FRG) Other

1 20 0 02 20 0 03 20 0 04 20 0 05 20 0 0

Abbreviations: FF, two fusion signals present; FRG, red, green, andfusion signals present; FISH, fluorescence in situ hybridization.


replicate, the presence of variant signal patterns, andvarying degrees of mosaicism.

Analytical Sensitivity

Considerations in determining analytical sensitivity inFISH studies include the fidelity with which the probe

hybridizes to its expected chromosome target (probesensitivity) and the performance of the assay as adiagnostic test with few false-negative results in compar-ison to a gold standard (assay sensitivity). Probe sensitiv-ity is defined as the percentage of chromosome targets thathave the expected normal signal pattern. Assay sensitivity

Abbreviations: Betainv, beta inverse function; F, 1 fusion signal only; FF, 2 fusion signals (normal); FFF, 3 fusion signals; FFFF, 4 fusion signals; FG, 1fusion, 1 green signal; FR, 1 fusion, 1 red signal; FFR, 2 fusion, 1 red signal; FFRG, 2 fusion, 1 red, 1 green signal; FRG, 1 fusion, 1 red, and 1 greensignal; SD, standard deviation.a Every signal pattern that was observed was recorded. For the less common abnormal signal patterns (FFF, FFR, FFRG, FFFF, other) data for both

technologists were combined. Normal cutoffs were determined using the beta inverse function 5BETAINV (available in Microsoft Excel).b T-1 and T-2 designate the 2 technologists who scored each specimen.c Sum of T-1 and T-2 for each pattern.

Table 2. Database of 200 Interphase Nuclei Scored for Mixed Lineage Leukemia (MLL) Gene Signal Pattern as Observedin 20 Normal Control Marrow Specimensa

Sample

Normal Abnormal Signal Patterns

FF FRG FG FR

T-1b T-2 % T-1 T-2 % T-1 T-2 % T-1 T-2 %

1 97 89 93.0 2 3 2.5 0 0 0.0 0 0 0.02 94 94 94.0 3 4 3.5 0 0 0.0 0 0 0.03 94 93 93.5 1 1 1.0 0 0 0.0 0 0 0.04 96 96 96.0 2 0 1.0 0 0 0.0 0 0 0.05 95 96 95.5 0 0 0.0 1 0 0.5 0 0 0.06 96 94 95.0 1 0 0.5 0 0 0.0 0 0 0.07 95 94 94.5 0 0 0.0 0 0 0.0 0 0 0.08 96 97 96.5 2 0 1.0 0 0 0.0 0 0 0.09 98 91 94.5 0 3 1.5 0 0 0.0 0 0 0.010 94 95 94.5 2 0 1.0 0 0 0.0 0 0 0.011 100 96 98.0 0 0 0.0 0 0 0.0 0 0 0.012 96 98 97.0 1 1 1.0 0 0 0.0 0 0 0.013 99 100 99.5 0 0 0.0 0 0 0.0 0 0 0.014 95 99 97.0 2 0 1.0 0 0 0.0 0 0 0.015 98 97 97.5 0 0 0.0 0 0 0.0 0 0 0.016 96 95 95.5 1 1 1.0 0 0 0.0 0 0 0.017 96 97 96.5 1 0 0.5 0 0 0.0 0 0 0.018 97 97 97.0 1 1 1.0 0 0 0.0 0 0 0.019 98 98 98.0 0 0 0.0 0 0 0.0 0 0 0.020 97 98 97.5 0 0 0.0 0 0 0.0 0 0 0.0

Mean 96.4 95.7 96.0 1.0 0.7 0.8 0.1 0.1 0.0 0.0 0.0 0.0Minimum 94 89 93.0 0 0 0.0 0 0 0.0 0 0 0.0Maximum 100 100 99.5 3 4 3.5 1 0 0.5 0 0 0.0SD 1.7 2.7 1.7 0.9 1.2 0.9 0.2 0.0 0.1 0.0 0.0 0.0Betainv (%) 6.3 2.3 1.5

Table 3. Correlation of Interphase Fluorescence In Situ Hybridization (FISH) Signal Pattern With Results ofMetaphase Cytogenetics

Case

Number of Cells With FISH Signal Patterna

KaryotypeFF FRG F FFF FFFF FG F2RG

1 20 176 2 2 46,XX,t(4;11)(q21;q23)[19]/46,XX[1]

2 25 175 46,XX,t(4;11)(q21;q23)[31]/46,XX[3]

3 18 182 46,XY,t(9;11)(p22;q23)[19]/46,XY[1]

4 200 45,XY,211[20]5 7 193 46,XY,del(11)(q13q23)[20]6 31 169 47,XX,+11[27]/46,XX[3]7 196 1 1 2 46,XY[20]8 197 1 2 46,XX[20]9 199 1 46,XX[30]10 199 1 46,XY[30]

Abbreviations: F, 1 fusion signal; FF, 2 fusion signals (normal); FFF, 3 fusion signals; FFFF, 4 fusion signals; FG, 1 fusion and 1 green signal; F2RG, 1fusion, 2 red, and 1 green signals; FRG, 1 fusion, 1 red, and 1 green signals.a 200 interphase cells were scored per case.


is determined by examining interphase signal patterns inknown normal and abnormal specimens (see discussionunder ‘‘Reference Range’’ later).

MLL probe sensitivity was determined by calculatingthe proportion of 11q23 target sites that had the expectedprobe signal. As seen in Table 1, the expected FF probesignal pattern was localized to 11q23 in 100 of 100metaphase cells examined (200 of 200 chromosome11q23 targets), yielding a probe sensitivity of 100%. TheAmerican College of Medical Genetics FISH Guidelines(www.acmg.net, accessed September 2010) recommendthat probe sensitivity should be at least 98%.

Analytical Specificity

Considerations in determining analytical specificity inFISH studies include the fidelity with which the probehybridizes only to its expected chromosome target (probespecificity) and the performance of the assay as adiagnostic test with few false-positive results in compar-ison to a gold standard (assay specificity). Probe specific-ity is defined as the percentage of probe signals thathybridize to the correct locus and to no other location.Assay specificity is determined by examining interphasesignal patterns in known normal and abnormal specimens(see discussion under ‘‘Reference Range’’ later).

MLL probe specificity was determined by examining thelocation (target versus nontarget) of all MLL probe signals in100 intact metaphase cells followed by calculating theproportion of the probe signals that hybridized to the 11q23locus. Hybridization of a probe to a nontarget locus isevidence for decreased probe specificity; this is manifest asan F or separated R or G signals at a site other than 11q23.For example, if 199 of 200 MLL probe signals scored arelocated at 11q23, the specificity (number of signals atexpected target/total number of signals) would be 99.5%.

As seen in Table 1, no cells with an abnormal signal patternwere identified, corresponding to a probe sensitivity of100%. The American College of Medical Genetics FISHGuidelines (www.acmg.net, accessed September 2010)recommend that probe specificity should be at least 98%.

Reportable Range

This parameter is not applicable to qualitative assays.

Reference Range (Normal Cutoff)

The reference range, as defined by the AmericanCollege of Medical Genetics FISH Guidelines (www.acmg.net, accessed September 2010), is the range of testvalues expected to occur in 95% of healthy individuals.The upper cutoff for normal results in a FISH assay can bedetermined by calculating the 95% confidence interval forprobe signal patterns found in normal control samplesthat are representative of the sample type to be analyzed.

All MLL probe signal patterns in a defined number (seelater) of interphase cells from each of 20 normal bonemarrow samples were analyzed and recorded. Scoringcriteria were used as defined in the familiarization phase.Because most laboratories do not have access to bonemarrow specimens from normal volunteer donors, thesesamples may be difficult to obtain. As a practical alternative,karyotypically normal bone marrow specimens referred forassessment of nonneoplastic conditions that would beunlikely to harbor a MLL rearrangement (or other probebeing validated) may be used for the validation.

It is useful for 2 individuals to independently score thesame slides, each recording the consecutive signalpatterns encountered in the first 100, 150, 200, and 250cells. When counts are totaled, this allows for constructionof a 200, 300, 400, and 500 cell database that may bevaluable later should unanticipated circumstances arise in

Total Scored

F

FFFc FFRc FFRGc FFFFc Otherc

Subtotal

T-1 T-2 % No. %

1 8 4.5 0 0 0 0 0 0 0.0 2002 1 1.5 2 0 0 0 0 2 1.0 2003 2 2.5 2 0 0 4 0 6 3.0 2002 4 3.0 0 0 0 0 0 0 0.0 2004 3 3.5 1 0 0 0 0 1 0.5 2002 1 1.5 3 0 0 3 0 6 3.0 2001 0 0.5 5 0 0 5 0 10 5.0 2002 3 2.5 0 0 0 0 0 0 0.0 2001 2 1.5 3 0 0 2 0 5 2.5 2004 2 3.0 1 0 0 2 0 3 1.5 2000 2 1.0 0 0 0 2 0 2 1.0 2003 1 2.0 0 0 0 0 0 0 0.0 2001 0 0.5 0 0 0 0 0 0 0.0 2003 1 2.0 0 0 0 0 0 0 0.0 2002 3 2.5 0 0 0 0 0 0 0.0 2003 1 2.0 1 1 0 1 0 3 1.5 2001 1 1.0 2 0 0 2 0 4 2.0 2002 2 2.0 0 0 0 0 0 0 0.0 2002 1 1.5 1 0 0 0 0 1 0.5 2002 2 2.0 0 0 0 1 0 1 0.5 200

2.1 2.0 2.0 1.1 0.1 0.0 1.1 0.0 2.2 1.1 2001 0 0.5 0 0 0 0 0 0 0.0 2009 8 3.5 5 1 0 5 0 10 5.0 2001.1 1.7 1.0 1.4 0.2 0.0 1.5 0.0 2.8 1.4 0.0

7.4 5.1 2.3 1.5 5.1 1.5

Table 2. Extended


the interpretation of clinical specimens (eg, need to scoreadditional cells to resolve discrepant results close to acutoff that could alter the diagnostic interpretation).Table 2 illustrates one way of documenting the numberof cells with various MLL signal patterns that wereobserved in a 200 cell analysis in each of 20 normalcontrol specimens. It is important to record results of allprobe signal patterns encountered during this phase of thevalidation because a different normal cutoff value canthen be calculated for each signal pattern. This alsoprovides a quality control tool with which to compare thereproducibility of results between individual readers.

The normal cutoff value for the MLL FISH assay wascalculated using the beta inverse function with a probabilityof 95% [5BETAINV(probability,alpha,beta,[A],[B]) availablein Microsoft Excel (Microsoft Corp, Redmond, Washing-ton)]. Variables to be entered into the function equationinclude probability, a desired level of probability associatedwith the beta distribution; alpha and beta, parameters of thedistribution; A, an optional lower bound of the distribution;and B, an optional upper bound of the distribution. Tocomplete the equation, probability equals 0.95, alpha equalsthe number of false-positive nuclei +1 found in the normalspecimen with the greatest number of false-positive nucleifor any given signal pattern, and beta equals the totalnumber of cells scored. For example, to calculate the normalcutoff value for the MLL signal pattern FRG listed in Table 2,note that of the 20 samples in the reference group, sample 2has the highest number of false-positive FRG signals (4 + 3 57). The beta inverse function as calculated in Microsoft Excelis therefore 5BETAINV(0.95, 7+1, 200) or 6.3%. Because agiven test result is considered abnormal if the percentage ofnuclei with that aberrant signal pattern exceeds the 95%confidence interval of the normal reference distribution, aminimum of 13 of 200 (6.5%) cells would be required for anFGR signal pattern to be considered an abnormal result. Thesignal patterns and respective calculated normal cutoffs forthe MLL probe set when 200 cells are scored are FRG (6.3%),FG (2.3%), FR (1.5%), F (7.4%), FFF (5.5%), FFR (2.3%), FFRG(1.5%), and FFFF (5.1%). For other less common abnormal-ities, such as deletion of the distal portion of the MLL gene, aminimum of 5 cells (2.5%) with a signal pattern of FG woulddenote an abnormal clone.

Normal cutoffs can also be calculated assuming aGaussian distribution of results in the normal referencepopulation for 2 or 3 standard deviations. The primarydifference between the cutoffs generated by the Gaussian orbeta inverse function is in the case where the number of false-positive cells is zero. Using the Gaussian statistical method,should a particular signal pattern be encountered in a patientsample that was not observedin the normal control dataset, asingle cell with that pattern would be considered abnormal.The beta inverse formula does not allow a cutoff of zero.19

Ten specimens, including normal cases and cases withMLL rearrangement confirmed by another method, werethen examined in a blinded fashion for detection ofdifferent MLL abnormalities in patient samples. Speci-mens known to have 11q23 rearrangement by cytogeneticswere selected as a gold standard to validate the cutoffvalues. It is also helpful to test cases with knownchromosome abnormalities such as variant translocations,interstitial or terminal deletions, or aneuploidy forchromosome 11 that are expected to generate variantMLL FISH signal patterns. Examples of MLL FISH signalpattern results from a test set of 4 normal cases, 3 cases

with 11q23 rearrangements, 2 with loss of chromosome 11,and 1 with gain of chromosome 11 are presented inTable 3. The FISH probe results correlated well with thecytogenetic results in all cases; in this example the assaysensitivity and specificity were each 100%.

IMPLEMENTATION OF THE TEST

The assay may be put into clinical use only aftercompletion of the assessment of probe performance andvalidation studies. Although such an extensive validationdoes not need to be performed on each new probe lot, eachlot must be shown to have comparable sensitivity andspecificity prior to being put into use for clinical testing.5,14

This may be accomplished by analyzing a patient samplewith both the old and new lots to document equivalentresults. Ongoing quality control of reagents and equipmentand continued monitoring of assay performance is required.Comments recorded on assay worksheets concerning thesize and brightness of probe signals, presence or absence ofbackground staining and/or cross hybridization, andhybridization efficiency can be monitored for acceptabilityas one method of ongoing assay performance verification.Similarly, ongoing monitoring of interobserver reproduc-ibility, accomplished in part by having 2 technologists readevery case, can help detect changes in the assay perfor-mance or loss of consistency in applying scoring criteria.Participation in external proficiency testing programs is animportant form of ongoing quality assurance.

References

1. Secker-Walker LMM. General report on the European Union concertedaction workshop on 11q23. Leukemia. 1998;12(27):776–778.

2. Cohen N, Novikov I, Hardan I, et al. Standardization criteria for thedetection of BCR/ABL fusion in interphase nuclei of chronic myelogenousleukemia patients by fluorescence in situ hybridization. Cancer Genet Cytogenet.2000;123(2):102–108.

3. Wiktor AE, Van Dyke DL, Stupca PJ, et al. Preclinical validation of fluorescencein situ hybridization assays for clinical practice. Genet Med. 2006;8(1):16–23.

4. Test and Technology Transfer Committee. Technical and clinical assess-ment of fluorescence in situ hybridization: an ACMG/ASHG position statement.I. Technical considerations. Genet Med. 2000;2(6):356–355.

5. CLSI. User Verification of Performance and Precision of Trueness; ApprovedGuideline. 2nd ed. Wayne, PA: Clinical and Laboratory Standards Institute; 2006.CLSI document EP15-A2.

6. Von Bergh A, Emanuel B, Van Zelderen-Bhola S, et al. A DNA probecombination for improved detection of MLL/11q23 breakpoints by double-colorinterphase-FISH in acute leukemias. Genes Chromosomes Cancer. 2000; 28(1):14–22.

7. Jennings L, Van Deerlin VM, Gulley ML. Recommended principles andpractices for validating clinical molecular pathology tests. Arch Pathol Lab Med.2009;133(5):743–755.

8. Bernard OA, Berger R. Molecular basis of 11q23 rearrangements in hematopoi-etic malignant proliferations. Genes Chromosomes Cancer. 1995;13(2):75–85.

9. Rubnitz JE, Behm FG, Downing JR. 11q23 rearrangements in acuteleukemia. Leukemia. 1996;10(1):74–82.

10. Young BD, Saha V. Chromosome abnormalities in leukemia: the 11q23paradigm. Cancer Surv. 1996;28:225–245.

11. Pui CH, Chessells JM, Camitta B, Baruchel A. Clinical heterogeneity inchildhood acute lymphoblastic leukemia with 11q23 rearrangements. Leukemia.2003;17(4):700–706.

12. Public Law 100-578, Clinical Laboratory Improvement Amendments of1988, October 31, 1988. Fed Regist. 1992;57(40):7002–7186.

13. Schwartz MK. Genetic testing and the Clinical Laboratory ImprovementAmendments of 1988: present and future. Clin Chem. 1999; 45(5):739–745.

14. Commission on Laboratory Accreditation. Laboratory Accreditation Program:Cytogenetics Checklist. Northfield, IL: College of American Pathologists; 2009.

15. NCCLS. Fluorescence In Situ Hybridization (FISH) Methods for MedicalGenetics; Approved Guideline. Wayne, PA: National Committee for ClinicalLaboratory Standards; 2004. NCCLS Document MM7-A.

16. Barch MJ, Knudson T, Spurbeck JL, eds. AGT Cytogenetics LaboratoryManual. 3rd ed. Philadelphia, PA: Lippincott-Raven; 1997:557–590.

17. Fluorescence in situ hybridization. Nat Methods. 2005;2(3):237–238.18. Rooney DE, ed. Human Cytogenetics. 3rd ed. Oxford, England: Oxford

University Press; 2001:175–209.19. Ciolino AL, Tang ME, Bryant R. Statistical treatment of fluorescence in situ

hybridization validation data to generate normal reference ranges using Excelfunctions. J Mol Diagn. 2009;11(4):330–333.


Documents

Validation of Fluorescence in Situ Hybridization Using An