Performance of the Maxim and SediaLimiting Antigen Avidity assays forpopulation-level HIV incidence

surveillanceJoseph B Sempa1dagger Alex Welte1 Michael P Busch23 Jake Hall4 Dylan Hampton2 Sheila

M Keating23 Shelley N Facente25 Kara Marson3 Neil Parkin6 Christopher DPilcher3 Gary Murphy4dagger Eduard Grebe12dagger on behalf of the Consortium for the

Evaluation and Performance of HIV Incidence Assays (CEPHIA)sect

1DST-NRF Centre of Excellence in Epidemiological Modelling and Analysis (SACEMA) StellenboschUniversity Stellenbosch South Africa

2Vitalant Research Institute San Francisco CA3University of California San Francisco San Francisco CA

4Public Health England London United Kingdom5Facente Consulting Richmond CA6Data First Consulting Belmont CA

daggerThese authors contributed equallysectMembership list can be found in the Acknowledgments section

Correspondence EGrebevitalantorg

12 March 2019Preprint

Abstract

Background Two manufacturers Maxim Biomedical and Sedia Biosciences Cor-poration supply US CDC-approved versions of the HIV-1 Limiting Antigen AvidityEIA (LAg assay) for detecting lsquorecentrsquo HIV infection in cross-sectional incidence estim-ation This study assesses and compares the performance of the Maxim and Sedia LAgassays for incidence surveillanceMethods We ran both assays on a panel of 2500 well-characterized HIV-1-infectedspecimens most with estimated dates of (detectable) infection We analysed concord-ance of assay results assessed reproducibility using repeat testing and estimated thecritical performance characteristics of a test for recent infectionmdashmean duration ofrecent infection (MDRI) and false-recent rate (FRR)mdashfor a range of normalized op-tical density (ODn) recency discrimination thresholds alone and in combination withviral load thresholds We further defined three hypothetical surveillance scenarios andevaluated overall performance for incidence surveillance defined as the precision ofincidence estimates by estimating context-specific performance characteristicsResults The Maxim assay produced lower ODn values than the Sedia assay on av-erage largely as a result of higher calibrator readings (mean calibrator OD of 0749

1

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

vs 0643) Correlation of non-normalized OD readings was greater (1198771113569 = 0938)than those of ODn readings (1198771113569 = 0908) and the slope was closer to unity (1054 vs0899) Reproducibility of repeat testing of three blinded control specimens (25 replic-ates each) was slightly better for the Maxim assay (CV 89 to 148 vs 132 to150) The MDRI of a Maxim-based algorithm at the lsquostandardrsquo recency discrim-ination threshold in combination with viral load (ODn le15 amp VL gt1000) was 201days (95 CI 180223) and for Sedia was 171 days (95 CI 152191) Commensur-ately the Maxim algorithm had a higher FRR in treatment-naive subjects (17 vs11) We observed statistically significant differences in MDRI using the ODn alone(le15) and in combination with viral load (gt1000) Under three fully-specified hypo-thetical surveillance scenarios (comparable to South Africa Kenya and a concentratedepidemic) recent infection testing algorithms based on the two assays produced similarprecision of incidence estimatesConclusions Differences in ODnmeasurements between theMaxim and Sedia LAgassays on the same specimens largely resulted from differences in the reactivity of cal-ibrators supplied by the manufacturers Performance for surveillance purposes was ex-tremely similar although different ODn thresholds were optimal and different values ofMDRI and FRRwere appropriate for use in survey planning and incidence estimation

1 BackgroundThe Limiting Antigen Avidity EIA (LAg-Avidity Assay) was developed by the US Centers for Dis-ease Control and Prevention (CDC) for detecting lsquorecentrsquo HIV infection for the purposes of cross-sectional incidence estimation [1] Two major manufacturers supply versions of the assay MaximBiomedical (Bethesda MD) and Sedia Biosciences Corporation (Portland OR) with both man-ufacturers currently utilizing multisubtype HIV-1 recombinant antigen supplied by the CDC Athird manufacturer Beijing King Hawk Pharmaceutical Co (Beijing PRC) has recently enteredthe market but without US CDC involvement [2]

TheMaxim and Sedia assays have generally been assumed to perform similarly and users of theMaxim assay have mainly used performance metrics (mean duration of recent infectionmdashMDRImdashand false-recent ratemdashFRR) estimated from calibration data produced using the Sedia assay [3]A recent comparison of the assays based on 1410 treatment-naiumlve specimens found substantiallylower normalized optical densities (attributed to differences in calibrators) and consequently a longerMDRI (at the lsquostandardrsquo threshold of 15) for the Maxim assay [4] We present the first large-scale independent evaluation of the Maxim LAg assay for surveillance applications including acomparative assessment of performance relative to the Sedia LAg assay previously evaluated on thesame large blinded specimen panel by our group [5 6]

In order to assess real-world performance we adopted the previously-proposed definition ofoptimal performance as the precision of incidence estimates obtainable using the algorithm underevaluationmdashimplying a trade-off betweenMDRI and FRR [7]mdashin specified surveillance scenariosFRR is inherently context-dependent depending strongly on epidemiological factors such as theantiretroviral treatment coverage abundance of elite controllers (spontaneous virus suppression)and distribution of times-since-infection in the surveyed population (see [8]) MDRI largely reflectsthe biological properties of the test (ie the post-infection biomarker dynamics) but is also affected

2

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

by the specific screening assay used in a survey to ascertain HIV-positivity the subtype mix in thepopulation etc We therefore defined three hypothetical surveillance scenarios based on real-worldsettings and evaluated performance of the Maxim and LAg assays under the assumptions definingthe scenarios

2 Methods

21 The CEPHIA Evaluation PanelThe CEPHIA specimen repository houses more than 29000 unique specimens from over 3000HIV-1-positive subjects The CEPHIA Evaluation Panel (EP) consists of 2500 plasma specimens [56] that were obtained from 928 unique subjects (1ndash13 specimens per subject) spanning a wide rangeof times since infection Most specimens are from subjects with HIV subtype B (57 of specimens)C (27) A1 (10) and D (5) The panel further contains multiple blinded aliquots of 3 controlspecimens (25 replicates of each) with antibody reactivity characteristic of recent intermediate andlong-standing infection to allow evaluation of the reproducibility of assay results and moderatenumbers of specimens from ART-suppressed and naturally suppressed (elite controller) participantsto assess the impact of viral suppression on FRR

The majority of subjects contributing specimens to the panel (68) had sufficient clinical back-ground data to produce Estimated Dates of Detectable Infection (EDDIs) which are obtained bysystematically interpreting diverse diagnostic testing histories into infection date lsquopoint estimatesrsquo(EDDIs) and plausible intervals of first detectability according to the method previously described[9] EDDIs represent the date on which a viral load assay with a 50 limit of detection (LoD) of 1RNA copymL would be expected to first detect the infection and consequently MDRI estimatesare lsquoanchoredrsquo to this reference test

The UCSF Institutional Review Board approved the CEPHIA study procedures (approval 10-02365) and all specimens were collected under IRB-approved research protocols

22 Laboratory ProceduresThe CEPHIA EP was tested with the Maxim and Sediatrade HIV-1 Limiting Antigen Avidity EIAassays (Maxim and Sedia LAg respectively) according to their respective product inserts [10 11]Both assays are microtitre-based with the solid phase of the microtitre plate coated with a multi-subtype recombinant HIV-1 antigen This antigen is coated in a limiting concentration to pre-vent crosslinking of antibody binding thereby making it easier to remove weakly-bound antibodySpecimen dilutions are incubated for 60 minutes and then a disassociation buffer is added for 15minutes to remove any weakly-bound antibody A goat anti-human horseradish peroxidase (HRP)-conjugated IgG is added and this binds to any remaining IgG a TMB substrate is added and a col-our is generated which is proportionate to the amount of HRP An optical density (OD) is measuredfor each sample and this is normalized by use of a calibrator specimen On each plate the calib-rator is tested in triplicate with the median of the three ODs used to normalize specimen readingsproducing normalized optical density (ODn) measurements

The procedures for both assays are essentially the same and both manufacturers source the re-combinant antigen from the CDC as part of their licensing agreement However other componentsof the assay such as the type of plates used the control and calibrator materials etc were sourced

3

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

or produced by the individual manufacturers The testing procedure for both assays requires thatspecimens producing an initial lsquoscreeningrsquo OD of le20 be subjected triplicate lsquoconfirmatoryrsquo testingThe median ODn of the triplicate results then serves as the final result [10 11] In the Maxim eval-uation a small number of specimens erroneously did not receive the triplicate confirmatory testing(940 out of 952 that should have received confirmatory testing did and 12 did not) A simulationinvestigation showed that this minor protocol deviation did not substantially affect our results Itis further recommended that specimens producing an initial ODn le04 be subjected to serologicaltesting to confirm HIV infection All subjects contributing specimens to the CEPHIA panel wereconfirmed HIV-1-positive and this step could be omitted

Laboratory technicians were blinded to specimen background data during testing which foreach of the assays was completed in batches over a one month period using LAg kits procured fromthe relevant manufacturer at the same time

23 Statistical AnalysisWe evaluated a Recent Infection Testing Algorithm (RITA) which consists of a screening assay toascertain HIV infection followed by a single immunoassay (either Maxim LAg or Sedia LAg) asprimary recencymarker and a quantitative viral load Recent infection was defined as anODn belowa tunable threshold and a quantitative viral load above a tunable threshold In addition the RITArequires the specification of a cut-off time denoted 119879 with recent results obtained from individualsinfected for longer than 119879 defined as falsely recent [12] A large number of ODn thresholds areinvestigated in addition to the lsquostandardrsquo LAg threshold of ODn le15 Performance without asupplemental viral load and viral load thresholds of 0 (meaning that no threshold was applied butanalysis is restricted to specimens that have viral load data available) 75 400 1000 and 5000cmLwere investigated We used a value of 2 years for 119879 throughout

The performance of a test for recent infection in cross-sectional incidence estimation is reflectedin two key characteristics the MDRI and FRR These characteristics and methods for estimatingthem are described in greater detail elsewhere [12 5 8]

Briefly MDRI is the average time an individual spends in the lsquorecent infectionrsquo state as definedby the biomarker or set of biomarkers having been infected for less than 119879 We estimatedMDRI byfitting binomial regression models for the probability of exhibiting the recent marker as a function oftime since detectable infection 119905 and integrated this function119875119877(119905) from 0 to119879 Confidence intervalswere estimated by means of subject-level bootstrap resampling (10000 iterations) MDRI may besensitive to HIV-1 subtype which affects post-infection antibody dynamics [5 6 8] so MDRIswere derived separately for different HIV subtypes in addition to an overall MDRI reflecting theCEPHIA evaluation panel and lsquoblendedrsquo MDRIs (weighted averages of subtype-specific MDRIs)for each surveillance setting

FRR (also referred to as the false-recent ratio) is simply the proportion of individuals infected forlonger than 119879 who nevertheless exhibit the lsquorecentrsquo biomarker The precision of incidence estimatesare highly sensitive to FRR and in most cases values above about 1 result in poor precision Asnoted earlier FRR depends strongly on context since viral suppression either as a result of antiret-roviral treatment or spontaneous suppression frequently results in partial seroreversion which leadsto the production of false-recent results on serological markers Inclusion of viral load in a RITA (ieviral load less than some threshold results in classification as long-term infection irrespective of ODnresult) ameliorates the impact of viral suppression It is important to consider a range of RITAs in-

4

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

cluding multiple ODn and viral load threshold combinations In practice a viral load threshold ofgt1000cmL is frequently used especially when dried blood spot (DBS) specimens are collected forrecency ascertainment which makes quantification of viral RNA at lower concentrations difficultNaiumlve FRR estimates (ie not adated to epidemiological context) and their confidence intervalswere obtained by estimating the binomial probability that an untreated individual would produce arecent result when infected for longer than 119879

To evaluate performance for surveillance purposes we estimated context-specific MDRI andFRR in three hypothetical (but realistic) scenarios The three scenarios were defined as follows

Scenario A (South Africa-like epidemic) 100 subtype C infection HIV prevalence of 189(SE 112) Incidence of 0990 cases100 person-years (SE 00004) ART coverage and viral sup-pression rates of 56 (SE 56) and 82 (SE 82) respectively survey sample size of 35000Scenario B (Kenya-like epidemic) 70 subtype A 25 subtype D and 5 subtype C HIV pre-valence of 54 (SE 036) Incidence of 0146 cases100PY (SE 0039) ART coverage and viralsuppression rates of 64 (SE 64) and 81 (SE 81) respectively survey sample size of 14000Scenario C (North American key population-like epidemic) 100 subtype B HIV prevalence of150 (SE 100) Incidence of 05 cases100PY (SE 0050) ART coverage and viral suppressionrates of 900 (SE 90) and 900 (SE 90) survey sample size of 5000 In all scenarios weassumed that we were able to classify all treated subjects as long-term using ART exposure testingTherefore the FRR in treated subjects (both suppressed and unsuppressed) is 0 This is very favour-able to the RITA but we relaxed this assumption in a sensitivity analysis reported in Figures A4ndashA6in the Appendix The screening assay was a laboratory assay with an average lsquodiagnostic delayrsquo of107 days relative to the 1cmL reference test to which infection date estimates are anchored asdescribed above

To obtain context-specific FRR estimates denoted 120598119879 we estimated FRR in untreated individu-als by fitting 119875119877(119905) for all times post-infection and weighted it by the probability density function oftimes-since-infection amongst the untreated population 120588(119905) the latter parameterized as a Weibullsurvival function whose shape and scale parameters were chosen to produce a weighting functionconsistent with prevalence and treatment coverage and normalized to recent incidence We es-timated the FRR in treated individuals 119875119877|119905119909 as the binomial probability that a treated individualinfected for longer than 119879 tests recent We then obtain a weighted FRR estimate as shown in equa-tion 1 below

120598119879 = 119888 sdot 119875119877|119905119909 + (1 minus 119888) sdot 1114009inf

119879120588(119905)119875119877(119905) 119889119905 (1)

where 119888 is the treatment coverage

120588(119905) = 119891(119905)intinf

119879119891(119905) 119889119905

(2)

and

119891(119905) = exp(minus( 119905120572)120573) (3)

5

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

with 120572 and 120573 in equation 3 the Weibull scale and shape parameters respectively This approachwas previously described in [8] and [13]

While we have declared hypothetical scenarios in which epidemiological parameters are lsquoknownrsquowe demonstrate the procedure that would be recommended in real-world settings by taking intoaccount uncertainty in these parameters To evaluate reproducibility of FRR estimates we boot-strapped (30000 iterations) both the calibration data and contextual parameters the latter drawnfrom truncated normal distributions with means and standard deviations as defined for the scenariosabove

Performance was defined as the precision of incidence estimates ie the relative standard error(RSE) on the incidence estimate given the epidemiological parameters survey sample size (assumingsimple random sampling) and test characteristics (incorporating uncertainty in these) estimated foreach scenario using a range of ODn thresholds and supplemental viral load thresholds with theinctools R package [14] and extensions thereto [15]

3 Results

31 Calibrators and reproducibility on replicate control specimensAs reported in Table 1 the meanOD for all Maxim LAg calibrators was 075 and for Sedia LAg was065 a difference of about 154 of the lower Sedia value When restricted to only the calibratorsused for normalizationmdashie the median value of the three ODs obtained from triplicate testingon each platemdashthe coefficients of variation (CVs) of Maxim and Sedia calibrators were 93 and142 respectively As shown in Figure 1 Maxim calibrators produced meaningfully higher ODreadings than the Sedia calibrators with a difference in mean OD of 0107 (95 CI 00900123)and a p-value obtained using a Welch two-sample t-test lt0001

Reproducibility on the three blinded control specimens was similar with CVs on OD and ODn(across 25 replicates) slightly higher for Sedia The Maxim assay produced lower ODn values onaverage and a much lower mean ODn on the low-reactivity specimen (labelled BC-3) of 054 vs102 on the Sedia assay In accordance with the manufacturesrsquo instructions for use specimen BC-3 was subjected to triplicate confirmatory testing on both assays The reported ODs were thoseobtained from the initial screening runs and the mean and CV on ODn results were computed onthe 25 final values

32 Performance on clinical specimensFigure 2 shows results of testing clinical specimens in the CEPHIA EP and the impact of thepreviously-noted higher Maxim calibrator readings ODn values in Figure 2B are concentratedbelow the diagonal line especially at lower ODn values in the range of plausible recency discrim-ination thresholds In fact correlation was stronger for non-normalized OD readings than for nor-malized ODn readings The slope for OD in Figure 2A (1048 95 CI 10371059) is closer tounity than the slope for ODn in Figure 2B (0899 95 CI 08880911) The linear regressionshown in Figure 2B was restricted to ODn values below 6 The Bland-Altman plots in Figure 2Cand 2D show that the Maxim assay tends to produce lower OD readings than the Sedia assay on thelow end of the dynamic range and higher readings at the top end When the calibrators are usedto normalize Maxim ODn values exhibit clear downward bias throughout the dynamic range

6

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure 1 Density plot of Maxim and Sedia calibrator ODs

0

2

4

6

00 02 04 06 08 10 12

Calibrator OD

Den

sity

Maxim Sedia

MDRI was estimated using treatment-naiumlve non-elite controller subjects with EDDI intervalsle120 days Using ODn le15 the MDRI for Maxim LAg without using a supplemental viral loadthreshold was 248 days (95 CI 224274) while the MDRI for Sedia LAg was 215 days (95 CI192241) This resulted in a statistically significant estimated MDRI difference of 327 days (95CI 229428) Applying a supplemental viral load threshold of gt1000cmL resulted in MDRIestimates of 201 days (95 CI 180224) and 171 days (95 CI 152191) for Maxim and Sediarespectively with an MDRI difference estimated at 309 days (95 CI 217407)

Table 2 shows MDRI estimates for all subtypes and by subtype (B C D and A1) for a rangeof ODn thresholds in combination with a viral load threshold (gt1000cmL) We did not observestatistically significant differences between subtype-specificMDRI estimates and the estimates for allother subtypes (using a two-sample 119885-test) for either assay at any ODn threshold MDRI estimatesfor a wider range of ODn and viral load thresholds are reported in the Appendix (Table A1)

TheMDRI-against-threshold and (naiumlvely estimated) FRR-against-threshold trajectories for thetwo assays were similar as shown in Figures A1 and A2 in the Appendix although the values atany given threshold differed (The MDRI calibration curve has a characteristic kink at an ODnthreshold of 20 the threshold that triggers confirmatory testing This shape is expected sincethere are more specimens with higher intrinsic reactivity which through random fluctuation pro-duce initial ODn values under the threshold and upon confirmatory testing is assigned a final valueabovemdashthus depressing the MDRI up to the retesting threshold)

7

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Table 1 Calibrator reactivity and reproducibility of results (assessed by repeattesting)

SpecimenMaxim Sedia

N MeanOD

CVOD()

MeanODn

CVODn()

N MeanOD

CVOD()

MeanODn

CVODn()

Calibrators

Alla 222 075 104 219 065 150

Median valuesb 74 075 93 73 064 142

Kit-supplied control specimens

Acute (low) 222 036 105 049 80 219 035 162 055 140

Chronic (high) 222 137 86 183 79 219 131 104 206 127

Blinded control specimens

BC-1 25 330 51 445 93 25 307 56 494 150

BC-2 25 304 59 408 89 25 283 62 45 132

BC-3 25 040 142 054 148 25 067 200 102 136aAverage over all calibrator values

bAverage over median calibrator values (one value per plate)

8

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure 2 Maxim vs Sedia OD and ODn measurements

y = minus0038 + 1048x

R2 = 0933

00

05

10

15

20

25

30

35

40

00 05 10 15 20 25 30 35 40Sedia OD

Max

im O

D

A

y = minus0036 + 0899x

R2 = 0908

00

05

10

15

20

25

30

35

40

45

50

55

60

00 05 10 15 20 25 30 35 40 45 50 55 60Sedia ODn

Max

im O

Dn

B

minus25

minus20

minus15

minus10

minus05

00

05

10

15

20

25

0 1 2 3 4Mean OD

Diff

eren

ce (

Max

im minus

Sed

ia)

C

minus25

minus20

minus15

minus10

minus05

00

05

10

15

20

25

0 2 4 6Mean ODn

Diff

eren

ce (

Max

im minus

Sed

ia)

D

AMaxim vs Sedia Optical Density (OD) BMaxim vs Sedia normalized Optical Density (ODn)The blue lines are linear regression fits and the red dashed lines show the diagonal (slope if the two assays produced

equivalent results)C Bland-Altman plot for Optical Density (OD) D Bland-Altman plot for normalized Optical Density (ODn)

The red lines represent zero bias the blue solid lines the mean differences and the blue dashed lines the 95 lower andupper limits

9

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Table 2 MDRI estimates for Maxim and Sedia LAg assays by HIV-1 subtype andODn threshold using supplemental viral load threshold of gt1000cmL

HIV Subtypes ODnleMaxim Sedia

MDRI (95 CI) p-value MDRI (95 CI) p-value

All 10 156 (139176) 122 (106138)

All 15 201 (180223) 171 (152191)

All 20 244 (220268) 204 (183227)

All 25 321 (294350) 278 (252305)

B 10 154 (119203) 0907 127 (91175) 0788

B 15 203 (162255) 0895 176 (132226) 0871

B 20 240 (191295) 0969 204 (160257) 0949

B 25 299 (245357) 0474 250 (201307) 0307

C 10 151 (130175) 0586 112 (97131) 0222

C 15 197 (170226) 0708 162 (141185) 0357

C 20 239 (207272) 0728 197 (170225) 0528

C 25 323 (285363) 0943 283 (245321) 0809

D 10 192 (109292) 0406 166 (86262) 0263

D 15 223 (140321) 0617 209 (126307) 0375

D 20 250 (164350) 0901 241 (152347) 0403

D 25 298 (203406) 0597 281 (186391) 0979

A1 10 182 (133240) 0340 147 (107192) 0240

A1 15 203 (148265) 0914 186 (137245) 0555

A1 20 261 (198332) 0536 205 (150268) 0950

A1 25 369 (299435) 0127 323 (258386) 0151 To obtain these p-values we compare HIV-1 subtype-specific MDRI with the MDRI for all other subtypes at therelevant ODn threshold using a two-sided Z-test

While naiumlvely-estimated FRRs at a given threshold were not identical between the Maxim andSedia assays the differences were not statistically significant The FRRs in ART-naiumlve subjects were326 and 217 for Maxim and Sedia respectively at ODn le15 without using supplementalviral load In combination with a viral load threshold of gt1000cmL the FRRs were 169 and112 respectively These estimates are shown in Table A1 in the Appendix Among treatedsubjects FRRs were extremely high when the RITA did not include a viral load threshold In early-treated subjects (time from infection to treatment initiation le6 months) the FRRs for Maxim andSedia were 98 and 96 respectively and in later-treated subjects (time from infection to treatmentinitiation gt6 months) FRRs were 38 vs 33 respectively Using a supplemental viral load

10

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

threshold reduced these FRRs to 0 given that all treated subjects in the CEPHIA panel were virallysuppressed

33 Performance in surveillanceThe performance of the two assays in the three hypothetical surveillance scenarios defined earlierare summarised in Figures 3 and 4

Figure 3 shows context-specific FRRagainstMDRI for RITAs that included a viral load thresholdof gt1000cmL and where we assume that ART exposure testing reduces false recency in treatedsubjects to zero Note that the MDRI values on the x-axis encode different ODn thresholds for thetwo assays The FRR rises at slightly lower MDRIs for Sedia-based RITAs than for Maxim-basedRITAs in all three scenarios To maintain FRRs below 2 both assays require a choice of ODnthreshold that produces maximal MDRIs of about 400 to 450 days In the Appendix we showcontext-adapted FRRs against ODn thresholds (Figure A3) and we relax the assumption that ARTexposure testing performs perfectly instead assuming that it reduces false recency in treated subjectsby an order of magnitude compared to that obtained in treated subjects when no supplementaryviral load threshold is used This is shown in Figures A4 and A5

Figure 4 shows the precision of the incidence estimate attained for a range of ODn thresholdsin combination with a viral load threshold of gt1000cmL At each ODn threshold assay-specificcontext-adaptedMDRIs and FRRswere computed for use in the incidence calculation In the SouthAfrica-like scenario Figure 4A the lowest value of RSE on incidence attained with the Maxim-based algorithm was 117 at ODn le30 and with the Sedia-based algorithm was 120 at thesame ODn threshold In the Kenya-like scenario Figure 4B the minimal RSE for Maxim was272 achieved at ODn le275 and for Sedia was 282 at ODn le30 In the North Americankey population-like scenario Figure 4C the lowest RSE for Maxim was 269 at ODn le325 andfor Sedia was 289 atODnle30 These nominally optimal thresholds were slightly different underthe alternative assumptions shown in the Appendix (Figure A4) Context-specific MDRI and FRRestimates and RSEs on incidence estimates are reported in Table A2 in the Appendix

4 DiscussionThe Maxim and Sedia LAg assays produce meaningfully different ODn results on the same speci-mens largely as a result of higher calibrator readings obtained from the Maxim-supplied kit calib-rators and consequently at any given ODn threshold RITAs based on the two assays have differentMDRIs and FRRs It is inappropriate to utilize published MDRI and FRR estimates for one assayin survey planning and incidence estimation where the other assay is being used or to switch fromone assay to the other within a study

It is possible to derive an approximate conversion factor (of 1172) between ODn values of thetwo assays from the slopes of the regression curves shown in Figure 2A and 2B It has further beensuggested that a threshold of 15 on Maxim is equivalent to a threshold of 20 on Sedia based ontesting of a set of specimens with reactivity spanning the dynamic range with both assays (personalcommunication B Parekh) Our analysis does indeed show that these thresholds yield very similarMDRIs when used alone (248 days vs 254 days) but the FRRs are also different Applying a con-version factor to the Sedia results of repeat-tested specimens does not perfectly predict the Maxim

11

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure 3 Context-specific false-recent rate (FRR) against MDRI in threedemonstrative surveillance scenarios

0

2

4

6

100 200 300 400 500

MDRI

FR

R (

)

A

0

2

4

6

100 200 300 400 500

MDRI

FR

R (

)

B

0

2

4

6

100 200 300 400 500

MDRI

FR

R (

)

C

Assay

Maxim Sedia

A supplementary viral load threshold of gt1000cmL is used throughout We assume ARV exposure testing classifies alltreated individuals as long-term This assumption is relaxed in Figure A5A Scenario similar to South African epidemic B Scenario similar to Kenyan epidemic C Scenario similar to NorthAmerican key population epidemic

ODn values obtained and a preferable approach is therefore to use appropriately-estimatedMDRIsand FRRs for any given RITA based on either assay

Our reproducibility analyses show little benefit to the normalization procedure with both theMaxim and Sedia assays showing greater variability in ODn values than in the raw optical dens-ities on blinded replicate specimens subjected to repeat testing Further the correlation between

12

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure 4 Relative standard error (RSE) of incidence estimate against ODn thresholdin three demonstrative surveillance scenarios

10

20

30

40

50

60

05 10 15 20 25 30 35 40

ODn Threshold

RS

E o

n in

cide

nce

()

A

10

20

30

40

50

60

05 10 15 20 25 30 35 40

ODn Threshold

RS

E o

n in

cide

nce

()

B

10

20

30

40

50

60

05 10 15 20 25 30 35 40

ODn Threshold

RS

E o

n in

cide

nce

()

C

Assay

Maxim Sedia

A supplementary viral load threshold of gt1000cmL is used throughout We assume ARV exposure testing classifies alltreated individuals as long-term This assumption is relaxed in Figure A6A Scenario similar to South African epidemic B Scenario similar to Kenyan epidemic C Scenario similar to NorthAmerican key population epidemic

Maxim and Sedia optical densities was greater than between ODn measurements on the same spe-cimens However at the time of each of these evaluations kits and reagents were sourced at thesame time kits were from a small number of lots and operators were highly experienced with as-says The purpose of the calibrators and normalization procedure is to reduce lot-to-lot variabilityand ensure stability of results over time and between manufacturers and laboratories This goal

13

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

requires that calibrators be highly consistent over time and between manufacturers which is notcurrently the case Fourteen laboratories participate in the NIAID-supported External Quality As-surance Program Oversight Laboratory (EQAPOL) proficiency testing program This programassessed consistency of results between and within assay manufacturers kit lots and participatinglaboratories using panels designed for quality assurance and proficiency testing and found similardifferences in calibrator reactivity and average ODn values between the two assays (Keating et alforthcoming) External quality assurance is critical for ensuring consistency between laboratoriesand kit manufacturers

It should be noted that our evaluation of both assays was restricted to plasma specimens Bothmanufacturers also produce kits for use with dried blood spot eluates and it has been shown thatspecimen type further impacts performance [16]

We did not observe any statistically significant subtype effects on MDRI although point estim-ates differed substantially especially with specimens from subtype D-infected subjects comparedto subtypes B and C (Table 2) With a larger dataset and more precise MDRI estimates subtypedifferences may be visible

Despite the systematic differences in calibrator readings and consequently in the ODn valuesobtained performance of the two assays for incidence surveillance was virtually indistinguishablemdashas long as appropriate assay- and context-specific MDRI and FRR estimates were used As a resultdifferent ODn thresholds were nominally optimal (ie produced the lowest variance on the incid-ence estimate) In all three hypothetical surveillance scenarios ODn thresholds between about 15and 30 (in combination with viral load) produced the best precision It is critical however thatappropriate MDRI and FRR estimates be used for the recency discrimination threshold chosenin order to obtain accurate incidence estimates It should also be noted that the triplicate lsquocon-firmatoryrsquo testing protocol mandates confirmatory testing when an initial ODn result is below 20which may be problematic for RITAs that use ODn thresholds above the lsquostandardrsquo threshold of15 It would also be a different subset of specimens reflexed to confirmatory testing on the twoassays Consideration should be given to a modified testing protocol in which confirmatory testingis performed on a larger subset of (or even all) specimens

A limitation of this study is that we did not have access to specimens from virally unsuppressedtreated subjects and we are therefore unable to rigorously estimate FRR in this group which maybe substantial in many surveillance settings [17] We urge survey planners and analysts to con-duct sensitivity analyses with respect to FRR when utilising either assay in cross-sectional incidenceestimation

Differences inODnmeasurements between theMaxim and Sedia LAg assays on the same speci-mens largely resulted from differences in the reactivity of calibrators supplied by the manufacturersThis resulted in systematically lower ODn measurements on the Maxim assay than on the Sediaassay and consequently longer MDRIs and larger FRRs at any given ODn recency discriminationthreshold While performance for surveillance purposes was extremely similar different thresholdswere optimal and different values of MDRI and FRR were appropriate for use in survey planningand incidence estimation The two assays cannot be treated as interchangeable should not bemixedwithin one study and care should be taken when interpreting and comparing results We summarizeour recommendations based on this comparative evaluation in Table 3

14

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Table 3 Summary recommendations for use of the Maxim and Sedia LAg assays

Issue Recommendation

Laboratory methods Assay procedures are similar butnot identical

Testing laboratories should ensurefull compliance with

manufacturerrsquos instructions for useespecially if both manufacturersrsquoassays are used in one laboratory

Quality assurance Lot-to-lot variation and differencesin laboratory staff proficiency mayfurther reduce reproducibility of

results

Participation in an external qualityassurance programme like

EQAPOL [18] is recommended

Software Although data capture and analysissoftware are similar interpretivecriteria for specific components

differ

The data analysis software isspecific to each assay and

laboratories should use the softwaresupplied by the manufacturer

Conversion Although it is possible to computean approximate conversion factorthis does not perfectly predict

equivalent ODn values

Rather than converting resultsappropriately-derived MDRI andFRR estimates should be utilizedfor each assay The same ODnthresholds may not be optimal

Descriptive title The names lsquoHIV-1 LimitingAntigen Avidity EIArsquo or rsquoLAg assaylsquodo not distinguish between the two

assays

Users should clearly identify themanufacturer of the kits used aswell as specimen type in allpublications and reports

Assay performance Despite differences in calibratorreactivity and consequently in

ODn values obtained on the samespecimens performance of the twoassays for surveillance purposes was

virtually indistinguishable

Both manufacturersrsquo assays aresuitable for use but they should not

be mixed within studiesappropriate performance

characteristic estimates must beused and care should be taken

when comparing results

Author contributionsJBS and EG analysed the data JBS GM and EG wrote the article GM managed the labor-atory testing JH conducted laboratory testing SNF DH SMK KM and EG managed thespecimen repository and clinical data NP provided scientific support and managed the primarygrant GM AW CDP and MPB were principal investigators for the study and provided inputon analysis and writing All authors reviewed and provided input on the article

15

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

AcknowledgmentsThe authors gratefully acknowledge the Centre for High Performance Computing which providedcomputational resources for this study

The Consortium for the Evaluation and Performance of HIV Incidence Assays (CEPHIA) com-prises Oliver Laeyendecker Thomas Quinn David Burns (National Institutes of Health) AlexWelte Joseph Sempa previously David Matten Hilmarieacute Brand Trust Chibawara (South AfricanCentre for EpidemiologicalModelling and Analysis) ReshmaKassanjee (University of Cape Town)Gary Murphy Jake Hall previously Elaine Mckinney (Public Health England) Michael P BuschEduard Grebe Shelley Facente Sheila Keating Dylan Hampton previously Mila Lebedeva (Vi-talant Research Institute formerly Blood Systems Research Institute) Christopher D Pilcher KaraMarson (University of California San Francisco) Susan Little (University of California San Diego)Anita Sands (World Health Organization) Tim Hallett (Imperial College London) Sherry MicheleOwen Bharat Parekh Connie Sexton (Centers for Disease Control and Prevention) Matthew PriceAnatoli Kamali (International AIDSVaccine Initiative) Lisa Loeb (TheOptions StudymdashUniversityof California San Francisco) Jeffrey Martin Steven G Deeks Rebecca Hoh (The SCOPE StudymdashUniversity of California San Francisco) Zelinda Bartolomei Natalia Cerqueira (The AMPLIARCohortmdashUniversity of Satildeo Paulo) Breno Santos Kellin Zabtoski Rita de Cassia Alves Lira (TheAMPLIARCohortmdashGrupoHospital Conceiccedilatildeo) Rosa Dea Sperhacke Leonardo RMotta Mach-line Paganella (The AMPLIAR CohortmdashUniversidade Caxias Do Sul) Esper Kallas Helena Tom-iyama Claudia Tomiyama Priscilla Costa Maria A Nunes Gisele Reis Mariana M Sauer Nat-alia Cerqueira Zelinda Nakagawa Lilian Ferrari Ana P Amaral Karine Milani (The Satildeo PauloCohortmdashUniversity of Satildeo Paulo Brazil) Salim S Abdool Karim Quarraisha Abdool KarimThumbi Ndungu Nelisile Majola Natasha Samsunder (CAPRISA University of Kwazulu-Natal)Denise Naniche (The GAMA StudymdashBarcelona Centre for International Health Research) InaacutecioMandomando Eusebio VMacete (The GAMA StudymdashFundacaoManhica) Jorge Sanchez JavierLama (SABES CohortmdashAsociacioacuten Civil Impacta Salud y Educacioacuten (IMPACTA)) Ann Duerr(The Fred Hutchinson Cancer Research Center) Maria R Capobianchi (National Institute for In-fectious Diseases ldquoL Spallanzanirdquo Rome) Barbara Suligoi (Istituto Superiore di Sanitagrave Rome)Susan Stramer (American Red Cross) Phillip Williamson (Creative Testing Solutions VitalantResearch Institute) Marion Vermeulen (South African National Blood Service) and Ester Sabino(Hemocentro do Satildeo Paolo) General inquiries may be directed to GaryMurphyphegovuk

Funding amp conflicts of interestCEPHIA was supported by grants from the Bill and Melinda Gates Foundation (OPP1017716OPP1062806 and OPP1115799) Additional support for analysis was provided by a grant fromthe US National Institutes of Health (R34 MH096606) and by the South African Department ofScience and Technology and the National Research Foundation Specimen and data collectionwere funded in part by grants from the NIH (P01 AI071713 R01 HD074511 P30 AI027763R24 AI067039 U01 AI043638 P01 AI074621 and R24 AI106039) the HIV Prevention TrialsNetwork (HPTN) sponsored by the NIAID National Institutes of Child Health and Human Devel-opment (NICHHD) National Institute on Drug Abuse National Institute of Mental Health andOffice of AIDS Research of the NIH DHHS (UM1 AI068613 and R01 AI095068) the Califor-

16

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

nia HIV-1 Research Program (RN07-SD-702) Brazilian Program for STD and AIDS Ministry ofHealth (914BRA3014-UNESCO) and the Satildeo Paulo City Health Department (2004-0168922ndash7) MAP and selected samples from IAVI-supported cohorts are funded by IAVI with the generoussupport of USAID and other donors a full list of IAVI donors is available at wwwiaviorg

MPB EG SNF DH AW and GM receive grant support from andor have a consultingagreement with Sedia Biosciences Corporation for evaluation of a separate assay

References[1] Duong YT Qiu M De AK Jackson K Dobbs T Kim AA et al Detection of Re-

cent HIV-1 Infection Using a New Limiting-Antigen Avidity Assay Potential for HIV-1 Incidence Estimates and Avidity Maturation Studies PLOS ONE 20127(3)1ndash9doi101371journalpone0033328

[2] Gao Z Yan H Feng X Wu L Qiu M Xing W et al Development of a New Limiting-Antigen Avidity Dot Immuno-Gold Filtration Assay for HIV-1 Incidence PLOS ONE201611(8)e0161183 doi101371journalpone0161183

[3] Duong YT Kassanjee R Welte A Morgan M De A Dobbs T et al Recalibration of theLimiting Antigen Avidity EIA to Determine Mean Duration of Recent Infection in DivergentHIV-1 Subtypes PLOS ONE 201510(2)1ndash15 doi101371journalpone0114947

[4] Schlusser KE Konikoff J Kirkpatrick AR Morrison C Chipato T Chen PL et al ShortCommunication Comparison of Maxim and Sedia Limiting Antigen Assay Performance forMeasuring HIV Incidence AIDS Research and Human Retroviruses 201733(6)555ndash557doi101089aid20160245

[5] Kassanjee R Pilcher CD Keating SM Facente SN McKinney E Price MA et al Independ-ent assessment of candidate HIV incidence assays on specimens in the CEPHIA repositoryAIDS 201428(16)2439ndash2449 doi101097QAD0000000000000429

[6] Kassanjee R Pilcher CD Busch MP Murphy G Facente SN Keating SM et al Viral loadcriteria and threshold optimization to improve HIV incidence assay characteristics AIDS201630(15)2361ndash2371 doi101097QAD0000000000001209

[7] Kassanjee R McWalter TA Welte A Short Communication Defining Optimality of a Testfor Recent Infection for HIV Incidence Surveillance AIDS Research and Human Retrovir-uses 201430(1)45ndash49 doi101089aid20130113

[8] Grebe E Welte A Hall J Keating SM Facente SN Marson K et al Infection Sta-ging and Incidence Surveillance Applications of High Dynamic Range Diagnostic Immuno-Assay Platforms Journal of Acquired Immune Deficiency Syndromes 201776(5)547ndash555doi101097QAI0000000000001537

[9] Grebe E Facente SN Bingham J Pilcher CD Powrie A Gerber J et al Interpreting Dia-gnostic Histories into HIV Infection Time Estimates Framework and Online Tool bioRxiv2018 doi101101323808

17

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

[10] Maxim Biomedical MaximHIV-1 Limiting Antigen Avidity EIA SingleWell Avidity EnzymeImmunoassay for Detection of Recent HIV-1 Infection Cat No 92001 2013 Available fromhttpwwwmaximbiocomimginsert920013-Product-Insertpdf

[11] Sedia Biosciences Corporation Sediatrade HIV-1 LAg-Avidity EIA single well avidity enzymeimmunoassay for detection of recent HIV-1 infection Cat No 1002 2016 Available fromhttpwwwsediabiocomLiteratureRetrieveaspxID=134692

[12] Kassanjee R McWalter TA Baumlrnighausen T Welte A A new general biomarker-based incid-ence estimator Epidemiology 201223(5)721ndash728 doi101097EDE0b013e3182576c07

[13] Grebe E Welte A Johnson LF van Cutsem G Puren A Ellman T et al Population-level HIV incidence estimates using a combination of synthetic cohort and recency bio-marker approaches in KwaZulu-Natal South Africa PLOS ONE 201813(9)1ndash16doi101371journalpone0203638

[14] Grebe E Baumlumler PMcIntosh A Ongarello SWelte A Incidence Estimation Tools (inctools)2018 Available from httpsdoiorg105281zenodo1493401

[15] Grebe E Incidence (combined method) PLOS ONE Publication 2018 Available fromhttpsdoiorg105281zenodo1420559

[16] Schlusser KE Pilcher CD Kallas EG Santos BR Deeks SG Facente S et al Comparison ofcross-sectional HIV incidence assay results from dried blood spots and plasma PLOS ONE201712(2)1ndash10 doi101371journalpone0172283

[17] WHO HIV drug resistance report 2017 World Health Organization 2017 Available fromhttpswwwwhointhivpubdrugresistancehivdr-report-2017en

[18] Duke Human Vaccine Institute EQAPOL Limiting Antigen (LAg) Avidity Incidence AssayEQA Program 2019 Available from httpseqapoldhvidukeeduprogramslag

18

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

A Appendix

Table A1 MDRI and FRR estimates in ART-naiumlve subjects for a range of ODn andviral load thresholds

ODn VL Maxim Sedia

le gt MDRI (95 CI) FRR (95 CI) MDRI (95 CI) FRR (95 CI)

050 None 129 (113146) 16 (0347) 91 (79105) 05 (0030)075 None 164 (144184) 16 (0347) 127 (111145) 16 (0347)100 None 197 (176220) 22 (0655) 158 (138179) 16 (0347)125 None 224 (200248) 27 (0962) 194 (172217) 22 (0655)150 None 248 (224274) 33 (1270) 215 (192240) 22 (0655)175 None 277 (251304) 38 (1577) 237 (213263) 44 (1984)200 None 297 (270324) 65 (34111) 254 (228280) 44 (1984)225 None 338 (311365) 76 (42124) 292 (265319) 60 (30104)250 None 382 (354410) 114 (72169) 333 (305362) 98 (59150)275 None 423 (394453) 169 (117231) 373 (344401) 130 (85188)300 None 471 (442500) 201 (146266) 427 (398457) 163 (113225)050 0 129 (113147) 17 (0449) 91 (79105) 06 (0031)050 75 118 (104132) 11 (0140) 83 (7394) 06 (0031)050 400 108 (96122) 06 (0031) 78 (6888) 00 (0021)050 1000 102 (90114) 06 (0031) 73 (6482) 00 (0021)050 5000 86 (7597) 06 (0031) 63 (5472) 00 (0021)075 0 165 (145186) 17 (0449) 127 (110146) 17 (0449)075 75 151 (133169) 11 (0140) 116 (102132) 06 (0031)075 400 138 (121155) 06 (0031) 106 (93120) 06 (0031)075 1000 127 (112144) 06 (0031) 99 (86112) 06 (0031)075 5000 106 (92122) 06 (0031) 83 (7196) 06 (0031)100 0 199 (177223) 23 (0657) 159 (139181) 17 (0449)100 75 183 (164205) 11 (0140) 145 (127164) 11 (0140)100 400 168 (149188) 11 (0140) 130 (114148) 06 (0031)100 1000 156 (139176) 11 (0140) 122 (106138) 06 (0031)100 5000 126 (109144) 11 (0140) 101 (86117) 06 (0031)125 0 226 (202251) 28 (0964) 197 (174221) 23 (0657)125 75 210 (188233) 17 (0449) 180 (161201) 17 (0449)125 400 191 (171212) 17 (0449) 164 (146184) 11 (0140)125 1000 179 (160199) 17 (0449) 153 (135172) 11 (0140)125 5000 141 (123160) 17 (0449) 122 (106140) 11 (0140)150 0 250 (225276) 34 (1372) 218 (194244) 23 (0657)150 75 233 (210256) 17 (0449) 202 (180224) 17 (0449)150 400 214 (193237) 17 (0449) 183 (163204) 11 (0140)150 1000 201 (180223) 17 (0449) 171 (152191) 11 (0140)150 5000 158 (138179) 17 (0449) 135 (118154) 11 (0140)175 0 279 (252306) 39 (1679) 239 (213266) 45 (2087)175 75 261 (236286) 17 (0449) 222 (199245) 23 (0657)175 400 242 (219266) 17 (0449) 203 (182225) 17 (0449)175 1000 227 (206251) 17 (0449) 190 (170211) 17 (0449)175 5000 175 (155198) 17 (0449) 148 (129168) 17 (0449)

19

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

200 0 299 (272326) 67 (35115) 254 (228282) 45 (2087)200 75 281 (255307) 51 (2394) 236 (212261) 28 (0964)200 400 261 (237286) 39 (1679) 217 (196241) 17 (0449)200 1000 244 (220268) 39 (1679) 204 (183227) 17 (0449)200 5000 188 (166210) 39 (1679) 160 (141180) 17 (0449)225 0 339 (312367) 79 (44128) 292 (265320) 62 (31108)225 75 321 (295348) 62 (31108) 274 (249300) 45 (2087)225 400 300 (276326) 45 (2087) 253 (230278) 34 (1372)225 1000 280 (255306) 45 (2087) 237 (214261) 34 (1372)225 5000 214 (190239) 39 (1679) 186 (164209) 28 (0964)250 0 383 (354411) 112 (70168) 335 (306364) 101 (61155)250 75 364 (336392) 96 (57149) 316 (289344) 84 (48135)250 400 343 (316371) 79 (44128) 296 (269322) 67 (35115)250 1000 321 (294350) 79 (44128) 278 (252305) 67 (35115)250 5000 252 (225280) 62 (31108) 220 (194246) 51 (2394)275 0 426 (397455) 174 (122238) 375 (346405) 135 (88194)275 75 406 (377434) 157 (107219) 357 (328386) 118 (75175)275 400 384 (355413) 135 (88194) 336 (308364) 101 (61155)275 1000 363 (334393) 135 (88194) 315 (287343) 96 (57149)275 5000 285 (256315) 107 (66162) 250 (223279) 62 (31108)300 0 475 (445505) 208 (151275) 431 (401461) 169 (117232)300 75 455 (426483) 191 (136257) 412 (382441) 152 (102213)300 400 430 (401459) 169 (117232) 391 (362420) 135 (88194)300 1000 408 (379437) 169 (117232) 369 (340396) 135 (88194)300 5000 326 (297358) 129 (84188) 296 (266327) 96 (57149)

Naiumlvely-estimated FRR ie not adapted to epidemiological context

20

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Table A2 Context-specific MDRI and FRR estimates from the three demonstrativesurveillance scenarios under different assumptions about impact of ARV exposure

testing on FRR

Assmuption 1 Assumption 2

Assay ODn Scenariosect MDRIdagger(95 CI)

FRR (95 CI) RSE FRR (95 CI) RSE

Maxim 05 A 88 (74104) 00 (0001) 167 06 (0124) 638Maxim 10 A 140 (119165) 03 (0008) 190 09 (0130) 468Maxim 15 A 186 (159214) 03 (0009) 156 10 (0233) 381Maxim 20 A 228 (196261) 04 (0110) 138 12 (0337) 342Maxim 25 A 313 (273353) 08 (0415) 118 17 (0744) 273Sedia 05 A 73 (6185) 00 (0000) 161 05 (0020) 635Sedia 10 A 101 (86120) 00 (0010) 147 07 (0126) 589Sedia 15 A 151 (130174) 01 (0002) 123 08 (0128) 421Sedia 20 A 186 (159215) 01 (0002) 117 08 (0130) 365Sedia 25 A 272 (234311) 07 (0313) 131 15 (0641) 300Maxim 05 B 88 (74104) 00 (0001) 167 06 (0124) 638Maxim 10 B 140 (119165) 03 (0008) 190 09 (0120) 468Maxim 15 B 186 (159214) 03 (0009) 156 10 (0233) 381Maxim 20 B 228 (196261) 04 (0110) 138 12 (0337) 342Maxim 25 B 313 (273353) 08 (0415) 118 17 (0744) 273Sedia 05 B 73 (6185) 00 (0000) 161 05 (0002) 635Sedia 10 B 101 (86120) 00 (0001) 147 07 (0126) 589Sedia 15 B 151 (130174) 01 (0002) 123 08 (0128) 421Sedia 20 B 186 (159215) 01 (0002) 117 08 (0130) 365Sedia 25 B 272 (234311) 07 (0313) 131 15 (0641) 299Maxim 05 C 88 (74104) 00 (0001) 167 06 (0124) 638Maxim 10 C 140 (119165) 03 (0008) 190 09 (0130) 468Maxim 15 C 186 (159214) 03 (0009) 156 10 (0233) 381Maxim 20 C 228 (196261) 04 (0110) 138 12 (0337) 342Maxim 25 C 313 (273353) 08 (0415) 118 17 (0744) 273Sedia 05 C 73 (6185) 00 (0000) 161 05 (0020) 635Sedia 10 C 101 (86120) 00 (0001) 147 07 (0126) 589Sedia 15 C 151 (130174) 01 (0002) 123 08 (0128) 420Sedia 20 C 186 (159215) 01 (0002) 117 08 (0130) 365Sedia 25 C 272 (234311) 07 (0313) 131 15 (0641) 299sectScenario A South Africa-like epidemic B Kenya-like epidemic C North American key population-like epidemicdaggerAdjusted for subtype mix and screening assayARV exposure testing classifies all treated subjects as long-termARV exposure testing reduces FRR in treated subjects to 10 of FRR without viral load

21

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure A1 MDRI point estimates against ODn thresholds at different supplementalviral load thresholds

0

100

200

300

400

500

600

0 1 2 3 4

ODn Threshold

MD

RI

Maxim

0

100

200

300

400

500

600

0 1 2 3 4

ODn ThresholdM

DR

I

Sedia

VL (copiesmL) 0 75 400 1000 5000

AMaxim B Sedia

Figure A2 FRR estimates in ART-naiumlve patients against ODn threshold at differentsupplemental viral load thresholds

000

005

010

015

020

025

030

035

040

045

050

0 1 2 3 4

ODn Threshold

FR

R

A

000

005

010

015

020

025

030

035

040

045

050

0 1 2 3 4

ODn Threshold

FR

R

B

VL (copiesmL) none 75 400 1000 5000

AMaxim B Sedia

22

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure A3 Context-specific FRR vs ODn threshold in three demonstrativesurveillance scenarios (assuming perfect ARV exposure testing)

02468

101214

05 10 15 20 25 30 35 40

ODn Threshold

FR

R (

)

A

02468

101214

05 10 15 20 25 30 35 40

ODn Threshold

FR

R (

)

B

0

2

4

6

8

10

05 10 15 20 25 30 35 40

ODn Threshold

FR

R (

)

C

Assay

Maxim Sedia

A supplementary viral load threshold of gt1000cmL is used throughout We assume ARV exposure testing classifies alltreated individuals as long-termA Scenario similar to South African epidemic B Scenario similar to Kenyan epidemic C Scenario similar to NorthAmerican key population epidemic

23

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure A4 Context-specific FRR vs ODn threshold in three demonstrativesurveillance scenarios (assuming imperfect ARV exposure testing)

02468

101214

05 10 15 20 25 30 35 40

ODn Threshold

FR

R (

)

A

02468

101214

05 10 15 20 25 30 35 40

ODn Threshold

FR

R (

)

B

02468

101214

05 10 15 20 25 30 35 40

ODn Threshold

FR

R (

)

C

Assay

Maxim Sedia

A supplementary viral load threshold of gt1000cmL is used throughout We assume ARV exposure testing reduces falserecency in treated individuals to 10 of that attained when no supplemental viral load threshold is utilizedA Scenario similar to South African epidemic B Scenario similar to Kenyan epidemic C Scenario similar to NorthAmerican key population epidemic

24

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure A5 Context-specific false-recent rate (FRR) against MDRI in threedemonstrative surveillance scenarios (assuming imperfect ARV testing)

0

2

4

6

8

10

100 200 300 400 500

MDRI

FR

R (

)

A

0

2

4

6

8

10

100 200 300 400 500

MDRI

FR

R (

)

B

0

2

4

6

8

10

100 200 300 400 500

MDRI

FR

R (

)

C

Assay

Maxim Sedia

A supplementary viral load threshold of gt1000cmL is used throughout We assume ARV exposure testing reduces falserecency in treated individuals to 10 of that attained when no supplemental viral load threshold is utilizedA Scenario similar to South African epidemic B Scenario similar to Kenyan epidemic C Scenario similar to NorthAmerican key population epidemic

25

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint

Figure A6 Relative standard error (RSE) of incidence estimate against ODnthreshold in three demonstrative surveillance scenarios (assuming imperfect ARV

exposure testing)

20

30

40

50

60

70

80

05 10 15 20 25 30 35 40

ODn Threshold

RS

E o

n in

cide

nce

()

A

20406080

100120140160180

05 10 15 20 25 30 35 40

ODn Threshold

RS

E o

n in

cide

nce

()

B

20406080

100120140160180

05 10 15 20 25 30 35 40

ODn Threshold

RS

E o

n in

cide

nce

()

C

Assay

Maxim Sedia

Assuming imperfect ARV exposure testing which reduces the FRR in virally unsuppressed treated individuals to 10 ofFRR in suppressed individuals (where no viral load threshold applied)A Scenario similar to South African epidemic B Scenario similar to Kenyan epidemic C Scenario similar to NorthAmerican key population epidemic

26

CC-BY-NC-ND 40 International licenseacertified by peer review) is the authorfunder who has granted bioRxiv a license to display the preprint in perpetuity It is made available under

The copyright holder for this preprint (which was notthis version posted March 18 2019 httpsdoiorg101101572677doi bioRxiv preprint