The ACC/ESC Recommendation for 99th Percentile of the Reference Normal Overestimates the Risk of an Acute Myocardial Infarct

Jerard Kneifati-Hayek, Salman Haq, M.D, Madeleine Schlefer, Larry Bernstein, M.D.

New York Methodist Hospital Cornell-Weill

Background

A joint committee of the American College of Cardiology and European Society of Cardiology (ACC/ESC) has established the criteria for acute recent or evolving AMI predicated on

a typical increase in troponin

in the clinical setting of myocardial ischemia,

which includes the 99th percentile of a healthy normal population.

Problems in Recommendation

1. A “Reference normal population doesn’t present to the emergency room”

2. The cutoff used to make a decision has an effect on overuse of hospital resources, especially costly and limited “telemetry”

3. The cutoff “selection” is not scientifically validated

4. The cutoff selection is informed by a potential projected risk extending to perhaps four months after the presenting event

Recommendation Challenged

1. Lin JC, Apple FS, Murakami MM, Luepker RV. Rates of positive cardiac troponin I and creatine kinase MB mass among patients hospitalized for suspected acute coronary syndromes. Clin Chem 2004.

2. Zarich SW, Bradley K, Mayall ID, Bernstein, LH. Minor Elevations in Troponin T Values Enhance Risk Assessment in Emergency Department Patients with Suspected Myocardial Ischemia: Analysis of Novel Troponin T Cut-off Values. Clin Chim Acta 2004.

Hypothesis

An AMI is better identified by a TnI value exceeding the 99th percentile of patients

seen with acute coronary syndrome (ACS)

who are subsequently excluded

than using the ACC/ESC guideline.

Patient selection

1480 successive patients who presented to the emergency room at New York Methodist Hospital in two month periods in 2003 and 2004 and were required to have troponin I (TnI) were identified.

No Randomization

Observational study

Exclusion Criteria: Pacemaker, ST elevation, patients under 30 years old

Inclusion Criteria: ST depression, T wave inversion, LBBB, anginal equivalent chest pain, short breath

Patient Demographics

Women 56% Men 44%

46% white, 32% Negro, 16% Hispanic, 6% Asian.

ECG Findings: 24% were normal,

73.3% were nonspecific,

and 55.2% were other than NSSTT changes.

ECG findings of AMI were present 4.6 percent of the time.

Laboratory Methods

The CKMB and TnI were measured on the Centaur (Bayer, Tarrytown, NY) at the time of presentation and at least 4 hours later.

Data Collection

The information collected was –

serum cardiac marker concentration,

discharge diagnosis,

ECG finding,

chest pain characteristics.

all relevant cardiac diagnoses, including, CHF, atrial or ventricular dysrhythmias, and other relevant diagnoses, such as, hypothyroid disease, hypertension and type 2 diabetes were included.

Computerized ECGs were reviewed by two cardiologists

Reference Normal Population

A control population of 140 blood donors was also measured for TnI.

The control population TnI values were all less than 0.1 g/L.

Statistical evaluation

The data was organized for paired comparisons of troponin I and CKMB.

SPSS 11.5 (Chicago, IL) software was used for ROC curves and basic statistics.

SYSTAT 10.0 (Chicago, IL) was used for logistic regression,

Ordinal Regression, GOLDminerTM 3.0 (Statistical Innovation, Inc., Belmont, MA)

and Latent Class Cluster Analysis, Latent Gold 2.0 (ibid).

Two Step Procedure

1. Trial set to determine best decision value

Trial set was itself partitioned by an initial set for which half the charts were unavailable because of transition to a new information system

2. Second trial set had 619 patients

3. Validation set

ROC curve on 2nd Training Set

The area under the ROC curve is 99%. The best cutoff selected is 0.60 0.77 µg/L at a sensitivity of 98.9% and a false positive rate at 4%.

This is actually lower than the 99th percentile of the non-MI population and significantly higher than the 99th percentile of the healthy donor population, approximately the 97.5th percentile of the ruled-out MI population.

Receiver Operating Characteristic (ROC) Plot

0.0 0.2 0.4 0.6 0.8 1.0False Alarm Rate

0.0

0.2

0.4

0.6

0.8

1.0

Hit

Rat

e

TnI Sens 1 - Spec.21 1.000 .101 .46 .989 .056.22 1.000 .098 .49 .989 .055.23 1.000 .094 .52 .989 .052

.24 1.000 .089 .56 .989 .050.25 1.000 .085 .57 .989 .049.26 1.000 .082 .59 .989 .047.27 1.000 .081 .64 .989 .046.29 .989 .079 .69 .989 .045.30 .989 .075 .72 .989 .043.31 .989 .073 .73 .989 .042.32 .989 .072 .75 .989 .040.33 .989 .069 .77 .989 .039.35 .989 .068 .80 .977 .039.36 .989 .066 .82 .966 .037.37 .989 .062 .83 .966 .036.38 .989 .060 .84 .966 .035

.41 .989 .058      

TnI Sens 1 - Spec

Coordinates of the ROC curve

Condition Troponin I Comment

Not AMI (619) 0.00 Median

MI (70) > 0.61 g/L  

(49) Neg Pos  

CHF 6 2 2 CKMBs > 6.25

tachycardia 6 6 3 TnI above 3.3

Cardiomyopathy 4 1 1 TnI above 3.3

Acute renal failure

0 1  

Resp Failure 1 2 Respiratory failure

Sepsis 1 1 discordant

Hypothyroid 1 0 Hypothyroid

other 13 17  

Latent Class Analysis

A latent class model partitions the data into separate, nonoverlapping sets (the fit measured by chi square).

We assume that within each latent class, each variable is statistically independent of every other variable. This is referred to as “conditional independence”.

However, within a latent class that corresponds to a distinct medical syndrome, while the presence/ absence of one symptom is viewed as unrelated to presence/absence of all others, this may NOT be the case.

MI NMI

other

Cluster1 Cluster2Overall Probability

0.9162 0.0838 Not MI MI

TNISCA g/L0 1.0000 0.0000 < 0.3 1 0.9997 0.0003 0.3 – 0.6 2 0.0001 0.9999 0.61 – 1.2 3 0.0000 1.0000 > 1.2

Eight percent of patients are in cluster 2, consistent with MI. A cutoff value of less than 0.61 is consistent with classification of not MI.

In the previous two models we fitted the data to a binary classification using the diagnosis for training (ROC), or using the information in the data combinations (LCA) to cluster into MI or not MI at a cutoff of 0.61.

We next explored a method that allows us to look at the probability of MI using the diagnosis as a training value, and scaled values of troponin for the Y variable in an ORDINAL RESPONSE.

Ordinal Regression

The situation we describe is an ordered response with graded categories.

Magidson has developed a unified maximum likelihood methodology for simultaneously assessing the statistical significance of treatment effects and the model fit when the response variable contains ordered categories.

He explores the fit for different logit model extensions (log-odds) to data derived under the assumption of bivariate normality and finds that the “parallel log-odds” model based on adjacent odds often gives a better fit than the “proportional odds” model based on cumulative odds.

Frequency cross-table, probabilities and odds-ratios for scaled TnI versus expected diagnosis

TnI scaled

Range Not MI MI N Pct in MI

Prob by TnI

Odds Ratio

0 < 0.45 655 2 657 2 0 1

1 0.46-0.6 7 0 7 0 0.03 13

2 0.61-0.75

4 0 4 0. 0.26 175

3 0.76-0.9 13 59 72 57.3 0.82 2307

4 > 0.9 0 42 42 40.8 0.98 30482

    679 103 782 100    

There are 13 TnI values between 0.76-0.9 that are not MI, that would be falsely classified as MI at a cutoff below 0.76,

17 at a cutoff below 0.61,

and 24 at a cutoff below 0.46.

The false positive rate at a cutoff of 0.61 is 2.5%.

We created a second data set including 322 patients from a population different than the trial population to validate the troponin I cutoff selected.

See the frequencies of TnI at a cutoff of 0.15 µg/L in rows versus the diagnosis of MI,

and the frequencies of TnI at 0.65 µg/L versus the diagnosis of MI.

Cutoff Test Result Not MI MI Row total

0.15 g/L Negative 239 100%86.9%

0 0%0%

239 100%

  Positive 36 44.5%13.1%

45 55.5%100%

81 100%

  Total 275 100% 45100%

320

         

0.65 g/L Negative 260 99.2%94.5%

2 0.76%4.4%

262 100%

  Positive 15 25.8%5.5%

43 74.2%95.6%

58 100%

  Total 275 100% 45 100% 320

Myocardial Risks Previous MI High Blood Pressure High Cholesterol Diabetes Age>65 Stroke, peripheral vascular disease Hypercholesterolemia Hypertension

LDL Cholesterol Level for Drug Consideration

Goal of Therapy

Without coronary heart disease and with fewer than two risk factors

190 mg/dL or higher*

less than 160 mg/dL

Without coronary heart disease and with two or more risk factors

160 mg/dL or higher

less than 130 mg/dL

With coronary heart disease

130 mg/dL or higher**

100 mg/dL or less

0=Non MI

1=MI0.0 0.2 0.4 0.6 0.8 1.0 1.2MEDSCORE2

0

1

2

3

4

5

TN

I

10

DXMI

Comparison of TnI Level with Number of Active Medications in Groups of MI and Non

MI

Medscore0= No Statins1= 1 Statin Drug

The documentation of chest pain characteristics in the medical record is inconsistent, sometimes absent, and often unclear. This results in reliance on TnI and emphasis on ECG evidence for the diagnosis of MI, even though many patients lack a definitive ECG pattern. This places greater importance on the presence of risk factors, such as, diabetes, hypertension, hypercholesterolemia, previous MI, and anginal equivalent findings in addition to TnI in assessing the probability of MI. A similar model can be constructed using TnI and risk factors.