| 1 | 1

Levels of Evidence for Foot and Ankle Questions on the OITE: 15-Year Trends

Cory T. Walsh, Louis C. Grandizio DO, John Parenti MD, Gerard J. Cush MD

GEISINGER ORTHOPAEDIC INSTITUTE

AOFAS Disclosure Statement

No conflict to disclose

Levels of Evidence for Foot and Ankle Questions on the OITE: 15-Year Trends

Cory T. Walsh MD

My Disclosure is in the Final AOFAS Mobile App

I have no conflicts with this presentation

| 2

| 3 | 3

BACKGROUND

• There has been a trend towards evidence based medicine (EBM) in

the orthopaedic surgery literature.

• Levels of Evidence (LoE) for articles in peer reviewed journals have improved in the past decade.

• The OITE functions as a yearly indicator of orthopaedic resident knowledge and has been found to correlate with performance on ABOS Part I Examination

• There is a paucity of literature pertaining to LoE supporting OITE questions.

| 3

| 4 | 4

PURPOSE & METHODS

PURPOSE • Determine if LoE for primary journal articles referenced within FA

content domain on OITE have increased over 15 year period

• Determine if both characteristics and taxonomy have changed during this same time period

METHODS

• All questions in FA content from 1995-1997 and from 2010-2012 were analyzed. Omitted questions excluded from official OITE

| 4

METHODS

Data collection from OITE score reports included: • Year of publication, journal, source type (primary journal article,

review article, ICL, textbook) • For each primary journal we documented the LoE for each using

AAOS Levels of Evidence Guidelines

Taxonomy

• Using Buckwalter Classification questions were assigned T1, T2, or T3

• T1 – recall basic facts • T2 – interpret an imaging study or make a diagnosis • T3 – treatment and management provided clinical situation

| 5

BASELINE CHARACTERISTICS

| 6

Overall 1995-97 2010-12 P-value

Total number of included questions, N 132 78 54 --

Buckwalter Classification

T1, N (%)

T2, N (%)

T3, N (%)

39 (29.5%)

25 (19%)

68 (51.5%)

29 (37%)

15 (19%)

34 (44%)

10 (18.5%)

10 (18.5%)

34 (63%)

.0286

Number of references per question

1, N (%)

2, N (%)

3, N (%)

4, N (%)

Types of reference per question

At least 1 primary journal article, N (%)

At least 1 journal review article, N (%)

At least 1 textbook, N (%)

At least 1 ICL, N (%)

24 (18%)

88 (67%)

19 (14%)

1 (1%)

64 (48%)

55 (42%)

76 (58%)

3 (2%)

24 (31%)

46 (59%)

7 (9%)

1 (1%)

34 (44%)

28 (36%)

51 (65%)

2 (3%)

0 (0%)

42 (78%)

12 (22%)

0 (0%)

29 (54%)

27 (50%)

25 (46%)

1 (2%)

<.0001

.1762

.1061

.0291

.7871

RESULTS

• 78 questions from 1995-1997, 54 questions from 2010-2012

• FAI was the most frequently cited source overall

TAXONOMY • Change in taxonomy distribution was statistically

significant – 34/54 were level T3

| 7

RESULTS

Levels of Evidence • 6/54 (11%) of FA questions from 2010-2012 were Level I or II,

while 3/78 (4%) from 1995-1997 were Level I or II (p level = .1035)

| 8

RESULTS

1995-97

OITE

(n=78)

2010-12

OITE

(n=54)

P-

value

Questions in each time period that cited level I-II evidence, N

(%)

Yes

No

Questions in each time period that cited level I-III evidence, N

(%)

Yes

No

3 (4%)

75 (96%)

5 (6%)

73 (94%)

6 (11%)

48 (89%)

8 (15%)

46 (85%)

.1035

.1111

| 9

Comparison of the LoE for questions in each time period

CONCLUSIONS

• There has been a trend towards creating OITE questions supported by higher LoE, which reflects the improved LoE in orthopaedic FA literature

• Increase in level 1 studies in comparing 1995-1997 to 2010-2012

• Change in question taxonomy with increased emphasis on clinical management questions compared to years past

• FAI was more frequently cited source

| 10

CONCLUSION

Summary Our results can be used to help improve resident self-study and suggest that reviewing recent FAI articles may aid OITE preparation.

| 11

References

Barr CR, Cheng I, Chou LB, et al. Foot and ankle questions on the orthopaedic in-training examination n: analysis of content, reference, and performance. Orthopedics.2012;35(6),472. doi: 10.3928/01477447-20120525-28.

Barske HL, Baumhauer J. Quality of research and level of evidence in foot and ankle publications. Foot Ankle Int, 2012;33(1):1-6. doi: 10.3113/FAI.2012.0001. Bernstein J, Kenniston JA, Nydick JA, et al. Levels of evidence are low for clinical management questions on the orthopaedic in-training examination. J Bone Joint Surg Am.

2010;92:508-511. doi: 10.2106/JBJS.I.00530. Buckwalter JA, Schumacher R, Albright JP, et al. Use of an educational taxonomy for evaluation of cognitive performance. J Med Educ. 1981;56:115-121. Dougherty PJ, Walter N, Schilling P, et al. Do scores of the USMLE Step 1 and OITE correlate with the ABOS Part I certifying examination?: a multicenter study. Clin Orthop and

Relat Res. 2010; 468(10):2797-2802. doi: 10.1007/s11999-010-1327-3. Hanzlik S, Mahabir RC, Baynosa RC, et al. Levels of evidence in research published in the journal of bone and joint surgery (american volume) over the last thirty years. J Bone

Joint Surg Am. 2009; 91:425-428. doi: 10.2106/JBJS.H.00108. Haughom BD, Goldstein Z, Hellman MD, et al. An Analysis of References Used for the Orthopaedic In-Training Examination: What are Their Levels of Evidence and Journal

Impact Factors? Clin Orthop and Relat Res. 2014; 472:4024-4032. doi: 10.1007/s11999-014-3895-0. Mankin HJ. The orthopaedic in-training examination (OITE). Clin Orthop Relat Res. 1971;75:108-116. Miyamoto RG, Klein GR, Walsh M, et al. Orthopedic surgery residents’ study habits and performance on the orthopedic in-training examination. Am J Orthop. 2007; 36(12):e185-

188. Razi Z, Abbassian A, Cro S, et al. Levels of evidence in foot and ankle surgery literature: progress from 2000 to 2010? J Bone Joint Surg Am. 2012;94:e112-1. doi:

10.2106/JBJS.K.01453. Sheibani-Rad S, Arnoczky SP, Walter NE. Analysis of the basic science section of the orthopaedic in-training examination. Orthopedics. 2012; 35(8), 1251-1255. doi:

10.3928/01477447-20120725-28. Srinivasan RC, Seybold JD, Kadakia AR, Analysis of the foot and ankle section of the Orthopaedic In-Training Examination. Foot Ankle Int. 2009;30(11):1060-1064. doi:

10.3113/FAI.2009.1060. Thordarson DB. Levels of evidence, evidence-based medicine, and Foot & Ankle International. Foot Ankle Int. 2008; 9(9):881-882. doi: 10.3113/FAI.2008.0881. | 12