1. Temporal Correction for Model Outdating / Treatment

Improvement:

MIST Runs Over the Cloud!

Amazon EC2 (Elastic Compute Cloud)

In a Nutshell: A League of Disease Models using High Performance Computing

The Reference Model for Disease Progression Sensitivity to Bio-Marker Correlation in Base Population The Reference Model Runs with MIST Over the Cloud!

Jacob Barhak Ph.D.

http://sites.google.com/site/jacobbarhak/

IMAG Multiscale Modeling (MSM) Consortium Meeting 2013 - 2-3 October

MI No CHD Survive MI CHD

Death

Stroke No Stroke Survive Stroke

Stroke Death

Alive Other Death

Death

Process CHD

Process Stroke

Process Competing Mortality

References: [1] J. Barhak, The Reference Model Scores Fitness of Models and Populations. Poster Presentation. ISPOR 18th Annual International Meeting, May 18-22, 2013, Sheraton New Orleans, New Orleans, LA, USA. Poster: http://sites.google.com/site/jacobbarhak/home/PosterTheReferenceModel_ISPOR_Submit_2013_05_14.pdf [2] J. Barhak, The Reference Model: Improvement in Treatment Through Time in Diabetic Populations, The Fourth International Conference in Computational Surgery and Dual Training. The Joseph B. Martin Conference Center at Harvard Medical School. Boston, MA, USA. December 9-10-11, 2012. Video: http://web.cs.uh.edu/~cosine/?q=node/140 , Presentation: http://www2.cs.uh.edu/~cosine/talks_cosine4/monday/MultidisciplinaryTalks/2_JacobBarhak.pptx [3] J. Barhak, The Reference Model for Chronic Disease Progression. 2012 Multiscale Modeling (MSM) Consortium Meeting, October 22-23, 2012, Abstract: http://www.imagwiki.nibib.nih.gov/mediawiki/images/7/77/Reference_Model_for_Chronic_Disease_Progression_Barhak.pdf Poster: http://www.imagwiki.nibib.nih.gov/mediawiki/images/c/c4/PosterTheReferenceModel_IMAGE_MSM_Submit_2012_10_17.pdf [4] J. Barhak, MIST: Micro-Simulation Tool to Support Disease Modeling. SciPy, 2013, Bioinformatics track, Abstract: https://github.com/scipy/scipy2013_talks/tree/master/talks/jacob_barhak Video: http://www.youtube.com/watch?v=AD896WakR94 [5] MIST - MIcro Simulation Tool software in the GitHub repository. Online (Accessed 12-Aug-2013): https://github.com/Jacob-Barhak/MIST [6] StarCluster home page. Online (Accessed 12-Aug-2013): http://star.mit.edu/cluster/ [7] B. Zaitlen, StarCluster Anaconda. Online (Accessed 12-Aug-2013): http://continuum.io/blog/starcluster-anaconda

Acknowledgments: The GPL modeling framework was supported by the Biostatistics and Economic Modeling Core of the MDRTC (P60DK020572) and by the Methods and Measurement Core of the MCDTR (P30DK092926), both funded by the National Institute of Diabetes and Digestive and Kidney Diseases. The modeling framework was initially defined as GPL and was funded by Chronic Disease Modeling for Clinical Research Innovations grant (R21DK075077) from the same institute. The Reference Model was developed independently without financial support.

Population 1

Population 2

Population n

Mu

ltiple Po

pu

lation

s

New!!!

The Reference Models for disease progression [1-3] uses High Performance Computing (HPC) to simulate a league of published disease models to compete for fitness to known clinical trial results. The resulting color coded fitness score matrix serves the modeler as a reference to improve model design and improve understanding of different models. The Reference Model regenerates baseline population data from publicly published data and does not require access to individual population information. Therefore the base for information is large, combining information from multiple trials, which provides a wider global view of phenomena observed. Using a Multi Scale modeling approach simulations are executed at the individual scale to reproduce information at the population scale and allows drawing conclusions at the multi-population scale. Never the less, the use of aggregate data instead of individual data brings up questions regarding the sensitivity of the model to assumptions and unpublished information such as bio-marker correlations. Fortunately it is possible to address these questions using HPC by running the assumptions and their fitness to observed results. This work demonstrates this by testing two extreme scenarios of biomarker correlation within populations: 1) Independent Bio-Markers with No Correlation, 2) Dependent Bio-Markers with Perfect Correlation. 2 X 34 cohorts of 6 diabetic studies :UKPDS, ASPEN, ADVANCE, ACCORD, KP, NDR, were generated from distributions using those scenarios and tested against 64 model variations composed of cardiovascular risk equations and modeling assumptions [1,2]. The results show the behavior of the model variations within the assumption scope and result rankings help identify superior models. Results consist of 4352=2x34x64 scenario simulations of 1000 virtual individuals and 10 Monte Carlo repetitions each for 10 years ~ 0.4 Billion virtual patient years. The use of HPC allowed running these computations in reasonable time. This simulation took about 36 hours on the cloud by leasing a computer cluster composed of 160 cores using 20 machines with 8 cores each. The Reference Model uses free Python based framework of software tools to run these simulations. MIST (MIcro Simulation Tool) [4-5] is used to run simulations in HPC environment. MIST runs over the cloud with the help of StarCluster [6] and an Anaconda AMI (Amazon Virtual Machine) [7].

Population Characteristics: Age, Male, Race, SBP, DBP, A1c, Smoke, BMI, HDL, LDL, Trig, TC, ACR, Lipid Ratio, Age At Diagnosis Of Diabetes, AF, Survive Stroke, Survive MI, MicroAlbuminuria, MacroAlbuminuria, Diabetes Type 2, Treated for Hypertension, Family History CHD, Rheumatoid Arthritis, Townsend Index, A1c Change, BMI Change, DBP Change, SBP Change, HDL Change, LDL Change, Trig Change, TC Change, Smoke Change, Year

Best Model

MODELS

Model Traits

A1c changes 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

BMI changes 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

BP changes 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Lipid change 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Smoke changes 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

MI Equation # 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4 1 1 1 1 2 2 2 2 3 3 3 3 4 4 4 4

Stroke Equation # 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

CHD Death Equation # 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Stroke Death Equation # 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Treatment Improvement Correction 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

FITNESS MATRIX: LOW SCORE = GOOD FITNESS (GREEN) , HIGH SCORE = BAD FITNESS (RED)

UKPDS

UKPDS33 Conventional 25.4 25.2 25.6 26.4 42.8 37.3 41.8 39.6 39.9 37.9 41.8 36.7 38.4 32.8 35.7 35.2 26.6 28.8 25.2 26.7 40.6 38.2 39.5 40.8 37.9 41.1 39.0 37.0 34.6 34.1 33.0 34.1 25.3 31.0 33.6 30.5 26.8 30.3 32.2 31.5 32.3 40.1 41.3 35.0 21.8 30.8 34.9 30.7 22.2 30.8 34.9 29.8 26.5 32.2 33.0 31.1 30.5 37.9 38.1 38.9 22.2 28.6 33.1 28.9

UKPDS33 Conventional Correlated 23.0 23.5 22.6 23.6 38.9 36.8 40.7 42.1 36.2 33.0 35.6 36.0 41.8 39.4 43.7 43.5 22.7 23.6 23.5 22.9 37.1 39.8 37.8 38.2 35.1 36.0 33.1 38.0 40.1 38.7 37.9 42.0 26.4 29.5 35.4 29.6 30.7 38.8 42.7 33.3 39.2 45.6 48.2 45.4 22.8 28.2 34.1 28.7 23.8 32.3 36.2 31.5 26.9 34.9 40.4 32.9 40.2 46.4 51.4 42.8 24.2 31.1 36.4 28.0

UKPDS33 Intensive 22.9 23.4 20.2 20.9 34.1 35.8 33.8 33.9 33.5 32.4 31.2 30.5 44.7 41.4 44.3 46.2 27.6 23.6 23.1 22.6 35.4 32.7 34.8 35.3 34.5 31.5 30.9 32.0 41.4 46.2 45.1 43.2 21.6 25.3 30.6 26.9 31.9 32.5 36.8 34.3 36.3 44.5 45.2 43.2 22.2 29.8 31.9 28.4 21.8 26.1 30.2 28.0 32.1 35.2 37.1 33.0 37.5 42.2 46.2 42.3 23.7 27.0 28.4 27.2

UKPDS33 Intensive Correlated 25.1 24.1 23.8 26.4 32.1 32.1 32.5 31.1 29.6 30.3 28.7 28.6 51.3 47.3 44.7 51.4 25.7 25.0 26.1 26.4 34.3 32.1 28.9 33.5 28.7 31.9 30.0 30.5 50.3 46.0 47.8 47.2 29.3 29.4 33.6 30.8 40.0 40.4 42.1 40.7 49.0 48.9 55.5 51.2 26.1 31.4 32.7 26.5 27.8 31.0 33.5 30.0 35.6 37.9 44.6 37.7 50.3 53.1 57.0 51.4 24.9 30.2 35.2 28.4

UKPDS33 Full 23.1 22.1 21.9 22.9 33.1 34.6 36.1 35.1 32.1 35.1 34.8 33.7 42.6 40.1 42.3 40.6 22.9 23.8 23.5 24.2 34.4 34.7 32.7 35.4 33.6 34.9 32.7 33.5 40.2 38.1 42.7 39.9 21.6 26.0 29.7 28.0 29.6 34.0 37.9 33.3 37.9 41.0 46.9 43.0 21.6 26.5 31.4 29.7 22.8 29.5 30.9 27.4 29.0 35.1 40.0 35.3 37.5 44.7 41.1 41.6 21.6 26.1 31.1 25.9

UKPDS33 Full Correlated 22.8 24.9 22.6 24.7 33.5 34.9 34.5 30.8 30.6 31.3 30.1 31.1 48.8 47.7 51.1 50.4 25.8 23.8 21.9 24.7 36.1 31.4 34.2 33.4 30.0 30.2 28.9 26.9 49.3 45.7 47.2 47.8 29.6 36.4 35.4 31.6 33.3 40.5 41.8 35.5 46.6 50.6 56.5 52.0 25.6 31.0 34.7 29.3 29.3 30.9 36.7 30.2 34.9 39.4 42.5 37.5 46.8 48.1 52.1 48.5 24.7 32.4 34.6 27.6

APSEN

ASPEN All Placebo 13.6 13.1 13.4 12.4 17.7 12.0 14.0 14.2 12.5 12.6 10.7 12.6 21.2 20.9 23.1 24.3 15.7 11.5 14.2 16.0 14.6 15.8 19.2 14.1 13.9 13.4 12.8 15.5 21.0 21.6 22.0 21.3 11.1 10.6 10.5 9.5 21.1 22.9 23.7 20.8 21.5 24.4 18.2 20.5 14.1 12.5 12.8 13.6 9.8 10.8 10.0 7.5 20.1 21.2 23.3 19.7 20.6 23.1 20.8 19.8 14.8 12.3 11.6 12.6

ASPEN All Placebo Correlated 37.7 33.4 32.2 31.4 24.7 22.1 18.1 21.8 22.9 18.2 20.8 21.2 36.8 36.8 34.6 34.0 33.6 34.5 31.1 34.8 21.6 22.0 23.4 20.1 20.7 20.9 20.1 17.5 41.3 36.9 34.1 35.9 9.3 10.9 12.3 8.4 26.1 29.4 29.7 25.9 29.7 29.4 29.1 29.4 10.2 11.6 9.7 8.9 9.7 12.0 13.1 12.2 27.0 27.8 27.6 28.8 31.7 31.1 33.5 31.2 8.2 11.5 12.0 9.3

ASPEN All Atorvastatin 18.6 17.1 16.8 16.6 20.7 18.5 21.0 20.6 19.8 20.6 17.7 15.8 30.6 28.2 28.9 29.9 13.4 10.0 13.2 13.2 15.8 16.3 19.1 16.9 14.3 13.1 16.4 14.9 18.3 22.0 17.5 18.1 9.5 9.9 11.5 8.5 30.6 27.6 27.1 28.8 27.6 28.6 28.5 25.7 9.5 12.4 12.3 10.1 9.3 8.5 11.1 8.5 21.2 26.1 25.4 25.8 22.0 23.7 23.8 25.5 11.8 13.9 12.3 12.8

ASPEN All Atorvastatin Correlated 40.4 38.0 37.6 37.3 26.8 27.1 28.6 24.4 26.2 22.7 25.8 27.0 42.6 43.2 40.9 42.5 29.2 29.7 28.4 29.1 24.5 24.4 22.4 22.7 22.2 21.2 21.9 20.9 36.4 27.4 29.9 32.3 15.6 20.2 22.4 16.7 37.0 36.4 35.1 37.0 35.3 41.5 38.6 38.2 6.1 12.1 13.9 8.2 10.2 11.6 11.2 11.3 29.4 32.0 30.8 29.8 32.8 31.5 34.6 35.2 8.7 10.2 12.7 7.5

ASPEN Primary Placebo 21.3 22.3 20.1 22.6 23.2 23.5 25.0 24.7 16.9 14.5 15.8 16.0 34.4 31.7 29.1 33.1 23.7 19.9 22.4 22.6 24.5 24.8 22.9 19.5 16.3 17.0 15.9 16.8 31.4 31.6 35.3 34.0 8.9 11.8 8.8 7.6 30.1 34.0 32.2 31.9 23.6 25.8 25.1 22.1 8.7 7.7 7.3 8.0 9.5 9.7 9.1 9.0 31.5 28.7 29.6 29.1 22.1 25.1 23.2 21.9 9.5 9.0 5.7 8.4

ASPEN Primary Placebo Correlated 39.8 40.1 43.3 38.2 30.3 29.3 29.7 28.3 23.6 26.1 24.7 23.2 48.3 45.6 43.6 41.5 42.4 40.6 40.6 38.4 29.5 29.6 27.7 30.0 23.3 22.0 21.5 22.5 46.3 44.0 47.2 45.7 18.4 19.1 18.3 18.0 35.8 34.6 33.7 37.0 32.5 33.3 31.2 32.8 9.3 11.2 12.1 9.5 18.0 18.7 21.9 15.5 34.2 36.2 34.1 35.5 34.9 31.3 31.9 32.5 9.4 11.8 9.8 10.0

ASPEN Primary Atorvastatin 22.2 21.4 23.3 20.1 25.7 23.7 24.4 24.1 17.6 19.7 17.5 15.4 34.5 35.4 33.6 32.1 15.8 15.2 15.3 16.1 19.8 18.9 18.3 19.2 14.6 14.7 15.9 14.5 23.6 24.6 21.8 25.3 11.4 11.9 12.5 12.8 30.1 30.9 32.7 33.8 26.1 25.5 23.7 25.1 10.2 12.2 9.3 10.1 10.5 8.9 9.3 10.9 26.1 28.7 27.5 26.1 20.8 23.3 22.4 22.6 11.6 9.7 10.5 12.3

ASPEN Primary Atorvastatin Correlated 41.0 38.8 40.3 39.0 30.0 29.8 26.8 27.1 24.4 24.8 24.6 23.4 46.1 42.7 47.5 44.3 28.6 31.5 32.0 31.0 27.0 25.9 25.8 25.7 20.4 20.9 20.0 19.7 34.6 34.3 37.2 33.2 18.7 20.3 18.7 18.8 35.5 38.4 35.7 38.8 34.6 36.2 35.3 33.7 11.7 12.4 12.2 11.0 15.1 15.7 13.9 15.0 31.6 30.1 31.4 28.3 31.5 29.4 31.1 29.9 10.0 12.6 11.2 9.2

ASPEN Secondary Placebo 23.2 23.7 20.9 20.9 21.4 20.2 18.7 19.1 19.0 14.0 15.3 12.3 19.8 14.8 18.1 15.8 23.6 18.1 21.3 21.2 21.4 21.4 20.6 20.4 17.8 13.3 14.2 14.0 21.0 14.5 19.2 16.9 39.9 40.6 43.3 39.2 9.5 10.0 18.8 9.3 19.4 17.0 20.7 20.8 46.7 48.5 50.6 48.6 37.9 39.7 43.4 39.4 10.0 10.4 20.3 12.4 17.6 20.7 23.1 18.3 48.2 47.9 50.6 47.8

ASPEN Secondary Placebo Correlated 29.3 18.0 22.5 20.9 23.1 17.9 19.5 19.8 23.7 17.0 22.0 21.6 25.1 17.5 21.8 17.7 28.3 18.7 23.0 20.7 24.6 16.0 20.7 18.9 24.9 20.1 24.2 19.6 27.4 16.7 20.7 17.6 26.9 28.7 31.9 24.4 7.7 9.8 20.4 8.7 26.2 30.1 36.4 28.1 41.7 44.0 46.1 44.0 25.8 24.4 31.9 23.8 7.1 11.7 23.8 9.5 34.3 31.4 36.9 29.7 42.6 43.9 46.2 42.1

ASPEN Secondary Atorvastatin 14.7 8.4 13.7 14.6 14.4 11.6 13.9 14.7 24.6 24.8 23.6 26.5 19.0 17.5 20.4 16.0 12.7 6.5 11.2 12.2 11.9 6.1 10.4 12.2 19.3 17.4 20.7 20.0 12.1 6.7 10.9 10.9 17.8 19.6 25.1 19.9 18.9 20.0 28.2 23.4 43.2 40.3 42.6 45.3 25.8 24.5 32.2 28.0 21.7 23.4 28.8 26.1 14.7 13.8 21.2 17.5 32.8 34.2 38.7 37.5 30.7 32.1 34.7 34.1

ASPEN Secondary Atorvastatin Correlated 33.3 26.2 31.1 26.8 26.0 16.3 21.5 19.0 42.3 38.9 35.8 37.4 35.8 27.8 29.5 28.9 21.1 18.9 22.2 20.0 15.6 9.3 18.0 12.7 35.6 27.0 30.9 30.6 20.9 17.7 19.7 19.6 7.7 12.8 27.8 14.5 23.5 28.8 33.8 26.1 50.0 48.9 54.3 52.5 23.7 24.9 33.0 24.3 10.6 15.0 23.2 18.5 14.2 19.6 28.3 21.0 46.2 44.8 49.1 46.9 24.0 27.9 34.2 31.1

ASPEN Full 9.8 7.4 8.2 8.0 9.5 11.4 12.4 9.1 7.7 6.7 7.7 7.0 18.8 17.3 16.1 15.9 8.5 8.6 8.5 9.2 7.3 10.1 8.3 9.4 7.4 7.4 9.3 7.7 17.5 17.7 16.4 19.9 12.1 10.9 10.8 10.8 16.7 21.6 16.8 14.4 15.0 16.5 15.4 14.7 18.2 18.1 17.6 14.9 15.0 14.8 13.1 12.5 18.4 15.5 17.3 17.2 16.9 16.1 15.7 17.0 16.7 16.3 17.3 15.1

ASPEN Full Correlated 26.2 27.6 25.7 29.0 16.2 16.6 15.0 15.4 17.8 14.4 14.5 15.4 28.4 32.6 32.4 33.0 26.0 25.4 27.8 29.0 15.1 16.4 15.7 15.7 19.7 16.3 11.9 13.0 31.0 29.4 27.3 27.4 5.3 8.6 11.5 8.0 23.3 22.9 24.7 25.7 25.4 24.3 26.8 24.3 12.7 13.5 11.8 11.5 5.3 9.2 10.9 6.8 21.8 22.6 22.1 21.7 23.8 27.9 25.9 26.1 10.6 14.2 12.9 13.4

ADVANCE

ADVANCE Standard 60.0 55.6 59.6 60.0 36.5 29.5 32.1 36.8 52.8 42.3 46.7 50.7 65.4 63.2 62.1 65.1 45.0 41.6 40.5 42.6 33.8 22.4 24.3 29.4 45.9 35.9 40.7 42.6 51.2 45.6 45.6 47.4 9.3 10.1 16.4 15.0 21.4 23.5 25.5 26.2 38.8 43.8 51.6 46.4 14.0 11.6 14.9 9.7 11.7 8.6 10.3 11.2 19.6 17.3 21.4 19.1 33.5 38.9 41.1 37.5 15.5 17.2 16.1 13.4

ADVANCE Standard Correlated 120.2 117.9 122.2 121.8 66.5 41.3 54.3 49.0 95.4 80.5 84.3 81.2 99.8 85.5 94.2 95.3 108.1 98.1 99.2 98.8 54.1 31.8 40.6 40.9 80.1 67.7 74.6 74.3 88.6 76.4 80.3 74.1 39.1 39.6 51.1 44.2 34.9 31.8 39.9 34.9 75.0 68.0 77.9 74.7 10.6 7.6 26.6 14.6 30.1 30.9 41.2 34.5 26.9 27.2 40.0 32.2 65.8 65.6 73.1 63.2 7.6 8.0 16.9 11.7

ADVANCE Intensive 66.9 61.4 59.8 68.7 45.0 33.5 38.0 35.0 61.9 51.4 51.4 52.3 68.4 62.2 69.0 71.6 38.9 35.9 38.5 39.0 28.5 25.4 28.5 27.6 39.9 36.2 40.7 39.6 48.1 44.6 41.7 47.9 12.0 12.5 17.9 16.0 24.3 27.9 30.3 27.6 48.7 49.7 54.9 50.8 9.6 6.1 15.1 10.7 10.7 7.1 9.0 9.1 19.3 25.4 26.5 24.3 32.0 35.1 38.6 40.3 15.9 12.2 11.5 12.3

ADVANCE Intensive Correlated 127.4 117.8 128.9 128.9 68.7 45.5 56.0 52.3 99.2 78.1 85.7 87.7 107.7 90.8 102.1 93.9 98.5 90.4 96.7 94.4 55.6 41.0 43.8 44.1 76.8 69.4 71.8 70.4 91.6 79.9 79.6 74.5 48.1 43.3 58.9 47.8 39.1 36.0 46.8 39.5 77.6 75.4 83.9 80.1 10.1 10.6 29.2 17.8 27.3 24.8 37.8 33.4 32.7 30.8 38.8 34.9 61.1 62.8 64.3 63.4 7.2 5.3 17.1 12.7

ADVANCE Asia Standard 66.6 54.1 64.2 61.3 29.3 27.5 30.7 24.1 52.3 49.2 47.2 51.4 74.4 65.5 70.3 67.9 40.5 50.0 50.6 48.7 23.2 21.7 21.9 23.6 40.6 36.9 39.7 41.7 50.8 47.2 53.2 51.8 19.2 16.2 12.5 16.9 25.2 22.2 21.6 22.2 42.9 45.9 45.7 51.3 20.9 18.8 11.1 13.0 22.4 19.2 11.7 11.4 23.2 20.2 17.1 21.2 37.9 38.1 37.3 37.6 28.6 25.0 16.0 16.2

ADVANCE Asia Standard Correlated 147.1 133.0 139.6 144.2 55.6 38.0 40.6 45.2 93.6 85.3 92.9 91.8 113.2 98.3 103.4 98.9 121.2 111.3 115.9 114.2 48.0 29.1 38.9 37.5 80.6 72.1 80.3 79.7 91.8 80.8 87.7 80.6 49.5 43.3 57.9 51.5 28.7 27.6 31.4 30.1 70.4 75.1 84.9 77.8 10.2 13.7 12.4 10.0 38.1 32.7 44.1 39.9 22.9 23.8 30.9 26.9 62.7 65.6 74.1 74.6 14.0 17.5 9.2 8.5

ADVANCE Asia Intensive 75.2 65.2 73.1 67.9 35.9 34.5 36.0 35.2 60.2 56.5 55.3 58.7 76.1 66.6 72.1 71.8 44.0 41.6 43.6 44.5 28.4 25.3 21.9 30.1 48.1 38.1 41.0 43.8 52.9 54.8 50.9 57.3 19.3 17.9 16.9 18.0 25.6 24.7 28.2 26.9 44.2 50.3 49.0 54.4 15.8 14.0 6.8 6.2 19.6 15.9 10.2 7.9 22.8 19.9 24.4 23.1 38.7 39.3 37.9 37.9 22.1 18.5 13.8 11.6

ADVANCE Asia Intensive Correlated 147.1 140.0 151.4 152.0 60.8 41.8 51.9 52.7 104.1 89.7 97.1 93.5 115.3 106.4 113.1 112.4 116.9 110.2 113.0 110.8 45.3 33.6 41.7 42.3 81.5 74.9 80.9 79.3 92.3 82.0 90.0 87.5 49.7 48.8 57.1 54.2 35.1 34.0 43.7 36.3 81.2 81.9 88.6 85.4 13.4 10.8 14.7 11.8 31.1 34.6 40.2 35.7 25.0 25.6 30.5 32.7 68.4 64.3 72.7 65.2 7.6 10.9 9.8 6.6

ADVANCE EME Standard 88.0 75.2 72.6 83.0 73.1 68.0 67.0 67.2 74.7 71.8 71.6 71.8 85.4 89.9 83.0 80.7 64.4 57.2 58.8 56.8 68.0 53.9 63.5 59.4 62.0 62.4 58.2 57.2 67.3 63.6 67.3 71.9 10.7 21.9 25.8 21.2 46.2 52.6 51.8 50.6 52.1 59.9 65.8 56.0 15.6 21.3 20.8 20.8 12.2 18.8 17.6 20.3 37.1 41.2 46.5 49.4 41.7 49.9 47.4 51.8 19.9 21.1 22.3 21.8

ADVANCE EME Standard Correlated 151.9 147.7 144.6 147.7 104.2 86.6 90.6 82.7 124.1 104.7 105.7 104.6 136.1 123.1 119.1 110.7 128.8 119.5 127.2 125.5 94.6 72.4 80.2 76.7 110.7 91.0 96.2 90.4 114.1 97.2 99.3 99.8 51.9 53.9 62.8 56.1 65.0 63.1 76.4 67.9 93.1 96.4 94.3 90.5 24.1 25.6 37.9 24.2 39.4 38.5 47.9 41.7 60.4 59.1 70.4 58.8 82.0 76.2 83.0 78.9 20.3 23.5 29.3 19.9

ADVANCE EME Intensive 93.8 82.9 89.9 93.3 80.9 73.2 69.9 77.7 83.4 79.3 78.0 78.3 97.0 83.3 87.5 87.1 58.4 58.2 48.3 54.8 68.8 64.1 60.9 62.0 61.4 56.0 54.7 53.8 72.4 73.6 59.9 68.7 18.0 27.3 27.3 23.1 50.1 62.0 59.6 58.9 57.2 61.8 62.2 63.5 14.0 20.9 23.0 18.1 11.6 17.7 18.4 18.9 45.8 49.2 49.1 51.7 43.1 47.6 48.7 46.7 16.5 19.1 19.5 17.2

ADVANCE EME Intensive Correlated 161.5 156.2 152.5 150.5 109.0 89.8 92.1 87.7 128.0 110.4 115.5 110.6 138.2 122.0 128.1 122.6 126.9 113.2 109.3 115.1 96.1 82.0 80.5 78.9 105.3 94.5 90.0 87.1 109.4 101.8 99.1 99.3 58.8 58.3 70.1 59.1 73.4 69.6 77.5 71.9 91.6 94.2 102.8 89.9 29.5 29.3 35.8 25.9 39.1 39.5 46.7 40.5 59.8 66.8 69.1 62.7 75.0 77.7 84.6 80.3 22.9 22.2 32.7 22.3

ADVANCE Eastern Europe Standard 71.7 69.1 72.1 68.5 61.8 60.2 60.1 58.7 62.2 53.8 60.0 55.5 77.2 74.9 73.7 70.4 54.5 50.4 47.8 52.4 59.1 47.9 50.0 53.3 54.4 49.3 47.3 45.7 58.1 55.1 49.4 58.2 15.4 17.4 19.3 16.9 44.2 49.0 46.7 45.3 47.8 50.4 51.2 50.7 25.2 23.2 21.2 22.7 20.2 16.4 17.7 17.3 37.4 37.8 44.3 37.7 36.9 47.2 42.9 41.0 29.1 23.9 25.1 25.8

ADVANCE Eastern Europe Standard Correlated 149.3 142.7 146.1 142.2 96.5 73.5 78.5 79.2 116.2 97.7 99.8 95.5 122.6 105.1 107.4 104.7 122.4 121.3 120.4 121.6 86.5 60.7 72.5 67.7 104.8 84.5 85.3 81.9 97.9 84.2 88.7 81.8 45.0 47.6 53.3 49.3 57.0 59.4 65.1 60.4 86.4 85.6 87.5 87.6 16.9 18.4 26.6 18.2 33.0 35.9 41.7 39.2 50.1 52.4 56.4 49.4 73.7 77.4 87.9 71.4 17.8 20.2 22.7 18.8

ADVANCE Eastern Europe Intensive 72.3 69.1 74.6 72.1 72.6 54.2 54.7 58.7 63.6 53.7 61.7 56.7 79.6 78.3 76.2 76.1 46.1 43.5 39.0 45.0 53.4 47.8 44.6 47.8 44.1 40.1 39.6 42.1 59.8 54.0 52.8 49.1 10.0 17.4 19.7 16.0 38.8 48.6 49.9 48.2 47.7 51.0 53.7 50.5 22.2 16.3 19.4 19.2 19.3 15.3 16.9 17.1 33.6 40.5 37.8 33.5 35.0 36.7 37.8 36.4 29.9 22.4 23.4 22.8

ADVANCE Eastern Europe Intensive Correlated 149.2 147.6 147.4 144.1 100.1 81.6 76.9 82.7 117.1 96.6 103.6 97.2 120.3 108.8 108.0 105.9 116.3 105.0 113.1 107.6 81.2 67.1 75.0 66.6 97.4 80.3 85.6 79.9 101.7 87.6 83.8 89.3 51.3 54.1 57.0 49.1 62.2 59.6 68.6 59.5 88.0 84.4 94.4 89.1 14.9 18.1 27.8 17.0 30.1 33.7 30.9 27.9 48.8 53.4 61.7 56.0 69.4 71.8 74.6 73.7 16.0 19.1 21.5 16.2

ADVANCE Full 53.9 53.5 59.3 54.3 55.3 58.4 53.2 55.5 54.2 54.5 58.5 53.7 58.4 60.9 64.4 60.5 35.6 40.5 37.1 37.4 46.1 46.7 44.5 46.2 46.0 43.4 45.5 44.7 50.6 47.3 46.1 48.4 10.4 12.7 16.6 14.6 40.5 45.9 49.9 49.0 46.8 52.4 53.9 47.2 14.2 13.3 15.0 11.8 14.5 8.9 11.6 10.0 36.7 42.0 42.4 40.6 42.3 39.3 44.2 40.8 20.6 12.5 12.7 13.2

ADVANCE Full Correlated 102.4 106.5 102.9 101.8 70.7 70.2 67.0 71.7 81.2 79.0 80.0 78.7 93.7 90.8 94.4 90.8 83.3 80.6 82.0 85.6 66.2 58.6 61.3 64.8 70.7 66.9 78.0 68.5 80.4 76.1 76.5 81.5 29.3 37.3 46.1 38.1 49.5 61.3 60.6 57.5 68.5 81.3 83.9 73.7 5.6 16.4 24.9 11.7 17.2 26.0 29.5 23.0 47.6 53.2 55.2 49.3 59.0 67.5 73.2 63.9 7.7 11.1 20.5 9.2

ACCORD

ACCORD BP Standard Therapy 53.0 51.5 55.7 55.7 47.3 44.7 54.4 51.7 56.5 60.9 62.7 68.0 60.2 58.0 62.1 60.7 46.7 46.6 51.5 52.8 47.7 43.8 46.7 51.1 59.7 60.3 62.6 66.7 52.1 52.7 57.2 59.2 25.2 28.9 32.7 27.3 19.4 27.6 33.7 30.9 35.1 41.2 47.1 48.2 32.5 32.6 38.5 34.3 28.0 29.5 35.3 30.6 19.3 26.1 34.2 30.2 36.0 39.8 46.5 45.1 35.0 35.3 36.3 33.8

ACCORD BP Standard Therapy Correlated 121.6 112.1 123.7 116.1 80.8 71.4 87.3 78.4 114.8 107.4 115.2 109.4 101.8 98.5 111.7 103.2 116.6 107.3 117.5 114.4 78.1 73.9 85.3 77.5 105.0 104.5 112.3 105.2 97.1 92.8 99.3 99.4 30.3 42.8 54.1 39.6 42.7 47.8 61.8 48.1 74.9 79.3 87.5 81.0 28.9 34.2 46.6 33.7 30.3 37.4 55.4 39.7 40.7 50.0 60.8 46.0 70.6 79.1 87.0 80.9 30.1 34.1 47.0 34.8

ACCORD BP Intensive Therapy 49.8 50.6 57.6 52.7 39.2 45.7 51.1 49.0 57.4 58.7 61.3 56.4 58.5 56.5 62.1 63.5 37.0 39.0 43.8 44.4 37.6 35.1 43.0 44.7 51.1 47.8 52.7 54.2 37.9 41.9 44.9 45.5 25.9 27.7 31.5 29.4 15.6 26.0 34.3 27.4 34.0 35.2 44.8 41.9 32.4 32.4 35.0 32.4 28.2 28.8 32.9 30.1 14.6 21.2 31.3 24.5 25.9 34.7 40.9 36.1 36.3 33.4 37.9 37.7

ACCORD BP Intensive Therapy Correlated 112.7 107.5 116.1 112.4 77.5 68.2 82.0 74.0 104.4 100.6 104.0 100.2 104.7 100.3 103.7 98.1 96.2 90.7 97.9 94.3 64.4 62.2 70.8 70.1 90.9 91.4 97.4 92.7 68.3 74.9 76.5 69.8 28.4 41.1 52.6 38.8 38.2 46.9 60.6 47.7 66.7 79.4 90.5 81.6 28.1 33.4 46.6 33.7 21.4 33.0 43.3 35.3 30.6 43.9 55.7 42.8 62.2 66.4 71.6 70.3 30.8 32.5 45.2 35.8

ACCORD BP Full 49.4 50.4 58.0 57.0 42.9 43.0 52.6 51.3 61.5 64.4 67.3 63.0 60.4 60.6 66.5 63.4 46.2 44.7 48.2 47.1 41.8 37.4 47.8 46.6 55.3 52.6 58.7 59.5 43.0 45.6 52.0 53.2 24.8 27.9 33.4 26.6 16.2 24.9 35.8 24.2 30.1 40.9 48.9 45.7 31.9 33.6 36.5 32.5 28.0 29.5 31.3 30.8 16.9 23.3 29.4 26.1 32.4 40.1 45.6 41.7 34.5 35.2 39.1 35.0

ACCORD BP Full Correlated 114.2 112.2 120.4 116.7 81.4 73.7 78.5 74.7 114.1 107.9 115.3 100.5 98.7 96.1 106.6 98.0 109.3 107.5 109.5 110.4 65.6 67.4 75.8 72.8 98.4 91.7 105.9 103.1 85.7 80.7 88.9 83.9 30.8 38.0 54.9 39.4 39.8 45.0 58.5 49.1 74.7 81.7 87.4 83.2 27.1 33.9 44.7 34.4 26.0 33.9 48.9 37.5 39.3 44.2 56.6 46.7 60.5 72.7 86.3 77.2 29.0 32.4 45.4 33.2

KP

KP 20-34 9.8 10.4 11.8 11.3 9.5 9.6 10.7 10.3 10.1 13.6 14.9 12.2 10.5 11.9 13.0 12.2 11.0 12.5 13.1 11.9 8.5 9.9 11.1 9.5 9.5 9.7 12.1 12.8 12.0 13.4 14.2 12.0 5.4 7.4 7.8 5.6 7.9 9.6 9.9 10.3 9.4 13.1 13.3 12.1 4.5 6.6 6.5 6.3 4.9 7.3 7.9 7.0 8.3 8.8 9.4 9.5 9.4 10.1 12.0 11.6 4.6 6.6 8.0 5.3

KP 20-34 Correlated 13.3 13.9 15.4 14.1 8.6 8.1 9.7 9.7 13.5 13.6 14.9 15.9 12.5 14.8 14.8 14.2 12.5 11.8 16.2 14.4 6.1 7.9 8.9 8.7 11.8 14.3 15.2 13.8 12.4 14.5 14.6 14.9 6.4 8.2 9.2 8.6 6.6 8.0 10.8 8.5 11.4 13.1 15.8 13.0 4.8 6.2 7.2 6.7 6.3 8.4 10.1 7.5 5.9 8.5 8.9 9.1 12.6 10.5 15.3 12.1 5.2 5.3 8.0 5.8

KP 35-50 20.9 23.5 25.8 24.0 21.4 25.5 27.4 23.4 20.6 24.4 22.9 20.5 38.2 41.1 41.2 38.9 21.8 22.3 24.1 24.3 19.0 25.7 25.2 24.5 23.1 25.4 22.4 20.2 37.6 40.6 43.1 38.2 6.5 9.6 11.8 9.4 17.3 17.7 22.6 18.9 14.6 19.1 22.2 21.7 6.7 11.5 12.9 8.8 7.0 10.0 11.0 9.4 14.0 18.5 21.6 19.3 17.5 20.4 21.1 22.1 6.4 9.2 10.6 10.4

KP 35-50 Correlated 30.8 34.0 33.1 33.5 20.0 25.8 25.0 25.2 25.0 27.3 29.6 26.8 43.9 49.7 48.8 45.0 28.1 37.1 31.0 34.7 23.4 27.2 29.7 25.3 25.0 29.7 30.9 27.5 45.8 47.8 50.1 46.8 10.3 12.7 16.6 12.7 17.5 22.5 25.7 19.7 21.4 23.7 26.0 27.3 8.2 14.1 13.0 10.8 10.1 14.9 14.9 14.2 17.6 22.8 23.9 20.3 20.9 26.0 26.7 25.6 8.9 12.5 13.9 11.3

KP 50-65 40.8 44.5 43.9 41.3 44.3 49.5 51.5 52.1 43.6 50.0 49.5 53.7 68.2 77.1 76.3 75.2 40.0 48.8 46.6 48.0 47.0 52.1 52.4 47.5 43.4 51.3 48.9 49.3 66.7 74.0 73.7 76.3 15.7 20.9 23.0 21.6 34.3 45.2 45.0 42.2 35.7 45.5 45.6 43.7 16.4 22.3 26.0 21.3 12.6 21.1 21.2 18.6 32.2 43.6 42.0 42.2 37.3 46.4 40.7 47.1 14.2 22.7 21.6 20.5

KP 50-65 Correlated 60.2 65.7 64.3 63.2 50.9 52.1 57.5 54.2 52.6 55.6 57.7 59.0 83.6 85.7 89.0 86.9 59.4 63.0 62.9 63.8 50.7 55.5 54.6 59.5 53.1 58.6 56.9 57.5 83.5 84.4 88.8 83.0 21.8 27.7 29.8 27.8 35.5 45.0 48.4 44.8 42.4 52.5 53.3 49.4 16.5 25.9 30.7 25.8 18.6 33.8 32.1 27.3 41.7 47.6 51.4 45.1 42.7 50.6 52.3 49.0 16.6 26.1 27.1 23.2

KP 65-75 70.3 71.3 76.0 83.7 81.7 84.5 86.5 96.2 87.7 95.4 95.6 102.0 80.6 87.9 84.1 96.3 73.9 72.9 76.8 83.9 81.0 83.6 83.8 95.7 85.7 90.3 93.4 108.1 83.1 85.2 89.6 94.0 33.7 46.4 45.8 47.5 62.3 72.6 75.0 81.0 73.6 85.3 88.5 91.3 31.5 40.8 43.4 45.0 37.2 43.5 47.1 51.2 61.7 73.2 75.4 75.9 73.2 84.9 84.8 91.1 31.3 39.0 41.2 44.9

KP 65-75 Correlated 97.1 96.7 104.0 106.8 90.1 86.1 94.4 99.1 103.6 108.4 107.9 110.2 96.5 92.6 100.9 105.2 102.8 98.4 99.8 110.2 94.1 90.5 89.9 100.3 97.9 101.4 107.1 105.0 97.6 103.2 99.1 103.4 40.4 52.3 57.8 55.4 69.1 80.6 81.2 83.4 85.3 93.7 95.8 96.6 33.3 38.5 46.2 46.2 43.5 50.5 58.4 55.3 70.4 78.1 83.5 79.9 84.8 89.7 94.3 98.3 33.3 40.2 45.3 49.0

KP 75+ 134.6 113.2 120.7 150.3 147.5 130.6 136.9 164.1 168.4 148.5 155.7 179.9 110.6 90.1 96.2 128.4 136.2 117.9 126.5 155.8 149.5 129.7 139.0 165.1 167.8 150.7 152.9 181.1 112.0 94.5 96.3 129.4 87.9 94.8 97.9 116.1 124.5 121.1 132.3 153.4 141.9 149.9 148.3 172.8 85.7 89.3 92.0 111.9 89.0 88.9 101.3 116.3 120.7 122.9 136.7 150.2 144.2 148.0 155.0 172.8 86.9 90.7 93.4 112.3

KP 75+ Correlated 152.9 140.3 141.0 164.8 155.6 132.7 144.1 166.2 178.8 156.4 159.3 186.5 125.4 100.1 102.8 135.6 151.5 140.5 142.6 163.7 151.4 134.5 136.8 164.3 180.8 156.6 165.9 184.8 122.5 101.4 104.2 131.2 97.4 101.1 107.8 120.2 123.6 127.4 138.2 151.9 157.8 159.1 163.7 180.3 90.3 92.9 97.4 107.5 96.5 102.4 108.8 122.1 127.0 126.6 144.5 148.9 152.2 161.2 163.1 179.8 88.2 93.6 98.7 110.4

KP Total 55.3 61.4 60.3 66.9 63.8 65.3 66.0 69.5 64.6 63.3 69.6 73.9 73.2 71.1 65.7 77.3 61.3 57.6 64.7 68.2 62.1 58.7 61.7 65.8 67.2 67.1 68.0 71.8 73.8 69.7 73.0 79.0 28.6 36.9 37.5 42.3 52.4 58.2 54.3 59.3 56.5 60.1 65.7 62.5 27.9 34.1 31.7 36.5 28.0 35.5 38.0 39.5 49.3 54.7 58.5 59.9 54.5 61.7 65.4 67.1 27.4 30.7 33.4 35.8

KP Total Correlated 90.0 88.7 91.9 93.5 71.3 67.2 75.7 76.2 86.2 81.5 81.0 83.6 84.3 83.1 87.0 87.5 90.6 86.5 92.1 91.5 68.0 72.3 71.0 74.3 84.0 77.2 84.5 82.6 86.5 88.1 88.6 87.7 40.9 50.4 56.7 49.7 55.3 62.8 67.8 64.2 73.2 80.5 82.0 75.5 35.5 37.4 43.4 39.6 45.4 49.9 52.4 50.0 59.2 64.8 65.2 63.1 66.8 76.7 82.0 80.5 34.2 36.9 42.7 37.2

NDR

NDR 20-34 12.4 12.7 12.1 11.7 12.8 13.1 13.4 14.0 12.0 12.4 11.4 11.2 19.4 19.7 21.7 21.9 10.5 11.7 13.4 11.1 14.5 13.8 13.0 11.5 10.9 11.1 10.2 10.7 19.0 21.5 21.3 20.9 6.8 6.5 8.0 7.1 12.3 13.3 13.2 12.7 14.4 13.9 11.8 12.2 6.9 8.7 8.2 6.3 6.6 8.6 8.6 6.9 11.0 13.6 13.6 14.0 12.4 13.9 13.4 11.8 6.4 8.6 9.0 7.0

NDR 20-34 Correlated 13.9 15.8 14.5 13.9 14.0 15.2 14.5 13.3 14.2 14.5 12.2 12.5 20.4 21.1 21.8 21.9 15.1 16.4 14.0 13.8 14.3 14.8 16.4 13.6 13.0 14.2 14.7 12.9 19.8 21.2 21.8 19.8 8.3 10.7 8.6 7.1 15.1 17.0 14.0 14.9 14.9 16.1 14.4 14.8 6.5 11.3 9.2 7.6 8.0 8.0 9.6 10.0 13.9 16.3 17.2 13.9 13.7 15.5 15.6 14.9 7.1 10.9 9.7 8.1

NDR 35-50 12.1 16.7 12.5 13.8 17.0 17.0 15.5 16.0 8.0 13.6 10.5 11.7 29.4 31.6 31.8 28.9 12.8 13.0 11.3 10.5 13.7 18.3 17.8 16.0 9.3 14.7 11.5 10.6 28.7 30.9 30.7 30.4 2.0 8.2 6.8 4.1 15.3 16.6 18.2 16.4 11.5 13.5 13.6 11.8 1.7 7.8 7.5 4.1 1.4 9.3 5.7 4.8 15.4 19.4 17.4 16.6 12.6 15.8 14.2 13.5 2.2 9.3 7.1 4.4

NDR 35-50 Correlated 18.2 23.6 20.5 21.0 19.7 19.0 20.0 20.5 12.7 17.2 14.6 14.4 35.7 35.8 35.4 37.4 19.1 20.9 21.0 19.3 18.9 21.3 18.2 17.9 15.5 15.7 15.7 14.1 39.4 41.0 35.4 35.9 5.0 11.2 10.8 5.9 18.7 22.8 20.6 18.1 14.0 17.5 18.0 16.0 4.1 10.0 11.4 7.2 4.5 10.5 10.5 7.2 19.1 20.9 19.0 21.1 15.5 17.7 18.0 19.3 2.8 11.3 9.5 7.7

NDR 50-65 32.9 34.7 32.2 30.9 39.1 37.7 38.2 39.7 28.1 25.9 26.3 28.1 56.3 54.1 57.0 58.3 30.9 35.8 30.0 35.9 35.3 36.6 38.0 38.8 27.3 28.0 27.0 29.2 55.8 57.2 56.8 52.5 11.4 13.3 12.8 9.3 39.9 40.3 42.3 39.7 32.3 31.6 30.3 33.3 9.2 12.3 8.9 8.0 9.9 10.7 11.8 9.1 40.3 41.3 39.1 34.9 34.1 32.0 31.9 33.4 10.0 12.2 8.8 7.6

NDR 50-65 Correlated 36.1 35.2 33.3 31.3 34.1 33.3 37.8 38.2 25.1 28.9 27.9 31.8 58.2 53.7 57.3 55.2 31.1 33.1 34.4 32.3 33.5 35.6 35.8 35.4 29.4 30.4 26.4 27.8 55.6 54.7 57.2 57.7 10.6 12.2 13.6 7.2 33.7 35.2 34.6 33.4 32.3 34.2 33.6 32.3 10.1 11.2 10.5 8.9 11.9 13.3 10.3 8.6 37.2 41.1 38.5 37.2 31.3 34.3 31.2 32.6 8.9 10.0 10.3 6.5

NDR 65-75 52.4 50.6 51.7 51.1 54.0 52.9 52.5 56.1 47.0 46.7 48.5 45.7 64.3 54.2 58.3 57.4 56.2 51.9 54.2 55.4 57.0 50.8 48.2 52.2 43.9 49.6 46.1 46.9 54.1 57.9 59.3 59.6 19.0 11.4 9.4 14.7 55.9 57.8 55.7 53.6 55.3 55.0 52.6 56.6 14.6 8.6 5.7 7.1 18.1 15.2 14.5 13.3 57.1 53.6 52.9 54.3 56.5 53.8 57.3 54.9 12.0 6.6 9.5 8.4

NDR 65-75 Correlated 79.2 83.3 81.6 83.8 65.5 55.9 61.5 62.1 62.1 61.7 59.4 60.7 70.5 72.6 76.6 68.2 79.4 82.2 82.7 85.6 62.1 60.3 60.5 59.2 65.5 59.2 55.7 57.8 71.3 71.6 75.7 72.1 34.2 29.5 32.1 29.5 62.5 61.3 59.2 59.3 70.8 68.8 68.9 67.4 4.8 9.1 10.2 5.0 25.8 28.9 28.9 31.8 58.8 61.8 61.0 60.3 70.1 66.3 66.5 67.5 5.7 12.4 10.8 6.6

NDR 75+ 77.1 63.2 69.3 71.1 61.5 55.0 55.1 54.9 70.1 62.7 56.8 65.1 50.5 20.2 22.5 25.3 77.4 68.2 64.5 65.5 63.3 52.5 48.5 50.6 72.1 59.7 60.8 58.9 52.6 25.1 22.2 30.9 8.1 9.0 5.7 14.7 50.9 53.4 51.2 55.9 73.8 75.5 73.4 78.3 37.3 36.7 40.8 40.7 5.2 10.2 8.2 18.0 52.7 50.0 55.8 56.9 73.0 72.0 73.3 73.6 37.8 36.8 38.9 40.3

NDR 75+ Correlated 111.7 106.9 107.9 106.3 86.1 55.7 58.3 56.3 98.8 82.7 75.7 85.2 77.1 37.8 32.0 42.1 114.9 109.5 105.7 105.4 82.8 59.1 56.2 58.6 96.0 78.4 78.1 83.6 74.9 37.4 41.0 39.5 41.5 34.9 33.4 39.4 65.4 58.7 60.7 63.7 100.0 92.9 98.9 94.4 35.4 32.3 36.3 34.6 37.0 37.3 40.5 34.7 64.7 65.0 63.0 60.6 100.7 96.7 92.2 98.6 33.5 29.0 36.1 34.1

NDR Total 43.6 42.3 35.9 42.5 44.6 42.1 43.3 43.2 35.1 37.5 33.5 40.9 44.9 45.7 48.4 51.6 44.4 40.7 38.6 41.5 40.8 42.4 40.9 41.8 37.0 37.7 37.5 35.6 48.8 49.2 42.4 50.0 12.2 5.4 7.6 8.2 48.1 44.1 45.9 43.9 45.7 46.9 42.5 41.7 11.8 8.2 8.5 5.8 11.5 7.1 8.9 9.3 42.4 44.8 39.4 46.2 43.2 44.0 42.1 42.2 9.1 10.7 5.7 7.9

NDR Total Correlated 82.4 75.7 78.0 77.7 52.5 51.7 54.2 55.0 57.7 53.7 50.0 54.4 61.5 68.0 69.5 64.6 75.7 76.6 74.8 79.0 54.7 50.9 54.6 53.6 56.8 53.8 55.7 52.2 68.2 63.4 64.3 64.6 29.3 32.3 31.6 29.9 52.8 55.3 55.3 55.6 59.9 63.7 65.7 60.9 1.6 5.7 10.8 8.3 31.0 29.1 33.4 32.4 54.7 56.8 56.8 53.7 64.6 59.3 59.7 63.2 1.6 9.7 10.5 8.5

OVERALL MODEL RANKING RESULTS

Model Traits

Method_A1c 1 0 1 1 0 0 1 0 1 0 1 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 0 0 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 0 0 1 0 0 1 1 0 0 0

Method_BMI 1 0 1 1 0 0 1 0 1 0 1 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 0 0 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 0 0 1 0 0 1 1 0 0 0

Method_BP 1 0 1 1 0 0 1 0 1 0 1 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 0 0 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 0 0 1 0 0 1 1 0 0 0

Method_Lipids 1 0 1 1 0 0 1 0 1 0 1 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 0 0 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 0 0 1 0 0 1 1 0 0 0

Method_Smoke 1 0 1 1 0 0 1 0 1 0 1 0 1 1 0 0 1 0 1 0 1 0 1 0 1 0 1 1 1 1 0 0 0 1 1 1 0 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 0 0 1 0 0 1 1 0 0 0

Method_MI 4 4 1 4 4 4 4 1 4 4 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 3 2 2 3 2 2 3 3 3 2 3 3 3 3 3 3 3 1 1 1 3 3 3 3 1 1 4 1 1 4 4 1 4 4 4 4 4

Method_Stroke 1 1 1 2 2 4 4 1 3 3 2 2 4 3 4 3 1 1 2 2 4 4 3 3 2 2 3 1 1 4 1 3 1 2 2 3 4 3 1 2 2 4 4 3 2 3 1 3 4 1 4 2 4 2 3 1 3 2 4 4 1 3 4 1

Method_DeathCHD 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Method_DeathStroke 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

Method_TimeImprove 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 1 0 0 1 0 0 0 1 0 0 1 0 0 1 0 1 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

SORTED Weighted Mean 21.19 21.22 23.84 24.29 24.31 24.79 24.8 25.99 26.04 27.05 27.58 28.45 28.78 29.79 29.89 31.14 44.24 45.15 47.91 48.48 48.62 50.17 50.64 51.01 51.24 51.75 52.2 53.49 54.48 54.89 55.04 55.27 56.32 56.47 56.57 56.58 56.77 57.83 58.04 58.4 58.59 58.72 58.75 58.94 59.21 59.62 60.2 60.38 60.47 60.57 61.58 62.32 62.5 62.63 63.57 63.86 63.88 64.92 65.14 65.2 65.27 67.33 69.01 69.87

Multiple Risk Equations

Sun Grid Engine (SGE)

Anaconda AMI (Amazon Machine Image)

StarCluster

MIST (MIcro Simulation Tool)

The Reference Model is good for: • Economic forecast: Cost/Quality of Life • Comparison amongst studies • Evaluation of models – “consumers report” • Better model development • Visualizing the bigger picture of our understanding of disease progression • MultiScale Modeling: individual to population to multi-population

The Reference Model Accomplishments so far: 1. Disease models go out of date and need for temporal correction 2. Identify models that behave better/worse 3. Using end of trial biomarkers in equations is beneficial 4. Identify which diabetic studies are easier / harder to explain 5. Bio-marker correlation is not the main reason for our gaps in understanding disease progression

Lay Explanation • Clinical trials do not typically publish correlation amongst biomarkers • Therefore it is impossible to distinguish between the following populations:

• This work tested both these extreme scenarios for each population • The fitness results show:

• Correlation matters since a range in results is visible • Correlation does not change the big picture much

• Conclusions: • Modeling at the individual level is important and makes a difference • Correlation is not the main hurdle in understanding disease progression

Independent: Not Correlated

BP Normal LDL High

BP Normal LDL Normal

BP HIGH LDL High

Population A Population B

BP HIGH LDL Normal

Dependent: Perfect Correlation

Better fitness

Worst fitness

In Lay terms the fitness score shows the difference between simulation results and end of trial results for 1000 patient s

Fitness Score:

No Model Correction Bio-Marker Correction Model Outdating Correction Model Outdating and Bio-Marker Correction

Fitness Diff due to correlation:

Avg. 15 ± 17SD 1000 individuals

Fitness Diff due to correlation:

Avg. 15 ± 17SD 1000 individuals

5. Fitness Diff due to correlation: Avg. 15 ± 17SD

1000 individuals

Best Model

4. No model used can explain the phenomenon

observed in this cohort

1. The best 24 model variations include the temporal correction

for model outdating/ treatment

Time

Model Data Time Interval

Simulated Time Stamp

Adjustment Time

3. Using end of trial biomarkers produces a slightly better model

Possible Future Directions: • Optimizing MIST performance • Adding more populations • Adding more risk equations • Adding more disease processes • Additional analysis modes • Using Individual data • Formulating alternative fitness functions

Reproducible!!!

2. Identify superior models according to weighted average fitness score

Implies Need for HPC

Better models needed