March 25, 2009 Seattle, WA

Crafting Integrated Strategies to Prevent and Manage Chronic Disease

Using System Dynamics

Chronic Disease Academy

March 25, 2009

Seattle, WA

http://www.cdc.gov/index.htm

Presenters • Phil Huang

– Medical Director for City of Austin Department of Health and Human Services, formerly Chronic Disease Director for TX

• Patty Mabry – Office of Behavioral and Social Sciences Research,

National Institutes of Health• Bobby Milstein

– Coordinator, Syndemics Prevention Network, Centers for Disease Control and Prevention

• Diane Orenstein – Technical Lead, Division for Heart Disease and Stroke

Prevention, Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention

• Kris Wile – Sustainability Institute, System Dynamics Facilitator

and Modeler


Workshop Agenda

Wednesday, March 25

- Why are we here?

- Introduction to System Analysis in Public Health

- Introduction to the CV/Chronic Disease Risk Model

- Testing Strategies for Reducing Preventable CVD and Chronic Disease Costs

Demonstration

Self-guided Exploration of Results

Lunch

- Building an Integrated Chronic Disease Strategy

- Conclusions from model

- Dialogue: Importance of Context

- Lessons Learned

- Dialogue: Future Opportunities

Adjourn


Office of Behavior and Social Science’s Vision at NIH

To mobilize the biomedical, behavioral, and social science research communities as partners in interdisciplinary research to solve the most pressing health challenges faced by our society.

Programmatic Directions to Achieve the Vision:

– Transdiciplinary science

– “Next generation”, basic science

– Problem-based, outcomes oriented strengthen the science of dissemination

– Systems - thinking for population impact

The Importance of Partnership for OBSSR


http://www.nhgri.nih.gov/

http://www.nia.nih.gov/

http://www.niaaa.nih.gov/

http://www.niaid.nih.gov/

http://www.nibib.nih.gov/

http://www.nichd.nih.gov/

http://www.nidcd.nih.gov/

http://www.nidr.nih.gov/

http://www.niddk.nih.gov/

http://www.nida.nih.gov/

http://www.niehs.nih.gov/

http://www.nigms.nih.gov/

http://www.ninds.nih.gov/

http://www.nlm.nih.gov/

http://www.cit.nih.gov/

http://www.csr.nih.gov/

http://www.fic.nih.gov/

http://nccam.nih.gov/

http://ncmhd.nih.gov/

http://www.ncrr.nih.gov/

http://clinicalcenter.nih.gov/

http://www.nimh.nih.gov/

http://www.nei.nih.gov/

http://www.nci.nih.gov/

http://www.niams.nih.gov/

http://www.niddk.nih.gov/

http://www.nhlbi.nih.gov/index.htm

Adapted from Glass, McAtee (2006). Soc. Sci. Medicine, 62: 1650-1671

Health as a continuum between biological, behavioral and social factors across the lifespan and across generations


Simulation Modeling and Experimentation

• Pandemic flu

• Tobacco use

• Obesity, Diabetes

• Health inequalities

• “Non-health factors”

• Chronic disease

• Health care delivery

• Stress, mental illness, worksites, policy……….

Understanding the “Whole” System


2000 2001 2002 2003 2004 2005 2006 2007 2008

Selected Examples from CDC’s Growing Portfolio of Simulation Studies for Health System Change

SD Identified as a

Promising Methodology for Health System

Change Ventures

Upstream-Downstream

Dynamics

Neighborhood Transformation

Game

National Health Economics & Reform

Health ProtectionGame

Overall Health Protection Enterprise

Diabetes Action Labs

Obesity Overthe Lifecourse

Fetal & Infant Health

Syndemics Modeling*

Cardiovascular Health in Context

Selected Health Priority Areas


Questions Addressed by System Dynamics ModelingExploring Strategies to Redirect the Course of Change

Prevalence of Diagnosed Diabetes, US

0

10

20

30

40

1980 1990 2000 2010 2020 2030 2040 2050

Mill

ion

pe

op

le

HistoricalData

Markov Model Constants• Incidence rates (%/yr)• Death rates (%/yr)• Diagnosed fractions(Based on year 2000 data, per demographic segment)

Honeycutt A, Boyle J, Broglio K, Thompson T, Hoerger T, Geiss L, Narayan K. A dynamic markov model for forecasting diabetes prevalence in the United States through 2050. Health Care Management Science 2003;6:155-164.

Jones AP, Homer JB, Murphy DL, Essien JDK, Milstein B, Seville DA. Understanding diabetes population dynamics through simulation modeling and experimentation. American Journal of Public Health 2006;96(3):488-494.

Why?

Where?

How?

Who?

What?

Markov Forecasting Model

Simulation Experiments

in Action Labs


Time Series Models

Describe trends

Multivariate Stat Models

Identify historical trend drivers and correlates

Patterns

Structure

Events

Increasing:

• Depth of causal theory

• Robustness for longer-term projection

• Value for developing policy insights

• Degrees of uncertainty

Increasing:

• Depth of causal theory

• Robustness for longer-term projection

• Value for developing policy insights

• Degrees of uncertaintyDynamic Simulation Models

Anticipate new trends, learn about policy consequences,

and set justifiable goals

Tools for Policy Planning & Evaluation


Different Modeling Approaches For Different Purposes

Logic Models(flowcharts, maps or

diagrams)

System Dynamics(causal loop diagrams, stock-flow structures,

simulation studies, action labs)

Forecasting Models (regression models, Monte Carlo models)

• Articulate steps between actions and anticipated effects

• Improve understanding about the plausible effects of a policy

over time

• Focus on patterns of change over time (e.g., long delays, better before worse)

• Test dynamic hypotheses through simulation studies

• Inspire action through visceral, game-based learning

• Make accurate forecasts of key variables

• Focus on precision of point predictions and confidence intervals


Brief Background on System Dynamics Modeling

Compartmental models resting on a general theory of how systems change (or resist change) – often in ways we don’t expect

– Developed for corporate policies in the 1950s, and applied to health policies since the 1970s

– Concerned with understanding dynamic complexity

• Accumulation (stocks and flows)

• Feedback (balancing and reinforcing loops)

– Used primarily to craft far-sighted, but empirically based, strategies

• Anticipate real-world delays and resistance

• Identify “high leverage” interventions

– Modelers engage stakeholders through interactive workshops

Forrester JW. Industrial Dynamics. Cambridge, MA: MIT Press; 1961.

Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. Boston, MA: Irwin/McGraw-Hill; 2000.


What is a System? What are Dynamics?

System (Structure) = Stocks + Flows + Feedback Loops +…

• Stocks are accumulations of flows (of population, resources, changing goals, perceptions, etc.)

• Feedback loops link accumulations back to decisions that alter the flows: only 2 types (goal-seeking, self-reinforcing)

• Delays complicate things further

• As do non-linearities (need for critical mass, saturation effects)

Dynamics = Behavior over time

• Patterns in time series data (growth, fluctuation, etc.)

• Visible relationships of two or more variables (move together, move opposite, lead-lag, etc.)

StockFlow

Feedbackinfluence


An (Inter) Active Form of Policy Planning/Evaluation System Dynamics is a methodology to…

• Map the salient forces that contribute to a persistent problem;

• Convert the map into a computer simulation model, integrating the best information and insight available;

• Compare results from simulated “What If…” experiments to identify intervention policies that might plausibly alleviate the problem;

• Conduct sensitivity analyses to assess areas of uncertainty in the model and guide future research;

• Convene diverse stakeholders to participate in model-supported “Action Labs,” which allow participants to discover for themselves the likely consequences of alternative policy scenarios


Simulations for Learning in Dynamic Systems

Morecroft JDW, Sterman J. Modeling for learning organizations. Portland, OR: Productivity Press, 2000.

Sterman JD. Business dynamics: systems thinking and modeling for a complex world. Boston, MA: Irwin McGraw-Hill, 2000.

Multi-stakeholder Dialogue

Dynamic Hypothesis (Causal Structure)

X Y

Plausible Futures (Policy Experiments)

Obese fraction of Adults (Ages 20-74)

0%

10%

20%

30%

40%

50%

1970 1980 1990 2000 2010 2020 2030 2040 2050

Fra

ctio

n o

f p

op

n 2

0-74


Getting Oriented

• Introduction – Name, Organization, What you do– What are you hoping to get out of today?

• Then talk with others at your tables:– What are the largest strategic issues you

see in chronic disease?

• After 10 minutes, we’ll return to large group to share highlights– Biggest strategic challenges?


CDC Diabetes System Modeling ProjectDiscovering Dynamics Through State-based Action Labs & Models

Jones AP, Homer JB, Murphy DL, Essien JDK, Milstein B, Seville DA. Understanding diabetes population dynamics through simulation modeling and experimentation. American Journal of Public Health 2006;96(3):488-494.


Inflow

Volume

Outflow

Developing

Burden ofDiabetes

Total Prevalence(people with diabetes)

Unhealthy Days(per person with

diabetes)

Costs(per person with diabetes)

People withDiagnosedDiabetes

Diagnosis Deaths

ab

People withUndiagnosedPreDiabetes

Developing

DiabetesOnset

c

d

People withNormal

Blood SugarLevels

PreDiabetesOnset

Recovering fromPreDiabetes

e

DiabetesManagement

DiabetesDiagnosis

Obesity in theGeneral

Population

PreDiabetesDetection &

Management

People withUndiagnosed

Diabetes

Deaths

Diabetes Model: Diabetes Burden is Driven by Population Flows


Diabetes Burden is Driven by Population Flows

Inflow

Volume

Outflow

Developing

Burden ofDiabetes

Total Prevalence(people with diabetes)

Unhealthy Days(per person with

diabetes)

Costs(per person with diabetes)

People withDiagnosedDiabetes

Diagnosis Deaths

ab

People withUndiagnosedPreDiabetes

Developing

DiabetesOnset

c

d

People withNormal

Blood SugarLevels

PreDiabetesOnset

Recovering fromPreDiabetes

e

DiabetesManagement

DiabetesDiagnosis

Obesity in theGeneral

Population

PreDiabetesDetection &

Management

People withUndiagnosed

Diabetes

Deaths

Standard boundary

This larger view takes us beyond standard epidemiological models and most intervention programs


Diabetes System Dynamics Modeling ProjectConfirming Fit to Historical Trends (2 examples out of 10)

Diagnosed Diabetes % of AdultsObese % of Adults

0%

10%

20%

30%

40%

1980 1985 1990 1995 2000 2005 2010

Obese % of adults

Data (NHANES)

Simulated

0%

2%

4%

6%

8%

1980 1985 1990 1995 2000 2005 2010

Diagnosed diabetes % of adults

Data (NHIS)

Simulated


The growth of diabetes prevalence since 1980 has

been driven by growth in obesity prevalence Obese Fraction and Diabetes per Thousand

1300.7

850.35

400

1980 1990 2000 2010 2020 2030 2040 2050Time (Year)

Diabetes Prevalenc

e

Obesity Prevalenc

e

Risk multiplier on diabetes onset from

obesity = 2.6


Prevalence=92 AND RISING

Baseline Scenario: Obesity to increase little after 2006, diabetes keeps growing robustly for another 20-25 years

Obese Fraction and Diabetes per Thousand1300.7

850.35

400

1980 1990 2000 2010 2020 2030 2040 2050Time (Year)

Diabetes Prevalenc

e

Obesity Prevalenc

e

Diabetes prevalence keeps growing after obesity stops

WHY?

With high (even if flat) onset, prevalence tub

keeps filling until deaths (4-5%/yr)=onset

Onset=6.3 per thou

Estimated 2006 values

Death=3.8 per thou

Risk multiplier on diabetes onset from

obesity = 2.6


Unhealthy days impact of prevalence growth, as affected by diabetes management: Past and one possible future

Unhealthy Days per Thou and Frac ManagedObese Fraction and Diabetes per Thousand1300.7

850.35

400

1980 1990 2000 2010 2020 2030 2040 2050Time (Year)

Diabetes Prevalenc

e

Obesity Prevalenc

e

5000.65

25001980 1990 2000 2010 2020 2030 2040 2050

3750.325

Unhealthy Daysfrom Diabetes

Managed

fraction

Diabetes prevalence keeps growing after

obesity stops

If disease management gains end, the burden

grows

Reduction in unhealthy days per complicated case if

conventionally managed: 33%; if intensively managed: 67%


A Sequence of What-if Simulations

People with Diabetes per Thousand Adults150

125

100

75

501980 1990 2000 2010 2020 2030 2040 2050

Monthly Unhealthy Days from Diabetes per Thou500

450

400

350

300

250

1980 1990 2000 2010 2020 2030 2040 2050

Base

Base

Start with the base case or “status quo”: no improvements in diabetes management or prediabetes management after 2006


What if there were further Increases in Diabetes Management?


125

100

75

501980 1990 2000 2010 2020 2030 2040 2050


450

400

350

300

250

1980 1990 2000 2010 2020 2030 2040 2050

Base

Diab mgt Base

More people living with diabetes

Keeping the burden at bay for nine years

longer

Diab mgt

Increase fraction of diagnosed diabetes getting managed from 58% to 80% by 2015. (No change in the mix of conventional and intensive.) What do you think will happen?

Diabetes mgmt does nothing to slow the growth of prevalence—in

fact, it increases it. As soon as diabetes mgmt stops improving, unhealthy days start to grow as

fast as prevalence.


What if there was a huge push for Prediabetes Management?


125

100

75

50

1980 1990 2000 2010 2020 2030 2040 2050


450

400

350

300

250

1980 1990 2000 2010 2020 2030 2040 2050

Base

PreD mgmt

Base

PreD mgmt

The improvement is relatively modest—the growth is not stopped

Increase fraction of prediabetics getting managed from 6% to 32% by 2015. (Half of those under intensive mgmt by 2015.) No increase in diabetes mgmt. What do you think will happen?

Diabetes onset rate reduced 12% relative to base run. Not nearly

enough to offset the excess onset due to high obesity. By 2050,

diabetes prevalence reduced only 9% relative to base run.


Diabetes Model: What if Obesity is Reduced?Two Scenarios

Obese Fraction of Adult Population

0.4

0.3

0.2

0.1

01980 1990 2000 2010 2020 2030 2040 2050

Base

Obesity 25%

Obesity 18%

What if it were possible—in addition to the prediabetes mgmt intervention - to gradually lower the fraction obese from 34% (2006) to the 1994 value of 25% by 2030? Or, to the 1984 value of 18%?


Diabetes: What if we Managed Prediabetes AND Reduced Obesity?

The more you reduce obesity, the sooner you

stop the growth in diabetes—and the more

you bring it down

… Same with the burden of diabetes


125

100

75

50

1980 1990 2000 2010 2020 2030 2040 2050


450

400

350

300

250

1980 1990 2000 2010 2020 2030 2040 2050

Base

PreD mgmt

PreD & Ob 25%

PreD & Ob 18%

Base

PreD mgmt

PreD & Ob 18%

PreD & Ob 25%

What do you think will happen if, in addition to PreD mgmt, obesity is reduced moderately by 2030? What if it is reduced even more?

Why is obesity reduction so powerful? Mainly because of its strong effect on onset rate among prediabetics; but,

also, because it reduces PreD prevalence itself. However, achieving significant obesity reduction takes a

long time.


What if Intervened Effectively Upstream AND Downstream


125

100

75

50

1980 1990 2000 2010 2020 2030 2040 2050


450

400

350

300

250

1980 1990 2000 2010 2020 2030 2040 2050

Base

PreD mgmt PreD mgmt

Base

PreD & Ob 25%

Pred & Ob 25%

All 3 --PreD & Ob 25% & Diab mgmt

All 3

With a combination of effective upstream and downstream interventions we could hold the burden of diabetes nearly flat

through 2050!

With pure upstream intervention, burden still grows for many years before turning around. What do you think will happen if we add the prior diabetes mgmt intervention on top of the PreD+Ob25 one?

Downstream improvement acts quickly against burden but cannot continue

forever. Significant upstream gains are thus

essential but will likely take 15+ years to

achieve. A flat-burden future is possible but

requires simultaneous action on both fronts.


CDC Obesity Dynamics Modeling Project Contributors

Core Design Team• Dave Buchner• Andy Dannenberg• Bill Dietz• Deb Galuska• Larry Grummer-Strawn• Anne Hadidx• Robin Hamre• Laura Kettel-Khan• Elizabeth Majestic • Jude McDivitt• Cynthia Ogden• Michael Schooley

System Dynamics Consultants• Jack Homer• Gary Hirsch

Time Series Analysts

• Danika Parchment

• Cynthia Ogden

• Margaret Carroll

• Hatice Zahran

Project Coordinator• Bobby Milstein

Workshop Participants• Atlanta, GA: May 17-18 (N=47)• Lansing, MI: July 26-27 (N=55)

Homer J, Milstein B, Dietz W, Buchner D, Majestic D. Obesity population dynamics: exploring historical growth and plausible futures in the U.S. 24th International Conference of the System Dynamics Society; Nijmegen, The Netherlands; July 26, 2006.

Cover of "The Economist", Dec. 13-19, 2003Cover of "The Economist", Dec. 13-19, 2003.


Focusing on Life-Course Dynamics

• Explore likely consequences of possible interventions affecting caloric balance (intake less expenditure) – How much impact on obesity prevalence?

– How long will it take to see?

– Should we target particular subpopulations? (age, sex, weight category; lack data for race, ethnicity)

• Consider interventions broadly but leave details (composition, coverage, efficacy, cost) outside model boundary for now– Available data inadequate

– Would require a separate research effort to estimate these details

– Not addressing feedback loops of reinforcement and resistance

– Not addressing cost-effectiveness


Obesity Dynamics Over the Decades Dynamic Population Weight Framework

Dynamic Population Weight Framework

Population by Age (0-99) and Sex

Flow-rates betweenBMI categories

Overweight andobesity prevalence

Birth Immigration

Death

CaloricBalance

Yearly aging

NotOverweight

ModeratelyOverweight

ModeratelyObese

SeverelyObese

Trends and PlannedInterventions

Changes in the Physicaland Social Environment

Weight Loss/MaintenanceServices for Individuals

Data source: National Center for Health Statistics, CDC: National Health Examination Survey (NHES) 1960-1970, National Health and Nutrition Examination Survey (NHANES) 1971-2002.

Homer J, Milstein B, Dietz W, et al. Obesity population dynamics: exploring historical growth and plausible futures in the U.S. Proc. 24th Int’l System Dynamics Conference; Nijmegen, The Netherlands; July 2006.


Alternative Futures for Adult Obesity


0%

10%

20%

30%

40%

50%

1970 1980 1990 2000 2010 2020 2030 2040 2050

Fra

cti

on

of

po

pn

20-

74

Base SchoolYouth AllYouth

School+Parents AllAdults AllAges

AllAges+WtLoss


Results of Simulated InterventionsEnvironmental change approach

(reduce caloric balances to their 1970 values by 2015 for selected age ranges)

• Youth interventions have only small impact on overall adult obesity (assuming adult habits determined by adult environments—not by childhood1)

• Slow decline in overall adult obesity, even when program covers all ages

Targeted weight loss approach(obese lose 4 lbs per year, program terminated 2020)

• Such a program could accelerate progress and “buy time” for environmental change (but first, need to find a cost-effective program with lasting benefits—minimal relapse)

Need to assure caloric balance throughout all ages, particularly adulthood.

Contrast today’s narrow national focus on school-age youth.

Also need research on extent to which adult habits are determined by childhood.2

Need to assure caloric balance throughout all ages, particularly adulthood.

Contrast today’s narrow national focus on school-age youth.

Also need research on extent to which adult habits are determined by childhood.2

1. Christakis and Fowler. NEJM 357, 2007.

2. Bar-Or O., PCPFS Research Digest Series 2, No. 4, 1995.


Simulating the Dynamics of Cardiovascular Health and

Related Risk Factors

Work in Progress

This work was funded by the CDC’s Division for Heart Disease and Stroke Prevention and by the National Institutes of Health’s

Office of Behavioral and Social Science Research. The work was done in collaboration with the Health and Human Services

Department of Austin/Travis County, Texas, and with Integrated Care Collaboration of Central Texas. The external contractors are

Sustainability Institute and RTI International.

Homer J, Milstein B, Wile K, Pratibhu P, Farris R, Orenstein D. Modeling the local dynamics of cardiovascular health: risk factors, context, and capacity. Preventing Chronic Disease 2008;5(2). Available at http://www.cdc.gov/pcd/issues/2008/apr/07_0230.htm

Homer J, Milstein B, Wile K, Trogdon J, Huang P, Labarthe D, Orenstein D. Simulating and evaluating local interventions to improve cardiovascular health. In submission to Preventing Chronic Disease.


Cardiovascular Disease and Risks Remain Among the Leading Causes of Death

United States Texas

1. Heart Disease 26.6% 1. Heart Disease 25.7%

2. Cancer 22.8% 2. Cancer 21.9%

3. Stroke 5.9% 3. Stroke 6.0%

4. Chronic Lower Respiratory Disease

5.3% 4. Accidents 5.5%

5. Accidents 4.8%5. Chronic Lower Respiratory Disease

5.1%

6. Diabetes 3.1% 6. Diabetes 3.6%

*US: CDC/National Center for Health Statistics, Vol. 56, No.10, April 2008; TX: TX Dept. of State Health Services Preliminary Vital Statistics Table 16

Fraction of total deaths in 2005*…


Reducing Disability & Risk of

Recurrent CVD

Detecting & Treating Acute CVD Events

Controlling Increased

CVD Risk

Preserving Low

CVD Risk

From Healthy People 2010: 4 Levels of Prevention for Cardiovascular Diseases


Disability and Risk of CVD Recurrence

Acute CVD Events

Increased CVD Risk

Low CVD Risk

4 levels of prevention correspond to 4 States of Cardiovascular Health:


NUTRITION, PHYSICAL ACTIVITY & STRESS

• Salt intake• Saturated/Trans fat intake• Fruit/Vegetable intake• Net caloric intake• Physical activity• Chronic stress

CVD RISK FACTORPREVALENCE

& CONTROL

• Hypertension• High cholesterol• Diabetes• Obesity• Smoking• Secondhand smoke• Air pollution exposure

UTILIZATION OF SERVICES

• Behavioral change• Social support• Mental health• Preventive health

COSTS (CVD & NON-CVD) ATTRIBUTABLE TO

RISK FACTORS

LOCAL CONTEXT

• Eating & activity options

• Smoking policies

• Socioeconomic conditions

• Environmental policies

• Health care options

• Support service options

• Media and events

Local capacity for leadership & organizing

LOCAL ACTIONS

ESTIMATED FIRST-TIME CVD EVENTS

• CHD (MI, Angina, Cardiac Arrest)

• Stroke

• Total CVD (CHD, Stroke, CHF, PAD)

Preventing and Managing Risk Factors for CVD

Disability and Risk of CVD Recurrence

Acute CVD Events

Increased CVD Risk

Low CVD Risk


NUTRITION, PHYSICAL ACTIVITY & STRESS

• Salt intake• Saturated/Trans fat intake• Fruit/Vegetable intake• Net caloric intake• Physical activity• Chronic stress

CVD RISK FACTORPREVALENCE

& CONTROL

• Hypertension• High cholesterol• Diabetes• Obesity• Smoking• Secondhand smoke• Air pollution exposure

UTILIZATION OF SERVICES

• Behavioral change• Social support• Mental health• Preventive health

COSTS (CVD & NON-CVD) ATTRIBUTABLE TO

RISK FACTORS

LOCAL CONTEXT

• Eating & activity options

• Smoking policies

• Socioeconomic conditions

• Environmental policies

• Health care options

• Support service options

• Media and events

Local capacity for leadership & organizing

LOCAL ACTIONS

ESTIMATED FIRST-TIME CVD EVENTS

• CHD (MI, Angina, Cardiac Arrest)

• Stroke

• Total CVD (CHD, Stroke, CHF, PAD)

Interventions Through Local Context

Homer J, Milstein B, Wile K, Pratibhu P, Farris R, Orenstein D. Modeling the local dynamics of cardiovascular health: risk factors, context, and capacity. Preventing Chronic Disease (in press).


Purpose of the Cardiovascular Risk Model

• How do local conditions affect multiple risk factors for CVD, and how do those risks affect population health status and costs over time?

• How do different local interventions affect cardiovascular health and related expenditures in the short- and long-term?

• How might local health leaders better balance their policy efforts given limited resources?

The CDC has partnered with the Austin (Travis County), Texas, Dept. of Health and Human Services. The model is calibrated to represent the

overall US, but is informed by the experience and data of the Austin team, which has been supported by the CDC’s “STEPS” program since 2004.

The CDC has partnered with the Austin (Travis County), Texas, Dept. of Health and Human Services. The model is calibrated to represent the

overall US, but is informed by the experience and data of the Austin team, which has been supported by the CDC’s “STEPS” program since 2004.

Homer J, Milstein B, Wile K, Pratibhu P, Farris R, Orenstein D. Modeling the local dynamics of cardiovascular health: risk factors, context, and capacity. Preventing Chronic Disease 2008;5(2). Available at http://www.cdc.gov/pcd/issues/2008/apr/07_0230.htm

Smoking

Obesity

Secondhandsmoke

Healthinessof diet

Extent ofphysical activity

Psychosocialstress

Diagnosisand control

First-time CVevents and

deaths

Access to and marketingof smoking quit products

and services

Access to andmarketing of mental

health services

Sources ofstress

Access to andmarketing of healthy

food options

Access to andmarketing of physical

activity options

Access to andmarketing of weight

loss services

Access to andmarketing ofprimary care

Particulate airpollution

Utilization ofquality primary

care

Tobacco taxes andsales/marketing

regulations

Smoking bans atwork and public

places

Junk food taxes andsales/marketing

regulations

Downwardtrend in CV

event fatality

Quality of primarycare provision

Chronic Disorders

Costs from CV and other riskfactor complications and

from utilization of services

Anti-smokingsocial marketing

High BP

Highcholesterol

Diabetes Populationaging

Air pollutioncontrol regulations


Direct Risk Factors

Smoking

Secondhandsmoke


deaths


Downwardtrend in CV

event fatalityChronic Disorders

High BP

Highcholesterol



Indirect Risk Factors

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths



care

Downwardtrend in CV


High BP

Highcholesterol



Tobacco and Air Quality Interventions

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services



care


regulations

Downwardtrend in CV



High BP

Highcholesterol



Air Quality Interventions

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress

Diagnosis andcontrol

First-time CVevents and deaths


and services



care


regulations


places

Downward trend inCV event fatality

Chronic Disorders


High BP

Highcholesterol

Diabetes


Populationaging


Health Care Interventions

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services




care


regulations

Downwardtrend in CV

event fatality


Chronic Disorders


High BP

Highcholesterol



placesAir pollution

control regulations


Interventions Affecting Stress

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress




care


regulations

Downwardtrend in CV

event fatality


Chronic Disorders


High BP

Highcholesterol



placesAir pollution

control regulations


Healthy Diet Interventions

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress


food options




care


regulations


regulations

Downwardtrend in CV

event fatality


Chronic Disorders


High BP

Highcholesterol



placesAir pollution

control regulations


Physical Activity & Weight Loss Interventions

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress


food options


activity options


loss services




care


regulations


places


regulations

Downwardtrend in CV

event fatality


Chronic Disorders


High BP

Highcholesterol

Diabetes


Populationaging


Adding Up the Costs

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress


food options


activity options


loss services




care


regulations


places


regulations

Downwardtrend in CV

event fatality


Chronic Disorders

Costs from CV and other riskfactor complications and from

utilization of services


High BP

Highcholesterol

Diabetes


Populationaging


Adding Up the Costs

Cardiovascular event costs• Medical costs (ER, inpatient, rehab)—for non-fatal & fatal events• Productivity (morbidity) losses* from non-fatal events• Productivity (premature mortality) losses* from fatal events

Non-cardiovascular complications of risk factors• Hospital costs due to non-CV complications of diabetes (e.g., kidneys,

eyes, feet), high BP, & smoking• Productivity (morbidity) losses* from non-fatal complications of diabetes,

high BP, smoking, & obesity• Productivity (mortality) losses* from fatal complications of smoking (e.g.,

cancer, COPD), diabetes, high BP, & obesity

Costs of managing risk factors• Medications & visits for diabetes, high BP, high cholesterol—by level of

care (high quality = 2 – 2.5x cost of mediocre care)• Other services: Mental health services, Weight loss services, Smoking

quit services & products

Human capital approach based on: Haddix, Teutsch, Corso, Prevention Effectiveness, 2003 (2nd ed, Tables 1.1b and 1.1c).


Relative size of included Complication Costs

CV EventsNon-CV

Complications of Diabetes

Non-CV Complications

of Hypertension


of Smoking


of Obesity

Direct medical costs of

complications++ ++* +* +* 0*

Indirect productivity

losses: disability

++ ++* +* +* +*

Indirect productivity

losses: premature

death

++ ++ + ++ +

* Non-CV hospitalization costs & lost workdays estimated from MEPS 2000-03 linked with NHIS. The regression analysis controlled for demographics, CVD, and unrelated diseases (e.g., HIV).


Data Sources for Modeling CVD Risk• Census

– Population, deaths, births, net immigration, health coverage

• AHA & NIH statistical reports – Cardiovascular events, deaths, and prevalence (CHD, stroke, CHF, PAD)

• National Health and Nutrition Examination Survey (NHANES) – Risk factor prevalences by age (18-29, 30-64, 65+) and sex (M, F)– Chronic disorder diagnosis and control (hypertension, high cholesterol, diabetes)

• Behavioral Risk Factor Surveillance System (BRFSS)– Diet & physical activity– Primary care utilization– Lack of needed emotional/social support Psychosocial stress

• Medical Examination Panel (MEPS) / National Health Interview (NHIS) – Medical and productivity costs attributable to smoking, obesity, and chronic disorders

• Research literature– CVD risk calculator, and relative risks from SHS, air pollution, obesity, and inactivity– Medical and productivity costs of cardiovascular events

• Questionnaires for CDC and Austin teams (expert judgment)– Potential effects of social & services marketing on utilization behavior– Effects of behavioral services on smoking, weight loss, stress reduction– Relative risks of stress for high BP, high cholesterol, smoking, and obesity


Calculating First-Time CV Events & Deaths

Based on well-established Framingham approach for calculating probability of first-time events & deaths in individuals• CVD = CHD (MI, angina, cardiac arrest) + Stroke/TIA + CHF + PAD

Modifies individual-level risk calculator for use with populations• Uses prevalences of uncontrolled chronic disorders by sex/age group

• Introduces secondhand smoke and pollution as additional risk factors

• Combines risks multiplicatively to account for overlapping conditions

• Adjustment exponents reproduce synergies seen in individual-level calculator

• Adjustment multipliers reproduce AHA event and death frequencies for 2003

- Anderson et al, Am Heart J 1991 (based on Framingham MA population N=5573, 1968-1987)

- Homer “Risk calculation in the CVD model” project document, June 19, 2007

- NHANES 1988-94 & 1999-04

- AHA Heart Disease and Stroke Statistics – 2006 Update


Interactive Model Guide

• Details key assumptions and sources of evidence for each relationship in the model

• On the CD ROM for participant today– Called “CVD interactive guide v8m.ppt”

– Functions in slide show mode

• Afterward, we will have an opportunity for remote Q&A with Jack Homer, Lead Modeler, and Justin Trogdon, Cost data expert.


Where are your efforts to manage chronic disease?

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress


food options


activity options


loss services




care


regulations


places


regulations

Downwardtrend in CV

event fatality


Chronic Disorders




High BP

Highcholesterol

Diabetes


Populationaging


Individual Strategy Fly-by Exercise

Your goal is to reduce CV deaths and reduce total risk factor consequence costs

… but you have limited resources.

• Pick 6 interventions for your strategy• Record your selected interventions on

your sheet under Section #1• Questions?• You have 10 minutes.


A Base Case Scenario for Comparison Assumptions for Input Time Series through 2040

• A plausible and straightforward scenario– Assume no further changes in

contextual factors affecting risk factor prevalences

– Any changes in prevalences after 2004 are due to “bathtub” adjustment process and population aging

– Provides an easily-understood basis for comparisons

• Prior to 2004, model reflects declining …– Fraction workplaces allowing

smoking (1990-2003)

– Air pollution (1990-2001)

– Youth smoking (rise 1991-99, decline 1999-2003)

– CV event fatality (1990-2003)

Total RF Complication Costs per Capita

2,000

1,000

0

1990 2000 2010 2020 2030 2040

Complication Costs per 1000 if all risk factors = 0

Also note: Cost minimum if all proximal risk factor prevalences were zero.

Consequence costs would decrease 80%CV death rate would be 60% below the base case.

3,000

No Further Changes in Drivers

No Further Changes in Drivers


Base run behaviors

CVD & Risk Factor Complication Costs and CVD Mortality

0.6

0.3

0

1990 2000 2010 2020 2030 2040

Smoking Prevalence

Air Pollution PM2.5

Diabetes Prevalence

High BP Prevalence

High Cholesterol Prevalence

CV Risk Factor Prevalences30

15

0

Obese Adults

Newly obeseadults

Becoming non-obese or

dying

2040

0

0.4% Obese

1990

Result: Past trends level off after 2004, after which results reflect only slow “bathtub” adjustments in risk factors

• Increasing obesity, high BP, and diabetes

• Decreasing smoking and air pollution

• Increases in risk factors and population aging lead to eventual rebound in deaths

(Air

pollu

tion

only

)

1990 2000 2010 2020 2030 2040

4

3

2

1

0

Deaths from CVD per 1000 if all risk factors = 0

Deaths from CVD per 1000

Complication Costs per 1000

Complication Costs per 1000 if all risk factors = 0

3,000

2,250

1,500

750

0


Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress


food options


activity options


loss services




care


regulations


places


regulations

Downwardtrend in CV

event fatality


Chronic Disorders

Costs from CV and other riskfactor complications and

from utilization of services


High BP

Highcholesterol


Base case behavior for 1990-2040

1

0Use of Primary Care Services

0.3

0

Stress Prevalence

0.8

0

Poor DietFraction

0.8

0

Inadequate Physical Activity

4

0

0.3

0Smoking

Prevalence

0.4

0

Obesity Prevalence

0.6

0

Secondhand Smoke

Exposure

0.6

0Diabetes

High BP

High cholesterol

30

0

Particulate Air Pollution

PM2.5

3,000

0

CVD & Risk factorcosts per capita

Uncontrolled

CVD Deaths per 1000

Prevalences

mcg per m3

Age 65+ fraction of the population

CV event fatality multiplier

0.3

0

1.5

0


New quality ofprimary care

PRIMARY CARE INTERVENTIONS

PHYSICAL ACTIVITY INTERVENTIONS

AIR QUALITY INTERVENTIONS

TOBACCO INTERVENTIONS

NUTRITIONAL INTERVENTIONS

INTERVENTIONS AFFECTING STRESS

New PC servicesmarketing

New access toprimary care svcs

New multiplier onair pollution

New multiplier onworkplaces allowing

smoking

New social marketingfor healthy diet

New access tohealthy diet

New junk food taxand sales restrict

New socialmarketing for PA New access to PA

New WL servicesmarketing

New access toweight loss svcs

New socialmarketing

against smoking

New tobacco taxand sales restrict

New SQ servicesmarketing

New access tosmoking quit svcs and

products

New multiplier onsources of stress

New MH servicesmarketing

New access tomental health svcs

INDIVIDUAL INTERVENTIONS SELECTOR

WEIGHT LOSS INTERVENTIONS

Area of effect

and type of intervention

– Increasing access – Marketing of

services– Social marketing – Taxes and/or

sales restriction– Others

Intervention Options


Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress


food options


activity options


loss services




care


regulations


places


regulations

Downwardtrend in CV

event fatality


Chronic Disorders




High BP

Highcholesterol

Diabetes


Populationaging

Tracing interventions through the system- Increasing access to physical activity options


Interpreting Cost Results

• Complication costs are for CV and non-CV related complications, both direct and indirect

• Management costs include

– Annual costs for services provided

– Medication costs

• When these costs are less than baseline, the difference is the per capita health cost savings per year – the maximum economically justifiable spending for the intervention

Complication & Management Costs per Capita

3,000

2,000

1990 2000 2010 2020 2030 2040

*

Average annual savings of *$ 49 per capita from interventions to

increase access to physical activity options from 2010 - 2040.

Base Case

Increased Access to Physical Activity options


Develop and Test Your Team Strategies

• Form groups of 4 or 5 people• Same goals –

Reduce CV deaths and total risk factor consequence costs with limited resources.

• Choose 6 interventions for your strategy• Prepare a flipchart to present your strategy.• You have 15 minutes.

• We will test some team strategies with the simulator.


Interactive Results Exploration

Purpose: To develop conclusions about which strategies are most effective to achieve our goals

1. Work in groups of 2-3 with a laptop.

2. As you explore, fill out Section #2 of worksheet.– Which interventions were more or less

powerful than you expected? – Your conclusions? Ideas?

• Please note your questions so we can explore them in the larger group.

• You have 25 minutes.


Break for Lunch

We will begin again promptly at 1:00 pm.

This afternoon, we will:– Revise and test team strategies.

– Share conclusions from this systemic analysis.

– Explore opportunities


Revising Team Strategies

1. Teams choose their goals. What are you trying to achieve? What is your timeframe?

2. Put together a strategy, choosing six different interventions that you think will work.

3. Record on flipchart.

4. We will try them out with the simulator.

5. You have 10 minutes.


Conclusions: Comparing intervention groups

Care• Primary Care Quality = 75%• PC Marketing = 100%• PC Access = 100%

Lifestyle• Physical Activity Access =

100%• Physical Activity Social

Marketing = 100%• Access to Healthy Nutrition =

100%• Healthy Nutrition Social

Marketing = 100%• Stress Multiplier = ½

Air• Tobacco Tax = 100% • Marketing Against Smoking =

100%• Air Pollution Multiplier = ½ • Smoking Bans = 100%

New quality ofprimary care

PRIMARY CARE INTERVENTIONS

PHYSICAL ACTIVITY INTERVENTIONS

AIR QUALITY INTERVENTIONS

TOBACCO INTERVENTIONS

NUTRITIONAL INTERVENTIONS

INTERVENTIONS AFFECTING STRESS

New PC servicesmarketing

New access toprimary care svcs

New multiplier onair pollution

New multiplier onworkplaces allowing

smoking

New social marketingfor healthy diet

New access tohealthy diet

New junk food taxand sales restrict

New socialmarketing for PA New access to PA

New WL servicesmarketing

New access toweight loss svcs

New socialmarketing

against smoking

New tobacco taxand sales restrict

New SQ servicesmarketing

New access tosmoking quit svcs and

products

New multiplier onsources of stress

New MH servicesmarketing

New access tomental health svcs

INDIVIDUAL INTERVENTIONS SELECTOR

WEIGHT LOSS INTERVENTIONS


Comparing Care, Air & Lifestyle Interventions

• Care provides – quick and sustained

reduction in CV events, – but little cost savings.

• Air provides – quick and growing

reduction in CV events, – and major cost savings.

• Lifestyle provides– Growing CV event

reductions over time, but little immediately

– Substantially increasing cost savings over time

Deaths from CVD per 10004

2

0

1990 2000 2010 2020 2030 2040

Base CaseCare

Care + Air

Care + Air + Lifestyle

If all risk factors = 0

Complication & Mgmt Costs per Capita3,000

0

1990 2000 2010 2020 2030 2040

Base Case

Care

Care + Air

Care + Air + Lifestyle

If all risk factors = 0


Cost Conclusions• AIR – Smoking and air quality interventions can save

lives quickly and can justify intervention spending up to $300 per capita for 30 years ($355 in ET).

• CARE – Improving utilization and quality of primary care services can save lives quickly, but should not be expected to save much on total costs. Justified intervention spending could be up to $25 per capita for 30 years ($35 in ET).

• LIFESTYLE – Improving nutrition and physical activity, and reducing sources of stress take longer to affect CV events though obesity and chronic conditions. However their contribution grows over time and intervention spending of up to $100 per capita could be justified ($177 in ET).


Comparing E. Travis to US: More Effective Individual Interventions

After 10 years (2015) After 35 years (2040)

CVD Death Rate

Compl + Mgmt Costs

CVD Death Rate

Compl + Mgmt Costs

ET US ET US ET US ET US

Social Marketing Against Smoking 4 4 1 1 4 3 1 1

Quality of Primary Care 1 1 3 1 1

Tobacco Tax and Sales Restrictions 5 2 2 2 2

Air Pollution 3 2 4* 3 5 2 4

Access to Primary Care 2 3 4 4 2 4

Access to Physical Activity 5 2 4 3 3

Access to Healthy Diet 4 4

Stress Reduction 4 5

*Duplicates ranks indicate ties.


Overall Conclusions• CV death rate has declined due to improvements

in acute care, and also reductions in smoking, second hand smoke, and air pollution.

• Risk factor consequence costs have decreased as a consequence, but also because of reductions in smoking related deaths.

• Smoking will probably continue to decline in growing elderly population, helping to lower costs.

• Of 19 interventions, at least 15 have the potential to reduce CVD events without increasing costs.


Revise Individual Strategies

With this additional information, would you adjust your individual strategy? Why?

• Please record selected interventions on your sheet under Section #3

• You have 10 minutes.


Integrating your contextual knowledge for integrated public health policy

1. Are there gaps in specific policy areas? Where?

2. Where are the opportunities?

3. What new partnerships can be identified now?


How we can make it happen

1. What partnerships or processes need to be created or strengthened?

2. Make a personal commitment

To making those connections and to collaborating

Indicate your commitment in Section 5 of your worksheet


Model Boundaries / Limitations

• More contextual information must come into play. • Decision support tool to inform multi-stakeholder

dialogue.• Local experts provide crucial link to relevant data

and implementation.

Multi-stakeholder Dialogue


X Y


X YX Y



0%

10%

20%

30%

40%

50%

1970 1980 1990 2000 2010 2020 2030 2040 2050

Fra

ctio

n o

f p

op

n20

-74



0%

10%

20%

30%

40%

50%

1970 1980 1990 2000 2010 2020 2030 2040 2050

Fra

ctio

n o

f p

op

n20

-74


Societal Dialogue Incorporates Model Omissions

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress


food options


activity options


loss services




care


regulations


places


regulations

Downwardtrend in CV

event fatality


Chronic Disorders




High BP

Highcholesterol

Diabetes


Populationaging

Other chronic disease

endpoints

Downstreaminterventions

and costs

Local implementationopportunitiesLocal

implementationstrengths

and success

Political will

• What measures of improvement ought to be included?

• What else is missing?

• What would be helpful to you?

SYSTEMDYNAMICS MODEL

STRATEGICDIALOGUE

Implementationactions and costs

Health inequities

Local leadershipcapacity

Ability to engage all

stakeholders

Borderline conditions


What we have learned

• The simulator and surrounding dialogue can be used to:– Create alignment among stakeholders– Initiate systemic thinking, increasing

leadership capacity– Spur people to action– Identify opportunities and build commitment

to address them– Inform the development of business cases

for investment in interventions


Sample Austin Implementation Worksheet

InterventionArea

Schools Community Worksites Healthcare

Physical Activity

Nutrition

Tobacco

CVD

Diabetes

Cancer


Integrative Modeling for Strategy Building

• Integrating large and varied sources of evidence in simulations is fundamentally useful.

• Complex problems, those with many causal pathways and significant time delays, are suitable.

• All models are limited, so they are useful as decision-support tools.

• Local context affects appropriate strategy. • Translucent box – making assumptions visible

can develop trust and leadership capacity. • Don’t forget the “ask”. Give people an

opportunity to take action.


What opportunities do you see for integrated policy making?

Smoking

Obesity

Secondhandsmoke

Healthinessof diet


Psychosocialstress



deaths


and services


health services

Sources ofstress


food options


activity options


loss services




care


regulations


places


regulations

Downwardtrend in CV

event fatality


Chronic Disorders




High BP

Highcholesterol

Diabetes


Populationaging

Other chronic disease

endpoints

Implementationactions and costs

Downstreaminterventions

and costs

Health inequities

Local implementationopportunitiesLocal

implementationstrengths

and success

Local leadershipcapacity

Political will

Ability to engage all

stakeholders

• We plan to extend this model– Borderline conditions, ex-

smokers– Downstream

interventions and costs

• Investigate transferability of this model to other locales

• Tools allowing wider dissemination

• What do need for this to be useful to you?

• Needs for other systemic analyses?

SYSTEMDYNAMICS MODEL

STRATEGICDIALOGUE

Borderline conditions


OBSSR at NIH

Vision: To mobilize the biomedical, behavioral, and social science research communities as partners in interdisciplinary research to solve the most pressing health challenges faced by our society.

27 NIH Institutes and Centers and the extramural community.

Programmatic Directions to Achieve the Vision:

– Trans-/inter-disciplinary science

– “Next generation”, basic science

– Problem-based, outcomes oriented strengthen the science of dissemination

– Systems science for population impact


What is the challenge?

• Other approaches alone have not solved intractable health problems

• Health problems are embedded in dynamically complex systems

• Policies, programs, interventions have limited resources and involve trade offs

• Could try “kitchen sink” approach, but resources are limited

• Could try a “thought experiment” but the human mind cannot execute beyond simple


Systems Science Activities at NIH

• 2007 Symposia Series on Systems Science & Health

• Institute on Systems Science and Health (annually)

• BSSR-Systems Science listserv - send email to [email protected]

• CDC SD Modeling with OBSSR and NHLBI

mailto:[email protected]


Examples of NIH Modeling Initiatives

Cancer Intervention and Surveillance Modeling Network (CISNET): http://cisnet.cancer.gov/about/

Interagency Modeling and Analysis Group (IMAG): http://www.imagwiki.org/mediawiki

http://grants.nih.gov/grants/guide/pa-files/PAR-08-023.htm

Models of Infectious Disease Agent Study (MIDAS): http://www.nigms.nih.gov/Initiatives/MIDAS

NIH Guide To Grants And Contracts http://grants.nih.gov/grants/guide/index.htmlTo Subscribe to the NIH Guide LISTSERV, send an e-mail to

[email protected] with the following text in the message body (not the "Subject" line): subscribe NIHTOC-L your name


Active NIH FOA’s in Systems Science and BSSR

• PAR-08-224 Using Systems Science Methodologies to Protect and Improve Population Health (R21). Expires Sept 2011. 3 receipt dates per year.

Contact Patty Mabry, OBSSR.

• PAR-08-212, -213, -214 Methodology and Measurement in the Behavioral and Social Sciences (R01, R21, R03). Expires September 2011. 3 receipt dates per year.

Contact Deb Olster, OBSSR.

• RFA-07-079, -080 Behavioral and Social Science Research on Understanding and Reducing Health Disparities (R01, R21) Expires September 2009. One receipt date per year Sept.

Contact: Ron Abeles, OBSSR.

• PAR-08-023 Predictive Multiscale Models of the Physiome in Health and Disease (R01). Expires September 2010. 3 receipt dates per year.

Contact: Grace Peng, NIBIB.


Grant Funded Systems Science and BSSR at NIH

• Joshua Epstein, Director’s Pioneer Award, NIGMS, OBSSR, 2008. Project Title: Behavioral Epidemiology: Applications of Agent-Based Modeling to Infectious Disease.

• David Lounsbury, R03, NIDA, 2008. Project Title: Dynamics Modeling as a Tool for Disseminating the PHS Tobacco Treatment Guideline

• David T. Levy, U01, NCI, 2002-2010. CISNET. Project Title: A Simulation of Tobacco Policy, Smoking and Lung Cancer.

• Linda Collins & Daniel Rivera, R21, 2007-2010. NIH Roadmap. Dynamical System /Related Engineering Approach /Improving Behavioral Intervention

• Daniel Rivera, K25, NIDA, OBSSR. Control Engineering Approaches to Adaptive Interventions in Drug Abuse Prevention.

• PAR-08-224 – Awards pending. • RFA-HD-08-023 (R01), Innovative Computational and Statistical

Methodologies for the Design and Analysis of Multilevel Studies on Childhood Obesity (R01). Awards pending.


For more information:

Patty Mabry, [email protected]

Office of Behavioral and Social Sciences Research

(OBSSR) National Institutes of Health

http://obssr.od.nih.gov


Check-out and evaluation

Please fill out Sections 4 and 5 of your worksheet.

Do you have any thoughts you would like to offer? Questions? Reactions? Plans? Feedback?


Chronic Disease Leaders

THANK YOU!

Small favors:

• Please leave your Strategy Worksheets with us.

• Take a CD-ROM for your own use. We encourage you to share.


Documents

March 25, 2009 Seattle, WA