Overview of Issues Related to Measuring Food and Nutrition

Policies and Environments

George A. Weiss University Professor

Workshop on Measurement Strategies for Accelerating Progress in Obesity Prevention

Overview of Issues Related to Measuring Food and Nutrition

Policies and Environments

Karen Glanz, PhD, MPHGeorge A. Weiss University Professor

University of Pennsylvania

Workshop on Measurement Strategies for Accelerating Progress in Obesity Prevention

Ø Scope of food environments and policies

Ø Types of measures & examples in settings

Ø Assessing food environments & policies: evaluating population strategies

Ø Limitations of measures to accelerate progress

Ø What we need to accelerate progress

Scope of food environments and policies

Types of measures & examples in settings

Assessing food environments & policies: evaluating population strategies

Limitations of measures to accelerate progress

What we need to accelerate progress

To make significant progress in the area of eating & nutrition environments…* we need valid, reliable measures of nutrition environments and policies… that are also practical

The balance between scientific rigor and practicality is challenging

To make significant progress in the area of eating & nutrition

we need valid, reliable measures of nutrition environments and policies… that are also practical

The balance between scientific rigor and practicality is challenging

An Ecological Framework Depicting the Multiple Influences on What People Eat

Individual Factors

(personal)

Social Environment(networks)

MacroEnvironments

(sectors)

Physical Environments

(settings)

HomeWorksitesSchool, AfterschoolChild-careNeighborhoods & CommunitiesRestaurants & fast food outletsSupermarketsConvenience & corner stores

=Access=Availability=Barriers=Opportunities

Cognitions (e.g. attitudes, preferences, knowledge, values)Skills and behaviorsLifestyleBiological (e.g. genes, gender, age)Demographics (e.g. income, race/ethnicity)

=Outcome expectations=Motivations=Self-efficacy=Behavioral capability

An Ecological Framework Depicting the Multiple Influences on What People Eat

Individual Factors

(personal)

Social Environment(networks)

Macro-level Environments

(sectors)

Physical Environments

(settings)

=Societal and cultural norms and values

=Food and beverage industry=Food marketing and media=Food and agriculture policies=Economic systems=Food production & distribution

systems=Government & political

structures and policies=Food assistance programs=Health care systems=Land use and transportation

=Practices=Legislative,

regulatory, or policy actions

=Family=Friends=Peers

=Role modeling=Social support=Social norms

Story et al., ARPH, 2008

Social Environment(networks)

MacroEnvironments

(sectors)

Physical Environments

(settings)

HomeWorksitesSchool, AfterschoolChild-careNeighborhoods & CommunitiesRestaurants & fast food outletsSupermarketsConvenience & corner stores

=Access=Availability=Barriers=Opportunities

Story et al., ARPH, 2008

Social Environment(networks)

Macro-level Environments

(sectors)

Physical Environments

(settings)

=Societal and cultural norms and values

=Food and beverage industry=Food marketing and media=Food and agriculture policies=Economic systems=Food production & distribution

systems=Government & political

structures and policies=Food assistance programs=Health care systems=Land use and transportation

=Practices=Legislative,

regulatory, or policy actions

=Family=Friends=Peers

=Role modeling=Social support=Social norms

MacroEnvironments

(sectors)

Physical Environments

(settings)

HomeWorksitesSchool, AfterschoolChild-careNeighborhoods & CommunitiesRestaurants & fast food outletsSupermarketsConvenience & corner stores

=Access=Availability=Barriers=Opportunities

Story et al., ARPH, 2008

Macro-level Environments

(sectors)

Physical Environments

(settings)

=Societal and cultural norms and values

=Food and beverage industry=Food marketing and media=Food and agriculture policies=Economic systems=Food production & distribution

systems=Government & political

structures and policies=Food assistance programs=Health care systems=Land use and transportation

=Practices=Legislative,

regulatory, or policy actions

Model of Community Nutrition Environments

[Glanz, Sallis, Saelens, & Frank 2005]

Policy Variables Environmental Variables

GovernmentGovernmentandand

Industry PolicyIndustry Policy

Community Nutrition Community Nutrition EnvironmentsEnvironments

•Type & Location of FoodOutlets (stores, restaurants)

•Accessibility – hours of Operation, drive-thru)

Organizational Organizational Nutrition EnvironmentsNutrition Environments

Home

Work

Consumer Nutrition EnvironmentConsumer Nutrition Environment•Available healthy options

•Price, promotion, placement

•Nutrition Information

Information Information EnvironmentEnvironment

(Media, Advertising)(Media, Advertising)

Model of Community Nutrition Environments

[Glanz, Sallis, Saelens, & Frank 2005]

Individual Variables Behavior

Organizational Organizational Nutrition EnvironmentsNutrition Environments

School

Other

Consumer Nutrition EnvironmentConsumer Nutrition Environment

Price, promotion, placement

SocioSocio--demographicsdemographics

PsychosocialPsychosocialFactorsFactors

Perceived Perceived NutritionNutrition

EnvironmentsEnvironments

EatingEatingPatternsPatterns

Community & ConsumerEnvironments

Community nutrition environments•Type & location of food outlets•Accessibility (e.g., hours, drive

Consumer nutrition environments = •Availability of healthful food choices•Pricing, promotion, placement•Information availability

Glanz, Sallis, Saelens & Frank, Am J Health

Consumer Nutrition Environments

nutrition environments = Type & location of food outletsAccessibility (e.g., hours, drive-thru)

nutrition environments = Availability of healthful food choicesPricing, promotion, placement

Am J Health Promot 2005

Nutrition/Food vs PA Environments

Food is a commodityFood products are big businessFood is highly regulated

(safety, taxation, hygiene)Complex (nutrients, foods)Organizational environments

play a large roleMore recent development

than PA environment measures

Nutrition/Food vs PA Environments

Food products are big businessFood is highly regulated

(safety, taxation, hygiene)Complex (nutrients, foods)Organizational environments

More recent development than PA environment

Food Environments & Policies: How do they go together?

Policies can shape environments school food policies, catering policies,price supports, food assistance policiesBUTEnvironments often evolve in the absence of specific policiesANDPolicies can be health-promoting or not

Food Environments & Policies: How do they go together?

Policies can shape environments àschool food policies, catering policies,price supports, food assistance policies

Environments often evolve in the absence of

promoting or not

Reasons for Measuring Food Reasons for Measuring Food & Nutrition Environments & Nutrition Environments

ObservationExplanationEvaluationAdvocacy/ActionSurveillance

Reasons for Measuring Food Reasons for Measuring Food & Nutrition Environments & Nutrition Environments

Related to ObesityRelated to Obesity

Advocacy/Action

Types of Measures & Data Sources

Self-report (surveys)

Observation/Audit

Archival, Existing DatabasesOften developed for purposes other than health research

GIS-based Measures/SpatialMost-used indicators: density, proximity

Combinations

Types of Measures & Data Sources

Archival, Existing DatabasesOften developed for purposes other than health research

based Measures/Spatialused indicators: density, proximity

Ideally, measurement involves…

Data source (s)ØExisting data ORØNew data collection

Definition of boundaries &/or samplingSound psychometric propertiesMetrics and plans for analysis“Packaging,” disseminationAdaptability

, measurement involves…

New data collection OR bothDefinition of boundaries &/or samplingSound psychometric propertiesMetrics and plans for analysis“Packaging,” dissemination

Ø Schools

Ø Worksites

Ø Food stores

Ø Restaurants

Ø Communities (local, state)

Examples of Food Environments & Policy Measures in Settings

Communities (local, state)

Examples of Food Environments & Policy Measures in Settings

Organizational Nutrition Environments

Home

Work

Organizational Nutrition Environments

School

Other

School Food Environments & PoliciesSchool Health Policies & Programs Study (SHPPS) [CDC]ØSince 1994; all statesØElementary/middle/senior high schoolsØEnvironment issues assessed: vending machines; offerings of fruit,

vegetables, french fries, high-fat baked goods

School Nutrition-EnvironmentState Policy Classification SystemØSystem to compare state policiesØOrdinal scoring system, 11 policy areasØModerate to high inter-rater agreement

Food BEAMS (Samuels & Associates)ØSystem to assess competitive foodsØLinks food item ID to nutrient databaseØHigh inter-rater reliability

School Food Environments & PoliciesSchool Health Policies & Programs Study (SHPPS) [CDC]

Elementary/middle/senior high schoolsvending machines; offerings of fruit,fat baked goods

State Policy Classification System

rater agreement

Links food item ID to nutrient database

Worksite Food Environments & Policies

Mainly used in intervention studies

Example: Working Well Trial (Biener, Glanz et al. 1998)ØMulticomponent assessmentØAccess to healthy food, nutrition informationØFound to be associated with intervention + self

Example: Section of CHEW (Australia)(Oldenburg, Sallis, et al. 2002; dev 1995)ØAudit tool, included nutrition information,

canteen (cafeteria), vending machinesØUsed NHF ‘tick’ to indicate healthful choicesØHigh reliability (0.8-1.0)

Worksite Food Environments & Policies

Mainly used in intervention studies

Working Well Trial (Biener, Glanz et al. 1998)

Access to healthy food, nutrition informationFound to be associated with intervention + self-report

Section of CHEW (Australia)(Oldenburg, Sallis, et al. 2002; dev 1995)Audit tool, included nutrition information,canteen (cafeteria), vending machinesUsed NHF ‘tick’ to indicate healthful choices

Community Nutrition Environments

•Type & Location of FoodOutlets (stores, restaurants)

•Accessibility Operation, drive

Community Nutrition Environments

Type & Location of FoodOutlets (stores, restaurants)

Accessibility – hours of Operation, drive-thru)

Community Food Environments: Objective Data Sources

Or, ‘where are places to get food’?

Public SourcesGovernment: Food licenses (retail & food service)Other: Yellow Pages, Online directories, etc.

Commercial SourcesDun & Bradstreet, InfoUSA, others

Issues: Completeness? Up-to-Wang et al. [IJBNPA, 2007] –• compared sources of historical data on food stores• State Board vs business directories: 127 vs 351 food stores• State Board had 36 added stores, directories showed 260 more

Community Food Environments: Objective Data Sources

Or, ‘where are places to get food’?

Food licenses (retail & food service)Yellow Pages, Online directories, etc.

Dun & Bradstreet, InfoUSA, others

-date? Accuracy? High turnover

compared sources of historical data on food storesState Board vs business directories: 127 vs 351 food storesState Board had 36 added stores, directories showed 260 more

Consumer Nutrition Environments

•Available healthy options•Price, promotion, placement

•Nutrition Information

Consumer Nutrition Environments

Available healthy optionsPrice, promotion, placement

Nutrition Information

Early Observational Measures of Food Store Environments

Sallis, Nader et al (1986, Pub Health Reports•San Diego Food Availability Survey•Supermarkets, groceries, convenience stores•Inventory of 71 ‘heart-healthy’ foods•78-99% interobserver agreement•Documented more HH foods in supermarkets (m=56.7)than neighborhood groceries (m=25.7) and conveniencestores (m = 12.2)

Cheadle et al. (1989, 1990, 1991, 1993)•Evaluation of Kaiser Family Fdn healthy communities•Focused on small # of items (e.g. skim milk); comparedhealthy to less healthy

•Examined shelf space

Measures of Food Store Environments

Pub Health Reports)

Supermarkets, groceries, convenience storeshealthy’ foods

Documented more HH foods in supermarkets (m=56.7)than neighborhood groceries (m=25.7) and convenience

Cheadle et al. (1989, 1990, 1991, 1993)Evaluation of Kaiser Family Fdn healthy communitiesFocused on small # of items (e.g. skim milk); compared

•Availability (of healthful choices)•Prices (compare healthy vs. less healthy; fastfood vs. sit-down)

•Promotion, Information•Facilitators & Barriers•Kids’ Menus

NEMS-R: Measures of Nutrition Environments in Restaurants

Availability (of healthful choices)Prices (compare healthy vs. less healthy; fast-

R: Measures of Nutrition Environments in Restaurants

Nutrition Environment Nutrition Environment Measures Survey:

A Case Study in Dissemination of

Measures

NEMS tools are research tested, valid and reliable instruments:• NEMS-S: Stores, NEMS• NEMS-R: Restaurants• NEMS-V: Vending machines

NEMS…* was originally developed for research* can be (and has been) used for community

assessment, advocacy, and intervention

NEMS tools are research tested, valid and reliable instruments:

S: Stores, NEMS-CS: Corner StoresR: RestaurantsV: Vending machines

* was originally developed for research* can be (and has been) used for community

assessment, advocacy, and intervention

Measures of the Nutrition Environment in Stores: NEMS

NEMS-S Measures:

• Availabilityo Of healthful choices

• Priceo Healthy vs. less healthy o By store type/neighborhood

• Qualityo For fresh produce

Measures of the Nutrition Environment in Stores: NEMS-S

S Measures:

AvailabilityOf healthful choices

PriceHealthy vs. less healthy By store type/neighborhood

QualityFor fresh produce

Measures of the Nutrition Environmentin Stores

Core Categories of FoodsMilk Ground BeefFruits Hot DogsVegetables Frozen DinnersSoda Fruit Juice

Measures of the Nutrition Environmentin Stores

Core Categories of FoodsGround Beef Baked Goods

BreadFrozen Dinners Chips

Cereal

Dissemination of NEMS: Tools & Methods

• Demand started during development• Initial funding support from RWJF• 2-3 day trainings & train-the• CD-Rom and online tools post• Assumed from the start that tools should be

customizable• Partnerships to stretch funds

departments, universities, CDC, etc• CDC State nutritionist group & webinars

Dissemination of NEMS: Tools & Methods

Demand started during developmentInitial funding support from RWJF

the-trainerRom and online tools post-training

Assumed from the start that tools should be

Partnerships to stretch funds - state health departments, universities, CDC, etcCDC State nutritionist group & webinars

Customization• The NEMS-S tool was designed to be easily

customizable to suit a projects’ needs• All measures are available in Word or Teleform

formats• NEMS adaptations have included WIC stores,

Latino versions, Texas adaptation, Brazil, ….• Other innovations: NEMS

for PDA and Tablet-PC

CustomizationS tool was designed to be easily

customizable to suit a projects’ needsAll measures are available in Word or Teleform

NEMS adaptations have included WIC stores, Latino versions, Texas adaptation, Brazil, ….

: NEMS-V, NEMS-CS, NEMS

Dissemination of NEMS: In Person Trainings

40 states in the U.S. with over 200 people trained, 3 from Canada, 3 from Singapore, and participants from Netherlands, Japan, and China

Dissemination of NEMS: In Person Trainings

40 states in the U.S. with over 200 people trained, 3 from Canada, 3 from Singapore, and participants from Netherlands, Japan, and China

NEMS Online Training

• NEMS-S and NEMS-R modules included• Initially launched in January 2010• Free to use• Won a 2010 bronze award from the United

States Distance Learning Association (USDLA) Best Practices in Distance Learning Programming

NEMS Online Training

R modules includedInitially launched in January 2010

Won a 2010 bronze award from the United States Distance Learning Association (USDLA) Best Practices in Distance Learning Programming

* As of July 2010

Dissemination of NEMS: Online Training

As of March 2011: 410 registrants, 131 completed the course; 31 outside the USA

Dissemination of NEMS: Online Training

: 410 registrants, 131 completed the course; 31 outside the USA

Assessing Food Environmentsand Policies in Large and Population-Based Programs

Assessing Food Environmentsand Policies in Large and

Based Programs

CDC's Common Community Measures for Obesity Prevention Project

Strategies and associated measures that local governments can use to plan & monitor environmentaland policy-level changes for obesity preventi

Examples--food related measuresüPolicy to apply DGA in government

facilities & large school districtü# of full service groceries

per 10,000 in 3 largest underservedcensus tracts in jurisdiction

üGovernment offers incentives to retailers to offer healthy food/beverages

CDC's Common Community Measures for Obesity Prevention Project (Khan et al., MMWR, 2009)

Strategies and associated measures that communities &can use to plan & monitor environmental

level changes for obesity prevention

food related measuresPolicy to apply DGA in governmentfacilities & large school district

per 10,000 in 3 largest underserved

Government offers incentives to retailers to offer healthy food/beverages

Logic Model Approach(Cheadle, Samuels et al., AJPH 2010)

üDescribe steps to change environs & individual health impacts

ü Indicators for each step, assessing those most directly exposed

üAttribute outcomes to interventions if they are:v preceded by meaningful

change in short-term and intermediate outcomes

v in a plausible temporal sequence

Logic Model Approach(Cheadle, Samuels et al., AJPH 2010)

Describe steps to change environs & individual health impacts

Indicators for each step, assessing those most directly exposed

Attribute outcomes to interventions if they are:

in a plausible temporal sequence

Philadelphia CPPW Enhanced Evaluation

Healthy Corner Stores (n=200)Pre-post assessmentsLow vs high intensity

EnvironmentNEMS-CS mobile

IndividualsIntercept interviewsCalories purchased

Philadelphia CPPW Enhanced Evaluation

School Wellness PoliciesPre-post assessmentsn=100 K-8 schools

EnvironmentCompetitive food environsDocumentation/validation

IndividualsBMI measures (standard)

Phila DPH, Food TrustU Penn, Temple U School District Phila

McKinnon, Reedy, Handy & Rodgers Eds. Website compilation of measures and articleshttps://riskfactor.cancer.gov/mfe/

Healthy Eating Research website: www.healthyeatingresearch.org

RESOURCES

McKinnon, Reedy, Handy & Rodgers Eds. – 2009 AJPM Supplement Website compilation of measures and articles

Healthy Eating Research website: www.healthyeatingresearch.org

RESOURCES

Research Design

Study designs:§ cross-sectional, correlational§ comparative§ longitudinal/prospective§ experimental/pre-post§ quasi-experimental

Research Design

correlational

longitudinal/prospectivepost

To evaluate environmental strategies, the RCT is not always the “gold standard”

To evaluate environmental strategies, the RCT is not always the “gold standard”

• May not be appropriate/feasible

• Helpful but not essential

• Can have threats to internal/external validity

Planning & Evaluation CycleHow do environment measures function

throughout the cycle?Grocery store food environment vs. individual level surveys

(Cheadle et al.)

CROSS-SECTIONALSignificant associations (1991)

TRACKING CHANGECorrelations weaker over time (1993)

EVALUATION OVER 3 TIME POINTSInconsistent & contradictory results (1995)

Planning & Evaluation CycleHow do environment measures function

throughout the cycle?

Planning

Implementation

Evaluation

Grocery store food environment vs. individual level surveys (Cheadle et al.)

CROSS-SECTIONALSignificant associations (1991)

TRACKING CHANGECorrelations weaker over time (1993)

EVALUATION OVER 3 TIME POINTSInconsistent & contradictory results (1995)

Unanswered research questionsØHow much environmental change is

needed?ØHow long will it take to improve

behavior AND health?ØWho changes after

environment/policy interventions?

Unanswered research questionsenvironmental change is

will it take to improve behavior AND health?

Limitations of measuresfor accelerating progress

Ø Complex nature of food/nutrition environmentsØ Variable rigor in measures developmentØ Practical limits of fieldØ Too little dissemination of measures Ø Lack of common metrics Ø Limited real-time (archival) measuresØ Unknown sensitivity to change

Limitations of measuresfor accelerating progress

Complex nature of food/nutrition environmentsVariable rigor in measures developmentPractical limits of field-based measuresToo little dissemination of measures Lack of common metrics

time (archival) measuresUnknown sensitivity to change

Bridging the Evidence Gap in Obesity Prevention LEAD FrameworkLocate evidenceEvaluate itAssemble itInform Decisions

Bridging the Evidence Gap in Obesity Prevention (IOM, 2010)

Recommendations

ØBuild resource baseØStandards for evidence qualityØSupport generation of evidence

• Core toolkit of valid, feasible, adaptable,

• Common metrics populations

• Measures at multiple levelssettings/organizations, communities, states, regions, national

• Measures tied to calories sources of fat and calories in diets

• Designs that permit reasonable inferences about impact

What We Need to Accelerate Progress

Core toolkit of measures: well-tested, valid, feasible, adaptable, disseminableCommon metrics to compare studies and

Measures at multiple levels: settings/organizations, communities, states, regions,

Measures tied to calories and major sources of fat and calories in diets

that permit reasonable inferences about

What We Need to Accelerate Progress

The Fork in the Road…The Fork in the Road…