The Behavioral Risk Factor Surveillance System/media/Files/Activity Files/Disease...The Behavioral Risk Factor Surveillance System ... Reduce alcohol use among those under 21 in

The Behavioral Risk Factor

Surveillance System

29 April 2010

Ali H. Mokdad, Ph.D.

Professor, Global Health

Outline

• Challenges for Surveys

• Behavioral Risk Factor Surveillance System (BRFSS)

• How BRFSS is dealing with the challenges

• Use of BRFSS for a national CVD surveillance system

Outline



• Ways to deal with challenges

• Use of BRFSS for a national Cardiovascular Disease (CVD) surveillance system

Goal: Optimize survey design to decrease total survey error for a given cost

Costs

MeasurementNonresponse

Coverage Sampling

Sampling

• Each element has a known and nonzero probability of selection from sampling frame

o Protection against selection bias

o Quantify sampling error

• Sampling by key modes:

o Face-to-face and telephone: well-developed techniques

o Mail: within-household selection techniques, quasi-random for general pop surveys

o Email: tend to be nonprobability; opt-in samples

Nonresponse

• Inability to obtain data from selected respondent:

o Unit nonresponse

o Item nonresponse

• Nonresponse by key modes:

o Highest in face-to-face

o Lowest in Internet

• Item nonresponse varies by mode and question

Measurement

• Measurement error occurs when a respondent’s answer to a question is inaccurate (departs from the

“true” value)

• Modes vary in terms of:

o Interviewer versus self-administered

o Stimuli / manner in which survey question is conveyed to respondent (and response is recorded)

05

10

15

pe

rce

nt re

spo

nd

ing

yes

Tu

rkey

Vie

tna

mK

aza

khs

tan

Sp

ain

Ge

org

iaB

os.

& H

erz

.P

aki

sta

nS

ri L

anka

Ch

ina

Lao

PD

RH

un

gar

yM

yan

ma

rU

kra

ine

Uru

gua

yS

lov

akia

Ru

ssia

n F

ed.

UA

EM

ala

ysi

aP

hill

ipin

es

Cro

atia

Cz

ech

Re

pu

blic

Gha

na

Est

oni

aM

exi

co

Ma

uriti

us

La

tvia

Par

agu

ay

Slo

ven

iaE

cuad

or

Ma

law

iZ

imb

ab

we

Na

mib

iaB

ang

lad

esh

Mau

rita

nia

Tu

nis

iaG

ua

tem

ala

Za

mbi

aE

thio

pia

So

uth

Afr

ica

Mo

roc

coD

omin

ica

n R

ep.

Ma

liC

om

oro

sB

urk

ina

Fa

soC

ote

d'Iv

oir

eS

wa

zila

nd

Ke

nya

Se

ne

gal

Ch

ad

Bra

zil

Ind

iaC

on

go

Ne

pa

l

A feeling that your thoughts were being directly interfered with or controlled by

another person, or your mind was being

taken over by strange forces?

Coverage

• Sampling frame must include all units of the population of interest

o Frame coverage errors

• Coverage by key modes:

o Face-to-face: expensive to develop (lists help)

o Telephone: cell phones and number portability

o Mail: USPS list not complete, but improving

o Web: no comprehensive list

Declining response rates

• Response rates decreasing significantly in the last 10–15 years.

• Decline has occurred for most types of surveys—particularly telephone and in-person interviews.

• Evidence of trends for mail surveys not as clear due to lack of long-term trend studies.

• Web surveys are too new to provide good trend data.

• Increase in nonresponse is a global problem.

• No single or clear explanation for these trends.

Outline



• How BRFSS is dealing with the challenges


Behavioral Risk Factor Surveillance System (BRFSS)

• Monthly state-based Random Digit Dialing (RDD)

survey of health issues

• Collected by health departments in all states, District of Columbia, Puerto Rico, Guam, and Virgin Islands

• 420,000+ adult interviews conducted in 2008

• From 2002 to 2008:

o Completed over 2,250,000 interviews

o Dialed over 21,000,000 telephone numbers

13

14

Distribution of Household Telephone Status for Adults, January-June 2009

Wireless Wireless

Only: 21.1%Only: 21.1%

Landline with SomeLandline with Some

Wireless: 49.7%Wireless: 49.7%

LandlineLandline

Only: 13.4%Only: 13.4%

Phoneless: 1.5%Phoneless: 1.5% Unknown: 0.4%Unknown: 0.4%

Questionnaire Content: Types and Numbers of Questions

15

CDC’s Role

• Questionnaire design and development

• Technical advice/support

• Data processing — cleaning, weighting, basic

analyses

• Analytic and epidemiologic support

• Data/information dissemination

Users of BRFSS Data

• State health departments

• CDC programs

• Department of Health and Human Services

• Other federal agencies

• Academic researchers and students

• Insurance companies

• Managed care organizations

• Nonprofit health organizations

State Health Departments and CDC

• Track health behaviors

• Identify emerging health issues

• Document health trends

• Evaluate prevention programs

• Measure progress toward health goals

• Develop programs and policies

BRFSS Strengths

• Flexible

• Timely

• Standardized

o Allows state-to-state comparisons

• Useful

• Relatively inexpensive

• Provides infrastructure for expansion and other studies

• True partnership

Flexibility

• Ability to add questions

• Ability to expand

• Ability to follow up

Timely

• States receive mid-year data reports

• States receive preliminary reports within a week of

submitting final year data

• Data files and prevalence tables are posted on the

Web site for public use within six months of the new year

Vaccine Shortage – Timeline

• Oct 5: Vaccine shortage announced

• Oct 5: Initial discussions within CDC

• Oct 19: Call with BRFSS state coordinators

• Oct 19-26: New questions developed and cognitively tested

• Oct 27: CATI specifications to states

• Nov 1: Data collection began

Morbidity and Mortality Weekly Report

• Dec 1-11: States collected December data

• Dec 13: Submitted files to CDC

• Dec 16: Dr. Gerberding holds press conference &

MMWR released on the CDC website

Standardized

• All states use same core questionnaire

• Standardized methodology for collection and analysis

• Allows for state-to-state comparisons

• Now allows for local-to-local area comparisons

Useful

• Identify emerging health problems

• Program development

• Policy development

• Tracking health risk trends

• Program evaluation

Develop Local Programs and Policies: SMART BRFSS in Fargo, ND

• Fargo, ND – 24.9% binge drinking vs. 16.4% nationwide

• Formed community coalition:

AMP (Alcohol Misuse Prevention)

• Mission: Reduce alcohol use among those under 21 in the Fargo-Moorhead area.

o Anti-binge drinking campaign

o Policy change sanctioning facilities

o Intervention with ER doctors

WEAT Startup Screen

WEAT Crosstabs Application Select Age by General Health

Challenges

• Gathers self-reported information only

• Increasing number of questions being added to survey

• Telephone undercoverage

• Increase in cell phone usage

• Competition from telemarketers

• Rising demand for data at the state and local levels

Outline

• Background





Ensuring Validity of BRFSS Estimates

• Monitoring data collection process

• Refining post-survey adjustments

• Benchmarking to other studies

• Testing alternative ways of collecting data

o Cell phone interviewing

o Address-based sampling (ABS)

Monitoring the Data Collection Process

What did we learn?

• Estimates are only as valid as the process in which the data were collected

• Tools for monitoring the quality of data collection and collecting valid data are only good:

o … if they are actually used

o … and if they are understood

o … and initiate follow-up action

Refining post-survey adjustments

Goals and limits of weighting

• Weighting and other post-survey adjustments are used to correct for imbalances in the data due to issues of:

o Coverage

o Sampling

o Nonresponse

• Weighting methodology affects the estimates produced

• Can only weight data you have

o Assumes no difference between respondents and nonrespondents on variables of interest

• Can only weight to external standards that exist

o Typically limits weighting to a handful of demographic variables, not “substantive” variables

Changes in estimate of health status

Percentage with Fair or Poor Health Status

19.0

20.0

21.0

22.0

23.0

24.0

25.0

BRFSS Final

Weight

Add Race Add Education Add Marital

Status

Add Age by

Educ

Add Age by

Race

% A

t R

isk

What did we learn?

• Modifications to post-survey adjustments can improve

the quality of the estimates produced

• Sometimes need to be innovative in the use of external data in developing population estimates

Benchmarking to external standards

Importance and challenges of benchmarking

• True standards rarely exist in health surveys – relative standards

o Better coverage, response

• No two studies are identical

o Populations

o Modes / procedures

o Wording / question order

o Post-survey adjustments / population standards

What did we learn?

• There are no “gold standards” in health statistics

• All comparisons are relative

o Surveys can vary in terms of back-end processing just as much as on front-end design and operational issues

• Determining if BRFSS compares favorably with

other surveys is a matter of perspective

Finding new ways of collecting data:

Cell phonesand

Address-based sampling

Cell phones and telephone surveys

• Reliance on cell phones is increasing

• Conducting surveys via cell phones can be operationally challenging:

o Cell phone sampling frame very inefficient

o Cannot use autodialers

o Charges for incoming calls/minutes used

o Safety concerns

o Potential mode effects / measurement errors

What did we learn?

• The part of the population we are missing due to

cell phones is different from those we interview ---and we cannot ignore them

• Missing critical information needed to integrate

landline and cell phone samples at the subnational level

o No reliable external standards denoting telephone usage at subnational level

Address-based sampling

BRFSS 2006 Address-based sample (ABS)mixed-mode pilot survey

• Six states: CA, FL, MA, MN, TX, and SC

• Address-based sampling frame: USPS Delivery Sequence File (sample provided by Marketing Systems Group)

• Mixed mode data collection:

o Initial mail survey

o Postcard reminder

o Second mail survey (to nonrespondents)

o Telephone survey follow-up (of nonrespondents)

• 75 questions from BRFSS core questionnaire

• Field period: June 20-Oct 4, 2006

• Compared to monthly RDD surveys from same time frame

Within household randomization

• Mail survey (3 approaches tested):

oMost recent birthday (25%)

– One questionnaire

oNext birthday (25%)

– One questionnaire

oAll adults in household (50%)

– Three questionnaires

– Toll-free number for additional questionnaire

• Telephone follow-up survey:

o Used BRFSS protocol – number of males / number of females

Type of household telephone access

1 SJ.Blumberg and JV Luke (2007). “Wireless substitution: Early release of estimates based on data

from the National Health Interview Survey, July – December 2006.” National Center for Health

Statistics E-States.

Household

Telephone

access

National Health

Interview Survey1

(%)

BRFSS ABS

mixed-mode

survey

(%)

Land line 85.0 88.4

-- Landline only --- 14.5

-- Landline and cellular phone --- 73.9

Cellular phone only 12.8 10.5

No telephone 2.2 1.1

Key findings fromABS mixed-mode design

• In states with low response rates, the address-based mixed-mode survey approach can yield response rates superior to RDD rates

• Approach reaches households without landline telephones

o Alternative to sampling from cell phone directories

• Mixed-mode data collection may result in cost savings

o Dependent on complexity of infrastructure put into place to integrate the approaches

What did we learn?

• We can reach cell phone population

• We can reach households with no telephones

Operational considerations

MULTIMODE

• May allow for total survey error for given cost

BUT

• Added complexity may produce mistakes and unexpected

consequences

Assessing Mode Effects – Key Enabler Is “Overlap”

MODE 1MODE 2

Outline

• Background





Surveillance System: Goals

• Surveillance of disease incidence and prevalence, functional health outcomes, measured risks, and public health and clinical response for the different Americas.

• Quantify the contribution of the major NCD risk factors to patterns and trends in disparities across the US

• Demonstrate an innovative model for surveillance that empowers local decision-makers with information in a decentralized health system that could be subsequently implemented on a wider basis

• Create an environment for conducting rigorous implementation research and evaluating the effectiveness of new health intervention programs

57

Basic Design

• Implement at the county level using an integrated multimode surveillance system.

• Each county would collect self-reported data, examination data, vital events, and provider data.

• Record linkage and repeat surveying of the same individuals would maximize the information content of the data collected.

• Surveillance data would be available for researchers in the public domain, with appropriate safeguards for privacy.

58

Surveillance System: Components

• Health and health care survey data

o Mixed-mode (telephone, cell phone, mail, in-person) interview survey

o Physical examination survey

• Administrative data

o Mortality data by cause from the vital registration system

o Health service provider data from hospitals, emergency rooms, and clinics

59

Role of BRFSS

• Provides local data (over 300 counties)

• Mixed-mode experience (use ABS as a starting point)

• Physical examination survey (pilots in 3 states in 2008, 2 states in 2010)

• Administrative data (advantage has been a state-based surveillance)

61

Ali H. Mokdad, [email protected]

Documents

The Behavioral Risk Factor Surveillance System/media/Files/Activity Files/Disease...The Behavioral Risk Factor Surveillance System ... Reduce alcohol use among those under 21 in