Upload
devin-willis
View
215
Download
2
Tags:
Embed Size (px)
Citation preview
Survey of Tobacco Retail Outlets in 50 United States, DC, and Territories
Lily Trofimovich, Karol KrotkiRTI International
RTI International is a trade name of Research Triangle Institute
2
Outline
I. Introduction 1. Synar Amendment and Regulation, Key Players, GAO Report2. Synar Survey
II. Methodology of the Survey1. Sampling Design2. Sample sizes and precision3. Inspector Characteristics (effect of gender and age on the survey results)
III. Coverage Study1. Introduction2. Guidelines3. State’s coverage studies: Results and Challenges
IV. Results1. RVRs and SE2. Trends
V. Conclusions
3
THE SYNAR AMENDMENT
In July 1992 Congress enacted the Alcohol, Drug Abuse, and Mental Health Administration Reorganization Act, which included the Synar Amendment aimed at decreasing minors’ access to tobacco products
Synar Amendment: Section 1926 of Title XIX of the Federal Public Health Service Act
Sponsored by Mike Synar (1950-1996) – late Representative (D) of Oklahoma
It required states to pass and enforce laws that prohibit the sales of tobacco to individuals 18 years of age or younger
4
THE SYNAR AMENDMENT – cont.
To determine compliance, the Amendment requires each State to:Enact State youth tobacco access control lawEnforce State lawConduct annual random, unannounced inspections
of retail tobacco outletsReport these findings to the Human Health and
Services Secretary
5
THE SYNAR AMENDMENT – cont.
Possible penalty for noncompliance with the Amendment:
Potential loss of a portion of SAPT Block Grant funding.
6
Players
The States
Center for Substance Abuse Prevention (CSAP)/Substance Abuse and Mental Health Services Administration (SAMHSA)
RTI provides technical assistance (statistical consulting) to CSAP:Verifying State’s resultsEvaluating sampling designs and coverage studiesOther
7
THE SYNAR REGULATION
January 1996:
SAMHSA issued the final Synar Regulation to provide guidance to States on implementing the Synar Amendment.
8
General Accounting Office (GAO) Review
GAO examined implementation of the Synar Amendment in 2000 in two key aspects: factors that can affect the quality and comparability of the retailer
violation rates reported by the States whether the States are seeking penalties from retailers as part of
their enforcement strategies
GAO made 3 recommendations to improve Synar implementation and oversight:
1. Help States improve quality and comprehensiveness of tobacco outlet lists
2. Develop a more standardized inspection protocol that promotes more uniform implementation across States and that better reflects research results
3. Ensure that RVRs exclude invalid inspections
9
SAMHSA Actions to Improve Synar implementation and oversight
1. Help States improve quality and comprehensiveness of tobacco outlet lists by:
• Assisting States assess the accuracy and completeness of lists through site visits
• Requiring coverage studies every 2 years for States using list frames
• Develop new guidance document on conducting coverage survey (2005)
10
SAMHSA Actions to Improve Synar implementation and oversight
2. Develop a more standardized inspection protocol that promotes more uniform implementation across States and that better reflects research results by:
• Recommending that all States use youth inspectors aged 15 and 16 and monitoring age and gender breakdowns each year as reported in the Annual Synar Report
• Requiring States use a standard format for reporting inspection protocol activities
11
SAMHSA Actions to Improve Synar implementation and oversight
3. Ensure that RVRs exclude invalid inspections by:
• Instituting review protocols that require States to exclude inspections that do not include age or gender of minor inspectors or outcome of inspection
• Developing a Synar Survey Estimation System (SSES) to assist States estimate and report their annual RVRs
(FYI: In FFY07, 48 States utilized the SSES)
12
Who is Required to Comply with the Synar Regulation?
All 50 States, DC and 8 Territories:
American Samoa
Commonwealth of the Northern Mariana Islands
Federated States of Micronesia
Guam
Marshall Islands
Palau
Puerto Rico
US Virgin Islands
13
Objective of the Synar Survey
To determine the retailer violation rate (RVR) for each State based on random, unannounced inspections of a sample of tobacco outlets accessible to youth.
The sample of outlets inspected must be representative of the geographic distribution of tobacco outlets in the State. The RVR obtained over a number of years will be used to assess the State’s progress towards achieving the overall Synar goal of RVR<=20%
14
Sampling implementation steps
Develop a sampling frame (which includes over-the counter and vending machines and includes at a minimum, 80% of the tobacco outlets in the state) and asses its quality (COVERAGE STUDY)
Determine appropriate sample size for Synar survey to meet SAMHSA’s precision requirement of having an RVR with no more than 3% margin of error with 95% confidence
Select a random sample of outlets that reflects the geographic distribution of outlets accessible to youth
Implement and monitor the Synar Survey within the federal fiscal year (FFY)
Analyze the results of Synar Survey
Report results to SAMHSA before January 1 of the same fiscal year
15
Eligibility Requirements
An over-the-counter or vending machine outlet is eligible for the Synar Survey if it sells tobacco
products and is accessible to youth under age 18. Ineligible tobacco outlets are bars, taverns, or other adult-only clubs that have enforced minimum age
restriction for entry of 18 or older.
16
Sampling Frame Considerations
Type of Frame List Frame (used by the majority of states)
Business List License List
Area Frame (used by two states: MA and IL) List – Assisted Area Frame (Hybrid Frame)
(implemented by two states)
Accuracy/Coverage
17
If SSES is used, Stratified Simple Random Sample with and
without FPC (for census, simple or systematic random sample)
Stratified Cluster Sample with and without FPC
Synar Survey Design Options
18
Sample Size Determination
Three sample size calculations are performed based on previous year’s survey Effective sample size Target sample size Original sample size
19
Effective Sample Size
Effective sample size:
where P is the previous year’s RVR
s.e is the standard error of the estimate for 3% margin of error for one-sided confidence interval (0.0183 for one-sided)
N is the total number of outlets in the sampling frame.
NPP
esne
1
)1(
.).(
12
20
Target Sample Size
Target Sample Size = Effective Sample Size * Design Effect
Where design effect is the ratio of the variance under the chosen design over the variance under the SRS design
The design effect is taken either from the previous year’s survey (SSES calculates the design effect) or hypothesized value is used
21
Original Sample Size
The original sample size is determined by:
where s is a safety margin (10% is recommended), rl is the expected eligibility rate, and rc is the expected completion rate.
cl
to rr
nsn )1(
22
Analysis of the inspection resultsEstimation of three essential items
Retailer violation rate (RVR = number of violations over number of eligible completed inspections)
Variance or standard error of RVR estimate95% confidence interval
Analysis of Survey Results
23
Analysis of the Inspection Results (continued)Estimation of other required items
Coverage rate of the surveyAccuracy rate of the list frame if appropriateCompletion rate
Analysis of Survey Results
24
Coverage Study - Background
States can chooseList frameArea frameHybrid (lists within selected PSUs)
HOWEVER, IF LIST FRAME IS ADOPTED, STATE MUST PROVIDE EVIDENCE FOR QUALITY OF LIST, ESPECIALLY COMPLETENESS, THAT IS, COVERAGE.
25
Characteristics of a List Frame
Coverage - Crucial
Accuracy – Less important
A low coverage rate is a potential source of bias, because the unlisted outlets might be different from outlets on the list with respect to their likelihood of selling tobacco to youth.
26
SAMHSA Requirements
Synar sampling frames must conform to the minimum coverage requirement of 80%.
A frame coverage study should be repeated every 3 years. If a State provides compelling evidence to SAMHSA that the coverage rate is +90% and the list frame used is comprehensive and stable, the State may be allowed to implement the coverage study at 5-year intervals.
27
Coverage Study Guidelines
Area frame - list of easily identifiable and manageable geographically designed areas that represent the full geographic extent of the State, without gaps or overlaps
SAMHSA recommends using census tracts (ZIP Codes and census blocks/block groups could also be used)
Rule of thumb: Area should contain 720 outlets on average
It is important to select a coverage study sample and implement the sample as close as possible to the time when the Synar survey is conducted, because the outlet population keeps changing over time
28
Sample Design for a Coverage Study – Cont.
To select a sample of areas from the area frame, a State must decide on the sample design:Design can range from SRS to a more complex
design (stratified or a multi-stage design) depending on a State’s specific situations.
Stratification always improves the quality of the design.
29
Corrective Actions for Low Coverage
Improve frame to reach the minimum coverage rate prior to drawing the Synar sample
Use additional data source in multi-frame mode
Use area frame for the Synar sample itself
30
State-Level Statistics
FFY 2007 Retailer Violation Rates for 59 Jurisdictions
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
Jurisdictions
RV
R
31
State-Level Statistics
FFY 2007 Standard Errors for 59 Jurisdictions
0.00%
0.50%
1.00%
1.50%
2.00%
2.50%
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61
Jurisdictions
Sta
ndard
Err
ors
32
National Weighted* Average Retailer Violation Rate, FY 97–05
*Weighted by State populationNote: All figures include data from the 50 States and the District of Columbia.
40.1
25.4
20.5 20.0
14.1 12.012.8
16.317.5
0
5
10
15
20
25
30
35
40
45
1997 1998 1999 2000 2001 2002 2003 2004 2005
Fiscal Year
Ret
aile
r V
iola
tion
Rat
e (%
)
33
States are Achieving Very Low Retail Violation Rates
Breakdown of States Achieving the 20% Retailer Violation Rate, FY05
4
17
22
6
2
0
10
20
30
≤ 5% 6-10% 11-15% 16-20% > 20%
Retailer Violation Rate
Nu
mb
er o
f S
tate
s
34
Coverage Rates as of FFY07
<80% - 6 States/Territories
[80-90)% - 9 States/Territories
[90-99)% - 20 States/Territories
99-100% - 10 States/Territories
No results – 14 States/Territories
35
Conclusions/Next Steps
The National Synar Retailer Violation Rate has significantly dropped since FY97; almost all States have achieved the 20% target
Most States are verifying the coverage of the frames
Most States implement efficient designs and analyze data using SSES
Next Steps:
Accuracy and Coverage of Frames should be assessed by all states with list frames
Correct and efficient implementation and reporting of the survey
Acknowledgements
Mel Tremper, Jennifer Wagner, Patty MartinJBS International
RTI International is a trade name of Research Triangle Institute