National Evaluation Enforcing Underage Drinking Laws Program EUDL National Leadership Conference Dallas, Texas September 19, 2002 Funding provided by the

National Evaluation National Evaluation Enforcing Underage Drinking Laws ProgramEnforcing Underage Drinking Laws Program

EUDL National Leadership ConferenceEUDL National Leadership ConferenceDallas, TexasDallas, Texas

September 19, 2002September 19, 2002

Funding provided by the Office of Juvenile Justice and Delinquency Prevention

Acknowledgements

All 50 state EUDL coordinators

All Key Actor Survey participants

All Law Enforcement Agency Survey respondents

OJJDP, PIRE, and USC

Evaluation DesignEvaluation Design

Process Evaluation

Key Actor Survey

Case Studies

Local Project Director Survey

50 States 50 States + D.C. 50 States + D.C.

1999 2000 2001

6 States 6 States

Impact Evaluation

Law Enforcement Agency and Youth Surveys

52 Intervention 52 Comparison





First Round of Discretionary Grants

Second Round of Discretionary Grants

3 States (BP)

216 Projects

Evaluation Design (cont.)Evaluation Design (cont.)

Process Evaluation

Key Actor Survey

Local Project Director Survey

50 States + D.C. 50 States + D.C. 50 States + D.C.

2002 2003 2004

Impact Evaluation

Law Enforcement Agency and Youth Surveys


Second Round of Discretionary Grants

Fourth Round of Discretionary Grants

36 Projects




Themes

Galvanized Activity and New Relationships

Increased Enforcement

Youth Behavior – Any Change?

Best and Most Promising Practices

State Government Agencies “Very Involved” in EUDL

3431

2624 23

0

10

20

30

40

50

Highway Safety State Police Substance AbusePrev. &

Treatment

ABC Juvenile Justice

Year 3

N = 49

State Government Agencies “Very Involved” in EUDL (cont.)

23

13 1311 10

0

10

20

30

40

50

Governor'sOffice

AttorneyGeneral

Public Health HumanServices

Education

Year 3

N = 49

Local Organizations “Very Involved” in EUDL

33

1713

7 6

0

10

20

30

40

50

LawEnforcement

YouthOrganizations

SubstanceAbuse Prev. &

Treatment

High Schools Local HealthDepts.

Year 3

N = 49

Other Groups “Very Involved”in EUDL

24

108 8

0

10

20

30

40

50

Citizens' Groups Judges/Judicial Org Alcohol Merchants State Legislators

Year 3

N = 49

Types of Interventions(No. of States “Very Involved”)

42%

55%

38%40%40%

82%

64%

48%

80%

52%

86%

28%

38%

88%

48%

70%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Coordination Public Education Enforcement Policy Change

Prior to EUDL

Year 1

Year 2

Year 3

N = 40-50

Additional Interventions Additional Interventions (No. of States “Very Involved”)(No. of States “Very Involved”)

23

11

21

27 27

12

0

10

20

30

40

50

Youth Involvement Targeting Colleges Programs for YouthOffenders

Nu

mb

er o

f S

tate

s

2000

2001

(N= 50)(N= 50)

Interventions Funded with Local EUDL Grants

2001 Local Project Director Survey

73%

50%

64%74%

76%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Enforcement YouthInvolvement

Education CoalitionBuilding

Policy Change

(N = 211)

Local Agency Focus on Underage DrinkingBefore and After Receiving EUDL Funding

41.7%36.0%

79.8%75.2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Block Grant Discretionary Grant

Per

cent

age

(%)

Involvement prior toEUDL

Involvement sinceEUDL

(N = 72 – 125)

Perceived Obstacles

36

26

19

12 11 10

30

36

12

20

14 14

27

32

1

16

22

10

0

10

20

30

40

50

InsufficientResources

Not Problem forPublic

Lack ofConsensus

Not Problem forState Decision

Makers

IndustryOppostion

Not Problem forLaw Enforcment

Year 1

Year 2

Year 3

N = 50

Themes





Enforcement Activities Enforcement Activities (No. of States “Very Involved”)(No. of States “Very Involved”)

Compliance Checks - 35 States

Cops in Shops – 22 States

Shoulder Tap program – 16 States

N = 49-50N = 49-50

Target of Enforcement Intervention (No. of States “Very Involved”)

41

34

2932

22

31

36

30

21

0

10

20

30

40

50

Merchants Underage Adult/Social

Year 1

Year 2

Year 3

N = 49-50

Impact Evaluation: Analysis Design

1999 Data Collection

Year 1 Year 2 Year 3





Round 1 Communities (104)

Round 2 Communities (38)


Median Number of Compliance ChecksMedian Number of Compliance Checks

20.020.0

24.1

14.0

0

5

10

15

20

25

30

35

40

45

50

Num

eric

al V

alue

Comparison

Intervention

P-value = 0.030

N (Comparison) = 76

N (Intervention) = 105

Year 1 Year 2

Median Number of “Cops in Shops” Median Number of “Cops in Shops” OperationsOperations

6.0

18.0

25.0

10.0

0

5

10

15

20

25

30

35

40

45

50

Comparison

Intervention

P-value = 0.005

N (Comparison) = 39


Year 1 Year 2

Median Number of Arrests for Median Number of Arrests for Purchase, Possession, or Use of Purchase, Possession, or Use of

AlcoholAlcohol

45

8 11

20

0

5

10

15

20

25

30

35

40

45

50

Comparison

Intervention

P-value = 0.049

N (Comparison) = 88


Year 1 Year 2

Other Enforcement Findings: No Statistically Significant Changes

Shoulder Tap ProgramMonitoring/patrolling parking lotsTracing alcohol sourceCitations/arrests for false IDCitations/fines/license suspension of

businesses

Percentage of Agencies Reporting Strong Percentage of Agencies Reporting Strong Local Support for Underage Drinking Local Support for Underage Drinking

EffortsEfforts

29%

32%

36%

45%

20%

25%

30%

35%

40%

45%

50%

Comparison

Intervention

P-value = 0.610

Odds Ratio = 0.82

N (Comparison) = 200


Year 1 Year 2

Themes





Alcohol Consumption in the Past Alcohol Consumption in the Past 30 Days30 Days

36%40%36%

41%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Comparison

Intervention

P-value = 0.662

Odds Ratio = 0.94

N (Comparison) = 2410


Year 1 Year 2

Other Youth Behavior Findings: No Statistically Significant Changes

Past 7 day alcohol consumptionBinge drinkingDrinking and drivingAttempts to purchase alcoholPerception of alcohol use among

peersConsequences of alcohol use

Why do Program Evaluations Seldom Detect an Impact on Behaviors?

Delayed, Incomplete, or Failed Program Implementation Flawed Program Theory Short Intervention Period Secular Trends Competing Programs Non-comparable Intervention and Comparison Groups Measurement Error Use of Wrong Measures Inadequate Statistical Power

Adapted from Lipsey and Cordray, 2000

Themes



Youth Behavior - No Solid Evidence of Change


Expert Survey Methods

Literature review identified experts in UAD research

Practitioner experts were identified using snowball sampling techniques

Web based survey N = 40Response rate = 75%

AnalysisEmpirical Evidence

Larger Quantity Smaller Quantity

Hig

hN

ot H

igh

Per

ceiv

ed E

ffec

tive

nes

s Best · ≥50% of experts perceived

practice to be highly effective and to have high quantity of empirical evidence

Promising · ≥50% of experts perceived

practice to be highly effective but did not perceive high quantity of empirical evidence

Ineffective · ≥50% of experts perceived

practice to not be highly effective and to have high quantity of empirical evidence

Questionable · ≥50% of experts perceived

practice to not be highly effective but did not perceive high quantity of empirical evidence

Compliance checks ¨ Conduct regular enforcement actions involving the use of underage

decoys who attempt to purchase alcohol

DWI Enforcement ¨ Enhance enforcement of drinking and driving laws ¨ Conduct sobriety checkpoints

Local Policy

¨ Restrict zoning (outlet locations and density)

State Policy ¨ Increase excise tax ¨ Restrict zoning (outlet locations and density) ¨ Enact .08 blood alcohol content laws for the general population

Findings: Best Practices

Findings: Promising PracticesState Policy

¨ Directly increase prices in "control" states in which prices are set by the state ¨ Enact, or strengthen existing, dram shop liability laws ¨ Restrict hours of sale ¨ Require or encourage the use of driver's license scanners ¨ Enhance driver's license to facilitate recognition of underage purchase attempts

and make license more difficult to falsify ¨ Enact and promote the use of civil penalties ¨ Ban concurrent sales of alcohol and gasoline ¨ Restrict alcohol marketing

School Policy

¨ Enact alcohol policies on college grounds and at college-sponsored events

Findings: Promising Practices

Local Policy ¨ Prohibit entry of persons under 21 into bars/nightclubs and other

"adult" locations ¨ Require or encourage the use of driver's license scanners ¨ Restrict the availability of alcohol at community festivals and other

community events ¨ Restrict alcohol industry sponsorship of public events ¨ Require conditional use permits ¨ Ban concurrent sales of alcohol and gasoline ¨ Restrict alcohol marketing

Findings: Promising Practices

DWI Enforcement Driving under the influence emphasis patrols

Other Enforcement Approaches (including training) Training of law enforcement officers to promote

better enforcement efforts Enforcement and education efforts focused on

parents and landlords who allow underage drinking parties to take place on their property

Evaluation Products

“Bulletin”Year 1, Year 2, and Year 3 ReportsDesign PaperOutcome PaperBest Practices PaperBest Practices Case StudiesPapers on Assorted Topics: LE, Sexual

Assault, Parenting

End of Leadership Conference Presentation

Documents

National Evaluation Enforcing Underage Drinking Laws Program EUDL National Leadership Conference Dallas, Texas September 19, 2002 Funding provided by the