Performance Performance MeasuresMeasuresfor Clean Airfor Clean Air

9th Annual Environmental Performance Summit9th Annual Environmental Performance SummitOctober 3-5, 2007October 3-5, 2007Mike Hadrick, US EPAMike Hadrick, US EPA

hadrick.michael@epa.govhadrick.michael@epa.gov

22

Outline of PresentationOutline of Presentation

A Matter of PerspectiveA Matter of Perspective Measure Types and UsesMeasure Types and Uses Analytical FrameworksAnalytical Frameworks Measures for Ground-level OzoneMeasures for Ground-level Ozone ChallengesChallenges Improvement EffortsImprovement Efforts ConclusionsConclusions Recommended ReadingRecommended Reading

33

A Matter of PerspectiveA Matter of Perspective

77

Measure Types and UsesMeasure Types and Uses

88

Measures for Lower ManagementMeasures for Lower Management

OutputOutput measures track activities and measures track activities and monitor progress and accomplishments in monitor progress and accomplishments in the short run.the short run.

Think of driving two days to a vacation spot Think of driving two days to a vacation spot and hitting milestones on the way...and hitting milestones on the way...

99

Short Run Measures for VacationShort Run Measures for Vacation

Are we on schedule or running late? Are we Are we on schedule or running late? Are we where we planned to be, or did we hit some where we planned to be, or did we hit some detours?detours?

What about our final destination? Will we still What about our final destination? Will we still have the time there to do the things we planned?have the time there to do the things we planned?

Based on the above, do we need to adjust Based on the above, do we need to adjust anything to have a nice vacation once we get anything to have a nice vacation once we get there?there?

1010

Measures for Upper ManagementMeasures for Upper Management

OutcomeOutcome measures track bigger picture or long- measures track bigger picture or long-term trends – such as goal achievement. Back to term trends – such as goal achievement. Back to the vacation...you arrived a couple days ago...the vacation...you arrived a couple days ago...

How’s it going? Are we having fun yet? Was this How’s it going? Are we having fun yet? Was this a good idea? Or is it more like “well, we’re not a good idea? Or is it more like “well, we’re not doing this again!”?doing this again!”?

There’s not much you can do about it now other There’s not much you can do about it now other then lessons learned for next year’s vacation.then lessons learned for next year’s vacation.

1111

Simple Logic Model for Any Type of ProgramSimple Logic Model for Any Type of Program

InputsInputs OutputsOutputs IntermediateIntermediateOutcomesOutcomes

EndEndOutcomesOutcomes

Hierarchy of Indicators – Specific to Environmental ProgramsHierarchy of Indicators – Specific to Environmental Programs

Actions by Actions by RegulatorsRegulators

Responses Responses of Regulated of Regulated Community Community or Societyor Society

Changes in Changes in Pressure or Pressure or

Stressor Stressor QuantitiesQuantities

Changes in Changes in Ambient Ambient

ConditionsConditions

Changes in Changes in Exposure, Exposure,

Body Body Burden, or Burden, or

UptakeUptake

Changes in Changes in Human Human

Health or Health or Ecological Ecological ConditionCondition

Analytical FrameworksAnalytical Frameworks

1212

Measures forMeasures forGround-level OzoneGround-level Ozone

1313

Program Purpose andProgram Purpose andRole of Air OfficeRole of Air Office

Protect human health and the environment Protect human health and the environment from the potential effects of air pollution from the potential effects of air pollution associated with ozone.associated with ozone.

Set health-based standards that protect with Set health-based standards that protect with human health and the environment and human health and the environment and assists states in implementing the standards. assists states in implementing the standards.

1414

Simple Logic Model for OzoneSimple Logic Model for OzoneEPA InputsEPA Inputs EPA OutputsEPA Outputs IntermediateIntermediate

OutcomesOutcomesEndEnd

OutcomesOutcomes

$$$$$$

FTEFTE

Establish air Establish air quality standardsquality standards

Develop technical Develop technical info, methods, & info, methods, & guidance to guidance to implementimplement

Conduct Conduct oversightoversight

Implement Implement federally if States federally if States do notdo not

Reduced Reduced emissions of air emissions of air pollutantspollutants

Lower ambient Lower ambient concentrationsconcentrations

Fewer days with Fewer days with unhealthy airunhealthy air

Reduced Reduced atmospheric atmospheric depositiondeposition

Reduced acidity of Reduced acidity of lakes and streamslakes and streams

Reduction in Reduction in prevalence and prevalence and incidence of air incidence of air pollution-related pollution-related health effectshealth effects

Improved Improved ecosystem ecosystem conditioncondition

1515

Hierarchy of Indicators for OzoneHierarchy of Indicators for Ozone

Actions by Actions by RegulatorsRegulators

ResponseResponseof Regulated of Regulated Community Community or Societyor Society

Changes inChanges inPressure or Pressure or

StressorStressorQuantitiesQuantities

Changes inChanges inAmbient Ambient

ConditionsConditions

Changes inChanges inExposure, Exposure,

BodyBodyBurden, or Burden, or

UptakeUptake

Changes inChanges inHuman Human

Health or Health or Ecological Ecological ConditionCondition

Establish air Establish air quality quality standardsstandards

Develop Develop technical info, technical info, methods, methods, guidance to guidance to implementimplement

Conduct Conduct oversightoversight

Implement Implement federally if federally if States do notStates do not

Implement Implement programsprograms

ComplianceCompliance

Take Take voluntary voluntary actionsactions

Implement Implement alternativesalternatives

Air pollutant Air pollutant or precursor or precursor emissions emissions from various from various source sector source sector categoriescategories

Air pollutantAir pollutantconcentrationsconcentrations

Deposition of Deposition of air pollutantsair pollutants

Lake and Lake and stream aciditystream acidity

Prevalence Prevalence and incidence and incidence of air of air pollution-pollution-related health related health effectseffects

Ecosystem Ecosystem conditioncondition

1616

Short Run MeasuresShort Run Measures

Timeliness of rulemakings, etc. (Deputy)Timeliness of rulemakings, etc. (Deputy)

Status of State Implementation Plan development by Status of State Implementation Plan development by states or review/processing by Regional Office (program states or review/processing by Regional Office (program office)office)

% of permits reviewed or issued within target timeframe % of permits reviewed or issued within target timeframe (PART)(PART)

NONOXX reductions from electric utilities (program office) reductions from electric utilities (program office)

1717

1818

Longer-term IndicatorsLonger-term Indicators

Mobile source emissions of ozone precursors (PART)Mobile source emissions of ozone precursors (PART)

Emissions of ozone precursors from all source Emissions of ozone precursors from all source categories (ROE)categories (ROE)

% reduction in population-weighted ambient % reduction in population-weighted ambient concentration of ozone in monitored counties (PART)concentration of ozone in monitored counties (PART)

Ambient ozone concentrations (ROE)Ambient ozone concentrations (ROE)

1919

2020

2121

2222

2323

2424

2525

End Outcome IndicatorsEnd Outcome Indicators

Asthma prevalence and mortality (ROE)Asthma prevalence and mortality (ROE)

Childhood asthma prevalence and mortality Childhood asthma prevalence and mortality (ROE)(ROE)

Chronic obstructive pulmonary disease (COPD) Chronic obstructive pulmonary disease (COPD) mortality (ROE)mortality (ROE)

Cardiovascular disease (CVD) mortality (ROE)Cardiovascular disease (CVD) mortality (ROE)

2626

ChallengesChallenges

Requirements of statutes, regulations, and court Requirements of statutes, regulations, and court ordersorders

Data not available in time for official reporting Data not available in time for official reporting cyclescycles

Time between activity or expenditure and Time between activity or expenditure and outcomeoutcome

2727

ChallengesChallenges

Who’s performance does measure actually Who’s performance does measure actually capture?capture?

Linking activities to outcomes (causality)Linking activities to outcomes (causality)

Measures overloadMeasures overload

2828

Improvement EffortsImprovement Efforts

2004 NRC Recommendations: 2004 NRC Recommendations: Air Quality Air Quality Management in the United StatesManagement in the United States

EPA Air and R&D offices working together on EPA Air and R&D offices working together on Air Air Accountability Framework Accountability Framework per NRCper NRC

EPA EPA Indicators InitiativeIndicators Initiative

Air office’s Air office’s Measures Improvement EffortMeasures Improvement Effort

2929

ConclusionConclusion

Be clear on the program’s logicBe clear on the program’s logic

Develop measures for each phase of programDevelop measures for each phase of program

Select measures for program evaluation or Select measures for program evaluation or reporting based on intended audience and usereporting based on intended audience and use

3030

Resolution & ZoomResolution & Zoom

- - - - - - - - - - - - - - - - - - - - - -

3131

Recommended ReadingRecommended Reading

Performance-Based Management HandbookPerformance-Based Management HandbookDOE – Oak Ridge Institute for Science and EducationDOE – Oak Ridge Institute for Science and Education

www.orau.gov/pbm/pbmhandbook/pbmhandbook.htmlwww.orau.gov/pbm/pbmhandbook/pbmhandbook.html

Performance Measurement Challenges and Performance Measurement Challenges and StrategiesStrategiesU.S. Office of Management and BudgetU.S. Office of Management and Budget

www.whitehouse.gov/omb/part/challenges_strategies.pdfwww.whitehouse.gov/omb/part/challenges_strategies.pdf

3232

Additional Air InfoAdditional Air Info

www.epa.gov/oar/airtrendswww.epa.gov/oar/airtrends

3333

EPA Report on the EnvironmentEPA Report on the Environment

Describes national environmental conditions and Describes national environmental conditions and trends using existing data and indicatorstrends using existing data and indicators

IDs data gaps and research needsIDs data gaps and research needs

Discusses challenges in filling those gapsDiscusses challenges in filling those gaps

www.epa.gov/roewww.epa.gov/roe

3434