United States Office of Policy December 2012 Environmental Protection (1807T) EPA-100-K-12-012 Agency

Measuring the Effectiveness of the Ocean Dumping Management Program Final Report December 15, 2012

ACKNOWLEDGEMENTS This evaluation of the Ocean Dumping Management Program was developed for the U.S. Environmental Protection Agency’s Office of Policy (OP) under Contract EP-W-10-002 between EPA and Industrial Economics, Inc. (IEc) of Cambridge, MA. Eastern Research Group (ERG) of Lexington, MA assisted in the evaluation as a subcontractor to IEc. Andy Schwarz, Tracy Dyke Redmond, and Josh Wolff represented IEc. Lou Nadeau and Melanie Sands represented ERG.

Ryan Gross and Davie Nguyen provided important background information on the Ocean Dumping Management Program. Matt Keene of OP’s Evaluation Support Division provided technical support and advice for the evaluation. Special thanks go to EPA staff at Headquarters and the Regions who shared their perspectives and feedback to inform the evaluation.

This report was developed under the Program Evaluation Competition, sponsored annually by EPA’s Office of Policy. Program Evaluation is one of the performance management tools US EPA uses to assure itself, the public, and other interested stakeholders that US EPA programs are protecting human health and the environment effectively and efficiently. The information obtained through program evaluations can shed light on whether US EPA programs are meeting their goals and objectives, provide the evidence and road map needed to replicate successes, and identify those aspects of US EPA programs needing improvement. To access copies of this or other EPA program evaluations, please go to EPA’s Evaluation Support Division website at http://www.epa.gov/evaluate.

i

TABLE OF CONTENTS GLOSSARY EXECUTIVE SUMMARY

CHAPTER 1 | INTRODUCTION AND BACKGROUND

Introduction 1-1 The Evaluation 1-2

Goal and Scope of the Evaluation 1-2

Evaluation Roadmap 1-3

Evaluation Questions 1-4

CHAPTER 2 | METHODOLOGY

Measures and Sources of Data 2-1 Literature Review 2-4

Pre-Interview Survey of EPA Staff 2-4

Interviews 2-4

Focus Groups 2-4

Analysis of Data 2-5 Overview 2-5

Data Storage 2-5

Interview Data Analysis 2-5

Survey Data Analysis 2-5

Document Review 2-6

Synthesis 2-6

Data Limitations 2-6

CHAPTER 3 | PROGRAM THEORY OF CHANGE

Ocean Dumping Management Program Logic Model 3-1 Findings 3-1 Program Processes and Activities 3-3

Headquarters 3-3

Regions 3-4

Program Assumptions 3-4 Program Information Flows and Feedback Mechanisms 3-5 Summary of Ocean Dumping Management Program in Practice 3-6

Allocation of Program Resources 3-10

Alignment of Partner Goals 3-11

ii

Conclusions 3-14

Recommendations 3-14 CHAPTER 4 | EMERGING TRENDS, OPPORTUNITIES, AND CHALLENGES

Findings 4-1 Historical Trends and Emerging Issues 4-1

Decreases in Ocean Dumping, Increased Focus on Beneficial Reuse 4-1

Advances in Science and Understanding of Contaminants 4-3

Changes in Program Resources 4-6

Staff Turnover 4-9

Trends and Emerging Issues Specific to EPA Regions 4-9

Obstacles and Delays 4-11

Opportunities for Improvement 4-12

Opportunity to Exercise Greater Coordination and Leadership from EPA HQ 4-13

Opportunity to Update Regulations and Guidance 4-14

Opportunity Related to London Protocol Ratification 4-14

Conclusions 4-15

Recommendations 4-16 CHAPTER 5 | RESOURCE ALLOCATION

Findings 5-1 Resources Required vs. Those That Can Be Reallocated 5-1

Opportunities to Improve Alignment of Resources with Intended Program Outcomes 5-1

Opportunities to Improve Alignment of Resources Across Regions 5-2

Opportunities, Risks, and Impacts of Resource Reallocation 5-3

Conclusions 5-3

Recommendations 5-4 CHAPTER 6 | PERFORMANCE MEASUREMENT

Findings 6-2 Current Measures 6-2

Strengths and Weaknesses of Current Measures 6-3

Potential Alternative measures 6-4

Outcome Measures 6-4

Output/Efficiency Measures 6-5

Conclusions 6-5 Recommendations 6-6

iii

APPENDICES:

APPENDIX A: SURVEY QUESTIONS

APPENDIX B: INTERVIEW GUIDES

APPENDIX C: FOCUS GROUP DISCUSSION GUIDE

APPENDIX D: FTE RESOURCES AND CONTRACT DOLLARS APPLIED TO THE

LOGIC MODEL

iv

GLOSSARY Acronym Definition

CWA Clean Water Act

ESD U.S. EPA Evaluation and Support Division

FTE Full Time Equivalent

FY Fiscal Year

HQ Headquarters

MPRSA Marine Protection, Research & Sanctuaries Act

NGO Non-Governmental Organization

ORISE Oak Ridge Institute for Science and Education

OW U.S. EPA Office of Water

OWOW U.S. EPA Office of Wetlands, Oceans, and Watersheds

PCB Polychlorinated Biphenyl

PEC U.S. EPA Program Evaluation Competition

SINKEX U.S. Navy Sinking Exercise

USACE U.S. Army Corps of Engineers

ES-1

EXECUTIVE SUMMARY

The U.S. Environmental Protection Agency (EPA) Marine Pollution Control Branch, within the Office of Water (OW), administers the Ocean Dumping Management Program in coordination with each of EPA’s seven coastal Regions (Regions 1, 2, 3, 4, 6, 9, and 10). The Ocean Dumping Management Program managers applied for and were awarded resources to evaluate the program in order to provide information that will help managers better align resources and activities with intended outcomes. This information should set the stage for future program improvements by increasing awareness among Ocean Dumping Management Program staff about how the program is working and leverage points where the program could change its approach to be more effective. Based on interviews with program staff and stakeholders, a survey of program staff, and reviews of the literature and program documentation, the evaluation team answered four primary evaluation questions and multiple sub-questions. These questions, and the findings, are summarized below:

1. What is the Ocean Dumping Management Program’s theory of change?

• What are the processes and activities that the Ocean Dumping Management Program uses to achieve its outcomes? The EPA HQ Ocean Dumping Management Program has five key processes/functions: 1) establishing environmental criteria/guidance for implementing MPRSA; 2) reviewing, concurring, and issuing ocean dumping permits; 3) designating ocean disposal sites; 4) establishing and managing Site Management and Monitoring Plans; and 5) enforcing MPRSA. To support these roles, the program seeks to undertake several activities, including: 1) developing and updating legislation; 2) working with international organizations and the State Department on international treaties and protocols; 3) developing policy on emerging issues; 4) developing regulations and policies, and ensuring that existing policies stay current; 5) issuing and updating guidance for implementing the program; 6) addressing petitions, letters, and other inquiries; 7) issuing special or general (non-dredge material) permits; 8) assisting Regions in reviewing and (where appropriate) concurring on dredged material permits issued by USACE; 9) assisting Regions in conducting site monitoring and site designation; 10) assisting Regions and U.S. Coast Guard with regulatory enforcement; 11) reviewing regional programs; 12) coordinating across Regions; and 13) conducting strategic planning. Evaluation findings suggest that the program is currently conducting most but not all of these actions.

• What are assumptions that lead from activities/outputs to outcomes? The program theory rests on assumptions that: 1) the regulations are current, 2) there is adequate information to make program decisions, 3) there is clear communication

ES -2

within EPA, 4) there are adequate program resources, and 5) program partners cooperate with EPA. The evaluation finds that these assumptions do not hold true in all cases.

• How are the program’s resources allocated across the outcomes? The program’s FTEs are allocated to activities intended to lead to the following long-term outcomes in descending priority order: environmentally acceptable conditions resulting from disposal (46 percent); coordinated and effective Ocean Dumping Management Program (29 percent); no legal vulnerability (15 percent); and maintaining an international leadership role (10 percent). The program’s contract dollars are deployed in a similar manner across outcomes: Environmentally acceptable conditions resulting from disposal (38 percent); maintaining an international leadership role (31 percent); coordinated and effective Ocean Dumping Management Program (22 percent); and no legal vulnerability (9 percent).

• What are the feedback mechanisms from outputs/outcomes to activity planning? Information flows can serve as a feedback mechanism that helps program staff and managers adapt their activities. Current information flows include: sharing information from EPA Regions; interagency meetings; complaints from program partners; Regional Dredging Team conversations; and public feedback. There are opportunities to improve information flows, e.g., by conducting focused regular meetings between the EPA Regions and HQ.

• At a high level, what are the goals of the non-EPA program partners? How do these goals align with the goals of the Ocean Dumping Management Program? For EPA Regions, the goals are to maintain environmentally acceptable conditions resulting from disposal. For USACE, the goals related to ocean dumping are to maintain navigation and meet regulatory responsibilities for dredged material with a focus on cost effectiveness. Other program partners include the EPA Clean Water Act 404 Program, the Coast Guard, Department of Interior, Environmental Non-Governmental Organizations, U.S. Fish and Wildlife Service, National Oceanic and Atmospheric Administration, State Agencies, the U.S. Navy, and individual ports and harbors. These partners have a wide array of goals, from natural resource protection to dredging, regulating mineral rights, and vessel disposal. As a regulatory program, EPA’s goals do not align with all of its customers’ and partners’ goals, which is to be expected. The one area of misalignment that seems to cause the most challenges for the Ocean Dumping Management Program is the tension in goals between EPA and USACE.

2. How can the program deal with emerging issues, chal lenges , and

opportun it ies for nat ional and regional implementat ion of the program?

• What are the emerging issues, challenges, and opportunities for the Ocean Dumping Management Program? Are these the same for the national program and the Regions? Historical trends and emerging issues common to all the EPA Regions include: 1) decreases in ocean dumping nationwide, and an increased focus on beneficial reuse, 2) advances in science and understanding of contaminants, 3)

ES -3

changes in program resources, and 4) staff turnover. Emerging issues specific to Regions include Gulf of Mexico restoration, major harbor and channel deepening projects in the southeastern US, ensuring proper disposal of debris from the Japanese tsunami on the west coast, ensuring proper disposal of fish waste in Alaska and on the west coast, and protecting sensitive near shore habitats (e.g., reefs) in the Caribbean from effects of dredged material transport.

• How have the environmental issues addressed by the program changed over the past 20 years, and how have these changes been reflected in program design and resource allocation? What changes in ocean dumping-related environmental issues and associated goals can reasonably be expected over the next 10 years? The environmental issues addressed by the program have indeed changed over the last 20 years, particularly since ocean disposal of sewage sludge was prohibited in all EPA Regions in 1992. Since that time, the focus of the program has shifted to primarily regulating ocean disposal of dredging material, and preventing ocean dumping of most other wastes. While we do not have budgetary information about the program that stretches back as far as the 1990s, interviewees report that the program has fewer resources now than it has had historically, and program staff say that the value of the program is less appreciated in recent years than it has been historically. Going forward, the Ocean Dumping Management Program is likely to be asked to address an emerging set of issues, particularly related to emerging contaminants and climate change.

• What guidance and other assistance should be provided to Regions to assist them in dealing with the emerging issues, challenges, and opportunities? Program staff and stakeholders express concern that the regulations and guidance documents have not kept pace with advances in science and therefore need to be updated. Regions have requested a range of guidance to assist them with emerging issues, challenges, and opportunities, as well as advances in science. Regions request guidance and coordination on emerging issues such as contaminants (e.g., pyrethroids and pharmaceuticals) and issues related to climate change. Additional requests include updating the testing guidance (Greenbook); guidance on fish waste, vessel disposal, site management, beneficial reuse and the Coastal Zone Management Act; advanced maintenance dredging/overdepth/sampling; and guidance on the Water Resources Development Act of 1992.

• What innovations or improvements are needed to deal with the emerging issues, challenges, and opportunities? Regions seek a centralized repository of checklists, resource lists, and other resources to improve consistency of program implementation. Staff praise the recent Ocean Dumping Coordinators meeting as an opportunity to foster coordination across the Regions and seek creative means to gather Ocean Dumping coordinators from across the country in focused conversations. Regions seek support from HQ establishing consistent expectations for USACE Districts

ES -4

• Is the program encountering obstacles or delays? If so, what are those obstacles/delays and what can be done to overcome them? The program is encountering obstacles, such as unclear program boundaries and priorities, conflicting goals of some partners, and lack of coordination. These obstacles lead to delays in activities such as updating guidance documents and reviewing regional programs. The program may overcome some of its obstacles by more clearly communicating its value, more clearly articulating its mission and priorities, and fostering greater communication and consistency across Regions.

3. Given s table or reduced program resources, what are the opportunit ies for

improv ing the al ignment between resource deployment and the program’s

intended or des i red outcomes? 1

• Does the current resource allocation reflect the desired allocation across program goals? If not, where should the emphasis be? Yes, the current resource allocation across its activities does reflect the program’s goals and priorities. However, resource alignment is not currently optimized, and interviewees generally agree that the program could do a better job of focusing its efforts, working more efficiently, and being more proactive. Program managers could improve resource alignment by confirming the program’s priorities, articulating the minimum resources needed to meet those priorities at both the HQ and regional level, and requesting support from senior leadership to focus the program’s limited resources on those priorities.

• What resources are required/fixed in the current allocation? What resources can be reallocated and under what conditions? It appears that the program’s current resources are required (and in fact may not be adequate to support the scope of the program’s work). Thus it does not appear that resources can be reallocated away from the program without reducing its scope. However, it may be desirable to reconsider the reallocation of resources across Regions to ensure equity given the variations in scale of work to be accomplished.

• Of the resources that can be reallocated, what risks and/or opportunities does their reallocation pose? Resource reallocation (whether within the program or outside of the program) poses risks of taking time and effort away from program work. Program staff and managers have not developed a shared view of the minimum requirements that HQ and the Regions must fulfill, and the resources needed to complete those tasks. Without such an agreement, it is difficult to pinpoint opportune areas for resource reallocation.

1 Additional sub-questions, which are answered as part of the findings for sub-questions described above, include: Would

resources being reallocated be as effective in other places/roles within the Ocean Dumping Management Program as in

their current place/role? What would be the impacts of resource reallocation on the program’s customers? Would those

impacts lead to feedback risks or opportunities for the program?

ES -5

• Do the ultimate goals/targeted outcomes reflect the program’s mission? Do the Regions share these goals and outcomes? Yes, the program goals are consistent with the program’s mission, and the Regions share these goals.

4. How can the Ocean Dumping Management Program measure i ts success?

• Are the current measures understood throughout the program? Yes, the current measure is understood, although it is not interpreted consistently.

• Are the current metrics valid measures for the program? A performance measure is valid if it accurately represents the condition or phenomenon that it is purporting to represent. The current measure would be valid if the key term “environmentally acceptable conditions” were consistently defined and interpreted.

• Are there other (more valid or reliable) measures? If so, what are the pros/cons of the other measures relative to the current ones? There are supplementary outcome and output measures that could help measure program results. Key criteria for assessing measures are the extent to which the program has control over the measure and the extent to which the measure reflects program goals.

• What can the current or potentially new measures tell us about program success? Supplementary measures could help assess the quality and efficiency of the program’s work.

In light of the findings above, the evaluation team concludes that the Ocean Dumping program faces numerous challenges, including significant budget restructuring, and at times operates in an inconsistent and somewhat reactive mode as new issues arise. We offer the following recommendations.

1. Clarify and communicate the importance of the program, focusing on why this program is essential to protecting ocean ecosystems. We suggest that program managers articulate the importance of the program with regard to managing dredge materials and encouraging beneficial reuse, and state the benefits of the program in terms of environmental benefits (as well as implementing regulations).

2. Seek to foster improved communication and partnership with USACE. We recommend that the Ocean Dumping Management Program work towards more productive communication with USACE by acknowledging differences in mission and organizational constraints, and focusing on areas of shared responsibility.

3. Update program guidance and use the London Protocol ratification process as an opportunity to update the regulations. We believe the program will need to identify approaches to take on the additional responsibilities that will come with ratification. As a part of updating program guidance, program managers at HQ should work to improve communications to Regions about how to implement the regulations, standard procedures, and expectations for program

ES -6

partners (e.g., USACE Districts). HQ should seek to facilitate communication and consistency across Regions.

4. Strengthen information flows to inform program adaptation, building on a suite of performance measures that encompass both outcome and output measures. We particularly recommend the program consider the following measures: volume of contaminated material not allowed to be ocean dumped (and of this amount, percentage that is used for beneficial uses); percentage of monitored sites found to be in compliance; average number of years since ocean disposal sites were monitored; and average number of years since Site Management and Monitoring Plans have been reviewed and updated.

5. Assess resource needs: We suggest that HQ define the minimum requirements that the Regions must fulfill. The program should then research what future program activity levels (e.g., permit review, site designation activities) can be anticipated in each Region, given trends in ocean dumping and port development. The program can then analyze what resources will be required in each Region to meet minimum program requirements given anticipated activity levels. Finally, the program should consider resource reallocation across Regions if necessary to ensure a balance between effort required and resources provided. HQ will also need to assess its own minimum resource requirements to carry out its core activities at the national level, given work on the London Protocol and the need to update the regulations and provide additional guidance to Regions.

1-1

CHAPTER 1 | INTRODUCTION AND BACKGROUND

This chapter introduces the purpose and scope of the Ocean Dumping Management Program evaluation. The chapter goes on to provide background on the program and its origins and describe the evaluation questions and key audiences for the evaluation findings.

INTRODUCTION

Enacted in 1972, the Marine Pollution, Research, and Sanctuaries Act (MPRSA) regulates transportation for the purpose of dumping of all types of materials into ocean waters and prevents or strictly limits the dumping into ocean waters of any material that would adversely affect human health, welfare, or amenities; or the marine environment, ecological systems, or economic potentialities.2 The MPRSA implements the requirements of the London Convention, an international treaty governing ocean dumping.3

Certain materials, such as high-level radioactive waste, medical waste, sewage sludge, and industrial waste, are banned from ocean disposal. The primary source of material that may be dumped in the ocean, with appropriate oversight and permitting, is navigational dredging undertaken to remove sediment from navigation channels and vessel berthing areas. Other items that may be disposed in the ocean include vessels that are no longer in operation, fish wastes, and human remains.

EPA is responsible for designating ocean disposal sites (both for dredged and non-dredged material), establishing and managing Site Management and Monitoring Plans for all ocean disposal sites, and establishing environmental criteria for the evaluation of permit applications. Though the U.S. Army Corps of Engineers (USACE) issues ocean dumping permits for dredged material (subject to EPA review and concurrence), EPA is responsible for issuing ocean dumping permits (special, general, research, or emergency) for all other substances.

EPA’s Marine Pollution Control Branch, within the Office of Water (OW) in Washington, D.C., administers the Ocean Dumping Management Program in 2 Title I of the Marine Protection, Research, and Sanctuaries Act (MPRSA), also referred to as the Ocean Dumping Act,

generally prohibits (1) transportation of material from the United States for the purpose of ocean dumping; (2)

transportation of material from anywhere for the purpose of ocean dumping by U.S. agencies or U.S.-flagged vessels; (3)

dumping of material transported from outside the United States into the U.S. territorial sea. A permit is required to deviate

from these prohibitions.

3 The U.S. has signed but not yet ratified the 1996 London Protocol, which is designed to update and clarify the London

Convention. As a signatory, the United States is obliged not to take any action to defeat the 1996 Protocol’s object and

purpose.

1 -2

coordination with each of EPA’s seven coastal Regions (Regions 1, 2, 3, 4, 6, 9, and 10). The Headquarters (HQ) program addresses national policy and programmatic issues, leads guidance development in coordination with the Regions, issues permits under the MPRSA, addresses cross-regional issues, and represents the program on/in intra-agency, interagency, and international workgroups and forums. The HQ program, in coordination with the regional offices, regularly addresses public and other agency inquiries related to dumping or placement of substances in the ocean and addresses a variety of issues during and following large-scale emergencies (e.g., Hurricane Katrina, Deepwater Horizon Oil Spill). Regional ocean dumping programs generally focus on dredged material disposal—including review and concurrence on USACE-issued permits, site designations, and site management and monitoring activities—as well as vessel, fish waste, burial at sea, and emergency issues.

The Evaluation

An evaluation of the Ocean Dumping Management Program’s past, present, and future is timely. MPRSA was enacted forty years ago and this is the first systematic evaluation of the program outside of OW. Like many programs within the federal government, the Ocean Dumping Management Program is operating within the context of diminishing resources. To better understand the program’s priorities and to improve management of program resources, Ocean Dumping Management Program managers applied for and were awarded evaluation resources through the EPA Program Evaluation Competition (PEC). The PEC is an ongoing Agency effort overseen by the Evaluation and Support Division (ESD), within the Office of Policy, that promotes the effective use and integration of program evaluation throughout the Agency.

This evaluation is a collaborative effort between ESD and Ocean Dumping Management Program staff, with contractor support. The overall evaluation design and objectives are the result of collective discussion and planning by the evaluation team, comprising a lead evaluator from ESD, one primary Ocean Dumping Management Program representative, an Oak Ridge Institute for Science and Education (ORISE) Fellow from OW, and the contracting team of Industrial Economics, Inc. (IEc) and its subcontractor Eastern Research Group, Inc. (ERG). In the remainder of this report, we use the term “the evaluators” or “we” to refer to the contractors who prepared this report under the oversight and guidance of the EPA staff.

Goal and Scope of the Eva luat ion

The goal of the evaluation is to provide information to allow the program to align resources and activities with intended outcomes. This information should set the stage for future program improvements by increasing awareness among Ocean Dumping Management Program staff4 about how the program is working and leverage points where the program could change its approach to be more effective. The evaluation team agreed on the following scoping decisions:

4 In general, we use the term “staff” to indicate both managers and staff. Where we specifically intend to indicate program

managers, we use that term rather than “staff.”

1 -3

• Focus: The evaluation will focus on the national Ocean Dumping Management Program and how it operates as a whole, including the regional components.

• EPA Regions as Partners: The evaluation will consider EPA Regions as partners of the national Ocean Dumping Management Program staff to the extent that they: a) contribute to the development of priorities for the program, and b) conduct activities that contribute to program outputs. Otherwise, the Regions are HQ’s customers in the process.

• Other Federal Agencies: Other Federal agencies (e.g., USACE) involved in implementing requirements under the MPRSA may provide input to the evaluation, as appropriate, but will not be the focus of the evaluation.

Evaluation Roadmap

Exhibit 1-1 provides an overview of the evaluation process and how it will inform program design. Starting at the top of the figure and proceeding clockwise, the evaluation team first established an understanding of the Ocean Dumping Management Program’s mission, goals, and objectives and then revised a logic model that had been provided by the program to reflect this information. Next, the team developed evaluation questions based on the logic model designed to support the goals of the evaluation. The evaluation questions then flowed into the team’s discussion of the discrete measures that would be most appropriate to address each question and the data that would be needed to develop these measures. We also considered how and from what sources the data would be gathered, and how they would be analyzed. The evaluation methodology, described in Chapter 2, documents each of these elements of the planned evaluation process. The team gathered data and analyzed findings in accordance with the methodology and prepared this report to present findings, conclusions, and recommendations for potential adaptations to the program and/or adjustments in how the program defines its work.

EXHIBIT 1-1. STEPS IN THE EVALUATION PROCESS

1 -4

EVALUATION QUESTIONS

The evaluation team, in consultation with representatives from the Ocean Dumping Management Program, identified four primary questions:

1. What is the Ocean Dumping Management Program’s theory of change? The purpose of this question is to better understand the various components of the Ocean Dumping Management Program (inputs, outputs, and outcomes), their relationships, and how they function in relation to the program’s intended goals and outcomes. Answering this question provides an opportunity to learn from key partners, particularly EPA Regions. For example, with regard to ensuring environmentally acceptable conditions resulting from disposal, the evaluation will address how the program expects its regulations, policies, and enforcement activities will help achieve the goal. The report will address where the program is currently focusing efforts and resources and how that focus aligns with the priorities of partners and stakeholders in the program. The findings will inform the remainder of the evaluation by pointing to places in the program where resource adjustment, improved communication, changes in information tracked, or a realignment of program focus may be needed to obtain targeted goals. Program staff may use the findings to communicate key aspects of their program and assumptions about how the program works.

2. How can the program deal with emerging issues, challenges, and opportunities for national and regional implementation of the program? This question serves three purposes: 1) to describe how changes in science, technology, and environmental issues have influenced the evolution of the program’s design and management over time, 2) to better understand issues and relationships that influence the program’s ability to meet its intended goals, and 3) to identify and explore aspects of the program (e.g., existing strengths or positive relationships) that can be leveraged to improve program functioning and resilience.

3. Given stable or reduced program resources, what are the opportunities for improving the alignment between resource deployment and the program’s intended or desired outcomes? This question takes stock of the Ocean Dumping Management Program’s resources and how they are allocated in order to determine whether or not this allocation reflects the program’s goals and priorities. This question delves into which program resources are fixed, which resources may be reallocated, the structure of resource flows and delays, and the implications of changing the resource allocation, should that be necessary to achieve different program outcomes or due to decreases in program funding.

4. How can the program measure its success? The purpose of this question is to understand how the program can assess and track the extent to which it is meeting its goals. The question helps identify current program measures of success, assess whether those measures are valid and/or easily understood by staff and external stakeholders, and suggest new or additional measures that might be beneficial in gauging success.

1 -5

Exhibit 1-2 identifies a series of sub-questions that are designed to inform the primary questions, along with the key audiences and communication mechanisms for the findings for each primary question.

EXHIBIT 1-2. EVALUATION QUESTIONS AND THE KEY AUDIENCE AND COMMUNICATION

MECHANISMS FOR FINDINGS

EVALUATION QUESTIONS AND SUB-QUESTIONS KEY AUDIENCES

COMMUNICATION

MECHANISMS FOR

FINDINGS

1. What is the Ocean Dumping Management Program’s theory of change?

a) What are the processes and activities that the Ocean Dumping Management Program uses to achieve its outcomes?

b) What are assumptions that lead from activities/outputs to outcomes?

c) How are the program’s resources allocated across the outcomes?

d) What are the feedback mechanisms from outputs/outcomes to activity planning?

e) At a high level, what are the goals of the non-EPA program partners? How do these goals align with the goals of the Ocean Dumping Management Program?

Program Manager for Ocean Dumping Management Program Director of the Oceans and Coastal Protection Division Director and Deputy Director and Director of the Office of Wetlands, Oceans and Watersheds (OWOW)

Program logic model and accompanying narrative theory of change. The evaluation team will document the evolution of the logic model over the course of the evaluation.

2. How can the program deal with emerging issues, challenges, and opportunities for national and regional implementation of the program?

a) What are the emerging issues, challenges, and opportunities for the Ocean Dumping Management Program? Are these the same for the national program and the Regions?

b) How have the environmental issues addressed by the program changed over the past 20 years, and how have these changes been reflected in program design and resource allocation? What changes in ocean dumping-related environmental issues and associated goals can reasonably be expected over the next 10 years?

c) What guidance and other assistance should be provided to Regions to assist them in dealing with the emerging issues, challenges, and opportunities?

d) What innovations or improvements are needed to deal with the emerging issues, challenges, and opportunities?

e) Is the program encountering obstacles or delays? If so, what are those obstacles/delays and what can be done to overcome them?

Same audiences as for Evaluation Question 1, and potentially, partner agencies (e.g., Coast Guard and USACE)

Narrative in final evaluation report

1 -6

EVALUATION QUESTIONS AND SUB-QUESTIONS KEY AUDIENCES

COMMUNICATION

MECHANISMS FOR

FINDINGS

3. Given stable or reduced program resources, what are the opportunities for improving the alignment between resource deployment and the program’s intended or desired outcomes?

a) Does the current resource allocation reflect the desired allocation across program goals? If not, where should the emphasis be?

b) What resources are required/fixed in the current allocation? What resources can be reallocated and under what conditions?

c) Of the resources that can be reallocated, what risks and/or opportunities does their reallocation pose?

d) Would resources being reallocated be as effective in other places/roles within the Ocean Dumping Management Program as in their current place/role?

e) What would be the impacts of resource reallocation on the program’s customers? Would those impacts lead to feedback risks or opportunities for the program?

f) Do the ultimate goals/targeted outcomes reflect the program’s mission? Do the Regions share these goals and outcomes?

Same audiences as for Evaluation Question 1

Narrative in the final evaluation report

4. How can the Ocean Dumping Management Program measure its success?

a) Are the current measures understood throughout the program?

b) Are the current metrics valid measures for the program?

c) Are there other (more valid or reliable) measures? If so, what are the pros/cons of the other measures relative to the current ones?

d) What can the current or potentially new measures tell us about program success?

Same audiences as for Evaluation Question 1

Narrative and examples in the final evaluation report

2-1

CHAPTER 2 | METHODOLOGY

This chapter describes how the evaluation team planned to answer the evaluation questions, specifically: 1) the measures used and sources of data; 2) approaches to analyzing data gathered; and 3) expected limitations of the analysis.

MEASURES AND SOURCES OF DATA

There are four key sources of information used in this evaluation:

1. Literature review

2. Survey of program staff

3. Interviews with:

• EPA Ocean Dumping Management Program HQ staff

• EPA staff from each of the seven Regions participating in the program

• Stakeholders, including representatives from EPA’s Clean Water Act Section 404 (inland disposal) Program, USACE, an environmental group that has engaged EPA’s Ocean Dumping Management Program in the past, and a former program manager (now retired).

4. Focus groups with EPA Ocean Dumping Management Program staff and managers.

Exhibit 2-1 shows the sequence of data collection used in the evaluation. These sources of information will inform measures that are used to answer the primary evaluation questions and their sub-questions. Exhibit 2-2 shows the measures that are associated with each evaluation question and the source of those data.

EXHIBIT 2-1. SEQUENCE

OF DATA COLLECTION

Survey of Program Staff

Stakeholder Interviews

Documentation Review

Program Staff Interviews

Regional Ocean Dumping Coordinators

Meeting

Focus Groups

Additional Program Staff

Interviews

2-2

EXHIBIT 2-2. EVALUATION DESIGN MATRIX: MEASURES AND DATA SOURCES

EVALUATION QUESTIONS AND MEASURES [A]

DATA SOURCES

LITERATURE

REVIEW

SURVEY OF

PROGRAM

STAFF

INTERVIEWS WITH PROGRAM STAFF FROM…

FOCUS

GROUPS

SUMMATIVE

FROM

FINDINGS[B] EPA HQ EPA REGIONS STAKEHOLDERS

1. What is the Ocean Dumping Management Program’s theory of change? Summary of the processes and activities that the program uses to achieve its outcomes (1a)

Description of the assumptions that lead from activities/outputs to outcomes (1b)

Quantification of the program’s resources (Full Time Equivalent (FTE) hours and contract dollars) allocated across the outcomes (1c)

Description of the feedback mechanisms from outputs/outcomes to activity planning (1d)

Description of the high-level goals of the non-EPA program partners and how they align with the goals of the Ocean Dumping Management Program (1e)

2. How can the program deal with emerging issues, challenges and opportunities for national and regional implementation of the program? Description of the emerging issues, challenges, and opportunities for the Ocean Dumping Management Program at the national and regional levels (2a)

Description of trends in environmental issues addressed by the program over the past 20 years and related changes in program design and resource allocation over the same period (2b)

Description of guidance/assistance needed to deal with the emerging issues, challenges, and opportunities (2c)

Description of innovations or improvements are needed to respond to emerging issues, challenges, and opportunities (2d)

Description of obstacles or delays encountered by the program and potential responses (2e)

3. Given stable or reduced program resources, what are the opportunities for improving the alignment between resource deployment and the program’s intended or desired outcomes?

Description of current resource allocation and emphasis (3a)

2-3

EVALUATION QUESTIONS AND MEASURES [A]

DATA SOURCES

LITERATURE

REVIEW

SURVEY OF

PROGRAM

STAFF

INTERVIEWS WITH PROGRAM STAFF FROM…

FOCUS

GROUPS

SUMMATIVE

FROM

FINDINGS[B] EPA HQ EPA REGIONS STAKEHOLDERS

Description of desired resource allocation (3a)

Description of required/fixed elements of the resource allocation (3b)

Description of elements that can be reallocated under what conditions (3c)

Risks and opportunities of reallocation (3c)

Description of the effectiveness of reallocating resources as compared to current place (3d)

Impacts of reallocation on the Ocean Dumping Management Program’s customers, e.g., EPA regional staff and potentially other stakeholders

Description of the alignment of targeted goals/outcomes with program mission (3f)

Description of the whether or not targeted goals/outcomes are shared by Regions (3f)

4. Given the answer to question #3, how can the Ocean Dumping Management Program measure its success?

Description of current program measures (4a)

Description of how program understands current program measures (4a) Description of the validity of current program measures (4b) Potential alternative measures and/or suggested revisions to current measures and their pros and cons (4c)

Description of how current or new measures gauge program success (4d) [A] Parenthetical references following the measures link the measure to the sub-questions listed in Exhibit 1-2. [B] Builds on findings from other evaluation questions

2-4

Each of the four data collection methods is described in further detail below.

Literature Rev iew

The evaluation team reviewed documents that are publicly available or made available by program staff in order to: 1) better understand the Ocean Dumping Management Program and its functioning, 2) identify key budgetary, technical, and managerial issues and trends related to ocean dumping, and 3) provide context on relevant historical trends and events.

Pre- Interv iew Survey of EPA Sta ff

Prior to interviews, the evaluation team distributed a brief online survey to 40 Ocean Dumping Management Program staff at HQ and the Regions. The purpose of the survey was to use staff time as efficiently as possible by collecting information for selected questions in a standardized way, and to identify topics to address in greater detail in subsequent interviews. The evaluation team received 13 responses (a 33 percent response rate), representing HQ and six of the seven Regions involved in the program. The survey gathered respondent opinions, perspectives, and feedback regarding emerging issues, challenges, and opportunities for national and regional implementation of the program. The evaluation team developed the survey questions with input from Ocean Dumping Management Program HQ staff. Appendix A contains a copy of the survey questions.

Interv iews

Following review of the survey results, the evaluation team conducted interviews with EPA staff and individuals outside the agency. The interviews occurred in two sets: first, the evaluation team interviewed EPA regional staff and a representative of EPA’s Office of Research and Development, which provides consultation to the Ocean Dumping Management Program. We summarized the findings of these interviews at the Regional Ocean Dumping Coordinators Meeting in August 2012. Following the meeting, the evaluation team conducted a second set of interviews with EPA HQ staff, a representative of EPA’s Clean Water Act (CWA) Section 404 program, and external stakeholders (including USACE, an environmental group, and a former Ocean Dumping Management Program employee). The evaluation team developed tailored interview guides for each set of interviewees in consultation with Ocean Dumping Management Program HQ staff. Exhibit 2-3 summarizes the interviews conducted. Appendix B contains the interview guides used.

Focus Groups

The evaluation team conducted three one-hour focus groups with program staff at the Regional Ocean Dumping Coordinators meeting in Washington, D.C., in August 2012 to add depth and detail to the information learned through the interview and survey. The discussions focused primarily on verifying and refining the theory of change, use of program resources, effectiveness of HQ activities, and engagement with partners. Appendix C includes the discussion guide for the focus groups.

2-5

EXHIBIT 2-3. SUMMARY OF INTERVIEWS CONDUCTED

GROUP INTERVIEWED

NUMBER OF INTERVIEWS

CONDUCTED

EPA INTERVIEWEES

Regional staff 6

Office of Research and Development 1

HQ staff 2

CWA Section 404 staff 1

EXTERNAL INTERVIEWEES

USACE 1

Environmental group (Clean Ocean Action) 1

Former Ocean Dumping Management Program employee 1

ANALYSIS OF DATA

Overv iew

The general analytical approach for the evaluation included a content analysis of the information gathered. In general, the analysis was qualitative, but where possible we included numerical analysis of quantitative data points such as resource and FTE trends.

Data Storage

The evaluation team developed a master set of all notes in Microsoft Word to serve as the foundation of the analysis. This file, along with the survey results, but excepting any information deemed confidential by the interviewees, will be available to EPA upon completion of the evaluation.

Interv iew Data Analys is

Given the manageable scope of data collected through the interviews, the evaluation team did not code interview responses, but rather identified common themes and gathered evidence from all data sources to address each evaluation question. The evaluation team compiled quotes where appropriate to illustrate key points (quotes are not attributed). We also noted where there were areas of consensus, and where there were differences in perspective across different groups of interviewees (e.g., across different groups of Regions, or between HQ and Regions).

Survey Data Ana lys is

The evaluation team compiled and analyzed survey data to look for trends and patterns related to the Ocean Dumping Management Program. The survey results include the percentage of respondents selecting specific responses or identifying particular themes.

2-6

Document Rev iew

The evaluators reviewed documents provided by the Ocean Dumping Management Program and searched for publicly available literature relevant to this evaluation using sources such as the EPA website, grey literature, and peer-reviewed journal articles. The literature review helped us understand the Ocean Dumping Management Program, as well as the historical context for the program, emerging issues, and resource trends.

Synthesis

To develop our conclusions and recommendations, the evaluation team synthesized information across the data sources. We used multiple data sources to answer each evaluation question and to verify apparent trends and patterns. Where different data sources suggested different conclusions, we noted this and, when appropriate, searched for additional data to confirm the analysis.

DATA LIMITATIONS

Throughout this study, the evaluation team sought to collect representative, objective, and robust data. However, as with all program evaluations, data collection and analytical limitations exist, and we make these transparent in our findings. We currently note the following limitations to this evaluation methodology:

• Non-experimental research design: This evaluation is not suited to use of comparison or control groups. However, the evaluation does involve comparisons, particularly comparing information about the program over time (e.g., current program activities and resource allocation, recollections about how the program operated in the past, and expected future opportunities and challenges).

• Limited information drawn from outside of EPA: The evaluation primarily drew on information from within the Ocean Dumping Management Program itself (e.g., program documentation, survey responses, and interview notes). The evaluation team did gather some perspectives from outside EPA (e.g., through a limited number of stakeholder interviews), but the evaluation does not necessarily present a representative summary of viewpoints from stakeholders outside EPA.

• Potential uncertainties in data on program resource allocation data: We analyzed program resource quantities (FTE hours and contract dollars) allocated across program outcomes based on data provided by program staff, including work plans with staff allocations to specific program activities. If these work plans and program staff input did not accurately reflect the activities that FTE hours and contract dollars support, the accuracy of our results would be compromised.

3-1

CHAPTER 3 | PROGRAM THEORY OF CHANGE

In this chapter we consider how the Ocean Dumping Management Program is intended to work, and how it works in practice, based on input from program staff and stakeholders. The section begins with a brief description of the program logic model that we developed in conjunction with program staff at the outset of the evaluation, and then describes our findings about particular facets of the program theory and experience, including:

• Processes and activities that the program uses to achieve its outcomes;

• Assumptions about how the program’s activities and outputs transition to program outputs;

• Program information flows and feedback mechanisms;

• Allocation of program resources; and

• Alignment of non-EPA partner goals.

OCEAN DUMPING MANAGEMENT PROGRAM LOGIC MODEL

The program logic model presented in Exhibit 3-1, which was developed by the Ocean Dumping Management Program and refined by the evaluation team, serves as a starting point in the evaluation. It depicts key elements of the HQ program, including program inputs, activities, outputs, customers, desired outcomes, and external factors that influence results. The evaluation team used the information provided in the logic model to clarify various aspects of the program’s theory of change. The logic model also serves as a template against which the program’s actual experience can be compared, to learn if there are aspects of the program that differ from the program design.

3-2

EXHIBIT 3-1. OCEAN DUMPING MANAGEMENT PROGRAM LOGIC MODEL

Develop and update legislation (i.e. MPRSA)

Develop policy on emerging issues (ie. CCS)

Develop regulations & policies, keep current w/

policy

Work w/ int’l orgs and State Dept

Legislation package (MPRSA Amendments)

Regulations & policies

International technical & training documents

Annual report to IMO on ocean dumping permits

US Congress

Gov’t entities, regions, regulated community,

env. groups, public

LC/LP contracting and non-contracting parties

Congress informed regarding MPRSA and LP

Stakeholders understand role and requirements of

ODP policy

EPA & State Dept. communicate w/ int’l community (LC/LP)

Congress amends MPRSA and ratifies LP

Stakeholders conform to ODP regulations and

policies

Limit ocean disposal of waste

US (EPA) maintains int’l leadership role

Address petitions, letters, and other inquiries

Issue special or general (non-dredged material)

permits

Assist regions to review & concur on dredged

material permits issues

Assist regions & USCG to enforce regulations

Develop guidance documents

Assist regions in site monitoring & designation

issues

Review regional programs

Conduct monthly call & semi-annual meeting for

OD coordinators

Response letters

Ocean dumping permits

Assistance w/ civil penalties

Guidance documents

Assistance with site monitoring data, SMMPs,

site designation

Regional program review reports

Coordinators’ call notes, meeting reports

Strategic PlanningEPA reports & planning

documents

Regulated community, env. groups, public

Ports and port-dependent industries

USACE and Permittees

Regional program offices

Division, office, OW management, public

Stakeholders understand & aware of requirements

for disposal

USACE & EPA aware of monitoring requirements and conditions at sites

Increased regional awareness of policy and

intent of policy

Mgmt aware of program activities and direction

Limit ocean disposal of wastes

No ocean disposal without permit

Only environmentally acceptable ocean disposal

Efficient and effective evaluation of permit

proposals

Guidance improves implementation of OD management program

Consistent program administration across

regions

Mgmt support of program activities and budget

Environmentally acceptable

conditions resulting from disposal

No legal vulnerability

Coordinated & effective OD mgmt

program

Activities & Partners Outputs Customers Short-term Outcomes Medium-term Outcomes Long-term Outcomes(Awareness) (Behavior) (Conditions)

3-3

Findings

PROGRAM PROCESSES AND ACTIVITIES

The program’s goal of ensuring that material placed in the ocean does not negatively impact the environment and/or human health through the implementation of MPRSA has remained constant since the program’s inception in 1972. However, since the program’s beginning, the types of materials being disposed of in the ocean have changed, shifting primarily toward the disposal of dredged material. Over time, this shift in the types of materials placed in the ocean has shaped the activities and emphasis of the Ocean Dumping Management Program. The program serves five primary functions, including:

• Establishing environmental criteria/guidance for implementing MPRSA;

• Reviewing, concurring on, and issuing ocean dumping permits;5

• Designating ocean disposal sites;

• Establishing and managing Site Management and Monitoring Plans; and

• Enforcing MPRSA.

These functions are carried out by EPA HQ and Regions, as described below.

Headquarters

HQ interviewees conveyed that their core program responsibilities mirror those listed in the Activity column of Exhibit 3-1 (e.g., dealing with statutory issues, providing guidance to the Regions), however, funding and near-term priorities have shaped HQ activities. For example, interviewees noted that the reduction in program funding has caused HQ to operate in a “bare bones” fashion; staff have had to focus on addressing issues requiring immediate attention, e.g., decommissioning the program’s research and monitoring vessel, the Bold.6 Interviewees report that some longer-term program initiatives, such as those listed below, are not receiving adequate attention and resources:

• Updating the ocean disposal testing manual,

• Developing an amendments package for MPRSA,

• Dealing with Region-specific issues (e.g., fish waste in Region 10), and

• Conducting certain types of activities related to strategic planning activities, such as regional reviews.

5 EPA is responsible for issuing ocean dumping permits (special, general, or emergency) for all material placed in the ocean

other than dredged material, which is permitted by the USACE and subject to EPA review and concurrence.

6 According to one interviewee, approximately half of the Bold’s activities supported monitoring related to ocean dumping.

In 2010, the Bold’s survey missions included monitoring ocean dredged material disposal sites, coastal eutrophication and

toxicity assessments, monitoring ocean outfalls, assessing critical coral reef habitats, and monitoring hypoxia in the Gulf of

Mexico. Source: EPA’s Ocean Survey Vessel Bold 2010 Annual Report,

http://water.epa.gov/type/oceb/assessmonitor/bold/reports.cfm

3-4

Regions EPA’s seven coastal Regions (EPA Regions 1, 2, 3, 4, 6, 9, and 10) implement the Ocean Dumping Management Program in conjunction with HQ. At the regional level, interviewees conveyed that they primarily focus on site designation, monitoring, compliance assistance, and stakeholder relationships. Although all the Regions work on many aspects of Ocean Dumping—including evaluation of project proposals and scoping documents, and development of and review of sampling/testing data— Regions emphasize different activities depending on the needs and priorities of the Region. For example, regional staff noted that they currently devote significant attention to the following activities:

• Enforcement (Region 2);

• Dredging teams (Regions 1, 6);

• Beneficial use (Regions 6, 9, 10); and

• Disposal of fish waste (Region 10).

PROGRAM ASSUMPTIONS

There are several assumptions that underlie the program’s theory about how activities will lead to intended outcomes.

• Current Regulations: the program is operating with up-to-date regulations that reflect current contaminants in material disposed of through ocean dumping.

• Adequate Information: the program has adequate information to establish program criteria, assess the suitability of material proposed for ocean dumping, and determine whether ocean dumping sites are meeting environmentally acceptable conditions.

• Clear Communication: HQ provides Regions with sufficient information that will enable them to implement the program consistently in accordance with MPRSA, and program managers at the Office of Water understand the purpose and function of the Ocean Dumping Management Program. For example, when delivering program outputs, such as reports, legislative packages, and guidances, the theory rests on the assumption that the HQ program clearly communicates information to their customers and partners.

• Adequate Program Resources: there is sufficient funding and staff time to support all of the activities necessary to achieve program goals.

• Program Partners Cooperate with EPA: program partners will cooperate, e.g., by sharing needed information, and thus help achieve program goals.

The evaluation examined the validity of these assumptions, and found that not all of them hold true. For example, see the section “Summary of Ocean Dumping Management Program in Practice” and Exhibit 3-3 for a discussion of examples where the assumptions are not valid.

3-5

PROGRAM INFORMATION FLOWS AND FEEDBACK MECHANISMS

According to systems theory, “missing feedback is one of the most common causes of system malfunction. Adding or restoring information can be a powerful intervention” that causes people to behave differently.7 Thus information flows can serve as a feedback mechanism that helps program staff and managers adapt their activities. Regional interviewees commented that they currently rely on the following sources of feedback to help gather information about the Ocean Dumping Management program and its functioning:

• Information and insights from Ocean Dumping Coordinators in the Regions;

• Interagency meetings;

• Complaints from program partners;

• Regional Dredging Team conversations; and

• Public feedback.

In addition, interviewees cited monthly conference calls for Ocean Dumping Coordinators as a potentially useful feedback mechanism that currently lacks sufficient focus and specificity to inform action. For example, instead of providing a brief overview of regional updates, focus group participants suggest that the calls could focus on specific topics or questions raised by the Regions.

Interviewees pointed out there is a need for more consistent transfer of information as to what is working and not working in the program. For example, one interviewee said, “I don’t think we’ve really had time to do that. We’ve been in a total crisis mode for years. The adaptation we’ve done is: when a crisis hits, we focus on it until the next crisis.” When survey respondents were asked to rate their level of agreement with the following statement: “I am provided or can access information about program results that helps inform my work,” less than one-third agreed or strongly agreed with the statement (See Exhibit 3-2).

7 Meadows, Donella. Leverage Points: Places to Intervene in a System. The Sustainability Institute, 1999.

3-6

EXHIBIT 3-2. ACCESS TO INFORMATION THAT INFORMS WORK EFFORTS*

*In Exhibit 3-2 and subsequent exhibits showing survey respondent agreement or disagreement

with a statement, agreement is shown in the lighter teal bar, and strong agreement (where it

occurs) is shown in the darker teal bar. Disagreement is shown in the lighter purple bar, and

strong disagreement (where it occurs) is shown with dark purple.

Summary of Ocean Dumping Management Program in Pract ice

We can assess how the program theory is working in practice by considering what HQ and regional staff and stakeholders said about their experience with program implementation. Exhibit 3-3 shows the program logic model, with three additional features identified based on interview notes: 1) leverage points that strongly influence program effectiveness, 2) process delays and need for improved processes, and 3) processes that are not occurring according to interviewees. Each of these elements is described below and illustrated in Exhibit 3-3.

Leverage Points

Systems theorists use the term “leverage points” to indicate “places within a complex system…where a small shift in one thing can produce big changes in everything.”8 We identify three leverage points within the system of agencies and individuals responsible for implementing MPRSA:

• The presence of current regulations seems to be a key leverage point that strongly influences program effectiveness. Lack of current regulations drives a feedback loop, whereby the USACE does not consistently cooperate with EPA and Regions do not have information to consistently implement MPRSA, which undermines the credibility and effectiveness of the program. Updating the regulations could lead to improved understanding and cooperation with USACE

8 Meadows, Donella. Leverage Points: Places to Intervene in a System. The Sustainability Institute, 1999.

0%

38%

31%

23% 8%

0% 5% 10% 15% 20% 25% 30% 35% 40% 45%

Don't Know/Not Sure

Neither Agree nor Disagree

Disagree/Strongly Disagree

Agree/Strongly Agree

"I am provided or can access information about program results that helps inform my work."

3-7

and more consistent and efficient implementation of the program at the regional level. Interviewees mentioned the following factors as impeding updates to the regulations: concern about litigation particularly from environmental NGOs, difficulty gaining agreement from other agencies (USACE and NOAA) about how to change the regulations, and concern that opening the regulations to revision could lead to weakening of the regulations. In order to change the current feedback loop, program managers will need to understand and address the factors leading to the current situation.

• The presence of current guidance is another leverage point that influences program effectiveness. As one interviewee said, “If regulatory reform is not going to occur, then the program needs to get [its] guidance to the best place possible so that people in the Regions can do the best job possible given the circumstances.” Lack of current guidance and training on key topics e.g., testing guidance, fish waste, vessel disposal, and artificial reefs impedes consistent program implementation and administration, which increases legal vulnerability and reduces program efficiency. Interviewees mentioned the following factors as impeding updates to the guidance: lack of updates to the regulations and competing priorities. Updating the guidance would lead to more effective program implementation, and could ultimately reduce resource demands by requiring less HQ staff time to assist Regions on a case-by-case basis. One interviewee said, “Guidance would save EPA and permittees time and money.” Another interviewee said, “Having strong consistent coordination and implementation and good guidance would really help our program would help new folks come on board. It would help us ensure we’re being environmentally protective across the board.”

• Effective coordination across Regions is a third leverage point that influences program effectiveness. This leverage point is shown in the logic model by the activity of conducting a monthly call and semi-annual meeting for ocean dumping coordinators. The program has not held regular in-person meetings in recent years, which has reduced the Regions’ ability to ensure consistent program implementation. Interviewee experience at a recent Ocean Dumping Coordinators meeting in Washington D.C. suggests that more regular focused conversations between Regions and HQ can improve information flows.

Process Delays and Need for Improved Processes

In Exhibit 3-3, yellow diamonds represent junctures where interviewees indicated that a program process or activity is delayed or impeded. For example, HQ staff indicated that their focus on certain near-term activities (e.g., resource reallocation) has shifted their focus away from addressing some programmatic strategic planning issues, so a delay symbol appears after the “Strategic planning” activity box. Interviewees also report delays associated with developing policy on emerging issues, and assisting Regions in site monitoring issues, which influence subsequent elements of the logic model. As discussed in the text on leverage points, there are also delays associated with updating guidance and a need for process improvement related to coordinating across Regions.

3-8

Regional interviewees also commented that process delays can also stem from friction in relationships with program partners. For example, although USACE and EPA may be “aware of monitoring requirements and conditions at sites,” regional staff require cooperation and adequate information from USACE in order to conduct “efficient and effective evaluation of permit proposals.”

Processes not Occur ing

The red stop sign symbols shown in Exhibit 3-3 indicate processes or activities that are currently not being undertaken by the program. For example, interviewees conveyed that regional reviews, conducted by HQ, have not been occurring. While HQ intends to conduct the regional reviews and does allocate resources to this effort in its work plans, other near term priorities have taken precedence. Regional personnel see the reviews as an opportunity to communicate their concerns and feedback to HQ and generate information that could help improve the program at the regional level. Therefore, the absence of these reviews represents a missed opportunity to gather feedback on program operations. In another example discussed above, regulations have not been updated, and the fact that this process has not occurred is strongly influencing work with USACE.

Examples of program processes not occurring, being delayed, or needing improvement point to places in the program theory where the underlying assumptions do not hold true. For example, the need for updated regulations and guidance suggests that the assumptions of current regulations, adequate information, and clear communication do not fully hold true. Instances of friction with USACE reduce the extent to which the program has adequate information and cooperation from program partners. Feedback from program staff raises questions about whether the program has adequate resources.

3-9

EXHIBIT 3-3. OCEAN DUMPING LOGIC MODEL REFLECTING PROGRAM LEVERAGE POINTS CONVEYED DURING INTERVIEWS

Develop and update legislation (i.e. MPRSA)

Develop policy on emerging issues (ie. CCS)

Develop regulations & policies, keep current w/

policy

Work w/ int’l orgs and State Dept

Legislation package (MPRSA Amendments)

Regulations & policies

International technical & training documents

Annual report to IMO on ocean dumping permits

US Congress

Gov’t entities, regions, regulated community,

env. groups, public

LC/LP contracting and non-contracting parties

Congress informed regarding MPRSA and LP

Stakeholders understand role and requirements of

ODP policy

EPA & State Dept. communicate w/ int’l community (LC/LP)

Congress amends MPRSA and ratifies LP

Stakeholders conform to ODP regulations and

policies

Limit ocean disposal of waste

US (EPA) maintains int’l leadership role

Address petitions, letters, and other inquiries

Issue special or general (non-dredged material)

permits

Assist regions to review & concur on dredged

material permits issues

Assist regions & USCG to enforce regulations

Develop guidance documents

Assist regions in site monitoring & designation

issues

Review regional programs

Conduct monthly call & semi-annual meeting for

OD coordinators

Response letters

Ocean dumping permits

Assistance w/ civil penalties

Guidance documents

Assistance with site monitoring data, SMMPs,

site designation

Regional program review reports

Coordinators’ call notes, meeting reports

Strategic PlanningEPA reports & planning

documents

Regulated community, env. groups, public

Ports and port-dependent industries

USACE and Permittees

Regional program offices

Division, office, OW management, public

Stakeholders understand & aware of requirements

for disposal

USACE & EPA aware of monitoring requirements and conditions at sites

Increased regional awareness of policy and

intent of policy

Mgmt aware of program activities and direction

Limit ocean disposal of wastes

No ocean disposal without permit

Only environmentally acceptable ocean disposal

Efficient and effective evaluation of permit

proposals

Guidance improves implementation of OD management program

Consistent program administration across

regions

Mgmt support of program activities and budget

Environmentally acceptable

conditions resulting from disposal

No legal vulnerability

Coordinated & effective OD mgmt

program

Activities & Partners Outputs Customers Short-term Outcomes Medium-term Outcomes Long-term Outcomes(Awareness) (Behavior) (Conditions)

Logic Model Assumptions include: 1) Current Regulations2) Adequate Information3) Clear Communication4) Adequate Program Resources5) Program Partners Cooperate with EPA

Process delays or need for improved process

Process not occurring

Leverage points that strongly influence program effectiveness

3-10

ALLOCATION OF PROGRAM RESOURCES

To assess the current allocation of Ocean Dumping Management Program resources, the evaluation team analyzed how resources in the program’s fiscal year (FY) 2012 work plan support the program’s long-term goals.9 Exhibit 3-4 shows the percentage of Ocean Dumping Management Program FTE hours and ORISE contract dollars10 that are allocated to each activity in the FY 2012 work plan, based on the total number of FTE hours and non-FTE hours (i.e. contract dollars). Based on this analysis, Exhibit 3-4 shows that the largest percentage of the program’s FTE hours are allocated to activities associated with developing policy on emerging issues (19 percent) and strategic planning (16 percent). In terms of ORISE contract dollars, the largest percentage of these resources is allocated to developing and updating legislation (24 percent) and developing policy on emerging issues, e.g., carbon capture and storage (21 percent). However, it is important to note that in practice staff do not conduct all of the activities that are allocated staff resource in the work plan. For example, as noted earlier, regional reviews have not occurred, even though they received an allocation of six percent of the FTE hours. Thus intended allocation may differ somewhat from actual resource allocation.

EXHIBIT 3-4. ALLOCATION OF FTE HOURS AND CONTRACT DOLLARS APPLIED TO OCEAN

DUMPING ACTIVITIES FOR FISCAL YEAR 2012

9 Program staff gave the evaluation team a copy of the FY2012 work plan.

10 The work allocation of ORISE fellows are considered non-FTE hours in the work plan and have been converted to contract

dollars for representation in the logic model. Contract costs associated with the OSV Bold are not included in this analysis.

7%

3%

3%

10%

24%

0%

0%

0%

0%

10%

14%

10%

21%

3%

3%

3%

3%

6%

6%

6%

6%

6%

6%

13%

16%

19%

0% 5% 10% 15% 20% 25%

Work w/ int’l orgs and State Dept

Develop regulations & policies, keep current w/ policy

Issue special or general (non-dredged material) permits

Develop guidance documents

Develop and update legislation

Assist regions to review & concur on dredged material …

Assist regions & USCG to enforce regulations

Assist regions in site monitoring & designation issues

Review regional programs

Conduct monthly call & semi-annual meeting for OD …

Address petitions, letters, and other inquiries

Strategic Planning

Develop policy on emerging issues

Allocation of FTE HRS Allocation of Contract Dollars

3-11

When the resources for specific activities are assigned to subsequent elements of the logic model based on the connections in the model, we find that activities leading to “environmentally acceptable conditions resulting from disposal” receive the largest percentage of FTE hours (46%) and contract dollars (38%), while the long-term goal of “no legal vulnerability” receives the fewest FTE hours (15%) and contract dollars (9%).11 Exhibit 3-5 shows the resource allocation for each of the four long-term program goals. (For the details of this analysis, and a depiction of resources across the logic model, see Appendix D.) As noted earlier, these percentages reflect intended allocations of effort, and in practice some staff time appears to be diverted to other near term priorities.

EXHIBIT 3-5. RELATIVE PRIORITIZATION OF PROGRAM GOALS FOR FY 2012

ALIGNMENT OF PARTNER GOALS

The Ocean Dumping Management Program has a variety of program partners whose goals span from protection of natural resources to maintaining navigational waterways. Exhibit 3-6 shows the program’s key partners, which include federal, state, and local entities, along with their goals related to ocean dumping. As noted in the introduction, the evaluation team decided to consider EPA Regions as partners of the national Ocean Dumping Management Program staff to the extent that they: a) contribute to the development of priorities for the program, and b) conduct activities that contribute to program outputs. EPA Regions are instrumental in achieving the program’s goals.

11 Contract costs associated with the OSV Bold are not included in this analysis.

31%

9%

22%

38%

10%

15%

29%

46%

0% 10% 20% 30% 40% 50%

Maintaining an international leadership role

No legal vulnerability

Coordinated and effective Ocean Dumping

management program

Environmentally acceptable conditions resulting from

disposal

Allocation of FTE HRS

Allocation of Contract Dollars

3-12

EXHIBIT 3-6. OCEAN DUMPING PROGRAM PARTNERS AND GOALS RELATED TO OCEAN DUMPING

PROGRAM PARTNER PARTNER GOAL(S) RELATED TO OCEAN DUMPING

EPA Regions Maintaining environmentally acceptable conditions resulting from disposal.

USACE Maintaining navigation and meeting regulatory responsibilities for dredged material with a focus on cost effectiveness.

Clean Water Act 404 Program

• Issuing permits for dredged material placed in ocean waters (from baseline12 to three nautical miles only) for a purpose other than dumping, i.e., for the purpose of raising the bottom elevation (e.g., beneficial reuse)

• Regulating discharge of dredged material to US waters, including wetlands

Coast Guard Surveillance and enforcement of dumping sites

Department of Interior Overseeing issues pertaining to mineral rights, carbon capture and storage, and platform/rig reefing and abandonment.

Environmental NGOs

Natural resource quality and protection Fish and Wildlife Service

National Oceanic and Atmospheric Administration

State Agencies Ensure consistency of state water quality standards/requirements for dumping that occurs in state waters

Navy Vessel Disposal (SINKEX)

Ports and Harbors Materials being dredged and/or placed in these locations

To assess how the goals of partners outside EPA align with the goals of EPA’s Ocean Dumping Management Program, the evaluation team asked program staff to rate their agreement with the following statement during the pre-interview survey: “The goals of the non-EPA program partners align with the goals of the Ocean Dumping program.” No respondents agreed with the statement, and over three-quarters of respondents disagreed or strongly disagreed with the statement (See Exhibit 3-7).

12 The United States baseline is the mean lower low water line along the coast, as shown on official United States nautical

charts. The baseline is drawn across river mouths, the opening of bays, and along the outer points of complex coastlines.

3-13

EXHIBIT 3-7. ALIGNMENT OF GOALS WITH NON-EPA PROGRAM PARNTERS

Interviewees specifically noted a disconnect with the goals of USACE. USACE’s goal is to maintain navigational waterways while meeting regulatory responsibilities in a cost-effective manner and with minimal delays. At times, this focus conflicts with the Ocean Dumping Management Program’s focus on ensuring adequate testing, monitoring, and review. This has led to friction between EPA and USACE in some Regions and Districts. For example, in EPA Region 6, the program struggles to obtain program-related data and information from USACE.

Aside from USACE, interviewees pointed out occasional overlaps in responsibilities and potential tension with other program partners. For example, while the Ocean Dumping Management Program, along with USACE, oversee the disposal of dredged material, the CWA 404 program also becomes involved when dredged material is placed in the ocean within 0-3 miles from the baseline for a purpose other than disposal (i.e., raising the bottom elevation). Where both MPRSA (ocean dumping) and CWA 404 programs intersect, some interviewees questioned which program should be providing guidance and authority. Interviewees indicated that at HQ, there is a clear delineation between the CWA 404 and Ocean Dumping Management Program staff, but in some Regions the same individuals may work in both programs. In another example, one interviewee noted that U.S. Fish and Wildlife Service has shifted their regional relationship from one of proactive collaboration to an increasing level of litigation.

15%

8%

69%

0%

8%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Don't Know/Not Sure

Neither Agree nor Disagree

Disagree/Strongly Disagree

Agree/Strongly Agree

"The goals of the non-EPA program partners align with the goals of the Ocean Dumping program."

3-14

Conclus ions

The program’s activities and outputs appear well structured to lead to desired outcomes, and the program’s intended resource prioritization is appropriate and consistent with EPA’s mission, since environmental protection is the top priority. Nevertheless, several assumptions that underlie the program theory do not hold true, and it will be important for the program to address these challenges.

In order to operate effectively the program will need to address key leverage points (e.g., updating regulations and guidance, and ensuring coordination across Regions).

While some misalignment amongst agency and organizational missions is typical, tension with USACE is posing a serious challenge for the Ocean Dumping Management Program, even though some Regions have a positive working relationship with USACE. Ensuring a more consistently productive relationship with USACE appears important to helping the program achieve its goals.

Recommendat ions

The evaluation team recommends that the Ocean Dumping Management Program:

• Update program regulations and/or guidance to improve consistency of program implementation and potentially improve relationships with USACE;

• Work towards more productive communication with USACE by acknowledging differences in mission and organizational constraints, and focusing on areas of shared responsibility;

• Clarify and communicate to senior EPA management the importance of the program, focusing on why this program is essential to protecting ocean ecosystems; and

• Consider rebranding the program with a new name and new focused outreach approaches designed to communicate to key audiences (e.g., co-regulators at federal and state levels, permittees, and environmental groups).

4-1

CHAPTER 4 | EMERGING TRENDS, OPPORTUNITIES, AND CHALLENGES

The Ocean Dumping Management Program has encountered significant changes over the past two decades, and faces emerging trends, opportunities, and challenges. This section describes our findings related to: 1) historical trends and emerging issues, 2) obstacles and delays, and 3) opportunities for improvement. At the end of the section we present our conclusions and recommendations.

FINDINGS

HISTORICAL TRENDS AND EMERGING ISSUES

We identified four historical trends important to understanding the context of the Ocean Dumping Management Program: 1) decreases in ocean dumping, and an increased focus on beneficial reuse, 2) advances in science and understanding of contaminants, 3) changes in program resources, and 4) staff turnover. Though there is some variation, these trends are common across all of the Regions. After considering common historical trends and emerging issues, we point out trends and issues that are specific to particular Regions.

Decreases in Ocean Dumping, Increased Focus on Benef ic ia l Reuse

Since the 1972 passage of MPRSA, ocean dumping of many types of substances has been phased out (with the notable exception of material from navigational dredging, which is discussed below). In particular, sewage sludge, which was dumped in the ocean by many cities starting in the 1920s, was outlawed by the Ocean Dumping Ban Act, which prohibited all ocean dumping of industrial waste and municipal sludge after December 31, 1991. The final city to cease dumping, New York City, ended the practice on June 30, 1992. On that date, the Deputy Administrator for EPA's Region 2 stated that "EPA is committed to … ensur[ing] that this problem is not just moved from the ocean to land, but that long-term, land-based disposal options are implemented. We will also continue to encourage the use of sludge as a benefit to the environment.”13

The theme of limiting ocean dumping and instead finding new uses for materials previously dumped in the ocean has continued, with a new focus on dredged material. Dredged material from navigation channels is now the predominant substance disposed of through ocean dumping, and managing this disposal has occupied a substantial portion of the Ocean Dumping Management Program’s time and resources. Between 1998 and 2009, there were 651 dredged material permits issued by USACE following EPA review 13 U.S. EPA Region 2 Press Release, “EPA Declares End of Ocean Dumping as New York City Ceases; EPA Committed to Long-

Term Beneficial Alternatives, “June 30, 1992, http://www.epa.gov/aboutepa/history/topics/mprsa/03.html

4-2

and concurrence. During this same period, an EPA general permit enabled the disposal of 50 vessels, seven emergency permits were issued by EPA, and two fish waste permits were issued (and reissued) by EPA.14 EPA has also issued permits to the U.S. Navy and the National Science Foundation for the transportation of target vessels and ice piers in Antarctica, respectively.

Exhibit 4-1 shows that on a national level, there was a pronounced decline in ocean dumping of dredged material starting in the early 2000s, with a record low in 2007, although in recent years the national trend has started to reverse. The amount of dredged material dumped at sea depends not only on permitting decisions by the Ocean Dumping Management Program and USACE, but also on the amount of dredging occurring, the contamination of dredged material, and the funds available for land disposal or beneficial reuse.

Dredging trends vary widely by Region and fluctuate frequently. Regions 4 and 6 have concurred on permits for the greatest volume of ocean dumping in most years of the program’s history. Ports in these EPA Regions are currently undergoing major harbor and channel deepening projects as they seek to accommodate post-Panamax vessels, a very large class of ship that will begin transporting goods internationally in coming years.15 The USACE has identified the Southeast and Gulf coast ports as top candidates for economically justified port expansion/dredging projects.16 One interviewee noted, “We have a number of ports that want to deepen for the post-Panamax vessels – [these projects] put a strain on the program, [because they require] a lot of testing to review a lot of material.”

14 National Ocean Disposal Activities Excel file, dated 9-11-12, provided by EPA Ocean Dumping Staff. There are also other

activities covered by EPA permits not listed in the spreadsheet. A burial at sea permit was issued in 1977; EPA does not

report individual burial at sea activities. The National Ocean Disposal Activities Excel file does not list activities related to

transportation for target practice permits in the time period described.

15 http://www.pancanal.com/common/maritime/advisories/2009/a-02-2009.pdf

16 Institute for Water Resources, U.S. Army Corps of Engineers, “U.S. Port and Inland Waterways Modernization: Preparing for

Post-Panamax Vessels: Report Summary.” June 20, 2012. http://www.iwr.usace.army.mil/index.php/us-port-and-inland-

waterways-moderinzation-strategy

4-3

EXHIBIT 4-1. OCEAN DUMPING OF DREDGED MATERIAL

Source: USACE. 2011. Ocean Disposal Database. http://el.erdc.usace.army.mil/Ocean Dumping/amount_by_all_districts.asp

EPA has encouraged beneficial reuse or land disposal rather than ocean dumping where feasible or necessary to protect marine habitats. USACE reports that currently “about 20 to 30 percent of port and waterway dredged material is used for habitat creation and other beneficial use.”17 Program staff note the connection between climate change, sea level rise, and beneficial reuse. For example, one survey respondent identified “climate change” as one of the top two emerging trends for the Ocean Dumping Management Program and said, “More and more, we are reviewing projects where sediments are being proposed for beneficial reuse, as protective measures for repair of dikes and providing sandy sediments for beaches and other coastal areas for protection of infrastructure.” Another respondent identified “alternatives to ocean disposal, especially in light of climate change and sea level rise” as one of the top two emerging trends. These responses highlight the extent to which EPA and its partners are increasingly considering beneficial reuse as an alternative to ocean dumping.

Advances in Sc ience and Understanding of Contaminants

Over the last two decades, EPA and USACE have jointly developed and updated several guidance documents for implementing various aspects of MPRSA. These include document:

17 Institute for Water Resources, U.S. Army Corps of Engineers, “U.S. Port and Inland Waterways Modernization: Preparing for

Post-Panamax Vessels: Report Summary.” June 20, 2012. http://www.iwr.usace.army.mil/index.php/us-port-and-inland-

waterways-modernization-strategy

4-4

• For evaluating alternative disposal methods for dredged material: Evaluating Environmental Effects of Dredged Material Management Alternatives -- A Technical Framework18 issued jointly by EPA and USACE in 1992 and revised in 2004. This document provides a general technical framework for evaluating the environmental acceptability of dredged material management alternatives, including open-water disposal (including ocean dumping), confined disposal (this includes disposal on nearshore or upland locations), and beneficial uses. The framework augments other existing program guidance, and is intended to foster enhanced consistency and coordination in USACE/USEPA decisions regarding dredged material management.

• For assessing the suitability of dredged material for ocean disposal: Evaluation of Dredged Material Proposed for Ocean Disposal Testing Manual,19 or the Green Book, which EPA issued jointly with USACE in February 1991. The manual contains technical guidance for determining the suitability of dredged material for ocean disposal through chemical, physical, and biological evaluations. The Green Book updated the prior testing manual, the Ecological Evaluation of Proposed Discharge of Dredged Material into Ocean Waters, published in 1977. The 1991 update introduced a tiered testing approach that sequentially increases the amount of information collected to fully consider biological impacts.20

• For preparing ocean dredged material disposal site management plans: Guidance Document for Development of Site Management Plans for Ocean Dredged Material Disposal Sites developed jointly by EPA and USACE in 1996. This document lays out a recommended framework for site management plan development and content.

EPA and USACE have provided several updates to these guidance documents. For example, in the 1990s, EPA made changes to its bioaccumulation testing procedures that had a substantial impact on the amount of material that could be placed in a major ocean dumping site off of the Port of New York and New Jersey. These changes resulted in re-designating the location as a remediation site that would accept only uncontaminated sediment.21 Between 2000 and 2003, EPA tightened its standards for substances contaminated with polychlorinated biphenyls, or PCBs, permitted for ocean dumping in the same EPA Region. These changes in guidance and standards were the subject of (and some say a product of) a series of lawsuits initiated by Clean Ocean Action and other environmental advocacy groups pushing for more rigorous bioaccumulation testing procedures, and a lawsuit from the United States Gypsum Company arguing for EPA to

18 http://water.epa.gov/type/oceb/oceandumping/dredgedmaterial/upload/2004_08_20_oceans_regulatory_ dumpdredged_

framework_techframework.pdf

19 http://water.epa.gov/type/oceb/oceandumping/dredgedmaterial/upload/gbook.pdf

20 Burroughs, Richard and Christine Santora, “Disposal of Contaminated Sediment from The Port of New York,” University of

Rhode Island Transportation Center, June 2004.

21 Ibid.

4-5

maintain its current PCB standards for the purpose of its permit application. The complex history of these lawsuits and related negotiations, public involvement process, and involvement of Congressional delegations and Vice President Gore is beyond the scope of the current document, but is described in existing literature.22

The most recent update made to the ocean dumping guidance was the 2004 framework for evaluating alternative disposal methods for dredged material. The underlying legislation (MPRSA) was last updated in 1992 by the Water Resources Development Act of 1992 (Public Law 102-580).23 As noted earlier, in FY2012, the Ocean Dumping Management Program prioritized activities including strategic planning and developing policy on emerging issues; in contrast the activity of developing guidance documents was the focus for only three percent of total FTE hours (see Exhibit 3-4).

Overall, program staff and stakeholders expressed concern that the regulations and guidance documents have not kept pace with advances in science and therefore need to be updated. Several program staff mentioned outdated regulations, new contaminants, and the need to revise testing and assessment methods as one of the top two emerging trends for the Ocean Dumping program. For example, one survey respondent discussed “emerging contaminants,” explaining that “Pyrethroids are coming to the forefront in various watersheds …we are beginning to look into this class of pesticides as a source of acute and chronic toxicity, but we will soon need guidance from HQ and ORD [the EPA Office of Research and Development]. Similarly, pharmaceuticals may be implicated in contributing to chronic toxicity in sediments.” Another respondent said that a top challenge is “sediment testing improvements (including ocean dumping regulation revisions): how/when to incorporate emerging contaminants, combining the Ocean and Inland Testing Manuals, etc.” For example, one interviewee said, “We know so much more about contamination and carcinogens nowadays, but we don’t sample for them because the 1970’s regulations don’t [cover] them.”

A new class of substances related to climate change that may be proposed for ocean dumping, including sub-seabed carbon sequestration and geo-engineering (e.g., ocean fertilization), constitutes an important emerging issue that will likely receive increased public attention over time, particularly in light of their potential for environmental impacts. For example, Canada’s environmental ministry is currently investigating an incident involving the Haida Salmon Restoration Corporation and of a California businessman who reportedly dumped iron dust approximately 200 nautical miles of the coast of British Columbia.24 The iron sparked the growth of plankton, which was intended to aid the recovery of the local salmon fishery for the native Haida as well as to sequester carbon, because plankton absorbs carbon dioxide and settles deep in the ocean

22 For a detailed history, see Burroughs, Richard and Christine Santora, “Disposal of Contaminated Sediment from The Port of

New York,” University of Rhode Island Transportation Center, June 2004. Additional analysis of the case is included in

Brown, N. and R.W. Knecht. 1998. “The New York/New Jersey Harbor dredging conflict.” In J.R. Waldman, W.C. Nieder

(eds.), Final Reports of the Tibor T. Polgar Fellowship Program, 1997. Hudson River Foundation.

23 http://www.epa.gov/owow/oceans/ndt/publications/pdf/1996_smmp_guidance.pdf

24 Fountain, Henry, “A Rouge Climate Experiment has Ocean Experts Outraged.” New York Times, section A1, Friday October

19, 2012.

4-6

when it dies.25 This issue was raised at the 29 October –2 November 2012 London Convention/London Protocol meeting where the Parties to the London Convention/London Protocol issued a statement of concern regarding this ocean fertilization activity. The EPA Ocean Dumping Management Program could be called upon to prevent or enforce against similar activities off the U.S. coast in the future.

Changes in Program Resources

As funding for OWOW and the Office of Water overall have increased, funding for the Marine Pollution Control Branch, which includes the Ocean Dumping Management Program, has declined 14 percent in the last six years (Exhibit 4-2). During the same period, while the number of HQ OWOW staff FTEs allocated to the Ocean Dumping Management Program has more than doubled, the overall FTE allocated to the HQ program remains relatively small (less than 4 FTE) (Exhibit 4-3). There is broad consensus among regional and HQ program staff interviewed that they are being asked to do “more with less,” and that they have fewer resources to do their jobs. Ten of 13 survey respondents cited reduced or insufficient funding as a top emerging issue or challenge for the program. One survey respondent said a top challenge for the program is “limited resources and a continually growing portfolio (a broad array of issues).”

Program staff have been actively discussing EPA’s disinvestment in the Ocean Survey Vessel Bold and its implications for the Ocean Dumping Management Program. Ten of 13 survey respondents identified the loss of the Bold or maintaining sufficient funding for monitoring as one of the top two emerging issues or challenges for the program. Interviewees from several regional offices, particularly those on the east coast, were concerned about the impact of the loss of the Bold on monitoring. One interviewee explained that the vessel was the program’s most important monitoring tool and its only solid asset. Interviewees worried that the loss of the ship would open the door to future budget cuts. As one interviewee explained, “One of the things about having a ship is that you can’t cut [the budget] below $5 million. [But] once you get rid of it you can cut it whenever you want.” Another interviewee remarked, “We need to be developing a strategy for that [monitoring] money [to replace the Bold] so we don’t lose it, so it’s predictable, [and] so we can plan.” Other interviewees expressed concern about the monitoring work the Bold used to do for programs other than the Ocean Dumping Management Program (e.g., related to corals and Total Maximum Daily Load monitoring), which will no longer be supported. On the other hand, some interviewees thought that the disinvestment from the Bold could bring more equity in regional monitoring resources. One interviewee said “With the east coast having the majority use of the vessel, their approach to monitoring [has been] really different than ours…. But now there’s an opportunity to have all the Regions look at how ocean disposal sites are managed and monitored in a consistent manner.”

25 Fountain, Henry, “A Rouge Climate Experiment has Ocean Experts Outraged.” New York Times, section A1, Friday October

19, 2012.

4-7

Program managers note that budget cutting exercises take time in and of themselves, which could be better directed toward substantive planning for the program. For example, one interviewee said, “We have spent a lot of time on budgeting exercises for the last year and a half … with all of those resources we could have done things that would have been more important. Rather than working on the budget level allocation, we should be working on planning [for the future]… for example how to meet required monitoring levels [and] how to even out resources between the Regions.”

Program staff report they are concerned that they are less involved in resource decisions than they have been in the past. One interviewee said, “Before the last two or so years, program staff across Regions had a bottom-up style of communication that fostered team decision-making and a stewardship approach to work. For example, if a budget cut had to made, we would all talk it out until we could come to some understanding of how best to proceed. Now, we have managers at the next level above us making decisions that we don’t necessarily agree with, and they inform us of those decision through rather poor messaging. …There is a lack of transparency.” One manager from the Office of Water reports that the Agency as a whole is under severe budget constraints, and that the Ocean Dumping Management Program needs to clearly communicate what it does and the value of the program to the EPA mission.

4-8

EXHIBIT 4-2. RESOURCE TRENDS WITHIN THE OFFICE OF WATER (DOLLARS IN THOUSANDS)†

PROGRAM

- RELATED GOAL(S)

PERCENT

CHANGE OVER

SIX YEAR

PERIOD

FY 2008

ACTUAL

BUDGET

FY 2009

ACTUAL

BUDGET

FY 2010

ACTUAL

BUDGET

FY 2011

ACTUAL

BUDGET

FY 2012

ENACTED

BUDGET

FY 2013

PRESIDENT’S

BUDGET

Office of Water - Clean and Safe Water/

Protecting America’s Waters 21% $3,119,201.2 $8,887,323.4 $4,989,963.6 $5,085,863.7 $4,094,452.5 $3,782,228.0

Office of Wetlands, Oceans, and Watersheds - Protect Water Quality

55% $1,658,310.4 $5,538,892.8 $3,375,542.5 $3,553,462.7 $2,798,913.5 $2,565,462.0

Marine Pollution (14%) $13,430.4 $13,064.7 $9,783.7 $15,570.5 $12,898.0 $11,587.0 † Source: EPA Congressional Justifications, available at http://www.epa.gov/planandbudget/archive.html. Includes contract costs associated with the OSV Bold.

EXHIBIT 4-3. RESOURCE TRENDS WITHIN THE OFFICE OF WATER (ANNUALIZED FULL-TIME EQUIVALENTS) †

PROGRAM

- RELATED GOAL(S)

PERCENT

CHANGE OVER

SIX YEAR

PERIOD

FY 2008

ACTUAL

BUDGET

FY 2009

ACTUAL

BUDGET

FY 2010

ACTUAL

BUDGET

FY 2011

ACTUAL

BUDGET

FY 2012

ENACTED

BUDGET

FY 2013

BUDGET

Office of Water - Clean and Safe Water/

Protecting America’s Waters 21.4% 2,815.1 2,868.3 3,471.3 3,510.3 3,423.6 3,418.9

Marine Pollution 2.1% 42.8 44 41.5 44.5 43.7 43.7

EPA HQ Ocean Dumping Management Program‡

OWOW FTE

128%

1.71

1.30

1.00

2.5

3.6

3.9

† Source: EPA Congressional Justifications, available at http://www.epa.gov/planandbudget/archive.html ‡ Source: Work plans for FY 08 through FY 13 provided by Ocean Dumping Management Program staff. Where possible, values taken from subsequent year’s work plan (to more closely account for FTEs actually used, not just planned).

4-9

Staf f Turnover

The Ocean Dumping Management Program has experienced considerable staff turnover in recent years. One interviewee noted that three of the seven Ocean Dumping Regions have lost staff with long tenures over the last seven years. Another interviewee noted that “It’s hard to know what to do and how to do it in this program unless you’ve been in it a long time. We need checklists… and other documentation to get people up to speed quickly.” Loss of institutional knowledge about the Ocean Dumping Management Program’s processes and procedures may lead to inefficiencies in permitting, monitoring, and other activities.

Trends and Emerg ing I ssues Speci f i c to EPA Reg ions

In addition to common historical trends and emerging issues cited by respondents, several issues were cited by only one or two Regions as a high priority in their jurisdiction. These issues include:

• Restoring the Gulf of Mexico, including post British Petroleum Deepwater Horizon work in the Gulf of Mexico related to Ocean Dredged Material Disposal Sites and harbors;

• Managing major harbor and channel deepening projects in the southeastern US. As noted earlier, ports in this area are expected to undergo major expansion in coming years to accommodate post-Panamax vessels;

• Ensuring proper disposal of debris from the Japanese tsunami on the west coast, and avoiding introduction of invasive species on the west coast;

• Ensuring proper disposal of fish waste in Alaska and on the west coast; and

• Protecting sensitive near shore habitats (e.g., reefs) in the Caribbean from effects of dredged material transport.

Survey respondents are relatively evenly divided over whether HQ and Regions face the same emerging issues and challenges: just over half (54 percent) of survey respondents report that EPA HQ and Regions face the same emerging issues, and a slightly smaller share of survey respondents (46 percent) agree that EPA HQ and Regions face the same challenges (see Exhibits 4-4 and 4-5). The survey responses do not suggest any notable patterns of agreement or disagreement based on the Region of the country or the tenure of staff responding.

4-10

EXHIBIT 4-4. SHARED HEADQUARTERS AND REGIONAL EMERGING ISSUES

•

•

•

•

•

•

•

•

•

•

•

•

•

EXHIBIT 4-5. SHARED HEADQUARTERS AND REGIONAL CHALLENGES

•

•

•

8%

0%

38%

46% 8%

0% 10% 20% 30% 40% 50% 60%

Don't Know/Not Sure

Neither Agree nor Disagree

Disagree/Strongly Disagree

Agree/Strongly Agree

"Headquarters and regional staff face the same emerging issues."

15%

8%

31%

46%

0% 10% 20% 30% 40% 50%

Don't Know/Not Sure

Neither Agree nor Disagree

Disagree/Strongly Disagree

Agree/Strongly Agree

"Headquarters and regional staff face the same program challenges."

4-11

OBSTACLES AND DELAYS

Findings from this evaluation suggest that the Ocean Dumping Management Program has faced obstacles and delays in recent years. Eighty-four percent of program staff surveyed agree or strongly with the statement that “Over the past three years the Ocean Dumping Program has encountered significant obstacles and/or delays to accomplishing its mission” (Exhibit 4-6). Obstacles cited include:

• Lack of adequate time and resources;

• Lack of clear priorities and “mission creep;”

• Limited senior management direction/feedback;

• Administrative “fire drills;”

• Loss of the Bold and associated increases in administration time to find other vessels to serve this function;

• Lack of funding for monitoring Ocean Dredged Material Disposal Sites so effects to marine species are not adequately understood;

• USACE disagreement or inflexibility and lack of communication with some USACE Districts;

• Lack of cooperation between federal and state agencies, despite best efforts of Regional Dredging Team; and

• Threat of lawsuits.

EXHIBIT 4-6. ENCOUNTERING OBSTACLES AND DELAYS

0%

15%

0%

69% 15%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Don't Know/Not Sure

Neutral

Disagree/Strongly Disagree

Agree/Strongly Agree

"Over the past three years the Ocean Dumping program has encountered significant obstacles and/or delays to

accomplishing its mission."

4-12

OPPORTUNITIES FOR IMPROVEMENT

Program managers seeking to improve the Ocean Dumping Management Program will need to anticipate emerging issues, challenges, and opportunities. Survey results suggest that to date, the Ocean Dumping Management Program has done a better job of anticipating emerging issues compared to anticipating challenges and opportunities, but in general the program has room to improve in both of these areas. Specifically, only 31 percent of respondents agreed or strongly agreed with the statement “Over the past three years, the Ocean Dumping program as a whole has done a good job of anticipating emerging issues” (roughly two-thirds of respondents disagreed with this statement or did not express an opinion) (see Exhibit 4-7). Only 15 percent of respondents agreed with the statement “Over the past three years, the Ocean Dumping program as a whole has done a good job of anticipating challenges and opportunities,” and no respondents strongly agreed (see Exhibit 4-8). These findings suggest that the Ocean Dumping program needs a renewed focus on strategic planning, anticipating emerging challenges, and seizing potential opportunities. In the following sections, we highlight three areas of opportunity for the Ocean Dumping program to focus on improvements: 1) exercising greater coordination and leadership from EPA HQ, 2) updating regulations and guidance, and 3) supporting London Protocol ratification.

EXHIBIT 4-7. ANTICIPATING EMERGING ISSUES

8%

31%

31%

23% 8%

0% 5% 10% 15% 20% 25% 30% 35%

Don't Know/Not Sure

Neither Agree nor Disagree

Disagree/Strongly Disagree

Agree/Strongly Agree

"Over the past three years the Ocean Dumping program as a whole has done a good job of anticipating emerging issues."

4-13

EXHIBIT 4-8. ANTICIPATING EMERGING CHALLENGES AND OPPORTUNITIES

Opportunity to Exercise Greater Coordinat ion and Leadership f rom EPA HQ

Interviewees and focus group participants suggest that, particularly in light of the recent staff turnover, EPA HQ could provide more leadership, facilitation, and coordination for the Regions. One survey respondent reported that a major challenge for the program was the “lack of consistency and standard operating procedures between Regions.” Regions seek a centralized repository of identified checklists, resource lists, and other resources. Participants in the Ocean Dumping Coordinators meeting acknowledged that the creation of training documentation may represent a large investment of time and effort in the short term but is a necessary step to address staff turnover and foster the long term resilience of the program.

Staff praised EPA HQ’s recent convening of an Ocean Dumping Coordinators meeting in Washington, D.C. as an opportunity to foster coordination across the Regions. Participants at the meeting acknowledged that regular in-person meetings might not be possible due to resource constraints, but suggested that EPA HQ seek creative means to gather Ocean Dumping coordinators from across the country in focused conversations (e.g., through facilitated conference calls and/or meetings using online document sharing and participation software such as GotoMeeting, WebEx, or Microsoft 365). Meeting participants also acknowledged that EPA HQ does hold regular conference calls, but pointed out that they tend to be an opportunity for updates, rather than focusing on a particular topic of concern.

8%

23%

54%

15%

0% 10% 20% 30% 40% 50% 60%

Don't Know/Not Sure

Neither Agree nor Disagree

Disagree/Strongly Disagree

Agree/Strongly Agree

"Over the past three years the Ocean Dumping program as a whole has done a good job of anticipating challenges and

opportunities."

4-14

In addition to coordination across Regions, staff seek greater leadership from EPA HQ. During the strengths, weaknesses, opportunities, and threats analysis at the Ocean Dumping Coordinators meeting, participants identified an absence of leadership as a key threat or weakness for the Ocean Dumping Management Program. One interviewee said, “There is a national consistency issue: [we need to] get HQ on the same page [and get] information coming down from the top telling [USACE] Districts what they need to do.” Another interviewee reported, “We have five or six voices telling you how to do things. [There is] unclear program direction [and a] lack of a unified voice from leadership.”

Opportunity to Update Regulat ions and Guidance

Although there is broad-based agreement among Ocean Dumping Management Program staff and program stakeholders that it is important to update the regulations and program guidance, several barriers make this task difficult. First, the regulations themselves are quite detailed and need to be at least partially updated, if the program is to fully update the technical guidance. Moreover, stakeholders do not agree on how to update the regulations. One interviewee explained that the program’s technical guidance cannot be changed because it was codified into the original regulations. Another challenge is that current resource constraints and demands on program staff time have taken focus away from efforts to update the regulations and guidance. As noted earlier, the program has updated its testing guidance in the past, which was the subject of litigation. One of the four long term outcomes that the program says it is trying to achieve is “no legal vulnerability.” One interviewee suggested that the program’s emphasis on avoiding litigation has “stymied program creativity.” Another interviewee remarked that “It would save everyone a lot of work to update the regulations because they are not working,” but that the “risk of changing the regulations is to EPA, not USACE,” presumably because once open for negotiation the regulations could become less protective of the marine environment.

The fact that the Ocean Dumping Management Program has not updated the guidance in recent years is straining relations with USACE and making program implementation more challenging for the Regions. The Ocean Dumping Management Program is planning to update the MPRSA regulations in conjunction with the London Protocol Ratification process, which could have the added benefit of potentially improving relations with USACE. The London Protocol Ratification process is described in the next section.

Opportunity Re lated to London Protocol Rat i f i cat ion

The London Convention, which the U.S. ratified in 1975, covers the deliberate disposal of wastes or other matter from vessels, aircraft, platforms, and other man-made structures at sea. In 1992, the 80 Parties to the London Convention began a comprehensive review of the Convention, resulting in a new treaty called the 1996 Protocol.26 The London Protocol, which the U.S. has signed but not yet ratified, updates and improves upon the London Convention. Currently, some Parties to the London Protocol are interested in

26 http://water.epa.gov/type/oceb/oceandumping/dredgedmaterial/londonconvention.cfm

4-15

developing a binding regulation to address marine geoengineering activities, including ocean fertilization, through an amendment to the London Protocol.

The United States signed the 1996 London Protocol in 1998. Since then, the United States has been working on the ratification process via implementing legislation (amendments to the MPRSA) and seeking necessary action by the Senate. In 2007, the White House submitted the London Protocol to the U.S. Senate for its advice and consent. The London Protocol is on the Obama Administration’s treaty priority list. The proposed implementing legislation submitted to Congress in 2008 is undergoing review prior to re-submittal to Congress. The Ocean Dumping Management Program is currently reviewing the MPRSA legislation package submitted to Congress in 2008 to align with the requirements of the London Protocol. One interviewee noted that, “Updating MPRSA would allow us …to fix a few things to be more efficient for how we implement the Ocean Dumping Management Program.” For example, one interviewee explained that “right now [the program] can only issue an emergency permit if it’s a threat to human health. We don’t have the authority to issue permits to deal with an emergency if the threat is to environmental health.”

If the MPRSA is revised, the Ocean Dumping Management Program will also have an opportunity to adjust the scope of the regulation. For example, the program may be called upon to help mitigate emerging issues like ocean acidification and climate change (e.g., ocean fertilization). Interviewees disagreed about whether the program should expand its scope to address emerging ocean problems, such as acidification, sea level rise, and carbon capture. Some program staff felt that the program should expand its purview beyond dumping to address these issues. For example, one regional interviewee suggested that EPA is likely “missing out” on helping with issues such as “ocean acidification, invasive species, and marine debris.” A survey respondent remarked “[The program] need[s] to expand [the] scope of [the] program beyond just ocean dumping; we need to be looking at other ocean issues, and there are a lot of them.” Other staff felt that expanding the scope of the program would be a distraction for a program with a narrowly focused role and a lack of resources. Some interviewees expressed skepticism about the viability of carbon sequestration in deep ocean waters. Considering which of these issues to include in its scope will be a major strategic decision for the Ocean Dumping Management Program going forward.

CONCLUSIONS

The Ocean Dumping Management Program has been challenged to clearly articulate the scope and benefits of the program, given shifts in the types of materials proposed for ocean disposal, multiple agencies involved in ocean dumping permitting, and reduced public attention to environmental and human health risks of ocean disposal.

The changing nature of contaminants and advances in monitoring and testing technologies require updates to EPA requirements and guidance. While the Ocean Dumping Management Program has jointly developed and updated several guidance

4-16

documents for implementing various aspects of MPRSA with USACE, the differences in perspective between the two agencies, multiple program priorities, and limits to EPA staff resources have constrained EPA’s ability to keep pace with needed updates.

The Ocean Dumping Management Program is working with a very small staff at HQ, and is challenged to provide effective coordination, leadership, and guidance to the Regions. Staff turnover and associated loss of institutional knowledge, insufficient guidance and documentation, limited opportunities for coordination across Regions, and varying levels of cooperation from USACE have hampered consistent implementation of the program across the country.

RECOMMENDATIONS

The evaluation team recommends that the Ocean Dumping Management Program:

• Update program guidance and use the London Protocol ratification process as an opportunity to update MPRSA;

• Identify approaches to take on the additional responsibilities that will come with ratification;

• As a part of updating program guidance on regulations, work to improve communications to Regions about how to implement the regulation, standard procedures, and expectations for program partners (e.g., USACE Districts);

• Foster communication and consistency across Regions; and • Seek to foster improved communication and partnership with USACE.

5-1

CHAPTER 5 | RESOURCE ALLOCATION

Given current resource trends and a broadening portfolio of work, the Ocean Dumping Management Program seeks to identify opportunities for improving the alignment between resource deployment and the program’s intended or desired outcomes. We group our findings into four main topics: 1) resources that are required vs. those that can be reallocated, 2) opportunities to improve alignment of resources with intended program outcomes, 3) opportunities to improve alignment of resources across Regions, and 4) opportunities, risks, and impacts of resource reallocation. Conclusions and recommendations follow the findings.

FINDINGS

RESOURCES REQUIRED VS. THOSE THAT CAN BE REALLOCATED

Most individuals we interviewed believe that the program is operating with minimal resources and that there is no opportunity to be gained by moving resources out of the program. For example, one interviewee said, “This is a program that in my view never had any ‘fat.’ This program is running on empty. It’s really hard for me to think what they could focus their resources on better; they’re only doing the absolute core essential elements of the program because that’s all they can do.” Another interviewee said, “Can the program be cut more and still meet the regulatory requirements? There are certain aspects of [the program] that really aren’t discretionary; when EPA looks at disinvestment, [it should] look at whether this is something some other agency is doing, [and whether the Ocean Dumping Management Program] has authorities that no one else has. [Resource reductions are] cutting into the bones of the agency we’re supposed to keep alive.” Interviewees point out that the Ocean Dumping Management Program has authority for enforcement, permits (specifically concurring on dredged material ocean dumping permits and issuing ocean dumping permits for all other substances), site management, and site monitoring, and that EPA cannot meet these responsibilities with fewer resources. Only one interviewee (outside of EPA) suggested that the program had resources to spare: this person said “EPA’s 35-person staff is still the same size as it was back when everything (not just dredged material) was dumped in the ocean.” Program FTE data shown in Exhibit 4-3 suggests that this size staff more likely relates to the Marine Pollution Control Branch as a whole, not just the Ocean Dumping Management Program.

OPPORTUNITIES TO IMPROVE ALIGNMENT OF RESOURCES WITH INTENDED

PROGRAM OUTCOMES

While not explicitly addressing opportunities to improve resource alignment, interviewees did suggest situations where resource alignment is not currently optimized.

5-2

For example, interviewees generally agree that the program could do a better job of focusing its efforts and working more efficiently. Some interviewees suggested that program staff were trying to work on too many disparate issues and that they needed to remain focused on core regulatory requirements. For example, one regional staff person said, “We have suffered from mission creep …HQ requires Regions to help enforce ocean dumping [requirements] but also help with the policy for the oceans27 … But we only get FTEs for the dumping portion. This [mismatch] in resource allocation and duties presents confusion in the program.”

Some interviewees suggested that the program is operating in a reactive mode because it is so under-resourced that it does not have the systems and procedures in place to anticipate and manage workloads. One person said, “We’re just trying to hold the key things together and deal with administrative [requirements], political directives, emergencies, and prepare for international meetings [related to the London Protocol]….We respond to emergencies well, but what happens is a lot of other stuff languishes….It would be great just to finish the guidance on the testing manual, but we’ve made slow progress. We don’t have the resources to get over the hump.” Interviewees also noted that lack of resources and attention to documenting procedures and ensuring consistency between Regions (e.g., establishing consistent site monitoring frequencies) leads to inefficiencies in work. The lack of effective manuals and guidance leads to increased time and resources needed to bring new staff up to speed when they replace experienced staff. This has been a particular challenge in Regions that have historically experienced management challenges and/or significant staff turnover. For example, one Region is working to create electronic filing systems, standard operating procedures, updated research, and a new understanding with USACE about what is required under the regulations. This renewed effort has required an investment of resources not only within the Region but also at EPA HQ.

OPPORTUNITIES TO IMPROVE ALIGNMENT OF RESOURCES ACROSS REGIONS

Several interviewees suggested changes to how resources are allocated across Regions, though there was no clear consensus on how to improve this allocation. Two interviewees said that resources should not be allocated equally between the Regions, but rather should be based on the number of ocean dumping sites within the Regions. However, another interviewee cautioned that this sort of pro-rated allocation had been tried in the past, and that it had led to some Regions that received relatively more funds redirecting funds away from ocean dumping to other activities.

A few interviewees suggested that there is a minimum level of resources required for monitoring, and that each Region should be allocated sufficient resources to conduct required monitoring. One interviewee (in the east) said, “What we’ve had to do is bare bones monitoring. We have adapted how we are monitoring sites by reducing the number of samples and replicates, and instead of doing everything the regulations say, we

27 The interviewee most likely was referring to the National Ocean Policy described at

http://www.whitehouse.gov/administration/eop/oceans/policy

5-3

have pulled back to the absolute minimum where we could still get useful information. At some point, data is so minimal it is not useful.” On the other hand, another interviewee said, “the east coast has had the [Bold] vessel available and has had an incentive to keep it busy, so they’ve done extra work, while we can’t even do the basics on the west coast. There needs to be a nationally recognized minimum monitoring policy.” Interviewees point out the importance of monitoring to protect human health and the environment. For example, one regional representative noted that monitoring is important to understand how activities during disposal affect marine environment. Another regional representative said, “Site monitoring is a feedback mechanism. [Monitoring allows us to] look for trends to see whether testing and management is adequate; without the monitoring results, we wouldn’t be able to make changes in how materials are placed [in the ocean to protect the marine environment.]” Another interviewee pointed out that ensuring adequate monitoring frequency is necessary to ensure that EPA is meeting its regulatory obligations and that the Agency is not vulnerable from a legal perspective. Until the minimum requirements for monitoring are agreed on and Regions determine how they will conduct monitoring without the Bold vessel, it remains unclear what are the minimum dollar resources Regions will need to contract out adequate monitoring.

With regard to allocation of staff across Regions, the current allocation is based on the historical number of ocean dumping sites in each Region. One interviewee argued this no longer makes sense. However, this interviewee said “it doesn’t matter because Regions allocate staff the way they want to anyway….and HQ does the same thing…. a lot of staff who have nothing to do with ocean dumping are officially ‘Ocean Dumping FTEs.’” One interviewee said that each Region should have a minimum of two staff persons, to allow for backup/redundancy in case one person leaves. In a group discussion among program staff, participants suggested that the Program assess the FTE allocation across regional offices and revisit the workload model from the 1990s.

OPPORTUNITIES, RISKS, AND IMPACTS OF RESOURCE REALLOCATION

As noted earlier, the Ocean Dumping program has recently undergone a major resource reallocation, namely disinvestment from the ocean monitoring vessel, the Bold. While several Regions lamented the loss of the vessel, others felt it would lead to an appropriate reallocation of resources. In addition, interviewees pointed out that the act of disinvestment itself has taken resources and has been a considerable distraction for the program in recent years. Some interviewees expressed concern that disinvestment from the Bold will actually end up costing EPA more resources in the long term.

CONCLUSIONS

The findings suggest that the Ocean Dumping Management Program does have potential to improve alignment of its resources with intended outcomes. We conclude that the program managers could help align resources by confirming the program’s priorities,

5-4

articulating the minimum resources needed to meet those priorities at both the HQ and regional level, and requesting support from senior leadership to focus the program’s limited resources on those priorities. Findings from interviews suggest that the HQ program is vitally important to coordinate, guide, and ensure consistency across Regions. In addition, HQ is essential to representing the Agency in cross-agency efforts, e.g., the interagency workgroup on the London Convention. In our opinion, these areas of activity should be the primary focus of the Ocean Dumping Management Program.

Whenever resources are shifted, there are likely to be “winners” and “losers,” and it is important to be sure that the change provides enough benefits to be worth the friction and distraction of implementing the change. Program staff and managers have not developed a shared view of the minimum requirements that HQ and the Regions must fulfill, and the resources needed to complete those tasks. Without such an agreement, it is difficult to pinpoint opportune areas for resource reallocation.

It is not clear that the program has adequate resources to address its core functions, take on new emerging issues, and cope with the transaction costs associated with budget restructuring. To operate with existing resources, the program may need to prioritize its activities with a focus on core program functions (e.g., providing guidance to the Regions), while devoting less attention to EPA initiatives that are important but not directly related to the program’s mission (e.g., administrative work associated with issuing monitoring grants in lieu of the Bold, and coordinating with other agencies on carbon capture and storage). On the other hand, in order to address emerging issues (e.g., related to climate change) the program may need more resources.

RECOMMENDATIONS

The evaluation team recommends that the Ocean Dumping Management Program take the following steps:

• Assess resource allocations for Regions going forward: Define the minimum requirements that the Regions must fulfill. Research what future program activity levels (e.g., permit review, site designation activities) can be anticipated in each Region, given trends in ocean dumping and port development. Analyze what resources will be required in each Region to meet minimum program requirements given anticipated activity levels. Finally, consider resource reallocation across Regions if necessary to ensure a balance between effort required and resources provided.

• Assess minimum resource requirements for the HQ program: given work on the London Protocol and the need to update the regulations and provide additional guidance to Regions, determine what resources will be needed to adequately carry out core program activities.

6-1

CHAPTER 6 | PERFORMANCE MEASURES

Performance measures provide essential feedback to assess program performance and to inform ongoing program learning and evaluation. There are a number of factors which should inform the design of performance measures for any program. These factors include, for example, the goals of the program, the aspects of the program over which program managers exert some control or influence, and the availability of reliable and meaningful data. Programs may develop a suite of performance measures that include both output and outcome measures intended to provide a comprehensive picture of program results. Not all aspects of a program can be measured; the best measures provide an indicator of program effectiveness (See Exhibit 6-1 for additional characteristics of effective program measures.)

The Ocean Dumping Management Program seeks to measure its success in a complex environment. First, the program is trying to achieve goals that have been articulated in several different ways and the definition of the goals is currently evolving. Traditionally, the program has articulated its goals in terms of implementing the MPRSA and the London Convention. The FY13 Congressional Justification describes the program’s major areas of effort as “Designating, monitoring, and managing ocean dumping sites and implementing provisions of the National Dredging Policy.” As part of the draft “EPA Operational Priorities for FY13 Ocean and Coastal Protection” the Ocean Dumping Management Program is helping define an operational priority for ocean dumping, which addresses monitoring of ocean disposal sites to ensure protection of environmental health, appropriate testing of material to be disposed, and encouraging beneficial reuse of materials. The program is also considering a process of engaging the users of ocean disposal sites in financing site designation and management costs.

EXHIBIT 6-1. CHARACTERISTICS OF

EFFECTIVE PERFORMANCE MEASURES

Data are:

• Available

• Measurable

• Verifiable

Measures are:

• Timely and actionable

• Responsive to the needs of stakeholders (e.g., staff, managers, public, funders)

• Comprehensible and credible to stakeholders

• Motivating

• Comparable across jurisdictions and programs, and over time

6-2

Program staff say they are working to minimize the amount of substances placed into the ocean and to ensure that any matter that is dumped is tested appropriately, meets applicable standards, and is managed and monitored appropriately. However, the program has no direct influence over the amount of material that is dumped; this is largely dependent on dredging projects overseen by USACE and influenced by economic conditions and priorities, as well as the amount of sedimentation in navigation channels. While the Ocean Dumping Management Program seeks to encourage beneficial reuse where possible, this approach is often logistically or economically infeasible.28 In the remainder of this chapter we present our findings regarding the current performance measures in use, strengths and weaknesses of the current measures, and potential alternate measures, along with our conclusions and recommendations.

FINDINGS

CURRENT MEASURES

Overall, the Ocean Dumping program focuses on measuring outcomes, namely the percentage of Ocean Dumping sites which meet specified conditions. Specifically, the Ocean Dumping program has a strategic measure which states that, “By 2015, 95 percent of active dredged material Ocean Dumping sites, as determined by 3-year average, will have achieved environmentally acceptable conditions (as reflected in each site’s management plan and measured through onsite monitoring programs).”29 (See Exhibit 6-2 for a description of performance on this measure). In addition, the Ocean Dumping program tracks the “number of active dredged material Ocean Dumping sites that are monitored in the reporting year.”30 The FY 13 Congressional Justification states that “During FY 2013, the EPA will collect environmental data from several offshore areas for use in the designation of… dredged material disposal sites and …monitor, as required, the 67 active dredged material ocean disposal sites.” In addition to documented measures, program staff shared that they track additional measures and are working on developing new measures. For example, one interviewee described an output measure under development that would track the number of projects and suitability determinations processed each year. Another interviewee said the program tracks the timeframes of how often site monitoring is conducted and when site management plans are updated, to track whether the site management plans are reviewed and revised every 10 years as required.

28 EPA Operational Priorities for FY13 Ocean And Coastal Protection, Draft August 27, 2012

29 Source FY 2011 – 2015 EPA Strategic Plan, http://www.epa.gov/planandbudget/strategicplan.html. The universe of active

dredged material ocean disposal sites was 60 in 2005, 65 in 2009, and is currently 67 sites.

30 EPA Office of Water, Fiscal Year 2013 National Water Program Guidance, Appendix A Measures Summary, April 2012.

6-3

EXHIBIT 6-2. PERFORAMANCE ON OCEAN DUMPING STRATEGIC PLAN MEASURE

MEASURE

Percent of Active Dredged Material Ocean Dumping Sites

that Will Have Achieved Environmentally Acceptable

Conditions (as Reflected in Each Site’s Management Plan). UNITS

FY08 FY09 FY10 FY11 FY12 FY13

Target 95 98 98 98 95 95 Percent sites Actual* 99 99 90.1 93

*Due to variability in the universe of sites, results vary from year to year (e.g., between 85 percent

and 99 percent). While this much variability is not expected every year, the results are expected to

have some change each year.)

Explanation of Results: Gulfport Western Ocean Dredged Material Disposal Sites (ODMDS) has exceeded

the minimum depth limitation. The Miami ODMDS has elevated PCB levels. In addition, multiple Gulf of

Mexico ODMDS likely do not meet environmentally acceptable conditions due to the Deep Water

Horizon Oil Spill and need to be evaluated.

Source: FY 13 Congressional Justification, page 853-854

STRENGTHS AND WEAKNESSES OF CURRENT MEASURES

Interviews with Ocean Dumping Management Program staff suggest that while the current strategic plan measure is understood in theory, different Regions have different definitions of what constitutes “environmentally acceptable conditions,” and thus the Regions do not use the measure in a consistent way. One interviewee stated that program staff have discussed what the term “environmentally acceptable conditions” means, but have reached no consensus. Another interviewee said that by definition, only sites that had achieved environmental acceptable conditions would be approved, and therefore by definition the measure should always be 100 percent. Another regional interviewee said that one state in the Region would not sign off on some site management plans (in an effort to make sure all dredge material was used for beneficial reuse) without the state’s agreement the Region could not say the sites had achieved environmentally acceptable conditions. These incompatible perspectives mean that the values reported for the strategic plan measure are not consistent or meaningful.

A review of performance on the strategic plan measure by Region shows that all but two Regions rate themselves as having 100 percent of their active dredged material ocean dumping sites that achieve environmentally acceptable conditions.31 For the two Regions that miss the target, one interviewee notes that one Region understands the issues at their sites (e.g., elevated PCB levels), and the other Region believes that environmental conditions at the ocean dumping sites are not adequately characterized. While the current measure does flag potential problems in these two Regions, it does not distinguish 31 EPA Office of Water, Fiscal Year 2013 National Water Program Guidance, Appendix E Detailed Measures Appendix, April

2012, page 12.

6-4

between the two different types of problems these Regions face. More importantly, it suggests that performance has been maximized in all the other Regions, which interviews suggest is not the case. Overall, definitions underlying the performance measures reflect different Regions’ conceptions of the measure more than variations in performance.

In addition to concerns about inconsistency in interpreting the strategic plan measure, interviewees express concern that the measure does not capture the quality of the work the program does, or the broad range of program activities conducted. Overall, it does not appear that the current measure provides feedback for staff about the effectiveness of their efforts, because staff feel they have little control over the measure.

POTENTIAL ALTERNATIVE MEASURES

Interviewees and focus group participants suggested a number of options for developing alternative performance measures, and both HQ and the Regions informally track some data in addition to the Strategic Plan Measure. Possible alternate measures for the program include outcome and output/efficiency measures, as described below.

Outcome Measures

Effective performance measures should track progress toward a goal (in addition to meeting the criteria described earlier). For the Ocean Dumping Management Program, goals include site designation and monitoring to ensure that substances dumped in the ocean do not have a negative impact on human health or the environment. Another objective is to increase the beneficial use of dredge material. Potential measures related to these goals include:

• Volume of material dumped at ocean dumping sites. Since one overall goal of the program is to minimize ocean dumping, this measure would indicate the environmental benefit of the program, if the volume of matter disposed of through ocean dumping declines over time.

• Volume and percentage of contaminated material proposed for ocean disposal that is not allowed to be ocean dumped (e.g., sent for land disposal instead). This measure speaks directly to the environmental benefit of the program, since it would show the extent to which the program prevents the dumping of contaminated matter.

• Percentage of monitored sites found to be in compliance. This measure would document whether sites are in compliance.

• Percentage of projects where material is diverted to beneficial reuse rather than ocean dumping. This measure would show whether the program is making progress in encouraging beneficial reuse for uncontaminated dredging material, which is a goal of the program.

All of the outcome measures described here may be strongly influenced by factors outside the program’s control. Thus while these measures are important to understanding progress toward the program’s goals, they must be interpreted carefully, and changes in an outcome measure should not be assumed to be cause by the Ocean

6-5

Dumping Management Program. Evaluation is a useful tool that can assess contribution of a program toward observed outcomes.32

Output/Ef f iciency Measures

In addition to outcome measures, the Ocean Dumping Management Program may wish to also include output or efficiency measures, which it can more directly control. Several options are listed below.

• Average number (and range) of years since site monitoring has occurred. This measure would document whether the program is meeting the expectation of monitoring all sites within the past 10 years, which is a key program activity. Note that some sites need to be monitored more frequently than every 10 years. In order for the measure to be comparable and meaningful, it would be necessary to define what constitutes adequate monitoring.

• Average number (and range) of years since Site Management and Monitoring Plans (SMMPs) that have been reviewed and updated. This measure would document whether the program is keeping site management plans up to date and meeting the requirements that SMMPs are reviewed and revised at least every 10 years.

• Average time to complete specific EPA activities (e.g., leading to site designation, suitability determinations, or permit reviews). This type of measure would track program efficiency, and managers would seek to reduce the measure over time to demonstrate improved efficiency. This type of measure would need to be balanced with an adequate review of quality of EPA’s work, in order to ensure there is no perverse incentive to rush the work.

• Count of EPA activities (e.g., inquiries EPA responds to, permits reviewed, etc.). This type of measure would not indicate program performance, but rather would provide context for other measures by indicating the extent to which there are ocean dumping activities that are potentially subject to EPA oversight and the volume of work EPA is completing.

CONCLUSIONS Overall, the current measure is not consistently interpreted by the Regions and does not provide particularly useful feedback for understanding program effectiveness. A valid performance measure is one that accurately represents the condition or phenomenon that it is purporting to represent. The current measure would be valid if the key term “environmentally acceptable conditions” were consistently defined.

32 In some cases, programs can attribute outcomes to a program, but this requires a higher standard of evidence, and is

difficult to do in the context of government program evaluations.

6-6

The Ocean Dumping Management Program could improve information flows by developing a suite of performance measures that reflect both outcomes related to the programs goals and outputs over which the program has direct control.

RECOMMENDATIONS

The evaluation team recommends that the Ocean Dumping Management Program:

• Strengthen information flows to inform program adaptation, building on a suite of performance measures that encompass both outcome and output measures; and

• Consider the following measures: volume of contaminated material not allowed to be ocean dumped (and of this amount, percentage that is used for beneficial uses); percentage of monitored sites found to be in compliance; average number of years since ocean disposal sites were monitored; and average number of years since Site Management and Monitoring Plans have been reviewed and updated.