EXCHANGE NETWORK RETURN ON INVESTMENT AND BUSINESS … · Ed Karmilovich Pennsylvania DEP (RCRA) Tim Lehman Pennsylvania DEP (SDWIS) ROI Model Final Report Page 3 of 131 Bill Sedlak

ROI Model Final Report

Page 1 of 131

EXCHANGE NETWORK RETURN ON INVESTMENT

AND BUSINESS PROCESS

ANALYSIS

FINAL REPORT SEPTEMBER 5, 2006

Prepared by: enfoTech & Consulting, Inc 11 Princess Road, Unit A

Lawrenceville, New Jersey 08648


Page 2 of 131

Acknowledgements The ROI Model Project team is comprised of participants from four states, ECOS, and an independent contractor. The following table lists the Project Members:

Primary Project Team State and ECOS Members Organization Michael Beaulac Michigan DEQ Molly O’Neill ECOS Kurt Rakouskas ECOS

Independent Contractor Organization Tony Jeng (Lead Consultant) enfoTech & Consulting, Inc. Brian Cerra enfoTech & Consulting, Inc. Douglas Timms enfoTech & Consulting, Inc.

Supporting States

Primary State Contact Organization Michael Beaulac Michigan DEQ (State Lead)

Deborah Sherrod Michigan DEQ (AQS) Jeffrey Jones Michigan DEQ (DMR) Ruth Borgelt Michigan DEQ (TRI) Robert Jackson Michigan DEQ (TRI) Kristen Philip Michigan DEQ (SDWIS)

Sherry Driber New Jersey DEP (State Lead) Mike Matsko New Jersey DEP (Node) Charles Pietarinen New Jersey DEP (AQS) Angela Witcher New Jersey DEP (AQS) Harry Chen New Jersey DEP (AQS) Theresa Pagodin New Jersey DEP (RCRA) Michael Gerchman New Jersey DEP (RCRA) Jim Bridgewater New Jersey DEP (RCRA/SDWIS) Phil Royer New Jersey DEP (SDWIS) Linda Sharkey New Jersey DEP (SDWIS) Russell Polo New Jersey DEP (SDWIS)

Nancy Imler Pennsylvania DEP (State Lead) Kirit Dalal Pennsylvania DEP (AQS) Jeff Miller Pennsylvania DEP (AQS) Bob Bauer Pennsylvania DEP (DMR) Troy Conrad Pennsylvania DEP (DMR) John Murtha Pennsylvania DEP (DMR) Rama Kapu Pennsylvania DEP (SDWIS) Ed Karmilovich Pennsylvania DEP (RCRA) Tim Lehman Pennsylvania DEP (SDWIS)


Page 3 of 131

Bill Sedlak Pennsylvania DEP (SDWIS) Jen Gumert Pennsylvania DEP (IT) Dudley Hackett Pennsylvania DEP (IT)

Debbie Stewart Washington ECY (State Lead) Miles Neale Washington ECY (Node) Phyllis Baas Washington ECY (AQS) Kathy Sundberg Washington ECY (AQS) Ed Bentley Washington ECY (RCRA) Dan Kruger Washington ECY (RCRA) Jean Rushing Washington ECY (RCRA) Idell Hanson Washington ECY (TRI) Joanne Phillipson Washington ECY (TRI)


Page 4 of 131

Version Control Summary Version Date Description Lead/Affiliation Change

1.0 June 28, 2006 ROI Final Report Tony Jeng, Brian Cerra & Douglas Timms

-


Page 5 of 131

Table of Contents

1 EXECUTIVE SUMMARY ................................................................................................................................3 2 INTRODUCTION ..............................................................................................................................................3

2.1 PURPOSE .......................................................................................................................................................3 2.2 BACKGROUND...............................................................................................................................................3 2.3 DATA FLOWS EXAMINED IN ROI ANALYSIS .................................................................................................3

3 OVERVIEW OF THE ROI ANALYSIS APPROACH...................................................................................3 3.1 TECHNICAL APPROACH.................................................................................................................................3

4 THE ROI MODEL .............................................................................................................................................3 4.1 OVERVIEW ....................................................................................................................................................3 4.2 BOUNDARY OF ROI ANALYSIS ......................................................................................................................3 4.3 DATA FLOW SPECIFIC ROI MODEL CHARACTERISTICS ................................................................................3

4.3.1 AQS ROI Model .......................................................................................................................................3 4.3.1.1 Business Process ............................................................................................................................................. 3 4.3.1.2 Cost Factors .................................................................................................................................................... 3

4.3.2 RCRA ROI Model ....................................................................................................................................3 4.3.2.1 Business Process ............................................................................................................................................. 3 4.3.2.2 Cost Factors .................................................................................................................................................... 3

4.3.3 SDWIS ROI Model...................................................................................................................................3 4.3.3.1 Business Process ............................................................................................................................................. 3 4.3.3.2 Cost Factors .................................................................................................................................................... 3

4.3.4 TRI ROI Model ........................................................................................................................................3 4.3.4.1 Business Process ............................................................................................................................................. 3 4.3.4.2 Cost Factors .................................................................................................................................................... 3

4.3.5 eDMR ROI Model ....................................................................................................................................3 4.3.5.1 Business Process ............................................................................................................................................. 3 4.3.5.2 Cost Factors .................................................................................................................................................... 3

4.4 POTENTIAL EXTENSIONS OF THE MODEL .......................................................................................................3 5 RESULTS ............................................................................................................................................................3

5.1 AQS..............................................................................................................................................................3 5.1.1 Michigan..................................................................................................................................................3 5.1.2 New Jersey...............................................................................................................................................3 5.1.3 Pennsylvania............................................................................................................................................3 5.1.4 Washington ..............................................................................................................................................3 5.1.5 Summary ..................................................................................................................................................3

5.2 RCRA...........................................................................................................................................................3 5.2.1 New Jersey...............................................................................................................................................3 5.2.2 Pennsylvania............................................................................................................................................3 5.2.3 Washington ..............................................................................................................................................3 5.2.4 Summary ..................................................................................................................................................3

5.3 SDWIS .........................................................................................................................................................3 5.3.1 Michigan..................................................................................................................................................3 5.3.2 New Jersey...............................................................................................................................................3 5.3.3 Pennsylvania............................................................................................................................................3


Page 6 of 131

5.3.4 Summary ..................................................................................................................................................3 5.4 TRI ...............................................................................................................................................................3

5.4.1 Michigan..................................................................................................................................................3 5.4.2 Washington ..............................................................................................................................................3 5.4.3 Summary ..................................................................................................................................................3

5.5 EDMR...........................................................................................................................................................3 5.5.1 Michigan..................................................................................................................................................3 5.5.2 Pennsylvania............................................................................................................................................3 5.5.3 Summary ..................................................................................................................................................3

5.6 CONCLUSIONS...............................................................................................................................................3 6 STATE ANALYSIS AND CONCLUSIONS ....................................................................................................3

6.1 STATE TOTALS..............................................................................................................................................3 6.2 THE NETWORK IMPACT.................................................................................................................................3

6.2.1 Michigan..................................................................................................................................................3 6.2.2 New Jersey...............................................................................................................................................3 6.2.3 Pennsylvania............................................................................................................................................3 6.2.4 Washington ..............................................................................................................................................3

6.3 QUALITATIVE BENEFITS: ..............................................................................................................................3 6.4 OVERALL SUMMARY ....................................................................................................................................3

7 APPENDICES.....................................................................................................................................................3 7.1 APPENDIX A: SUMMARY DATA.....................................................................................................................3

7.1.1 AQS..........................................................................................................................................................3 7.1.2 RCRA .......................................................................................................................................................3 7.1.3 SDWIS......................................................................................................................................................3 7.1.4 TRI ...........................................................................................................................................................3 7.1.5 eDMR.......................................................................................................................................................3

7.2 APPENDIX B: RAW DATA..............................................................................................................................3 7.2.1 AQS..........................................................................................................................................................3 7.2.2 RCRA .......................................................................................................................................................3 7.2.3 SDWIS......................................................................................................................................................3 7.2.4 TRI ...........................................................................................................................................................3 7.2.5 eDMR.......................................................................................................................................................3

7.3 APPENDIX C: ANTICIPATED (AND EXISTING) DATA FLOWS IMPLEMENTED AT EACH STATE NODE...............3


Page 7 of 131

1 Executive Summary The Exchange Network Return on Investment (ROI) and Business Process Analysis Project funded by the Environmental Council of the States (ECOS) was conducted to better understand the effects Exchange Network Technologies have on the quality and efficiency of environmental data exchanges for states, tribes and local agencies. As a result of this project, a Return on Investment Model was developed to quantify the savings associated with the implementation of Exchange Network related projects. The Project Team selected five data flows to include in the ROI analysis. These flows are:

• Air Quality System (AQS) • Resource Conservation and Recovery Act (RCRA) • Safe Drinking Water Information System (SDWIS) • Toxics Release Inventory (TRI) • Electronic Discharge Monitoring Report (eDMR)

The first four data flows were selected for the following reasons:

(1) The U.S. EPA planned to model its internal ROI from implementing these same data flows; (2) Many of these data flows are already in production in several states; and (3) Many states and tribes are prioritizing these flows for implementation over the next few years.

The fifth flow, eDMR, was chosen to ensure that the ROI model could be extended out to capture eGovernment (industry to state) related activities that are directly tied to the National Environmental Information Exchange Network (Exchange Network). Four state agencies participated in the development of this study: the Michigan Department of Environmental Quality, the New Jersey Department of Environmental Protection, the Pennsylvania Department of Environmental Protection, and the Washington Department of Ecology. Each agency selected 3-4 data flows to be analyzed out of five total flows that were included in the ROI analysis. Site visits were conducted at each of the participating states. At each of these visits, individual meetings were conducted with the Subject Matter Experts (SMEs) from each of the data flow program areas in which the state was participating. The ROI analysis consisted of five major steps:

1. Business process analysis: Because a state’s business process supporting a particular data flow may be unique, the overall business process for each data flow was examined in detail at each state. The analysis included every step in the data flow, from the facility or laboratory through to a state agency submission to the U.S. EPA.

2. Identification of cost factors: Once the Business Process was defined, each step in the process was characterized as a potential cost factor for Return on Investment analysis.

3. Quantification of each cost factor: Each cost factor was then studied to determine the exact input parameters that would define the cost of the factor. Equations were then developed to describe each cost factor based on these input parameters. Cost Factor values were found for each cost factor both before and after implementation of an Exchange Network project.

4. Calculation of the ROI: Once the data was collected for operational scenarios before and after the implementation of Exchange Network technologies, the net savings (or loss) was calculated to determine the Return on Investment.

5. Qualitative assessment: For any benefit that could not be quantified in step 3, a qualitative description of the benefit was captured.

These steps above were conducted not only to determine the ROI values for specific flows within the participating states, but to also develop a Return on Investment Tool that can be used by other states or


Page 8 of 131

tribes to analyze data flows within their agency. This can be used as a screening tool, as it incorporates average data compiled during this project in order for a user to perform a quick estimate of the potential return for implementing the data flow in their particular state or tribe. The model is designed to provide an estimate with only a very minimal amount of state- or tribal-specific information. Default values for each input parameter are supplied in the model and will be scaled up or down based on a user’s response to a few key questions. If a user would like a more accurate analysis, the default values can be substituted with actual state-specific values and the ROI values will be calculated automatically from this data. Although this report will only included analyses from the five data flows mentioned earlier, the five steps described above can be used as a standard approach for assessing the Return on Investment for any data flow. The ROI analysis can be very valuable before implementation of an Exchange Network project. By analyzing each cost factor, those aspects of the business process that are in most need of improvement can be determined. The project can then be tailored to maximize the impact of the project. Overall the results from the study show a positive return for most of the data flows analyzed. For the flows with a larger volume of data being exchanged (SDWIS and eDMR) there were considerable savings due to the implementation of a data collection server to receive data electronically from laboratories or facilities. These savings were the result of the elimination of the costs associated with a paper-intensive data flow, including mailing costs and dual data entry costs. The AQS and RCRA data flows had varying results depending on the pre-implementation business process that each state was using to flow the data. The project team determined that the implementation of multiple flows increased the return realized by each state. This is because many of the implementation costs (such as IT infrastructure development) and subsequent maintenance could be spread across multiple flows. This allows subsequent data flows to be implemented by leveraging the infrastructure changes made while implementing initial data flows. In the table below are the total results for multiple flows found at each state: Michigan New Jersey Pennsylvania Washington Data Flows Examined AQS

eDMR SDWIS TRI

AQS RCRA SDWIS

AQS eDMR RCRA SDWIS

AQS RCRA TRI

Total Annual Operational Cost Pre-EN ($/year) $1,622,120 $1,266,519 $1,352,807 $311,560 Total Annual Operational Cost Post-EN ($/year)1 $834,407 $672,789 $710,181 $272,294 Initial Capital Investment ($)2 $1,095,940 $845,233 $1,002,390 $139,422 Post-EN Annual Maintenance Cost ($/yr) $107,983 $75,831 $105,609 $19,014 Post-EN Annual State Savings ($/year) 3 $679,730 $535,192 $537,017 $20,252 ROI Summary Average ROI (5 year basis) 62% 61% 54% 15% Payback period (years) 1.6 1.6 1.9 6.9 This table shows the total cost to operate all of the data flows implemented by each state (of the five that were included in this study) both before and after the implementation of Exchange Network technologies.

1 These values are the total annual operational costs for all examined flows within each state, based on a five year average of the Post Exchange Network operational costs. 2 Total implementation cost for all data flows that were included for each state in this study. 3 These values are average annual savings based on 5-year post Exchange Network scenario


Page 9 of 131

By dividing the net savings of the project by the initial investment, a Return on Investment percentage was determined. As can be seen from these summary results, all states participating experienced a positive return on their investment in Exchange Network technologies to flow data. Implementation of additional flows would likely improve these individual state ROI values.


Page 10 of 131

2 Introduction 2.1 Purpose The purpose of this study is to analyze the effects of the implementation of Exchange Network technologies have on different environmental data exchanges. This analysis includes an in-depth review of the four participating states’ specific business processes for up to five different data flows. This review compares the business processes for each data flow before and after the implementation of Exchange Network technologies in order to estimate the total cost savings as a result of the implementation. A Return on Investment Model is then applied to all of the data flows. The results of these analyses allowed for the development of a generic ROI Model for other, non-participating agencies to use. This tool can allow users to develop the economic incentive and implementation strategies for investing in the Exchange Network infrastructure and technologies.

2.2 Background The business of managing and protecting the natural environment has become highly information intensive. Data on air, water, and waste are routinely exchanged among regulated facilities, states, tribes and the U.S. Environmental Protection Agency (EPA). Today, most environmental data is stored in electronic data management systems. These systems are often incompatible or structured differently from one another, even within the same regulatory agency. The proliferation of heterogeneous data systems has resulted in complex and often burdensome approaches to exchanging environmental data. Business processes historically utilized manual data entry, email, or batch file processing to move data from facilities to state and federal regulatory agencies. In the late 1990’s, state environmental agencies and the U.S. EPA formed the Information Management Workgroup (IMWG) to collaborate on improving the management of environmental data. Recognizing that existing methods of exchanging data had become inefficient, burdensome and costly, the IMWG initiated plans for a new model for data exchange. The result was the National Environmental Information Exchange Network (now known simply as the Exchange Network —a secure Internet- and standards-based approach for exchanging environmental data. Using universally applicable technologies such as eXtensible Markup Language (XML), Nodes and Web services, States, the U.S. EPA and regulated facilities are using the Exchange Network to increase access to environmental data and make the exchange of data more efficient.

2.3 Data Flows Examined in ROI Analysis In the context of this ROI project, a data flow is defined as a collection of one or more exchanges of information between either environmental agencies or with the regulated community, to support the information management of a particular segment of an environmental regulation. Based on the limited scope of this ROI project, the team selected five data flows. The selected data flow needed to be one which is affected by the implementation of an Exchange Network project and which multiple states (especially one or more of the five project states) have or are currently implementing, in order to collect sufficient ROI data and to be relevant for an Exchange Network ROI analysis. Based on these criteria, the data flows examined were:

• Air Quality System (AQS) – This data flow focuses on the State’s primary data collection and data exchange of ambient air quality data as required by the Clean Air Act. This includes the


Page 11 of 131

collection of ambient air quality data at the State level and its exchange to the U.S. EPA’s national AQS system. AQS is a national database that contains ambient air pollution data and meteorological data, collected by the U.S. EPA, state, local and tribal air pollution control agencies from thousands of monitoring stations.

• Resource Conservation and Recovery Act (RCRA) – RCRAInfo is the national database containing hazardous waste data collected from delegated States. This data flow focuses on the collection of Waste Handler, Permitting, Compliance Monitoring & Enforcement (CME), Corrective Action, and Waste Activity information at the state level and its exchange to the U.S. EPA’s Office of Solid Waste (OSW), which regulates all solid and hazardous waste under the RCRA.

• Safe Drinking Water Information System (SDWIS) – This data flow focuses on the State’s primary data collection and data exchange activities to support their management of the Safe Drinking Water Act. This includes the collection of various safe drinking water analytical reports from laboratories and water systems, the collection of operating reports from water systems, and the exchange of sampling, compliance, and enforcement information from the state to the U.S. EPA’s national SDWIS system,

• Toxics Release Inventory (TRI) – This data flow involves the collection of annual TRI reports from regulated facilities by states and the U.S. EPA, and the recent business process change introduced by the Exchange Network allowing facilities to submit data electronically to the U.S. EPA. It is then sent automatically to states; this relieves facilities of the burden of duplicate submissions and relieves the States of the burden of data entry and/or processing diskette submissions. .

• Electronic Discharge Monitoring Report (eDMR) – This data flow focuses on the State’s primary data collection and data exchange activities to support their management of the National Pollution Discharge Elimination System (NPDES) program. This program regulates point sources that discharge pollutants into waters of the United States. The data flow includes the collection of Discharge Monitoring Reports (DMRs) from regulated facilities at the State level and the exchange of this information, along with permitting, compliance, and enforcement information to the U.S. EPA’s national Permit Compliance System (PCS).

The first four data flows were selected for the following reasons: (1) The U.S. EPA planned to model its internal ROI from implementing these same data flows; (2) many of these data flows are already in production in several of the participating state agencies; and (3) many states and tribes are prioritizing these flows for implementation over the next few years. The fifth flow, eDMR, was chosen to ensure that the ROI model could be extended to capture eGovernment (industry to state) related activities that are directly tied to the National Environmental Information Exchange Network (Exchange Network).


Page 12 of 131

3 Overview of the ROI Analysis Approach 3.1 Technical Approach The technical approach used to develop the ROI model is described in the diagram below:

Business Process Analysis

ROI Research

Cost Factors

Initial ROI Model

Develop a diagram of each data flow’s business processIdentify the cost factors of each step in each process

For each Cost Factor found in the Business Process Diagram, determine the following:

Input ParametersDefault ValuesAssumptions

Research other ROI ModelsDetermined that MI Evaluator would be a suitable basis for the development of the ROI Model to be developed in this project

Incorporate the Cost Factors (including all input parameters and default values) into the modelDevelop a Financial ModelDevelop calculations

ROIPayback PeriodTotal Cost of Ownership

State Interviews

Evaluate Business ProcessesInvestigate missing Cost FactorsCollect actual state dataRefine default value assumptionsCollection of Qualitative Factors

1

2

Final ROI Model

Revise Initial Model based on site visits

Add/remove cost factorsIncrease usabilityChange analysis boundary

3

4

5

6

7

Business Process Analysis To begin the development of the initial ROI Model, the business process for each data flow was first diagrammed to clearly identify each step in the process and all associated cost factors. A cost factor is any step in the business process that has the potential to result in a quantitative cost, adding to the total operational cost of the data flow. This business process analysis was used in brainstorming exercises during the state site visits with program staff to ensure that all potential cost factors were identified. This analysis also established the boundary that would be used to describe each data flow, by determining which cost factors were part of the data flow, and which cost factors were independent to the data flow implementation.

Cost Factors Once the cost factors were identified in the business process analysis, each factor was deconstructed to determine what influenced the factor. Input parameters were defined for each cost factor as a means of quantifying the factor. Once each input parameter was created, an initial “first guess” default value was assigned to it based on high-level discussions with staff knowledgeable of that data flow process. These initial default values served as a starting point for the site visit discussions and were later modified based on more detailed input from the state program area experts. The


Page 13 of 131

modified defaults would later be used in the Final ROI model for a user to perform a very quick analysis on a potential data flow implementation for screening purposes. Below is an example cost factor from the eDMR data flow with the required input parameters needed to quantify the factor:

Cost Factor Name DMR Data entry cost (by state) Cost Factor Description The cost incurred by the state to have state staff manually

enter DMR data (daily and/or summary) into a state database

Input Parameters • Hourly rate of state staff ($/hour) • Number of DMRs entered each year (DMRs/year) • Time spent to enter each summary DMR (hours/DMR) • Time spent to enter each daily DMR (hours/DMR) • Accuracy of data entry (% entered correctly)

Note: Throughout this analysis it will be assumed that any cost relating to staff salary will be a fully-loaded cost rate for the personnel to include fringe/indirect employer costs. Each state had particular methods for estimating this cost.

ROI Research The team conducted research into previously established ROI techniques used by states and also coordinated with a similar U.S. EPA effort running in parallel with this project. As part of this research, the team reviewed Michigan’s “MI Evaluator” tool. During the ROI research phase, the team concluded that the MI Evaluator tool would be a suitable basis for the ROI model developed in this project. This tool factored in the total project benefit, project maintenance cost, and project implementation cost to calculate a Return on Investment. This tool also calculated the Payback Period and Total Cost of Ownership (both of which were used in the ROI model for this project). While providing a good foundation, the model could not be easily applied to Exchange Network data flows. The tool was database-application-development focused, was not created to model data exchanges and, accordingly, did not consider costs and benefits to non-state participants (i.e., permitted facilities, labs, etc.) in data exchange processes.

Initial ROI Model An initial ROI model was then developed for each data flow, based on the previous business process analysis and the ROI research. The ROI model takes all of the cost factors identified in each business process and generates a dollar value based on certain input parameters. Once all of the cost factors are quantified, these values along with a financial model are used as the basis for the ROI calculation.

State Interviews Next, the team conducted site visits in order to collect the cost data that would populate the ROI model and determine if there were any cost factors or potential operational scenarios that were missing from the initial analysis. These visits were used to improve the default value assumptions, determine if there were any cost factors that were missing from the initial analysis, and to understand different potential ‘operational scenarios’ depending on a state’s own unique circumstances. The team created a survey form for each participating state to complete. This survey was divided into three sections:


Page 14 of 131

• Part 1: Business Process Analysis. This section was used to determine the exact business process for each data flow in each state. Every state has different variations for flowing data from one source to another. These variations can greatly affect the way the Cost Factors are calculated. States were asked to verify the initial, more generic business process that was used to generate the list of cost factors and to add or remove any cost factors (business process steps) that did not apply to the operational scenario in their state.

• Part 2: Assessment of State Costs. In this section, states were asked to provide costs for each individual parameter that made up each cost factor. These responses were used to quantify the total operational cost before and after implementation of Exchange Network technologies, as well as the cost required to implement and maintain those technologies. States provided actual costs when available and estimates in cases where a data flow had yet to be implemented.

• Part 3: Qualitative Factors Survey. This section helped identify the qualitative factors associated with the implementation of an EN project. These factors are sometimes difficult to quantify in dollars, but still provide a benefit to the project.

This survey was distributed to the Subject Matter Experts at each state. During the site visits, each data flow was analyzed and the responses to the surveys were discussed during each state’s site visit.

Iterative follow-up with Participating States After the site visits were concluded, all of the data was complied for review by the participating states. State subject matter experts filled in any data gaps and responded to the team’s follow up questions.

Final ROI Model After collecting all of the data, the team created the Final ROI Model. This model incorporated the lessons learned during the site visits and subsequent follow-up activities. A final list of cost factors was established for each data flow and incorporated into a data flow specific ROI Model. The information gained from each state was used to populate the Final ROI Model with default cost data. Users of the model can easily scale these default values up or down by entering a small set of key information about their particular business processes. This will allow other Network partners to perform an approximation of their expected ROI with relatively little effort. To produce a more refined and customized ROI analysis, users can easily edit the cost data specific to their organization’s situation.


Page 15 of 131

4 The ROI Model 4.1 Overview The Final ROI Model was developed as a tool to estimate the return on investment for five particular Exchange Network data flows.

Key Questions

NPDES Permittees 1200Project development duration (Years)

1Key values for ROI Analysis

1 Operational Scenario

Choose facility DMR submission method

Paper

Choose DMR types: BothChoose DMR tracking method:

State database and PCS

Business Scenario Questions:

2

Financial Model5

% Participation by facilities (a projected

goal)

% Project spending (% of total

implementation Cost)Year 0 0% (Development) 50%Year 1 0% (Development) 50%Year 2 25% (Production Year)Year 3 37%Year 4 50%Year 5 60%Year 6 70%

Default Values

Facility to State Cost Factors:(A4) Data entry cost (by state)

Pre - EN Post - EN

Hourly rate of state staff ($/hour)

28.5 0

Number of Permittees 1200 1200Frequency of submissions 12 12# of DMRs/year (Data entered)

14400 14400

Time spent to enter each DMR (hours/DMR)

0.29 0.00

Time spent to enter each monthly DMR

0.87 0.00

Accuracy of data entry (% entered correctly)

100% 100%

SUBTOTAL COST ($/year)

$474,240.00 $0.00

State to EPA Cost Factors:(5) Submission to EPA

Pre - EN Post - EN

Hourly rate of state staff ($/hour)

28.5 37.5

Average time spent preparing submissions (hours/month)

0 0

Number of DMRs entered 14400 0Time to enter each DMR into PCS

0.29 0

Time spent to fix submission problems (hours/year)

N/A 60

SUBTOTAL COST ($/year)

$118,560.00 $2,250.00

4

3

Add to all other facility to statecost factors

Add to all other state to EPAcost factors

ROI Calculations

Year 2 $446,475Year 3 $379,781Year 4 $307,530Year 5 $251,952Year 6 $196,3745 year

average $316,422

Annual Facility to State operational Cost (Post-EN)

Implementation Cost6

7

Average Annual Operational Cost Pre-EN ($/year) (5 year basis) $703,980Average Annual Operational Cost Post-EN ($/year) (5 year basis) $318,672Initial Capital Investment ($) $506,850Annual Maintenance Cost ($/year) $50,685Annual Savings ($/year) $334,623ROI Summary

Average ROI (5 year basis) 66%Payback period (years) 2.1Expected % Participation after 5 years 70%% Participation needed for positive ROI 0%

Facility to StateSUBTOTAL COST $270,850.00

State to EPATOTAL NODE COST $240,000Total # of flows implemented on the Node within first 3 years (estimate)

10

Shared Node Cost $24,000Implementation $212,000

SUBTOTAL COST $236,000.00TOTAL IMPLEMENTATION COST $506,850.00

Pre - EN Post - ENTotal cost from Facility to State

($/year)$585,420 $29,640

Pre - EN Post - ENTotal cost from Facility to State

($/year)$118,560 $2,250

Annual State to EPA operational Cost (Post-EN)

$2,250


Page 16 of 131

By calculating the total operational cost before and after the implementation of an Exchange Network project, the model can compare the total savings and be compared to the total project implementation (and maintenance) cost. In the diagram above, the final ROI Model is described in detail. The example values in the diagram come from an example eDMR Model.

Key questions Users first need to answer a few “key” questions to be used as a basis for scaling the default values for each of the Cost Factors. Typically these key factors relate to the amount of data being processed. The values provided for these key questions are then linked to each of the cost factors, and will either positively or negatively affect the cost factor’s value. Listed below are the key questions that have been used for each data flow:

Key Questions AQS eDMR RCRA SDWIS TRI 1. # of Monitoring

sites 2. Average # of

parameters per site

3. Frequency of Submissions to the U.S. EPA

4. Project development duration

1. # of NPDES Permittees


1. # of facilities 2. # of Handler

Notifications 3. # of CME Reports 4. # of Permitting

Reports 5. # of Corrective

Action Reports 6. # of Waste Activity

Reports 7. Frequency of

reporting to the U.S. EPA


1. Number of Water Systems

2. Total # of analysis reports sent to state/year

3. Total # of sample results sent to state/year

4. Total # of MORs sent to state/year


1. # of facilities 2. Total # of

Form R 3. Total # of

Disks submitted

4. Total # of Paper Form R submissions


Operational scenario As part of the Business Process Analysis, the project team observed that each cost factor can be interpreted differently depending on a state’s unique business process for managing a particular program. For example, some states will accept electronic submissions of data from the regulated facilities, while other states may rely on paper submissions. For each data flow there were certain operational scenarios that greatly affect how the cost factors are quantified. Below are the operational scenario questions posed by the model for each data flow:

Operational Scenario Questions AQS eDMR RCRA (repeated

for each module) SDWIS TRI

1. What is the submission method of lab data to the state? (Paper, electronic,

1. What is the submission method of DMR data to the state? (Paper, electronic, or both)

2. What types of DMRs are

1. Is this module entered into a state database?

2. What is the data entry method? (RCRAInfo as primary source,

1. Are MORs tracked by the state?

2. What is the submission method of MOR data to the state? (Paper, electronic, or both)

3. Is lab analysis data

1. Does the state track TRI data received?


Page 17 of 131

Operational Scenario Questions AQS eDMR RCRA (repeated

for each module) SDWIS TRI

or both) processed (daily, monthly, or both)

3. What is the tracking method for the DMRs (Direct PCS user, date entered into a state database, or both)

4. Will this project include the implementation of a data collection server?

State as primary source, Dual entry)

tracked by the state? 4. What is the submission

method of lab analysis data to the state? (Paper, electronic, or both)

5. What type of database is used by the state? (SDWIS-State, or a state database)

6. Will this project include the implementation of a data collection server?

Determination of Cost Factor Values Once the key questions and operational scenario questions have been answered, the ROI Model calculates default values for each of the components of each cost factor. These values are to be considered rough estimates to be used for screening purposes. If a user would like a more accurate estimate, then actual state values can be plugged in for each input parameter. The Cost Factor values developed for each of the participating states in this project were calculated based on the responses from each state survey and visit. Each input parameter was compared to all the responses received from the states and it was determined if the Key Questions or Operational Scenarios had any affect on the differences between states. If there were correlations that could be made, the input parameter would be dependent on the values supplied in the first section of the model. If that was not the case, a constant was estimated. To determine constants, the number of comparable responses was first counted. If only two states supplied an answer to a particular input parameter, then the constant was set to the average of the two responses. If there were more than two responses, both the median and average were compared. If the responses were similar, the average was taken; if responses were not similar, a reasonable value between the average and median was taken instead.

Sub-flow division The ROI model has divided each data flow into two primary sub-flows: (1) Facility/Lab to State data flow and (2) State to U.S. EPA data flow. This allows users to independently analyze each section of the flow. Some of the data flows also included an analysis of a State to “Exchange Partner” data flow. An Exchange Partner is someone who would request data from the state for a non-regulatory purpose (for example, FOIA data requests).

Financial Model It is important to note that implementing an Exchange Network project does not happen immediately. The project length may stretch over multiple years. When the Exchange Network is implemented in data flows that rely on information from data suppliers such as laboratories or regulated facilities, participation rates will not reach 100% as soon as the project implementation is complete. These groups will require time to transition to a new way of doing business. This time lag may greatly affect the pay back period of the project because a Network partner may not recognize the full


Page 18 of 131

potential of the project until a larger percentage of data suppliers are participating in the new process. For example, in the eDMR and SDWIS flows, the operational cost of moving data from labs or facilities to the state depends on the number of labs or facilities making electronic versus paper submissions. The more facilities and labs that participate in the electronic flow, the greater impact this project will have. The Financial Model embedded in the ROI Model asks the user to estimate these factors to account for their influence. The two questions in the Financial Model are:

1. Percent of total project cost spent per year. 2. Percent participation. This is the annual percentage of data suppliers that have shifted to a

reporting method that is compatible with the implementation of Exchange Network technologies.

The user is asked to estimate this participation percentage for the first five years after the data flow was put into production. These estimates will impact the summary table in the first section of the Model Calculation Worksheet. See the example below: The estimates provided by the user:

% Participation by facilities (a projected

goal)

% Project spending (% of total implementation

Cost) Year 0 0% (Development) 50% Year 1 0% (Development) 50%

Year 2 25% (Production

Year) Year 3 37% Year 4 50% Year 5 60% Year 6 70%

In the ROI Model, the timeline of events is defined as follows:

• Year 0: Represents the (pre-implementation) operational scenario at the exact point in time before the Exchange Network project began.

• Year 1: Represents the (pre-implementation) first year after work on the project began. • Year 1 plus the Project implementation time (Year 2 in the above example): This is the

year that the project moved into the production phase. This year and all that follow are considered to be operating “Post – implementation.”

The average annual operational cost post-implementation follows the following calculation method:


Page 19 of 131

Facility to state State to EPA

Pre- Implementation $585,420 $118,560

Post - Implementation $29,640 $2,250

State to EPA

Participating Cost NON participating Cost TOTAL TOTAL TOTAL COSTYear 1

(Pre -Implementation) 0% (Development) 100% $585,420 $118,560 $703,980

Year 2 25% 75% $7,410 $439,065 $446,475 $2,250 $448,725

Year 3 37% 63% $10,967 $368,815 $379,781 $2,250 $382,031

Year 4 50% 50% $14,820 $292,710 $307,530 $2,250 $309,780

Year 5 60% 40% $17,784 $234,168 $251,952 $2,250 $254,202

Year 6 70% 30% $20,748 $175,626 $196,374 $2,250 $198,624

AVERAGE Annual Cost Post Implementation: $318,672

$585,420

Facility to state

Operational Cost

% Participating facilities

% NON-Participating facilities

Facility to state: 1. Calculate the Participating facility cost. This is the total Post-Implementation cost for the

facility to state sub-flow times the percentage of facilities participating • Example: Year 2: $29,640 x 0.25 = $7,410

2. Calculate the Non-participating facility cost. This is the total Pre-Implementation cost for the facility to state sub-flow times the percentage of facilities not participating

• Example: Year 2: $585,420 x 0.75 = $436,065 State to U.S. EPA: 3. The State-to-U.S. EPA cost does not depend on the participation rate; it is just the Post-

Implementation cost for the state-to-U.S. EPA sub-flow. Average Operational Cost Post Implementation: 4. Calculate the annual operational cost for each year (sum of steps 1-3), then take the average

over the five year period. Applying the Financial Model estimates to the Cost Calculations shows that as the participation of data suppliers increases, the annual benefit increases.

Implementation Costs All data flows in the ROI model will include the cost to implement a state-to-U.S. EPA submission of data. This implementation cost will account for the cost incurred by the state to implement an Exchange Network Node and the particular data flow using the Node. There is also an associated implementation cost for the facility/lab-to-state sub-flow for the eDMR and SDWIS data flows. This implementation accounts for a data collection server (and associated accoutrements) to collect data electronically from the facilities or labs and store it electronically in the state database. For the state to U.S. EPA or state to Exchange Partner sub-flows, the implementation cost is for a Node-to-Node data flow. This cost takes into account the cost of the Node, its associated hardware, and the cost to implement the data flow. Any implementation cost that was shared by more than one data flow was divided by the number of shared flows a state either has or plans to implement. Because a state Node will most likely support the flow of multiple distinct data exchanges, the total Node cost is first calculated, and then divided by the total numbers of data flows that it will support. This “shared” cost is then applied as the


Page 20 of 131

shared Node cost in the implementation cost section of the ROI Model. In the analysis, states are asked to estimate the total number of expected data flows to be implemented using their Node at the end of a three-year time period; this is the number by which the total Node cost is divided. The flow implementation costs are divided or shared among the total number of existing or planned flows to prevent one data flow is not unfairly bearing the total Node implementation costs.

ROI Calculations

A. For data flows that are affected by the participation rate of data suppliers (eDMR and SDWIS): Based on the responses in the Model Input section, three important values are generated in the Model Calculation Worksheet:

Annual Operational Cost before Exchange Network implementation

Implementation Cost (Project Investment)

Annual Operational Cost after Exchange Network implementation

The Financial Model incorporates these values and generates the following chart (this is an example using the eDMR data flow):


Page 21 of 131

As displayed in the chart above, as the amount of participating facilities increases, the “TOTAL Annual Data flow Cost After EN Project” decreases (approaches the “Annual Post-Project Cost” as one approaches 100% participation). The Model uses the data in this chart to calculate the following values for use in the Payback Period and ROI calculations:

• Annual Operational Savings: This is the net difference between the Total Annual Data flow

Cost Before Implementation and After Implementation.

• Cumulative Project Capital Investment: This is the running total of the cost to implement the Exchange Network Project. It is equal to the total spent in all previous years plus the amount spent in the current year.

• Cumulative Balance of Combined Investment and Savings: This value is used to calculate the

Payback Period and the ROI. To determine this value, the program uses the following equation:

If Year “n” is less than the first production year (Year 3 above): (Cumulative Balance)Year n = (Cumulative Project Capital Investment)Year n If Year “n” is greater than or equal to the first production year (Year 3 above): (Cumulative Balance)Year n = (Cumulative Balance)Year n-1 + (Annual Operational Savings)Year n

This is summarized over a seven-year period in the table below:

This can also be shown graphically (see below) to illustrate the payback period of the Exchange Network Implementation Project:

ROI Analysis

-$1,000,000-$500,000

$0$500,000

$1,000,000$1,500,000$2,000,000$2,500,000$3,000,000$3,500,000

0 1 2 3 4 5 6 7

Years

Operational Savings ($/year)

Cumulative Project Investment ($)

Cumulative Balance of CombinedInvestment and Savings ($)Dataflow in production

Payback

Payback

The ROI Model uses linear extrapolation to estimate where the “Cumulative Balance of Combined


Page 22 of 131

Investment and Savings” curve (blue curve in above graph) crosses the x-axis. This is the point where the project begins to turn a profit, also known as the Payback Period.

The Payback Period is defined as the time it takes for the investment to begin to show a profit. As can be seen in the graph above, this happens just after Year 2. Because the first year was spent implementing the data flow, the actual Payback period is 1.3 years. The ROI calculation also takes into account the participation level of the facilities. Because the Annual Project Benefit is affected by the % of participating facilities, a ROI value is calculated for each year after production by the following equation:

100Investment Initial

Benefit)Project Annual Total( xROI =

Where: Total Annual Project Benefit = (TOTAL Annual Data flow Cost Before EN Project) –

(TOTAL Annual Data flow Cost After EN Project)

The ROI is presented in a table that utilizes the data from above:

Notice again that as the participation level increases, the ROI also increases.

B. For data flows unaffected by the participation rate of data suppliers (AQS, RCRA and TRI): For data flows with a constant post implementation operational cost, the ROI calculations are much simpler. The only values needed are the following:

a. Annual operational cost before implementation b. Annual operational cost after implementation c. Annual maintenance cost d. Implementation cost

ROI Calculations (using the above values):

dcbaROI −−=


Page 23 of 131

ROIPeriodPayback 1=

Qualitative Analysis Not included in the diagram above, but very important to the ROI analysis, is the identification of the qualitative benefits that are the result of implementing an Exchange Network project. Some data flows may not have a high financial return on investment; however, there may be some very valuable benefits that cannot be quantified. As part of the site visits, states were surveyed to determine what these qualitative benefits were. The most common responses included improved data quality and improved data timeliness. These qualitative factors are described in detail for each data flow in Section 5.

4.2 Boundary of ROI analysis When each data flow was examined, the Team discovered that the boundaries of analysis were unique to each flow. Some implementation projects included major business process changes for how the state received data from a facility or laboratory, others only included a business process change for how the state submitted data to the U.S. EPA. The ROI analysis also included the costs associated with sharing data with Exchange Partners. These Partners may include other states, local agencies, universities, or environmental groups. These boundary differences are illustrated in the diagram below:

Nod

eIm

plem

enta

tion

Dat

a C

olle

ctio

n S

erve

rIm

plem

enta

tion


Page 24 of 131

4.3 Data Flow Specific ROI Model Characteristics Each data flow has unique characteristics for its ROI analysis. These unique characteristics are tied to the unique business process for each data flow. The following sections describe the ROI Model for each of the five investigated data flows:

4.3.1 AQS ROI Model The Air Quality System (AQS) contains ambient air pollution data collected by the U.S. EPA, state, local and tribal air pollution control agencies from thousands of monitoring stations. AQS also contains meteorological data, descriptive information about each monitoring station (including its geographic location and operator), and data quality assurance/quality control information. The U.S. EPA’s Office of Air Quality Planning and Standards (OAQPS) and other AQS users rely upon the system data to assess air quality, assist in Attainment/Non-Attainment designations, evaluate State Implementation Plans for Non-Attainment Areas, perform modeling for permit review analysis, and other air quality management functions. AQS information is also used to prepare reports for Congress as mandated by the Clean Air Act. Before the Exchange Network, State, Local and Tribal agencies used a manual internet application to submit their data to the U.S. EPA’s Central Data Exchange (CDX). Next, they use an application (provided by AQS) to move the data from CDX to AQS, perform Statistical and Critical Review on the data, and post the data directly into AQS’s production database. There are two main sources of data:

1. Continuous Monitor Data: Data that is automatically collected by data loggers and transmitted electronically to the state.

2. Laboratory Data: Data that must be analyzed by a laboratory. The results are then sent to the state and in some cases submitted directly by the lab to the U.S. EPA.

4.3.1.1 Business Process Below are two diagrams of generic business processes for the AQS data flow for both Pre- and Post-Exchange Network Implementation. All steps labeled with a number have been included in the ROI analysis. Those steps not included represent costs incurred by the laboratory or monitoring station. Pre-Exchange Network:


Page 25 of 131

State

USEPA

AQS

Exchange Partner

6

Air Program Staff

Air Program Staff

Exchange Partner

Staff

5`7

AQSDatabase

Monitor

(1) State Laboratory

Data Logger

Automatically sent to the state

Lab Staff

`

Submissionto EPA

Automatically update the state

database

My CDX

Validate

3AQSFile

2

(2) Federal Contract Lab

Lab Staff

`

StateDatabase

Courtesy copy

Send electronicfile to the state

(Excel, PDR, etc.)

1

Air Program Staff

Already inAQS format

4

Air Program Staff

Lab to State (not included in ROI analysis)4 There are two types of air quality data submitted to the state: (1) Continuous data and (2) Non-continuous data. Continuous data is submitted automatically to the state from data loggers that are spread across the state (typically via phone lines). The non-continuous data needs to be analyzed by a laboratory. This information can be submitted to the state in several ways depending on the type of lab that is performing the analysis:

1. State Laboratory: If a state lab performs the analysis, the results are sent to the state for submission to AQS. This data can be sent to the state in one of two methods:

a. The state lab may have direct access to the state database, allowing them to automatically add the data directly to the database

b. The lab may instead send the state data in an electronic format (typically Excel, Access, or PDF)

2. Federal Contract Lab: States often contract with federal labs to perform analysis on certain pollutants. Under these contracts the lab will provide a courtesy copy of the data to the state for review before they submit the data (once approved) on behalf of the state to AQS.

State processing

Data Entry: Once all of the non-continuous data is submitted to the state, it will need to be incorporated into the state database. This step is not always performed, some states will keep the non-continuous data separate and store it in the format in which it was submitted by the laboratory.

4 These cost factors were analyzed during state interviews but were not included in the final ROI analysis because they were largely unchanged by Exchange Network technology implementation.


Page 26 of 131

Data Validation: All data sent to the state is reviewed before it is submitted to AQS. In general, the data is checked for anything that seems out of the ordinary based on historical patterns, slope consistency, persistence, etc. Once an error is found, it will be flagged.

State to U.S. EPA

Data Reformatting: State systems that receive continuous data will have a reporting option to extract data from the system in the AQS transaction format (80 character string). However, some non-continuous data may be stored separately (Excel, Access, PDF, etc.). This data will need to be reformatted before it is submitted to AQS.

Submission to U.S. EPA: Once the data has been validated and prepared for submission, state staff can then upload the AQS transaction file to CDX. Then the state staff must log into AQS through an application to transfer the data from CDX to the AQS staging database, perform a Critical and Statistical Review test on the data, and then post the data to the AQS production database.

Federal Contracted Lab to U.S. EPA (not included in the ROI analysis) Federal Contracted lab submits a subset of non-continuous data to AQS on behalf of the state. This Cost Factor was not included in the ROI analysis since it is primarily an effort conducted directly between contract labs and the U.S. EPA. State to State/Exchange Partner

Data sharing request: The state may need to request data from other states or Exchange Partners for analysis (typically for data validation). If this data is not yet available from AQS (there is a time lag before data is available on AQS), then they will have to request it directly from the state.

Data sharing response: Other states, local agencies, universities, environmental groups will often request data from the state. State staff will then need to extract the data from the state database and send it to the requester.

Data entry cost (partner data): If a state requests data from another exchange partner for analysis, it may be returned in a format that requires manual data entry into an analysis program (Excel, Access, etc.)


Page 27 of 131

Post-Exchange Network:

State

USEPA

AQS

Exchange Partner

6

Air Program Staff

Exchange Partner

Staff

5

`7

AQSDatabase

Monitor

(1) State Laboratory

Data Logger

Automatically sent to the state

Lab Staff

`

Submissionto EPA

Automatically update the state

database

CDX

Validate

3

AQSFile

2

(2) Federal Contract Lab

Lab Staff

`

StateDatabase

Courtesy copy

Send electronicfile to the state

(Excel, PDR, etc.)

1

Air Program Staff

4

StateNode

Air Program Staff

Lab to State (not included in ROI analysis)5 Implementation of Exchange Network-enabled efficiency improvements will not affect the business process for laboratories to submit data to the state. Therefore it is not included in the ROI analysis. State processing

Data Entry: Some states may store non-continuous data separately from the continuous data stored in the state’s main database. These states would need to enter this data into their main database to allow it to be accessible to the state Node.

Data Validation: The process for validation will be unaffected by the implementation of the AQS flow as an Exchange Network data flow. However, implementation may cause all data to be stored in one central location making it easier to perform validation.

State to U.S. EPA

Data Reformatting: The process for data reformatting will be unaffected by the implementation of the AQS flow as an Exchange Network data flow.

Submission to U.S. EPA: Network partners can automate submissions to the U.S. EPA by scheduling the Node to submit the data to CDX directly. Once the data has been submitted to CDX, state staff

5 These cost factors were analyzed during state interviews but were not included in the final ROI analysis because they were largely unchanged by Exchange Network technology implementation.


Page 28 of 131

will still be required to log into AQS and perform each of the manual steps to eventually post the data into AQS’s production database.

Federal Contracted Lab to U.S. EPA (not included in the ROI analysis) Federal Contracted lab submits a subset of non-continuous data to AQS on behalf of the state. This Cost Factor was not included in the ROI analysis since it is primarily an effort conducted directly between contract labs and the U.S. EPA. State to State/Exchange Partner

Data sharing request: States can access data from other participating states or Exchange Partners directly by querying/soliciting another state’s Node.

Data sharing response: Any data requester with valid credentials can access the data directly from the state database via the state’s Node.

Data entry cost (partner data): Data returned by the Node will be in a XML format which can then be readily imported to other applications.

4.3.1.2 Cost Factors Listed below are the cost factors that contribute to the overall cost to operate the AQS data flow. Also listed under each cost factor are the input parameters that are required to quantify each cost factor, as well as the equation used to calculate each cost.

Cost Factor Input Parameter

Data entry cost (by state) a. Hourly rate of state staff ($/hour)

b. Time spent to enter each submission (hours/submission)

c. Number of submissions (submissions/year) d. Accuracy of data entry (% entered correctly) Equation: a*b*c*(2 – d)

Data Validation a. Hourly rate of state staff ($/hour)

b. Time spent to perform validation (hours/year) Equation: a*b

Data reformatting cost a. Hourly rate of state staff ($/hour)

b. Time spent to perform reformatting (hours/year) Equation: a*b

State submission to the U.S. EPA

a. Hourly rate of state staff ($/hour) b. Time spent to prepare each submission

(hours/submission) c. Time to Post data to AQS (hours/submission) d. Number of submissions (submissions/year) Equation: a*(b + c)*d

Data sharing request cost a. Average number of requests (requests/year)

b. Cost of mailing ($/request) c. Average amount of lost mail (% lost)


Page 29 of 131

Cost Factor Input Parameter Equation: a*b*(1 + c)

Data sharing response cost a. Hourly rate of state staff ($/hour)

b. Time spent to prepare results of request (hours/request)

c. Cost of mailing ($/request) d. Average number of requests (requests/year) e. Average amount of lost mail (% lost) Equation: (a*b*d* + c*d)*(1 + e)

Data entry cost (partner data) a. Hourly rate of state staff ($/hour)

b. # of records/year (records/year) c. Time spent to enter each record (hours/record) d. Accuracy of data entry (% entered correctly) Equation: a*b*c*(2 – d)


Page 30 of 131

4.3.2 RCRA ROI Model The Office of Solid Waste (OSW) regulates all solid and hazardous waste under the Resource Conservation and Recovery Act (RCRA). RCRA's goals are to:

1. Protect people from the hazards of waste disposal 2. Conserve energy and natural resources by recycling and recovery 3. Reduce or eliminate waste, and 4. Clean up waste, which may have spilled, leaked, or have been improperly disposed.

Solid and hazardous waste comes in many shapes and forms. Chemical, metal and furniture manufacturing are some examples of processes that create hazardous waste. RCRA tightly regulates all hazardous waste from "cradle to grave." RCRA also controls garbage and industrial waste. Common garbage is municipal waste, which consists mainly of paper, yard trimmings, glass and other materials. Industrial waste is process waste that comes from a broad range of operations. Some wastes are managed by other federal agencies or state laws. Examples of such wastes are animal waste, radioactive waste and medical waste. RCRA is divided into the following data modules:

1. Handler – This module tracks the handlers of solid and hazardous waste 2. Compliance Monitoring and Enforcement (CME) - Inspections, record reviews, sampling,

and other activities are used by the U.S. EPA and the states to determine a waste handler's compliance with the RCRA requirements. Both the EPA Regional Offices and states carry out compliance monitoring activities.

3. Permitting - The RCRA hazardous waste permitting program helps ensure the safe treatment, storage, and disposal of hazardous wastes by establishing specific requirements that must be followed when managing those wastes. Permits for the treatment, storage, or disposal of hazardous wastes are issued by Authorized States or by EPA Regional Offices.

4. Corrective Action – The Corrective Action Program tracks accidents or other incidents at facilities resulting in the release of hazardous waste or hazardous constituents into soil, ground water, surface water, or air and issues requirements for the clean up or recovery of the contaminated site.

5. Waste Activity Reporting (biennial) – This module requires the reporting of Waste Activity every two years.

New Jersey, Pennsylvania and Washington all participated in the RCRA ROI analysis. However, only the Handler module was assessed for all three states. The permitting module was analyzed for both New Jersey and Pennsylvania, and the Waste Activity Module was analyzed for Washington. Although no data was obtained for the CME and Corrective Action modules, they were left in the ROI model for future use.

4.3.2.1 Business Process Below are two diagrams of the generic business processes for the RCRA data flow for both Pre- and Post-Exchange Network Implementation. All steps labeled with a number have been included in the ROI analysis. Those steps not included represent costs incurred by the facility.


Page 31 of 131

Pre-Exchange Network:

USEPAState

State Staff

RCRAInfo

State Database

State Staff

Facility

FacilityStaff

`

Facility mailsdata to the state

Facility sends datato the state

electronically

Inte

rnet

2b

1

2a

Exchange Partner

4

Exchange Partner

Staff

3

`5

Facility to state (not included in ROI analysis) Facilities submit data to the state for all five of the modules listed above. Depending on the module, this data can be either submitted by mail (paper submission), or uploaded to a state data collection server electronically. State processing Each module may be processed by the state differently. There are three potential methods for processing each module:

1. Direct entry into RCRAInfo: RCRAInfo is the primary source of the data; no data is entered into a state database

2. Direct entry into a state database: The state database is the primary source of the data; data is submitted to RCRAInfo in a batch upload from the data in the state database

3. Dual data entry: State staff enters data directly into the state database and RCRAInfo.

Data Entry: For some (or all) of the RCRA Modules, a state may enter data directly into its state database.

If data is submitted through a data collection server, state staff will not need to manually enter data; it will be automatically sent to the state database. State to U.S. EPA submission

Submission to the U.S. EPA: Once data has been entered into the state database, the state performs a submission to CDX for all RCRA modules that are not entered directly into RCRAInfo.

Direct data entry into RCRAInfo: For some (or all) of the RCRA Modules, a state may enter data directly into RCRAInfo. This data entry may be in addition to entering data into the state database


Page 32 of 131

(dual data entry). State to State/Exchange Partner (These steps were not included in the ROI analysis)6

Data sharing request: The state may need to request data from other states or Exchange Partners for analysis.


Data entry cost (partner data): If a state requests data from another exchange partner for analysis, it may be returned in a format that requires the manual entry into an analysis program (Excel, Access, etc.)


USEPAState

RCRAInfo

State Database

Facility

FacilityStaff

Facility sends datato the state

electronically

Facility mailsdata to the state

Inte

rnet

1CDXState

Node

2a

State Staff

Exchange Partner

4

Exchange Partner

Staff

3

`5

Facility to state (not included in ROI analysis) Implementation of the Exchange Network will not affect the business process for facilities to submit data to the state. State processing

Data Entry: State staff will only enter data directly into the state database (no direct entry into RCRAInfo)

If data is submitted through a data collection server, state staff will not need to manually enter data, it will

6 These cost factors were added to the ROI Model after ROI analysis had been conducted. Therefore they are not included in the analysis but will be present in the ROI Model.


Page 33 of 131

be automatically sent to the state database. State to U.S. EPA submission

Submission to the U.S. EPA: Once data has been added to the state database, the state Node will submit this data directly to RCRAInfo automatically. Typically there will be a nightly update of RCRAInfo in which the state Node will send all data added or updated during the previous day.

State to State/Exchange Partner (These steps were not included in the ROI analysis)

Data sharing request: States can access data from other participating states directly by querying/soliciting another state’s Node.

Data sharing response: Any data requester can access the data directly from the state database via the state’s exchange Node.

Data entry cost (partner data): Data returned by the Node will be in a XML format which can then be readily imported to other applications.

4.3.2.2 Cost Factors Cost Factor Input Parameter

Data Entry into state database (REPEAT FOR EACH MODULE) a. Hourly rate of state staff ($/hour) b. Number of submissions sent to the state

(submissions/year) c. Time spent to enter each submission into the state

database (hours/submission) d. Accuracy of data entry (% entered correctly) Equation: a*b*c*(2 – d)

Submission to the U.S. EPA (REPEAT FOR EACH MODULE) a. Hourly rate of state staff ($/hour) b. Time spent to submit data to the U.S. EPA

(hours/submission) c. Number of submissions sent to the U.S. EPA each

year (submissions/year) Equation: a*b*c

Direct data entry into RCRAInfo (REPEAT FOR EACH MODULE) a. Hourly rate of state staff ($/hour) b. Number of submissions sent to the state

(submissions/year) c. Time spent to enter each submission into

RCRAInfo (hours/submission) d. Accuracy of data entry (% entered correctly) Equation: a*b*c*(2 – d)

Data sharing request cost a. Average number of requests (requests/year)

b. Cost of mailing ($/request) c. Average amount of lost mail (% lost)


Page 34 of 131

Cost Factor Input Parameter Equation: a*b*(1 + c)

Data sharing response cost a. Hourly rate of state staff ($/hour)

b. Time spent to prepare results of request (hours/request)

c. Cost of mailing ($/request) d. Average number of requests (requests/year) e. Average amount of lost mail (% lost) Equation: (a*b*d* + c*d)*(1 + e)

Data entry cost (partner data) a. Hourly rate of state staff ($/hour)

b. # of records/year (records/year) c. Time spent to enter each record (hours/record) d. Accuracy of data entry (% entered correctly) Equation: a*b*c*(2 – d)


Page 35 of 131

4.3.3 SDWIS ROI Model Under the Safe Drinking Water Act, states monitor the drinking water systems within their jurisdictions to ensure that each public water system meets state and the U.S. EPA standards for safe drinking water. Each state, at its own option, may use any data system to collect the information necessary to run its Public Water Supply Supervision (PWSS) program. Some states have developed extensive data systems to meet their specific requirements; others have less sophisticated tracking systems. The Safe Drinking Water Information System (SDWIS) is a system developed by the U.S. EPA to manage drinking water testing and compliance data. It contains information about public water systems and their violations of the U.S. EPA's drinking water regulations. These statutes and accompanying regulations establish maximum contaminant levels, treatment techniques, and monitoring and reporting requirements to ensure that water provided to customers is safe for human consumption. Two versions of the software are available: a state version for use by states to manage their data (states can use this version or develop their own system), and a Federal version used to collect data from all states. Two types of reports that are submitted to State agencies have been considered in the ROI analysis described in this section:

1. Lab Analysis Reports: These reports contain the actual chemical results returned by lab analysis. The lab or the Water System may be responsible for submitting this data to the state.

2. Monthly Operational Report (MOR): These reports contain information concerning the operation of the Water System. The Water System is responsible for reporting this information to the state.

4.3.3.1 Business Process Below are two diagrams of the generic business processes for the SDWIS data flow for both Pre- and Post-Exchange Network Implementation. All steps labeled with a number have been included in the ROI analysis. Those steps not included represent costs incurred by the laboratory or Water System. Pre-Exchange Network:


Page 36 of 131

Inte

rnet

Water System to Laboratory (not included in ROI analysis)7 The Water System sends water samples to the laboratory for analysis. Laboratory to Water System (not included in ROI analysis) The laboratory performs analysis on the samples sent from the Water System. Results are then sent back to the Water System. Laboratory to State (not included in ROI analysis) The laboratory may also be responsible for sending the lab results to the state. This cost factor was not included in the ROI analysis because it is a lab cost; however, this step will be eliminated if a data collection server is implemented at the state. Water System to State (not included in ROI analysis) The Water System may send sample results and MOR data to the state. State processing

Mail processing: State staff must then process all of the mail that is returned (this includes sorting,

7 These four cost factors were not included in the ROI analysis because they involve cost to the reporting community as opposed to the states and will not be suitable for a determination of state Return on Investment calculations.


Page 37 of 131

time stamping, filing, etc.). If a state has implemented an electronic SDWIS solution, the data will be automatically sent to the state via a data collection server and added to the state database with no paper to process.

Data Entry: The data returned from the laboratory and Water System is manually entered into the state database. The state may be using SDWIS-State as its database, a state specific database, or both (a subset of the data may be entered into different databases).

State to U.S. EPA submission

Creation of SDWIS-FED DTF files: The state will need to extract data from its database to be submitted to the U.S. EPA. This step will depend on the type of database being used by the state. If a state uses SDWIS-State, this extraction is a simple function provided by the U.S. EPA.

Submission to the U.S. EPA: Once the submission files have been created, state staff submits the files to CDX.

Validation and resubmission to the U.S. EPA: After the data is submitted to the U.S. EPA, an error report will be created and the state will need to fix any errors and then resubmit the corrected data.

State to State/Exchange Partner (These steps were not included in the ROI analysis)8

Data sharing request: The state may need to request data from other states or Exchange Partners for analysis.


Data entry cost (partner data): If a state requests data from another exchange partner for analysis, it may be returned in a format that requires the manual entry into an analysis program (Excel, Access, etc.)


8 These cost factors were added to the ROI Model after ROI analysis had been conducted. Therefore they are not included in the analysis but will be present in the ROI Model.


Page 38 of 131

Inte

rnet

Water System to Laboratory (not included in ROI analysis) Implementation of the Exchange Network will not affect the business process for Water Systems to submit samples to the laboratory. Laboratory to Water System (not included in ROI analysis) Implementation of the Exchange Network will not affect the business process for laboratories to submit results to the Water System. Laboratory to State (not included in ROI analysis) The laboratory may also be responsible for sending the lab results to the state. This cost factor was not included in the ROI analysis because it is a lab savings; however, this step will be eliminated if a data collection server is implemented. Water System to State (not included in ROI analysis) The Water System may be responsible for sending the lab results, as well as MOR data to the state. This cost factor was not included in the ROI analysis because it is a Water System savings; however, this step will be eliminated if a data collection server is implemented at the state. State processing

Mail processing: This step in the business process will be eliminated if a state moves from a paper SDWIS data submission method to an electronic solution. Otherwise it will be unchanged.


Page 39 of 131

Data Entry: This step in the business process will be eliminated if a state moves from a paper SDWIS data submission method to an electronic solution. Otherwise it will be unchanged.

State to U.S. EPA submission

Creation of SDWIS-FED XML files: Submission files are created using SDWIS-FedRep, most of the validation has been moved to this step in the process.

Submission to the U.S. EPA: The Node can be scheduled to submit the SDWIS XML files directly to CDX automatically.

Validation and resubmission to the U.S. EPA: Most of the validation has been moved to the state, there should be little validation and resubmission after the initial submission to the U.S. EPA once the flow is implemented.

State to State/Exchange Partner (These steps were not included in the ROI analysis)

Data sharing request: States can access data from other participating states directly by querying/s

Documents

EXCHANGE NETWORK RETURN ON INVESTMENT AND BUSINESS … · Ed Karmilovich Pennsylvania DEP (RCRA) Tim Lehman Pennsylvania DEP (SDWIS) ROI Model Final Report Page 3 of 131 Bill Sedlak