Institute for Tribal Environmental Professionals, Tribal Air Monitoring Support Center

Guest Speaker: Mike Papp, OAQPS

April 7: QA Systems, EPA definitions, PQAOs and common sense – Mike Papp

April 14: Routine Quality Control and Data Management (1-pt QC, flow rate, and instrument stability checks)

April 21: Audits Overview (NPAP, PEP, Annual PE, Flow Rate Audits) – Jeremy Howe

April 28: Calculating Bias and Precision and AQS reports

May 5: 40 CFR 58 App. A- Gaseous Pollutants – Glenn Gehring

May 12: 40 CFR 58 App. A- Ozone – Brenda Jarrell May 19: 40 CFR 58 App. A- PM filter and continuous

methods –Brandy Toft

2

Why QA? Quality systems Terminology QMPs, QAPPs, and other monkeys EPA’s Graded Approach CFR online Appendix A intro PQAOs

3

4

London Killer Fog

Donora PA 1948

5

“(2) Establishment of a national network to monitor, collect, and compile data with quantification of uncertainty in the status and trends of air emissions, deposition, air quality, surface water quality, forest condition, and visibility impairment and to ensure the comparability of air quality data collected in different States and obtained from different nations.”

{

6

E4- Part of quality management focused of providing confidence that quality requirements will be fulfilled.

Another Definition- A system of activities whose purpose is to ensure that information derived from measurements are of a quality that the decision maker is willing to risk making an inappropriate decision.

Some decisions will be inappropriate (wrong) due to

data uncertainty…

…the difference between

your measurement (estimate) and the “truth”

Premise 1 - All estimates have error so all decisions made with estimates have risks.Premise 2 - We can’t afford 100% certainty in our decisions

The purpose of QA is to help you get the data you need to answer the questions you have as cheaply and quickly as possible

It can quantify uncertainty and comparability QA done right does save you time and money QA makes you:

◦ Plan ahead◦ Ensure everyone agrees to the:

goals and equipment used, what their jobs are, the schedule, and who reports what to whom when

7

National consensus standard authorized by American National Standards Institute (ANSI)

Developed by American Society for Quality (ASQ) Consistent with ISO standards (there are ISO

standards for everything, drill bits, toilets, diapers, tires)

EPA adds to E4 with:◦ EPA Quality Policy 2106.0 and Procedure for Quality

Policy 2106-P-1.0◦ EPA Guidance for Quality Management Plans: EPA QA/G-2◦ EPA Guidance for Quality Assurance Project Plans: EPA

QA/G-5

EPA Quality Staff Web site http://www.epa.gov/quality1

8

Quality System

Quality Management

Quality Assurance Quality Control

A blue print of how an organization produces quality products.

develops and implements the quality policy defined in a monitoring organization's QMP.

includes strategic planning, allocation of resources and other systematic planning activities (e.g., planning, implementation, assessing and reporting) pertaining to the quality system.

must have sufficient technical expertise and management authority to conduct independent oversight and assure the implementation of the organization's quality system relative to the ambient air quality monitoring

should be organizationally independent of environmental data generation activities.

10

11

Program that makes sure the quality requirements are fulfilled

Activities that provide checks on specific areas of a process to help keep them acceptable

Quality System: have enough money to pay the bills and plan for a fun retirement

QA are the activities of:◦ Setting up a long range goal◦ Establishing monthly budget◦ Allocating $$ for checking and saving◦ Reviewing IRA statement monthly, determine reallocation annually ◦ Depositing checks

QC is:◦ routinely balancing our checkbooks and calculating whether checks

written or online payments are going to bounce◦ Checking bill statement for errors

• QMP• Describes orgs quality

system• Establishes capability

◦ QA Project Plan• Identifies the reasons for

collecting data and for collecting it in a specific way

• Documents how the data are collected and how quality is maintained

SOP• Ensures consistency

• From day to day• From one person to

the next

Ford

◦ Mustang

engine

12

Quality is Job 1

Quality Management Plans reflect activities and policies common to all projects.

Quality Assurance Project Plans reflect specific projects.

QMP

QAPPQAPP QAPPQAPP

Mission and quality policy of the organization

Specific roles and responsibilities with respect to QA and QC activities

Means and structure to assure effective communication

Processes used to plan, implement , assess and report work and the effectiveness of QA and QC activities

Process for continual improvement of the Quality System

14

Must provide sufficient detail to show that:

•The project objectives are identified

•The intended measurements are appropriate for achieving project objectives

•The assessments (audits QC checks, data reviews) are sufficient for confirming that data of the type and quality needed are obtained

•Data use limitations are defined (e.g.; non-regulatory monitor type data can’t be used for NAAQS)

15

Saves money Saves time Helps communication No later misunderstandings Everything contained in or referenced in

one document Public document in case of litigation

16

QA and QC requirements based on:

Importance of work

Available resources

Unique needs of organization

Consequences of making an wrong decision with the data

17

http://www.epa.gov/ttn/amtic/qaqcrein.html

Category 1 projects are to compare against the NAAQS standard, and require all 24 elements

Category 2-3 projects would include looking at air pollution transport, air toxics, exploratory studies, or evaluating whether further measurements should be made, and require even fewer of the 24 elements of a QAPP

Category 4 projects are education and outreach and may make a few qualitative measurements and only require 6 elements of a QAPP (see example on course website)

18

Important Decisions

Less Important Decisions

19

Taking the overall EPA Requirements and bringing them into the Ambient Air Monitoring

Program

21

Criteria Pollutants◦ Ozone◦ CO◦ SO2

◦ NO2

◦ Pb◦ Particulate Matter

PM10 PM2.5

Air Quality

Standard

AmbientAir Data

Adjust

Classification

Continue

Air Quality

Measurement

EPA Responsibility

Monitoring Org

Responsibility

Control

Strategy

State/Tribal

Implementation

Plan

Emergency

Control

Real TimeReporting (AQI)

ResearchTrends

Analysis

Attainment of Air Quality Standards

NAAQS

Demo how to download sections at e-CFR Copies available in course website

22

23

Regulations◦ Are requirements◦ Must be followed◦ Usually minimum

requirements.. more is better

Guidance◦ More details on

regulations◦ Provides additional

suggestions or strongly suggests

◦ Are not mandatory, but you need an acceptable alternative

EPA QualityProgram PolicyCIO 2106.0

OAQPS40 CFR Part 58 Appendix A

Mon. OrgsQMP, QAPP, SOPs

R-RequirementsG-Guidance

Redbook Guidance

• EPA Regs– QA Policy

– CFR Contracts and Grants

– R2- QMP

– R5 – QAPPs

– 40 CFR Part 58 App A

• Guidance– G2- QMP

– G5-QAPP

– QA Handbook Vol II

– QA Handbook Volume IV

– TADs, White Papers, etc

24

Go to AMTIC Home page click on

25

Section 1◦ General Info◦ Measurement Uncertainty◦ Measurement Quality Checks◦ Assessment and Reports

Section 2◦ QMPs/QAPPs*◦ Independent quality

management function*◦ DQOs- (EPA)

PM2.5, O3, PM10-2.5

◦ NPAP/PEP◦ TSAs-(EPA Regions)◦ NIST Traceable Standards

Section 3 ◦ PQAO definition◦ Quality Control and

Assessment 1-point QC Annual PE Flow Rate Verification Flow Rate Audit Collocated Sampling PEP

Section 4 ◦ QA Stats

Section 5-◦ Reporting Req.

List- Site ID/PQAO by pollutant Quarterly submission of QA per

58.16 reporting requirements Calculation of QA (EPA) once

annual submission occurs

} Gaseous pollutants

}PM/Pb

* EPA QA Policy (covered earlier)

There is so much more to QA beyond Appendix A

26

Deviations from the “true” concentration Defined in terms of data quality indicators:

◦ Bias◦ Precision◦ Completeness◦ Detectability

} accuracy

Other data quality indicators like comparability and representativeness are important, but since they relate to spatial/temporal variability or data outside the control ofa monitoring organization, they are not included in Appendix A.

27

Acknowledging that the quality control checks in Section 3 are required to be reported to AQS.

One exception- flow rate verifications (with the exception of automated PM10) are not required to be reported to AQS. However, they can be reported if desired.

Each primary quality assurance organization shall be defined such that measurement uncertainty among all stations in the organization can be expected to be reasonably homogeneous, as a result of common factors. Common factors that should be considered by monitoring organizations in defining primary quality assurance organizations include, but all are not strictly necessary (see Region 5 and OAQPS PQAO guidance):

 (a) Operation by a common team of field operators according to a

common set of procedures;(b) Use of a common QAPP or standard operating procedures;(c) Common calibration facilities and standards;(d) Oversight by a common quality assurance organization; and(e) Support by a common management, laboratory or

headquarters.

28

Development of one QAPP and SOPs Fewer sets of calibration equipment Independent QA management for

consolidated PQAO Reduced requirements (and resources) for

collocation, PEP and NPAP Data Reporting

29

The Aroostook Band of Micmac Indians, the Passamaquoddy Tribe and the Penonobscot Nation have sites that are consolidated with the State of Maine.

The Wampanoag Tribe of Gay Head is consolidated with the State of Massachusetts.

The Little River Band of Ottowa has consolidated with the State of Michigan.

The Little Travers Bay Band has consolidated with the Inter-Tribal Council of Michigan

The Fond du Lac Band of Minnesota Chippewa and the Mille Lacs Band of Minnesota Chippewa have consolidated with the State of Minnesota.

The Bad River Band of Lake Superior Tribe of Chippewa Indians and the Forest County Potawatomi Community have consolidated with the State of Wisconsin

30

Work with the EPA Regions. Regions that have performed consolidation have some guidance developed.

Assess common factors with monitoring organizations that are consolidating.

Establish how requirements for QA documentation/implementation will be met.

Prepare some agreement documentation and formalize.

31

Anyone expecting their data to be respected must have some form of quality system (remember this is just good management plus documentation, in a QAPP)

Anyone receiving grant funds from a federal agency must have a quality system

Even 1-person programs can have data that is legally and scientifically defensible by following the guidelines

32

QA/QC comes down to common sense and documentation

Subsequent webinars will cover pollutant specifics

Course website: http://itep68.itep.nau.edu/itep_downloads/

Our emails:◦ Papp.michael@epa.gov◦ Melinda.ronca-battista@nau.edu◦ Christopher.lee@nau.edu

33

34