Data Data ManagementManagement

ConceptsConceptsJames PayneJames Payne

Environmental Protection Environmental Protection DepartmentDepartment

Morongo Band of Mission Morongo Band of Mission IndiansIndians

22

Why data management?Why data management?

Ensure data are Ensure data are scientifically validscientifically valid i.e., you have done what you said you’d do, in the way i.e., you have done what you said you’d do, in the way

you said you’d do ityou said you’d do it

How? Equipment checks, flagging data, scheduled How? Equipment checks, flagging data, scheduled

maintenancemaintenance

Ensure data are Ensure data are legally defensiblelegally defensible i.e., you can prove that you did it with evidencei.e., you can prove that you did it with evidence

How? Chain of custody forms, logbooks/logsheets, audit How? Chain of custody forms, logbooks/logsheets, audit

reports reports

Remember: If you don’t write it down, it didn’t happenRemember: If you don’t write it down, it didn’t happen

33

How How youryour data is managed data is managed and assessed depends and assessed depends

on…on… How the data will be used and by whomHow the data will be used and by whom How good the data needs to beHow good the data needs to be

Data quality objectivesData quality objectives How the data will be collectedHow the data will be collected

Newly collected data vs. Previously collected Newly collected data vs. Previously collected datadata

To which databases it will need to be To which databases it will need to be uploadeduploaded EPA’s Air Quality System (AQS)EPA’s Air Quality System (AQS) EPA’s National Emission Inventory (NEI)EPA’s National Emission Inventory (NEI) OtherOther

44

Collecting the Data Collecting the Data

Requires Requires Data Management SystemData Management System InvolvesInvolves

Planning data collectionPlanning data collection Implementing data collectionImplementing data collection Assessing collected dataAssessing collected data

Mechanism to continually improve Mechanism to continually improve system system

EPA provides data management “tools”EPA provides data management “tools”

55

EPA’s Quality EPA’s Quality Management ToolsManagement Tools

Quality Management Plans (QMPs)Quality Management Plans (QMPs) Management Systems Reviews (MSRs)Management Systems Reviews (MSRs) Data Quality Objectives (DQOs)Data Quality Objectives (DQOs) QA Project Plans (QAPPs)QA Project Plans (QAPPs) Standard Operating Procedures (SOPs)Standard Operating Procedures (SOPs) Technical Systems Audits (TSAs)Technical Systems Audits (TSAs) Data Quality Assessment (DQA)Data Quality Assessment (DQA)

66

Quality Assurance Project Quality Assurance Project Plan (QAPP)Plan (QAPP)

Detailed description of Detailed description of whatwhat, , wherewhere, ,

whenwhen, , whowho, , howhow and and whywhy of project of project

activitiesactivities

Opportunity to review systems in place for Opportunity to review systems in place for

quality assurance (QA)quality assurance (QA)

Documentation of QA (if not written, didn’t Documentation of QA (if not written, didn’t

happen for QA purposes)happen for QA purposes)

77

Why do I need a QAPP?Why do I need a QAPP?

Provides overview of project goals, organizationProvides overview of project goals, organization

Details information for every aspect of the Details information for every aspect of the

projectproject

Outlines clear chain of authority for QAOutlines clear chain of authority for QA

Describes need for the measurementsDescribes need for the measurements

Defines QA/QC for projectDefines QA/QC for project

Usually required for EPA-funded air monitoring Usually required for EPA-funded air monitoring

activitiesactivities

88

What is in a QAPP?What is in a QAPP?

Program staff & management Program staff & management structurestructure

Samplers/equipmentSamplers/equipment Project activities & Standard Project activities & Standard

Operating Procedures (SOPs)Operating Procedures (SOPs) Lab & contract staffLab & contract staff Data reviewers & auditorsData reviewers & auditors

99

What is QA/QC?What is QA/QC?

Quality Assurance (QA)Quality Assurance (QA): integrated system of : integrated system of management activitiesmanagement activities involving involving PlanningPlanning ImplementationImplementation DocumentationDocumentation AssessmentAssessment ReportingReporting Quality improvementQuality improvement

Good QA ensures that a process is of good qualityGood QA ensures that a process is of good quality

1010

What is QA/QC? (cont.)What is QA/QC? (cont.)

Quality Control (QC):Quality Control (QC): overall system of overall system of

technical activitiestechnical activities Measures performance of process against Measures performance of process against

defined standardsdefined standards

Verifies that processes meet requirements Verifies that processes meet requirements

established by QAPPestablished by QAPP

QC is part of QAQC is part of QA

1111

Morongo QA ActivitiesMorongo QA Activities

Standard Operating ProceduresStandard Operating Procedures Equipment checksEquipment checks

Automated status checkAutomated status check Daily data reviewDaily data review Weekly physical checkWeekly physical check Quarterly solar adjustmentQuarterly solar adjustment Semi-yearly independent equipment Semi-yearly independent equipment

calibration/performance testingcalibration/performance testing Inter-tribalInter-tribal

1212

QA Activities (cont.)QA Activities (cont.)

Data checksData checks Daily data reviewDaily data review

Summary tables; daily, monthlySummary tables; daily, monthly

Event logEvent log

Second-person check needed but currently Second-person check needed but currently

not donenot done

Check against local air quality Check against local air quality

management district datamanagement district data

1313

Morongo QC ActivitiesMorongo QC Activities

Technical activities that control data quality, should be part Technical activities that control data quality, should be part

of standard operating procedures (SOPs)of standard operating procedures (SOPs)

Weekly automated calibrationsWeekly automated calibrations

Quarterly Calibration machine checkQuarterly Calibration machine check Sampler auditsSampler audits

Check against local air quality management district dataCheck against local air quality management district data

(Field blanks)(Field blanks)

(Collocated replicate samples)(Collocated replicate samples)

(Duplicate analyses)(Duplicate analyses)

(Lab QC)(Lab QC)

(Data handling checks)(Data handling checks)

1414

Data-Collection MethodsData-Collection Methods

Ambient air monitoringAmbient air monitoring 40 CFR Part 50–40 CFR Part 50–National Primary and Secondary Ambient Air National Primary and Secondary Ambient Air

Quality StandardsQuality Standards

Source samplingSource sampling 40 CFR Part 60–40 CFR Part 60–Standards of Performance for New Stationary Standards of Performance for New Stationary

SourcesSources

Emissions inventoryEmissions inventory

Other sourcesOther sources EPA, states, tribes, otherEPA, states, tribes, other

Data provided by facilityData provided by facility

Data from prior studiesData from prior studies

Technical literature & reportsTechnical literature & reports

1515

Examples of Data UsesExamples of Data Uses

Source emissionsSource emissions Ambient concentrationAmbient concentration

Baseline dataBaseline data Attainment/Non-attainmentAttainment/Non-attainment

Source complianceSource compliance AQSAQS Code developmentCode development Permit writingPermit writing

1616

Morongo Data UsageMorongo Data Usage Establish baseline and seasonal trends of Ozone Establish baseline and seasonal trends of Ozone

& PM-2.5 levels on tribal lands& PM-2.5 levels on tribal lands

Determine meteorological trendsDetermine meteorological trends

Determine feasibility of realignment of non-Determine feasibility of realignment of non-

attainment boundaryattainment boundary

Upload into AQSUpload into AQS

Monitor long-term ozone & PM-2.5 changesMonitor long-term ozone & PM-2.5 changes

Regulate tribal activitiesRegulate tribal activities

Issue alertsIssue alerts

1717

Data Quality Data Quality ConsiderationsConsiderations

Is the quality of the Is the quality of the data known?data known?

Are errors within Are errors within acceptable limits?acceptable limits?

Are the data Are the data usable?usable?

Were enough data Were enough data collected?collected?

1818

Data Quality Objectives Data Quality Objectives (DQOs)(DQOs)

Method detection limits (MDLs) Method detection limits (MDLs)

Data completeness Data completeness

Accuracy of measurements Accuracy of measurements

Spatial coverage (representativeness)Spatial coverage (representativeness)

Data quantity (frequency of Data quantity (frequency of

measurements)measurements)

1919

Morongo Monitoring Morongo Monitoring DQOsDQOs

Measure continuous ozone & PM-2.5 Measure continuous ozone & PM-2.5

concentrations on tribal lands using 1-concentrations on tribal lands using 1-

hour averageshour averages

Measure meteorological conditions on Measure meteorological conditions on

tribal lands using 1-hour averagestribal lands using 1-hour averages

Maintain data recovery and quality at Maintain data recovery and quality at

>75%>75%

2020

Basic “Toolbox” For Basic “Toolbox” For Assessing Data Assessing Data

UncertaintyUncertainty

Collect and analyze Collect and analyze

specificspecific QC samplesQC samples Use basic statistical Use basic statistical

methods to calculate and methods to calculate and

evaluateevaluate Sampling variabilitySampling variability Measurement errorMeasurement error

2121

Data Review/ValidationData Review/Validation

Using all available QC informationUsing all available QC information Review Data Quality Objectives (DQOs)Review Data Quality Objectives (DQOs) Weed out any big problems (flag bad data)Weed out any big problems (flag bad data) Determine MDLs, accuracy of resultsDetermine MDLs, accuracy of results

Determine if any corrective action is requiredDetermine if any corrective action is required Can be an expensive and time-consuming Can be an expensive and time-consuming

stepstep Most important step in data managementMost important step in data management

““No data is better than bad data”No data is better than bad data”

2222

Tools for Data ValidationTools for Data Validation

Microsoft Excel has numerous tools for reviewing/validating Microsoft Excel has numerous tools for reviewing/validating datadata Sorting/filteringSorting/filtering Calculations (built-in and custom)Calculations (built-in and custom) Charting/graphing functionsCharting/graphing functions Excel is relatively straightforward and easy to useExcel is relatively straightforward and easy to use

Microsoft AccessMicrosoft Access Powerful database applicationPowerful database application Extremely useful for managing large, complex sets of dataExtremely useful for managing large, complex sets of data Not as intuitive or accessible as MS Excel for the “casual” userNot as intuitive or accessible as MS Excel for the “casual” user

Software specificSoftware specific Datalogger software will send email for set criteriaDatalogger software will send email for set criteria Tribal Data ToolboxTribal Data Toolbox

2323

Data ReportingData Reporting

Internal and project-related reportingInternal and project-related reporting Reports to tribal councilReports to tribal council

Progress reports to EPA or other funding Progress reports to EPA or other funding

agencyagency

External reportingExternal reporting Reports to community and other audiencesReports to community and other audiences

Reporting data to national databases Reporting data to national databases

(AQS, NEI)(AQS, NEI)

2424

Data Reporting (cont.)Data Reporting (cont.)

Air Quality System (AQS) not easy to useAir Quality System (AQS) not easy to use Data requires specific format to be accepted into Data requires specific format to be accepted into

databasedatabase

Not known for user-friendlinessNot known for user-friendliness

Tribal concerns about data sharingTribal concerns about data sharing

Critical for increasing our collective Critical for increasing our collective

understanding of effects of air pollutionunderstanding of effects of air pollution

Necessary for continued political support of Necessary for continued political support of

tribal monitoring activitiestribal monitoring activities

2525

Other Data SharingOther Data Sharing

National Emissions Inventory (NEI) databaseNational Emissions Inventory (NEI) database Tribal EI data has large gapsTribal EI data has large gaps

ITEP attempting to help tribes enter dataITEP attempting to help tribes enter data

Tribal Exchange (TREX) Network ProjectTribal Exchange (TREX) Network Project Nine tribes from Southwest uploading continuous Nine tribes from Southwest uploading continuous

data into national network – 4 others showing data into national network – 4 others showing

interestinterest

Walker River Paiute Tribe (NV) leads the projectWalker River Paiute Tribe (NV) leads the project

2626

Preserving/Protecting Preserving/Protecting DataData

Make sure data is accessibleMake sure data is accessible Changing sensor and acquisition hardwareChanging sensor and acquisition hardware

Changing software and operating systemsChanging software and operating systems Bad ITBad IT

Backup dataBackup data Automated backupAutomated backup

Multiple location repositoryMultiple location repository

HardcopyHardcopy

2727

Key Take-Home MessagesKey Take-Home Messages

Good data management requires Good data management requires

organizational commitment to organizational commitment to Good planningGood planning

Realistic allocation of resources (esp. staff Realistic allocation of resources (esp. staff

time)time)

Consistent executionConsistent execution

Good documentationGood documentation

NoNo data is better than data is better than badbad data! data!

2828

Take-Home Messages Take-Home Messages (cont.)(cont.)

QAPP is vehicle by which data-QAPP is vehicle by which data-

management process is defined for a management process is defined for a

specific projectspecific project Data Quality ObjectivesData Quality Objectives

Quality-control measuresQuality-control measures

Data-validation stepsData-validation steps

Data reportingData reporting

Preserve/protect your dataPreserve/protect your data