1

Freshwater Sediment Standards

Science PanelAugust 25, 2010

Presenters:Russ McMillan – Department of EcologyTeresa Michelsen – Avocet Consulting

2

Freshwater Sediment StandardsGoal For Today

Are the approaches used to develop chemical and biological criteria

scientifically defensible?

3

Freshwater Sediment StandardsDiscussion Points For Today

Introduce regulatory and policy contextPresent proposed biological and chemical

criteria and framework Identify and discuss scientific and

technical issues

4

Freshwater Sediment StandardsPolicy Decisions

Consistency with SMS regulatory framework.

Adopt biological and chemical criteria.Two tier structure: SQS and CSL.Allowance of some adverse effects.Biological override of chemical criteria.

5

Confirmatory bioassays override chemistry

Two tier structure: SQS and CSL Bioassay suite – Multiple species/sensitive

life-history stages Minimum of 3 endpoints Both acute and chronic tests

Proposed Biological Sediment Standards

Regulatory Framework

6

Narrower choice of species than SMS marine criteria due to limited bioassay availability.

Built on EPA and ASTM protocols well established in the Northwest:

Hyalella 10-day mortality – 366 Hyalella 28-day mortality – 312 Hyalella 28-day growth – 79 Chironomus 10-day growth – 525 Chironomus 10-day mortality – 568

Proposed Suite of Bioassays

7

Consistent w/SMS designation of sediment quality SQS: Single SQS level hit CSL: 2+ SQS level hits; 1+ CSL level hit

Consistent w/SMS sensitive endpoints.

Proposed Suite of Bioassays

8

Bioassay suite will include sensitive endpoints from chronic and acute tests:

3 Endpoints 2 Species 1 Chronic test 1 Sublethal endpoint

Requirements for Proposed Bioassay Suite

TestAcute

BioassaysChronic

BioassaysLethal

EndpointSub-lethal Endpoint

Hyalella azteca  10-day mortality X X 28-day mortality X X 28-day growth X XChironomus dilutus  10-day mortality X X 10-day growth X X20-day mortality X X 20-day growth X X

Bioassays: Acute and Chronic and Endpoint Effects Levels

9

10

Is the proposed bioassay suite scientifically defensible as being appropriately protective

of the benthic community?

Proposed Bioassay SuiteQuestion

11

Differs from SMS marine interpretation due to difficulty in identifying appropriate reference areas.

Due to lack of reference sites, interpretation of an SQS and CSL hit is based on a comparison to control.

Comparison to control is a more conservative interpretation than comparison to a reference sediment.

Proposed Freshwater Bioassay Interpretation

Test  QA limits Control

QA limits Reference SQS CSL

Hyalella azteca 10-day mortality  C 20% R  25% T – C > 15% T – C > 25% 28-day mortality  C 20% R  30% T – C > 10% T – C > 25%

28-day growth  CF 0.15 mg/ind

RF  0.15 mg/ind T/C < 0.75 T/C < 0.6

Chironomus dilutus 10-day mortality  C 30% R  30% T – C > 20% T – C > 30%

10-day growth  CF 0.48 mg/ind RF/CF  0.8 T/C < 0.8 T/C < 0.7

20-day mortality  C 32% R  35% T – C > 15% T – C > 25%

20-day growth  CF 0.48 mg/ind RF/CF  0.8 T/C < 0.75 T/C < 0.6

C=Control, R=Reference, T=Test, F=Final, (SQS &CSL hits statistically sign. diff.)

Bioassay Interpretation: Comparison to Control

12

13

Is it scientifically defensible to base interpretation of a bioassay hit by using a comparison to

control rather than a comparison to a reference sediment?

Is it scientifically defensible to base the designation of sediment quality on a suite of bioassays

comparing test to control without the benefit of a reference sediment?

Question

14

15

History of Freshwater SQV Development Early work on FW Apparent Effects Thresholds

(AETs) & Floating Percentile Method (FPM; Portland Harbor) throughout the late 1990s

2002 – Formal evaluation of FW AETs and other existing SQV sets (TELs/PELs, etc.)

Decision that a new approach was needed: FW AETs not sufficiently conservative TELs/PELs, etc., far too conservative National evaluations were not looking at both types

of statistical errors

16

Statistical Digression False Negative (FN) = Predicting that a sample

will be non-toxic when it is actually toxic False Positive (FP) = Predicting that a sample

will be toxic when it is actually non-toxicExisting national methods were focused on reducing false negatives at lower screening levels and false positives at upper screening levels, creating substantial errors and inefficiencies in between, where most actual data are located.

We focused on reducing both types of errors at the same time, for all levels of effects.

17

History, cont. 2003 – Developed interim FPM values, used as

guidance by Ecology and in the regional dredging manual (SEF)

2007 – Regional Sediment Evaluation Team (RSET) OR/WA workgroup formed to update FPM SQVs

2008 – RSET technical work completed; final SQV selection still under discussion and review

2009/2010 – Ecology begins rule revision, finalizes the values, begins peer review

18

Past and current review Presentations at 5 national and regional scientific

conferences (1999-2009) DEQ-led peer review & public meetings during Portland

Harbor (2001 state site) Public and agency review of 2003 Ecology report Presentations at 4 SMARMs (2003-2010) + numerous

RSET public meetings SMS Advisory Workgroup peer review of approach and

draft SQV report (2010) Science Panel and national peer review (2010)

19

Projects/Guidance to Date 1999, 2003, 2008 Portland Harbor, OR 2001 Onondaga Lake, NY 2003 Los Angeles Harbor, CA 2004 San Francisco Bay/Oakland Harbor, CA 2003 Draft Ecology Freshwater Guidelines, WA 2006 Interim Sediment Evaluation Framework

Guidelines for WA/OR/ID 2010 Updated Freshwater Guidelines, WA

20

Question 1

General Approach:- Is it appropriate to use sediment bioassays to represent effects to the benthic community?- Is the use of a multivariate model to empirically derive chemical SQVs scientifically defensible?

21

Use of toxicity tests Marine AETs were derived based on toxicity

tests, benthic community studies, and chemistry Benthic community studies were considered

equivalent to a chronic bioassay Freshwater benthic community data were

searched for but not found in OR, WA, or ID Substantial diversity of freshwater sites

compared to marine areas complicates use of benthic community data, if there were any

22

Floating Percentile Method Goal: Minimize false negatives and false positives

simultaneously Approach:

Data QA, screening, and summing Determine “true” toxicity based on bioassay results Search for the most predictive chemical

concentrations, allowing each chemical to move independently to the level at which it appears to be toxic

23

Features Uses synoptic chemistry and bioassay field data Multivariate → considers all chemicals at once Incorporates measures to address covariance Requires selection of false negative target Follows with optimization of false positives Repeat for a range of false negative targets Thoroughly evaluates reliability Now automated using a series of Excel

spreadsheets

24

25

Covariance All chemicals are addressed simultaneously – avoids

inappropriate assignment of toxicity Covariance analysis can be run ahead of time Model results allow visual identification of covariance

patterns Appropriate chemical classes can be summed Those that can’t be summed but often covary are

subjected to multiple runs with different starting points to find the low concentration for each chemical, then selection of the combination with the highest reliability

26

Question 2

Data Issues:- Is the data set sufficiently robust and representative?- Has appropriate data screening and QA been conducted?

27

Data Set – Chemistry

Oregon and Washington West and east of the Cascade Mountains Lakes, rivers, small and large Various geochemical environments 50 analytes and sums → 105 chemicals Rigorous QA/QC applied (PSEP QA2) –

qualifiers rectified among data sets

28

Data Set – Bioassay Endpoints

Hyalella 10-day mortality – 366 Chironomus 10-day mortality – 550 Chironomus 10-day growth – 504 Hyalella 28-day mortality – 319 Hyalella 28-day growth – 79 Rigorous QA/QC applied – recent ASTM

protocols, QA2 review of lab sheets, etc.

29

Question 3Reliability Testing:

- Is the reliability testing that was conducted an appropriate method for evaluating SQVs?- Is the comparative reliability analysis that was conducted an appropriate way of making decisions while fine-tuning the approach?- Are the reliability measures that were used the right ones and were the relative weights given to them appropriate?- Are there alternative methods of validation that could have been used without collecting additional data?

30

Reliability Sensitivity (100% – false negatives) Efficiency (100% – false positives) Predicted no-hit reliability Predicted hit reliability Overall reliability

All measures of reliability were used for ALL effects levels – endpoints given greater weight are shown in yellow

31

Reliability Measures Diagram

32

Reliability Goals

The RSET Workgroup set the following goals before beginning SQV development:

SQS (%) CSL (%)Sensitivity  80 – 90 75 – 85Efficiency 70 – 80 75 – 85Predicted Hit Reliability

70 – 80 75 – 85

Predicted No-Hit Reliability

80 – 90 75 – 85

Overall Reliability 80 – 90 80 – 90

33

34

Freshwater Standards Reliability

Values are averages across relevant assays

No Effect Level

Minor Effect Level

35

Comparative Reliability Analysis East side vs. west side vs. combined TPH vs. PAH vs. combined Microtox – include? Hyalella growth – include Portland Harbor? Ammonia and sulfides issues N-qualified pesticides Blank-correction standardization Comparison to control vs. reference

36

SQV Validation RSET made a decision early on to not withhold

part of the data set for validation For such a large heterogeneous area, we needed all

the data to develop the best possible model Most other SQV sets have followed the same

approach, with independent validation following after Independent validation will require a large,

representative data set, not just a few projects Other validation methods available?

37

Question 4

Data Interpretation and Regulatory Decision-Making:- Is the overall framework and selection of final SQVs consistent with the SMS and marine SQVs?- Is the approach used to select final SQVs scientifically defensible?

38

Challenges – Criteria Selection

Expectation: SQS values clustered below CSL values SQS CSL

39

Challenges – Criteria Selection

Reality: Differences between bioassays were much greater than differences between endpoints → try

species sensitivity distribution approach

40

“>” values- no toxicity observed for that endpoint up to the listed concentration. Sample concentrations at or above this level should undergo toxicity testing.

Approach for selection of CSL: “next significantly different value”

41

Questions?

42

Petroleum ToxicityPAHs contribute to the greatest number of errors: PAHs do not sufficiently capture petroleum

toxicity No form of normalizing or summing solves the

problem Addressed through side-by-side PAH/TPH &

combined model runs Best results when both were included May be legacy issues with not enough TPH data

43

Summation of Chemical Classes PAHs/TPH classes PCB Aroclors Dioxins/Furans Chlordanes DDT, DDE, DDD isomers Heptachlors