VSP Monitoring 101

Brian Cochrane, Levi Keesecker, and Bill Eller

December 8, 2020

VSP Monitoring 101

Outline:

What is Monitoring? Back to Basics

Anatomy of Monitoring in VSP

Worked Example: analyzing and reporting data for VSP

(I)

(II)

(III)

What is monitoring?

Monitor verb

monitored; monitoring\ ˈmä-nə- t(ə- )riŋ \

Definition of monitor (Entry 2 of 2)

transitive verb

: to watch, keep track of, or check usually for a special purpose


https://www.merriam-webster.com/dictionary/verb

https://www.merriam-webster.com/dictionary/transitive

“For a special purpose” = why?

What, where, when?

How? (and how $$?)

What ….

is your

quest?


why?

• Regulatory requirement?

• Implementation monitoring?

• Effectiveness monitoring?

Specific parameter and

method;

Specific place;

Specific frequency;

Specific response to critical

value.

Or …

“Thou shalt monitor”

• Compliance with permit conditions

• Basis for enforcement• Evaluating efficiencies• Characterization for re-

issuance• Bad poetry

The regulators might have a completely

different set of reasons why ….

why?Did the thing get

built/installed/planted?

Did the thing get

installed to spec?

Is the thing still there?




why?




Did the thing work like

we anticipated?

How well is it working?

Is the sum of the things

in an area doing what we

wanted them to?

How are the things

performing over time?

Why’s corollary:

“how good answer

to you need?”

Or, put another way …

When will you do

something with the

monitoring answer?

Two other parts to

the question: space

and time.

• Over what time frame (and at what frequency)?

vs.

• At what

time

frame?

• At what

spatial

extent?

Monitoring:

• is done for a purpose (the

why)

• Answers a question

• Has an action as an

outcome

• Must address why BEFORE

jumping to what, where,

when and how

Anatomy of Monitoring in VSPWetlands Geologically Hazardous Areas

Fish & Wildlife Habitat Frequently Flooded Areas

Critical Aquifer Recharge Areas

Critical Areas

Anatomy of Monitoring in VSPImplementation Monitoring

Example: “What BMPs are occurring on the

landscape, how much, and where?”

Conservation Tillage

Cover Cropping

Hedgerows





Livestock Fencing

Cover Cropping

HedgerowsMicro Sprinklers




VSP Checklists

Conservation Practice Databases

(e.g, CPDS, NRCS, or Conservation District records)

Farm Management Plans

Aerial/Satellite Imagery

Common Methods


Livestock Fencing

Cover Cropping

HedgerowsMicro Sprinklers

Anatomy of Monitoring in VSPEffectiveness Monitoring

Example: “How are BMPs affecting critical

area functions and values?”

Flood StorageWater Quality ImprovementShoreline & Erosion Control

Natural Products (food/medicines)Food/Habitat for Fish & Wildlife

Wetlands






Wetlands

Frequently Flooded Areas

Flood StorageReduced Erosion/Sedimentation

Groundwater RechargeHydrologic Connectivity

Food/Habitat for Fish & Wildlife






Wetlands





Geologically Hazardous Areas

Erosion PreventionLandslide Prevention






Wetlands







Fish & Wildlife Habitat

Food Provisioning (e.g., fisheries)Sustains Native Species/EcosystemsPollination of Wild/Cultivated PlantsPredation of Pest Insects by Native

Species/Communities






Wetlands









Species/Communities

Critical Aquifer Recharge Areas





(1) What’s your question?

(e.g., which functions/values?, which critical area types?, what is important in our county/watershed?)

(1) How good (accurate/precise) of an answer do you need?

(e.g., presence/absence vs. concentrations vs. categories)

(3) What is the appropriate scale to ask the question?

(e.g., parcel vs. subwatershed vs. basin)


Effectiveness Monitoring



Example 1: Wetlands

Wetlands



Example: “How are BMPs livestock fences/watering stations/nutrient mgmt. practices affecting critical

area wetland functions and values in WRIA 35?”

Example 1: Wetlands

Wetlands





Example 1: Wetlands



Wetlands





Example 1: Wetlands



Wetlands

Which functions and values are important to our community/work group?

Example: Flood Storage





Example 1: Wetlands



Wetlands



How good of an answer do we need?

Example: (a) Presence/absence/amount – how many wetlands in 2016 compared to 2011? How much area?





Example 1: Wetlands



Wetlands




Example: (a) Presence/absence/amount – how many wetlands in 2016 compared to 2011? How much area?

Examples:

GPS Locations and Photo Verify

Mapping and Remote Sensing

Water Gauge/Water Level (rebar, staff gauge, etc)


Examples:

Wetlands Assessments (Dept. of Ecology, Assoc. Wetland Managers, misc. rapid assessments)

Water Gauge/Water Level (rebar, staff gauge, etc)




Example 1: Wetlands



Wetlands




Example:

(b) Quality - how have wetland conditions/quality changed from 2011 to 2016?

Degraded Improved





Example 1: Wetlands

Wetlands

Detail of Information

(i.e., “how good of an answer?”)Low High

Presence/Absence/Amount

Wetlands Assessment -Detailed

Wetlands Assessment - Rapid





Example 1: Wetlands

Wetlands




Wetlands Assessment -Detailed

Wetlands Assessment - Rapid

Identify question and need first, then look for tools that

can helpWetland Biologist

Survey

Modified Rapid Assessment –Field Tech/Landowner

GPS Wetland Locations, Photo Verify, and/or

Water Gauge


Type of “response” that is being monitored depends on question and approach

Wetland is present (yes/no)

Wetland is present & has water at critical periods

Wetland is present & has water at

critical periods & has few invasive

plants

Semi-quantitative wetland scores

(poor, fair, good, very good)

Wetland classification and

multiple functions

Detail of Information (i.e., “how good of an answer?”)

Low High

Wetlands

- % of wetlands in good/very good condition

- Average wetland scores/ratings (e.g., 0 – 10)

- Change in wetland scores from 2011 to 2016

- presence/absence/density of invasive or keystone species



Wetland is present (yes/no)

Wetland is present & has water at critical periods

Wetland is present & has water at

critical periods & has few invasive

plants

Semi-quantitative wetland scores

(poor, fair, good, very good)

Wetland classification and

multiple functions


Low High

Wetlands

- # of total wetlands

- Total acreage of wetlands

- Total wetland acreage lost from

2011 to 2016

- # of total wetlands w/ water in critical periods

- Total acreage of wetlands with water in critical

periods

- Total wetland acreage with water in critical

periods lost from 2011 to 2016

- Amount of water stored in wetlands during critical

period

- Hydrologic function and connectivity assessment

- Change in wetland hydrologic storage capacity

from 2011 to 2016


Effectiveness MonitoringExample: “How are BMPs cover crops, tillage mgmt., and managed grazing affecting critical area fish and wildlife

functions and values in WRIA 24?”

Example 2: Fish and Wildlife








Species/Communities







Species/Communities


Example: Food Provisioning (e.g., fisheries)







Species/Communities




Example: (a) Presence/Absence/Amount – how many stream miles have fish in 2016 compared to 2011? How many fish?

Examples:

Fish distribution maps – often presence/absence/historic (e.g., WDFW, USFW)

Fish counts

Etc







Species/Communities




Example:

(b) Quality - how have fish habitat conditions/quality changed from 2011 to 2016?

Examples:

Suspended solids/turbidity (Imhoff cone, water probe)

Stream Visual Assessment Protocol (SVAP, others)

Riparian shade cover (densitometer)

Water depth (staff gauge)

Macroinvertebrates

Pool:riffle:run (measuring tape, visual assessment)









Stream/Riparian Assessment - Detailed

Stream/Riparian Assessment - Rapid









Stream/Riparian Assessment - Detailed

Stream/Riparian Assessment - Rapid

Identify question and need first, then look for tools that

can help

Geomorphic surveys (habitat)

Stream flow (habitat)

Surface/groundwater interactions (temperature/habitat)

Imhoff cone (turbidity)

Thermometer (temperature)

Macroinvertebrates (indicator of quality/conditions)

Densiometer (riparian shade cover)

Mapping

Presence/Absence

Indicators



Fish are/were present (yes/no)

Fish are/were present & stream

has water at critical periods


has water at critical periods & riparian

shade cover is intact

Semi-quantitative habitat scores (poor,

fair, good, very good)/water samples

Quantitative/qualitative riparian stream surveys

& water quality data


Low High


- % of streams in good/very good condition

- Average stream/riparian habitat scores/ratings (e.g., 0

– 10)

- Change in turbidity from 2011 to 2016

- presence/absence/density of invasive or keystone

species














Low High- # of total stream

miles with presence of fish

- Change in # of stream miles with presence of fish

from 2011 to 2016

- # of total stream miles w/ fish

presence & water in critical periods

- Change of total stream miles w/ fish presence & water in critical periods from

2011 to 2016

- Detailed habitat conditions, e.g.,

geomorphology, hydrology

- Detailed water quality, e.g., turbidity, nutrients, toxics, microbial source

tracking

- Change in habitat/water quality conditions from

2011 to 2016


- Changes in riparian shade

cover from 2011 to 2016

- Changes in # river miles w/ water in

critical period














Low High


When will you do something with the monitoring

answer?

Anatomy of Monitoring in VSPFish & Wildlife Habitat


answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1

Background

Level

20 NTU



answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1

Background

Level

20 NTU

“Do

Something”

Level

30 NTU



answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1

Background

Level

20 NTU

“Do

Something”

Level

30 NTU

Where did

this data

come from?



answer?

Where did

this data

come from?

Each approach may have different:

Accuracy

Precision

Timescale (1/yr, 1/mth, 1/week vs. continuous)

Cost $

Staff Time


2010 2020

Turb

idit

y (

NTU

)

Station 1

Background

Level

20 NTU


2010 2020

Turb

idit

y (

NTU

)

Station 1

Background

Level

20 NTU


answer?

“Do

Something”

Level

30 NTU



answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1

Background

Level

20 NTU

“Do

Something”

Level

30 NTU

Where did

this data

come from?

2017 2018

3 samples/year

1 station

Ex: Secchi disk



answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1

Background

Level

20 NTU

“Do

Something”

Level

30 NTU

Where did

this data

come from?

2017 2018

Noise, error, accuracy,

precision of

instrument/approach

3 samples/year

1 station

Relates to standard

deviation



answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1

Background

Level

20 NTU

“Do

Something”

Level

30 NTU

Where did

this data

come from?

2017 2018

Noise, error, accuracy,

precision of

instrument/approach

3 samples/year

1 station

Trend line shows

reduction in turbidity

over time*

BUT

May not be better

than random

Relates to standard

deviation



answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1 2017 2018Trend line shows

reduction in turbidity

over time*

Relates to standard

deviation

• Microsoft Excel can create trend lines, but they are

not necessarily “statistically significant”

– i.e., not better than a random relationship given the

data

• How good does your data need to be?

A statistically significant difference indicates only that the difference is unlikely to have occurred by chance.



answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1

Additional

samples

What if there

were more

samples?



answer?

2010 2020

Turb

idit

y (

NTU

)

Station 1

What if there

were more

samples?

Could change interpretation of data

More likely to “see” what’s really going on

More likely to be better-than-random

Is the turbidity level really

going down?


2010 2020

Turb

idit

y (

NTU

)

Station 1 Where did

this data

come from?

2017 2018

Ex: Water sample analyzed

In a lab Noise, error, accuracy,

precision of

instrument/approach

Some methods reduce “noise” in data via increased

accuracy/precision

There are time and $ considerations

“How good of an answer to I need?”


2010 2020

Turb

idit

y (

NTU

)

Station 1 Where did

this data

come from?

2017 2018

Ex: Imhoff conesNoise, error, accuracy,

precision of

instrument/approach

Some methods reduce “noise” in data via increased

accuracy/precision

There are time and $ considerations

“How good of an answer to I need?”


2010 2020

Station 1

2010 2020

Station 1

No change in turbidity

Trend is significant (P < 0.05)

Decrease in turbidity

Trend is weakly significant

(P < 0.1)

2010 2020

Station 1

Increase in turbidity

Trend is not significant

(P > 0.05 or 0.1)

Low High

Number of Samples


2010 2020

Station 1

2010 2020

Station 1





(P < 0.1)

2010 2020

Station 1



(P > 0.05 or 0.1)

Regardless of method, in general, more samples:

- give best chance to see differences (if there)

- Reduce “noise”


2010 2020

Station 1

2010 2020

Station 1





(P < 0.1)

2010 2020

Station 1



(P > 0.05 or 0.1)

Still important to ask

“where, why, when?”


2010 2020

Station 1





“Where is Station 1?”

Green areas are

“agricultural activities”


2010 2020

Station 1






Green areas are


Surface water sub-watershed


2010 2020

Station 1






Green areas are



Is this station location meaningful for

VSP?


2010 2020

Station 1






Green areas are




VSP?

Turbidity here could be affected by

residential construction

Little or no “agricultural activities” in

the sub-watershed


2010 2020

Station 1






Green areas are




VSP?

Turbidity here could be affected by

residential construction

Little or no “agricultural activities” in

the sub-watershed

Example: “How are BMPs cover crops, tillage mgmt., and managed grazing affecting critical area fish and wildlife


“What’s the Question?”


2010 2020

Station 1






Stations selected to pick up the

“signal” of agricultural

activities are more likely to

result in actionable information


2010 2020

Station 1






Upstream/downstream stations

or multiple stations in selected

(sub)watersheds often improves

our ability to “see” what’s

happening to functions and

values of critical areas


2010 2020

Station 1

No change in butterfly counts


Sample locations should depend on the

question being asked

Upstream/downstream stations

may not be helpful if counting

butterflies

• What is the central tendency? (mean/average)

• How much noise? (variability/standard deviation)


2006 2016

Station 1

Background

Level

20 NTU

“Do

Something”

Level

30 NTU

Turb

idit

y (

NTU

)


• How much variability? (noise/standard deviation)

2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Baseline Comparison



2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

“Do

Something”

Level

30 NTU

Background

Level

20 NTU

Baseline Comparison



2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

“Do

Something”

Level

30 NTU

Background

Level

20 NTU

Baseline Comparison

Baseline

Average ~ 29 NTU



2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

Baseline Comparison

Baseline

Average ~ 29 NTU

100

50

100 samples were 29 NTU

10

10 samples were 20 NTU



2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

Baseline Comparison

Baseline

Average ~ 29 NTU

100

50

10



2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

Baseline Comparison

Baseline

Average ~ 29 NTU

100

50

10

When the variability is high,

difficult to determine central

tendency and/or differences



2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

Baseline Comparison

Comparison

Average ~ 22 NTUBaseline

Average ~ 29 NTU

Are they different?



2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

Baseline Comparison

Comparison


Average ~ 29 NTU

Are they different?



2006 2016

Station 1


Background

Level

20 NTU

“Do

Something”

Level

30 NTU

July 2011

Turb

idit

y (

NTU

)

Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

Baseline Comparison

Comparison


Average ~ 29 NTU

Are they different?




Turbidity (NTU)

Fre

qu

en

cy

of

sa

mp

les

Comparison


Average ~ 29 NTU

Are they different?

Standard deviation can help

make sense of the central

tendencies

When? How?

Desired accuracy (power)

Number of samples (sample size)

Observed mean (central tendency/average)


The 5-yr Report Template will prompt for this type of

information

Standard deviation (variability/noise)

Is observation statistically significant (P-value/better

than random?)




When? How?

Population vs. Sample


Worked ExampleFish & Wildlife Habitat

First,

let’s access some

data


Selected Samples:

• July 2006 to July 2016

• Streams and rivers

• Stations with >10 samples

• July 2006 to July 2011 = baseline (pre 2011)

• July 2011 to July 2016 = comparison (post 2011)


Selected Samples:

• July 2006 to July 2016

• Streams and rivers

• Stations with >10 samples

• July 2006 to July 2011 = baseline

• July 2011 to July 2016 = comparison


Many important elements of

“Is it better than random?”

can be calculated in Excel


Number of

Samples

Standard

Deviation

(noise)

Central

Tendency

(mean/average)


Worked Example

Dissolved Oxygen


answer?

How many samples do you need to

detect a difference?

More samples needed to detect

smaller differences

Worked Example

Turbidity


answer?

How many samples do you need to

detect a difference?

Depends on the “thing being

measured”

• Which of these are measurable at the scale and time frame of interest?

• Can I use surrogates?

Ask yourself:

• What are the functions of (x) critical area?• Which of these are of greatest interest (biologically?,• economically?, politically?)

Additional Staff and Support Regarding VSP Monitoring

Brian Cochrane (SCC) - [email protected]

Levi Keesecker (SCC) – [email protected]

Michel, Leslie (WSDA) - [email protected]

Williams, Sean R (WDFW) [email protected]

Regional representatives for WDNR, ECY, NRCS, DOH, etc

WSU, USGS – often have related research/projects in related areas

Monitoring Resources

Monitoring Tips for Local Voluntary Stewardship Workgroups (SCC) https://assets.website-

files.com/5ec2d4f7da309c68cdc0655a/5f57caeee815a00779b7f345_VSP-Monitoring- for-WGs.Feb-2017.pdf

What does Monitoring Look Like? A VSP Primer (SCC/WDFW) - https://uploads-

ssl.webflow.com/5f0507f9cf344a5f8752f27a/5f7e39783c18833bbb1ea7c7_VSP%20Reg

ional%20Fall%202018%20Monitoring%20PPT%20slides.pdf

Power Analysis, Sample Size, and Data Variability: Applications using Existing State/Federal Data (SCC) –

https://uploads- ssl.webflow.com/5f0507f9cf344a5f8752f27a/5f7e39bab9232515420f9027_PowerAnaly

sisSummaryMay10_2020.pdf

Statistical Analysis and Assumptions -

https://www.afit.edu/stat/statcoe_files/Model%20Building%20Process%20Part%201%

20Checking%20Model%20Assumptions%20V2.pdf

Field Sampling Design, Statistics, and Variability

http://unix.eng.ua.edu/~rpitt/Class/ExperimentalDesignFieldSampling/Module%205/M

5%20Statistical%20Analyses.pdf

https://assets.website-/

https://assets.website-files.com/5ec2d4f7da309c68cdc0655a/5f57caeee815a00779b7f345_VSP-Monitoring-for-WGs.Feb-2017.pdf

https://assets.website-files.com/5ec2d4f7da309c68cdc0655a/5f57caeee815a00779b7f345_VSP-Monitoring-for-WGs.Feb-2017.pdf

https://uploads-ssl.webflow.com/5f0507f9cf344a5f8752f27a/5f7e39783c18833bbb1ea7c7_VSP%20Regional%20Fall%202018%20Monitoring%20PPT%20slides.pdf



https://uploads-ssl.webflow.com/5f0507f9cf344a5f8752f27a/5f7e39bab9232515420f9027_PowerAnalysisSummaryMay10_2020.pdf



https://www.afit.edu/stat/statcoe_files/Model%20Building%20Process%20Part%201%20Checking%20Model%20Assumptions%20V2.pdf

https://www.afit.edu/stat/statcoe_files/Model%20Building%20Process%20Part%201%20Checking%20Model%20Assumptions%20V2.pdf

http://unix.eng.ua.edu/~rpitt/Class/ExperimentalDesignFieldSampling/Module%205/M5%20Statistical%20Analyses.pdf

http://unix.eng.ua.edu/~rpitt/Class/ExperimentalDesignFieldSampling/Module%205/M5%20Statistical%20Analyses.pdf

Documents

VSP Monitoring 101