Facilitating end-user access to VIIRS data

PIs: Cara Wilson, Roy Mendelssohn & Dave Foley

NMFS/SWFSC Environmental Research Division

Pacific Grove, CA

JPSS Science Seminar, January 27, 2014 1

RIP

DAVID G. FOLEY

1966-2013

Dave passed away Dec 8, 2013

Memorial website: http://www.forevermissed.com/david-g-foley 2

• Two primary components: 1) Facilitate access to VIIRS data by

(a) placing it on the ERDDAP server at the SWFSC ERD facility

(b) revising the EDC tool (an ArcGIS interface to ERDDAP) to work with telemetry data

2) Education, training and outreach via the 2013 NOAA ocean satellite course

• Targets two broad groups of users: ArcGIS users, and marine biologists who work with tagged animal data

Facilitating end-user access to VIIRS data

PIs: Cara Wilson, Roy Mendelssohn (& the late Dave Foley) NMFS/SWFSC/ERD

3

• Two primary components: 1) Facilitate access to VIIRS data by

(a) placing it on the ERDDAP server at the SWFSC ERD facility

(b) revising the EDC tool (an ArcGIS interface to ERDDAP) to work with telemetry data

2) Education, training and outreach via the 2013 NOAA ocean satellite course

• Targets two broad groups of users: ArcGIS users, and marine biologists who work with tagged animal data

Facilitating end-user access to VIIRS data

PIs: Cara Wilson, Roy Mendelssohn (& the late Dave Foley) NMFS/SWFSC/ERD

✔

✔

4

Part I:

ERDDAP (Environmental Research Division’s Data Access Program)

Facilitating end-user access to VIIRS data

PIs: Cara Wilson, Roy Mendelssohn (& the late Dave Foley) NMFS/SWFSC/ERD

5

What is ERDDAP? • The Environmental Research Division’s Data Access Program. • Developed at SWFSC/ERD by Bob Simons. • ERDDAP is a web application (for humans with browsers) and a web service

(with services for computer programs) designed to provide easier access to datasets for both people and machines.

• Multiple data formats are available (ncdf, grib, csv, ESRicsv, JSON, ODVtext, mat, text and more)

• Multiple image files formats are available (png, transparent png, pdf, kml) • Allows temporal and spatial subsetting • ERDDAP is “RESTful”, meaning the URL completely defines the data you

want, in the format you want. • Its better demonstrated than described • http://coastwatch.pfeg.noaa.gov/erddap

6

Without ERDDAP

7

With ERDDAP

8

9

Since March 2013 daily, weekly and monthly composites of global fields of VIIRS chlorophyll, Diffuse Attenuation Coefficient K490 and Remote Sensing Reflectance at 671 nm from NASA/OBPG have been available on ERD’s THREDDS and ERDDAP servers

Part II:

EDC (Environmental Data Connector)

Facilitating end-user access to VIIRS data

PIs: Cara Wilson, Roy Mendelssohn (& the late Dave Foley) NMFS/SWFSC/ERD

10

The Problem Satellite data have been largely inaccessible for those working with GIS tools, such as fisheries scientists and marine resource managers. Without going through the painful experience of bringing satellite data into ArcGIS using only the native tools, it is hard to comprehend how truly onerous this task can be. 11

The Solution

• Allows easy selection, and importation, of both gridded and non-gridded datasets served by THREDDS, OPeNDAP, SOS, or ERDDAP

• Works with ArcGIS 9.2, 9.3 or 10, Matlab, R and Excel • Data can be subset temporally and spatially • Developed by Applied Science Associates, Inc. (ASA) • Funded by the NOAA R&O Tiger team, IOOS, the NOAA IEA

Program and the JPSS RRPG program • Free distribution, available since May 2008

www.pfeg.noaa.gov/products/EDC 12

The Solution Is it really necessary?

“I had been struggling with satellite data in ArcGIS for three or four years. I found several workarounds. But they all tended to be very laborious, and they worked only marginally. On any given satellite data project, I was spending 90% of my time just getting the data into a usable format for ArcGIS. It was painful. So I was more than a little skeptical about the EDC product. I thought if it saves a little time, it might be worth it. Then in the first 10 minutes of using it I was amazed. Not only did it work, but it seamlessly pulled the data into ArcGIS directly from the OpenDap servers. It turned all-day data import, cleanup, and temporal animation exercises into a 10-minute operation. That is not an exaggeration. Not only does it pull the data in cleanly, but it is already structured to temporally animate, which is a huge step forward. I think that everybody who uses it will be thrilled with the way it handles temporal data. It is a must-have for anybody working with satellite data. Thanks NOAA.” Jonathan Kamler* 17th US Coast Guard District *graduate of the 2008 NOAA Satellite Course

13

EDC Modifications proposed for this project

• Get the EDC to work with the new versions of Arc (V10) & Java • Simplify the install process • Get the EDC to work with ArcGIS V10.2 • Enable the EDC to extract data along a user-defined track • Enable the EDC to extract data from a user-defined polygon • Add search capabilities to the EDC • Develop a web-based version of the EDC (to be used by people

who are not allowed to install software on their computers, which is fairly common in the government)

A version released June 2013 with these changes ✔ ✔

Our initial cost estimate was too low to make all the desired revisions to the EDC

+ $6.5K

+ $17K

14

Work done via a contract to ASA

15

Bailey et al. [2012] Identification of distinct movement patterns in Pacific leatherback turtle populations influenced by ocean conditions. Ecological Applications, 22(3), pp. 735–747

135 tags, with an average of 234 days of data per tag

NMFS/SWFSC

Part III:

The 2013 NOAA Satellite Course

Facilitating end-user access to VIIRS data

PIs: Cara Wilson, Roy Mendelssohn (& the late Dave Foley) NMFS/SWFSC/ERD

16

We VIIRS!!! • Funding from the JPSS Proving Ground and Risk Reduction

initiative paid for participant travel to the 2013 NOAA ocean satellite course, through a grant designed to facilitate end-user access to VIIRS data.

• Thanks to Ted Strub and (the now defunct) CIOSS (Cooperative Institute for Oceanographic Satellite Studies) for paying for the use of the Digital Earth classroom

17

NOAA Satellite Data Training Course

• Course was developed by Cara Wilson and Dave Foley, both at NMFS/SWFSC/ERD, in conjunction with Ted Strub at the (now defunct) Cooperative Institute for Oceanographic Satellite Studies (CIOSS) at Oregon State University in Corvallis, OR.

• Conducted almost every year since 2006: DATE LOCATION # Aug 20-22, 2013 CIOSS/OSU, Corvallis, OR 35* Mar 13-15, 2012 NOS/CSC, Charleston, SC 7 Mar 23-25, 2011 CIOSS/OSU, Corvallis, OR 26 Aug 9-11, 2010 MATE/MPC, Monterey, CA 19 Aug 2-4, 2010 MATE/MPC, Monterey, CA 22 Mar 24-26, 2008 CIOSS/OSU, Corvallis, OR 19 Mar 26-28, 2007 CIOSS/OSU, Corvallis, OR 28 Aug 22-24, 2006 CIOSS/OSU, Corvallis, OR 31 *Funded by JPSS PGRR Initiative

18

NOAA Satellite Data Training Course

• 3-day (free!) course aimed at NMFS or NOS participants who want to learn how to access & use satellite data

• Objective is to help people access and use satellite data in the environment they are used to working in – a challenging task! Focus has been on GIS, Matlab and R applications.

• Participants bring projects to work on.

• Course initiated by funding from NOAA’s R&O project in 2006. The course was conducted with no funding support 2007-2012, funding was obtained for the 2013 course.

• The learning experience goes two ways. From conducting these courses we get a better idea of users’needs and wants, and are better able to address those needs. 19

Course Participants

176 participants since 2006

Other Federal or State agencies OSU Other Universities Private Sector

20

Course Participants

129 NOAA participants since 2006

11 of the 14 different marine sanctuaries have participated

NMFS 78%

NOS 19%

21

Course Participants

101 NMFS participants since 2006

22

Applicants & participants

NMFS – 20

NOS - 6

OSU – 8

Private sector – 1

35 Participants

23

2013

NOAA Ocean Satellite Data Course

NOAA: 26 NMFS: 20

AFSC: 2 NEFSC: 6 NWFSC: 3 SEFSC: 3 SER: 3 SWFSC: 4 PIFSC: 1

NOS: 6 NCCOS: 2 ONMS: 2 ORR: 2

*OSU: 8 *Industry: 1 (auditing)

*JPSS funds were used to pay the travel costs of NOAA participants only

35 Participants

24

2013

NOAA Ocean Satellite Data Course

Participant’s Software Usage

• Our objective is to help people access satellite data and use it in the environment they are used to working in.

• We emphasis the use of tools and services developed at NMFS/SWFSC/ERD, including our CoastWatch browser, ERDDAP, x-tractomatic tools for Matlab & R, and the EDC.

• This year we wrote a “user’s manual” to complement the course instruction.

2013

NOAA Ocean Satellite Data Course

25

How does the “wet” side of NOAA plan to use VIIRS data?

Feedback from the 2013 NOAA Ocean Satellite Data Course

26

Some statistics

• 34 participants in the 2013 NOAA Satellite Data Course

• 23 returned an evaluation form (66%) Of those 23: 20 had never heard of VIIRS data before (87%) 21 plan on using VIIRS data (91%)

• 9 (27%) contributed a powerpoint slide illustrating usage

27

Not a complete photo – 9 people missing 28

Where are the turtles???

I assist with aerial surveys, looking for leatherback turtles off the west coast of the US. One of the main things I’ve learned from those surveys is that it’s very hard to find a few turtles in a big ocean!

My hope is that data from VIIRS will help us find the best places to look

Good turtle habitat

Dan Prosperi, NMFS/SWFSC/PRD 29

-10

0

10

20

30

40

50

60

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012

Age-

2 sa

blef

ish (

mill

ions

)

Sablefish

Predicted

Residuals

Developing ecological indicators for sablefish recruitment

ST(t-2) Chl(t-2) Sable(t-1)Beta 0.37 1.16 -0.33S.E. 0.14 0.14 0.11t-test 2.59 8.14 -3.07Prob(t) 0.03 0.00 0.01

Figure 1. Age-2 sablefish modeled as a function of Chl-a (t-2), sea temperature (ST) (t-2), and age-2 sablefish (t-1) with data collected in situ, 1999-2010 (Martinson et al. 2013).

Coastal rearing habitat for young sablefish

Objectives 1. Support an ecosystem approach to management 2. $ 142 million fishery for sablefish in U.S. 3. Develop indicators for sablefish recruitment 4. Use satellite color data to index chl-a, blooms 5. Quantify blooms in rearing areas 6. Link to future sablefish recruitment

High age-2 recruitment in 2002 was linked to high chlorophyll-a in the late summer in 2000.

http://www.afsc.noaa.gov/ABL/MESA/mesa_default.php

Sablefish (Anoplopoma fimbria)

Ocean survey results

Future: Use satellite data to develop spatial and temporal indicators of ocean productivity for sablefish rearing areas.

Model output

Ellen Martinson, NMFS/AFSC Auke Bay Laboratories 30

Biogeographical assessment of new MPA in Puerto Rico

Will Sautter, NOS/NCCOS

• Very interested in using 750m ocean color and SST data from VIIRS • Through the course was put in contact with Mark Eakin, who has a JPSS PGRR funded project

to develop high resolution VIIRS products for coral reefs • Need data in a format that is easy to pull into ArcGIS

31

C. Hayslip, MMI

By integrating satellite-derived locations of gray whales with remotely-sensed oceanographic variables (SST, VIIRS Ocean Color, SVW) we hope to develop predictive habitat models for these animals in the Pacific Northwest. Such models will be very useful in siting decisions for marine hydrokinetic energy development being proposed for the area.

Gray whale habitat along the Pacific Northwest coast Barbara Lagerquist, Oregon State University Marine Mammal Institute

Barbara Lagerquist, OSU 32

NOAA Ocean Satellite Data Course August 20-22, 2013

Oregon State University, Corvallis, OR

How does wind affect commercial and recreational fishing behavior? Methods: I will use the Environmental Data Connector (EDC) to download daily satellite wind climatologies into ArcGIS (see left) for a ten-year period. I will then summarize the percentage of days over 15 knots (~ 8m/sec), Where effective fishing operations are likely constrained. The percentage of windy days will be spatially assigned to 547 blocks.

*Wind is just one model covariate that I will use for my analysis. Now that I can easily access satellite data, I could also use SST or CHL-a from VIIRS.

Each block will be summarized for total recreational and commercial catch from 1930-2010 where wind will be a potential model covariate.

Rebecca Miller, GIS Analyst, NMFS, Santa Cruz, CA. rebecca.miller@noaa.gov

m/s

March 2, 2009

Rebecca Miller, NMFS/SWFSC 33

Ecosystem monitoring in the Gulf of Mexico

1998 El Nino anomaly?

2005 severe red tide event

Chlorophyll concentrations for the West FL Shelf – weekly means by year

Mandy Karnauskas, NMFS/SEFSC 34

Is chlorophyll-a correlated with loggerhead sea turtle distribution? • VIIRS

chlorophyll-a 8-day average for 06-01-2013

• Tracks of tagged loggerhead sea turtles for the same week

Melissa Warden, NMFS/NEFSC U.S. Department of Commerce | National Oceanic and Atmospheric Administration | NOAA Fisheries | Page 35

Early Marine Ecology of Juvenile Chinook Salmon on the Yukon Delta

• Evaluate the spatial distribution and feeding ecology of juvenile salmon in the Yukon River plume

• Assess abiotic factors affecting fish community composition in Yukon River plume

• Satellite data will be used with in situ oceanographic measurements to track plume dynamics during FY14 and FY15 sampling seasons

Katharine Miller, NMFS/AFSC

Yukon river delta and plume as viewed from the Landsat 7 satellite on Sept. 22, 2002. (NASA Earth Observatory)

36

Comparison of In-situ Chl-a and VIIRS Chl-a David S. Hanisko and John Quinlan, NOAA/NMFS/SEFSC

y = 0.5012x R² = 0.726 R=0.852 N=160

0

2

4

6

8

10

12

14

0 2 4 6 8 10 12 14

VIIR

SN D

aily

4km

Chl

orop

hyll

a

In Situ Extracted Chlorophyll-a

In-situ chlorophyll (chl-a) data was collected during Spring and Fall SEAMAP (Southeast Area Monitoring and Assessment Program) Plankton Surveys conducted by the NMFS Southeast Fisheries Science Center. Surface chl-a was extracted from triplicate 200 ml seawater subsamples at each station using a modified Welshmeyer method and was reported as the average value of the subsamples. 562 observations of in-situ chl-a from the 2012 and 2013 surveys coincided with the operational status of the VIIRS sensor from the Suomi National Polar-Orbiting Partnership (NPP) Mission. Daily observations from 160 in-situ chl-a samples were matched in time and space with VIIRS daily sensor data. In-situ and VIIRS chl-a data were highly correlated (r=0.852). The majority of samples available for comparisons were primarily from open ocean observations with low chl-a values and very few observations from areas of high primary productivity.

Feb. 2013

NMFS/SEFSC collects chl-a data as part of plankton surveys every spring and fall. They are willing to give their data to the VIIRS cal/val effort if additional data is needed. 37

Comparison with glider data

Cara Wilson, NMFS/SWFSC/ERD

Villareal & Wilson, A comparison of the Pac-X trans-Pacific Wave Glider data and satellite data (MODIS, Aquarius, TRMM and VIIRS), PlosOne, in revision, January 2014

Comparison of Pac-X trans-Pacific wave Glider fluorescence data to VIIRS chl-a data

38

Current/Future Plans

Facilitating end-user access to VIIRS data

PIs: Cara Wilson, Roy Mendelssohn (& the late Dave Foley) NMFS/SWFSC/ERD

• We are working on hiring a replacement for Dave Foley.

• Now that the VIIRS SST data have been deemed provisional (Jan 7 2014) we will start the process of making them available on our ERDDAP server.

• When the MSL12 Ocean Color data are available from CoastWatch we make them available on our ERDDAP server.

• Work continues (through our contract with ASA) on the improvements to the EDC. Next version expected out in April 2014.

• We acquired some funds through the JPSS Visiting Science Program to support the travel of 5 participants to the 2014 NOAA Ocean Satellite Course.

39

40

Questions?