Science Review Panel Meeting Biosphere 2, Tucson, AZ - January 4-5, 2011 Vegetation Phenology and Vegetation Index Products from Multiple Long Term Satellite

Science Review Panel MeetingScience Review Panel MeetingBiosphere 2, Tucson, AZ - January 4-5, 2011Biosphere 2, Tucson, AZ - January 4-5, 2011

Vegetation Phenology and Vegetation Index Products from Multiple Long Term Vegetation Phenology and Vegetation Index Products from Multiple Long Term Satellite Data RecordsSatellite Data Records

ESDRs Distribution and User ESDRs Distribution and User Support – Miscellaneous TopicsSupport – Miscellaneous Topics

Calli JenkersonERT, USGS EROS Center

NASA MEASURES #NNX08AT05A

U.S. Department of the InteriorU.S. Geological Survey

MEaSUREs VIP Miscellaneous TopicsJanuary 2011

Calli Jenkerson, ERTUSGS EROS Center

Miscellaneous Topics Beta ESDR File Transfer Metrics GMCD Entry ESDR Metadata Long Term Archive ESDR Access

MEASURES VIP ESDRs Science Review Panel-3-

Beta ESDR File Transfer

Expectations Test data set characteristics Infrastructure Schedule Current test results Plans Questions


Beta ESDR File Transfer ESDR production at University of Arizona Storage and distribution at USGS EROS

Expected final totals: Number of Files <=87,656 Uncompressed Volume ~5.7 TB Compressed Volume ~1.5 TB

based on 30 year record 2 daily products (VI, Phenology) 4 data sets (AVHRR2, AVHRR3, MODIS, SPOT)



Data Set File Format

Time Range # Files

Volume (GB)

AVHRR V2 (from LTDR2) HDF 1998001-1998365

365 88

AVHRR V3 (from LTDR3) HDF 1998001-1998365

365 63

MOD09CMG HDF 2003001-2003365

359 121

SPOT_CMG HDF 1998091-1998358

98 37

TOTAL 1,187 309

Beta collections available as of June 2010


Beta ESDR File Transfer Utilize FTP

UofA pushes to public staging area at EROS ~2 TB disk Unique VIP user

EROS moves files across firewall to VIP storage ~6 TB disk


Beta ESDR File Transfer Schedule

Roughly once every year July 20 - August 3, 2009 June 7-11, 2010 June 2011 December 2011? June 2012?

As data are available



Test results

2009 Transfer Test Alpha data collection Pushed 230 GB Obstructed by limited connections (5) to staging area <1 Mbs average throughput


Beta ESDR File Transfer 2010 Transfer Test

Beta data collection Pushed 309 GB Initiated multiple sessions to maximize number of

connections to staging area ~1.7 Mbs continuous throughput

Transfer Statistic June 7-11, 2010Total FTP Transactions 1243

Total Transaction Time June 7 12:07:09 CST – June 11 17:20:19

Total Uploaded Gigabytes 308.07 GB

Total Continuous Upload Time Hours 61.66 Hr

Average Continuous Upload Rate Kilobytes per second (3 connections)

1692 Kbs


Beta ESDR File Transfer Perform transfers to the EROS VIP environment as

ESDR data are available

Not likely to explore performance enhancement Timeliness of public product availability is not

currently an issue for this historical collection Future plans to archive ESDRs in LP DAAC will

not use the beta transfer model for ingest



Questions?


Metrics

Expectations Tools Record of Report ESDSWG-MPARWG Plans Questions


Metrics Report project metrics as appropriate to mark

progress and results Ingest metrics: volume and number of files stored

in EROS VIP systems

Distribution metrics: Web site hits, distinct users, services invoked, file downloads

User metrics: registered users, user services contacts, method of contact, subject of inquiry, escalation level, response time


Metrics

Earth Science Data Systems (ESDS) Metrics Collection Tool (MCT) Established for REASoN projects Carried over for MEaSUREs

Earth Science Data and Information System (EOSDIS) Metrics System (EMS) Established for Earth Observing System (EOS)

data centers Potential for implementation of EMS-MEaSUREs


Metrics

VIP Metrics Reporting Record: ESDS MCT entry 2009 July 2010 July, September, November Ingest and Distribution metrics only

Describes primarily test community activity User metrics will be added as Project-specific after

public release of ESDRs


MetricsVIP Client Ingest Metrics (since June 2009)

Data Set Layers in Client

Time Range Total # Files

Total GB

AVHRRV2 (from LTDR2)

NDVI, EVI2 1998001-1998365 (daily) 365 88

AVHRRV3 (from LTDR3)

NDVI, EVI2 1998001-1998365 (daily) 365 63

MOD09CMG NDVI, EVI2 2003001-2003365 (daily) 359 121SPOT_CMG NDVI 1998091-1998358 (daily) 98 37MOD13C2 NDVI, EVI 2000061-2009273 (monthly) 114 2eMODIS CONUS

NDVI 2000049-2009278 (weekly) 457 180

TOTAL 1,758 files

491 GB


MetricsVIP Client Distribution Metrics

(since June 2009)38 Web Page Visitors

43 Distinct Users

31 Domain .gov

12 Domain .edu

49 Products Delivered

5.13 GB Delivered

18,547 Services Provided


Metrics

Solicitation requirement to participate in the Earth Science Data System Working Groups (ESDSWG)

VIP has interested parties in Standards and Interfaces, Technology Infusion, Architecture and Reuse Working Groups

VIP is a contributing member of the Metrics Planning and Reporting Working Group (MPARWG)


Metrics

Attended ESDSWG Meetings 2009, 2010 Participated in ESDSWG MPARWG Telecons

2009, 2010 Contributions:

Experiences Goals Feedback Recommendations


Metrics

Continue membership in MPARWG Respond as needed to MPARWG chairs Continue quarterly reporting in ESDS MCT Refine metrics collection methods to more

precisely express services rendered


Metrics

Questions?


GCMD Entry

Global Change Master Directory NASA sponsored repository for Earth and

environmental science information Data sets Data services

Intended to consolidate such things into a one-stop-find-everything-site


GCMD Entry Uses keywords to navigate through a topical

database Every NASA product is described by

Entry ID Summary Geographic Coverage Data Set Citation Temporal Coverage Location / Science Keywords ISO Topic Category

Platform Instrument Originating Center Data Center Distribution Contact Pubs/Refs


GCMD Entry

Adding a data set is a simple matter of completing a web form

Approval for addition is at GCMD discretion

Preference given to data sets with: Compliant metadata (ECS or ISO) Relevance to at least one of about 1,500 science

topics


GCMD EntrySample GCMD Entry


Documents

Science Review Panel Meeting Biosphere 2, Tucson, AZ - January 4-5, 2011 Vegetation Phenology and Vegetation Index Products from Multiple Long Term Satellite