AMSR-E and AMSR-E Validation Status at NSIDC

AMSR-E and AMSR-E ValidationStatus at NSIDC

Amanda Leon, NSIDC AMSR-E LeadJoint AMSR-E Science Team MeetingOxnard, CA4-5 September 2013

Outline

2

• Standard Products • Status• Metrics

• AMSR-E Related Products Status Metrics

• Data Access & Discovery • User Support• Data Quality Information• Long Term Archival

Sea ice derived from the AMSR-E 89 GHz Tbs. Image courtesy

of NASA/GSFC Visualization Studio

Standard Products: Status

• Level-2A Version 3 announced to public on 14 June 2013 Reprocessing completion announced 26 July 2013 Version 2 will be publicly available for 6 months post reprocessing (27 Jan)

• Level-3 Sea Ice Products Reprocessing initiated in September 2012 to integrate a new

Antarctic land mask. Integration of mask caused valid data in SH to be masked in 12.5 &

25 km Sea Ice Concentration and 6.25 Sea Ice Drifto Removed affected data and republished previous versiono Awaiting another reprocessing effort

3

nsidc.org/data/amsre

Archive Metrics: Volume

4

1 June 2002 – 31 July 2013

Stats from EMS

Current Version Archive Volume

Total Current Version Archive: 16.4 TB

Total Archive (V001, V002, V003): 28 TB

Archive retention plan:

• Keep the Current Version and the Previous Version of all products.

• Keep All Versions of L2A

Product Volume (GB) FilesVersion 1AE_SID 3.56 123Version 2AE_5DSNO 1.34 666AE_DYOCN 45.77 3360AE_DYSNO 6.75 3359AE_L2A 4981.65 97524AE_LAND 68.07 97451AE_LAND3 203.24 3361AE_MOOCN 1.53 112AE_MOSNO 0.22 112AE_OCEAN 1163.64 97474AE_RAIN 1039.80 97471AE_RNGD 0.02 112AE_SI12 194.87 3360AE_SI25 64.22 3360AE_SI6 291.06 6725AE_WKOCN 6.58 483Version 3AE_L2A 5089.10 98162AMSREL1A 3610.31 98146

Distribution Metrics: Total

5

1 June 2002 – 31 July 2013

Stats from EMS

Total Distribution:589 TBs

24.3 million files

Product Files Volume (TB)AE_L2A 13,083,100 491.05AE_RAIN 3,158,305 64.64AE_LAND 1,859,228 29.46AE_OCEAN 1,773,523 24.57AE_LAND3 1,541,277 19.28AE_SI12 769,340 16.76AMSREL1A 670,238 7.86AE_DYSNO 583,694 3.45AE_SI25 262,371 1.46AE_SI6 243,321 1.21AE_DYOCN 212,409 0.86AE_5DSNO 94,481 0.13AE_MOSNO 20,137 0.10AE_WKOCN 12,561 0.05AE_RNGD 10,308 0.03AE_MOOCN 8,216 0.02AE_SID 8,058 0.00Grand Total 24,310,567 589.24

Sorted on Volume

Distribution Metrics: Files

6

Jan 2004 – July 2013 File Distribution

Stats from EMS

2004 2005 2006 2007 2008 2009 2010 2011 2012 20130

500,000

1,000,000

1,500,000

2,000,000

2,500,000

3,000,000

3,500,000

4,000,000

4,500,000

5,000,000

AE_SID

AE_MOOCN

AE_RNGD

AE_WKOCN

AE_MOSNO

AE_5DSNO

AE_DYOCN

AE_SI6

AE_SI25

AE_DYSNO

AMSREL1A

AE_SI12

AE_LAND3

AE_OCEAN

AE_LAND

AE_RAIN

AE_L2A

201101

201102

201103

201104

201105

201106

201107

201108

201109

201110

201111

201112

201201

201202

201203

201204

201205

201206

201207

201208

201209

201210

201211

201212

201301

201302

201303

201304

201305

201306

2013070

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

Distribution Metrics: Files

7

Antenna Spin Down10/4/2011

Jan 2011 – Jul 2013 Files/Month

Dec 2012: L2A & L2B

Land, Ocean, Rain

Jan 2013: L2A

Mar 2013: L2A & L2B Rain

Distribution Metrics: Volume

8

2004 2005 2006 2007 2008 2009 2010 2011 2012 20130

20,000

40,000

60,000

80,000

100,000

120,000 AE_RNGD

AE_SID

AE_MOSNO

AE_MOOCN

AE_WKOCN

AE_5DSNO

AE_DYSNO

AE_DYOCN

AE_LAND

AE_SI25

AE_SI6

AE_OCEAN

AMSREL1A

AE_RAIN

AE_SI12

AE_LAND3

AE_L2A

Jan 2004 – July 2013 Volume (GB) Distribution

201101

201102

201103

201104

201105

201106

201107

201108

201109

201110

201111

201112

201201

201202

201203

201204

201205

201206

201207

201208

201209

201210

201211

201212

201301

201302

201303

201304

201305

201306

2013070

5,000

10,000

15,000

20,000

25,000

Distribution Metrics: Volume

Joint AMSR-E Science Team Meeting

Amanda Leon - NSIDC DAAC 4-5 September 20139

Jan 2011 – Jul 2013 GB/Month

Antenna Spin Down10/4/2011

Jan 2013: L2A & L3 Land

Mar 2013: L2A & L3 Land

Distribution Metrics: Users

10

1 June 2002 – 31 July 2013

Stats from EMS & USO TRAK Database

2,519 Total Distinct Users

U.S. Users – 46%

Foreign Uses - 54%

Product Distinct Users

AE_LAND3 864

AE_L2A 802

AE_SI12 398

AE_DYSNO 366

AE_LAND 334

AE_RAIN 210

AE_5DSNO 209

AE_SI25 190

AE_OCEAN 185

AE_MOSNO 143

AE_SI6 134

AE_DYOCN 115

AMSREL1A 107

AE_RNGD 89

AE_MOOCN 59

AE_WKOCN 31

AE_SID 7

User Type Distinct Users

FOREIGN OTHER 699

FOREIGN UNIVERSITY 504

US UNIVERSITY 493

OTHER 151

NASA 150

ECS OTHER 137

FOREIGN GOVERNMENT 121

US OTHER 101

US COMMERCIAL 38

FOREIGN COMMERCIAL 31

NOAA 29

US FEDERAL GOVERNMENT 28

US GOVERNMENT 16

US STATE/LOCAL GOVERNMENT 11

US GENERAL PUBLIC 8

WDC 2

Distribution Metrics: Users

11

Stats from USO TRAK Database

Jan 2003 – Jul 2013 Total Distinct Users per Year

2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 20130

100

200

300

400

500

600

52

198

336

429

493510

466

414

458

258

165

Distribution Metrics

12

Top 10 Countries by Fiscal Year (by Volume, Files, and Users)

FY

201

2F

Y 2

013

china376

united states127

other (31 countries)54

colombia27

india27

canada8

korea, republic of8

italy7

japan6

netherlands6

russian federation6

FY 2013 End-User Distribution for AE_LAND3 (Users)Rank = 1

china220

united states114


india18

canada16

japan12

germany11

united kingdom

9

argentina7

russian federation7

FY 2013 End-User Distribution for AE_L2A (Users)Rank = 2

united states56

china48


canada14

japan11

united kingdom9

norway7

germany5

korea, republic of5

FY 2013 End-User Distribution for AE_SI12 (Users)Rank = 3

united states979,385

china468,300

canada467,005

other (22 countries)91,706 argentina

26,579

germany25,707

taiwan12,849

denmark10,180

japan9,290

FY 2013 End-User Distribution for AE_L2A (Files)Rank = 1 by Files

united states122,396

china69,413

korea, republic of11,682

other (25 countries)10,657

canada7,701

united kingdom

5,861saudi arabia

4,626

netherlands3,869

FY 2013 End-User Distribution for AE_LAND3 (Files)[Rank = 6 by Files]

AMSR-E Related Products

AMSR-E Validation Data nsidc.org/data/amsr_validation/

• No changes

Other AMSR-E Related Products• John Kimball’s MEaSUREs Global Record of

Freeze/Thaw nsidc.org/data/nsidc-0477.html

Version 2 Release

• AMSR-E Global Land Surface Emissivity nsidc.org/data/nsidc-0543.html

Hamidreza Norouzi; New York City College of Technology, The City University of New York (CUNY)

• On the horizon: Aquarius Soil Moisture; AMSR2; SMAP

18

http://nsidc.org/data/amsr_validation/



http://nsidc.org/data/nsidc-0477.html


AMSR-E Validation: Distribution Metrics

19

Total Distinct Users Per YearJan 2011 – July 2013

Statistics from our University FTP logs

2011 2012 20130

200

400

600

800

1,000

1,200

1,400

1,600

Soil MoistureRainfallCryosphere


20

Total Files Distributed Per YearJanuary 2011 - July 2013

Statistics from our University FTP logs

2011 2012 20130

50,000

100,000

150,000

200,000

250,000



21

Total Volume (MB) Distributed Per YearJanuary 2011 – July 2013

Statistics from FTP logs

2011 2012 20130

100,000

200,000

300,000

400,000

500,000

600,000

700,000

800,000


Other AMSR-E Related Products

ID Authors TitleNSIDC-0301 Knowles,

Savoie, Brodzik, Armstrong

AMSR-E/Aqua Daily EASE-Grid Brightness Temperatures

NSIDC-0302 Knowles, Savoie, Brodzik, Armstrong

AMSR-E/Aqua Daily Global Quarter-Degree Gridded Brightness Temperatures

NSIDC-0450 Tedesco, Miller

Co-Registered AMSR-E, QuikSCAT, and WMO Data

NSIDC-0451 Jones, Kimball Daily Global Land Surface Parameters Derived from AMSR-ENSIDC-0464 Long, Stroeve Enhanced-Resolution SSM/I and AMSR-E Daily Polar Brightness Temperat

ures

NSIDC-0477 Kimball, Kim, Glassy, McDonald

MEaSUREs Global Record of Daily Landscape Freeze/Thaw Status

22









Other AMSR-E : Distribution Metrics

23

.25 Deg TBs Co-Reg WMO EASE TBs F/T High Rez TBs Land Surf. Parm.

0

200,000

400,000

600,000

800,000

1,000,000

1,200,000

1,400,000

2007200820092010201120122013

Jan 2007 – Jul 2013 Files per Year

Other AMSR-E: Distribution Metrics

Joint AMSR-E Science Team Meeting

Amanda Leon - NSIDC DAAC 3-4 September 201324


0

500,000

1,000,000

1,500,000

2,000,000

2,500,000

2007200820092010201120122013

Jan 2007 – Jul 2013 MBs per Year

Other AMSR-E: Distribution Metrics

25


0

50

100

150

200

250

300

350

400

2007200820092010201120122013

Jan 2007 – Jul 2013 Distinct Users per Year

Discovery & AccessData Pool GUI Retired• HDF-EOS to GeoTIFF (HEG) data services now accessible in Reverb• Data Pool FTP site remains for direct data access

REVERB reverb.echo.nasa.gov

• FTP download All Products

• Data services Most Standard Products Spatial & Parameter Subsetting, Reformatting to GeoTIFF, Projection Conversion

OpenDAP nsidc.org/opendap/ Standard Products

• Provides data variable display and output to ASCII and binary

Polaris http://nsidc.org/data/polaris/ EASE-Grid & .25 deg TBs

• Data services: Spatial & Parameter Subsetting, Reformatting, Projection Conversion

26

nsidc.org/data/methods.html

http://reverb.echo.nasa.gov/reverb/granules?utf8=%E2%9C%93&spatial_map=satellite&spatial_type=rectangle&keywords=ae_l2a&datasets=C98817885-NSIDC_ECS

http://nsidc.org/opendap/



http://nsidc.org/data/polaris/

http://nsidc.org/data/polaris/

Discovery & Access: NSIDC Data Search

27

Metadata Project – Completed • Improved metadata efficiency and accuracy • Better data search results• More complete data documentation

NSIDC Search – In Work• User feedback: Challenging to

find and select data which best meets needs

• Plans: • Improved search algorithms• Faceted browse• Data comparison

• Timeframe: Initial release before end of 2013

User Support

28

User Services Knowledge Base support.nsidc.org/forums

• Dynamic FAQs and other data support

The Drift nsidc.org/the-drift/

• Consolidated online data announcements, data highlights, tools, and tips into a blog-style Web site

• Replaces NSIDC Notes

Most Common User Questions1. Provide guidance on which data set to use or which data within a data set2. Provide guidance on information in the documentation3. How do I open/read [data set] in [format] with [tool]?4. How do I convert from [x data format or x projection] to [y data format or y

projection]?5. Is there a way to extract data for only the region I need in [data set]?

nsidc.org/data/amsre

https://support.nsidc.org/forums

http://nsidc.org/the-drift/

Data Quality Information

• Aqua Senior Review Committee requested a centralized source of AMSR-E uncertainly information.

• NSIDC agreed to host an "AMSR-E Data Product Confidence" Web page Synopsis on sources of uncertainty. The best estimates of data uncertainty under optimal conditions, expressed

in terms appropriate to each measurement. Where possible, express this as "95% confidence interval".

A description of how to interpret quality flags to understand the conditions under which uncertainty may be greater.

• Completed: Sea Ice, SWE, Soil Moisture

• Suggestion by Marco: develop a 'standardized' measure so that it would be easier to look at the results of the survey.

29

Long Term Archival• ESDIS developed the NASA Earth Science Data Preservation

Content Specification (423-SPEC-001, Nov 2011) “This document defines the contents of data, metadata and associated

documentation to be preserved beyond the life of missions funded by NASA’s Earth Science Division (ESD).”

• NSIDC will coordinate the collection and archival of material Doug Fowler (ICESat Data Management Lead)

• Science Team, TLSCF, SIPS, and JAXA will be essential to the LTA collection process Need to capture knowledge before people move on

• Documents should be archived in a persistent format, like PDF are either submitted to the NASA Technical Reports Server (NTRS) or become

an unofficial part of the LTA

30

Long Term Archival

ESDIS LTA Category Description

Preflight/Pre-Operations * Instrument/Sensor characteristics including pre-flight/pre-operations performance measurements; calibration method; radiometric and spectral response; noise characteristics; detector offsets

Science Data Products Raw instrument data, Level 0 through Level 4 data products and associated metadata

Science Data Product Documentation

Structure and format with definitions of all parameters and metadata fields; algorithm theoretical basis; processing history and product version history; quality assessment information

Mission Data Calibration * Instrument/sensor calibration method (in operation) and data; calibration software used to generate lookup tables; instrument and platform events and maneuvers

Science Data Product Software

Product generation software and software documentation

Science Data Product Algorithm Input

Any ancillary data or other data sets used in generation or calibration of the data or derived product; ancillary data description and documentation

Science Data Product Validation

Records, publications and data sets

Science Data Software Tools

Product access (reader) tools

31 * Proved difficult to collect with ICESat

AMSR-E Team at NSIDC

32

Thanks to the AMSR-E Team at NSIDC!

AMSR-E

Chris Bond – Operations

Karla LeFevre – Technical Writer

Shannon Leslie– User Services

Siri Jodha Khalsa – ECS Science Liason

? Walt Meier @ GSFC now – Scientist

Cathy Fowler – Database Administrator