EOSDIS Survey Overview

Carol.L.Boquist@nasa.govHDF and HDF-EOS WorkshopNov. 4, 2009

Why we survey

NASA’s Earth Observing System Data and Information System provides data products and associated services for interdisciplinary studies to a diverse user community.

Beginning in 2004, the Earth Science Data and Information System (ESDIS) Project has conducted annual surveys to measure user satisfaction using the American Customer Satisfaction Index (ACSI).

The results are one of the ESDIS Project’s metrics sent to the Office of Management and Budget (OMB).

The Easy Part

We start by setting up an Interagency Agreement between NASA and the Department of Interior.

The survey is conducted by CFI Group under contract with the Federal Consulting Group (FCG) of Department of Interior’s National Business Center.

The FCG is Executive Agent in government for the ACSI. They work with OMB to ensure adherence to applicable Federal regulations such as the Privacy Act and the Paperwork Reduction Act.

EOSDIS Data Centers

SEDACHuman Interactions

in Global ChangeLP DAACLand Processes

& Features

NSIDC DAACCryosphere, Polar

Processes

PO.DAACOcean Circulation

Air-Sea Interactions

ASF DAACSea Ice, Polar

Processes

ASDCRadiation Budget,Clouds, Aerosols,Tropo Chemistry

ORNL DAACBiogeochemical

Dynamics, EOS Land Validation

GES DISCAtmos Dynamics, StratoComposition, Hydrology,

Biosphere, Radiance

MODAPSAtmosphere

OBPGOcean Biology &Biogeochemistry

GHRCHydrological Cycle &

Severe Weather

CDDISSolid Earth

Global Hydrology Resource Center—GHRChttp://ghrc.nsstc.nasa.gov/Hydrologic Cycle, Severe Weather Interactions, Lightning, Atmospheric Convection

Alaska Satellite Facility DAAC—ASF DAAChttp://www.asf.alaska.eduSynthetic Aperture Radar (SAR) Products, Sea Ice, Polar Processes, Geophysics

Land Processes DAAC—LPDAAChttp://LPDAAC.usgs.gov Surface Reflectance, Land Cover, Vegetation Indices

Physical Oceanography DAAC— PO.DAAChttp://podaac.jpl.nasa.gov Sea Surface Temperature, Ocean Winds, Circulation and Currents, Topography and Gravity

Oak Ridge National Laboratory DAAC—ORNL DAAChttp://daac.ornl.gov Biogeochemical Dynamics, Ecological Data, Environmental Processes

National Snow and Ice Data Center DAAC—NSIDC DAAC http://nsidc.org Snow and Ice, Cryosphere, Climate Interactions, Sea Ice

EOSDIS Data Centers

Crustal Dynamics Data Information System—CDDIS http://cddis.gsfc.nasa.govSpace Geodesy

Goddard Earth Sciences Data and Information Services Center—GES DISC http://disc.gsfc.nasa.gov Atmospheric Composition, Atmospheric Dynamics, Global Precipitation, Hydrology, Solar Irradiance, Global Modeling, Multi-Sensor Research Products

MODAPS Level-1 Atmospheres Archive and Distribution System— MODAPS LAADShttp://ladsweb.nascom.nasa.govMODIS Level-1 and Atmosphere Data Products

Ocean Biology Processing Grouphttp://oceancolor.gsfc.nasa.govOcean Biology, Sea Surface Temperature, Biogeochemistry

Atmospheric Science Data Center—ASDC LaRChttp://eosweb.larc.nasa.gov Radiation Budget, Clouds, Aerosols, Tropospheric Chemistry

Socioeconomic Data and Applications Center—SEDAC http://sedac.ciesin.columbia.edu Human Interactions, Land Use, Environmental Sustainability, Geospatial Data, Multilateral Environmental Agreements

EOSDIS Data Centers

E-mail addresses are obtained from:•Orders from registered users•Inquiries•Anonymous FTP access•Data center lists

The number of e-mail addresses per data center varies greatly because the data centers vary greatly:

•Discipline-specific•Number or frequency of available products•Size of user communities•Number of anonymous ftp users•Restricted access or cost

Multiple invitations to the same user•A user with multiple e-mail addresses will receive multiple invitations•A user accessing multiple data centers will receive multiple invitations

Who do we survey?

Survey Questions

The survey contains comment fields and several types of questions:• Demographic questions• Questions to aid recall• Rating questions for the ACSI and EOSDIS models• Non-modeled rating questions

The questions were originally based on a previous survey of the members of the User Working Group for each of the data centers.

Formulating the questions is a combined effort of project personnel, EOSDIS User Services Working Group (USWG), and the CFI Group.

The challenge is to keep the survey short enough so users are willing to respond, yet long enough to reflect the complexity of the data and differences in data centers, and more importantly, understand the needs of the diverse user community.

How did EOSDIS Compare in 2008?

The ACSI is the #1 national economic indicator of customer satisfaction. The ACSI is produced by the Stephen M. Ross Business School at the University of Michigan, in partnership with the American Society for Quality (ASQ) and the international consulting firm, CFI Group.

The ACSI is used to measure over 40 industries and 200 organizations covering 45% of the U.S. economy. Over 70 U.S. Federal Government agencies have used the ACSI to measure more than 120 programs/services.

Survey Results 2004-2008

Although the survey contains over 50 questions, the ACSI is based on responses for only three questions.

N= number of responses used (sample size).

See Slide 9 for confidence levels.

The EOSDIS Model

A series of questions are asked for each of the 6 elements of the EOSDIS model:•Product Search•Product Selection and Order•Delivery•Product Quality•Product Documentation•Customer support

CFI’s methodology quantifiably measures and links satisfaction levels to performance and prioritizes actions for improvement.

2008 Results and Priorities

YYR

ACSI 2004 2005 2006 2007 2008

EOSDIS 75 78 74 75 77

Federal Government*

72 71 72 68 69

Overall 74 73 74 75 75

EOSDIS ACSI Score Comparison 2004-Comparisons 2004-2008

Federal Government*Notes:•These numbers are from the end of year results available at thecsi.org.•The charts in our annual presentations may show a slightly different score because the EOSDIS ACSI score is computed before the end of the calendar year.•Our presentations have either the score from a previous year or a quarterly ACSI average of the scores available at the time that our EOSDIS ACSI score was computed.

Top-line Results 2004-2008

2004 2005 2006 2007 2008Customer Satisfaction Index 75 78 74 75 77Likelihood to Recommend 86 89 86 85 86Likelihood to use Services in Future 86 91 88 87 89

Top-line Results 2004-2008 (cont.)

2004 2005 2006 2007 2008Product Search 70 74 70 72 75Product Selection and Order 73 76 72 74 77Delivery 84 85 79 79 81Product Quality* 68 71 71 72 74Product Documentation - - 72 74 75Customer Support 84 84 82 83 84

* Product Quality includes format questions.

2004 HDF/HDF-EOS

In what format were data or products provided?

HDF-EOS 49%HDF 39%NetCDF 5%Binary 14%ASCII 12%GeoTIFF 19%Other 7%

2005 HDF/HDF-EOS

HDF-EOS 39%HDF 30%NetCDF 3%Binary 6%ASCII 7%GeoTIFF 9%Other 3%

HDF-EOS 22%HDF 20%NetCDF 7%Binary 8%ASCII 10%GeoTIFF 25%OGC, Other GIS, Other

6%

Format preferred …Format received …

Q27. In what format were

your data products

provided to you?

Q28. What format would

/ do you prefer?

HDF-EOS 37% 21%HDF 30% 21%NetCDF 3% 8%Binary 6% 7%ASCII 6% 9%TIFF or GeoTIFF 10% 23%JPEG, GIF, PNG 3% 4%OGC Web services 1% 2%Other 3% 5%

In 2005, 9% said products were provided in GeoTIFF and 25% who said it was their preferred method.

2006 HDF/HDF-EOS

2007 HDF/HDF-EOS

In 2006, 67% said products were provided in HDF-EOS and HDF and 42% said they were their preferred method.

Format data or products was

provided*

Format preferred

HDF-EOS/HDF 73% 41%NetCDF 8% 7%

Binary 11% 7%

ASCII 11% 8%

TIFF or GeoTIFF 20% 21%JPEG, GIF, PNG 9% 4%

OGC Web services 0% 0%

GIS 6% 1%

Don't know 4% 8%

Other 2% 3%

*Multiple responses allowed

*

2008 HDF/HDF-EOS

*Multiple responses allowed

Format data products were provided* Format preferredHDF-EOS/HDF 74% HDF-EOS/HDF 41%NetCDF 9% NetCDF 8%Binary 9% Binary 6%ASCII 13% ASCII 8%GeoTIFF 16% GeoTIFF 20%JPEG, GIF, PNG, TIFF 11% JPEG, GIF, PNG, TIFF 5%OGC Web services 1% OGC Web services 1%GIS compatible 4% GIS 6%KLM, KMZ 2% KLM, KMZ 1%Don't know 4% Don't Know 0%Other 2% Other 4%

2009 Format QuestionsWhat format(s) would/do you prefer? (select any that apply)

•HDF-EOS/HDF•NetCDF•Binary•ASCII•GeoTIFF•JPEG, GIF, PNG, TIFF•OGC Web services (WMS, WCS, WFS, etc.)•GIS (e00, shp, etc.)•CEOS•KML, KMZ•Other (Please specify another format or comment on specific version, etc.

In what format(s) were your data products provided to you? (select any that apply)

•HDF-EOS/HDF•NetCDF•Binary•ASCII•GeoTIFF•JPEG, GIF, PNG, TIFF•OGC Web services (WMS, WCS, WFS, etc.)•GIS (e00, shp, etc.)•KML, KMZ•CEOS•Don’t know•Other (please specify and/or comment

Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate…

•Ease of using the data product in the delivered format•Overall quality of the data product•Overall usability of the data product

2009 Documentation Questions

Still using the 10-point scale on which “1” means “Poor” and “10” means “Excellent,” how would you rate…

•Overall quality of the document (i.e., technical level, organization, clarity)•Extent to which the data documentation helped you use the data

What documentation did you use or were you looking for?

•Instrument specifications•Science algorithm•Product format•Tools•Science applications•Data product description•Production code•Other

Was the documentation•Delivered with the data•Available online•Not found (Skip to Customer Services)

Customer Service

Was it •By phone•By E-mail•Both by phone and e-mail

Have you requested assistance from <Data center name>’s user services office during the past year?

•Yes, continue•No: Go to Closing

Think about the user services staff you interacted with when you requested assistance from <Data center name> user services. On the same scale from 1 to 10 where 1 means “Poor” and 10 means “Excellent,” how would you rate the user services staff on…

•Professionalism•Technical knowledge•Accuracy of information provided•Helpfulness in selecting/finding data or products•Helpfulness in correcting a problem•Timeliness of response

2009 Survey Responses

Data CenterNumber of addresses

Bounced Back

Received Invitation Responded

Response Rate

GHRC 1174 161 1013 82 8%

ASF SAR DAAC 1372 144 1228 127 10%

CDDIS 1597 612 985 143 15%

GES DISC 2248 812 1436 101 7%

SEDAC 2338 184 2154 131 6%

ORNL DAAC 2351 229 2122 164 8%

LaRC ASDC 2386 410 1976 144 7%

PO.DAAC 3770 1425 2345 173 7%

OCDPS-OBPG 3841 381 3460 266 8%

NSIDC DAAC 4673 911 3762 312 8%

MODAPS LAADS 8198 1119 7079 696 10%

LP DAAC 11585 916 10669 1503 14%

Total 45533 7304 38229 3842 10%

Thank you! any questions?

http://www.humanfactors.com

Some thoughts on users …

Some thoughts on web sites …

And just a few more …