Embed Size (px)
Citation preview
2005 Customer Satisfaction Study
September 2005
NASA Earth Observing System Data and Information Systems
2
What is the ACSI?
• The only standardized measure of customer satisfaction covering more than 200 companies in 40 industries; about 66% of the U.S. economy
– Including over 70 U.S. Federal Government agencies
• A quarterly snapshot of the national economy’s health
• Results published quarterly
– For more information, see www.theacsi.org
3
Who is FCG?
A franchise of the Department of the Treasury
Assists agencies in managing change
Has three centers of excellence:
• Management consulting
• Executive coaching
• Customer/employee satisfaction metrics• Executive Agent for American Customer
Satisfaction Index• Generic Office of Management and Budget
clearance • Streamlined contracting process
4
Who is CFI Group?
• Founded in 1988; headquartered in Ann Arbor, Michigan
• 13 offices, 150 full-time consultants and researchers worldwide
• CFI system is the source for University of Michigan’s American Customer Satisfaction Index (ACSI)
5
Which NASA EOSDIS customer segment was surveyed?
This customer segment includes individuals who have accessed NASA EOSDIS data and/or products by means of a NASA EOSDIS Data Center (DAAC).
How were the NASA EOSDIS customers identified?
NASA EOSDIS provided a list of 21,871 email addresses for people who have used NASA EOSDIS data and/or products. The data center was identified within the customer list. After cleaning the sample list CFI Group sent out 17,424 email invitations asking customers to participate in the online survey. During the initial mailing over 3,800 emails were bounced back to CFI Group due to an invalid address.
The survey was available online June 28, 2005 – July 20, 2005. A total of 1,395 surveys were completed. CFI Group used 1,263 surveys for analysis.
Project Background
6
78*
NASA EOSDIS Results
The Customer Satisfaction Index for NASA EOSDIS is…
NASA EOSDIS Aggregate Segment
The Customer Satisfaction Index score is derived from customer responses to three questions in the survey:
– How satisfied are you overall with the products and services provided by the Data Center (82)?
– To what extent have the data, products and services provided by the Data Center fallen short of or exceeded your expectations (73)?
– How well does the Data Center compare with an ideal provider of scientific data, products and services (76)?
This score is 6 points higher than the 2004 American Customer Satisfaction Index for the Federal Government overall (72.1).
7
Comparison Across Government Agencies
73
72
75
72
75
78
ACSI Aggregate Q2 2005
U.S. Federal GovernmentACSI 2004
Info. Providers (10 agencyaggregate 2004)
NASA main website(www.nasa.gov)
NASA EOSDIS 2004
NASA EOSDIS 2005
Customer Satisfaction Index
+3
8
NASA EOSDIS Customer Satisfaction Model
Future Use
RecommendCustomer
SatisfactionIndex(CSI)
Product Search
Product Selection
Delivery
Product Quality
Customer Support
9
NASA EOSDIS 2004 vs. 2005 Comparisons Significant increases from 2004 in Product Selection and Product Quality
2004 2005
Product Search 70 74 áProduct Selection and Order 73 76 N/A*Delivery 84 85 Product Quality 68 71 áCustomer Support 84 84 Customer Satisfaction Index 75 78 áLikelihood to Recommend 86 89 áLikelihood to Use Services in Future 90 91 áComplaints 32% 33%
Sample Size 1016 1263
*N/A due to questionnaire changes
10
NASA EOSDIS Customer Satisfaction Model
Future Use
RecommendCustomer
SatisfactionIndex(CSI)
Score
89
78
Impact of CSI 3.3
2.5
91
Product Search
Product Selection
Delivery
74
0.8
0.8
0.2
76
85
Impact on CSI
Product Quality
71
0.9
Customer Support
84
1.4
11
x1
x2
x3
x4
x5
x6
x1
x 3
x4
x5
x 6
y1
y2
y3y
3
y2
y 11
2
xi xi t i , for i=1,2,3 t=1,2y j yj j 1 , for j = 1,2,3
1 1 1 2 2 1
x2
The Math Behind the Numbers
A discussion for a later date…or following this presentation for those who are interested.
12
Areas of Opportunity for NASA EOSDIS Focus on the quality and accessibility
Top Priority
Product Quality
Product Search
Product Selection and Order
(71)
(74)
(76)
13
71
68
69
68
67
72
71
70
Product Quality
Ease of using the data
product in the deliveredformat
Thoroughness of dataproduct documentation
Clarity of data product
documentation
2005 2004
Product QualityImpact: 0.9
Only 28% had documentation delivered with
the data…
70% of data products were
HDF-EOS or HDFformat
“If possible send along the software to open HDF”
“DAAC products are acceptable however their documentation could be a little better. I find the information provided regarding processing and things like data projections and units very confusing.”
“If possible send along the software to open HDF”
“DAAC products are acceptable however their documentation could be a little better. I find the information provided regarding processing and things like data projections and units very confusing.”
Note: All score increases are statistically sig.
14
74
70
73
69
68
75
74
72
Product Search
How well the searchresults met your needs
Ease of finding data
Ease of using searchcapability
2005 2004
Product SearchImpact: 0.8
63% used EOS Data Gateway to search for data and products which remains consistent with 2004 results (67% in 2004)
Note: All score increases are statistically sig.
15
Product Search Score ComparisonBy method for most recent search
Product Search Rating
73
73
74
79
65 70 75 80
1
Data center search capability
Global Change MasterDirectory
EOS Data Gateway
Data pool / data cache
“The search engine should produce only the data requested. For example, when I type MOD12Q1 in the search engine, I get every MODIS product
generated.”
“The search engine should produce only the data requested. For example, when I type MOD12Q1 in the search engine, I get every MODIS product
generated.”
16
76
74
70
72
74
76
78
Product Selection andOrder
Ease ofrequesting/ordering
data products
Ease of selecting dataproducts
Description of dataproducts
2005 2004
Product Selection and Order Impact: 0.8
Question wording changed from 2004
“For the different data formats, have summary of
basic data use and
interpretation, particularly for
level 3 data, along with a
test read file to make sure user is reading and converting the data correctly.”
Over 50% are requesting results for Earth Science Research and 1/2 are from higher education
Over 50% are requesting results for Earth Science Research and 1/2 are from higher education
17
Kudos for Customer Support PersonnelScore of 84 maintained vs. 2004
While only 21% used Customer Support during their most recent request/order, they are rating their experience well and this component has a substantial impact on overall Satisfaction with EOSDIS.
Overall Satisfaction with EOSDIS is higher for those who did use customer support when compared to those who did not (80 vs. 77)
18
Customer SupportImpact: 1.4
84
84
83
84
87
85
85
84
82
83
86
85
84
84
Customer Support
Professionalism
Technical knowledge
Accuracy of information provided
Helpfulness in selecting/finding data orproducts
Timeliness of response
Helpfulness in correcting a problem
2005 2004
19
DeliveryImpact: 0.2
69% of customers’ data was delivered through FTP – which is consistent with 2004 survey results
69% of customers’ data was delivered through FTP – which is consistent with 2004 survey results
85
84
86
81
87
83
Delivery
Convenience ofdelivery method
Timeliness of deliverymethod
2005 2004
“Compress data before delivery - will cut
download times
drastically”
20
Desired Outcomes
NASA EOSDIS wants its customer to recommend and use EOSDIS services in the future. Desired customer behaviors were measured with a single question each. – “How likely are you to recommend the Data Center to a
colleague?” scored an 89
• 43% state the way they heard about gathering data from EOSDIS is from a colleague.
– “How likely are you to use the services provided by the Data Center in the future?” scored a 91
NASA EOSDIS customers were also asked if they had ever contacted the Data Center’s user services office to report a problem.– 33% of of customer say they have contacted the Data
Center’s user services office to report a problem.
21
Summary of Findings
NASA EOSDIS performs well compared to various Government benchmarks
NASA EOSDIS saw a significant increase in their overall Customer Satisfaction score when compared to 2004 results
Likelihood to Recommend increased in 2005 and almost half of respondents heard about NASA EOSDIS from a colleague (word of mouth really does matter!)
Higher priorities remain centered around the quality of products and the ability to access the products
Customer Support continues to score well, and remains important to those customers using the service
22
Score ComparisonData Centers show a 10-point variation in Satisfaction
78
78
77
76
73
72
82
81
80
78
Customer Satisfaction Index
GHRC - Global Hydrology and ResourceCenter
ORNL DAAC - Oak Ridge NationalLaboratory DAAC
PO DAAC - Physical Oceanography DAAC
GES DAAC - GSFC Earth Sciences DAAC
NSIDC DAAC - National Snow and Ice DataCenter DAAC
LP DAAC - Land Processed DAAC
LaRC DAAC - Langley Research CenterDAAC
SEDAC - Socioeconomic Data andApplications Center
ASF DAAC - Alaska Satellite Facility DAAC
LP DAAC - Land Processes DAAC
2% n=29
5% n=65
8% n=96
28% n=359
11% n=138
33% n=419
7% n=92
3% n=35
2% n=30
% reflects percentage of aggregate score
23
NASA EOSDISOver 3
months ago3 months
or lessProduct Search 69 75Product Selection and Order 73 78Delivery 82 86Product Quality 68 72Customer Support 79 87
Customer Satisfaction Index 74 79
Likelihood to Recommend 84 91Likelihood to Use Services in Future 87 94Complaints 26% 36%
Non-modeledData for free 94 96Overall Quality 79 82Overall Usability 77 81
Sample Size 351 862
Score ComparisonTime of Last Data Product or Service Request
Results consistent with 2004 where those whose last request was less than 3 months ago are rating EOSDIS higher across the board.
Results consistent with 2004 where those whose last request was less than 3 months ago are rating EOSDIS higher across the board.
24
Score ComparisonHigher satisfaction outside of the USA
Respondents outside the USA continue to have a higher overall Satisfaction score with EOSDIS but as in 2004 their rating of Customer Support continues to be lower and both have approximately 20% who
used Customer Support
Respondents outside the USA continue to have a higher overall Satisfaction score with EOSDIS but as in 2004 their rating of Customer Support continues to be lower and both have approximately 20% who
used Customer Support
USAOutside the
USA
Product Search 72 74Product Selection and Order 74 77Delivery 86 85Product Quality 70 72Customer Support 88 82
Customer Satisfaction Index 75 79
Likelihood to Recommend 88 89Likelihood to Use Services in Future 91 91Complaints 36% 30%
Sample Size 452 795
25
Recommendations
Product Quality – Use terminology most users would understand and consider delivering documentation with the product
Product Search – Focus on lessons learned from data center search capability which can be applied to EOS Data Gateway
Product Selection and Order – Clearly describe data and give examples for how the particular data can be used
Customer Support – Praise staff for continuing to bring a high level of service to customers when needed
o Identify different practices used for customer support for USA users and outside of USA users
Use data centers strengths to create “best practices” which can be shared across the data centers
26
Additional Information
• Detailed survey results for all of the Federal services, including trends in performance and customer satisfaction, were updated in June 2005 and can be found on the website www.customerservice.gov
• Many agencies share best practices through the interagency customer service forum
• Contact information– Sheri Teodoru
– Bernie LubranFederal Consulting [email protected]
27
Review of ACSI Results
A component score is a weighted average of the set of survey questions comprising a component or activity area. Responses to survey questions are given on a 1-10 scale, which is converted to a 0-100 scale for score reporting.
An impact predicts the increase in satisfaction that would result from a 5-point increase in a component or input score.
Areas for improvement are those components or activities with a relatively low score and a relatively high impact on satisfaction.
In the simplified example shown here,
Product Selection would be an area of
improvement due to its lower score and higher
impact.
CustomerSatisfaction
Index(ACSI)
78Product
Selection
Delivery
0.8
0.2
76
85
Score
Impact on ACSI
EXAMPLE
