Q3 2013 e-gov commentary FINAL.pdf

7/27/2019 Q3 2013 e-gov commentary FINAL.pdf

1/40


2/40

ACSI E-GOVERNMENT SATISFACTION INDEX (Q3 2013)

EXECUTIVE SUMMARY

This quarters report on the American Customer Satisfaction Index (ACSI) E-Government Satisfaction Index

has three parts:

> Part 1: Introduction Mobile Naturalization in Digital Government. Citizen satisfaction scores

have hovered around 75 for the past four years. The bad news is that citizen satisfaction scores have

not increased over this period. One major inuence impacting citizen expectations is the growing

availability of mobile as a direct line for citizens to interact with government organizations,

agencies, and departments. Thus, as mobile is the next natural step for citizens, it should be the

logical next step for the digital federal government to measure and manage. This section also

shares mobile insights across a handful of agencies, departments, and organizations that asked

citizens additional questions regarding mobile use.

> Part 2: Citizen Satisfaction. This is a quarterly update on citizen satisfaction with e-government at

the aggregate level, including individual satisfaction scores for the 105 federal government websites

participating in the Index. Key ndings include:

Citizen satisfaction with the e-government experience remains steady in Q3 2013. The third

quarter of 2013 registered an average citizen satisfaction score of 74.9, which is insignicantly

down from 75.0 in Q2 2013. The Social Security Administrations (SSA) Retirement Estimator, SSAs

iClaim, and Extra Help with Medicare Prescription Drug Plan Costs each scored a 90 to lead the

Index. See a complete list of individual agency scores on page 8.

E-government outperforms overall government in citizen satisfaction. Each year, it is

clear that citizens prefer to interact with the federal government via the Internet. Average citizen

satisfaction with e-government continues to outperform average citizen satisfaction with the

overall federal government, recording a score of 74.9 for Q3 2013 compared to 68.4 for the overall

federal government in the ACSIs 2012 U.S. Federal Government Index.


3/40


Why measure satisfaction? Highly satised citizens are 92% more likely to use the federal

website they evaluated as a primary resource, 96% more likely to recommend the site, 64% more

likely to put their trust behind the agency, 54% more likely to return to the site, and 47% more

likely to participate with the agency in the future. See the chart and explanation on page 22.

Search, Online Transparency, and Navigation are top priorities for improvement. By using a

scientic approach to measure e-government experiences through the eyes of the citizen, agency

managers and government ofcials can determine which improvements will have the largest impact

on satisfaction, thereby affecting future behaviors, such as those described on page 23.

> Part 3: Appendix A Keeping Pace with Mobile: ForeSees Five Tips for a Better Mobile

Experience. Eric Feinberg, senior director of mobile, ForeSee, shares ve tips to help improve the

mobile experience in this abridged reprint of his whitepaper of the same name. While the examples

used are taken from the private sector, Feinberg believes that mobile excellence transcends all

industries, public and private.

PART ONE: INTRODUCTION

THE NATURALIZATION OF MOBILE IN

THE DIGITAL GOVERNMENT

In spite of undulating citizen expectations and opinions of the federal government through the past decade,

citizen satisfaction with the web arm of the federal government has actually remained fairly steady for more

than four years, hovering around the 75 mark.

Citizen satisfaction scores, in fact, have remained at 75 or higher (on the studys 100-point scale) for 15 of

the last 17 quarterssince Q3 2009. The good news for e-government is that there has not been a major

decline in citizen satisfaction. The bad news is that citizen satisfaction scores have not increased over this

period either. Ideally, government ofcials need to address this problem by continuing to improve the

experiences they offer in order to meet the challenge of evolving citizen expectations.


4/40


One major inuence impacting expectations is the growing availability of mobile as a direct line for citizens to

interact with government organizations, agencies, and departments.

We are seeing mobile becoming a prime catalyst for interaction. Across all industries, ForeSee clients see

an average of 10% to 20% of overall requests to a given companys digital experiences coming from mobile

devices (including mobile web and apps), according to Keeping Pace with Mobile: ForeSees Five Tips for

a Better Mobile Experience. This means for a company with one million unique monthly visitors, 100,000

to 200,000 individuals are inuenced by a mobile experience with that brandand that percentage is

continuously increasing.

Furthermore, according to a June 2013 Nielsen report, three out of ve (61%) American adults owned

smartphones during the three-month period of March to May 2013, up more than 10% since early 2012.

Pew Research shows that in June 2013, one-third (34%) of U.S. adults owned tablets, which is almost two

times as many as a year ago (18%).

While mobile accessibility is a major focus for the federal government in its efforts to be more citizen-centric

as set forth by the Digital Government: Building a 21st Century Platform to Better Serve the American

People report (May 2012), this sudden advancement of mobile can create certain challengesmuch like

what we see in the private sector. Most notably, consumers do not distinguish between the different

channels an organization offers. Although todays customers (or in this case citizens) are busy traversing

across multiple touch points with multiple devices, they see the experience as singular and expect it to be

seamless. If citizens are unable to complete their tasks via a mobile device (whether it is a transaction or

informational search or something else) as easily as they can through a web interaction, their satisfaction

with the organization as a whole could suffer.


5/40


Mobile implementation is a huge undertaking, as is ensuring that the experiences meet or exceed citizen

expectations. We continue to receive numerous inquiries from both concerned clients and curious prospects

regarding the best practices for mobile accessibility. In Appendix A of this report (page 27), Eric Feinberg,

senior director of mobile, ForeSee, shares ve tips for a better mobile experience.

As mobile is becoming the next natural step for citizens to take when engaging with federal government, it

should be the logical next step for government leaders to measure and manage.

MOBILE IN GOVERNMENT UPDATE

To get a better idea of how mobile use measures up in the digital government, ForeSee examined 19 federal

Index participants who added questions to their surveys regarding citizen mobile experiences. These addition-

al survey questions received more than 20,000 responses.

On average across all the participating agencies, more than half (55%) of government website visitors

reported ever using a mobile phone or tablet to access the Internet, showing a small increase from 52% last

quarter and from 48% when rst measured in Q4 2012 as the use of mobile devices for web access becomes

more common.

More than one-third (37%) of visitors (also a benchmark average) reported having accessed any federal

government website using a mobile phone or tablet, compared to 34% in Q2 2013. Furthermore, 13% of

visitors reported they had not used a mobile device to access federal government websites butplannedto do

so; 36% of visitors reported they had not used a mobile device to access a federal government website but

maydo so in the future; and 14% of visitors reported they had not used a mobile device to access a federal

government website and had no plans of doing so in the future.


6/40


On average, 31% of visitors to federal government websites participating in the survey reported that they

had used a mobile phone or tablet to access the specic website they were on; 21% of visitors reported they

had not used a mobile phone or tablet to access the website butplannedto do so; 37% of visitors reported

they had not used a mobile phone or tablet to access the website but maydo so in the future; and 11% of

visitors reported they had not used a mobile phone or tablet to access the website and had no plans of doing

so in the future.

PART TWO

SATISFACTION WITH E-GOVERNMENT REMAINS STEADY

Satisfaction with federal government websites continues to outperform satisfaction with the federal

government overall. In fact, e-government recorded a satisfaction score of 74.9, whereas the overall

government scored a 68.4 in the ACSI 2012 U.S. Federal Government Report released in February 2013.

Federal government websites have long lagged behind their private-sector counterparts in customer

satisfaction. However, a recent slide for the e-business sector broke this trend by scoring 71.8 in the ACSI

annual report on the sector in July 2013. These scores indicate that people are actually more satised with

federal government websites than they are with comparable private-sector e-business websites (such as

portals, search engines, news and information websites, and social media sites).

Government agencies have an obligation to the public to be scally responsible by using the federal budget

as wisely as possible. There are denite cost savings associated with offering a highly satisfying website

experience. Therefore, government agencies need to maintain the highest online standards by meeting or

exceeding citizens expectations. Federal websites that are already successfully meeting users needs should

take note of their achievements without becoming complacent. If done right, improving the citizen

experience will increase users likelihood to participate with government in the future, use the government

website as a primary resource (rather than costlier channels such as branch locations and contact centers),

and recommend the government site to others.


7/40


On the studys 100-point scale, a satisfaction score of 80 or higher is typically considered the threshold for

excellence and can be achieved only if the organization is doing an outstanding job of meeting and

exceeding citizen expectations. This quarter, 30% (32 websites) of the federal participants in the Index

scored 80 or above. A score in this range shows that public-sector websites can live up to, and in some cases

exceed, expectations that the private sector sets in large part.

The ACSI E-Government Satisfaction Index is a comprehensive reection of the citizen experience with

federal government websites, and it serves as a critical checkpoint for evaluating the success and

performance of the federal governments online initiatives. More than 270,000 responses were collected

across 105 federal government websites for the 2013 third-quarter Index. This demonstrates that

citizens are willing to share their voices to help agencies and departments improve. The use of ForeSee

technology and ACSI methodology then enables agency leaders to determine which website

improvements will have the greatest impact on future usage and recommendations.

Satisfaction

Score

(100-pointscale)

Q3

03

Q4

03

Q1

04

Q2

04

Q3

04

Q4

04

Q1

05

Q2

05

Q3

05

Q4

05

Q1

06

Q2

06

Q3

06

Q4

06

Q1

07

Q2

07

Q3

07

Q4

07

Q1

08

Q2

08

Q3

08

Q4

08

Q1

09

Q2

09

Q3

09

Q1

10

Q2

10

Q3

10

Q4

09

70 69 70 70 71 72 71 72 73 73 73 74 73 73 73 73 73 72 72 72 73 74 73 73 75 75 74 7575

ACSI E-GOVERNMENT INDEX

Satisfaction by Quarter 2003 - 2013

Satisfaction Q4

10

75

Q1

11

Q2

11

Q3

11

75 75 75

Q4

11

75

Q1

12

75

74

Q2

12

Q3

12

75 75

Q4

12

75

78

77

76

75

74

73

72

71

70

69

68

Q1

13

75

Q2

13

75

Q313

74


8/40


Three websites from the Social Security Administration (SSA)Retirement Estimator, iClaim, and Extra

Help with Medicare Prescription Drug Plan Costseach scored a 90 to lead the top overall scores in the

E-Government Satisfaction Index. These three SSA sites also match or outperform some of the top measured

private sites to date in 2013, such as Mercedes-Benz (88), Apple (87), FedEX (85), and Amazon (85 in 2012).

The following table displays scores for all 105 participating federal websites in the E-Government Satisfaction

Index. The highlighted agencies represent those that have reached an excellent score of 80 or higher.

Later pages of the commentary show scores by category for more specic benchmarking purposes.

Q3 2013 ACSI E-Government Satisfaction Index

Department Website Satisfaction

Average 74.9

SSA Extra Help with Medicare Prescription Drug Plan Costs -- socialsecurity.gov/i1020 90

SSA SSA iClaim -- socialsecurity.gov/applyonline 90

SSA SSA Retirement Estimator -- ssa.gov/estimator 90

HHS MedlinePlus -- medlineplus.gov 88

SSA SSA Electronic Access/Online Statement -- ssa.gov/mystatement 88

DHSU.S. Citizenship and Immigration Services Resource Center --uscis.gov/portal/site/uscis/citizenship

88

HHS MedlinePlus en espaol -- medlineplus.gov/esp 87

HHS National Womens Health Information Center (NWHIC) main website -- womenshealth.gov 85

SSA Social Security Business Services Online -- ssa.gov/bso/bsowelcome.htm 85

SSA Social Security Internet Disability Report -- ssa.gov/applyfordisability 85

HHS National Library of Medicine AIDS Information -- aidsinfo.nih.gov 84

HHS NIH - Senior Health -- nihseniorhealth.gov 84

HHS HHS Healthy People -- Healthypeople.gov 83HHS National Cancer Institute main website -- cancer.gov 83

HHS National Cancer Institute Site en Espaol -- cancer.gov/espanol 83

HHS National Institute of Diabetes and Digestive & Kidney Diseases -- www2.niddk.nih.gov 83

CIA Recruitment website -- cia.gov/careers 83

DHS U.S. Citizenship and Immigration Services Espaol -- uscis.gov/portal/site/uscis-es 83

SEC U.S. Securities and Exchange Commission -- investor.gov 83

HHS CDC main website -- cdc.gov 82

AgenciesScoring

80+


9/40


Q3 2013 ACSI E-Government Satisfaction Index (continued from page 8)


DHS Federal Emergency Management Agency Ready Campaign -- ready.gov 82

DOD DOD Pentagon Channel -- pentagonchannel.mil 81

DOJ National Institute of Justice -- nij.gov 81

HHS NIAMS public website -- niams.nih.gov 81

DOSU.S. Department of State Bureau of Educational and Cultural Affairs alumni website --https://alumni.state.gov

81

Treasury U.S. Mint Online Catalog and main website -- usmint.gov 81

Boards, Commissions,and Committees

American Battle Monuments Commission -- abmc.gov 80

DOD DoD Navy -- navy.mil 80

FTC FTC OnGuardOnline -- onguardonline.gov 80

NASA NASA main website -- nasa.gov 80

DOCNational Geodetic Society, National Oceanic and Atmospheric Administration website --ngs.noaa.gov

80

HHS National Institute of Dental and Craniofacial Research -- nidcr.nih.gov 80

DOL Department of Labor Job Listings -- doors.dol.gov 79

DOJ FBI main website -- fbi.gov 79

DOI National Park Service main website -- nps.gov 79

DOS Recruitment website -- careers.state.gov 79

HHS SAMHSA Store -- store.samhsa.gov 79

NIHThe National Center for Complementary and Alternative Medicine (NCCAM) -- nccam.nih.gov

79

NRC U.S. Nuclear Regulatory Commission website -- nrc.gov 79

PBGC MyPAA -- https://egov.pbgc.gov/mypaa 78

PBGC MyPBA -- https://egov.pbgc.gov/mypba 78

HHS National Library of Medicine main website -- nlm.nih.gov 78

DOS Bureau of Consular Affairs -- travel.state.gov 77

DOL Bureau of Labor Statistics -- bls.gov 77

HHS National Institute of Allergy and Infectious Diseases -- www3.niaid.nih.gov 77

HHS National Library of Medicine AIDS information -- aidsinfo.nih.gov 77

SBA SBA main website -- sba.gov 77

HHS U.S. Food and Drug Administration main website -- fda.gov 77

DOI U.S. Geological Survey -- usgs.gov 77

HHS Agency for Healthcare Research and Quality -- ahrq.gov 76

AgenciesScoring

80+


10/40




DOD Department of Defense portal -- defense.gov 76

FTC FTC main website -- ftc.gov 76

GAO GAO main public website -- gao.gov 76

USDA Recreation One-Stop -- recreation.gov 76

SSA SSA iAppeals - Disability Appeal -- ssa.gov 76

DHS U.S. Citizenship and Immigration Services -- uscis.gov/portal/site/uscis 76

USDA ERS main website -- ers.usda.gov 75

HHS HHS National Health Information Center -- Healthnder.gov 75

DOJ National Criminal Justice Reference Service -- www.ncjrs.gov 75

NIST National Institute for Standards and Technology main website -- nist.gov 75

DHS U.S. Citizenship and Immigration Services -- uscis.gov/e-verify 75

DOD DoD Air Force -- af.mil 74

DOT Federal Aviation Administration -- faa.gov 74

FTC FTC Complaint Assistant website -- ftccomplaintassistant.gov 74

HHS National Library of Medicine Clinical Trials website -- clinicaltrials.gov 74

GSA GSA main website -- gsa.gov 73

HHS Health Resources and Services Administration main website -- hrsa.gov 73

Treasury Making Home Affordable -- makinghomeaffordable.gov 73

OPM Recruitment website -- usajobs.gov 73

DOT U.S. Department of Transportation -- fhwa.dot.gov 73

PBGC U.S. PBGC main website -- pbgc.gov 73

DOD DoD Marines -- marines.mil 72

FDIC FDIC main website -- fdic.gov 72

Treasury Financial Stability -- nancialstability.gov 72

GSA GSA Auctions -- gsaauctions.gov 71

HHS SAMHSA website -- samhsa.gov 71

SSA Social Security Online: Frequently Asked Questions -- ssa-custhelp.ssa.gov 71

ITC U.S. International Trade Commission main website -- usitc.gov 71

FDIC FDIC Applications -- www2.fdic.gov 69

NARA NARA main public website -- archives.gov 69

Treasury Treasury main website -- treasury.gov 69

DOS Department of State main website -- state.gov 68

DOC U.S. Census Bureau main website -- census.gov 68


11/40




EPA U.S. Environmental Protection Agency -- epa.gov 68

SEC U.S. Securities and Exchange Commission main website -- sec.gov 68

DOC BEA main website -- bea.gov 67

DHS Department of Homeland Security main website -- dhs.gov 67

SSA Social Security Online main website -- socialsecurity.gov 67

HHS Girls Health -- girlshealth.gov 66

VA VA main website -- va.gov and myhealthva.gov 66

DOT DOT Research and Innovative Technology Administration website -- rita.dot.gov 65

Treasury Department of the Treasury, Alcohol and Tobacco Tax and Trade Bureau website -- ttb.gov 65

DOT Federal Railroad Administration main website -- fra.dot.gov 64

DOT Federal Motor Carrier Safety Administration main website -- fmcsa.dot.gov 63

USDA Forest Service main website -- fs.usda.gov 63

USDA NRCS website -- nrcs.usda.gov 63

GSA Ofcial Site to Buy U.S. Government Property -- govsales.gov 63

DOE U.S. Department of Education -- ed.gov 63

HHS HHS -- grants.gov 62

DOL Disability -- Disability.gov 59

Treasury IRS main website -- irs.gov 59

Treasury TreasuryDirect -- treasurydirect.gov 59

USDA FSIS main website -- fsis.usda.gov 58

DOD TRICARE -- tricare.mil 58

DHSFederal Emergency Management Agency mainwebsite -- fema.gov

55


12/40


TOP GAINERS

Any time a federal government agency, program, or department website shows signicant improvement in

satisfaction (of three points or more), it should be noted as a success, as it is sometimes difcult to keep pace

with the ever-changing citizen expectations. These entities show that they are denitely doing something right

by increasing citizen satisfaction, and others, whether in the same category or not, should take note of how

they are achieving this success.

The following chart shows the eight websites that demonstrated signicant increases (three points or more)

in citizen satisfaction from Q2 2013 to Q3 2013.

E-Gov Top Gainers (Quarter to Quarter)

Department WebsiteSatisfaction

Gain





DOD Pentagon Channel -- pentagonchannel.mil 4




SATISFACTION BY FUNCTIONAL CATEGORY

In this report, federal government websites are organized by both functional category and organizational

structure to allow for benchmarking against peers. The functional website categories include: News

and Information; Portals and Department Main Websites; E-commerce and Transactional; and Career

and Recruitment.


13/40


Since missions can vary greatly by category, it is useful for a government website to benchmark its score

against other sites in the same category in addition to comparing against the overall aggregate average.

To provide the most accurate and precise data, the ForeSee standard requires that a category consist of

at least ve websites before an average is calculated. Only four websites are measured in the Career and

Recruitment category; therefore, this category does not include an overall average. However, we do list the

scores of the individual websites within the Career and Recruitment category at the end of this section.

Federal E-Commerce and Transactional Websites

The following chart shows the citizen satisfaction scores for all 16 federal government E-Commerce and

Transactional websites measured in this category.

Federal E-Commerce and Transactional Websites


E-Commerce Aggregate 79

SSA Extra Help with Medicare Prescription Drug Plan Costs -- socialsecurity.gov/i1020 90

SSA SSA iClaim -- socialsecurity.gov/applyonline 90

SSA SSA Retirement Estimator -- ssa.gov/estimator 90

SSA SSA Electronic Access/Online Statement -- ssa.gov/mystatement 88

SSA Social Security Business Services Online -- ssa.gov/bso/bsowelcome.htm 85

SSA Social Security Internet Disability Report -- ssa.gov/applyfordisability 85

Treasury U.S. Mint Online Catalog and main website -- usmint.gov 81

HHS SAMHSA Store -- store.samhsa.gov 79

PBGC MyPAA -- https://egov.pbgc.gov/mypaa 78

PBGC MyPBA -- https://egov.pbgc.gov/mypba 78

USDA Recreation One-Stop -- recreation.gov 76

SSA SSA iAppeals - Disability Appeal -- ssa.gov 76

FTC FTC Complaint Assistant website -- ftccomplaintassistant.gov 74


GSA Ofcial Site to Buy U.S. Government Property -- govsales.gov 63

Treasury TreasuryDirect -- treasurydirect.gov 59


14/40


E-commerce and Transactional websites remain the top-scoring category in this Index, recording a score

of 79 in Q3 2013 (matching the category score from Q2 2013), with the scores of websites in this category

ranging from 59 to 90. However, this category still trails behind private sector e-commerce websites, which

scored 81.1 in the ACSI E-Commerce annual report published in February 2013.

The good news is that the aggregate average for this category has steadily improved over the last year,

increasing from 77 in Q4 2012 to 78 in Q1 2013, and now 79 in both Q2 and Q3 2013. This is welcomed

stability to a category that has been erratic at best in its performance through the years, ranging from 75

in 2007, to 82 in 2009, to 77 in 2012.

Federal government E-commerce and Transactional websites could be on the way to stability. Only time will

tell, however. As long as they use a scientic, reliable, and accurate measurement system to monitor citizen

experiences, agency leaders should see improvements categorically and individually.

The Social Security Administration continues to set the bar high with six websites leading this category whose

scores range from 85 to 90. Three of the websites score a 90, outdoing every top-performing private-sector

e-commerce website measured by the ACSI, including Amazon.

The GSA Auctions site was the only site in the category to show a marked improvement (three points or

more) in Q3 2013, increasing their citizen satisfaction score four points to 71.

Federal News and Information Websites

The following chart shows the citizen satisfaction scores for all 53 of the federal government News and

Information websites.


15/40


Federal News and Information Websites


News/InformationAggregate

75

HHS MedlinePlus -- medlineplus.gov 88

DHSU.S. Citizenship and Immigration Services Resource Center --uscis.gov/portal/site/uscis/citizenship

88

HHS MedlinePlus en Espaol -- medlineplus.gov/esp 87

HHS Nat ional Womens Health Information Center (NWHIC) main website -- womenshealth.gov 85


HHS NIH - Senior Health -- nihseniorhealth.gov 84

HHS HHS Healthy People -- Healthypeople.gov 83

HHS National Cancer Institute Site en Espaol -- cancer.gov/espanol 83

HHS National Institute of Diabetes and Digestive & Kidney Diseases -- www2.niddk.nih.gov 83

SEC U.S. Securities and Exchange Commission -- investor.gov 83

DHS Federal Emergency Management Agency Ready Campaign -- ready.gov 82

DOD DOD Pentagon Channel -- pentagonchannel.mil 81

DOJ National Institute of Justice -- nij.gov 81

DOSU.S. Department of State Bureau of Educational and Cultural Affairs alumni website --https://alumni.state.gov

81

Boards, Commissions,

and CommitteesAmerican Battle Monuments Commission -- abmc.gov 80

DOD DoD Navy -- navy.mil 80

FTC FTC OnGuardOnline -- onguardonline.gov 80

DOCNational Geodetic Society, National Oceanic and Atmospheric Administration website --ngs.noaa.gov

80

NIHThe National Center for Complementary and Alternative Medicine (NCCAM) -- nccam.nih.gov

79


DOS Bureau of Consular Affairs -- travel.state.gov 77

DOL Bureau of Labor Statistics -- bls.gov 77

HHS National Institute of Allergy and Infectious Diseases -- www3.niaid.nih.gov 77


DOI U.S. Geological Survey -- usgs.gov 77

HHS Agency for Healthcare Research and Quality -- ahrq.gov 76


HHS HHS National Health Information Center -- Healthnder.gov 75

DOJ National Criminal Justice Reference Service 75

DHS U.S. Citizenship and Immigration Services -- uscis.gov/e-verify 75


16/40


Federal News and Information Websites (continued from page 15)


DOD DoD Air Force -- af.mil 74

DOT Federal Aviation Administration -- faa.gov 74

HHS National Library of Medicine Clinical Trials website -- clinicaltrials.gov 74

DOJ U.S. Department of Justice, Ofce of Justice Programs -- www.ojp.usdoj.gov 74

HHS Health Resources and Services Administration main website -- hrsa.gov 73

Treasury Making Home Affordable -- makinghomeaffordable.gov 73

DOT U.S. Department of Transportation -- fhwa.dot.gov 73

DOD DoD Marines -- marines.mil 72

Treasury Financial Stability --nancialstability.gov 72

SSA Social Security Online: Frequently Asked Questions -- ssa-custhelp.ssa.gov 71

FDIC FDIC Applications -- www2.fdic.gov 69


SEC U.S. Securities and Exchange main website -- sec.gov 68

DOC BEA main website -- bea.gov 67


DOT DOT Research and Innovative Technology Administration website -- rita.dot.gov 65

Treasury USTTB website -- ttb.gov 65

DOT Federal Motor Carrier Safety Administration main website -- fmcsa.dot.gov 63

USDA Forest Service main website -- fs.usda.gov 63

USDA NRCS website -- nrcs.usda.gov 63

HHS HHS -- grants.gov 62

USDA FSIS main website -- fsis.usda.gov 58

DOD TRICARE -- tricare.mil 58

The News and Information aggregate score remained at 75 for Q3 2013, with scores for individual

websites ranging from 58 to 88. The aggregate score for Federal News and Information Websites

was higher than that of private sector news and information websites, which scored 73 in the most

recent ACSI E-Business report.


17/40


Eighteen websites (34%) in this category are considered top performers (scoring 80 or higher). The U.S.

Citizenship and Immigration Services Resource Center and the English version of the MedlinePlus site led this

category, each with a score of 88. The Spanish version of the MedlinePlus site scored 87, followed by the

National Womens Health Information Center main website with an 85.

This category also accounted for 63% of the websites that experienced a signicant increase of three points

or more in citizen satisfaction since Q2 2013 (see chart on page 12). The Health and Human Services Girls

Health website saw the largest score increase for this category and index, jumping six points to 66.

Also in this category, the Department of Defenses Pentagon Channel site jumped into the top performer

category with a four-point increase in satisfaction to 81. The U.S. Nuclear Regulatory Commission website

(79), the U.S. Department of Agricultures Economic Research Service main website (75), and the U.S. Census

Bureau main site (68) all increased by four points from last quarter.

Looking ahead, the challengeas it is with any business or government agency that experiences an increase

in satisfactionwill be not just to maintain, but to grow and improve. If these federal government websites

continue to measure the expectations and experiences of their visitors, they should be able to make the

improvements necessary for continued success.

In spite of the number of top performers and the signicant satisfaction increases that some websites

experienced this quarter, the aggregate category score did not improve and has not improved much over

time. In fact, this category has recorded a citizen satisfaction score of about 75 for the past 10 quarters.

This stagnation is a sign that underperforming websites are not gaining and, more likely, are losing ground.

In this quarter, 15 websites witnessed declines in citizen satisfaction.


18/40


Federal Portals and Department Main Websites

The following chart shows the citizen satisfaction scores for all 33 of the Federal Portals and Department

Main websites in this category.

Federal Portals and Department Main Websites


Portal and DepartmentMain Sites Aggregate

72

HHS National Cancer Institute main website -- cancer.gov 83

DHS U.S. Citizenship and Immigration Services Espaol -- uscis.gov/portal/site/uscis-es 83

HHS CDC main website -- cdc.gov 82

HHS NIAMS public website -- niams.nih.gov 81

NASA NASA main website -- nasa.gov 80

HHS National Institute of Dental and Craniofacial Research -- nidcr.nih.gov 80

DOJ FBI main website -- fbi.gov 79

DOI National Park Service main website -- nps.gov 79

HHS National Library of Medicine main website -- nlm.nih.gov 78

SBA SBA main website -- sba.gov 77

HHS U.S. Food and Drug Administration main website -- fda.gov 77

DOD Department of Defense portal -- defense.gov 76

FTC FTC main website -- ftc.gov 76

GAO GAO main public website -- gao.gov 76

DHS U.S. Citizenship and Immigration Services -- uscis.gov/portal/site/uscis 76

NIST National Institute for Standards and Technology main website -- nist.gov 75

GSA GSA main website -- gsa.gov 73


FDIC FDIC main website -- fdic.gov 72

HHS SAMHSA website -- samhsa.gov 71

ITC U.S. International Trade Commission main website -- usitc.gov 71

NARA NARA main public website -- archives.gov 69


DOS Department of State main website -- state.gov 68

EPA U.S. Environmental Protection Agency -- epa.gov 68


19/40


Federal Portals and Department Main Websites (continued from page 18)


DHS Department of Homeland Security main website -- dhs.gov 67

SSA Social Security Online main website -- socialsecurity.gov 67

VA VA Main website -- va.gov and myhealthva.gov 66

DOT Federal Railroad Administration main website -- fra.dot.gov 64

DOE U.S. Department of Education -- ed.gov 63

DOL Disability -- Disability.gov 59

Treasury IRS main website -- irs.gov 59

DHS Federal Emergency Management Agency main website -- fema.gov 55

The Portals and Department Main Websites category scored 72 for the third quarter of 2013, which

is one point off from Q2 2013. The websites in this category had scores ranging from 55 to 83. While

federal e-government Portals and Department Main Websites are the lowest-scoring category in this report,

they also lag behind the private-sector measure of Portals and Search Engines, which scored 76 in the

July 2013 ACSI E-Business Report.

The National Cancer Institute main website and the Spanish version of the U.S. Citizenship and Immigration

Services site lead the category, with both sites scoring an 83.

Two sites from this category landed on the top gainers chart (see page 12). The U.S. Pension Benet Guaranty

Corp main website increased satisfaction with the citizen experience by ve points (the second highest in the

index this quarter) for a 73. After gaining three points in Q2 2013, the Treasurys main site improved another

three points in Q3 to 69.

As with the News and Information Websites category, some Portals and Department Main Websites saw

improvement, while others experienced signicant decreases that, in turn, lowered the overall category score.

Again, websites that fail to meet the expectations of citizens may not see a return on investment (ROI) if they

do not begin or continue to measure the experiences that citizens have with their websites.


20/40


ForeSee recently conducted an analysis in the private sector to quantify the impact of customer satisfaction

on future revenue among 91 of the top 100 e-retailers. Predicting year-end 2011 revenue with spring 2011

customer satisfaction, we found that a one-point increase in customer satisfaction translated to a 10.6 %

increase in revenues, which corresponded to roughly $72 million. This relationship between customer

satisfaction and revenuesis based on data from the largest 100 e-retailers as designated by Internet Retailer.

This excludes Amazon because Amazon is atypical, even among the top 100. Individual companies may nd

that this relationship differs somewhat when modeling their own data; however, it is clear that there is a

relationship between customer satisfaction, as measured with the ACSI, and actual revenue.

While increasing satisfaction impacts organizations in the public sector differently (such as cost savings

versus revenue) than it does the private sector, the above ndings make a strong case for the critical

importance of providing an excellent customer experience and having tools in place that help quantify ROI.

If measured and managed properly, federal agency and government leaders can make a difference in the

e-government landscape.

Federal Career and Recruitment Websites

Federal Career and Recruitment Websites


CIA Recruitment website -- cia.gov/careers 83

DOL Department of Labor Job Listings -- doors.dol.gov 79

DOS Recruitment website -- careers.state.gov 79

OPM Recruitment website -- usajobs.gov 73


21/40


We have not seen many changes in the Career and Recruitment category, with only four websites currently

measuring citizen satisfaction. None have signicant increases or decreases in satisfaction this quarter.

However, there is a 10-point score range among the participating entities, which means the websites trailing

in this category have ample opportunity to improve considerably. Those who lead should continue to meet

and exceed citizen expectations.

Career and recruitment websites are dependent upon citizens returning to, participating with, and

recommending them in order to be benecial to the agencies maintaining these websites. Therefore, it is

critical for the leaders of these agencies to keep in mind future behavior scores (see Why Satisfaction

Matters below) and the elements that drive satisfaction and impact behavior (see Common Elements of Satis-

faction on page 23).

Why Satisfaction Matters

Satisfaction has been shown to have a direct impact on behavior. If federal government agencies focus on

improving their websites priority areas, citizen satisfaction with their websites should also improve.

Every quarter, this Index compares less-satised website visitors (with satisfaction scores of 69 or less) to

highly satised website visitors (with satisfaction scores of 80 or higher) to produce likelihood scores of what

citizens will do in the future (i.e., participate again, use the website as a primary resource, recommend the

website to others, return to the website, and trust the agency). The following page contains a graph that

shows the range of satisfaction for each measured future behavior.

Based on likelihood scores, citizens who are highly satised with a federal government website rate their trust

in the agency 64% higher. Highly satised citizens also report being 47% more likely than those who are less

satised to participate with the government by expressing their thoughts to the agency.

Satisfaction also increases the likelihood that the citizen will return to the website again (54%), use it as a

primary resource (92%) as opposed to more costly channels, or recommend the site to others (96%).


22/40


HighlySatised

Citizens (80+)

Dissatised

Citizens (< 70)

Difference

in ScoresThe Impact of Higher Website Satisfaction

FutureParticipation

63 43 47%Citizens are more likely to participate with and express their thoughtsto their government, which strengthens the democratic process andmay provide useful feedback.

Return to Site 97 63 54%Government departments and agencies have an ongoing channel toprovide information and services to citizens efciently and relativelyinexpensively.

RecommendSite

96 49 96%Use of government websites will grow as citizens recommend them totheir friends, family, and colleagues.

Use Siteas Primary

Resource

92 48 92%

Cost-savings for departments and agencies can result as citizensare right-channeled to web; citizens get information from a crediblegovernment source, rather than another online/ofine source (in cases

where options exist, e.g., health-related information).

Trust 90 55 64%Citizens believe the agency is trustworthy and acting in their bestinterests, which fosters faith in the democratic process.

These gures illustrate how vital it is to measure citizen experience and demonstrate that the results the

ForeSee methodology produces are more than just numbers. The information here demonstrates that

technology-driven customer satisfaction analytics, when done right, can predict website visitors future

behaviors. Furthermore, the data-driven ndings can guide agencies to make improvements that will increase

desired citizen behaviors.

High customer satisfaction is important for federal websites, but it can be increased.

The federal websites that are using ForeSee to measure citizen satisfaction also measure a number of website

elements, or drivers of satisfaction. Although there are variations in the set of elements that are relevant to

each website, the most common elements are: Search, Functionality, Online Transparency, Navigation,

Look and Feel, Content, and Website Performance.


23/40


Common Elements of the Website Experience

Element What It MeasuresPriority for

Improvement

SearchThe relevance, organization, and quality of search results available onthe site. (Although this element is not applicable universally, it is oftenextremely impactful for sites where it is relevant.)

Priority 1 = Top Priority

FunctionalityThe usefulness, convenience, and variety of online features and toolsavailable on the website.


Online TransparencyHow thoroughly, quickly, and accessibly the website discloses informationabout what the agency is doing.


Navigation The organization of the site and options for navigation. Priority 2

Look and Feel The visual appeal of the site and its consistency throughout the site. Priority 3

ContentThe accuracy, quality, and freshness of news, information, and contenton the website.

Priority 3

Site Performance The speed, consistency, and reliability of loading pages on the website. Priority 3

By measuring these elements, federal agencies can pinpoint and prioritize areas of improvement from

the citizens perspective, which leads to increased satisfaction. If federal agencies fail to scientically

measure and analyze the results, they will have difculty making the changes that will enhance their value

and usefulness to citizens in a cost-effective manner.


24/40


ABOUT THE AUTHORS

Dave Lewan manages ForeSees sales organization focused on the public sector, including government

departments and agencies, non-prot organizations, and higher education institutions. He leads ForeSees

continuing expansion in these markets to help organizations measure and manage satisfaction of

their website visitors. Dave brings more than 20 years experience in sales, leadership, online strategy,

technology, and consulting. Most recently, he served as Vice President of Product Marketing and

Management at Gevity HR, where he acted as the primary leadership and planning force for Gevitys

product and solutions strategy. Dave graduated from the University of Minnesota with a degree in

speech communication.

As president and CEO of ForeSee, Larry Freed is responsible for managing the companys strategy and

signicant growth since the companys founding in 2001. Larry brings more than 20 years of experience

in senior management and in directing ForeSees e-commerce and technology initiatives. An expert on

the cross-channel customer experience and author of Innovating Analytics Word of Mouth Index:

How the Next Generation of Net Promoter Can Increase Sales and Drive Business Results and Managing

Forward: How to Move from Measuring the Past to Managing the Future, Larry speaks extensively on the

topic at private and public sector industry events. He has also been quoted in numerous publications and

media outlets, including CNN, The Wall Street Journal, The Washington Post, Investors Business Weekly,

Internet Retailer, Multichannel Merchant, DM News, Computerworld, Federal Computer Week,and

Government Executive, among many others.

Senior Director of Mobile, Media & Entertainment Eric Feinberg provides leadership to ForeSees

mobile solutions as well as solutions related to the media and entertainment industries. He is responsible

for working with product, delivery, sales, and marketing teams to ensure that ForeSee brings innovation

and operational excellence to its mobile offerings. Eric brings 15 years of customer-focused experience to

the team and is currently on the Board of Directors for the Digital Analytics Association (DAA), formerly the

Web Analytics Association.


25/40


ABOUT THE RESEARCH TEAM

Julie Anderson, research analyst at ForeSee, has over 13 years of consulting and research experience

in public and private organizations, with a focus on large-scale data analytics. Through her work as a

satisfaction research analyst at ForeSee, she has extensive experience with customer experience and

customer satisfaction methodologies, and in providing voice-of-customer analytics with a special focus

on digital analytics.

ABOUT THE ACSI E-GOVERNMENT SATISFACTION INDEX

The ACSI E-Government Satisfaction Index is a special quarterly report of the American Customer

Satisfaction Index (ACSI) in partnership with customer experience analytics rm, ForeSee.

The ACSI, created at the University of Michigan, is the only uniform, national, cross-industry measure of

satisfaction with the quality of goods and services available in the United States, both in the private and

public sectors. In 1999, the federal government selected the ACSI to be a standard metric for measuring

citizen satisfaction. More than 100 federal government agencies have used the ACSI to measure

citizen satisfaction with more than 200 services and programs and more than 100 websites. The report

on ofine federal government services is released annually in December. The E-Government Index is

released quarterly.

ForeSee collects and analyzes the data for the e-government websites included in the report. The

e-government scores were calculated based on data gathered from voluntary online surveys of randomly

selected site visitors. Each government website was rated by its visitors on various components of overall

satisfaction. The ratings were converted to a score on a 100-point scale, using the ACSI methodology.


26/40


The ACSI methodology identies key drivers of online satisfaction (such as navigation, look and feel, search,

site functionality, etc.) and quanties their relationship to overall citizen satisfaction. This cause-and-effect

methodology demonstrates the impact of website enhancements in these areas on overall customer

satisfaction. In turn, customer satisfaction, measured in this way, has been proven to predict how citizens

will behave in the future. Improvements to customer satisfaction will make citizens more likely to choose

to interact with an agency online (the more cost-effective channel), return to the site, and recommend it to

others. Monitoring and improving customer satisfaction has a tangible impact on citizen usage of the web

channel and on the bottom line.

ABOUT THE ACSI

The American Customer Satisfaction Index (ACSI) is a national economic indicator of customer

satisfaction with the quality of products and services available to U.S. consumers. It is updated quarterly

with new measures for differing sectors of the economy, building on the previous years data. The overall

ACSI score for a given quarter factors in scores from more than 200 companies in 44 industries, and from

government agencies over the previous four quarters. The Index was founded at the University of

Michigans Ross School of Business and is produced by ACSI, LLC.

ABOUT FORESEE

As a pioneer in customer experience analytics, ForeSee continuously measures satisfaction across

customer touch points and delivers critical insights on where to prioritize improvements for maximum

impact. Because ForeSees superior technology and proven methodology connect the customer experience

to the bottom line, executives and managers are able to drive future success by condently optimizing

the efforts that will achieve business and brand objectives. The result is better business for companies

and a better experience for consumers. Visit www.foresee.com for customer experience solutions and

original research.


27/40


APPENDIX A

KEEPING PACE WITH MOBILE: FORESEES FIVE TIPS FOR A

BETTER MOBILE EXPERIENCE

WHY MEASURE MOBILE

Theres a saying at ForeSee: You have to know what you know, and you have to know what

you dont know.

What most people know in the mobile space right now relates to the quantity of things: the number

of app downloads, the number of mobile website visitors, and a menagerie of other behavioral data.

This is good information to have; however, theres a bit of a challenge there. Since the popularity of

mobile is increasing, and the number of people using mobile is increasing, the basic metrics around the

experience are increasing, regardless if a company is doing an excellent job or not. This could lead to

misinformation and misguided decisions.

What people need to know is what they dont knowthe quality of things and what is motivating

someone to take action. As mobile use continues to grow, companies need to benchmark against

themselves, their peers, and the market. They also need to understand how mobile is being used. What

makes mobile so perplexing is that it has an impressive duality where it is both a stand-alone channel where

people can engage with it alone and a companion channel where it supports other experiences they might

be having with the brand. This duality makes for a complex, sometimes hard-to-navigate environment.

Knowing exactly how you are doing and where you can improve, whether you are a mobile veteran or

newcomer, starts with listening to your customers. The voice of the customer is louder than any other voice

out there and should be listened to rst and foremost. Start by measuring it.


28/40


29/40


30/40


31/40


32/40


33/40


34/40


In analytics, mobile analytics especially, its all about measurement, measurement, measurement. Here are four

things that will help mobile initiatives succeed:

The most important thing is to do these things continuously. This continuous measurement is vital as mobile

initiatives are being designed, getting implemented, changing, and moving forward. Not only are companies

changing over time, but so are their customers: their needs should be met and their expectations exceeded.

FORESEE SATISFACTION ANALYTICS FOR MOBILE

Measure effectiveness and success

> Apply science to continuous customer experience measurement across mobile-optimized sites,

tablets, and apps

> Measure satisfaction and drivers of satisfaction and impact on future behaviors

>

Gain intelliegence

>

Who is your mobile audience: personas, segments, demographics

Where are your consumers using mobile and when in the lifecycle

Why do they use mobile

> Diagnose strengths, weaknesses and opportunities

Prioritize improvements

> Predict impact of change on satisfaction and behavior

> Provide actionable insights

Benchmark your performance

> Compare against peers, competitors, best-in-class, yourself over time

Measure how mobile inuences and is inuenced by other channels

Prole your mobile users


35/40


36/40


TIP 5: CONNECT THE DOTS

Connecting the dots is a simple exercise of linking a seemingly random group of points in a way that, when

done, results in a complete picture. Connecting the dots with deep linkages to inventory is powerfulits where

everything is headed. This is a technical hurdle to help sew the systems together. However, once achieved theres

still a need to provide a fantastic customer experience for the mobile visitor.

Since the mobile experience is becoming more and more fragmented, using a consistent and powerful

technology to measure across the board will help connect those pieces and bring it all together to make a

complete pictureone representing the best multichannel customer experience possible.

Connecting the dots is all about recognizing that mobile inuences other channels. It might inuence a store

experience, a call experience, a web experience, a social media experience, and/or an email experience. We also

need to understand that mobile is inuenced by these other channels, creating this wonderful bidirectional ow.

As people are engaging at a store and want to look something up, they reach for their mobile device, creating

an extraordinary and quality nexus point to measure in.

When mobile is measured thoroughly with a scientic measurement, executives will get a clear picture of not

just who is coming to the site and why, but also the inuence back to these other channels. Tip 3 talked about

knowing what you know (the quantity of things) and knowing what you dont know (the quality of things).

These together deliver a complete picture of the behaviors and the attitudes of what people are engaging in.


37/40


Attitudes will drive what people do (purchase, purchase again, return to the site, remain loyal, and recommend

to others), and thats an important component when connecting the dots.

The mobile experience continues to get more complex when looking at the impact that mobile has on other

channels. What happens when customers go from the web to mobile to a store, or they go to a store, then

to their mobile device, then to the web, then to social media to ask friends, then back to mobile because they

were using that for their social experience, and then make the nal purchase on the web? How can company

leaders get the most reliable data from a multiple-device experience like that? Better yet, how can they predict

what a multi-device consumer will do next? With the right technologyone that is accurate, precise, and

sensitivethey can measure across all touch points, see where the different experiences intersect, and actually

predict what the consumer will do next.

BEHAVIORS ATTITUDES

FUTURE

BEHAVIORSImpact on

Future Behaviors

RECOMMEND MOBILE SITE

PURCHASE

2.5

2.9

3.4

70

72

78

RETURN TO MOBILE SITE

Dashboard Dashboard

17,311Visits

1.74Visits

69.61%Bounce Rate

Dashboard

Visitors

Traffic Sources

Content

Goals

About the Report

Beta Feedback

Report Finder

Common

Questions


38/40


39/40


Mobile lives before, during, and after key experiences. So when measured wellthe behaviors and attitudes

of what people are doingthe full picture of the customer experience begins to emerge, lending a sense not

only of mobile attribution but of mobile contribution.

CONCLUSION

The mobile experience is becoming more challenging every day because theres so much more to navigate.

There are phones operating across mobile sites and apps; there are different-sized screens and diverse

functions among the various smartphones; and there is an increasing array of operating systems. This makes

mobile a hard and confusing landscape for companies to navigate.

ACQUISITION SOURCES & INFLUENCERS

CUSTOMER

STORES

MOBILEWEB

SOCIAL

MEDIA

CONTACT

CENTER

EMAIL

SOCIAL

MEDIA

EMAILADVERTISINGBRAND SEARCH


40/40


Tracking and counting through traditional metrics can be a little misleading when trying to traverse through

this new multichannel landscape. Just because a companys analytics or information technology teams

measure more people coming to their sites doesnt necessarily mean the company is doing a good job of

meeting the expectations of its visitors. It could mean that more people are using mobile experiences, but it

doesnt mean they are satised with the experiences provided. It doesnt mean theyre going to buy anything,

or subscribe, or come back again.

Mobile is cool and exciting, but executives and decision makers rst have to grow out of the mindset that,

just because theyre developing a mobile experience, it will serve the customer admirably. It might; it might

not. The customer is the key here. Until the customer experience is measured, the dots remain just dots. Only

when they are connected, through measuring both behavioral and attitudinal, can business leaders create the

experiences that customers are looking for and deserve.

As far as mobile has come already, it is still a new and nascent industry full of possibilities and open to

innovation. Instincts are important when developing mobile experiences and should always be trusted.

If there is an adventurous and exciting mobile idea on the boardroom table that everyone believes will

work, it probably will.

Mobile is the one channel where theres still the possibility of piquing peoples interest, changing the ways they

engage with companies, and offering them the ability to do things that are easier, faster, and more convenient

than ever imagined. Theres a certain exibility in mobile right now that offers companies an opportunity to

do what they want and to do it in the best interest of the consumer to positively inuence the other channel

experiences that they have control over.

Before implementing any of these tips, though, companies need to rst talk with their customers and nd out

what they want, need, and expect, by measuring their experiences. Then, from their perspective, companies

can design a mobile experience that works for the customer and for the company.

Documents

Q3 2013 e-gov commentary FINAL.pdf