1 June 2013

State of web readineSS: e-commerceE-commerce sites are some of the worst performing as a category. Even though availability and performance directly impact their bottom line.

3

The resulTs of This, our second report on the state of web readiness, surprised us.We analyzed the data from over 5,000 performance tests and more than 500

user surveys and concluded that, on average, e-commerce sites perform well below expectations; and often with profound performance issues. The reason for our sur-prise is that, based on usage patterns on loadimpact.com, e-commerce sites spend substantially more on performance testing than other categories. Performance, it would seem is very important to them, and with good reason; a number of studies have shown the direct correlation between increased latency and decreased custo-mer conversion.

The data also revealed that e-commerce sites are slower than other website cate-gories. The average time it took to load a page on an e-commerce site was almost 8 seconds while other sites exhibited page load times somewhere between 4 and 5.5 seconds. The average for all sites, including e-commerce, was 5.6 seconds.

According to survey participants, e-commerce sites more often lose money and visitors due to performance or stability problems than other site categories. 40% of e-commerce respondents reported losing money in the last year due to performan-ce/stability issues. This compared to 25% of sites in other categories. Clearly there is a need for change.

Good performance is crucial to e-commerce, as the correlation between perfor-mance and increased sales is well understood. Perhaps even more important is the impact of poor performance on the brand of a site. So why is it that despite all this, e-commerce sites seem to be doing worse than others? We believe it’s a combina-tion of several factors:

e-retailerS inveSt the moSt in web performance but have SloweSt SiteS

4

• Rapid growth in e-commerce traffic means that reality outstrips projections; and the sites have no reliable means of tracking this.

• E-commerce requires more complex, dynamic, user generated content – and latency inducing back-end calls - than other types of websites.

• E-commerce websites may simply have a higher awareness of these issues since performance failures directly impact sales statistics.

Conclusion: E-retailers are more sensitive to and invest more in performance than other groups. Yet e-commerce websites are, as a category, the slowest and even they rate their own performance as inadequate. Worst of all, it is more common for e-commerce sites to report losing money due to performance than it is for non e-commerce sites.

The Load Impact TeamJune 2013

5

brief• Sixty-eight percent of all survey respondents experienced performance or

stability problems with their website last year. Lack of resources was the main reason for these problems, both among e-commerce sites and other websites.

• Thirty-nine percent of e-retailers claimed they lost money last year due to performance or stability problems. In comparison, only 24 percent of the non e-retailer respondents claimed they lost money.

• Sixty percent overestimated their site’s capacity to handle traffic according to their load testing results. This level of overconfidence was similar across all website categories.

• Ninety-eight percent of e-retailers thought sub two-second response time was desirable. However, load time for e-commerce sites analyzed exceeded this re-sponse threshold by a wide margin. The average was 7.9 seconds - higher than for non e-commerce sites.

• Low latency, while important for all websites is crucial for e-retailers. Also e-commerce sites are more likely to proactively analyze and predict response times.

• Even though it is common to deploy mobile versions of sites it is uncommon to load test of mobile versions of sites.

METHodoLoGY• Based on analysis of Load Impact test data from over 5,400 independent

tests in 118 countries.• Based on 538 user survey responses.• Tests and survey conducted September – october 2012.

6

failure to perform: poor performance and itS effecton revenue

68 %

Yes

No

32 %

Sixty-eight percent of all respondents claimed to have experienced performance or stability problems with their sites in the last year. The response of e-retailers was not significantly different from non e-retailers.Sixty seven percent of e-retailers responded that they had problems during the last year.

Q: Have you experienced any performance or stability problems on your site in the last year?

7

The most common reason for performance problems were ‘lack of resources’, and ‘unexpected events’.

0 %

10 %

20 %

30 %

40 %

50 %

OtherProblem with

a cloud service

Unexpected events

BugsLack of information

on the expected number

of visitors

Lack of information about the

sites capacity

Lack of resources

46 %

27 % 26 %

35 %37 %

16 %

6 %

Site ownerS – all categorieSQ: What are the major reasons behind performance problems on your website?

8

‘Lack of resources’ was less common among e-commerce sites as compared to other categories. However it remained the most common reason for performance problems, with ‘bugs’ coming in a close second.

0 %

5 %

10 %

15 %

20 %

25 %

30 %

35 %

40 %

OtherProblem with a cloud

service

Unexpected events

BugsLack of information

on the expected number

of visitors

Lack of information about the

sites capacity

Lack of resources

40 %

26 %

29 %

39 %

29 %

13 %

7 %

Site ownerS – e-commerceQ: What are the major reasons behind performance problems on your website?

9

E-commerce sites say they lose revenue due to performance or stability problems far more often than other sites. Thirty-nine percent of e-commerce site owners claimed they lost money in the last year.In comparison, only 24 percent of the other respondents claimed to have lost money.

0

10

20

30

40

50

60

Don't knowDidn’t losemoney or customers

Lost moneyor visitors

53 %

15 %

28 %

33 %

25 %

E-commerce

Other

48 %

Q: Have you lost money or visitors due to poor performance or stability on your site during the last year?

10

When analyzing the load tests in the survey we identified a trend we came to call the ‘overconfidence factor’. The overconfidence factor describes sites that make incorrect assumptions regarding their capacity. It is produced by looking at how many concurrent users a load test was configured to simulate, and comparing that to the load level at which page load time had increased by 100 %.

falSe confidence overeStimating capacity iS prevalent

58 percent of owners had overconfidence in their site’s capacity. The overconfidence was fairly even throughout the field, but media sites stood out as being slightly more realistic about their site capacity - only 50 % of media sites overestimated their capacity.

0 %

10 %

20 %

30 %

40 %

50 %

60 %

70 %

80 %

TotalOtherOnline service

MediaE-commerceCorporate, product

and interest

Campaign and event

56 %61 % 62 %

50 %

61 %58 % 58 %

How many have overconfidence in the capacity of their site to handle traffic?

11

reSponSe time

Analysis of results from 5,240 load tests show that average initial (i.e. with no simulated traffic on the site) page load time for e-commerce sites is 7.5 seconds – much higher than other websites. This contrasts with the e-retailer sentiment that acceptable response times are 2 seconds or less.

0

1

2

3

4

5

6

7

8

OtherOnline service

MediaE-commerceCorporate, product

and interest

Campaign and event

4.8 4.8

7.5

4.8

4.2 4.3Seco

nds

User load time on average by category

12

Professed tolerance for load times seems to correlate with industry latency /conversion data which shows a dramatic fall off in conversion after 1 sec page load times

0 %

5 %

10 %

15 %

20 %

25 %

30 %

35 %

Over 4 sec2 – 4 sec1 – 2 sec0.5 – 1 sec0.25 – 0.5 secUnder 0.25 sec

8 %

4,317 %

22 %

31 %

21 %

2 %

Resp

onse

per

cent

Q: What’s an acceptable load time on an e-commerce site?

13

0 %

10 %

20 %

30 %

40 %

50 %

60 %

Very importantImportantLess importantNot important

1 %

4,2

0 %

6 %3 %

42 %

37 %

52 %

59 %

Resp

onse

per

cent

E-commerce

Other

Q: How important is a short response time?

Short response times are important to everyone, but especially so for e-retailers.

14

0 %

10 %

20 %

30 %

40 %

50 %

Don't knowWe don't monitor

the response time on our site

AlarmsLoad testing before

predicted tra�c peaks

Preventive load testing

before technical changes

Scheduled load testing

42 %

E-commerce

Other

28 %

34 %31 %

39 %

30 %

45 %

36 %

14 %

23 %

4 %3 %

14

E-retailers actively work to monitor or predict response times. Only 14 percent of e-commerce sites claimed they did no performance testing. This compared to 23 percent of sites in other categories.

Q: Which activities do you engage in to monitor or predict response times on your site?

15

download this and future reports atloadimpact.com/readiness

Even though it is common to deploy mobile versions of sites it is uncommon to load test these. However, the e-commerce site owners tend to test more often (17 % ) when compared to other site owners (13 %).

going mobileQ: Do you perform load tests on the mobile version of your site?

0 %

10 %

20 %

30 %

40 %

50 %

60 %

We don't have a mobile version

Don't knowNoYes

17 %

E-commerce

Other

13 %

50 %

55 %

4 %2 %

29 % 30 %

0 %

10 %

20 %

30 %

40 %

50 %

60 %

We don't have a mobile version

Don't knowNoYes

17 %

E-commerce

Other

13 %

50 %

55 %

4 %2 %

29 % 30 %

about load impactLoad Impact is the world’s largest cloud-based, on-demand load testing tool, with more than 800,000 load tests executed since the service was launched in 2009. Load Impact has earned the trust of over 50,000 users who continuously use the service to ensure the quality of their online presence. This allows each organization to allocate the appropriate amount of infrastructure resources, and to spot potential performance problems before they become an issue, providing an accurate, scalable and proactively optimized service for the end user. Load test your site at loadimpact.com