Shopzilla's "You Get What You Measure" - Velocity 2009

Shopzilla Site Redesign – We get what we measure

Phil Dixon, VP Engineering

Velocity 2009

June 23rd, 2009 | Velocity 2009

Phil Dixon – VP, Engineering | Velocity 2009 | June 23rd, 2009

2

Shopzilla, Inc. - Online Shopping Network

100M impressions/day

20-29M UV’s per Month

8,000+searches per second

100M+Products


3

Background• In 2000, the business was simple - bizrate.com in the US

• Co-brands, API and shopzilla.com

• Multivariate testing, Europe and multiple data centers

• Always in a rush

• Difficult to change, differentiate

• Site had become slow between 4-6 second page loads


4

Our Build Approach

• Start over, stay simple

• Build 2 weeks at a time, deliver every page ASAP

• Manage risk by managing exposure


5

Shopzilla Site Technology Overview


6

Anatomy of a Page Request


7

digitalcameras.bizrate.com – Server Side

SessionizationTaxonomy

KeywordTaxonomy

SEO ContentProduct Search

AttributionUser Content

Category SearchSponsored Links

Search Log 1Product Reviews

User ContentSEO ContentSearch Log 2SEO Content

TaxonomyRelated Searches

Product DetailsRelated Searches

BannerBanner

AttributionBannerBannerBannerBannerBanner

Access Log 1Access Log 2

0 25 50 75 100 125 150 175 200 225 250


8

SLA’s – Key to High Performance

• 650ms server side

• 1.5s user experience

• At the 95th percentile

• Under peak loads

• 0% error rate


9

Banners; 19%

Images; 30%

Other; 51%

Most Performance Gains in the Browser

Server Response Time

Rest of the Page

Images to non-cookie domain resulted in

0.5% top line revenue increase!


10

Can you spot our release dates?

June 1, 2008 November 30, 2008

7/1/2008 10/1/2008

7/1/2008Released remaining

shopzilla.com pages

11/17/2008bizrate.com full

site launch

11/3/2008bizrate.com launched to

40% of traffic10/13/2008Released 100% traffic

shopzilla.com

shopzilla.com 10/13/08

bizrate.com 10/17/08


11

Gratuitous Graph Slide


12

Yeah, yeah… but did we make money?


13

Site conversion rates increased 7-12%

Revenue = sessions x conversion % x CPC


14

9/1/

2008

9/5/

2008

9/9/

2008

9/13

/200

89/

17/2

008

9/21

/200

89/

25/2

008

9/29

/200

810

/3/2

008

10/7

/200

810

/11/

2008

10/1

5/20

0810

/19/

2008

10/2

3/20

0810

/27/

2008

10/3

1/20

0811

/4/2

008

11/8

/200

811

/12/

2008

11/1

6/20

0811

/20/

2008

11/2

4/20

0811

/28/

2008

84.00%

86.00%

88.00%

90.00%

92.00%

94.00%

96.00%

98.00%

100.00%

Sessions to Google Clicks Ratio

Bizrate

Shopzilla

Performance Impacts Abandonment

June 1, 2008 November 30, 2008

7/1/2008 10/1/2008

7/1/2008Released remaining

shopzilla.com pages

11/17/2008bizrate.com full

site launch

11/3/2008bizrate.com launched to

40% of traffic10/13/2008Released 100% traffic

shopzilla.com


15

Performance Penalties & bizrate.co.uk

~120% SEMSessions


16

Performance Summary

• Conversion Rate +7% - 12%

• Page View’s +25%

• US SEM Sessions +8%

• Bizrate.co.uk SEM Sessions +120%

• Infrastructure Required (US) -50% (200 vs 402 nodes)

• Availability 99.71% 99.94%

• Product Velocity +225%

• Release Cost $1,000’s $80


1717

Is Performance Worth The Expense?

YES!


18

Key Decision: Refactor or Rebuild


19

Simplicity, Speed and Qualityare design decisions


20

Performance Gain Can Be Unexpected


21

We Get What We Measure


22

Lots More To Come


Thank You!

Blog: tech.shopzilla.com

Email: [email protected]

Jobs: jobs.shopzilla.com

June 23rd, 2009 | Velocity 2009