Testing Ideas for Ecommerce Websites

Embed Size (px)

Citation preview

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    1/45

    Common Testing Ideas forE-commerce Websites

    A B

    1Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    2/45

    About this Deck (2)

    Homepage (3)

    Search Results (12)

    Category and Subcategory (15)

    Product Page (20)

    Shopping Cart (25)

    Checkout Flow (30)

    Broad Strategies (35)

    2Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    3/45

    This deck provides Optimizely customers with a resource for learning about commontesting ideas on e-commerce sites.

    The specific examples were chosen at random from Optimizely clients and non-clients alike. The examples are purely a representation of what couldbe tested and are

    not necessarily representative of actual tests that have been run.

    No explicit test data has been shared without the explicit consent of the clients.

    3Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    4/45

    The following slides highlight common optimization ideas for homepages.

    4Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    5/45

    WhatMost search boxes simply read Search or have notext at all.

    Tests

    Show text suggestive of the range of productsyour site offers or the ways search can be used.

    Test showing a search CTA and the value of anicon CTA vs. a word CTA.

    Test Benefits

    Easy to setup Low risk Easily targeted to any page on your site

    Potential Impact

    Increase product page views and conversion bydriving users to products faster Increase average order value by highlighting

    products not previously considered.

    ConsiderIf your search results page is not strong but isschedule for a revamp, this is a great test to see ifyou can drive more users in that direction.

    Alternate Concepts

    Common Default

    Product Specific

    Inspirational

    Functionality-focused

    Functionality-focused

    Functionality-focused

    5Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    6/45

    Alternate Concepts

    Common Default WhatMany scrolling banners span the width of the page,move too quickly, make it hard to navigate betweenthem and have inconsistent placement of the CTAbetween images.

    Tests

    Eliminate auto-scroll through the hero Increase the prominence of hero navigational

    controls with color, location or hover states Reduce the hero to 2/3 width and show other

    top content above the fold Grid layout of images instead of single image Use of javascript scrolling banners instead of Flash Product images vs. Lifestyle images

    Potential Impact Reduce homepage bounce rate Increase awareness and click-through on

    important hero messaging

    ConsiderThere may not be an increase in hero click-throughbut be aware of changes to other key metrics downthe funnel.

    If your hero content will be changing during the runof the test, consider how this may impact certainmetrics. (For example, prominent sale messagingduring the second week of the experiment maydrive a conversion rate increase that week but adecrease in average order value.)

    Finally, consider targeting your hero content orlayout based on previous on-site behaviors(categories visited, product page visited, itemcurrently in cart etc.).

    Easy navigation

    Consistent CTA location

    No rotation

    Centered hero

    Text promos on left

    Grid layout, small images onright

    Difficult navigation

    Fast rotation

    Inconsistent CTA location

    6Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    7/45

    Alternate Concepts Continued

    No rotation

    Simple CTA links

    Simple CTA link

    Clear navigation

    Content on right

    Lifestyle image

    7Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    8/45

    WhatMany homepages use the space below the hero to showcase special

    offers and featured products, much of which is likely to be ignored in favorof a user clicking the top-level navigation.

    Tests

    Showcase category content Varying amount of text to describe categories (simple title vs. title and

    description

    Clear and highly noticeable hover state to increase a feeling ofinteractivity

    Special offers

    Individualproducts

    Common Default

    Potential Impact

    Reduce homepage bounce rate Increase in AOV as additional products/categories are considered

    ConsiderWhat are your top user paths from the homepage? If its interaction withcertain navigational elements, test showcasing those outside of thenavigation.

    Alternate Concept

    No hero (top andside banners instead)

    Special offers belowcategories

    8Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    9/45

    Alternate Concepts Continued

    Category images

    Clear categorylabeling

    9Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    10/45

    WhatMany sites do not make clear what they do or whythey are different. Simply because a user has landedon your site, doesnt mean theyre convinced theyshould buy from you.

    Tests

    Test an alternate tagline or marketing slogan Make tagline more prominent by your logo Explicitly state your value props

    Test Benefits

    Easy to setup - star t with adding text Low risk - A/B testing is the simplestand cheapest

    way to test slogans and taglines

    Easily targeted to any page on your site

    Potential Impact

    Increase conversion

    Consider

    You may not get statistically significant results withthis test - particularly if you only target it to thehomepage which is far removed from your ultimateconversion event - but measure other metrics whichcan tell you if youre moving in the right direction.

    For example, an increase in product page views ortime on site when you have a price pointdifferentiation tagline may lead to a variety of othertests focused on price messaging.

    Alternate Concepts

    Common Defaults

    Tagline not focused on price differentiation

    Tagline hard to read

    Opportunity to communicate how theservice works is missed

    Scope of offerings and pricevalue made clear

    Tagline with clear explanation, directly above a link to learn more -great for sites with lots of organic, non-branded terms

    Site has six layers of navigation above the hero images. Value props (FreeShipping, International Shipping, No Hassle Returns) can easily be missed.

    10Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    11/45

    Alternate Concept

    User has previously visitedMaternity, the Maternityhero display on default.

    Common Default

    User has previously visitedMaternity, but that content

    does notdisplay on default.

    WhatDedicate a section of the homepage to servetargeted content based on pre-defined criteria. Thismay be based on buckets of search terms, geo-location or previous on-site behavior.

    Tests

    Location of targeted content (e.g. above/belowthe fold, main hero vs. sidebar etc.)

    Specificity of targeted content (e.g. target withproduct image vs category image)

    Test Benefits

    Moves your site in the direction ofpersonalization - opportunity to take your testingto the next level

    Potential Impact

    Increase conversion Increase RPV

    ConsiderStart with a simple targeted test that will get a lot oftraffic. This may be a common organic search term,or reading a cookie that was set based on a uservisiting a section of your site.

    11Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    12/45

    The following slides highlight common optimization ideas for search results pages.

    12Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    13/45

    WhatMany sites default their search results to FeaturedItems. This doesnt communicate to the useranything meaningful about the way the resuls areshown.

    Tests

    Default to a user generated search order like toprated, most popular or best sellers.

    Default based on price (low to high or high tolow)

    Default based on recency like New Arrivals

    Test Benefits

    Easy to setup - with a bit of simple code (orpossibly a redirect to a different URL), there is no

    change to the layout of the page

    Potential Impact

    Increase in conversion Increase in product page views Decrease in bounce rate Increase in searches

    Common Default

    Defaults to Featured Items

    Default to...

    Price (low to high) Price (high to low) Customer Top Rated Best Sellers New Arrivals

    13Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    14/45

    WhatMany sites default to showing all their filters on theleft side of their page expanded. This makes itdifficult to see the filtering options available or tonotice the most important filtering options.

    Tests

    Defaulting to all the filters collapsed Defaulting to some(but not all) of the filters

    collapsed - only showing the most commonlyused ones expanded

    Changing the order of the filtering options tomake the most important ones show at the top.

    Test Benefits

    Easy to setup - with a bit of simple code (or

    possibly a redirect to a different URL), there is nochange to the layout of the page

    Potential Impact

    Increase in conversion Increase in product page views Decrease in bounce rate Increase in searches Increase in interaction with filters (bringing

    broader awareness of product offerings)

    Common Default

    Only one main filtering option isshown above the fold with

    many sub-categories shown.

    Alternate Option

    All the main filtering options areshown above the fold and noneof the sub-categories aredisplayed.

    14Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    15/45

    The following slides highlight common optimization ideas for category and subcategory pages.

    15Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    16/45

    Common Default

    The subcategory page has five images across.

    What

    Subcategory pages that show a grid of images often are too small make out anydetail for the individual products, requiring users to click through to the productpage to determine if theyre interested.

    Tests

    Test using larger images so the product details can be more easily made out.

    Potential Impact

    Fewer product page views per session but more subcategory page views anda higher conversion rate

    Alternate Concept

    The subcategory page has larger images and shows only threeper row.

    Consider

    This may be a technically difficult test if done on every subcategory page of thesite. But a simple way to do a test like this is choose one subcategory, or evensimply the first page of a subcategory, and manually code that page with larger

    images. If you get a higher conversion rate on those products, this may be a signthat its worth trying on other subcategories in a more automated fashion.

    16Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    17/45

    Common Default

    The subcategory page has no descriptive text for any of theproducts.

    WhatTheres a balance between having too much clutter on the page andproviding a bit of extra detail that might entice viewers to take a closerlook.

    Tests

    Add a bit of detail, either product description, or specification details(sizes and measurements material types etc.).

    Potential Impact

    Fewer product page views per session but more subcategory pageviews and a higher conversion rate.

    Alternate Concept

    The subcategory page has the first part of the descriptionfrom the product detail page and a read more link.

    Consider

    This may be a technically difficult test if done on every subcategory pageof the site.

    But a simple way to do a test like this is choose one subcategory, or evensimply the first page of a subcategory, and manually code that page withthe first part of the descriptive text. If you get a higher conversion rate onthose products, or a higher product page view rate on those products, thismay be a sign that its worth tr ying on other subcategories in a moreautomated fashion.

    17Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    18/45

    Common Default

    The compare box is far from the image and thelast item before the price.

    WhatIf your site offers the option to compare products, this can be a significant toolto help users make decisions on products with many details.

    Tests

    Move the Compare button closer to the product image. Change the text to read Compare Products. Add hover state text that explains how it works (e.g. Select up to three

    products to compare features.)

    Make the Compare text larger

    Potential Impact

    Increase in conversion rate Increase in Compare feature use

    Alternate Concept

    The compare box is immediately below the image.

    Test BenefitsThis is a relatively easy test to setup and apply to a wide variety of pages.

    Consider

    If youre applying this to a variety of category or subcategory pages, make surethose pages are all using the Compare feature and have the same basictemplate.

    18Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    19/45

    Common Default

    The FAQs, Buying Guide, andcustomer service phone number arein the far corners of the page andnot likely to be noticed. (FAQs andBuying guide link are below the fold.)

    WhatIf your site sells products that require a significant amountof research prior to making the purchase, test giving moreprominence to FAQs, buying guides and customer servicefeatures.

    Tests

    Move the FAQs, Buying Guides or other support optionshigher on the page above the results filter.

    Move the support options as a row between theproduct rows to make them obviously noticed after theuser has scanned the first products.

    Potential Impact

    Increase in conversion rate Increase in time on site Increase in return visitors as users think of your site as an

    industry expert and a place to star t their research

    Test BenefitsDepending on your site, this should be very easy to set upusing only the Optimizely visual editor using therearrange drag and drop functionality.

    Consider

    Even if your site doesnt have extensive buying guides, testthe prominence of your help features and any FAQs youmay have. If youre considering creating content like buyingguides, this will give you the data to support your case forprioritizing those efforts.

    Alternate Concept

    Need more help?Call us!

    1300 000 500

    The FAQs, Buying Guide, andcustomer service phone number areplaced high on the page. Users willstill be likely to find the NarrowResults option because they areused to finding this in the left column.

    19Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    20/45

    The following slides highlight common optimization ideas for product pages.

    20Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    21/45

    What

    Every product page has product images. But not every page has lifestyleimage which can help communicate additional information (such as sizeand fit, common uses) and provide inspiration.

    Tests

    Add lifestyle images in the primary image area Add lifestyle images in the description area of the page

    Common Defaults

    One product imageis shown for twodifferent product

    colors. Product isnot shown in alifestyle context.

    Multiple productimages are shownincluding two

    lifestyle images in theproduct descriptionarea.

    Test Benefits

    Learn if users are motivated by lifestyle imagery and test this concept inother places on your site

    Test on many pages or only a few pages

    Potential Impact

    Increase conversion Increase time on site

    Alternate Concept

    21Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    22/45

    What

    Finding the right balance of showing related products and not distractingusers with too many options (creating FUD - Fear, Uncertainty, Doubt) isunique to your product offerings and your site. This makes it a great test.

    Tests

    Remove showing related products Show fewer/more related products Test alternate algorithms (higher/lower price point, same/different brand

    only, highly rated etc., users who viewed vs. users who bought)

    Common Defaults

    Users are showncompeting productsimmediately by the

    description of theproduct page theyare currently on.

    Competing productsare removed andonly People who

    bought alsobought... is shownlow on the page.

    Test Benefits

    This test can easily be done using Optimizelys visual editor.

    Potential Impact

    Increase conversion

    Consider

    What works for one product category may not work for others. Createsegments to determine what types of products users are converting onmost or run this as multiple test at separate times on different categories.

    Alternate Concept

    22Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    23/45

    Common Defaults

    Free Shipping is displayedabove the navigation andbelow the product details

    as part of the generalizedvalue props.

    WhatFree shipping is regularly shown to significantly affectuser behavior. Before you optimize your offers based onprice, make sure users are seeing the offer in the first

    place.

    Tests

    Increase Free Shipping prominence by displaying it inthe product details

    Duplicate the same look and feel for the generalizedmessage as for the this product qualifies message

    Test various price points Test adding a shipping logo instead of simply text

    Test Benefits

    This test can teach you how promotional offers canimpact user behavior (do users spend more or convertmore to get special offers) and its also easy to setup.

    Potential Impact

    Increase in conversion Increase in average order value (AOV) Increase in revenue per visitor (RPV)

    Consider

    Consider creating conversion goals for multiple price-

    points. For example, does your free shipping offer workbest for users who are buying over a particular dollaramount?

    Also, if your site doesnt currently show (on theproduct page) when a product qualifies for freeshipping, using a bit of custom javascript, you add thisfunctionality in with Optimizely and measure its impact!

    Alternate Concept

    The same Free Shippingstyle shown above thenavigation is shownimmediately below theprice in the product details

    area.

    23Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    24/45

    Common Defaults

    The ratings and reviewscontent may be difficult tonotice since its surrounded

    by a lot of other content.

    WhatThe prominence of ratings, reviews or even usertestimonials (on ecommerce but non retail sites) canstrongly influence users behavior for certain types of

    products. Make sure this content gets noticed.

    Tests

    Move ratings and reviews content closer to the price Add hover states to the ratings so it feels more

    interactive and may be noticed on incidental hovers

    Use larger ratings and reviews stars Make testimonials stand out by adding large quotes

    or other stylistic changes Make testimonials stand out by adding a users image

    Test BenefitsThis test can often be done by moving existingelements around on the page using the visual editor.

    Potential Impact

    Increase in conversion Increase in average order value (AOV) Increase in revenue per visitor (RPV)

    Consider

    Try an extreme version with very large review stars.

    Create segments for products with no reviews, lowreviews, medium reviews and high reviews (based onstar ratings).

    The goal of this is to see if interaction with reviewsgoes up and by segmenthow the more prominentversion affected conversion.

    Alternate ConceptThe ratings and reviewscontent is immediatelybelow the price in an areathat almost all users eyesare likely to look. Thecontent is also given its owntab at the top of the page.

    24Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    25/45

    The following slides highlight common optimization ideas for shopping cart pages.

    25Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    26/45

    WhatUsers who arrive on the shopping cart page aredoing only one or two things: reviewing their cart toconfirm they have the right items in the rightquantities (and making changes if they dont), and

    moving on to the checkout page.

    In short, the shopping cart page is about usability.The most usable pages dont make the user think.Having an obvious checkout button that draws theusers attention is key.

    Tests

    Use a button color that stands out (its ok if its abit off brand - as long as it stands out)

    Provide more empty space around the button

    Use a larger button and text

    Show checkout buttons above and belowthe cartto make them easy to find

    Add a hover state to the button to make it standout

    Test Benefits

    Easy to setup - image buttons can easily bereplaced, HTML-based buttons can modify theircolor with basic CSS

    Close to your final conversion event, small liftscan translate to big ROI

    Potential Impact

    Increase conversion

    ConsiderNot every small change on this page will make adifference for your ultimate conversion event. But ifyou move more people into the checkout flow,youre probably on the right track.

    Common Default

    The checkout CTA is thesame color as the topnavigation.

    Alternate Concept

    The checkout CTA is acontrasting color to the restof the page. There are avariety of things that could beoptimized on this button andpage but if nothing else, the

    button immediately standsout.

    26Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    27/45

    WhatWhen there is an empty field - particularly on pageswhere users need to take some kind of action -users attention becomes focused on it and will oftensearch the web for a coupon code which takes

    them away from your site resulting in a lowerconversion rate.

    Hiding this field behind a link preserves the essentialfunctionality for users who genuinely do need toenter a code, but does not distract users who donthave it.

    Tests

    Hide the field behind a link and expose the fieldwhen the link is clicked

    Remove the CTA associated with the promocode field and capture the coupon details on thenext page

    Test Benefits

    No backend manipulation is required if the field issimply hidden behind a link

    Potential Impact

    Increase conversion Decrease page bounce rate

    ConsiderIf possible, try testing the page within the checkoutflow that is optimal for displaying the coupon code.The example here shows it on the shopping cartpage but it may be better to have it on the paymentpage when a user is more deeply committed tobuying from your site.

    Common Default

    The coupon code field isexposed and has a prominentCTA alongside it.

    Alternate Concept

    The coupon code field ishidden behind a link. Whenthe link is clicked, the field andthe CTA are exposed.

    27Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    28/45

    WhatRelated, or recommended products are often shownon the shopping cart page. Depending on theproducts in the cart and what your business sells,this may make sense.

    But it may also distract the user from completingtheir purchase or even confuse them if theyre tryingto confirm what items they are about to purchase.

    Tests

    Remove related products altogether Show products only within a certain price range

    (usually lower works better) Show products only from different categories

    than what is in the users cart Clearly label the recommended products as

    Related items or Users also bought

    Test Benefits

    If you add nuanced segments based on averageproduct price (in car t), total cart value, productcategories etc., you have the ability to gain in-depth insights into the patterns and behaviors ofyour users. For example, High spenders , buyingelectronics, over $300, are likely to add onproducts when presented with electronicaccessories but users spending less than $50 are

    not.

    Potential Impact

    Increase conversion Increase average order value (AOV)

    ConsiderThere is tons of opportunity for personalization andtargeting based on the results of these types of tests.

    Common Default

    Recommendedproducts within thesame product categoryare shown in the rightsidebar.

    Alternate Concept

    Only one recommendedproduct is shown and it isnot a related product.

    28Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    29/45

    WhatUsers are often given multiple options on theshopping cart page that drive them away fromchecking out. There is a balance between givingusers options and suggesting new products and

    driving users towards the primary goal. Test your siteto see what works best.

    Tests

    Remove the top navigation and replace it with acontinue shopping or return to shopping link

    Keep the top navigation but remove thecontinue shopping CTA next to the checkoutbutton

    Test Benefits

    Easy to setup

    Potential Impact

    Increase conversion Increase revenue per visitor (RPV)

    ConsiderWhat other distractions do yo have on the shoppingcart page that may be driving users away from theprimary goal?

    Common Default

    Alternate Concept

    The top navigation is shownand immediately to the left ofthe checkout button there is a

    continue shopping button.

    The top navigation is notdisplayed but is replaced witha

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    30/45

    The following slides highlight common optimization ideas for checkout flows.

    30Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    31/45

    WhatOnce the user has indicated he is ready tocheckout, the last thing you want to do is makethem believe they must sign up for an account. Evenif your preference is to have them sign up, a

    conversion without signing up is better than noconversion at all.

    Tests

    Exposing only fields for one section Do not expose fields or a CTA for any section

    until the user has made a selection, then exposefields and a single CTA

    Listing easy-to-read value props Use a title at the top that makes it clear the user

    has a choice (e.g. Begin Checkout: Select an OptionBelow)

    Display choices left to right or top to bottom

    Test Benefits

    Results are fairly clear-cut - users either advanceddown the funnel at a higher rate or they did not

    Potential Impact

    Increase conversion

    Common DefaultExposed sections show all theirfields.

    Guest checkout is hidden.

    Alternate Concept

    Only one section shows fields.

    Clear value props are listed foreach section.

    31Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    32/45

    WhatWhich of the forms on the left would you rathercomplete? Forms equal work. When a user sees abunch of fields it can be overwhelming. However onshipping and billing pages, users often are entering

    the same information twice.

    Instead of forcing the user to make a selectionindicating their information is the same, force themto make the selection only if their information isdifferent. This will reduce the likelihood of the userfeeling overwhelmed and abandoning.

    Tests

    Select the shipping same as billing checkbox bydefault and hide the fields

    Potential Impact

    Increase conversion

    Consider

    This test does nothave to involve backend codingchanges.

    With a bit of custom javascript code entered inOptimizely, the fields can simply be hidden. As theuser enters information in the Billing section, theinformation will also be entered in the Shipping

    section. Then, if the user deselects the same ascheckbox, the information is removed and the usercan enter fresh information.

    Common Default

    The same as box is notselected by default and allthe shipping fields areexposed.

    Alternate Concept

    By default, the same asbox is selected and thefields are hidden fromview.

    32Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    33/45

    WhatMuch like an empty form field, when users see adrop-down menu, they often think they need (orfeel compelled) to interact with it. If you have ashipping cost estimator or similar functionality, those

    can drive users away from converting.

    Tests

    Hide the shipping cost estimator behind a link Move the shipping cost estimator functionality

    deeper into the funnel

    Test Benefits

    By preserving functionality on the page, nobackend changes are required to run this test.

    Potential Impact

    Increase conversion

    Common Default

    Users can estimateshipping costs here butthis is not actually where

    they select the shippingmethod.

    Alternate Concept

    All the functionality is the

    same but it is hiddenbehind a link.

    33Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    34/45

    WhatWhat are the most important messages for users toread in your checkout funnel? Do you know iftheyre reading them? Are they obvious and easy tosee? Are they located in the right place? Test making

    the messages in your funnel more prominent.

    Tests

    Use larger text for messages Place a box around the message or add a graphic

    (perhaps showing an ! to indicate importance)

    Move it earlier or later in the funnel

    Test Benefits

    Easy to run - minutes to setup

    Potential Impact

    Increase conversion

    ConsiderA messaging test like the example shown not onlyhelps you fix a potential usability problem, but ithelps you better understand the fears, uncertainties,and doubts users may be having on your site.

    In this example, its a travel site that allows you tobook a vacation home directly from the owner. Ifthis alternate concept were to win, it might signal

    that showing additional trust messaging elsewhereon the site or within the conversion funnel mightlead to an even higher conversion rate.

    Common Default

    The message isgray and hard toread.

    Alternate Concept

    The text is black,large and easy toread.

    34Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    35/45

    A successful testing program relies on more than just a few good ideas.

    Your organizational structure, the metrics you use to evaluate success, and your ability toprioritize your teams efforts, will affect how much testing positively impacts your business.

    The following slides highlight what has worked for a number of e-commerce clients.

    35Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    36/45

    These are the most common testing metrics for e-commerce sites and the pages likely to

    impact those objectives. These should be measured with every test.

    Description Key ages

    Conversion Rate Percentage of site (campaign) visitors who make a

    purchase Checkout Funnel Product Pages

    Category Pages Homepage

    RPV Revenue Per Visitor (RPV) - average revenue fromeach visitor to the site Product pages Homepage

    Checkout FunnelSteps

    Each step (checkout, shipping address, billing address,payment, etc.) within the checkout funnel.

    Checkout Funnel

    Add to Cart Rate Percentage of site (campaign) visitors who add a

    product to their carts Homepage Product Pages Category Pages

    Product Page

    Views

    Number of product page views - measured either asan average number per session or a percentage ofusers who view at least one product page

    Homepage Product Pages Category Pages

    Category PageViews

    Number of category page views - measured either asan average number per session or a percentage ofusers who view at least on category page

    Homepage Product Pages Category Pages

    36Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    37/45

    Ultimately, every page with reasonably high traffic should be tested. Here are some things to

    consider.

    Benefits Things to Consider

    Homepage

    Entry point to your site, sets the tone for your brandsvalue and sites offering

    High traffic volume

    Far from purchase event - traffic not yetqualified

    Consider metrics beyond purchases to judgesuccess (product page views, add to carts etc.)

    Category/

    SubcategoryPages

    Page has ability to turn unqualified traffic (browsers)into qualified traffic (buyers) Ability to set a cookie for users broad interests for

    future targeting

    Somewhat far from purchase event Consider testing manual changes on one categorypage before running on all

    Product Page

    Sweet spot of high traffic volume and qualified traffic Not all changes will work across all products, beaware of varying page templates used

    Shopping Cart Tipping point - the r ight offer (free shipping, discountsetc.) can make the difference Last (good) opportunity to up-sell - experiment tosee what works with your site

    Checkout Funnel

    Users are highly qualified Even small conversion lift directly correlates with

    revenue

    Low traffic volume means tests may need to runlonger

    Focus on usability and few distractions

    37Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    38/45

    38Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    39/45

    Every business is different and so is every team. Your team may have one person who performs

    a variety of functions or many people who each perform a very specific function. Whatever thecase, the following represent the key rolesand competenciesin any successful testing program.

    38Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    40/45

    Every business is different and so is every team. Your team may have one person who performs

    a variety of functions or many people who each perform a very specific function. Whatever thecase, the following represent the key rolesand competenciesin any successful testing program.

    Testing Program ManagerThis person is the primary person whomanages testing effor ts. This per sonprioritizes test order, coordinates resources,understands how to interpret test results,and ensures the Executive Sponsor receivesregular and timely updates on test results.

    38Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    41/45

    Every business is different and so is every team. Your team may have one person who performs

    a variety of functions or many people who each perform a very specific function. Whatever thecase, the following represent the key rolesand competenciesin any successful testing program.

    Testing Program ManagerThis person is the primary person whomanages testing effor ts. This per sonprioritizes test order, coordinates resources,understands how to interpret test results,and ensures the Executive Sponsor receivesregular and timely updates on test results.

    Executive SponsorThis person is usually a VP or ExecutiveDirector. This person is not typically involvedin day-to-day testing decisions but believestesting and data-driven decision-makingshould affect business decisions and iscurious to see test results on a regular basis.

    38Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    42/45

    Every business is different and so is every team. Your team may have one person who performs

    a variety of functions or many people who each perform a very specific function. Whatever thecase, the following represent the key rolesand competenciesin any successful testing program.

    Testing Program ManagerThis person is the primary person whomanages testing effor ts. This per sonprioritizes test order, coordinates resources,understands how to interpret test results,and ensures the Executive Sponsor receivesregular and timely updates on test results.

    DesignerThis person is usually someone with

    usability and/or graphic design experience.His or her ability to create subtle tosignificant stylistic changes can help create avariation that works. Alternate buttonimage, navigation style, or a new icon orimage all can have a significant impact onhow users use your site.

    Executive SponsorThis person is usually a VP or ExecutiveDirector. This person is not typically involvedin day-to-day testing decisions but believestesting and data-driven decision-makingshould affect business decisions and iscurious to see test results on a regular basis.

    38Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    43/45

    Every business is different and so is every team. Your team may have one person who performs

    a variety of functions or many people who each perform a very specific function. Whatever thecase, the following represent the key rolesand competenciesin any successful testing program.

    DeveloperThis person is extremely comfortable with

    HTML and CSS and also has a strongunderstanding of javascript. He or she canefficiently manipulate code to implementmore complex experiments when using themore advanced functionality of theOptimizely platform.

    Testing Program ManagerThis person is the primary person whomanages testing effor ts. This per sonprioritizes test order, coordinates resources,

    understands how to interpret test results,and ensures the Executive Sponsor receivesregular and timely updates on test results.

    DesignerThis person is usually someone with

    usability and/or graphic design experience.His or her ability to create subtle tosignificant stylistic changes can help create avariation that works. Alternate buttonimage, navigation style, or a new icon orimage all can have a significant impact onhow users use your site.

    Executive SponsorThis person is usually a VP or ExecutiveDirector. This person is not typically involvedin day-to-day testing decisions but believes

    testing and data-driven decision-makingshould affect business decisions and iscurious to see test results on a regular basis.

    38Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    44/45

    Small, Big, Small (Again)Start with small iterative changes and concepts. Many of the changes highlighted in this deckare small changes and may represent quick wins. But dont be afraid to Go Big with alarger redesign and then Go Small Again to iteratively improve upon that.

    TargetingTesting can involve more than just visual changes. Test displaying targeted content or layouts.

    Targeting can be based on geo-location, cookies (based on the users account type, pagesvisited etc.), referrer, browser etc.

    Test ConceptsDont worry if the exact version youre testing is exactly how you would implement it if youwere doing it permanently. Get your tested version close enough to validate the concept.

    Then, test again. For example, if you want to test if larger images have an impact, start byhardcoding larger images on a few key pages. If that has an impact, you now know its worthyour design or development teams time to create the process of automatically pulling in thelarger images across all pages.

    39Friday, October 11, 13

  • 8/12/2019 Testing Ideas for Ecommerce Websites

    45/45

    Thank you for making Optimizely the worlds number one A/B testing platform!