Embed Size (px)
Citation preview
You never know the potential of your effort until you test.
Testing best practices
and examples
Carol Worthington-Levy
2
TESTING
Combining art, copy and the science of testing to create measurable
success.
4!
Direct marketing is different than other advertising
What’s the difference? General Advertising:
makes a point, supports your brand— This includes MarCom, ads, outdoor boards…
Direct Marketing Advertising: Generates action, leading directly to a sale or a relationship which becomes measurable in Return on Investment (ROI)
5!
Which is better?
Both are important in a healthy mix of marketing.
If you are short on budget, direct marketing is the way to go since it’s accountable and very targeted.
Creative work (design and copy) is different for each, but they can be integrated.
6!
Direct response is a science Designers and writers who do DM should follow special rules that have been gathered through research and testing:
Some typefaces won’t be read as easily as others
Some photos will turn off a prospect, rather than get them interested
Some colors will upset the reader
Black backgrounds with white type are rarely read at all!
Sometimes ‘mystery’ works (blank outgoing envelopes) and sometimes teasing works better – this must be tested for YOUR audience
What we like personally is often not what our prospects like!
7!
The key ingredient for all successful advertising…
USP - Unique Selling Proposition.
This is something that you have that nobody else offers
Something that you do better than anyone else
The uniqueness that is a good lever for you to attract customers and sell your product or service
Who wants to volunteer to state what OTA’s unique selling proposition is?
8!
What makes up a direct mail effort? A direct mail package typically has: An outgoing envelope (OE) A letter A reply form And often has: A “buckslip” for special messaging that does not fit in the letter A reply envelope for times that mail reply may be possible A self-mailer: Can be a postcard although the message must be simple and straightforward Is more likely a one-piece mailer that has different ways to tell the “story”, Must be easy to for the reader to “scan” and find the information they Must be on heavy enough paper to mail without getting destroyed since no envelope protects it Which works better? Testing will tell us.
9!
What’s a dm “Control” package? The direct marketing effort that beats all others in a test so that it’s got… • Best response in ‘front end’ (response to the effort) or • Best response in the ‘back end’ (best qualified leads for a lead-generation effort) or • Largest order sizes … etc. To be successful in ongoing mail efforts, we must determine what should be repeated in future efforts. This is true for direct mail, ads, email, broadcast and more
10!
Can I mail a great “control” forever? No! – Why? If your mailing lists are limited in size, the prospects start recognizing the package as something they have seen before Times change and a direct mail package must reflect those times to be relevant to the audience Some people are more attracted to some kinds of mail than others. For example:
Invitation package feels personal Self-mailer is promotional, easy ‘read’ Oversized OE is promotional but holds a ‘secret’ inside
Some copywriting tells the story in a different way, and the message does a better job of inspiring certain people.
11!
In direct mail, how do prospects learn about your USP?
WE tell them as clearly as we can – in a way that shows them how it BENEFITS them – in the letter, in a buckslip, and often on the OE
Features: what a product or service has as its construction or what defines it
Benefits: What good will come of your prospect’s life as a result of having purchased this product or service
Testing in direct marketing prove that benefits trump features when you are trying to start a relationship
12!
How can we measure what works in a marketing campaign?
1. We create each mail package concept with a purpose: generate either a purchase, or a lead
2. We set up a TEST - to measure how this dm package performs vs. how how another direct mail package did.
3. We measure the way these mailings do, in terms of: percentage of response order size return on investment (ROI), and repeat orders. (lifetime value)
13!
Why test? Learn the best ways - media and methodology
- to talk to your customers The step from theoretical to reality!
Learn what voice, visual signals, format, offer works best short-term and long-term.
For example – teaser vs. no teaser on outgoing envelope? Brochure or not? White paper or ivory paper? Picture on the envelope, or plain? You might be very surprised!
When you get these answers, you can repeat your success instead of searching blindly in the future.
14!
What should we test? Mailing lists
Offers
Copy style or messaging
Concept and Design (including paper and format testing)
Timing of the drop or insertion (one week, one month apart might make a difference)
Contact strategy: do they respond more when getting two, instead of one, in a row? Would a postcard followed by direct mail work better?
All of this can be tested, and measured.
15!
Do the homework, and trust the numbers.
What works with this customer may not be something that is our own personal style – but the numbers don’t lie. Ignoring test results throws away opportunity for success
We analyze what we can keep that worked, and how can we leverage it into a new test
Until you test something, you won’t really know how it will work. Mailing without testing is like Russian roulette with marketing dollars.
16!
What questions we ask as we develop a DM test …
What has worked in the past? What has not? What do we want to accomplish with THIS test? What is our budget, and how do we best use that? What names do we have, what names can we get,
to test this time? How do we plan to measure the results of this test? What do we want to accomplish in the FRONT END, and
the BACK END? What will we do once we get our results (next steps)?
17!
What do we do with that research/information?
1. We set up a test to measure a few different ways to reach our customer… and after the tests are done, we compare all the criteria mentioned.
2. We look over the results and draw conclusions which will help us decide what our next move will be - what we will mail, what we will offer, etc.
18!
What next? 1. We take our “winner” of the test — and we call
it our “control”. 2. We compare the other tests — list testing, offer
testing, creative concept testing… and we determine from our losses and gains what we should test next.
3. We develop another test. But meanwhile, our “Control” gets the lion’s share of the mailing names so that it makes the mailing most profitable.
19!
In a nutshell… Testing is the way we discover what works best to our audience.
Quantity mailed must be enough to get statistical accuracy — (You must get 100 responses in a category or the numbers lose their statistical integrity.)
Testing must be controlled by keeping all elements the same except ONE, so that you know what made response better or worse.
20!
How do we know what wins?
Response is tracked by using codes that are on each piece, or numbered on the ad (or different response phone numbers or URLs) Testing example 1: Test the same direct mailer to five different lists. Testing example 2: Test a direct mail pack versus a postcard. Keep list and offer the same. Testing example 3: Test two postcards, one with offer A and one with offer B
21!
Check out this offer test… Testing an offer to encourage prospects to sign up for the BMW Card (a credit card by BMW Financial) and USE the credit card • This was going to their database of BMW owners • Both of these gifts matches the psychographic profile of the BMW driver/enthusiast: they have ‘model home kitchens’ and they like to travel for pleasure Left offer: $20 gift certificate to Cooking.com Right offer: 2 round trip airfare tickets (with travel restrictions)
Which do you think won?
22!
Envelope tests for a continuity program
Which would you respond to? What do you think won?
23!
Envelope tests for a continuity program
The left package got the most response. But the right package got us the best customers since they stayed with the program and bought a lot more.
Offer testing can yield surprises
We offer-tested a white paper about their topic (Permission email) versus a chance to win a Tablet PC.
The list was all opt-ins from the past year. Pretty qualified.
Which do you think got the most responses?
Which do you think got the best quality responses?
25!
Cover tests for catalog This company loved their bright, clean copy-free covers. But testing had told us that teasers on a catalog cover often made it work better. So we tested… which do you think did better?
26!
Cover tests for catalog The one with teasers beat their longtime control by 123%.
27!
Concept tests for theWine of the Month Club
Which won this 3-way test? Above: – a 10-year control – a Check in the Mail has a check for $10 that shows through window – it covers your first order Right: concept: You never pay for a wine you don’t like. Contains a $10 coupon for first order. Right, top – 2-color OE Right, below – full color OE
28!
Envelope tests for a continuity program
Response: The Check in the mail came in 20% lower than the “You Never Pay…”
Then: The 2-color OE beat the 4-color OE by 2%. And the 2-color OE had a 50% higher lifetime value than Check in the Mail at the start…
But then… the 2-color one (in the back-end) doubled the lifetime value of the Check in the Mail in the back-end
29!
Test, test, and test again!! Online Trading Academy creative tests in the past 12 months included:
The 2006 control vs. our ivory invitation package Our ivory invitation (the new control) package vs. the same but with one copy change at
the start of the letter (the copy change bombed) Our ivory invitation package (still the control) vs. a self-mailer
(self mailer did not do badly, just not as well as control — so we can use it in our “mix” for future mailing to keep lists fresh)
6 dates vs. 9 dates on the RSVP (6 dates won) Our ivory invitation package vs. letter-shaped ivory package (the letter-shaped package
bombed) Our ivory (still a control) invitation package vs. a white paper version (white paper won) Our ivory invitation package with the “essential trading offer” vs. the “risk management”
offer (highlighted by teaser on OE) – (new offer won) An entirely new copy test with a different ‘voice’ (did not win) Our new control at this point became the white paper version with “risk management” offer. Our new control vs. the same but with something lumpy inside (still gathering results to
decide winner) Our white paper/risk management control VS an Oversized OE with “can you…”
questions or True/false (still gathering results to decide winner) Getting ready to test: Special down-market copy test, check in the mail test, Magalog,
entirely new design and copy approach, additional offer tests, and more!
30!
Only one thing can change for a test to measure accurately
Which is the better test… What would be learned? Which would not be a good test? Why?
1. Test a letter/traditional direct mail package with an outgoing
envelope, a letter, and a reply form with a free music CD offer Versus - a self-mailer with a free music CD offer.
2. Test a letter-shaped direct mail package with an outgoing envelope, a letter, and a reply form with a free music CD offer Versus - a self-mailer with no offer.
3. Test a letter-shaped direct mail package with an outgoing
envelope, a letter, a reply form and a buckslip with a free music CD offer – Versus - a square-shaped direct mail package with an outgoing envelope, a letter, a reply form, a buckslip and a lift-note, with a free music CD offer.
31!
Only one thing can change for a test to measure accurately Answer: 1 is the only one that would be a true and accurate test. Because no. 2 has two different formats AND two different
offers… and no. 3 has two different formats AND an additional piece in the envelope.
1. Test a letter/traditional direct mail package with an outgoing
envelope, a letter, and a reply form with a free music CD offer Versus - a self-mailer with a free music CD offer.
2. Test a letter-shaped direct mail package with an outgoing envelope, a letter, and a reply form with a free music CD offer Versus - a self-mailer with no offer.
3. Test a letter-shaped direct mail package with an outgoing
envelope, a letter, a reply form and a buckslip with a free music CD offer – Versus - a square-shaped direct mail package with an outgoing envelope, a letter, a reply form, a buckslip and a lift-note, with a free music CD offer.
32!
NOW, try putting a test together, yourself…
Remember that there must be just ONE thing changing to make the test valid. Otherwise, you don’t know what made the test win or lose.
What do you want to learn as a result of the test? What media are you using to run the test – mail, email, web, etc?
INTERESTED IN LEARNING MORE ABOUT OFFER TESTING?
Call me — and let’s talk about your upcoming
campaigns, and what kinds of tests will be useful.
Thank you!
Carol Worthington-Levy
408.269.6981 [email protected]
