Wood for trees 1wood for trees 1 Testing Jon Kelly, Director of Analysis @ Wood for Trees March 2015 Outstanding M arketing S ervices

wood for trees 1wood for treeswww.woodfortrees.net

1

Testing

Jon Kelly, Director of Analysis @ Wood for Trees

March 2015

Outstanding Marketing Services Provider 2011 (Database Marketing awards); Supplier of the Year 2012 (IoF SIG Awards); Innovation in Analytics for Marketing 2013 (Database Marketing Awards)


2

Some thoughts on testing

Determine the reason for the test – what are you expecting to find out

Beware of averages

Beware of balancing effects

Test as wide as possible

Explore behaviour changes within sub populations of the test audience

Action the test

Re-test

Understand the longer term effects


3

Prompt Testing – Top Level Results

Top level views and analysis previously done shows

that the testing had little effect.

However looking at certain segments in more detail

tells a different story..

Audience Prompts Response Rate Average Gift Income per Thousand

Control £5, £18 ,£25, £50, £150 10.30% £11.48 £1,183Test £10 10.43% £11.46 £1,195


4

Response Pattern

Overall response pattern in line with other campaigns

However, different response picture where testing took place


5

May 2012 By Last Amount

This behaviour is the same across all giving levels, so

£5 donors give £10 – GOOD

£25 donors give £10 – BAD

As long as responder volumes remain consistent……

Previous Gift =

£5

Previous Gift =

£25


6

May 2012 by Last Amount

Low value responders were prompted to give more but less

responded

High value responders were prompted to respond more but to give

less

Previous Gift = £5Prompts Response Rate Average Gift Income per

Thousand£5, £18 ,£25, £50, £150 13.04% £6.59 £1,210

£10 12.26% £7.34 £1,093

Previous Gift = £25Prompts Response Rate Average Gift Income per

Thousand£5, £18 ,£25, £50, £150 10.60% £23.26 £2,475

£10 11.34% £19.59 £2,222


7

TEST - £5 and under

Test 1: £5, £8, £10, £15, Other

Test 2: £8, £10, £15, £20, Other

Test 3: £10, £15, £20, £25, Other


8

Cold Mailing - £5 and under

HR £0 - £5.01 Mailed Responded Response Rate Index Sig Average Gift Index Sig Income per 000 Mailed IndexControl 2,500 205 8.2% £5.76 £472.20Test Total 7,500 548 7.3% 89 84.7% £6.89 120 100.0% £503.67 107Test 1: £5, £8, £10, £15, Other 2,500 203 8.1% 99 8.2% £6.47 112 95.6% £525.60 111Test 2: £8, £10, £15, £20, Other 2,500 161 6.4% 79 98.3% £6.92 120 100.0% £445.80 94Test 3: £10, £15, £20, £25, Other 2,500 184 7.4% 90 73.3% £7.33 127 100.0% £539.60 114TOTAL 10,000 753 7.5% £6.58 £495.80

All tests show a fall in response rate which is perhaps unexpected, although only the £8 test is significant.Average gift is up around £1 on all tests as supporters are encouraged to give at higher amounts.Around £30 extra per 000 across all tests.So tests are likely to increase income but there might be a small hit in responses.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

2014 - April Warm Cash Mailing - HR £0 - £5.01 Prompt Control

2014 - April Warm Cash Mailing - HR £0 - £5.01 Prompt Test 1



b:<£5

c:£5

d:£5-£10

e:£10

f:£10-£15

g:£15

h:£15+


9

Max Warm Donation in last 5 years

<=£10

Responded to a previous campaign

Max Response

Amount Ever > £12

Max Response

Amount Ever £6 - £12

Max Response

Amount Ever < £6

Not responded to

a previous campaign

Mailed once or less

(Warm) in last year

First payment in

last 15 months

First Payment

before last 15 months

Mailed more than once in

last 12 months

First relationship within last 2

years

Given via self sponsorship

Not given via self

sponsorship

First relationship more than 2 years ago

GROUP A: £12 ASK – Max Warm Donation in last 5 years <=£10


<=£10


Max Response

Amount Ever > £12

Max Response


Max Response

Amount Ever < £6

Not responded to

a previous campaign

Mailed once or less

(Warm) in last year

First payment in

last 15 months

First Payment

before last 15 months

Mailed more than once in

last 12 months

First relationship within last 2

years

Given via self sponsorship

Not given via self

sponsorship

First relationship more than 2 years ago


10


<=£10

Responded to a

previous campaign

Max Response Amount

Ever > £12

Max Response Amount

Ever £6 - £12

Max Response Amount

Ever < £6

Not responded

to a previous campaign

Group A1


11

Ask: £5 £12 £15

No. mailed: 1,431 4,310 1,418

Response rate:

8.4% 7.3% 7.8%

Average gift:

£12.20 £15.16 £20.05

ROI: 2.56 2.76 3.89

Group A1

Group A1: Max donation in last 5 years less than £10 –

but previously responded at more than £12

Winner = £15

Ask: £5 £12 £15

No. mailed: 1,431 4,310 1,418

Response rate:

8.4% 7.3% 7.8%

Average gift:

£12.20 £15.16 £20.05

ROI: 2.56 2.76 3.89


12

Max Warm Donation in last 5 years <=£10


Max Response

Amount Ever > £12

Max Response


Max Response Amount

Ever < £6

Not responded to a previous campaign

Group A3


13

Ask: £5 £12 £15

No. mailed: 6,016 17,814 5,966

Response rate:

16.3% 13.1% 12.2%

Average gift:

£4.43 £4.57 £4.48

ROI: 1.80 1.49 1.36

Group A3

Group A3: Max donation ever is less than £6

Winner = £5

Ask: £5 £12 £15

No. mailed: 6,016 17,814 5,966

Response rate:

16.3% 13.1% 12.2%

Average gift:

£4.43 £4.57 £4.48

ROI: 1.80 1.49 1.36


14

Most effective ask levels - summary

Less than before Same as before More than before

How the asks changed

75%

15%

10%

Documents

Wood for trees 1wood for trees 1 Testing Jon Kelly, Director of Analysis @ Wood for Trees March 2015 Outstanding M arketing S ervices