PUBLIC RELATIONS PRACTICE

2014 Week 6

!

DR KANE HOPKINS

EVALUATION

Evaluation definedThe practice of evaluation involves the systematic collection of information about the activities, characteristics, and outcomes of programs, personnel, and products for use by specific people to reduce uncertainties, improve effectiveness, and make decisions with regard to what those programs, personnel, or products are doing and affecting !

Patton (1982, p. 15)

1. Evaluation is research 2. Evaluation looks both ways 3. Evaluation is user- and situation-dependent 4. Evaluation is short term 5. Evaluation is long term 6. Evaluation is comparative 7. Evaluation is multi-faceted

Principles of evaluation

1. Evaluation is research 2. Evaluation looks both ways 3. Evaluation is user- and situation-dependent 4. Evaluation is short term 5. Evaluation is long term 6. Evaluation is comparative 7. Evaluation is multi-faceted

Poindexter says:!if you want more info the title is a link to the article

Principles of evaluation

Evaluation: the reality • Practitioners viewed evaluation very narrowly and

lack confidence in promoting evaluation methods to employers and clients

• Most relied on output measurement of media coverage. Few undertook research or pre-testing when preparing campaigns

• Evaluation was not undertaken because of a lack of time, budget and knowledge of methods

• Evaluation was feared because it could challenge the logic of practitioners’ advice and activities

Evaluation: the reality • 75% claimed to undertake some form of evaluation,

with 62% of programmes being evaluated, mainly by the project manager

• Little was spent on evaluation, with 74% indicating that the total proportion of budget was zero to 5%

• The picture that emerged was of the practitioner as a ‘doer’ rather than an advisor

The Barcelona Declaration of Research Principles• A declaration of standards and practices to guide

measurement and evaluation of PR was first adopted at the 2nd European Summit on Measurement in Barcelona, Spain, in June 2010

• Organised by the International Association for Measurement and Evaluation of Communication and the Institute for Public Relations

• The Barcelona Principles will be the subject of ongoing dialogue as industry professionals seek to apply the standards to the public relations practice

Seven measurement principles 1. Goal setting and measurement are fundamental aspects of any

public relations programme 2. Measuring the effect on outcomes is preferred to measuring outputs 3. The effect of public relations programs on business results can and

should be measured where possible 4. Media measurement requires quantity and quality. Overall clip

counts and general impressions are meaningless 5. Advertising Value Equivalents (AVEs) do not portray the value of

public relations and do not inform future activity 6. Social media came in should be measured, although there is no

single ‘metric’ 7. Transparency and replicability are paramount to sound

measurement

Goal setting & measurement• Goal-setting and measurement are fundamental

aspects of any public relations program • Goals should be as quantitative as possible and

address who, what, when and how much the PR program is intended to affect

• Measurement should take a holistic approach, including representative traditional and social media; changes in awareness among key stakeholders, comprehension, attitude, and behaviour as applicable; and effect on business results

Measuring the effect on outcomes• Outcomes include:

– shifts in awareness – comprehension – attitude and behaviour related to purchase – donations – brand equity – corporate reputation – employee engagement – investment decisions, – other shifts in stakeholders regarding a company, NGO,

government or entity, as well as the stakeholder’s own beliefs and behaviours

Measuring the effect on outcomes• Practices for measuring the effect on outcomes should

be tailored to the business objectives of the PR activities. Quantitative measures such as benchmark and tracking surveys, are often preferable. However, qualitative methods can be well suited or used to supplement quantitative measures

• Standard best practices in survey research including sample design, question wording and order, and statistical analysis should be applied in total transparency

Measuring business results• To measure business results from consumer or

brand marketing, models that determine the effects of the quantity and quality of PR outputs on sales or other business metrics, while accounting for other variables, are a preferred choice

Measuring quantity & quality• Overall clip counts and general impressions are usually

meaningless. Instead, media measurement, whether in traditional or online channels, should account for:

– Impressions among the stakeholder or audience – Quality of the media coverage including:

• Tone • Credibility and Relevance of the Medium to the Stakeholder or

Audience • Message Delivery • Inclusion of a 3rd party or company spokesperson • Prominence as Relevant to the Medium

– Quality can be negative, positive, or neutral

AVEs are not the value of PR• AVEs measure the cost of media space and are rejected as a

concept to value public relations • Where a comparison has to be made between the cost of

space from earned versus paid media, validated metrics should be used, stated for what they are, and reflect:

– negotiated advertising rates relevant to the client, as available – quality of the coverage, including negative results – physical space of the coverage, and the portion of the

coverage that is relevant • Multipliers intended to reflect a greater media cost for earned

versus paid media should never be applied unless proven to exist in the specific case

Measuring social media• Social media measurement is a discipline, not a tool; but there

is no “single metric” • Organisations need clearly defined goals and outcomes for

social media • Media content analysis should be supplemented by web and

search analytics, sales and CRM data, survey data and other methods

• Evaluating quality and quantity is critical, just as it is with conventional media

• Measurement must focus on “conversation” and “communities” not just “coverage”

• Understanding reach and influence is important, but existing sources are not accessible, transparent or consistent enough to be reliable; experimentation and testing are key to success

Transparency & replicability• PR measurement should be done in a manner that is

transparent and replicable for all steps in the process, including specifying:

– Media Measurement: • Source of the content (print, broadcast, internet, consumer

generated media) along with criteria used for collection • Analysis methodology – for example, whether human or automated,

tone scale, reach to target, content analysis parameters – Surveys:

• Methodology – sampling frame and size, margin of error, probability or non-probability

• Questions – all should be released as asked (wording and order) • Statistical methodology – how specific metrics are calculated

Use of output evaluation methods in campaigns

 Evalua'on  category  Evalua'on  methods %  use

 Output Distribu'on  sta's'cs 28.81Media  monitoring 64.41

Media  content  analysis 45.53

Audience  analysis 4.24

Sta's'cal  analysis 6.78

Response  rates 66.10

Coding  material 8.47

AItude  and  image  studies 4.24

Communica'on  audit 1.69

Organisa'onal  culture  study 0.85

Analysis  of  complaints 2.54

Use of output evaluation methods in campaigns  Evalua'on  category  Evalua'on  methods %  use

 Output Distribu'on  sta's'cs 28.81Media  monitoring 64.41

Media  content  analysis 45.53

Audience  analysis 4.24

Sta's'cal  analysis 6.78

Response  rates 66.10

Coding  material 8.47

AItude  and  image  studies 4.24

Communica'on  audit 1.69

Organisa'onal  culture  study 0.85

Analysis  of  complaints 2.54

Use of output evaluation methods in campaigns  Evalua'on  category  Evalua'on  methods %  use

 Output Distribu'on  sta's'cs 28.81Media  monitoring 64.41

Media  content  analysis 45.53

Audience  analysis 4.24

Sta's'cal  analysis 6.78

Response  rates 66.10

Coding  material 8.47

AItude  and  image  studies 4.24

Communica'on  audit 1.69

Organisa'onal  culture  study 0.85

Analysis  of  complaints 2.54

Poindexter says:!That’s a link to more info on the methods

Use of output & outcome evaluation methods in campaigns

Evalua'on  category  Evalua'on  methods %  use

Outcome Focus  group  discussions 4.24

In-­‐depth  interviews  surveys 4.24

Pre  and  post  tests 3.39

Unobtrusive  data  collec'on 17.80

Quasi-­‐experimental  study 0.00

Ac'vity  outcome 29.66

Use of output & outcome evaluation methods in campaignsEvalua'on  category  Evalua'on  methods %  use

Outcome Focus  group  discussions 4.24

In-­‐depth  interviews  surveys 4.24

Pre  and  post  tests 3.39

Unobtrusive  data  collec'on 17.80

Quasi-­‐experimental  study 0.00

Ac'vity  outcome 29.66

Evaluation as a new beginning• Measuring the effectiveness of a public relations effort

frequently provides new direction and emphasis for an on-going programme.

• Even if a campaign is cancelled, the lessons learned concerning its effectiveness will be useful in future activities.

• Knowledge gained through careful evaluation is an important payoff to prevent future mistakes.

• Careful measurement of successful efforts will help reproduce positive elements in future programmes.

Case study: Driving Dogs

Case study: Driving Dogs• Issue

– In New Zealand, the number of dogs in SPCA shelters far outweighs public interest in their adoption

• Research – People think because a dog is in a shelter, it must be

damaged goods – A media coverage audit revealed that exposure of

bad dog behaviour in the news was up 44% – It also suggested that news media would be key in

changing perceptions

Case study: Driving Dogs• Objectives

1. Change people’s negative perceptions of SPCA shelter dog behaviour, in order to drive interest in SPCA adoptions

2. Increase awareness of MINI’s association with SPCA by 10%

3. Double engagement with the MINI brand on Facebook: from 5% to 10%

Case study: Driving Dogs• Audience: Potential dog adoptees nationwide

– People who had ever considered, or could consider adopting a dog or encouraging someone they know to do so

• Messages – “Dogs this smart deserve a home”

• The SPCA in association with MINI is proving that you can teach a rescue dog new tricks

• SPCA wants to showcase just how gifted, intelligent and talented SPCA animals really are

• MINI is a long term supporter of the SPCA and wanted to do something significant to drive adoptions

Case study: Driving Dogs• Strategy

– To deliver an extreme display of SPCA’s dogs’ intelligence and trainability to PROVE they didn’t have behavioural issues

– Prove that SPCA dogs are smart by teaching them to drive a mini – They used news and social media influencers to take the nation

on a journey from disbelief to belief in three steps 1. Create disbelief – use a credible platform to engage NZ with the idea

that dogs could drive MINIs 2. Engage the nation – drive engagement with this idea and encourage

people to get behind the dogs through news, social and paid media 3. Prove it to the nation – create one mass, undeniable, media moment to

prove that the idea is actually possible – Dogs this smart deserve a home

Case study: Driving Dogs• Implementation

– New Zealand’s #1 animal trainer and selected three SPCA shelter dogs to undergo a world first eight week “doggy driving” training course in a modified MINI

• Create disbelief: They convinced Campbell Live to be our editorial partner on the project. They would be given the “reveal” exclusive and therefore be given access to the dogs during the training process to capture the back story

• Engage the nation: Facebook, YouTube channel, A MINI USB stick on a SPCA branded dog collar was sent to15 New Zealand media “super influencers”

• Prove it to the nation: staged a demonstration of SPCA dogs driving a MINI live on Campbell Live – ensured those who had already engaged with us via

social media or had run the teaser story received a copy of the video so as many people as possible could see it with their own eyes and further share the content

– This included online news sites, journalists, Twitter followers and Facebook fans.

– Also shared it through our YouTube platform

Case study: Driving Dogs

Results• Objective: Change people’s negative perceptions of

shelter dog behaviour, in order to drive interest in SPCA adoptions. – Perceptions were changed

• People were twice as likely to overcome perception barriers once they’d seen the dogs driving

Evaluation: Survey• To test the efficacy of the key messages and to

document whether or not they had achieved our objectives: – Driving Dogs drove a 40% increase in awareness of

MINI’s association with SPCA – There was a 113% increase in the number of people

with a positive perception of shelter dog behaviour – Once people had seen the campaign, they were twice

as likely to have overcome their perception barrier

Evaluation: Media Analysis• Campbell Live ran the story not twice as originally

planned but four times to their audience of 304,000 people nationwide

• A total of 483 media articles • 213 minutes of broadcast coverage ran altogether on the

story • Broadcast coverage was syndicated to 1008 media

outlets globally • Reach via media articles to an estimated audience of

230,540,469 worldwide

CONTROLLED COMMUNICATION

What is that exactly?

Controlled communication is any form of media that allows the communicator to control the content, timing, frequency, format and placement of the communication

Controlled media can be... • Videos • Posters • Websites • Personally addressed letters • Brochures + booklets • Advertising • Fact sheets • Exhibitions and displays • Newsletters – external + internal audiences

Posters• Fast and effective communication • The difference is size and cost • Advances in computers make them easy and cheap

to produce

Brochures and bookletsPurpose: • Arouse interest • Answer questions • Direct to further information • Sit under Public Information or Publicity models of PR • Usually function as support pieces in a larger campaign

• Display rack • Press kit • Direct mail

Video• A good way to repurpose content, such as media

releases, events etc • Creating a video news release (VNR) • Political campaigns • Product launches • Building trust and credibility

Video• A good way to repurpose content, such as media

releases, events etc • Creating a video news release (VNR) • Political campaigns • Product launches • Building trust and credibility

The Galifianakis bump

Fact Sheet– A quick reference tool for reporters

– Summarises the key points about and event, product, issue or organisation

– Don’t include too much information – Supports other material

Email• Low cost • Immediacy • Ease of distribution • Target messages • Easily allow for feedback

Exhibitions and Displays• Rely primarily on visual messages.

• Use many of the tactics talked about here

• Valuable for many types of industries • Tourism • Electronic • Motor vehicles

TO READ THE READINGS THIS WEEK. I MEAN, WOULD IT KILL YOU?