Upload
eric-schwartzman
View
1.655
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Barcelona declaration of measurement principles final with results voting. 20.06.10
Citation preview
Barry LeggetterExecutive Director AMEC
Summit Director
Creating the Barcelona Declaration of Measurement Principles
Moderator
David B. RocklandPartner/CEO
Ketchum Pleon
Barcelona Declarationof Measurement Principles
Global AllianceICCO
Institute for Public RelationsPublic Relations Society of America
AMEC U.S. & Agency Leaders Chapter
Why Are We Here? For the Common Good of Our Industry.
• The communications landscape is changing rapidly
• A lack of clear standards and approaches to PR measurement results in the profession not always being taken seriously; it doesn’t count unless you can count it
• AVEs, random use of multipliers, and other silly metrics and practices diminish the integrity of the profession and the market size
•As a profession, we spend more time arguing amongst ourselves rather than building the field
•We are at a moment in time to make difference, and alter the path we are on, taking advantage of the opportunity to “own” the new communications environment
AMEC International Business Monitor 2010 6
New AMEC research shows strong demand for AVEs
Q8 Current trends in the measurement and evaluation market (base: 34)
47% 32%There is continuing demand for AVEs as a
significant metric for measuring PR programme effectiveness
Agree slightly Agree strongly
Chart does not include % saying “neither agree nor disagree”
Proportion disagreeing
6%
AMEC International Business Monitor 2010 7
Increased Interest in International Measurement and Social Media; ad hoc programmes on the decline
Q8 Current trends in the measurement and evaluation market (base: 34)
Chart does not include % saying “neither agree nor disagree”
Proportion disagreeing
6%
47%
3%
Towards an evaluation standard for marketing PR – market mix modelling
Speaker
Richard HoughtonPresident
ICCO on behalf of New York University
Lou CapozziImmediate Past President, ICCO and Professor, New York
University
• Team of graduate students from NYU’s Masters Degree program in public relations and corporate communications
• Supervised by Prof. Capozzi
• Gained a grant from NYU and support form the International Communications Consultants Organisation
• Broad review of the literature
• Goal – seek commonalities
NYU Student Audit
• Large PR firm models
• Country models• UK, Canada and Germany
• PR measurement vendor models
• Advertising models
• Market mix modelling vendors
NYU Student Audit -Areas Covered
• Real need and appetite for a standardise approach
• Enormous breadth and diversity of existing approaches
• Variation is too great to be rationalised into a single model
• All models have a common flaw
NYU Student Audit –Conclusions
• Broadly adopted by large-scale marketers
• Readily available for PR to connect to
• Enables comparisons between all the elements in the marketing mix
• Fits with many existing PR evaluation techniques
Why market mix modelling?
• Focused tightly on marketing PR, especially for large-scale programs
• Vendor neutral
• Strikes an effective balance between cost and credibility
• Presented in terms familiar to marketers
• Practical with low barriers to entry
Model requirements
• Total PR program costs (negotiated rates)
• Markets covered
• Circulation/audience reach (no multipliers)
• Impressions
• Tone/Quality score
• Impact score
Potential Data Elements
• Up front commitment required
• Historical data needed for MMMs
• Weekly tracking generally required
• Larger PR investments required to show impact
• Budgets need to be tied to PR program elements
Challenges
• Meet with thought leaders• AMEC US and agency research leaders• Corporate pioneers
• Meet with MMM vendors, in-house operations
• Propose a standard
• Get AMEC support
• Seek support from major PR associations
• Promote
Next Steps
John PaluszekChair
Global Alliance For Public Relations and Communication Management
• Mission: To raise standards and promote public relations globally
• Mantra: "We can -- and we must -- all learn from each other"
• Members: National PR associations in 70 countries and leading international PR associations:
• Serving 175,000 practitioners and educators.
• Wide range of services: "Landscapes" of PR in 22 countries . Global PR education study
The Global Alliance For Public Relations and Communication Management
Signature Event
The Stockholm Accords
" Stockholm Accords" Measurement Opportunities
• Effectiveness of public relations communications/messages
• Credibility of source
• Credibility of content
• Familiarity of content
• Long term effectiveness of "Stockholm Accords"
• Integration of public relations into university management curriculum
• Appointments of public relations professionals in C-suites
• Improved respect for the profession in business and general media
The Consultancy ViewProgramme Measurement Principles
Richard Houghton
International Communications Consultancy Organisation
• Umbrella organisation for 1500 PR agencies in 28 countries
• Our members are PR trade associations
• Remit:• Standards – CMS in 15 countries• Forums – senior management• Insight – World Report
• Huge variety in the maturity of individual markets
•
ICCO Members’ Views
The Consultancy View
Principles need to be…
• Based on straight forward and easily understood methodology
• Needs to be understood and used by all account handlers at all levels
• Applicable across PR disciplines
• Structured to allow for growing digital and social media communications
The Consultancy View
We like to think evaluation differentiates• Many agencies feel that their proprietary
evaluation processes give them a competitive edge
• We may, of course ,be kidding ourselves!
• If clients adopt the principles, consultants will too
The ugly reality
AVEs work for the finance guy (mostly)• No they don’t give provide insight into the
effectiveness of a programme or campaign
• No you can’t really compare advertising with PR
• No they don’t take in account tonality or stance
• No they are no use at all for the vast majority of online activity
• But….• …if finance guys want them consultancies
are likely to provide them
Creating the Declaration of Measurement Principles
• Institute for Public Relations Commission on Public Relations Measurement and EvaluationPauline Draper – Chair, IPR Commission
The Commission
Who are we and what do we do?
The Commission is unique in that it comprises some of the leading thinkers from• academia• corporate/government/not-for-profit• agency• research providerOur purpose• to establish best practice for PR research and measurement• to issue authoritative White Papers• to promote excellence in PR research and measurement through the Jack Felton Golden Ruler Awards and partnering with others
Example White Papers
Available from InstituteforPR.org
•A primer on measuring corporate reputation: a case study on the Dow Chemical Company
•Doing Measurement Right: One Organization’s Experience Creating a Best-In-Class Measurement Program from Scratch
•Isolating the Effects of Media-based Public Relations on Sales: Public Relations Optimization Through Marketing Mix Modeling
•Using Web Analytics to Measure the Impact of Earned Online Media on Business Outcomes: A Methodological Approach
•A New Paradigm for Media Analysis: Weighted Media Cost
•Guidelines for Setting Measurable Public Relations Objectives: An Update
Definition: The calculation of space or time used for earned media (publicity or news content) by comparing it to the cost of that same space or time if purchased for advertising. AVE is used because it assigns a monetary value to public relations using a metric that is simple to understand, simple to calculate, and uses data that is readily available.
•AVE suggests that the space/time occupied by earned media is equivalent to the same space/time purchased•AVE is not a proxy for measuring public relations’ ROI
Advertising Value Equivalents
Why not?
Outcomes rather than Outputs or Outtakes
•Measurement and evaluation practices should demonstrate the effectiveness of public relations to help meet organizational goals.•Measuring the quality of public relations output is necessary in order to demonstrate its contribution to reaching these goals.•Any measure of media coverage should include variables such as tone, prominence, placement, appearance of key messages, the portion of story that applies to the organization or its key messages, and the credibility and targeted reach of the medium in which the message appears.•Whenever possible, it is best to link media coverage with expected outcomes. These outcomes should also be tied to organizational goals.
Advertising Value EquivalentsMeasuring the Value of this Marketing
Communication
Measuring OUTCOMES
• Benchmarking:– To past performance– To closest rival– To competitors
• Map outputs to other metrics:– Share price– Sales– Web analytics data
• Specific research:– Pre/post survey– Message testing– Journalist audit
• Utilize other data:– Surveys– Focus groups– Marketing mixed modeling /
econometric modeling– Client and Employee
satisfaction studies
Practices for taking measurement further
Barcelona Declarationof Measurement Principles
Public Relations Society of AmericaGary McCormick
Chairman and CEO
As a first step, ensuring that four basic questions are asked regarding the work that is to be undertaken:
•Whom are you seeking to affect?•What about them are you seeking to affect?•How much must they be affected to be successful?•By when does this effect need to occur?
The Business Case for Public Relations
A framework for building measurement into public relations programs – a
lexicon and recommended approaches.
Financial•Generates Revenue, Sales, Profit; Marketing public relations drives sales.•Investor public relations drives investment, valuation.•Public relations drives donations and membership for relevant organizations.Reputation / Brand Equity•Increases likelihood to purchase / consider your brand(s).•Minimizes the effects of a crisis and rebuilds trust. Establishes credibilityEmployees and other Internal Publics•Increases employee satisfaction and engagement, leading to greater efficiency, increased retention, reduced turnover, lower recruiting costs and higher productivity.Public PolicyCreates public awareness, understanding and support for legislation, regulation and political candidates.Affects voter behavior. Helps pass legislation, regulation and initiatives.
As a second step, connect the desired outcomes to strategic goals – one of four
key areas:
Financial•Consumer response: Field consumer survey; determine purchase levels and exposure to public relations results; isolate causal effects through statistical analysis. •Market Mix Modeling / Econometric Modeling: Gather public relations output / outcome data by function, market, region and time period; factor revenue-generation by market, by region and over time. Apply regression analysis.Reputation / Brand Equity•Benchmark reputation / relationship metrics via survey prior to a campaign; repeat every three to six months.•Correlate attitudinal studies with customer purchase attitudes and behavior.Employees and other Internal Publics•Compare control groups to employee populations exposed to public relations activities.•Focus on performance outcomes, not attitudes or awareness, such as employee satisfaction and engagement findings, employee turnover statistics and other recruitment data.Public PolicyUse tracking services to track awareness, correlate to public relations activity, and connect to actual votes recorded and legislative outcomes.
Third, measure the desired outcomes not in terms of outputs – clips or activity – but in terms
of outcomes:
Barcelona Declarationof Measurement Principles
AMEC U.S. & Agency Leaders Chapter
Tim MarkleinExecutive VP, Measurement & Strategy
Weber Shandwick
AMEC’s first international chapter. Launched in fall 2009. Provides a networking forum for U.S. research and agency leaders. Represented on AMEC Board by inaugural chair David Rockland of Ketchum. Includes 16 members:
Existing AMEC Members:BurrellesLuce, Cision, Dow Jones, Echo Research, Report International, VMS
Large Agency Research Leaders:Edelman/StrategyOne, Fleishman-Hillard, Hill & Knowlton, Ketchum, MS&L, Ogilvy, Weber Shandwick
Midsize Agencies and Specialist Research Firms:Chandler Chicco, Evolve24, Leading Communicators
U.S. & Agency Leaders Chapter
Topic #1. Ad Value Equivalency
Building on IPR Commission’s Oct’09 vote to “condemn the name, concept and practice of AVEs,” we agreed to the following fundamental principles:
1.This topic is germane to AMEC.2.Outcome based metrics – attributes, behavior, business results, etc. – should always be used in preference to output metrics, whenever possible.3.When measures are based on cost of media space or time, they should be referred to as “costs” not “values.” 4.If media costs are used, they should be adjusted for quality (tone, message, etc.) and for the actual space or time occupied by a client or competitors.5.Multipliers should not be used unless validated by third-party audits, surveys or market mix models.6.Negotiated rates should be used (not rate cards).7.Methodologies must be transparent and repeatable.
Topic #2. Social Media Measurement(part one of two)
Building on AMEC U.S. member experience, we agreed to the following fundamental principles:
1.Social media can be measured.2.Social media measurement is a discipline and a process, not a tool.3.There is no “single metric” for social media.4.Media content analysis has a fundamental role in social media measurement – and should be supplemented by web and search analytics, sales and CRM data, survey data and other methods.5.Organizations need clearly defined goals and outcomes for social media in order to measure it.
• Refer to PRSA “Business Outcomes” outline:http://comprehension.prsa.org/?p=628
• Refer to Altimeter Group “Social Marketing Analytics” outline: http://bit.ly/dldIHf
Topic #2. Social Media Measurement(part two of two)
Building on AMEC U.S. member experience, we agreed to the following fundamental principles:
6.Evaluating quality and quantity together is critical, just as it is with conventional media.7.Given the scale and volume of social media, technology-assisted analysis is necessary – but human reading and coding is still valuable for precision analysis of tone, messaging, nuance, etc.8.Measurement must adapt to the media, focusing on “conversation” and “communities” not “coverage.”9.Understanding reach and influence is important, but existing sources are not accessible, transparent or consistent enough to be reliable.10.Experimentation and testing are key to success – though measurement and communication fundamentals still apply.
Topic #3. Market Mix Modelling
Building on the initial NYU market mix modeling report and AMEC U.S. member experience, we agreed to the following fundamental principles:
1.Clients are creating demand for market mix models to evaluate the impact of consumer marketing.2.The PR industry needs to understand the value and implications of market mix models for accurate evaluation of consumer marketing PR, in contrast to other measurement approaches.3.The PR industry needs to develop PR measures that can provide reliable input into market mix models.
• Such measures might focus on media quantity and quality, messaging and audience impact, traditional versus social media, consistency with other marketing disciplines, etc.
Topic #4. TransparencyClients are entitled to know how their content was obtained and the methodology used to append any metadata to that content:
•Sources:Print - Physical clippings, aggregator feeds, website linksBroadcast – closed caption text, speech to text, manual observationInternet – aggregators, spiders, text scrapes, screen viewsBlogs – main post only or post plus comments and forwards
•Qualitative assessments:Human or machine scored, sampled or all coverageTone – criteria for scoring should be clear and sharedReach metrics – the source of the data and the criteria they are based upon should be clear, i.e. persons 2+, persons 18+, daily or monthly visitors, unique visitors, etc.Other – content analysis of messages, scorecards, calculated measures such as impact scores or value scores should be understandable and not a “black box”
AMEC International Business Monitor 2010 46
Networking Refreshment Break
A Declaration of Measurement Principles
David Rockland Partner / CEO, Managing Director
KetchumAMEC Board Member
IPR Measurement Commission, Past Chairman
TELE-VOTING SYSTEM
The debate: how to vote
1.You will be advised when to vote2.You will be given clear options on which to vote3.Please click your preferred option during the voting time4.To change your option, click again. The last one you record will be the one accepted by the system5.The result will be shown on the screen once all voting finishes6.Summit staff will collect the voting keypads at the end of the conference7.Further opportunity for comment to [email protected] until July 15
Our Summit Voting Test!
Q. Is Catalan the officially spoken language in Barcelona?
• YES: Press 1
• NO: Press 2
Seven Principles
1. Importance of Goal Setting and Measurement 2. Media Measurement Requires Quantity and Quality3. AVEs are not the Value of Public Relations 4. Social Media Can and Should be Measured 5. Measuring Outcomes is Preferred to Measuring
Media Results 6. Business Results Can and Should Be Measured
Where Possible 7. Transparency and Replicability are Paramount to
Sound Measurement
Principle 1:Importance of Goal Setting and Measurement
• Goal-setting and measurement are fundamental aspects of any public relations program
• Goals should be as quantitative as possible and address who, what, when and how much the PR program is intended to affect
• Measurement should include representative traditional and social media; target audience changes in awareness, comprehension, attitude, and behavior as applicable; and business results
The Seven Principles
Principle 1Importance of Goal Settingand Measurement
• YES: Press 1
• NO: Press 2
Principle 2: Media Measurement Requires Quantity and Quality
Overall clip counts and general impressions are usually meaningless. Instead, media measurement, whether in traditional or online channels, should account for:
•Impressions among the target audience
•Quality of the media coverage including:ToneCredibilityMessage DeliveryInclusion of a 3rd party or company spokespersonProminenceVisual dimensions
•Quality can be negative, positive, or neutral
The Seven Principles
Principle 2 Media Measurement Requires
Quantity and Quality
• YES: Press 1
• NO: Press 2
Principle 3: AVEs are Not the Value of Public Relations
• Advertising Value Equivalents (AVEs) do not measure the value of public relations and do not inform future activity; they measure the cost of media space
• Where a comparison is to be made between the cost of space from earned versus paid media, validated metrics such as Weighted Media Cost should be used and reflect:
• negotiated advertising rates• quality of the coverage (see Principle 2), including negative results• physical space of the coverage, and the portion of the coverage that is relevant
• Multipliers intended to reflect a greater WMC for earned versus paid media should never be applied unless proven to exist in the specific case
The Seven Principles
Principle 3
AVEs are Not the Value ofPublic Relations
• YES: Press 1
• NO: Press 2
Principle 4: Social Media Can and Should Be Measured
• Social media measurement is a discipline, not a tool; but there is no “single metric”
• Organizations need clearly defined goals and outcomes for social media
• Media content analysis should be supplemented by web and search analytics, sales and CRM data, survey data and other methods
• Evaluating quality and quantity is critical, just as it is with conventional media
• Given the scale and volume of social media, technology-assisted analysis is necessary – human reading and coding is valuable for precision analysis of tone, messaging, nuance, etc.
• Measurement must focus on “conversation” and “communities” not “coverage”
• Understanding reach and influence is important, but existing sources are not accessible, transparent or consistent enough to be reliable; experimentation and testing are key to success
The Seven Principles
Principle 4 Social Media Can and Should
Be Measured
• YES: Press 1
• NO: Press 2
Principle 5: Measuring Outcomes is Preferred to Measuring Media Results
• Outcomes include shifts in awareness, comprehension, attitude and behavior related to purchase, donations, brand equity, corporate reputation, employee engagement, public policy, investment decisions, and other shifts in target audiences regarding a company, NGO, government or entity, as well as the audience’s own beliefs and behaviors
• Benchmark and tracking survey research are the preferred practices for quantitative measurement
• Standard best practices in survey research including sample design, question wording and order, and statistical analysis should be applied in total transparency
The Seven Principles
Principle 5
Measuring Outcomes isPreferred to Measuring Media Results
• YES: Press 1
• NO: Press 2
Principle 6: Business Results Can and Should Be Measured Where Possible
• Models that determine the effects of the quantity and quality of PR outputs on sales or other business metrics, while accounting for other variables that drive sales, are a preferred choice for measuring consumer or brand marketing. Related points are:
Clients are creating demand for market mix models to evaluate
the impact of consumer marketing The PR industry needs to understand the value and implications
of market mix models for accurate evaluation of consumer marketing PR, in contrast to other measurement approaches
The PR industry needs to develop PR measures that can provide reliable input into market mix models
Survey research can also be used to isolate the change in purchasing, purchase preference or attitude shift resulting from exposure to PR initiatives
The Seven Principles
Principle 6
Business Results Can andShould Be Measured Where Possible
• YES: Press 1
• NO: Press 2
Principle 7: Transparency and Replicability are Paramount to Sound
MeasurementPR measurement should be done in a manner that is transparent and replicable meaning the detailing of:
Media Measurement:• Quantitative – source of the content: print, broadcast, internet, consumer generated media along with criteria used for collection• Qualitative – human or automated, tone, reach to target, content, overall media impact
Surveys:• Methodology –sampling frame and size, margin of error, probability or non-probability• Questions – all should be released as asked• Statistical methodology- how specific metrics are calculated
The Seven Principles
Principle 7
Transparency and Replicability are
Paramount to Sound Measurement
• YES: Press 1
• NO: Press 2
Next Steps:
Panel Organizations
Delegates
AMEC
THE RESULTS Yes No1. Importance of Goal Setting and Measurement 97% 3%
2. Media Measurement Requires Quantity and Quality 97% 3%
3. AVEs are not the Value of Public Relations• Do not measure the value of PR or future activity• Where comparisons made validated metrics should be used• Multipliers never be applied unless proven to exist
70%92%57%72%
30%8%43%28%
4. Social Media Can and Should be Measured 93% 7%
5. Measuring Outcomes is Preferred to Measuring Media Results 87% 13%
6. Business Results Can and Should Be Measured Where Possible 86% 14%
7. Transparency and Replicability are Paramount to Sound Measurement
95% 5%