Upload
compusense-inc
View
119
Download
5
Embed Size (px)
DESCRIPTION
Check-All-That-Apply (CATA) questions are increasingly being incorporated into consumer tests because they provide a simple mechanism for consumers to communicate their perceptions of products being evaluated. We review existing and propose new approaches for analysing data obtained from such a study. Contingency tables are well known, and can be pictured using mosaic plots. Correspondence analysis (CA) using the χ2 distance provides dimensionality reduction, but Hellinger's distance is often preferred where rarely cited attributes skew results. Word clouds can be used to determine citation frequency for responses that might be entered in open comment format by consumers (e.g. upon checking "other" in a CATA question). Cochran's Q test provides a univariate test for differences between 3 or more products, and the sign test can be used to assess pairwise differences. To our knowledge no omnibus hypothesis test is available for assessing global differences. We propose such a test, based on randomization and Cochran's Q statistics, in which the null distribution is formed from data re-randomizations. Multidimensional alignment (MDA) is suggested to investigate the relationship between products and CATA attributes. The φ-coefficients, proposed to understand relationships between CATA attributes, are readily visualized using MDS. Consumers can be asked to evaluate an ideal product, and the gaps between the real and ideal products can inform product improvements. Penalty and penalty-lift analyses can reveal (positive and negative) hedonic drivers. Methods are illustrated by means of CATA study on whole grain breads.
Citation preview
Existing and new approaches for the analysis of CATA data
Michael Meyners Procter & Gamble B. Thomas Carr Carr Consulting John C. Castura Compusense Inc.
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
Are 2 products different?
Sign Test (exact)
McNemar’s Test (approximation)
Selected Not Selected
Selected a b
Not Selected c d
Product B
Pro
du
ct A
Are 3 (or more) products different?
Cochran’s Q Test (approximation)
extends McNemar’s Test
Testing strategy
Attributes Products Test 1 All 1 2
Cochran’s Q Test Sign Test
Significant?
Testing strategy
Attributes Products Test All All 1 All 1 2
? Global Test ? Cochran’s Q Test Sign Test
Significant?
Significant?
Global Test
Test Statistic: Sum of 𝑄 statistics
Approximate Test
Chi-square distribution formed using property of additivity
Problem: null distribution incorrect
(chi-square distributions are not independent)
Global Test
Solution:
Randomization test (quasi-exact):
null distribution is valid because it is based on repeated and appropriate reshufflings of observed data
Testing strategy
Attributes Products Test All All 1 All 1 2
Randomization Test Cochran’s Q Test Sign Test
Significant?
Significant?
With CATA data, can we…
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
1 2 3 … Ideal Total
Fresh 89 70 71 … 142 642
Warm 6 7 4 … 90 135
Crusty 24 4 32 … 47 193
Soft 76 95 75 93 570
Sweet 102 18 30 85 466
… … … … … ... …
Total 2169 1144 1539 … 2579
Contingency table
Bar charts
Problem: This is a mess.
We need a visual summary of this data.
Bar charts
Correspondence Analysis
Reduces dimensionality of contingency tables (Think: PCA for ordinary data)
Attributes
Pro
du
cts
Correspondence Analysis
Could use…
Chi-square distances
Sensitive to attributes with low counts
Hellinger distances
Low counts not a problem
Correspondence Analysis
Sweet
2
4
3
1
5 6
Molasses
Exciting
Tempting
Satisfying
Tasteful Fullness Comfort
Dense
Firm
Moist Warm
Homemade
Fresh Chewy Traditional
Soft Springy Wheat
Brown
Healthy
Nutritious
Crusty
Texture
Coarse
Sesame
Rustic Nutty
Crunchy
Grainy Seeds
Component 1 (79.2%)
Co
mp
on
ent
2 (
11
.9%
)
Component 1 (79.2%)
2 3
1
5 6 Traditional
Soft Springy Brown
Nutty
Crunchy Grainy
Seeds
4
Correspondence Analysis
Can only look at 2 (or 3) dimensions at once Problem: True relationships between products and attributes might be obscured.
Multidimensional Alignment (MDA)
Solution: Look at each product individually.
Positive association with these attributes
Negative association with these attributes
With CATA data, can we…
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
φ-coefficient
Measures correlation between 2 binary attributes
ClassicalMDS on matrix of 𝟏 − 𝝋 distances
Fresh
Warm Crusty
Soft
Moist
Nutty
Tempting Sweet
Dense
Texture
Wheat
Exciting
Crunchy
Homemade
Seeds
Satisfying
Coarse
Nutritious
Molasses
Tasteful Rustic
Traditional
Springy
Sesame
Firm
Fullness
Healthy
Grainy
Comfort
Brown
Chewy
ClassicalMDS on matrix of 𝟏 − 𝝋 distances
Fresh
Warm Crusty
Soft
Moist
Nutty
Tempting Sweet
Dense
Texture
Wheat
Exciting
Crunchy
Homemade
Seeds
Satisfying
Coarse
Nutritious
Molasses
Tasteful Rustic
Traditional
Springy
Sesame
Firm
Fullness
Healthy
Grainy
Comfort
Brown
Chewy
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
Penalty-Lift Analysis
Problem: What is the relationship between what consumers perceive and their hedonic response? Solution: Look at elicitations of attributes and association with liking
change in liking (attribute elicited vs. not elicited)
change in liking (attribute elicited vs. not elicited)
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
Comparison with Ideal
Problem: How to modify a product to delight consumers? Solution 1: Close gap between product and ideal (accounting for natural variability!)
Difference in elicitation rates
Product – Ideal
Difference in elicitation rates
Product – Ideal
Penalty Analysis
Problem: How to modify a product to delight consumers? Solution 2: Focus on the most relevant attributes for consumers’ hedonic responses
% consumers
Higher Relevance for Many
Consumers
+2
+1
0
10 30 50 40 20
Penalty analysis based on “must have” attributes
Liki
ng
Fresh Warm
Crusty
Soft
Moist
Chewy
Nutty
Tempting
Sweet
Grainy
Dense
Texture
Healthy
Exciting
Crunchy
Homemade
Brown
Seeds
Satisfying
Coarse
Fullness
Nutritious
Molasses
Tasteful
Rustic Traditional
Springy
Sesame
Wheat
Comfort
know if products are different?
characterize products?
know how attributes are correlated?
know what drives consumer liking?
focus reformulation efforts?
With CATA data, can we…
Yes!
Thank you for your attention!
For more information see the following publication and references therein:
Meyners, M., Castura, J.C., Carr, B.T. (2013). Existing and new approaches for the analysis of CATA data. Food Quality and Preference, 30, 309-319.
B. Thomas Carr Carr Consulting
Michael Meyners Procter & Gamble
John C. Castura Compusense Inc.