Head or Heart: The Conflicts of Political Polling

HEAD OR HEARTThe conflicts of political polling

alex wheatley

Copyright © 2016 Lightspeed, LLC. Proprietary and Confidential. All rights reserved.

• Day before vote• 450 Nat Rep• Undecided asked lean

POLLING THE UK’S E.U. REFERENDUM

1

48% 52%

Remain Leave *the result was 48% remain 52% leave

WE CALLED IT!


THIS WAS JUST LUCK

interview 96% of voters and still call Remain victory

One sample:

82%

82%

18%

Remain Leave


AS LONG AS IT’S REPRESENTIVE

Don’t know the causes we need to represent!

WWW

NAT REP NAT REP


“I’m voting remain”


NOT A SIMPLE QUESTION

IT’S SENSITIVE1…so context has an impact

IT’S A SELF OBSERVATION2…so it needs breaking down

IT’S A BEHAVIOUR PREDICTION3…we’re not good at those


FAILED PREDICTION

272 F.T. 58% OF POLLS GOT IT WRONG

CAN WE PREDICT OTHERS?


NEED SOME EXPERT PREDICTORS

“Best way to measure predictive abilities is to longitudinally measure accurate predictions”


SOME EXPERT PREDICTORS

GOT IT RIGHT!

0.37

37%

63%

Remain Leave


MORE WORK NEEDED

Emotions bias

prediction

2015 Conservative victory makes

sample self selecting


NOT A SIMPLE SOLUTION

• 2 years of experimentation• More than 10 studies

Out of the box solution?No experts needed


ASKED TO “PREDICT WHO WILL WIN?”

GOT IT WRONG!0.61

61%

39%

Remain Leave


INFORMED PREDICTION


ONLY A SLIGHT CHANGE

Initial Prediction After Information After Information of Confident & Correct

61% 58%

76%

*percentage predicting Remain victory



MORE WRONG IF THINK THEY ARE CORRECT


61% 58%

76%




MORE WRONG IF THINK THEY ARE CORRECT


61% 58%

76%


RED FLAG FOR CONFIRMATION BIAS


STRUGGLE WITH HEAD SO WHAT ABOUT HEART?

I.A.T. showed clear lead for Cameron across positive attributes

WORKED IN 2015


TESTING BREXIT

Thumbs up/down I.A.T. of

images

Undecided Voters

52% 48%

DIDN’T WORK THIS TIME

*percentage thumbs up of undecided voters


NOT A CLEAR READ

Undecided Voters

40%

*percentage thumbs up of undecided voters

NEGATIVE REACTION

A clearer picture?


POLLS BALANCED & NO SINGLE MEASURE TIPS SCALE


ADD IT ALL UP

• Emotions


ADD IT ALL UP

• Emotions• Prediction


ADD IT ALL UP

• Emotions• Prediction• Biases


ALL FALLING ON ONE SIDE


NO LONGER A COIN TOSS!

The next test…

Data & Analytics

Head or Heart: The Conflicts of Political Polling