Statistical methods for election forecasting in the United Kingdom 1970–90

This article was downloaded by: [UQ Library]On: 04 November 2014, At: 06:59Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

British Elections andParties YearbookPublication details, including instructionsfor authors and subscription information:http://www.tandfonline.com/loi/fbep18

Statistical methods forelection forecasting in theUnited Kingdom 1970–90Clive Payne a ba Director of the Computing and ResearchSupport Unit, Social Science Faculty ,University of Oxfordb Fellow of Nuffield College , OxfordPublished online: 08 Nov 2007.

To cite this article: Clive Payne (1992) Statistical methods for electionforecasting in the United Kingdom 1970–90, British Elections and PartiesYearbook, 2:1, 138-158, DOI: 10.1080/13689889208412917

To link to this article: http://dx.doi.org/10.1080/13689889208412917

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy ofall the information (the “Content”) contained in the publicationson our platform. However, Taylor & Francis, our agents, and ourlicensors make no representations or warranties whatsoever as to theaccuracy, completeness, or suitability for any purpose of the Content.Any opinions and views expressed in this publication are the opinionsand views of the authors, and are not the views of or endorsed byTaylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources ofinformation. Taylor and Francis shall not be liable for any losses,actions, claims, proceedings, demands, costs, expenses, damages,and other liabilities whatsoever or howsoever caused arising directly

http://www.tandfonline.com/loi/fbep18

http://www.tandfonline.com/action/showCitFormats?doi=10.1080/13689889208412917

http://dx.doi.org/10.1080/13689889208412917

or indirectly in connection with, in relation to or arising out of the useof the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of accessand use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014

http://www.tandfonline.com/page/terms-and-conditions

http://www.tandfonline.com/page/terms-and-conditions

Statistical methods for election forecasting in theUnited Kingdom 1970-90

Clive Payne

Introduction

Forecasting the outcome of elections has long been regarded by political scientists andthe media as an integral part of the coverage of elections in the United Kingdom. Tothis end statistical techniques of varying degrees of sophistication have been used,ranging from crude extrapolation to sophisticated model building. While the primaryinterest has been in election-night forecasting for the notional elections to the Parlia-ment at Westminster and for elections of the UK representatives to the EuropeanParliament, there has been growing interest in forecasting outcomes in local electionsand in individual by-elections for Westminster constituencies. Recently a number ofpolitical scientists have developed time-series models of government support, usingpolitical and economic indicators, which can be used for long-term election fore-casting. ' This article is concerned only with short-term election forecasting, usuallyin the period immediately preceding the election and on the election night itself, whenactual results become available.

The article reviews the statistical methods used by the BBC, and their perfor-mance, in a range of types of election in the last twenty years. The main focusis on the prediction of the national elections, where a ridge regression model hasbeen used in all elections since 1974; brief details of this method are given here, asfull details have been published elsewhere.2 However, the paper presents a completereview and assessment of the performance of the method in this period and sets outthe modifications made to the basic prediction algorithm to take account of changesin the political situation and the adjustments made for elections of different types.

The methods applicable for election forecasting are determined by the particularelectoral and party system. A major factor here is the system used to translate votesinto winning candidates; in the United Kingdom this is the 'plurality' or 'first-past-the-post' system. Other important determinants are the way election results areactually declared and received. The paper thus starts with a brief description of thespecial features of the UK electoral system which constrain the choice of statisticalmethod.

The UK system

Types of election

The Westminster Parliament currently has 634 constituencies, plus 17 in Ulster,averaging about 68,000 electors.3 Constituencies are made up of wards which aremade up in turn of polling districts.

138

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014

Statistical methods for election forecasting in the UK 139

The European Parliament has 78 constituencies, plus 3 in Ulster, voting every 5years (1979,1984 and 1989) for UK representation in the European Parliament. TheEuro-constituencies are aggregates of between 8 and 13 Westminster constituencies,giving electorates of about 500,000.

Local elections for county, district and borough councils are held on a fixed day inMay every year. The cycle and coverage of the local elections vary considerably, witheither whole council elections every four years or one-third council elections eachyear. The main interest in local elections has been in estimating the national standingof the parties, particularly near the time of general elections, and then forecastingthe national outcome in seats in Parliament. Problems are presented by the com-plexity of local elections, especially their varying coverage, the uneven patternsof candidacy of the major parties, the strength of independents, the existence ofmulti-member seats in some councils, and boundary changes.4 There is nowincreasing interest in forecasting the results of local elections, especially as to whichparties will have control in individual councils.

The main features of the electoral system which constrain the prediction methodare as follows:

1. There is a plurality system to determine the winning candidate in single-memberconstituencies, though there are some multi-member seats in local elections.

2. There are regular changes of constituency boundaries to obtain reasonablyuniform constituency electorates.

3. Constituency results are not collected and published by official authority.4. Results are not available at levels lower than the constituency in the national

elections.5. There is a lack of correspondence between the national and the local elections in

the political units involved. The only political unit common to all elections isthe ward, except in county-level elections, which frequently cut across wardboundaries. The district, county and borough boundaries (except the Londonboroughs which contain 2-A Westminster seats) do not coincide with parlia-mentary ones.

6. There is very little information available on the socio-economic characteristics ofthe political units, as these do not generally coincide with administrative areas.The exception is the Census, conducted in the first year of each decade, whendata are produced for most levels, including parliamentary constituencies.

Thus very little information is available on the socio-political-economic character-istics of the electorates at any particular election at any level, or of changes in thesebetween elections. In particular, the annual local election results do not provide asatisfactory basis for estimating recent partisanship in constituencies for the nationalelections. So forecasting seats has to be based on estimating changes from theprevious election, with very little information being available on covariates otherthan previous party strength.

UK forecasting compared to that in other countries

There are some aspects of elections in the United Kingdom which make electionforecasting more difficult than in most other countries.5 The following factors, indecreasing order of importance, determine the level of difficulty encountered:

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014

140 Articles: Research Methods, Data and Techniques

1. Whether results are available at a disaggregated level, such as polling locations ordistricts. If they are, the forecasting problem is much easier, as predictions canbe based on a suitable sample of the locations. The sample of locations can beselected either to achieve representativeness (for example in Spain) or to giveindicator swingometric or barometric districts (for example in New Zealand).The availability of results at polling-district level also makes the problem ofselecting a sample of voters to interview about their voting intentions orbehaviour much easier. The greater volume of district results also allows therapid calculation of adjustments for bias in declared results (for example inAustralia). The United Kingdom is one of the few countries where results are notavailable at polling district level.

2. The system for converting votes into seats. Here there is a basic distinctionbetween proportional systems and other systems. The prediction problem ismuch easier in proportional systems, as used in other European countries.Forecasting here is essentially an exercise in sampling voters (usually after theyhave voted) in representative districts, to estimate the shares of vote for eachparty. The conversion to seats is then straightforward, using simple algorithmssuch as the d'Hondt rule, usually subject to a threshold which a party has to passto qualify for seats. For other electoral systems the conversion to seats is moredifficult, particularly in those where voters rank candidates in order, such as thesingle transferable vote used in Ireland (although this is a proportional system)and the alternative vote used in Australia, where preference distributions amongvoters must be built in to the conversion procedure. In this respect the UnitedKingdom's plurality system represents an intermediate level of difficulty.

3. How many significant parties there are and whether the parties' candidatesand their patterns of support are unevenly distributed. The United Kingdompresents considerable difficulties in this respect as there are three major partiesstanding throughout the country, so that the forecasting is at least a three-dimensional problem, which also has to take account of the uneven patterns ofsupport of the Labour and Conservative parties in particular. There are twofurther significant parties, the Scottish and Welsh nationalists, which havean important influence on the predictions in their respective countries. Theimplication is that data must be obtained from a range of types of constituency,particularly for regional types, before any real confidence can be attached to theforecasts.

4. The quality of the socio-political-economic data available for the constituencies.In the UK constituencies there is a paucity of up-to-date data either to use ascovariates in election-night prediction, or on which to base sample selection forvoting intention/behaviour polls.

5. Whether there is significant bias in the declaration order of results on electionnight. In the United Kingdom counting of votes is usually done at one place ineach constituency, so that results from urban areas are generally more likely to beannounced sooner than those in rural areas. This introduces the possibility ofsignificant bias in the order of results declared.

6. Who collects the results and how they are declared. In some countries results aremade available instantly in computer-readable form by an official authority (suchas the Electoral Commission in Australia). In the United Kingdom, each mediaorganization has to employ so-called 'stringers' at each count to supply the resultsto them; this is an error-prone procedure. Another key factor is whether results

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


are made available on a progressive basis (as in Australia), rather than only whenthe whole constituency result is announced (as in the United Kingdom).

Types of forecasting

Three types of forecasting problem in the United Kingdom can be distinguished.The first concerns the extrapolation from voter intention polls or local electionresults to seats in Parliament; these are used to translate the estimates of the parties'current standing to give some indication of what the composition of Parliamentwould be if a national election had been fought at the same time. Only crudemethods of extrapolating shares of the vote to seats can be used here. The secondtype concerns the prediction of the parties' shares of vote, and the winningcandidate, in an individual constituency election. This is essentially an exercise insampling the electors who have just voted, although account must be taken of thehighly clustered nature of the sample which is usually obtained. The third and mostimportant type deals with the forecasting of the final composition of Parliament onthe day of a national election when actual results are declared; these forecasts havetwo components - a prior forecast and an updated, results-based forecast asconstituency results become available. This type presents the opportunity to usesophisticated statistical techniques based on the modelling of the set of resultsdeclared.

Extrapolation from polls and local election results

In the interval between national elections, voting intention polls are regularlycarried out. The aim is to translate voting intentions into a pre-election forecast ofthe number of seats that would be won by each party if a national election were heldon the date of the poll. The annual local elections also provide an opportunity toestimate the national standing of the parties; here local election results areaggregated to a sample of notional Westminster constituency results. There isspecial interest in this forecast in the period leading up to a national election,especially a general election, when the governing party is keen to assess the numberof seats it is likely to win given its current standing, and so help decide when to callthe election.

The data available for this type of forecast are very sparse. Usually they are nomore than an estimate of the projected national shares of the vote for each party,based on a quota poll of no more than 1,000 electors. Occasionally larger polls arecarried out and disaggregated vote share estimates, usually on a regional basis, maybe available. Separate polls are often carried out in Scotand. The local electionsprovide a richer data base containing separate vote share estimates by region andtype of political contest. Thus a model is required to convert national voting sharesinto seats in Parliament in a plurality system. There is not a great premium onaccuracy, as a real election outcome is not being predicted and the polling data are ofcourse themselves subject to sampling and non-sampling errors. There are variousalternative models which can be used for this purpose, and their efficacy can betested empirically by retrospective application to previous general election results.

In the 1970s, pre-election forecasts in the United Kingdom were based on 'swing',a measure of the change in the relative shares of the vote for the two major parties,

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


Conservative and Labour. If d and L, are the percentage of the vote forConservative and Labour respectively at the previous election, and C2 and L2 are thepercentages forecast by the poll, the swing, S, is defined as

5 = 0.5*[(C2-C,) + (L1

Then, given the assumption that all other electoral movements involving otherparties and abstention cancel out, swing measures the net transfer of votes betweenConservative and Labour. The forecast is obtained by applying the swing 5 to eachconstituency in turn. The forecast Conservative share is d + S; Labour's is Li - 5;and the percentage shares of all the other parties are assumed to remain unchanged,so that the predicted winner is the party with the highest predicted percentage. Astandard presentational technique is to rank seats in order of the swing required tochange hands, and to display this as the election 'battleground'. A 'swingometer',calibrated with the predicted seats for various levels of swing, was a regular featureof the TV coverage of general elections in the 1970s, and presenters took pleasure inpitting the swingometer against the more sophisticated, computer-based forecasts.

The swing model worked reasonably well up to the 1980s, as the minor partieswon very few seats. It gave reasonable predictions of seat totals, but often the

jg 30^Conservative majority

Balance ofpower held

by third party

25L... *.'.,20 25

• 1983 election result,

40

Alliance's share of vote (%)

Note: Chart assumes uniform nationwide swings from 1983 result.Source: The Economist.

Figure 1 Projections for the uniform change model for the 1987 election

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


outcomes in individual seats were badly predicted, with sometimes as many as fiftywrongly predicted individual winners. There was no attempt to attach a probabilityestimate to the predicted outcome in an individual seat, or to calculate confidenceintervals for seat totals.

With the rise of the Alliance parties in the 1980s, modifications were needed tocope with the threat to the hegemony of the major parties (in terms of seats). Anatural extension of the basic swing model is to apply the predicted change inpercentage share of the vote for each significant party uniformly across allconstituencies to give a projected winner in each seat - the 'uniform change' model.This model has been used by the BBC from 1974 onwards. It has provided goodprior forecasts when used on the election-night programmes, and retrospectivetesting on actual election outcomes has confirmed that this refined model issufficiently accurate for pre-election use. The natural extension of the two-dimensional swingometer is a set of three-dimensional contour plots, where the xand>> axes represent the predicted vote percentages for pairs of parties and the z axisplots the projected number of seats for a particular party. Figure 1 shows a usefuldevice, extracted from these three-dimensional plots, for demonstrating the projec-tions for the uniform change for Conservative, Labour and Alliance versionsof this model for the 1987 election.

An important question for the application of these models is whether betterforecasts are obtained with disaggregated polls than with those based on a uniformnational projection. There has been an understandable assumption that forecastswould be better if the models were applied separately by region using disaggregatedpolling information. The United Kingdom has experienced marked regional dif-ferences in electoral behaviour in the 1979-87 general elections. There has beenthe emergence of a north/south divide, with Labour doing worse in the south, wherethe Alliance parties have been the main opponents of the Conservatives. However,this is essentially an empirical question about whether the model actually performsworse when applied nationally, where errors in individual regional forecasts may toa large extent cancel out. Table 1 presents the results of a retrospective forecast of thelast two general elections, comparing the uniform swing and the uniform changemodel both nationally and regionally based. The main conclusions to be drawn arethat the uniform change model performed well for both elections and that regionaldisaggregation does not necessarily improve the predictions. Note that the swingmodel did very well in the last (1987) general election, indicating that the 1983-7decline in the Alliance vote benefited, on average, the two major parties roughly inproportion to their ratio of support in each seat.

Other models have been proposed to cope with the three-party structure. Inparticular, Reece proposed what he termed the 'proportional loss model'.6 Hismodel assumes that the percentage of people deserting their previous allegiance is(statistically) constant over the country, and the ratio in which these deserters areattracted to other parties is also constant. Using the notation above, and assumingthat support is declining for both Conservative and Labour, then the projectedConservative vote in a particular constituency is:

(Previous Conservative vote)

and the projected Labour vote is:

(Previous Labour vote) *L2/Li,

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


Table 1 Comparison of swing and uniform change models1987 general election projected from 1983

ConservativeLabourAllianceNationalistsOthers (Ulster)Con/Lab majorityOverall majority

Change

Regional

372236

196

1713694

National

381230

184

17151112

Swing

Regional

368235

237

1713386

National

375228

264

17147100

Actual

376229

226

17147102

1983 general election projected from 1979 (BBC/ITN notional results)

ConservativeLabourAllianceNationalistsOthers (Ulster)Con/Lab majorityOverall majority

Change

Regional

395211

215

18184140

National

392213

197

19179134

Swing

Regional

409212

65

18197168

National

407215

55

18192164

Actual

397209

234

17188144

Note: Regions: London, south, Midlands, north, Wales and Scotland.

with the deserters from both parties going to the Alliance. Empirical investigationshave demonstrated that this model often did worse than the uniform change model,which has the additional virtue of simplicity.

Forecasting individual election results

Interest in predicting outcomes in individual constituency elections using exit pollsarises in two contexts. There has been an attempt to use 'bell wether' or 'indicator'seats in general elections; the predicted individual result is projected across similarconstituencies. The primary application is now, however, to forecasting the resultsof Westminster by-elections, which have become an important focus of the media'scoverage of the political scene in the United Kingdom. The BBC commissionedMORI to conduct three by-election polls in the mid-1970s; these were not judged tobe successful and the practice stopped until just before the 1983 general election.Since then the BBC has conducted exit polls at nearly all the politically interestingby-elections, and these have generally had a very successful record.

Exit polls in individual constituencies in the United Kingdom present specialsampling problems, as the sample of voters is highly clustered at polling stations.There may be practical reasons which preclude the choice of some polling stations,particularly in rural constituencies. There are also very few political data availableon which to base the choice. Finally, exit polls are characterized by very high refusal

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


rates, which can introduce significant bias into the estimates. Payne et al. give adetailed description of the by-election exit polls carried out by the BBC in the periodof 1981-6.7 These involved choosing a systematic sample of polling stations from alist of stations ranked by partisanship, and then interviewing voters throughout theday of the election at a rate which depended on the size of the station's electorate.A noteworthy feature of the procedure was that better predictions were usuallyobtained when the voter transition matrix (recalled previous general election votecross-tabulated with vote in the by-election) was scaled to fit the known margin forthe previous general election vote. An imputation procedure was used to deal withrefusals.

These polls have generally provided very good predictions, usually within theexpected sampling error for the sample size (typically 2,000). Since the 1987 generalelection, the BBC by-election polls have been carried out by NOP, who have used arather different sampling scheme. They construct a sample of both time periods andof polling locations, which are ranked by partisanship and selected with probabilityproportionate to electorate size. Their record has also been very good.

Election-night forecasting

Forecasting the final composition of the House of Commons has long been anintegral part of TV and radio programmes on general election night. These forecastshave a very high profile and there is a high premium on accuracy, as the final resultwill soon be known. Election-night forecasting has two components. First is theproduction of a prior forecast made before any actual results have been declared.This is based on various sources of prior information abut the electorate's votingintentions, usually including specially commissioned polls for this purpose; thesimple statistical techniques described above are used to translate projected voteshares into seat projections. The second component is the production of updatedpredictions as actual constituency results are received. In the United Kingdom themain problem in this respect is that the first constituency results declared do notform a representative sample of all constituencies. Figure 2 gives a plot of thedifference between the Conservative-Labour swing estimated from the resultsdeclared so far and the final swing, for the last five general elections. The plotsindicate a considerable bias in the early stages, but this is not systematically in favourof one party, and then a consistent bias towards Labour. Statistical techniques ofvarying sophistication have been devised to cope with this problem.

The two main TV companies, BBC and ITV, have produced election-nightforecasts since 1951. This article gives brief details of the statistical method usedby the BBC since 1974. ITN has never published details of its procedures, althoughit is believed that these are based on the application of predicted swings separatelyfor seats of different types; the BBC used a similar method in the 1970 generalelection.

The prior forecast

As well as the choice of a model for translating vote shares into seats, a key questionis what sort of information to use as the basis for the forecast. Four sources have beenused, often in combination, but each has cost implications:

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014

5?

•55

o

c3

a>

1

Key

+x

— +X

1974

1974

1979

1983

1987

FebOct

to5aa.

Number of seats declared

Figure 2 Bias in swing by declaration order

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


1. Poll-of-polls. This is a weighted average of the final pre-election polls conductedby the major polling organizations giving predicted national shares. This in-formation is essentially free, but the forecasts of shares are sensitive to a lateswing and there are problems of identifying respondents who will actually vote.

2. Quota polls. A specially commissioned quota poll of large size (typically 2,000-5,000) voters is carried out on the day before the election and on the day itself.The poll is usually carried out to elicit voter attitudes as well as voting behaviour,but has booster samples in marginal seats. The poll gives projections of shares ofthe vote, usually disaggregated by region and type of political contest (categor-ized by the previous top two parties and the lead of the winner over the secondparty). This source is subject to the usual problems with quota polls, especiallythat of distinguishing people who are actually going to vote.

3. National exit polls. A national sample of polling stations is chosen, often confinedto marginal seats, and a sample of voters leaving the polling stations is askedhow they voted. Samples of 10,000-15,000 voters are typical. Estimated changesin parties' shares are applied to seats of similar type in order to project thewinning party. However, exit polls are very expensive, and there is a problemover the choice of the sample of polling stations, as there is very little informationon previous voting (unless a portfolio of stations is established over a long period,which requires a lot of effort initially and needs regular updating). The BBC haschosen one polling station at random within each selected constituency; ITN hasattempted to select a polling station which is politically representative of theconstituency. Other problems arise from the highly clustered nature of thesample and high refusal rates. Exit polls have been used by ITN in every generalelection since October 1974, and occasionally by the BBC (October 1974 generalelection and the 1989 European election).9

4. Indicator seats. Here there is an attempt to find 'swingometric' seats whosepolitical movements and characteristics match those of the average marginal seat,so that their results are taken as indicators of the national political trend. An exitpoll is conducted in these seats and the projected changes are used for the wholeset of marginal seats. The BBC achieved a spectacular success with the first use ofexit polls in the Gravesend constituency at the 1970 election. The predictedswing in Gravesend was very close to the national average, although the use of theexit poll was rather underplayed in the election coverage itself because it wasout of line with that indicated by the opinion polls (which, as it turned out,considerably underpredicted the swing from Labour to Conservative). Threeindividual exit polls were also carried out at the October 1974 and the 1979elections, but they were not very successful.

This method is reasonably cheap but has proved to be rather risky. Apart fromprevious votes, there is little information on which to base the choice of suitableseats. There can be large and unknown changes in the socio-economic composi-tion of individual electorates between elections. So, despite the seductiveness ofthis approach, it has been more or less abandoned.

The BBC prior forecast methodThe BBC has used information from all the above sources in varying degreesthroughout the last two decades. In the last two general elections the main source ofinformation on which to base the prior forecast was a Gallup quota poll, whichprovided estimates of change in party shares of the vote separately for different

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


Table 2 General election prior forecasts, 1974-87: errors in forecast of majorityof winning party over second party

Election

1974 (Feb)1974(Oct)197919831987

BBC

6137(28*)

42

70

Main BBC source

Exit polls in 3 seatsNational exit pollPoll-of-pollsQuota pollQuota poll

ITN

5014223431

Polls

3475146036

Notes* Brown/Payne computer-based forecast. The actual forecast used was swingo-meter-based.11n all cases the main BBC sources were supplemented by Scottish polls and expertopinion about particular seats for the computer-based forecasts.2 The Polls column gives an estimate of the error if the final poll-of-polls had beenused, assuming uniform national swing.3 BBC European election prior predictions: errors in the Conservative-Labour seatdifference were: 1979 (expert priors only) — 1; 1984 (Marplan quota poll) — 6; 1989(NOP exit poll) —12.

regions and for types of seat characterized by the nature of the political contest. Thepriors also incorporated expert opinion, expressed in terms of probabilities, foroutcomes in seats where special local factors were thought to be operating. Thuspredictions of the vote shares in each constituency were based on a weightedcontribution from each source of prior information. Projected probabilities of eachparty winning the seat were then calculated from the change estimates, using abivariate normal integration as used in the results-based forecast described below.l0

The prior predictions obtained in 1983 were within one seat of the final outcome,although post-mortem analysis indicated that there were compensating errors in theregional change estimates provided by the poll. The same methodology was used inthe 1987 general election, but this led to some embarrassment, with an under-estimation of 70 in the projected Conservative majority over Labour (35 seats inerror). This was primarily due to a poor Gallup poll, which, while giving goodestimates of the minor party shares, underestimated the overall Conservative-Labour lead by 7 per cent, an error which had a large effect on seat projections.Correct poll figures would have given an acceptable error of 16 in the majority.

Prior predictions of seats, 1974-89Table 2 gives the errors in the predictions of the majority of the winning party overthe runner-up for the prior forecasts produced by the BBC and ITN. The averageBBC (computer-based) error is about twenty (that is within ten seats), ITN's isabout thirty, and the crude poll-of-polls-based forecast is over forty. ITN's priorpredictions were consistently reasonable, and their exit-poll-based predictions ofnational shares of the vote were very good. The BBC's prior forecasts were moreoften than not very good but occasionally very poor.

Prior predictions of shares of the vote, 1970-87These are given in Table 3 (poll-of-polls) and Table 4 (election-day surveys). For thepurposes of seat forecasts, the key performance indicator is the error in theprediction of the gap between the first and the second party's share of the vote.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014

Table 3 Accuracy of forecast opinion polls in British general elections, 1945-87

Election

194519501951195519591964196619701974 Feb1974Oct197919831987Mean

Outgoinggovernment

CoalitionLabLabConConConLabLabConLabLabConCon

No. offorecast

polls3

12323435655763.8

Deviation of mean estimate from shareof Great Britain vote obtained by:

Con

+2+ 1+2+ 1—+ 1- 1- 2-

- 2-+3- 1+0.4

Lab

- 2- 2- 4+ 1- 1+ 1+3+4- 2+3+ 1- 2+2

0.0

Lib

+ 1+2+ 1- 1—

- 1- 1- 1+2—-—

- 1

+0.1

Others

- 1-—

- 1——-

- 1_

- 1-

- 1-

+0.4

Mean errorper party

1.51.32.31.00.31.01.32.01.01.50.31.51.6

1.3

Mean erroron gap betweenfirst and second

party

- 4- 3- 6-+ 1-

+4- 6+2+5- 1+5- 3

3.1

Coatis

I's.

|^

~*s.S'

1Sa '

3'

rs

Note: a A 'forecast' poll is defined for 1959-87 as any published on polling day, and for earlier elections as any described as such.Fieldwork dates vary slightly, but for elections since February 1974 will almost always be the day preceding the election. |Source: Ivor Crewe, 'Sources of British elections: problems of design, analysis and response', paper presented to the Round Table onPolitical Campaigns and Electoral Surveys, Madrid, October 1983.

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014

Pollingorganization

ORCforlTN

Gallup for BBC7979 election result (GB)

Harris for ITNGallup for BBC7983 election result (GB)

Harris for ITNGallup for BBC

1987 election result (GB)

Dates offieldwork

3 May 1979

2-3 May 19793 May 1979

9 June 19838-9 June 19839 June 1983

June 1987

11 June 1987

Table 4 Ele

Sample design

3,702 randomexit2,435 quota

10,000 exit4,141 quota

4,500 exit4,806 quota

Source: Crewe, 'Sources of British elections'.

ction-day surveys, 1979-87

Con

44

4545

(44.9)

424443

(43.4)

41.840.4

(43.3)

Forecast

Lab

38

3838

(37.7)

282928

(28.2)

32.035.4

(37.5;

Lib/SDP

14

1314

(14.1)

262626

(25.9)

23.422.5

(23.1)

Others

4

33

(3.3)

423

(2.5)

2.81.7

(2.1)

Con leadover Lab

6

77

(7.2)

141575

(15.2)

9.85.0

(11.8)

Meanerror

per party

0.5

0.4

0.50.8

0.752.0

Erroron gap

1

1

2.06.8

Maximumerror

1

1

11

1.53.9

o

Articles: R

esearch Methods, D

ala and \

i1'

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


ITN's national exit poll has given excellent results in the 1979-87 elections. Thepoll-of-polls has given reasonable results for predictions of individual party shares,but predictions of the lead of the first over the second party have been rather poor(except in 1979), and this has led to poor seat forecasts.

DiscussionProperly designed national exit polls are best for vote-share projection. The case forpreferring exit polls in marginal seats for seat projection is not proven, as ITN's seatprojections have rarely been particularly good. It is not known if this results fromtheir method of conversion of shares into seats - the BBC might have been ableto do rather better with the same information. However, ITN has never given aparticularly bad forecast of seats and can claim that it has always got the basic storyright. At the very least, a forecast based on an exit poll in carefully selected pollingstations in marginal seats seems to have been an insurance against the sort ofproblems which the BBC experienced in October 1974 and in 1987.

Results-based predictions

A variety of statistical methods has been used for the production of forecasts usingthe actual results as they become available. In the early 1950s, the method used wasbased on the cube law, which asserted that the number of constituencies won by aparty was proportional to the cube of its share of the vote." This law was based onthe empirical fact that the shares of the vote obtained by a particular party followeda normal distribution with a particular shape. This method worked well in thepredominantly two-party system then existing, but suffered from the deficiency thatit did not give individual seat projections. This deficiency was met by the use ofswing, which has provided the basis for most prediction algorithms since 1951.The mean swing from the previous election for the sample of results declared iscalculated and then applied to each undeclared constituency in turn, as describedabove. This method has been extended in various ways, primarily by groupingconstituencies (for example, by region and type of political contest) so that adjust-ments are made to the mean swing for each type.12 The swing-based methodsworked well in the elections fought by the two predominant major parties in the1960s, but have been found to be seriously deficient in recent elections where therehave been stronger minor parties.

A more sophisticated method used by the BBC since 1974, based on ridge regres-sion, has given better predictions than the swing-based methods.13 Constituenciesare first classified into two main types:

1. Two-party seats - where the contest is considered to be between Conservativeand Labour only.

2. Three-party seats - where at least one party, other than Conservative or Labour,is considered to have a chance of winning.

The method has three stages:

1. Multiple ridge regression on the set of declared results is performed to predicteither the Con-Lab swing (for two-party seats) or the changes in the shares ofthe electorate for all significant parties (for three-party seats), using predictorvariables such as the parties' shares at the previous election, regional dummies,

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


incumbency and type of political contest. The ridge modification has the impor-tant practical advantage that problems of multi-collinearity are dealt with auto-matically; these have special importance in the early stages of the prediction. Italso performs better for estimation purposes than least squares.14

2. The regression coefficients are then applied to form predictions of changes invoting behaviour in the undeclared seats. These are used to calculate posteriorprobabilities of each party winning. The methods used depend on the type ofundeclared seat, as follows:

(a) Two-party. Here a regression model is used to predict the swing, S, fromLabour to Conservative in each undeclared seat, in order to give a predictedConservative share of the two-party vote C + S, where C is the Conservativeshare of the two-party vote in the previous election.

The probability of a Conservative win is then estimated as

Pr(Con) = 0{(C + S- 0.5)1 a)}

where 0 is the cumulative standard normal distribution function, 0.5 is thetie value and a is the prediction standard error for the seat. The probabilityof a Labour win is then

Pr(Lab) = 1 - Pr(Con)0

All other parties have a zero probability of winning.

(b) Three-Party. The regression coefficients are used to predict the party sharesin the undeclared seats. For the top three predicted parties in a seat the vectorof predicted shares of the electorate Zi = (Z|, Zi, Zj) is assumed to betrivariate normal with a variance-convariance structure V, obtained fromStage 1.

The posterior probability Pr(Pj) that party i would win the constituency isthen calculated as:

Pr[P,] = Pr{[Z, - Z,.) > 0] U [(Z,- - Zr) > 0]}

i = i' = i" = 1, 2, 3 where (Z,- — Z,--, Z, — Z,") is bivariate normal obtaineddirectly from Z, and V. The calculation thus involves a bivariate normalintegration. Any remaining parties are assumed to have zero probabilityof a win.

In the elections in the 1970s as many as 50 per cent of the seats were treatedas two-party seats. In the 1980s, with the rise of the Alliance, most seats weretreated as three-party seats. There is no evidence that the proportion of seatstreated as three-party introduced any significant bias into the prediction.

3. The forecast of the number of undeclared seats won by each party is thenobtained by summing the probabilities over the undeclared seats. Confidenceintervals are calculated for party seat totals. For presentation purposes theuncertainty in the forecasts is given in terms of probabilities (or odds) of variousoutcomes, such as that Conservatives would achieve an overall majority in seats.

The same procedures were used for the prior forecast. For three-party seats thepredicted shares of the vote Z,- are obtained from the prior information, and V is a

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


prior variance-covariance matrix estimated from previous election results. Similarly,for two-party seats, predicted swings and the prediction standard errors are obtainedfrom prior information. The final forecast is actually a weighted function of the priorand posterior probabilities, where the weight given to the prior is chosen accordingto the strength of belief in the value of the prior information. The prior probabilitieswere also usually given a higher weight for types of constituencies where relativelyfew results are expected in the early declarations. The weighting function used is:

Predicted probability = Prior * W + posterior * {1 - W)

with W = P/(P + N), where N is the number of declared results on which theposterior is based, and P is the number of seats for which the prior is consideredequivalent. Typically, P is set in the range 1-5.

In any particular election, there are some constituencies where special localfactors mean that it would be unwise either to use these to forecast other seats or toforecast them from more normal seats. Typical reasons are change of winning partyat a by-election, and former members of a party who are standing against an officialcandidate. These seats are identified a priori and fixed prior probabilities are usedthroughout for them, although the prediction program has the facility for manualmodification of these priors during the forecasting process if the priors seem out ofline. Typically about fifty seats are put in this category; many of them have verypredictable outcomes and there is no evidence that these fixed priors have intro-duced any significant bias into the forecasts.

The BBC's performanceTable 5 shows the errors in the BBC results-based forecast for the majority of thewinning party over the second party, by number of results declared, for generalelections 1974-87.15

The prediction traces show that the forecasts have almost always been within tenseats after fifty results (an error of twenty in the projected majority), after which theforecast becomes very stable. After half the results have been declared the forecastsare within a few seats of the final outcome. The worst forecasts were obtained in the1987 election. Here the Gallup quota poll-based prior was given a weight equivalentto five actual results, and it took some time before its effect was purged from theforecast (a prior weight of zero would have produced predictions always within tenseats of the final outcome!). The ridge regression method always gave betterforecasts than the swingometer-based methods, although sometimes the differencewas very small.

Table 5 BBC results-based forecasts for general elections, 1974-87: errors inmajority of the winning party over the second party

Election

1974 (Feb)1974 (Oct)197919831987

0

62842

70

5

213133

39

Results declared

10

161527

46

50

1270

1020

100

489

1620

300

24365

Actualmajority

44270

188147

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


Adjustments for the changing political situation

Since 1970 various modifications have been made to take account of changes in thepolitical situation and the type of election.

Significant minor partiesThe BBC method depends on identifying the top three parties, usually Conserva-tive, Labour and Liberals (Alliance since 1983), so that if other parties areconsidered to have a non-zero chance of winning in a significant number of seats thealgorithm has to be accommodated to deal with them. In this situation the procedureadopted was to do separate regressions for the significant minor parties and thenchoose the top three parties in any particular seat for the calculation of posteriorprobabilities. Since the October 1974 election, the SNP in the 72 Scottish seats hasbeen treated in this way. The Plaid Cymru party, standing in the 36 Welsh seats, hashad a real chance in at most 3 seats; for these 3, fixed prior probabilities have beenused throughout.

The SDP and the AllianceThe 1983 election saw the emergence of a new party, the Social Democratic Party, inwhich 28 former Labour MPs defected. The SDP and the Liberals formed analliance in which all but 3 of the British seats were shared out between the twoparties; complete agreement was reached in the 1987 general election. Thus theprediction algorithm had to be modified for this new situation. The two parties couldhave been treated as separate parties, each with its own regression predicting changein share of the electorate from a previous Liberal base; alternatively, the Alliancecould have been treated as one party, regressing the change in share for the Alliancevote (SDP or Liberal according to the seat) from the previous Liberal base. Thelatter approach was adopted because the indications were that the electorate viewedthe Alliance as one party. However, some further modifications were necessitated bythis change. First, major changes were required to the program to treat the Allianceas one party for predicting shares of the vote, but as separate parties for predictingseats. Second, a dummy variable indicating whether the Alliance candidate was aLiberal or a Social Democrat was introduced into the Alliance regression to allow forany differential attraction of the two parties. The many seats in the 1983 electionwhere former Labour MPs stood as SDP candidates needed special treatment, withthe use of a 'Defector' dummy variable. In the event the predictions for the Allianceparties were very good in both elections, vindicating the decision to treat theAlliance in the way described. (With the demise of the SDP in 1990, thesemodifications had to be undone for the 1992 election!)

European elections 1979,1984 and 1989The UK elections to the European Parliament are based on aggregations of theWestminster constituencies. The major parties stood in all constituencies and thenationalist parties stood in Scotland and Wales. In 1979 and 1984 there were nosignificant minor parties, but a new feature in 1989 was the increased strength ofthe Green party, which stood in all British seats and averaged 15 per cent of thevote, although it had no chance of winning seats under the plurality system. TheWesminster prediction algorithm was easily modified to cope with the Europeanelections. The approach adopted was to aggregate votes for the Westminster seats at

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


the previous general election in order to form notional Euro-constituency results,and then to predict changes at the actual European election from this base. Despitethe Green Party's expected good performance in 1989 it was decided not to do aseparate regression for them; this decision was vindicated by the good performanceof the method on election night.l5

For all three European elections the results-based forecasts were extremelyaccurate, being within two seats of the final result from the initial stages. The verylarge size of the Euro-constituencies, which smooth out individual variations in thecomponent Westminster seats, seems to bring a large degree of uniformity in theirelectoral behaviour.

However, prior forecasts are more problematic in European elections, as there isoften very little prior information available in the form of opinion polls. This isbecause the elections seem to be of little interest to the media, and also because of thedifficulties of identifying likely voters in quota polls when only about 30-40 per centof electors will actually vote on election day. The BBC again used a variety of sourcesfor the prior forecast. These gave excellent prior forecasts in 1979 and 1984, but the1989 prior, based on an NOP exit poll in 100 polling stations throughout GreatBritain, overestimated the Labour lead over Conservatives by 7 per cent, giving anerror of 6 seats in the projected Labour lead over the Conservatives.

Boundary changesIn the United Kingdom, periodic boundary changes to the constituencies are carriedout. The last major changes were introduced in 1983 when the number of West-minster constituencies was increased from 635 to 650; one third of the constituencieshad major changes, and another third had minor changes. Since UK electionprediction is founded on predicting changes from the previous election, this pre-sents a major challenge to the forecaster. The solution adopted has been to form a setof notional results for the new constituencies - that is, what the votes would havebeen if the previous election had been fought on the new boundaries. This involvesa detailed examination of local election results at the time of the previous election,aggregating wards to the new constituencies.17

Discussion

The complex method for election-night forecasting has worked better than theswingometer-based methods, and has proved easy to adapt for changes in thepolitical situation and different types of plurality election. For each election therelevant data base of past results has to be established, and the parameters for theprogram, such as the definition of two- and three-party seats, have to be set. Thecovariates for the regressions then need to be chosen; these will include basic variatessuch as party's previous shares of the electorate and regional dummies. Election-specific covariates concerned with political issues salient to the electorate at the timeof the election need also to be specified. A good example is the constituencyproportion of electors who live in council housing, which worked well as a predictorof swing in the 1983 and 1987 elections, given the Conservatives' policy of sellingcouncil houses. Finally, election-specific deviant seats must be identified and theirprior probabilities fixed.

As the method has worked well for the results-based forecasting, there has notbeen a need for detailed post-mortem exercises to investigate fine tuning or to

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


improve the prediction system. The BBC used essentially the same system as set upin 1974 throughout the next fifteen years. Of course if one were to start again onewould consider other approaches. One obvious possibility would be to use log-linearmodels, where the proportion of the constituency vote going to a party is assumed tohave an expectation which is log-linear in the set of covariates. In fact a logisticmodel was used by the BBC for the prediction of the 1975 Referendum on entry tothe European Common Market, where the voting alternatives were either for oragainst entry.18 Ideally a log-linear model treating the vote response as a three-valued one to include abstention would have been used, but the software ran tooslowly with this model using the hardware that was available then. The requirementfor fast updating of prediction in the election-night context militates against the useof complex iterative statistical models.

The next general election was in April 1992 and the ridge regression method againproved a suitable technique for its prediction, although the BBC will in the futurehave to give some attention to improving the sources of information on which theprior forecast is based. In the 1992 general election the United Kingdom returned toa predominantly Conservative versus Labour contest, and the final outcome wasmuch closer than in the last three general elections. This places a premium onaccurate predictions. The prediction algorithm had to take account of several newfeatures during the election. These include the enfranchisement of a number ofBritons living abroad and the splitting of the Milton Keynes constituency into twoseats. Account was also taken of the political effects of the introduction of thecommunity charge (poll tax) by the 1987-92 Conservative government.

Media aspects

The main objective of this article has been to give a statistical treatment of short-term election forecasting in the United Kingdom. However, the primary use ofelection-night forecasting is by the media, and this has implications for the way suchforecasts are both used and presented. There are also constraints on what can bedone because of the hectic election-night context. Some implications of the contextare as follows:

1. There is a high premium on results-based forecasts only when the election isclose. There is always a very high premium on the prior forecast, which plays akey role at the start of the election night coverage, filling the gap between the endof voting and the declaration of results. There is a strong demand for frequentpredictions in the very early stages, when the forcasts are likely to fluctuate a lot.

2. There is a reluctance to present the uncertainty associated with the forecasts.This could be done in various ways, such as confidence intervals or probabilitydistributions for seat totals, probabilities or odds for parties winning in eachconstituency, and odds on particular global outcomes, such as who will bemajority party. This reluctance has several explanations. One is the discretenature of maps which are used to present predicted outcomes in individual seats;there is also a requirement to use bold graphics (Labour seats all in red,Conservatives all blue), without shading to indicate uncertain outcomes. Asecond explanation is a deterministic view commonly held by presenters ('if theaverage swing to Labour is X then all seats where Labour is less than X/2 behind

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


will be won by Labour'). Most importantly, there is a commonly held view thatthe viewer will not understand betting odds or probabilities, let alone probabilitydistributions.

3. Forecasts are now presented in increasingly sophisticated dynamic graphicaldisplays, and the on-line prediction system is a small component in the election-night computing system. Technical requirements impose constraints on thefrequency with which forecasts can be updated and on the sophistication ofthe statistical method used. This is because the prediction sub-system must bothcompete and integrate with other components of the election-night computingsystem. In particular, a complex interface is required to pass individual seatpredictions to the graphical display sub-system and to make sure that the twosub-systems are in phase.

As a result of these media aspects, Payne's Eleven Laws of media electionforecasts have been formulated:

1. Few people understand that UK forecasting is more difficult than that in mostother electoral systems.

2. Prior forecasts are at the mercy of the pollsters.3. Good forecasts are soon forgotten.4. Bad forecasts are forever remembered and brought to mind (even by prime

ministers).5. You are only as good (or as bad) as your last forecast.6. Only statisticians can understand why ridge regression should always beat the

swingometer.7. There is a positive correlation between the proportion of the TV audience

watching your channel and the error in your forecast.8. Hyping good forecasts always results in a red face for the subsequent one.9. Expressing uncertainty about forecasts is prohibited even though the informa-

tion on which they are based is allowed to be subject to error.10. Graphics (and their presenters) rule!11. Relative performance (that is, BBC versus ITN) rather than absolute perfor-

mance is what matters.

Notes

1. See, for example: David Sanders, Hugh Ward and David Marsh, 'Governmentpopularity and the Falklands War: a reassessment', British Journal of PoliticalScience, 17, 1987, pp. 281-313; Paul Whiteley, 'Electoral forecasting from poll data:the British case', British Journal of Political Science, 9, 1979, pp. 219-36.

2. Philip Brown and Clive Payne, 'Election night forecasting (with discussion)',Journal of the Royal Statistical Society (Series A), 138, 1975, pp. 463-98.

3. For forecasting purposes the Ulster MPs are regarded as representing minor partiesindependent of the mainland GB parties. The voting system used in Ulster also varieswith the type of election - a plurality system in the Westminster election and a singletransferable vote system in the European election. Forecasting the Ulster Europeanseats is not covered here. See Fred Smith, 'Election forecasting: transferable votesystems', in S. Kotz and N. Johnston, eds, Encyclopedia of Statistical Sciences (NewYork: Wiley, 1982).

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014


4. A description of the problems of using local election results to estimate the nationalstanding of the parties is given in John Curtice and Clive Payne, 'Local elections asnational referendums in Great Britain, Electoral Studies, 10, 1, 1991, pp. 3-17.

5. Descriptions of election forecasting in a number of countries are given in HughMorton, ed., 'Election night forecasting', The New Zealand Statistician, 25, 1990,pp. 36-84.

6. Gordon Reece, 'How people vote' (unpublished manuscript, 1982).7. Clive Payne, Philip Brown and Vincent Hannah, 'By-election exit polls', Electoral

Studies, 5, 1986, pp. 227-87.8. Full details are given in Brown and Payne, 'Election night forecasting'.9. A description of the ITN exit polls conducted for the 1987 general election is given in

Robert Waller, 'The Harris/ITN exit poll 11 June 1987', Journal of the MarketResearch Society, 29, 1987, pp. 419-28.

10. Full details are given in Philip Brown and Clive Payne, 'Forecasting the 1983 Britishgeneral election', The Statistician, 33, 1984, pp. 217-28.

11. See Maurice Kendall and Alan Stuart, 'The law of cubic proportions in electionresults', British Journal of Sociology, 1, 1950, pp. 183-96.

12. See David Milledge and Michael Mills, 'Forecasting election results', ComputerJournal, 2, 1960, pp. 195-8.

13. See Brown and Payne, 'Election night forecasting', for a detailed description of themethod and justification for the use of ridge regression.

14. The ridge regression technique is used to deal with the potential problem of highintercorrelations between predictor variables in a regression. The regression designmatrix is modified so that the intercorrelations are reduced by a constant fractionbefore regression calculations are done.

15. Detailed prediction traces for the two 1974 and the 1983 elections are set out inBrown and Payne, 'Election night forecasting', and Brown and Payne, 'Forecastingthe 1983 British general election'.

16. Full details of the 1979 prediction are given in Clive Payne and Philip Brown,'Forecasting the British election to the European Parliament', British Journal ofPolitical Science, 11, 1981, pp. 235-48.

17. For the last boundary change an agreed set of notional results was estimated - see TheBBC/ITN Guide to the New Parliamentary Constituencies (Chichester: ParliamentaryResearch Services, 1983).

18. For this forecast the GLIM software system was used successfully - see Robert Bakerand John Nelder, The Glim System: Release 3 (Oxford: Numerical AlgorithmsGroup, 1978).

Dow

nloa

ded

by [

UQ

Lib

rary

] at

06:

59 0

4 N

ovem

ber

2014

Documents

Statistical methods for election forecasting in the United Kingdom 1970–90