Opinion Poll Accuracy

'As every politician know the pollsters got it wrong in 1992, so why should we trust them now?' Bill Bush, in The BBC News General Election Guide Harper Collins, 1997.

ICM gets it right

The election result demonstrated that the final opinion polls were more than 8 % adrift of reality. An enquiry by the Market Research Society put the errors down to three main factors - a late swing, wrong quotas and a high refusal rate. The late swing was thought to account for only 1-2 % of the error. Wrong sample quotas accounted for a further 2 % error. This left a massive 4-5 % which was accounted for by 'shy' Tory voters who had refused to answer the pollsters questions.

The polling organisations have changed their methods of calculation since 1992. Quotas have been adjusted to include less voters from Labour-inclined groups such as manual workers and council tenants and more from Tory-inclined groups like white-collar workers and owner-occupiers.

The inaccuracies caused by 'shy' Tory voters who refuse to respond to pollsters questions have been dealt with in different ways by the various organisations. For example, ICM assumes that 60 per cent of those who say they don't know which way they will vote will return to the party they voted for at the previous election, while NOP allocate the don't knows according to which party they identify most closely with or which has the best policies on the economy.

So just how accurately did the pollsters predict the general election result?

ICM provided an extremely accurate prediction of the 1997 general election, with a 'cumulative error' of only 1.8 %. They predicted the Lib Dem vote correctly, and were only 0.4 % out for Labour, 0.5 % out for the Conservatives and 0.9 % out for the SNP.

NOP did much worse with a 'cumulative error' of 10.0 %. They were 1.0 % out for the Lib Dems, 2.1 % out for the SNP, 3.4 % out for Labour and 3.5 % out for the Conservatives.

The worst prediction from a professional polling organisation was the System 3 eve of election poll with a 'cumulative error' of 16.0 %. They were 3.5 % out for the Conservatives, 4.0 % out for the Lib Dems, 4.1 % out for the SNP and 4.4 % out for Labour. To be fair, System 3 did state that they expected the Tories to poll more than their poll rating due to their 'traditionally higher turnout'. In fact, System 3's previous weekly poll (23 rd April 1997: Lab 47 %, SNP 24 %, Con 15 % Lib Dem 12 %) was much more accurate with a 'cumulative error' of only 7.0 %.

The least accurate poll was the Scotland 500 poll for STV/Grampian, with a 'cumulative error' of 27.2 %. However, as the Scotland 500 poll sample was a studio audience and not a true cross section of the population, its failure to predict the result correctly can be excused somewhat more easily. The 500 poll was 4.5 % out for the Conservatives, 6.0 % out for the Lib Dems, 8.1 % out for the SNP and 8.6 % out for Labour.

Therefore, ICM takes the gold medal for the most accurate prediction of the 1997 general election result. NOP and System 3, on the other hand, still appear to have some work to do.