Revealed Preferences and the Death of Polling

So the pollsters got it wrong again. After the general election last year and then Brexit, it is perhaps not surprising. What is surprising is just how wrong they were. The real problem is the enormous confidence with which they pronounced that Clinton would win.

The Princeton Election Consortium was probably top of the class, stating that Clinton had a 98 to 99 per cent chance of winning. Even the top Bayesian statistician, Nate Silver, who shot to fame by calling all 50 states correctly in 2012, gave Hillary a 71.4 per cent probability of victory.

Economists have been suspicious of opinions elicited by surveys for a long time. A fundamental concept in economic theory is that of “revealed preference”. The idea goes back much further than Adam Smith, the 18^th century founder of modern economics. In the Bible, we find the phrase “by their deeds, ye shall know them”. In other words, it is not what people say what matters, it is what they do. If someone says repeatedly that he prefers Pepsi to Coke, but never buys Pepsi and always buys Coke, we can reasonably infer that, despite his words, he does in fact prefer Coke. His actions reveal his preference.

Readers above a certain age will recall the 1980s. Then, pollster after pollster reported that public opinion was firmly in favour of both more public spending, and higher taxes to pay for it. Yet in election after election, voters just as firmly returned Mrs Thatcher and the Conservatives to power. They revealed a preference for lower spending and lower taxes.

A great deal of environmental policy is guided by hypothetical questions in surveys of what people would be willing to pay to, say, preserve a species of newt or prevent an oil spillage. This approach even has its own name, that of “contingent valuation”. Peter Diamond is an MIT economist who has won the Nobel Prize. Jerry Hausman, also of MIT, might very well get one. Referring to a paper they co-authored in the early 1990s, in 2012 Hausman wrote “at the time Peter’s view was that contingent valuation was hopeless. I was merely dubious. But 20 years later, after millions of dollars of government funded research, I have concluded that Peter was correct”.

A fundamental problem is that people overstate how much they would be willing to pay in such surveys, compared to how much they will pay when they really have to – just like the British electorate in the 1980s.

A great deal of expertise has been built up over the years in how to put together carefully constructed surveys to find out what voters and consumers think. But their useful life is at an end. Instead, social media conversations have the potential to discover what people really do prefer. For all their chaotic and often incoherent nature, these unstructured conversations can reveal what people really are thinking and doing. Economists, with their concept of revealed preference, need to make common cause with computer scientists.