The Best Writing on Mathematics 2017
The article, Great Expectations: the past, present and future of prediction, was originally published in the April 2016 issue of the Royal Statistical Society’s Significance magazine and was the cover story that month.
The Best Writing on Mathematics 2017, published by Princeton University Press
I didn’t know all that much about statistical forecasting and prediction methods before I started the research. Fortunately, my interviewees — Rob Hyndman, Leighton Vaughan Williams, David Orrell, and Walter A. Friedman — were very generous with their time and patient explanations.
Statistical forecasting techniques are used for all manner of everyday business tasks, from gauging electricity demand to setting stock levels for retailers. But it most often hits the headlines when it’s used to predict the outcome of elections.
The results of the Brexit and US Presidential elections confounded forecasters
After I wrote the article, two elections produced outcomes — Brexit and Trump — that not one expert correctly predicted. At least as far as I know.
I was as surprised as anyone, but I probably shouldn’t have been. One of the last quotes in my article was by Leighton Vaughan Williams, an expert in prediction markets. He made a pertinent point that probabilities are far from certainties.
“In 2008, Hillary Clinton had a 20 per cent chance of winning the New Hampshire primary and she won it. People said the prediction markets had got it all wrong. But as any statistician would know, what they’re saying is that one time in five it’s going to happen.”