Nate Silver called every state correctly in the last presidential race, and was wrong about only one in 2008. In 2012 he predicted Obama’s total of the popular vote within one tenth of a percent of the actual figure. His powers of prediction seemed uncanny. In his early and sustained prediction of an Obama victory, he was ahead of most polling organizations and my fellow political scientists. But buyers of his book, The Signal and the Noise, now a deserved best seller, may be in for something of a surprise. There’s only a short chapter on predicting elections, briefer than ones on baseball, weather, and chess. In fact, he’s written a serious treatise about the craft of prediction—without academic mathematics—cheerily aimed at lay readers. Silver’s coverage is polymathic, ranging from poker and earthquakes to climate change and terrorism.
We learn that while more statistics per capita are collected for baseball than perhaps any other human activity, seasoned scouts still surpass algorithms in predicting the performance of players. Since poker depends as much on luck as on skill, professionals make a living by having well-heeled amateurs at the table. The lesson from a long chapter on earthquakes is that while we’re good at measuring them, they’re “not really predictable at all.” Much the same caution holds for economists, whose forecasts of next year’s growth are seldom correct. Their models may be elegant, Silver says, but “their raw data isn’t much good.”
The most striking success has been in forecasting where hurricanes will hit. Over the last twenty-five years, the ability to pinpoint landfalls has increased twelvefold. At the same time, Silver says, newscasts purposely overpredict rain, since they know their listeners will be grateful when they find they don’t need umbrellas. While he doesn’t dismiss “highly mathematical and data-driven techniques,” he cautions climate modelers not to give out precise changes in temperature and ocean levels. He tells of attending a conference on terrorism at which a Coca-Cola marketing executive and a dating service consultant were asked for hints on how to identify suicide bombers.
Much is made of ours being an era of Big Data. Silver passes on an estimate from IBM that 2.5 quintillion (that’s seventeen zeros) new bytes (sequences of eight binary digits that each encode a single character of text in a computer) of data are being created every day, representing everything from the brand of toothpaste you bought yesterday to your location when you called a friend this morning. Such information can be put together to fashion personal profiles, which Amazon and Google are already doing in order to target advertisements more accurately. Obama’s tech-savvy workers did something similar, notably in identifying voters who needed extra prompting…
This is exclusive content for subscribers only – subscribe at this low introductory rate for immediate access!
Unlock this article, and thousands more from our complete 55+ year archive, by subscribing at the low introductory rate of just $1 an issue — that’s 10 digital issues plus six months of full archive access plus the NYR App for just $10.