Nate Silver is a Wunderkind. Not yet 35, Silver has already developed the leading statistical tool for assessing prospective professional baseball players, been consulted by the 2008 Obama presidential campaign to help assess the implications of opinion polls for the election’s outcome, writes a widely read New York Times blog and in 2009 was named by Time magazine as one of the 100 most influential people in the world. Along the way, and apparently in his spare time, Silver made more than $400,000 playing professional poker (before losing about a third of it and moving on). Silver’s initial foray into professional poker turned a $100 initial stake into $15,000, which was apparently enough to convince Silver to quit his day job, which has been to the benefit of the rest of us. Silver adds to this impressive list of accomplishments with the publication next week of his first book The Signal and the Noise: Why Most Predictions Fail – But Some Don't (Penguin USA).
This generally well-researched and ambitious book covers a lot of ground. It describes Silver’s evolution as a Fox (to use Philip Tetlock’s terminology favored by Silver) who is “tolerant of nuance, uncertainty, complexity, and dissenting opinion.” Silver discusses prediction in sports, geosciences, economics, politics and health. He clearly has done lots of reading in the academic literature. In most of these chapters he has tracked down and spoken to the characters that show up in the book -- a nice touch for a work that combines analysis with a bit of reporting. I’m not one of those characters, but I did engage in a long phone conversation with Silver while he was researching the book to discuss various aspects of prediction. In our discussion, I was impressed by his thoroughness and attention to detail, and have been looking forward to the final product.
Silver’s book is at its best when he writes about the things he knows best. That is to say, when he is writing about sports, political predictions and poker, Silver’s book is original and insightful. However, when he writes about weather, climate, earthquakes, the stock market and terrorism, the book doesn’t quite reach this high standard, and sometimes the discussion of these complex topics seems a bit cursory or hurried, which is decidedly contrary to Silver’s approach to analysis for which he has become well known.
Consequently, there is a brilliant book here, but it is a subset of the volume that was produced. The brilliant book shares Silver’s personal experiences in building the PECOTA system to evaluate baseball prospects a la Moneyball, his remarkable successes (and lessons learned) as a professional poker player and his experiences developing the FiveThirtyEight election forecasting system now hosted at the New York Times. I got my PhD in political science, and have long been a critic of the silly election forecasting methods proposed by academics. Silver’s work is much more solid than that of the academics, and probably the best of its kind. Silver also writes insightfully about chess, conditional probabilities (and the importance of Bayes theorem) and the difficulty of distinguishing predictive skill from pure chance.
The book’s thirteen chapters appear to have been written such that with few edits each might appear as stand-alone essays in places like the New York Times magazine (indeed, the chapter on weather forecasting recently was the basis for exactly such an essay). This makes the book easy to read but also makes it difficult for Silver to sustain a consistent argument throughout. For instance, after introducing the reader early on to Frank Knight’s definitions of “risk” (where you can readily quantify probabilities, such as in the roll of a die) and “uncertainty” (where you can’t readily quantify probabilities, such as the expected price of copper in 2057), Silver then leaves this distinction behind and subsequently throughout the book uses the word “uncertainty” to mean what Knight called “risk.”