Superforecasting: The Art and Science of Prediction
The average human forecaster is no better than a monkey throwing darts. Evolutionarily, we’ve developed a simple three-dial system for making decision: Do I see a huge dangerous predator? Yes, run. Maybe, stay alert / run. No, relax. Whenever we do venture into predictions, it’s with a vague vocabulary filled with rubbery words: may, soon, highly likely, unlikely, .. The statement “Greece may default in the near future” really doesn’t mean anything: may is completely uncertain, and the near future could be a year, decade or tomorrow depending on who you ask. If the medium for predictions is ambiguous English, how are we supposed to evaluate and therethrough make anyone accountable for their predictions?
However, aggregate a large enough sample of average dart throwing humans and you’ll get a much more useful result. If you have enough people guessing the weight of an ox, the average will run quite close to reality. People all come with different backgrounds, biases and bits of information that they boil down to a single number. Combine enough of those numbers, and a remarkable amount of information is captured in the final average. This is exactly how a stock market works, oodles of traders push new information into the stock.
This strategy can be applied just as well to your own predictions. Instead of thinking twice to take another angle, think 12 times, even better 100 times—become your own supplier of diverse views, and aggregate these views. As new information submerges, update your predictions, but only move them little at a time. Super forecasters think in probabilities, not three dial notches, and they’re excellent distilling facts into numbers. What sets them apart is their ability to see through confirmation bias, and consider as many angles as they can possibly find. They’re experts at bias awareness and balance. They grunt at the smell of false dichotomy. They don’t substitute questions for whether they’d do it, but absorb the full context to understand whether the person the prediction question is about will do it. Their growth mindset is what makes this possible, they refuse to believe that everything cannot be learned through hard work—versus a fixed mindset, where you think your only job is to reveal skills you were born with.
When an effective forecaster look at a new problem, they start with a baseline. It’s easy to get primed by an inside perspective here, but a super forecaster always start from outside. Vietnam and China border dispute? Look at history and see how often it’s happened in the past, rather than compiling only from information available right this week which is subject to the availability bias. They don’t go incredibly deep into one branch of an issue, but rather develop a nuanced, broad perspective. Ferme predictions are a weapon of choice, breaking a problem into many that can each have a reasonable probability associated with it. They know that the aggregation will result in a reasonable prediction. After developing an outside perspective, they’ll dig inside and come back up merging the two into their final prediction.
Super forecasters do remarkably in groups, effectively aggregation of aggregations. This is how Nate Silver works too, and 2-level (or even higher) of aggregating can be remarkably effective. They’re aware of groupthink: that consensus should not be confused with having found the best possible solution. Friendliness may not spur enough diverse opinions. Chaos is an accepted reality among them, and they understand that the further into the future we venture the more we invite chaos and unpredictability. Taleb, the author and Kahnemann all agree it’s unreasonable to predict anything 10 years out. Predictions excel in the 3-18 month range, as longer it becomes subject to the butterfly effect and it becomes more like a seasonal weather prediction than anything. When the book discusses chaos, it takes a detour into Prussian war strategy where localized decision power was always maximized. The higher ups would compose the overall plan, but the field generals would make the final decisions. The vision was shared, but the execution was up to the people with the most information. The famous quote here being “plans don’t survive contact with the enemy”.
The author does a good job throughout of applying what he’s preaching, questioning what he’s saying and arguing against it.