Superforecasting is one of those books that takes a subject most of us think we understand and demonstrates how casually we often approach it. Philip E. Tetlock and Dan Gardner examine why some people are consistently better than others at predicting future events and, more importantly, whether the habits that make them successful can be learned. Their answer is broadly optimistic. Good forecasting is not primarily the product of extraordinary intelligence, secret information or inspired intuition. It is more often the result of disciplined thinking, careful use of evidence and a willingness to change one’s mind.

The book grew out of Tetlock’s long running research into political judgement and, particularly, the Good Judgment Project, in which thousands of participants made forecasts about real world political, economic and international events. The striking result was that a relatively small group of participants repeatedly produced more accurate forecasts than the average forecaster. Tetlock calls these people “superforecasters”, although the book is careful not to turn them into some mysterious class of gifted prophets. Their success generally comes from method rather than magic.

One of the strongest arguments in the book concerns the way a forecasting problem is approached. Rather than immediately deciding what will happen, successful forecasters break large questions into smaller ones, examine relevant historical frequencies, establish a reasonable starting probability and then modify that judgement as new information becomes available. A question such as whether a government will collapse, a war will begin or an international agreement will be reached is therefore not treated as a simple yes or no proposition. It becomes a collection of smaller questions concerning political stability, previous cases, leadership, economic pressures, public support, military capability and other relevant factors.

This emphasis on probability is particularly valuable. People have a tendency to speak about the future with far greater certainty than the available evidence warrants. Tetlock and Gardner instead encourage forecasts expressed as probabilities and demonstrate why distinctions between, for example, a 55 percent and an 80 percent chance actually matter. Once predictions are expressed numerically they can also be recorded and tested. The forecaster can no longer quietly reinterpret an unsuccessful prediction after the event and claim that this was what they really meant all along.

Perhaps the most important characteristic of the successful forecasters described in the book is intellectual flexibility. They are prepared to revise their conclusions, sometimes repeatedly, when circumstances change. This is not presented as weakness or indecision. It is a recognition that forecasting concerns an uncertain future and that new evidence should alter the probability assigned to an outcome. The poor forecaster becomes attached to a theory and then interprets everything through it. The better forecaster treats the theory as provisional.

The discussion of expertise is equally interesting. Tetlock’s earlier research had already demonstrated that impressive qualifications, professional standing and media prominence do not automatically produce forecasting accuracy. Specialists may possess enormous knowledge of their subjects while still being poor at judging what will happen next. Indeed, expertise can sometimes encourage excessive confidence because the expert has developed a powerful explanatory model and becomes reluctant to abandon it. Superforecasting does not dismiss expertise, but it makes a convincing distinction between possessing knowledge and using that knowledge effectively when making predictions.

There is also an important lesson here for historians and wargamers. Historical knowledge makes outcomes appear far more inevitable than they actually were to those living through them. Once we know that an offensive failed, an empire collapsed or a particular commander made the wrong decision, it becomes remarkably easy to construct an explanation showing why the result was unavoidable. Tetlock and Gardner’s approach encourages us to reverse the process and ask what probabilities could reasonably have been assigned before the outcome was known. That is a useful discipline when designing scenarios, campaign systems and decision based games because it reminds us that historical participants were dealing with possibilities rather than completed narratives.

For a wargame designer the book has another obvious attraction. Rules are filled with forecasts disguised as mechanisms. A designer is effectively predicting how frequently troops will break, weapons will hit, commanders will react successfully or particular tactical decisions will produce favourable results. Superforecasting reinforces the value of recording those expectations before testing them. If a designer believes an event should occur about one time in four, that assumption can be stated, tested and revised rather than defended after the game because the resulting narrative happened to feel convincing.

The book is not without weaknesses. Tetlock and Gardner occasionally spend too much time celebrating the achievements of the Good Judgment Project, and some of the individual examples could have been shortened without weakening the argument. There is also a danger that readers may come away believing that almost every uncertainty can be reduced to a sufficiently precise numerical probability. Real world forecasting remains dependent upon incomplete information, changing circumstances and events that may have few useful historical comparisons. The authors recognise these limitations, but the enthusiasm generated by successful forecasting sometimes threatens to overshadow them.

There is also an inevitable question about how far the techniques demonstrated in forecasting competitions transfer to the most difficult historical and geopolitical problems. A carefully defined question with a specified deadline is easier to score than a broad prediction about the direction of a society, the consequences of a war or the long term effects of technological change. The methods remain useful, but their precision should not be mistaken for certainty.

Despite these reservations, Superforecasting is an excellent examination of how people think about uncertainty. Its greatest strength is that it does not promise a method for seeing the future. Instead, it offers a disciplined way of being slightly less wrong about it. Breaking problems into manageable parts, beginning with sensible base rates, expressing conclusions probabilistically, updating them as evidence changes and keeping records of successes and failures are deceptively simple practices, yet together they provide a powerful corrective to intuition and overconfidence.

For historians, analysts and particularly wargame designers, there is a great deal here worth considering. We spend considerable time asking what might have happened, how likely particular outcomes were and whether our models produce believable results. Tetlock and Gardner provide a useful reminder that the quality of those judgements depends not simply upon how much we know, but upon how carefully we think about what we do not know.