- Combining multiple perspectives, viewpoints, theories and ways of analyzing the problem. Combining analytical and theoretical approaches with data-driven extrapolations. Combining different theoretical frameworks and different data sources, including contradictory ones. Each will have its flaws and oversights and by combining them, you have the greatest chance of not missing anything important. This is very similar to the wisdom-of-crowd effect of forecast aggregation, just done within one person. Not being overly attached to one theory or worldview (e.g. examining everything from an economic point of view, with a liberal or conservative bias, overly theoretically or overly empirically etc.). Tetlock calls this being a fox rather than a hedgehog.
- Using the outside view, often as a starting point. For example when trying to forecast whether there will be a conflict in some region during a period of time, you start by just extrapolating historical data about how often conflicts in this area tend to occur, before even exploring specific reasons or mechanism why it should or should not occur at this particular time. This is often called reference class forecasting (i.e. looking at data about other events or subjects roughly similar to the one you are asked about) and using base rates. Another example may be when forecasting the success of a specific startup company, to first simply look at the average success rate of startups in this general area, prior to even examining their product, strategy, funding etc. It’s mostly about extrapolating very general trends. Of course then you need to adjust the outside-view forecast based on the specifics of the actual question subject.
- Continuously updating your beliefs and estimates as you encounter new evidence and data. Even continually updating the way in which you do forecasting, the tools and ways of thinking you use, learning from what works and what does not. Being flexible and actively open-minded, having a growth mindset and a desire to always learn new things. Being introspective and self-critical. Be aware of your own biases and try to correct them.
- Having nuanced and probabilistic thinking. Not seeing things as black and white, sure or impossible. Operating with varying degrees of uncertainty. Being humble in some ways, accepting that reality is complex, there are many unknown unknowns and your models cannot capture everything.
- Using fermi-izing, i.e. breaking the problem into smaller subproblems that are easier to analyze. This comes from a technique known as Fermi estimation. For example, when asked whether there will be a larger lion population in Africa next year than this year, a forecaster can ask the following: (1) How has the total world lion population evolved in recent years? (2) How many lions are there in Africa? (3) What country has the most lions and are there any plans to significantly reduce them? (4) How many cubs does one lioness have on average?
- Doing the hard work. Sometimes forecasting requires digging through a lot of data, reading a lot of articles, considering and integrating a lot of contradictory points. Conscientiousness helps a lot.
What makes a
good forecaster?
In his research, Philip Tetlock tried to identify common traits, habits and ways of thinking shared by the best forecasters. Here are some of them, in no particular order.