forecasting tournaments

Find out who the best forecasters in your organization are and whose judgement you should know at the time of high uncertainty or crisis.

Take advantage of a forecasting tournament as an efficient team building activity and improve internal coordination and collaboration mechanisms within your org by establishing a culture of clear expression of uncertainty.

Forecasting tournaments can be organized in various entities, including academic institutions, private companies, or government agencies, and can cover a wide range of topics, from financial markets to political elections or sports competitions.

How to run a forecasting tournament

Forecasting tournament is a competition in which participants submit predictions for future events, and the accuracy of their predictions is evaluated against the actual outcomes of those events

1. Decide on the amount and type of questions

A typical forecasting tournament consists of 15 to 100 questions. The fewer the questions, the greater the influence of luck on the outcome. Conversely, a larger number of questions leads to more accurate insights into who the top forecasters in your group are. However, increasing the number of questions also raises the time commitment for both organizers and participants.

A well-established approach is to post four questions per week over a six-week period. This structure helps participants stay focused on the current questions without feeling overwhelmed. Based on past experience, participants spend an average of one hour answering each question.
‍
At the start of the tournament, it's important to decide on the type of questions you'll be asking your participants. If you choose to include only yes-no questions, you can take advantage of Brier scoring, which is built into the Confido app. This will allow you to automatically view participants' scores. Learn more about the automatic scoring feature here.

However, if you prefer to use a different scoring method for yes-no questions or to include numeric range questions, you'll need to export the results as a CSV file (Confido supports CSV export) and compute the scores yourself according to whatever formula you choose.

3. Think about questions you want to ask

You can not only find out who the best forecasters in your org are but you can also find out better data about the future of your industry. Therefore you can think about what kind of topics you would like to cover in your tournament.

Of course, you can also forecast just for fun - sports, fashion, celebrities etc.

4. Develop quantified questions about the future

‍
Once you have your Confido workspace, create a room "xxx forecasting tournament." Then add all your questions. Every forecasting question should be specific, clear and resolvable. You can also read more on what does a good forecasting question look like on our website.

For every forecasting question you can set a schedule: when it opens (will be seen by participants), when it closes (when forecatsing on this question ends) and when it will be scored.
‍

‍

9. Resolve your questions

Once you know what the correct answer was, resolve the question. Participants usually prefer to answer questions that will resolve quite soon. Quick feedback will also help them improve as the tournament progresses.

In almost every tournament, at least one question cannot be resolved because of unclear definition and disputes.

10. Evaluate the accuracy of participants' forecasts

The evaluation of accuracy can be done in a variety of ways, but typically involves comparing the participants' predictions to the actual outcomes using some sort of scoring system.
‍
The Brier score is the most common scoring system in forecasting tournaments. It measures the average squared difference between the predicted probability of an event and the actual outcome. Other systems, such as log-likelihood or precision-recall, use different metrics. The Confido app includes built-in Brier scoring, allowing you to automatically generate scores for all participants. You can see set it up with a few clicks.

If you choose a different scoring method, you can still use the Confido app by exporting a CSV file of all predictions and manually scoring participants' responses based on your preferred system.
‍

Get all the benefits from running a forecasting tournament

Improved decision-making

Forecasting tournaments help participants hone their prediction skills, which can lead to better decision-making in various strategic domains within the organization.

Collective intelligence

By aggregating forecasts from diverse participants, forecasting tournaments can produce more accurate predictions than any single individual, leveraging the wisdom of the crowd.

Incentivizing learning

Tournaments often offer rewards or recognition for top performers, which can motivate participants to learn and improve their forecasting abilities.

Identifying talent

Tournaments can help organizations identify individuals with exceptional forecasting abilities who may be valuable assets in strategic planning or other decision-making processes.

Encouraging collaboration

Tournaments can foster a spirit of collaboration among participants, as they often involve sharing insights, discussing methodologies, and learning from one another.

Generating valuable data

The forecasts and accompanying data generated by tournaments can be useful for the organization itself.

Confido is a free app designed to run

forecasting tournaments

How to run a forecasting tournament

1. Decide on the amount and type of questions

2. Decide about the level of collaboration among participants

3. Think about questions you want to ask

4. Develop quantified questions about the future

5. Inform your participants

6. Help your participants to prepare for the tournament

7. Start the tournament and share questions with your participants

8. Talk with participants

9. Resolve your questions

10. Evaluate the accuracy of participants' forecasts

11. Announce winners