Crystal Balls to Expose Bad Pundits

Marco Ottaviani (Department of Economics) and Debrah Meloso (ESC Rennes School of Business) experimented with crystal balls to test strategic misreporting of information. Consider an economic forecaster who analyzes data and comes up with an opinion on what's going to happen to the economy. The forecaster wants others to think he is well informed. Will he report his opinion truthfully? The literature argues that forecasters could have incentives to misreport.

To capture this situation, Ottaviani and Meloso designed a urn and balls scheme. Each ball has an outer shell and an inner core. The inner core itself is a crystal ball filled with either blue or orange liquid. The outer shell is opaque and is also either blue or orange. There are two urns corresponding to the quality of information of the forecaster. The informative urn corresponds to the forecaster who perfectly knows the future and the inner core perfectly matches the color of the outer shell. The uninformative urn captures the forecaster who has no ability to predict the future: the color of the shell does not predict the color of the core.

The game goes like this: First, the ball is drawn from either the informative or the uninformative urn with equal probability, but neither the forecaster nor the evaluator know which urn the ball is coming from. Second, the forecaster reports to the evaluator the color of the outer shell he sees (without seeing the inner core). The evaluator then observes the color of the inner core and assesses the probability that the forecaster has observed a ball drawn from an informative urn. The forecaster is paid according to evaluator's assessment that the ball was drawn from an informative urn.

The theory predicts that if more than 75% of the balls have a blue core the forecaster has an incentive to report blue when he actually sees an orange shell – an instance of misreporting. Intuitively, even after seeing an orange shell the forecaster thinks that a blue core is more likely than an orange one. So, by misreporting, the forecaster can increase the probability of correctly guessing the core.

The actual behavior in the experiment is rather erratic but there is a tendency for more misreporting when theory predicts it.