Meet “Polly,” the political forecasting expert. She picks up others’ predictions about election outcomes and repeats them almost exactly as she hears them. I say “almost” because beyond simply reproducing them, she performs a simple calculation: She averages them.
Polly’s formula integrates four different methods. They are polls, the Iowa Electronic Market, econometric models and experts’ judgments. This yields the PollyVote, which stands for “many” and “political.”
Combining forecasts is one the most powerful tools available to forecasters. A meta-analysis of 30 studies estimated that the combined forecasts yielded a 12 percent reduction in error compared to the average error of the components. The reductions of forecast error ranged from 3 to 24 percent. In addition, the combined forecasts were often more accurate than the most accurate component. Combining forecasts is especially useful if the forecaster wants to avoid large errors, and if there is uncertainty about which method will be most accurate. (Armstrong 2001.)
Imagine, for instance, that you have two forecasts of a quantity: How big would the error of the second forecast need to be in order for the error of the average of the two forecasts to be larger than the error of the first forecast alone?
The answer is the error of the second forecast would need to be bigger than the error of the first forecast if the error was in the same direction (sign) but more than three times the error of the first forecast if the error had the opposite sign. So the key is to use methods and data that differ substantially so as to bracket the true value. Under such favorable conditions, combining reduces forecasts errors almost by half (Graefe et al., 2010).
What’s not to like about combining? Interestingly, people often lack the intuition to understand the benefits of combining. In a series of experiments with highly qualified MBA students, a majority of participants thought that the average of estimates would reflect only average performance (Larrick & Soll 2006).
Polly made her first appearance in 2004, when she delivered a near-perfect prediction of the percent of the two-party vote that President Bush won against Senator Kerry. Four years later, she repeated this in the Obama-McCain contest. At no time during either year—Polly’s forecasts began at six to 12 months before Election Day—did Polly’s forecast show either eventual winner losing the election. Those who used Polly in each of those elections would have made money on the betting markets.
This year, Polly has been in contact with Alfred Cuzán, Professor of Political Science at the University of West Florida, about forecasts for the outcome of the midterm elections for both the U.S. House of Representatives and Senate. For the latest forecast, visit pollyvote.com. Forecasting is obviously an uncertain business, but I would not advise that you bet against Polly. She would beat you most of the time.
But why stop there? This is a relatively inexpensive procedure that can improve forecasts in any organization. To get some idea as to which forecasting techniques might be relevant to your situation, go to the forecasting selection tree at forecastingprinciples.com.
References
Armstrong (2001). Combining forecasts. In: J. S. Armstrong, Principles of Forecasting, Kluwer Academic Publishers, Boston, pp. 417-440. (In full text at ForPrin.com)
Graefe, A., Armstrong, J. S., Jones, R. J. and Cuzán, A. G. (2010). “Combining forecasts: An application to U.S. Presidential Elections,” Working paper, Available at: http://dl.dropbox.com/u/3662406/Articles/Graefe_et_al_Combining.pdf
Larrick, R. P. & Soll, J. B. (2006). “Intuitions about combining opinions: Misappreciation of the averaging principle,” Management Science, 52, 111-127.
