Comparison of predictions with outcomes

Here is a graph of last-week polls compared with real outcomes. The graph is a bit rough but will do for now. Note, with a few exceptions, the excellent agreement of averaged opinion polls with real outcomes. Only states with at least five polls completed on or after October 25 are included. Colorado was an unusual case because the polls were exceptionally scattered.

Comparison of last-week polls with real outcomes

Here are the same data, divided by the standard error of the mean (SEM) of the pre-election polls ("z-score"), and plotted as a function of final Bush-Kerry margin. The z-score is a measure of how anomalous the real result was. Note that Missouri (MO) and Florida (FL) have very different results, suggesting that some extraneous factors were at work in these states, either in polls or voting. In the case of Missouri, the SEM is very small, 0.5%, suggesting that in this case the anomaly comes from the fact that pre-election polls happened to cluster close to one another. Therefore the only remaining anomaly is Florida.

Comparison of last-week polls with real outcomes

Finally, here is a graph comparing exit polls and real outcomes. Exit polls are noisier and therefore somewhat harder to interpret. However, the data generally fall around a line about 3 points below the zero diagonal. The point at (+1, -5) is Florida.

Comparison of last-week polls with real outcomes

Postscript: Jim G. writes in with county-by-county analysis of Florida to counter my argument that something strange happened there. I am not entirely convinced, but he has done more than I have and it's worth reading.