I was wondering when this was going to happen. Democrats are assailing Rasmussen Reports for its polling results that often show President Barack Obama and Democratic agenda items tracking a few points lower in public support than do some other polling outfits. Politico reported the story over the weekend and Hot Air followed up yesterday with analysis.
There is certainly room for error in polls that can be caused by a variety of reasons, but advances in the field have enabled independent and respected pollsters like Scott Rasmussen to forecast election outcomes with high degrees of accuracy (RR predicted the winner’s percentage of the popular within less than one point in 2004 and 2008, among other high-profile successes). There is no control mechanism (like an election) to objectively assess the accuracy of survey results on a given policy issue, but if the same sources are using the same methods then we should expect the results to be similarly accurate for these sorts of questions.
Rasmussen includes only the responses of likely voters in its results, whereas some organizations may include all adults or all registered voters. For measuring electoral strength and the actual potency of political issues, the former is a more accurate indicator. Furthermore, Rasmussen’s samples include a more realistic partisan respondent composition than that of some firms (Morrissey mentions the New York Times, CBS, ABC and Washington Post), which have been known to over-represent the amount of Democrats. These two differences in procedure may contribute to the difference in results.
Other kinds of variation may result from how a poll asks a question. Some perennial questions are fairly standard in their formulation and may vary little if any among polling operations. Others vary more considerably, which naturally affects results. Of course, some organizations ask particular questions that others avoid entirely. Finally, how the results are published and reported also matters. For instance, Rasmussen has developed a presidential approval index, “calculated by subtracting the number who Strongly Disapprove from the number who Strongly Approve.” Updated daily, the index portrays Barack Obama with [weak public support. However, Rasmussen also used the index during the Bush presidency.]
I’ll continue to rely on Rasmussen Reports because I think it provides sound polling on important issues of the day. I’ll also continue to look at the Real Clear Politics average, which aggregates multiple poll results on a running basis. Other polls will continue to be used by various sources and cited in the media, and that’s fine too. This whole dust-up illustrates a larger story at play, which is the continuing pervasiveness of polls on the media and political landscape. It’s all part of the horse-race style coverage and political point-scoring we’ve come to know and expect when it comes to journalism and punditry, respectively.
There’s nothing inherently wrong with polls, but undue attention to them shouldn’t distract from other matters. Beyond that, if you dislike the results of a poll, discussing actual methodology more than alleged motive may be the more objective way to discern whether bias exists.
* This article was edited on 01/05/10 at 2:50 p.m. CST to correct a sentence fragment, indicated by brackets above.