Anyone following the 2012 election polling has undoubtedly noticed that there has been a significant divergence between the results of some of the more notable polls released recently.
As of today, Rasmussen Reports is showing an approval rating of 44 percent for President Obama and 55 percent disapproval, while Gallup shows a 49/45 approval/disapproval rating. These polls were both conducted May 8 – May 10 and have a margin of error of +/- 3.
Many other polls also vary widely from the two shown above. I chose these polls because they were conducted over the same time period and have the same sample size.
That’s a difference of 15 percent. How can two polls, both with a sample size of 1,500, be different by such a wide margin?
The first, and most obvious, answer is that the Gallup poll used a sample of American adults, while the Rasmussen Reports poll sampled likely voters. Likely voters probably have different opinions than randomly selected adults, but can this really explain a 15 percent difference in the polling results?
To answer this question, I looked into the methodologies used by Rasmussen and Gallup for their presidential approval polling.
Rasmussen Reports uses a telephone survey to determine presidential job approval. This poll does not include cell phones, but Rasmussen attempts to compensate for this by using an “online survey tool”.
Gallup’s methodology is slightly different. It also uses a telephone survey that includes a “minimum quota” of cell phone respondents. The Gallup poll will generally contain around 400 cell phone respondents and 600 landline respondents per 1,000 adults.
Both organizations weight their data to better reflect the overall population in terms of race, gender, age, and party, as well as other factors.
It’s also possible that Gallup and Rasmussen word their questions in a way that leads to different results. However, as a rule, reputable polling agencies try to phrase their questions in as unbiased a manner as possible.
While both organizations make efforts to adjust for the decrease in landline use, they make this adjustment in different ways. It’s possible that the inclusion or exclusion of cell phone results in the polling data may be a significant factor in the difference between the Rasmussen and Gallup presidential job approval data.