The greatest bias in traditional political polling is the way "leaners" are counted.
Leaners are people who say they are undecided when polled, but when pushed to pick someone they are leaning towards, they come up with a name, usually someone with strong name recognition.
For example, suppose a political poller called you and asked; “If the election were held today, would you vote for [blank], [blank], or [blank], etc.?”
Response; “I’m not sure.”
Poller; “Well, if you had to pick someone, who would you be leaning towards?”
Response; “Well, er, um, I’m not sure, maybe, … , well I might pick McCain?”
Poller; “McCain?”
Response; “Yup, McCain, I think.”
Poller; “Are you sure?”
Response; “Yep.”
BINGO! In this poll, the respondent will be listed as voting for McCain.
Most polls do not list what percentage of responses are leaners except sometimes in the fine print, but usually about 1/3 of the support of any candidate in any poll is a leaner.
Let’s look at a typical poll to see how the way leaners are counted distort perceptions.
Saint Anselm College Poll for New Hampshire, October 2007, Republican Primary
Results:
Mitt Romney 32% Rudy Giuliani 22% John McCain 15% Ron Paul 7% Mike Huckabee 6% Fred Thompson 5% Tom Tancredo 1% Sam Brownback 1% Duncan Hunter 1% Other 1% None/No answer?Didn’t know 9%
[The poll does not list a margin of statistical error. Most traditional polls have a margin of error of 4% or less, so we will assume a 4% margin of error]
This poll makes it appear that it is mathematically impossible for anyone but Romney, Giuliani, or McCain to win, with Romney almost having it wrapped up. The others have no chance to win.
For Giuliani, if he takes his 22% and gets all of the 9% undecided vote, he is still at 31%, not enough to win. So he needs the margin of error to fall in his direction as well. Not likely.
For McCain, if he takes his 15%, gets all 9% of the undecided vote, and the 4% margin of error falls in his favor, he will be at 28%, still not enough to win. Unless Romney drops the 4% margin of error to 28%, then it’s a tie.
Very unlikely.
Paul has absolutely no chance to win.
But lets look at what would happen if the poll did not include leaners in the candidate numbers, assuming 1/3 of all support is leaners:
Mitt Romney 21% Rudy Giuliani 15% John McCain 10% Ron Paul 5% Mike Huckabee 4% Fred Thompson 4% Others 2% UNDECIDED 39%
If the poll were reported this way, it would look like EVERYONE had a chance to win. And in fact, they do, if they are able to overcome the polling bias that leads to less media coverage for underdog candidates. For example, in this poll, Paul would only need 1/2 of the undecided vote to beat Romney by 3%.
The next time you hear someone says someone has no chance to win because of a poll, please ask them if they have analyzed the effect of leaners.
November 5, 2007
