Sunday, November 04, 2012

WHY YOU SHOULDN'T BELIEVE THAT FOSTER McCOLLUM POLL OF MICHIGAN

In contrast to a lot of other polling from Michigan showing Obama ahead, the local Fox affiliate has a poll of the state showing the race tied. Don't be fooled.

Why shouldn't you believe this poll? Because the polling firm, Foster McCollum White Baydoun, has a likely voter screen that's utterly absurd, as Nate Silver explained a while back:
In Michigan, there are two local polling firms in particular that have shown poor numbers for Mr. Obama. One is ... Foster McCollum White Baydoun, which had Mr. Romney ahead by four points in its last survey.

Foster McCollum White Baydoun actually does survey one state apart from Michigan: it released a poll of Florida last week. In that survey, it showed Mr. Romney ahead by almost 15 points -- perhaps the biggest outlier we’ve seen in any individual poll so far this year.

As I mentioned when that Florida poll came out, this firm anticipates a big decline in participation among groups like younger voters that are ordinarily inclined to support Democratic candidates. The initial news release of the Florida poll appeared to show that the firm anticipated that just 11 percent of Florida voters would be under age 50 -- in contrast to about half the Florida electorate in 2008, according to exit polls. A principal of the firm, Eric Foster, later wrote me to say that the news release had been incomplete, and voters under the age of 50 had actually been weighted to make up 27 percent of the poll instead.

That's still an implausibly large drop-off, in my view, from 2008. It may be because the Foster McCollum White Baydoun likely-voter model looks for voters who, as they described it, participated in "odd year municipal and county elections" -- where turnout is tiny as compared with what it will be on Nov. 6. In any event, this method seems likely to produce extremely Republican-leaning results -- whether applied to Michigan, to Florida or to any other state.
A firm that thinks three-quarters of the electorate will be 50 or older, even in Florida, is not going to get believable results.

I've seen a tweet, which I can't find now, that says the current Michigan poll has a similar huge age distortion, greatly tipped toward older voters. (Curiously, when you go to the Fox story and try to click on the link to get full results, the link doesn't work.)

Don't believe this poll.

6 comments:

Victor said...

WHAAAAAAA?

You mean an election for your local dogcatcher in an off-year, doesn't have the same turn-out as a Presidential election?
WHODATHUNKIT?!?!?!?!?!?!

Anonymous said...

According to their website they have 3% unner age 30 (16% in 2008); 15% ages 31-50 vs 25% in 08; 38% 51-65 vs 30% in 08; and 44% ages 65+ vs 29% in 08. Not reasonable.

Dark Avenger said...

"There are lies, damned lies, and statistics."

Mark Twain.

Anonymous said...

After re-reading the report, my comment above may not be accurate. My apologies.

atlasfugged said...
This comment has been removed by the author.
atlasfugged said...

I looked through this report yesterday and it looks like they aren't making the same kind of absurd assumptions in their method of weighting of different age cohorts. It's weighting assumes that the 60% of registered voters in MI will be over 50 years old, which is not outrageously higher than the 54% that the Marist poll shows, for example. It also assumes that 16% of voters will be aged 18-30, which actually slightly higher than the 15% which the Marist poll found. It's assumptions for percentage of African-American registered voters are also in line with other polls. (The Latino sub-sample wasn't weighted because the percentage of respondents closely matches the the percentage of registered Latino voters in its projections of the electorate. It also matches well with the findings of the Marist poll).

Nevertheless, that additional 6% weight for respondents over 50 probably does "skew" the results toward Mitt Romney, but the weighting method used in this poll doesn't introduce the same ludicrous amount of skew as the method used in the firm's FL poll - which had Romney up by 14% in August!

What I found unusual was the age distribution of its sample of respondents. Its respondents consisted of 82% of people over 50, including 44% of people over 66, while only 3% and 15% of the respondents were aged 18-30 and 30-50, respectively. It's as if they went out of their way to call mostly old people. The samples of most other polls, even ones (e.g. http://bit.ly/VqNhWG) that have the race tight in MI, consist of a far greater (and more representative) proportion of voters aged 18-50 than the sample this Fox-sponsored poll was able to obtain. Presumably, the pollster would attempt correct this lopsided sample using its weighting methodology, but the sample is so disproportionately old that it should have been thrown out. It was a really bad, unrepresentative sample of respondents.