There are lies, damned lies, and exit polls. A purported connection between race and homophobia has recently made the rounds, prompting big think pieces from the likes of the Washington Post. We’ve been told that 7 out of 10 African-Americans who went to the California polls voted yes on Proposition 8 — a measure that passed on Tuesday overruling the California Supreme Court judgment that legalized same-sex marriage.
Even more amazing than this is the way this correlation is getting a free pass. The only way you can bring a demographic into election statistics is through the exit poll. But exit polls have problems. Back in 2006, Mark Blumenthal initiated a helpful series of posts summarizing some of the flaws: where the interviewer is standing in relation to the polling place, how well-trained the interviewer is, the tendency for voters who volunteer to participate upon seeing the interviewer with the clipboard, the inclination for the polls to favor Democrats in presidential election since 1988, and so forth. In 2005, the Washington Post reported that interviewing for the 2004 exit polls was “the most inaccurate of any in the past five presidential elections.” Large numbers of Republicans refused to talk with interviewers, and this, in turn, led to an inflated estimate for John Kerry. But despite these problems, exit poll faith is a bit like stubborn fabric softener sticking to a hard wonk’s argyle sweater. In a longass Rolling Stone article, even Robert F. Kennedy, Jr. believed in the gospel, suggesting that exit polling was the first indicator that the 2004 election had been stolen. Political slickster Dick Morris went further, stating that “exit polls are almost never wrong.”
Edison Media Research and Mitofsky International were the team behind the 2004 polling botch, and this dynamic duo also spearheaded this week’s California exit polling. The hard data is not yet available at the Edison/Mitofsky site. But the Associated Press has reported that 2,240 California voters (of these, 765 were absentees interviewed by landline telephone), interviewed in 30 precincts, represented the total number of people that Edison/Mitofsky interviewed. Which means that some percentage of these voters were African-American. Let’s give Edison/Mitofsky 50%. That leaves us with a mere 1,120 voters.
A quick jaunt to the California Secretary of State’s website reveals that there are 25,423 precincts in California and that 10.5 million people turned out on Tuesday. In other words, Edison/Mitofsky is making a major claim based on 0.11% (a little more than one-tenth of 1%) of the total precincts, and a sample of voters smaller than a crab louse dancing in a thorny thatch of hair. Is this really large enough? Exit polls have proved somewhat accurate in relation to simple binary choices, but I’m wondering if it all turns to bunk when it comes to correlation. Perhaps a legion of statistics experts can help explain why Edison/Mitofsky can get away with this. Because I’m tempted to view this as a strange offshoot of the Bradley effect.