Update: 8/27/24: Current Mood: Hope and Determination! Lately there has been constistant upwards polling trends for Harris/Walz. This encouraging news shows that potential voters are tuning in. Now we need to convert this enthusiam to actual ballots cast. Robert Hubbell (First Edition) says it best:
Despite significant improvement in every swing state, the race remains within the margin of error in reputable polls. If we can continue the progress and momentum through September, we have every reason to believe that Kamala Harris will pull ahead of Trump in swing states by a margin that indicates an incontestable lead.
In other words, we have every reason to be hopeful but no reason to be complacent. We must sustain the intensity of the last month and follow-through on our personal commitments to help Kamala and Tim win!
May 31, 2024 Are you stressing about the latest poll showing alarming support for Trump? We can’t just say “ignore polls” because we use them as well to show overwhelming support for legal abortion, contraception, IVF and gun safety. What you need to know is that all polls are not created equally. Let’s look at some examples of the different ways polls can be manipulated, and how to evaluate them.
Who Created the Poll and Why?
In addition to the obvious right wing “think tanks,” the profit-seeking media also has a different goal than objective truth. Robert Hubbell described it so succinctly in his May 14th newsletter:
“Polling has become big business for major media outlets. They get to write news stories in which they are star actors. The NY Times produces presidential polls on a monthly basis, guaranteeing exclusive stories in which it designs, conducts, and interprets polls that become front-page news for the Times. Create the news, report on the news you created, and interpret that same news for the public….
The NYTimes polls consistently overweight Trump’s prospects, which the Times dutifully reports, even though most of its findings show an effectively tied race. The Times analysis delves into subgroups with sample sizes so small the error bars render them meaningless.The Times treats its own polls as front-page news, but when polls showing more favorable results for Biden are reported by other outlets in the ensuing weeks, they are ignored by the Times. Because, well, those polls weren’t designed, conducted, and interpreted by the Times…”
Who is the pollster talking to?
In Dan Rather’s newsletter “Steady,” he talks about how many times polls have been poor predictors of actual results. In 2020, Biden won by a narrower margin than predicted. Rather explains that “It turns out pollsters didn’t factor in enough white, non-college-educated voters, who apparently are less likely to answer pollsters’ questions. They also happen to be Trump’s biggest support base.”
In addition, pollsters still have not adjusted to modern technology, depending on techniques developed well before cell phones existed. “Pollsters are struggling to keep up with changing technology. Not long ago, data was collected by calling voters at home on landlines. Now with the ubiquity of cell phones with caller ID, answer rates for pollsters have been plummeting. Also, some folks, maybe more than we think, just flat-out lie, to mislead the pollsters.”
What Questions are the Polls Asking?
The Roper Center, an old and highly respected pollster, has listed 20 questions that journalists should ask before they report a poll and their editor slaps on a highly provocative title to get clicks. They recommend examining the wording of the questions asked, which can reveal the bias of the group paying for the poll. But did you know the order of the questions matter too? The answers can be subtly manipulated, intentionally or not. For example, in economically troubled times, if the interviewer asks about the economy before he asks about the person’s approval of the president, the reported approval will be lower than if the order of questions was reversed.
How is the data reported?
Talking Points Memo says that “As we’ve noted a few times, this has become a consistent pattern in this presidential race: Joe Biden does substantially better in likely voter polls. … A Likely Voter screen is the pollster’s attempt to poll the actual voting electorate as opposed to the population of registered voters. So that distinction is very key. And if they diverge you really want to be doing better with Likely Voters, as Biden is.” Often the media does not distinguish between these two populations in their reporting, especially in the headlines.
What is the margin of error?
Pew Research has a very detailed description of what the margin of error means. Whenever you are reading a poll, pay attention to what the pollster describes as the margin of error. (Notice that this is still calculated by the pollster, who might have an ulterior motive.) Errors should get larger the smaller the sample size, because extrapolating that to the total population is less accurate.
Aren’t Poll Aggregators More Accurate?
The poll aggregator 538 (no longer owned by Nate Silver) attempts to take all the different polls and aggregate the results into one overall number. They have ranked different pollsters by 538’s evaluation of their error, bias and transparency. But basically, they mash all the polls together and report the average. My opinion? I don’t care how much you fiddle with weighting these polls, I believe in the truism: Garbage In, Garbage Out.
Can we get anything out of these polls?
As Rather notes, “Analysts from the polling aggregator FiveThirtyEight explained the 2022 election results this way. “[A poll’s] true utility isn’t in telling us who will win, but rather in roughly how close a race is — and, therefore, how confident we should be in the outcome. Historically, candidates leading polls by at least 20 points have won 99 percent of the time. But candidates leading polls by less than 3 points have won just 55 percent of the time. In other words, races within 3 points in the polls are little better than toss-ups — something we’ve been shouting from the rooftops for years.”
So What Should We Do?
If you want a more nuanced reading of the polls, turn off cable news, and ignore mainstream newspaper’s clickbait headlines. Instead, check out free newsletters like Robert Hubbell’s First Edition, Simon Rosenberg’s Hopium Chronicles or even Heather Cox Richardson’s Letters From an American to get context. The polls are a snapshot; and we need to be aware of the pollster’s focus. We also need to be aware of media bias – clickbait headlines lead to more profit.
That said, although we shouldn’t panic, we must not be complacent. When you think about the margin of error, think of yourself as working in the “margin of effort.” In other words, “Do More, Worry Less” as Rosenberg often says!