[btomba_77]

[h1]Polling Warning Signs Are Flashing Again[/h1] [link=https://www.nytimes.com/2022/09/12/upshot/polling-midterms-warning.html]Nate Cohn[/link]: Democratic Senate candidates are outrunning expectations in the same places where the polls overestimated Mr. Biden in 2020 and Mrs. Clinton in 2016.

[link=https://messaging-custom-newsletters.nytimes.com/template/oakv2?campaign_id=9&emc=edit_nn_20220912&instance_id=71706&nl=the-morning&productCode=NN&regi_id=4500752&segment_id=106056&te=1&uri=nyt%3A%2F%2Fnewsletter%2F1cc6748a-42ab-55b6-9bf6-1fbd85722697&user_id=4ae52eb7f8b8fa824c1c3801a38579f0]David Leonhardt[/link]:  [b]Are Democrats again about to be disappointed by overly optimistic polling?[/b]

 In each of these states and some others pollsters failed to reach a representative sample of voters. One factor seems to be that Republican voters are more skeptical of mainstream institutions and are less willing to respond to a survey. If thats true, polls will often understate Republican support, until pollsters figure out how to fix the problem.

This possibility offers reason to wonder whether Democrats are really doing as well in the midterm elections as the conventional wisdom holds.
 

[kayla.meyer_144]

Catch-22 rock vs hard place, FUD vs everything is puddle wonderful.
 
I agree with Dan here, pollsters have not exactly been showing their accurate Cassandra skills for a few years now. Let Republicans suppress the vote, not the media or complacent Blue voters. & re. Leonhardts admonition, Republican/Trumpy voters, even posting here on AM had been bragging in the past of deliberately lying to pollsters.
 
Get out & vote. Vote Blue.

[satyanar]

No doubt it’s about turn out. I predicted in a different thread that women will be more likely to stay passionate in this election. Should help the Blue.

[btomba_77]

Quote from Thread Killer

No doubt it’s about turn out. I predicted in a different thread that women will be more likely to stay passionate in this election. Should help the Blue.

 
Theoretically, the “likely voter” model is supposed to model expected turnout.
 
_________________

[link=https://fivethirtyeight.com/features/politics-podcast-its-not-yet-time-to-start-worrying-about-the-polls/]https://fivethirtyeight.c…rying-about-the-polls/[/link]
 

Nate Silver and Galen Druke talk about the polling for the 2022 mid-terms on the 538 podcast. Good listen if anyone wants. (start about the 27 minute mark)

They also give a reminder about 30%-ish probabiity events (Like Dems taking the House or Republicans taking the Senate respectively). They happen all the time

[btomba_77]

It’s 2012 all over again … “Unskewed” polls are back, baby.

RCP now has a real time poll unskewer

[link=https://www.realclearpolitics.com/epolls/2022/senate/pa/pennsylvania_senate_oz_vs_fetterman-7695.html]https://www.realclearpoli…vs_fetterman-7695.html[/link]

[img]https://pbs.twimg.com/media/FeovMGPX0AYOTaN?format=png&name=medium[/img]

[btomba_77]

“Unskewed polls 2.0”

[link=https://www.wsj.com/articles/how-2022-midterm-polls-performed-in-senate-races-11668037633]Wall Street Journal[/link]: Across the eight most competitive races, Democrats on average did about three points better than the final poll averages calculated by [i]Real Clear Politics[/i]. And a number of those averages camouflage a wide disparity among individual polls.

Wapo –

[link=https://www.washingtonpost.com/politics/2022/11/09/why-was-midterm-result-such-surprise/]https://www.washingtonpost.com/politics/2022/11/09/why-was-midterm-result-such-surprise/[/link]

RealClearPolitics, which has traditionally had a straightforward average of polls, introduced an effort to adjust polling to reflect perceived anti-GOP shift. Perhaps because of that expectation that polls were overestimating Democrats, Tom Bevan, co-founder of the site, publicly [link=https://twitter.com/TomBevanRCP/status/1590135840810106883]challenged[/link] the Economists G. Elliott Morriss presentation that the Senate was a true toss-up. Which, of course, it was.

To some extent, Republicans and right-wing commentators got caught up in enthusiasm about an expected trouncing. Predicting Democratic doom is always good for attention on the right, and attention is the coin of the modern realm. Republicans were confident in private but effusive in public.

Democrats, meanwhile, were still shellshocked from 2016. (Dont believe me? Sneak up behind a Democrat and whisper Did you see what [link=https://www.nytimes.com/interactive/2022/11/08/us/elections/results-needle-forecast.html]the needle[/link] did? in their ear.) They, too, were [link=https://www.washingtonpost.com/politics/2022/08/31/shadow-stalking-democrats-improving-numbers-can-polls-be-trusted/?itid=lk_inline_manual_39]ready to believe[/link] that things were going to go better for Republicans than the polls indicated. Yet it was often the case that the predictions of dominance were unmoored from any evidence and simply reflections of personal desire, the rights driving energy or both.

[/QUOTE]

[satyanar]

Nothing about failure of polling should be surprising at this point. In fact these results were not surprising at all. 

[btomba_77]

After looking great in 2020, it looks like Trafalgar is going to be the worst or near-worst polster of the 2022 cycle.
 
 
It’s unknown as to why (again, because their methodology isn’t public) but my guess is that the :what does your neighbor think” approach falls down when the right-leaners are in a bubble of their own … and that led them to believe that inflation/economy/crime were much bigger issues than abortion/democracy preservation …. when it turns out those things were a wash.

[satyanar]

I think the biggest problem with the polls is the inability to get representative people to answer these days. It doesn’t matter what questions you ask if you are asking too few or the wrong ones.
 
Trafalgar found a way around it when people were still picking up the phone. Now not so much.
 
 
 
 

[btomba_77]

[h3][link=https://www.vox.com/policy-and-politics/2022/11/27/23475262/midterms-red-wave-democrats-election]The guy who got the midterms right explains what the media got wrong[/link][/h3] [b]The media fell for a false red wave narrative.[/b]

[b]Real election results werent given enough credence over polls [/b]
 
[b] [/b]Real voting is more important than polling, Rosenberg said. The way you interpret an election is looking at how people vote.

…..

[b]Polls were misinterpreted  [/b]
 
[b] [/b]
When the polling averages narrowed in the fall, it was partially because [link=https://www.politico.com/news/2022/11/01/biden-gap-senate-surveys-00064362]partisan polls[/link] commissioned by Republican organizations were bringing them down for Democrats. Rosenberg was one of the first to identify the phenomenon, which he described as an unprecedented campaign by Republicans to flood the polling averages in the final month to create this impression of the red wave.
If you were looking at polling averages that included Republican polls, you were looking at a completely different election than we were looking at, he added.
When Rosenberg stripped out the partisan polling, he foresaw an election in which New Hampshire, Arizona, Georgia, and Pennsylvania were leaning Democrat, Nevada was too close to call, and Ohio, North Carolina, and Wisconsin were leaning a little Republican. Thats consistent with what actually transpired.

[b]The enduring salience of [i]Roe[/i] was underestimated [/b]
 
[b] [/b]The [link=https://fivethirtyeight.com/features/abortion-was-always-going-to-impact-the-midterms/]evidence was always in the polls[/link]: Interest in abortion didnt drop over time and, among Democrats, actually increased across [link=https://morningconsult.com/2022-midterm-elections-tracker/]several[/link] [link=https://today.yougov.com/topics/politics/trackers/most-important-issues-facing-the-us?crossBreak=democrat]tracking[/link] polls. And Rosenberg thinks its salience wont wane anytime soon.
 
This could be creating a millstone around the Republican Partys neck not just in this election but for many elections to come, and potentially may alienate an entire generation of young people, he said.
[/QUOTE]
 

[satyanar]

Thats a cool analysis. You could feel that playing out on this forum. Helps to not have a partisan approach. 

[btomba_77]

[b]The (Nonpartisan) Polls Were Pretty Accurate[/b][/h1]  
 
 
[link=https://www.thebulwark.com/the-nonpartisan-polls-were-fine-actually/]The Bulwark[/link]: The nonpartisan polling was actually pretty good in 2022. Most of the phantom Republican strength in pre-election statewide polling was a function of junk firms with poor data quality and low transparency spamming the polling averages with bad polls.
 
In reality, an aggregation of nonpartisan polls predicted the correct winner in every Senate battleground and would have predicted the margin substantially more accurately than the partisan GOP pollsters which flooded the averages in almost every major race.

 

[btomba_77]

Trafalgars self-described approach was akin to artificially adding Republicans to account for the supposed inbuilt bias in polling. But their underlying problems werent fixed even then; [link=https://twitter.com/Nate_Cohn/status/1587173827607511042]as Nate Cohn noted[/link], the crosstabs of Trafalgar polls were unbelievable, with Pennsylvania crosstabs suggesting a tight race despite Democrats [i]winning[/i] white voters, and their numbers showing the heavily-Democratic Philadelphia metro area voting the same as the rest of the state. Both sets of numbers strained credulity even pre-election and were not explainable by the variance inherent to poll crosstabs. They could have thus been dismissed outright by people with familiarity of the states voting patterns. But residual scarring from 2020 may have left many observers wary of looking into this in detail.
 
There were similarly egregious misses in Georgia, where the aggregate of nonpartisan polls showed a Warnock lead while the partisan polls showed Herschel Walker up. The polls that showed Walker winning, however, all shared one key element: They kept showing an extremely Republican electorate in which Walker was pulling around 20 percent of the Black vote. This would have been a phenomenal showing for any Georgia Republican, especially considering that Trump got only 6 percent of black Georgians. When polls show seismic demographic shifts like this, they are worthy of deeper examination and occasional skepticism, especially when the shift is only observed by partisan pollsters (many of whom [link=https://www.nytimes.com/2022/11/05/upshot/polling-averages-midterm-election.html]do not adhere to standard data quality procedures[/link]). Instead, they appear to have been taken at face value by many observers worried that polls would always overestimate Democrats.
 
That is a dangerous approach, because over the long term, polling error is much more random than people think. Historically, it [link=https://fivethirtyeight.com/features/will-the-polls-overestimate-democrats-again/]does not correlate cycle to cycle[/link]. So the best approach to using polls to understand the political landscape probably remains the same as it always has been: Trust the work that is methodologically sound, with robust methodology paired with transparent data collection and reporting. And be skeptical of outfits with partisan motivations who wont show their work.
 

[btomba_77]

[link=https://fivethirtyeight.com/features/2022-election-polling-accuracy/]FiveThirtyEight

[/link]
[h1]The Polls Nailed The 2022 Election[/h1] [h2]Weve updated our pollster ratings. You can read exactly [link=https://fivethirtyeight.com/methodology/how-our-pollster-ratings-work/]how we calculate pollster ratings here[/link].[/h2]

Lets give a big round of applause to the pollsters. Measuring public opinion is, in many ways, [link=https://www.nytimes.com/interactive/2022/10/24/opinion/frustrated-with-polling-pollsters-are-too.html]harder than ever[/link]  and yet, the polling industry just had one of its most successful election cycles in U.S. history. Despite a loud chorus of naysayers claiming that the polls were either [link=https://thehill.com/homenews/campaign/3718434-optimistic-democrats-insist-the-polls-are-wrong/]underestimating Democratic support[/link] or [link=https://gelliottmorris.substack.com/p/the-polling-website-where-republicans]biased yet again against Republicans[/link], the polls were more accurate in 2022 than in any cycle since at least 1998, with almost no bias toward either party.

…
Theres been a lot of interest in statistical bias in recent years specifically, whether polls are [link=https://fivethirtyeight.com/features/trump-supporters-arent-shy-but-polls-could-still-be-missing-some-of-them/]systematically biased against Republicans[/link] nowadays. These concerns stem primarily from polls [link=https://fivethirtyeight.com/features/why-did-republicans-outperform-the-polls-again-two-theories/]overestimating Democratic support[/link] in the 2016 and 2020 cycles; as the table below shows, the polls in 2015-16 had a weighted-average bias of D+3.0, and the polls in 2019-20 had an even worse weighted-average bias of D+4.7. A lot of people assumed the polls would have a similar bias again in 2022. But that assumption was wrong: For 2021-22, polls had a weighted-average bias of just D+0.8.

…

Unsurprisingly, the top of the list is generally populated by academic and media-affiliated pollsters that have been trusted names in polling for a long time. But special congratulations are due to [link=https://projects.fivethirtyeight.com/pollster-ratings/suffolk-university/]Suffolk University[/link] and [link=https://projects.fivethirtyeight.com/pollster-ratings/siena-college-the-new-york-times-upshot/]Siena College/The New York Times Upshot[/link], which had the lowest average errors of any pollster that conducted at least five qualifying polls last cycle. As a result, Suffolks pollster rating has increased from B+ to A-. Siena College/The New York Times Upshot already had an A+ grade, so it didnt get a ratings boost. Still, its stellar performance did push it past [link=https://projects.fivethirtyeight.com/pollster-ratings/selzer-co/]Selzer & Co.[/link] for the distinction of most accurate pollster in America (at least by FiveThirtyEights reckoning). 
Meanwhile, the bottom of the list features quite a few Republican-affiliated pollsters that systematically overestimated the GOP in 2022: RRH Elections, InsiderAdvantage, co/efficient, Moore Information Group.[link=https://www.auntminnie.com/Forum/"https:/projects.fivethirtyeight.com/polls/governor/2022/oklahoma/"]FiveThirtyEights final polling average[/link] of the gubernatorial race, but he won by almost 14 points.
 
“>14[/sup] But the most famous of these is probably [link=https://projects.fivethirtyeight.com/pollster-ratings/trafalgar-group/]Trafalgar Group[/link], a pollster whose methods are [link=https://twitter.com/NateSilver538/status/1322301003090268162?s=20]notoriously opaque[/link] but that [link=https://nymag.com/intelligencer/2022/11/trafalgar-groups-robert-cahaly-explains-his-polling-miss.html]played a significant role[/link] in shaping the ultimately untrue narrative that a red wave was building with its 37 (!) qualifying pre-election polls. Trafalgars polls were quite accurate in 2020, when its Republican-leaning [link=https://fivethirtyeight.com/features/calculating-house-effects-of-polling-firms/]house effects[/link] helped it avoid the big polling miss that other firms experienced. As a result, it went into 2022 with an A- pollster rating. But its poor performance last cycle has knocked it down to a B making it the only pollster to fall two notches in our ratings this year.[link=https://fivethirtyeight.com/features/2022-election-polling-accuracy/#fn-15]15[/sup][/link]
 

[/QUOTE]

Quick thread. Warning: not super exciting.

1. In the course of our reporting on Trafalgar Group—part of the due diligence we often do while entering polls—we've learned that some of their polling was done for partisan clients that weren't clearly disclosed.

— Nate Silver (@NateSilver538) October 30, 2020