So, I've been hearing a lot of crap takes this election cycle when the data doesn't fit one's perceptions. "Polls dont real", "but the fundamentals", something something Allen Lichtman's 13 keys, and now given how close the race is getting, people are starting to get weird over the websites their polling data from.
I use realclearpolitics, because I've ALWAYS used them. They've been around since I was in college and they were the first polling aggregation site ever that I recall. And part of me just refuses to change. Especially because I never had particularly bad experiences with them. They normally get 48 states right per election cycle and in pre 2016 polls at least, they were often very close in the margins.
Fivethirtyeight was on the up and up because Nate Silver happened to guess the election perfectly and went 50/50 in 2012, which was a feat, but one of dumb luck rather than him just being smarter than everyone else. Seriously the hype surrounding 538 was always massive. I mean he got one election right and everyone treated them, especially silver, like a god.
In 2016, when everyone got 2016 wrong in the polling community, people were saying, WELL NATE SILVER GUESSED IT BETTER THAN EVERYONE ELSE AND GAVE TRUMP A 30% CHANCE! Oh? I used RCP's data and guessed a 56% chance for Hillary. I had the elction literally as a toss up. I had it more right than anyone else.
In 2020, 538 and Nate Silver had this methodology of fundamentals, and weighting and culling the polling averages, and had Biden at like a 89% chance of winning. And the election was way closer than it was expected? Gee, what went wrong? Well, I know, because I made the same mistake.
You see, there was a lot of controversy over polling itself. There were these right wing pollsters, like Trafalgar, InsiderAdvantage, etc., who just added a bunch of points to Trump to account for the shy trump voter effect. This was considered garbage polling science. 538 decided to exclude these polls from their averages. So did I. And I got the same 89% roughly 538 did.
And election night was ROUGH. I literally had north carolina and florida going Biden, and seeing them not just go Trump but go by several points freaked me out. Then Ohio was a lot more red. Michigan and Wisconsin were barely holding on blue despite being "safe" states in my estimation. PA was +5 in polling but also came down to the wire. And yeah. If one recalls I was literally freaked out in my election takedown afterwards.
What went wrong? In retrospect, it was removing all of those right wing polls. Trafalgar, InsiderAdvantage, Rasmussen, etc., they might be biased and hacky, but removing them made the polls skew in an insanely liberal direction. And it didnt line up with reality on election day. So you know what? I've decided....never again. RCP says a polling average is such and such, you know what? It's probably pretty darned close. Because historically it is. The problem with 2020 was all of this poll weighting nonsense, and leaning into fundamentals and blah blah blah. 538 and sites like that might want to lean into that but to me it sophistry to appeal to psuedointellectuals.
I get decent results from polling only. I mean, by this point I've gone back over 20 years of data going back to 2004 and I find that polling generally gets it right. Even if it gets some minor details wrong, if anything, most mistakes I've made over time came from the fact that tried to go against the polls. I try to rely on conventional wisdom, or weighting polling averages, and I end up making my predictions WORSE.
And that brings us back to the big complaint liberals are making. Sites like 538 weigh polls and have Harris in the lead. RCP doesn't and has Trump still winning. Now, both sides are engaging in their own copium over this. Trumpers are showing RCP's election map like YEAH WERE STILL WINNING, ignoring that their lead is well within the margin of error and its a functional toss up. And then liberals are like, "well they have a conservative bias because 538 weighs polls and blah blah blah." And uh...YEAH. AND THAT'S ONE OF THE REASONS I DON'T CARE ABOUT 538. 538 is first and foremost a site that provides their own predictions and electoral analysis. They love to make complex models and dazzle you with BS, and as you can tell, I dont think they're that much more accurate than anyone else is. Why? Because there's so much uncertainty in the data that no one actually knows, and just because they got lucky with a few takes doesn't make them right. It's just sophistry to appeal to pseudointellectuals who think they're smarter than they are.
And you know what? At this point, have at it. You wanna trust their "expert analyses" and weighted polls, go for it. I'll just present raw data, use a simple statistical method to estimate probabilities, and then give you my opinion from there. In the past, like 2016 and 2020, my own analyses dont vary a ton from theirs, especially the stripped down stuff. My own model is basically what used to be their "lite" model when silver was still there. Ya know, polls only, straight percentages, no BS.
Btw, Silver aint their any more and 538 sucks now. They're terrible. They had Biden at like 50-50 which was a joke. I admit my own polling model is more sensitive to plling volatility and more inclined give low percentages than most models do, but I'm also just presenting data and going by where things stand now. Sure, if you wanna predict how things will work out 3-4 months from now, youre gonna be a lot less certain. because a lot more can happen between now and then.
So yeah im not really gonna be using 538 for more than just getting polls I cant get from RCP (like nebraska CD2 polls...). Not that I would anyway. It kinda defeats the purpose of doing my own predictions. But if you want someone to do you thinking for you...have at it.
But yeah. I'll be using RCP's polling averages for the duration of this election. I dont believe in weighting polls. I dont need more sophisticated analyses that they do. I just want data that I can use to make my own predictions.
EDIT: Also looking at polling averages on RCP as per my election predictions, removing trafalgar/insideradvantage/rasmussen wouldnt change the polling averages in a meaningful way in most states. We're talking tenths of a percent to the final average. In some states it would actually hurt Harris. And the only states where those polls have an outsized right leaning effect are Nevada where 2/3 polls are from the offending companies and without them Harris would be up by 2, and Florida, where 1/2 polls are and Harris would be down 7 instead of 8.5. Beyond that, the effect just isnt large enough to justify removing those polls from the averages. Again, we're talking shifts of tenths of a percent in the margins and not all of them are in Harris's favor.
So yeah. The concern is overstated and only matters in Nevada and maybe Florida.
No comments:
Post a Comment