So, I did an exercise where I went through the 5 previous election cycles' polling data and used my simulator to predict 100 outcomes. So I want to compare its predictive power to the results that we got, and the more linear probability model.
2004
Outcome: Bush won
Linear probability: 56%
Simulator probability: 70%
I mean, both got the right outcome. And some simulations probably came close to the actual result, although many were kind of strange and off the wall due to the "weighted coin flip" nature of my simulator.
2008
Outcome: Obama won
Linear probability: 95%
Simulator probability: 100%
This one is the easiest election to predict. And I also spammed the button to make the simulator keep spitting out tons of predictions every second, I didn't see it flip to McCain for even a single flash. As for the simulator, I did recall seeing a simulation where Obama won Indiana too and resembled the real outcome.
2012
Outcome: Obama won
Linear probability: 76%
Simulator probability: 85%
I mean, the linear one probably did a better job, as the result was basically what was expected + flipping Florida, although the simulator probably spat one out like that.
2016
Outcome: Trump won
Linear probability: 44%
Simulator probability: 19%
So, this is where things really broke down with politics in general. Linearly, Trump winning wasnt that improbable. Anything within the 40-60% range is a de facto tossup, and Trump did prove that the linear model is the correct one to apply. However, the simulator had advantages. I mean, Trump's actual win was weird. Clinton did win New Hampshire and Nevada, but lost Pennsylvania, Michigan, and Wisconsin. It's hard to predict an exact outcome like that. It really did have randomness to it, even if Trump systemically overperformed. I would say some of Trump's wins did resemble the actual outcome, but yeah, it's hard to get that outcome with just randomness too. I'd say that the linear/trend model is probably more accurate and true to form, but the simulator is good at coming up with some of the more off the wall outcomes hypothetically, and yes, my simulator can spit out outcomes like we get. But given the sheer number of combinations of states going every which way, it will also spit out a lot of random outcomes that aren't close too. So it gets it right in a "brute force" way of considering random probabilities, while it seems to overly favor the winner. Given 2016 was the year the polls "got it wrong", the linear model did seem closer than the simulator in terms of guessing the right prediction and its probabilities.
2020 (original prediction)
Outcome: Biden won
Linear probability: 89%
Simulator probability: 100%
So, this is one of those times where my prediction was laughably wrong. yes, it did predict the correct outcome, but it was way too bullish on democrats in general and was comically off by the margins. The fact was, I thought we had it in the bag almost Obama 2008 style and the end result was an extremely narrow win for Biden.
Linearly, I had it at a bullish 89% probability. The simulator also didn't predict any Trump outcomes, although it did predict outcomes similar to and worse than what we actually got (like down to 280-300 electoral votes for Biden). It also did simulate outcomes similar to what we got, so it was within the realm of its predictive power.
One thing I will say is just like 2008, I spammed the button to see if I could generate ANY Trump outcomes, and I did see like ONE. So that's better than 2008, but we're still talking a probability of generating a Trump outcome at less than 1%.
I guess the simulator was technically right since Biden won and we did get similar and even worse outcomes from it, but yeah, nothing that gave Trump any sort of win really.
2020 (Corrected/Revised prediction)
Outcome: Biden won
Linear probability: 62%
Simulator probability: 95%
So, this one is interesting. Linearly, I only would give Biden a 62% probability in my updated prediction after the fact. And it was more accurate and true to the final outcome in my opinion. The simulator still spat out mostly Biden outcomes though. I guess by random probability it was a hard map for Biden to win. once again a point in favor of the linear trend/wave model of elections, as opposed to the randomness model. Of course, the randomness model CAN predict similar outcomes again, but also while spitting out a bunch of nonsense outcomes that werent close.
Once again, both models have advantages. The linear model is probably more correct ON THE WHOLE. It generally gives you a better idea, in my opinion, of whether we're gonna get a close, competitive election, or a blowout.
The simulator has unique advantages though. It does generate random outcomes that can simulate things that the linear model misses, like a D+2 state going red but a R+1 state going blue or something. But still, it tends to generate the right outcome by brute forcing a lot of wrong outcomes. So you'll never know before hand which outcome is actually correct. Finding the right one is like finding a needle in a haystack. It's in there, but unless you have the power of hind sight, you'll never know which one is actually right.
In a sense, the linear model is probably more like the oracle from the Matrix, knowing what the actual outcome is, roughly, but lacking in specificity. The simulator model is like the architect in that it will brute force tons of possibilities and one will be correct, but you'll never know which one it is before hand.
With that said, let's discuss some of the 2024 predictions based on this simulator:
Biden 2024 (as of July 21, 2024)
Predicted outcome: Trump wins
Linear probability- 87% chance
Simulator probability- 96% chance
So yeah, it's no mistake that Biden was heading to almost certain defeat prior to his dropping out of the 2024 race. Trump had a commanding lead in the polls, to the point Biden had almost no chance of coming back...yet a small minority of simulator outcomes still had a Biden win, strangely enough. What did these predictions look like? Well, they were off the wall, and most of them involved flipping Florida or Texas in some way. So...not very likely to actually happen. But not impossible. I'm just gonna call it and say Biden was heading to certain defeat here.
Harris 2024 (as of last week)
Predicted outcome: Split
Linear probability- 52% Trump
Simulator probability- 58% Harris
Here, we had the closest possible prediction. Razor thin margins, Trump slightly favored, but then the simulator just favored Harris for whatever reason. Once again, I think it's stuff like the possibility of flipping Texas or Florida that's doing it for her here. Not very likely in the linear model but in the simulator? It's a random possibility.
Harris 2024 (as of today)
Predicted outcome: Harris wins
Linear probability- 58%
Simulator probability- 73%
Once again, we're seeing the favored candidate in the linear overperforming in the simulator. That seems to be a weakness of the simulator. Still, the simulator likely has the real outcome in there somewhere. We just don't know which one it actually is. And just because Harris is favored, don't mean she'll win. Look back at 2016. 56% chance Clinton won, 80% in the simulator, and she lost. Still, one of those 19 simulator outcomes for Trump came close to or accurately predicted the actual outcome. So, yeah. It can happen.
So that said, what do I expect out of 2024? Well, my actual linear probability is the ACTUAL probability, but the simulator can help identify interesting outcomes of note that should be considered. I don't think it's good as a predictive tool for the winner. But it's good at predicting how things can go wrong (or right) for either party. Most outcomes it spits out will be wrong in some way, sometimes massively so, but it also will be able to identity outlying outcomes that CAN happen that my linear brain can't necessarily predict.
Conclusion
And yeah. Again, the linear model is more accurate on the whole, as the simulator is often too bullish on the winner, but if we get strange outcomes that seem to confuse and confound, similar to 2016, the simulator probably predicted it at some point if i run enough simulations. As such, i won't use it as my primary predictive tool, but it is potentially useful at identifying outlying outcomes.
No comments:
Post a Comment