So....let's discuss how my models did for the election cycle.
Senate
A lot of people were like "how could you have guessed the exact map we got?" And we couldn't have. My model is linear in the sense that it assumes a wave uniformly washes over the country one way or another. But here, we lost Louisiana but won south carolina, we lost Nebraska, but won Mississippi. We couldn't have guessed that. No one could have. And while my model could have, I literally did an experiment where I had to have it go 523 times before it finally spat out the right map, and after 500 I applied a D+3 bias to tilt the scale a bit. So no, you couldn't have reasonably predicted the exact map.
Still, if we look at history, these maps are NEVER perfect, they're close, but 1-2 wrong is par.
Of the tossups, there were 2 of them, I got both right. That's 100%.
Of the leans, there were 7, I got 5 right. That's 71%.
Of the likelies, there were 6, I got all 6 right, that's 100%. That's a pretty good cycle all things considered.
Of the safe seats, i guessed 3 and got 3 right, that's 100%.
Governors
I'm doing governors yet because it's easier than doing the house. I am gonna be doing the house somewhat separate as that was...a mess.
With the governors, I guessed 21 different races and only got 1 wrong, that being Hinojosa winning in Texas. I thought Hot Wheels (Abbott) would hold into it, but alas, the blue wave was so massive he didn't. Again, oregon I assumed would go blue, because again, all 0 point races were estimated to go blue, and it is normally a blue state anyway.Of the 4 tossups, I got all 4 right, that's 100%.
Of the 3 leans, i got 2 right, that's 67%.
Of the 8 likelies, I got 8 right, that's 100%.
Of the 6 safes, I got all 6 right, that's 100%.
House
Oof, here's where it gets painful.
So it should be noted not all races are formally called, but we know enough where some are projecting the outcomes and we have an idea where they're going. Here I guessed 88 races and got 23 wrong. I also got an additional wrong, TN-04, which experts said wasn't in play.
Most of the incorrect races were red leaning districts that went blue. 4 of my errors were democratic races, and 19 were republican ones. 20 if you include TN-04.
Of the 4 tossups, I got only 1 right, that's 25%. (target: 50-60%)
Of the 19 lean races, I got only 9 correct. That's 47%. (target: 61-84%)
Of the 23 likely races, I got 16 right, that's 70%. (target: 85-98%)
Of the 41 safe races, I got 37 right, that's 90%. (target: 98%+)
Why did I get so much wrong? Well, at first I thought it was my model. I made A LOT of assumptions with it and it was VERY experimental. I didn't have a lot of data to go by, and I basically shifted 2024 results a uniform amount, which was difficult given that so many districts were redistrcted for this race.
But...it looks like my model performed on par with everyone else's. Nate silver, for example, made 21 errors in his forecast. I guess it happens. When I'm performing similarly to Nate Silver, i can't complain. That IS the benchmark I aspire to. but MAN this was a slaughterfest, and I should maybe use a larger margin of error with future house races if I'm gonna bomb this badly.
Total
Of 10 tossups, I got 7 right. That's 70% accuracy, above what I would expect.
Of 29 leans I got 16 right. That's 55% accuracy, below what I would expect (reversing tossup and lean numbers would be about on par though strangely enough).
Of 37 likely races, I got 30 right. That's 81% accuracy, slightly below my desired accuracy range.
Of 50 safe races, i got 46 right, that's 92% accuracy. If I subtract 1 for the one wrong not even counted race, that's 45/50, or 90%. I'll go with that to keep myself honest.
With that said, the model in 2026 underperformed expectations, precisely due to the house model. I wasn't even gonna count the house numbers, but I decided to last minute. Mainly because even if my model took a beating, I didn't really perform any worse on average than experts seemed to. My predictions were mostly in line with theirs. We just happened to have a REALLY HISTORIC blue wave that screwed up the model, and no one really got it right. So...I'm gonna count it. The model did what it was expected roughly, the data was just really that crazy.
Combining with historical data
Now we have a total of 53 tossups, with 35 correct guesses. That's a 66% success rate (50-60% par expected).
We have a total of 114 leans, with me getting 80 right. That's a 70% success rate (61-84% expected)
We have 109 likely races, with me getting 97 right. That's a 89% success rate (85-98% expected).
Like always I wont bother counting solids. While my success rate is trash here, I'm also not excluding the hundreds of literally safe races i got right that didn't even make it on the map because they weren't in place. I mean, think about it, if I got 24 races wrong in the house, I still got 411 right, which is a 94% success rate (and some people are framing Nate Silver's success rate in those terms).
All things considered, the model still LARGELY performs within expectations. it took a beating this year due to the house, but it seems like everyone took their lumps this year because of the house. It truly was a historical wave election where democrats just vastly underperformed expectations.
And yeah. That's how I performed.



No comments:
Post a Comment