So, I recently did a post about how the evolution of election predictions are like an evolution in economic growth, where labor saving machines save labor, but then create more labor, and we kind of gotta consider the tradeoffs. This post, I'm going to outline a bit more about how this actually is creating more work, and how while it makes my predictions more sophisticated, this doesnt necessarily mean it's better.
So...I specifically wanna focus on my simulator for this part. I largely believe that for better or for worse, my excel spreadsheet idea is more efficient and allows me to more quickly compile more data which saves me time in the long term. While it took me a day of work to get it up and running and some additional time tweaking it, most time spent on it is actually using it. If anything, I update it multiple times a day now and am obsessed with following polls. This isn't necessarily mean it creates more work, that's more my passion and obsession with the topic, but I do spend more time on it as my means to track stuff has become more advanced. So I end up working in other ways. I'm fine with this, and this leads to a better final quality of the product, but yeah. I do see how I don't save a ton of time in some regards. But given it's time I'm willing to spend, what's the problem.
The simulator, I'm a bit less inclined to believe it's worth it. As a matter of fact, this is why I abandoned it after some early testing of it back over the summer and only brought it back for the final weeks of the campaign. Because that big simulator? The one that does 1000 outcomes at once? It's actually...well...a lot of work.
So I made 3 versions of it. 2 presidential ones (the second one just looks at the big 7 states) and a senate one. Each took two hours to make. TWO. HOURS. And I know, from the early testing on the beta version, that if I add or remove states or mess with my data beyond just swapping out a few margins and reordering the states, that actual changes to it...are gonna break it. It will just blank out, the cells won't line up, and it will take even more significant amounts of time to fix. So that's MORE hours duct taping a bad product. This is kind of true with employment. Say you have one guy whose job it is to push a button to get simulations and then record the data. You can have a guy sit there for an hour recording 1000 simulations one at a time. Or you can have a guy work 2 hours to give you 1000 simulations with the push of one button. Sounds efficient, right? Well, sure, you just displaced the job of the guy whose job it was to push the button, but now you have to hire someone who is more educated to fix the simulator and keep duct taping it whenever it breaks. A lot of BS jobs are duct tapers and in my hypothetical polling business, the job of button pusher would be replaced by mr programmer who has to fix my spreadsheet whenever it breaks. So, in some cases, I'm saving labor, but also creating other labor. This is what's meant by automation "creating more jobs", often times, youre replacing a guy in a factory who pushes a button with a technician who has to fix the automatic button pusher you created because it breaks every 5 minutes. And we seem more sophisticated but we're not. Even worse, in a world where automation displaces low skilled work (pushing a button) with high skilled work (button pusher technician), the guy who originally pushed the button struggles to find another job while we're paying the technician the big bucks due to higher skills in being able to fix this simulator. You know what I'm saying?
Since I'm doing it all myself, I can make an objective statement about whether the tradeoff is worth it, and generally, it is not. I mean, this big 1000 outcome simulator is NICE. It gives me a lot of info and the data analysis alone into my own model is worth the few hours of effort alone, but let's be honest. It takes me 5 minutes to generate 100 outcomes like I'd do. And now I can do 1000 in like the 5-10 seconds it takes for the data to refresh in excel, but is it actually helping? In a way it's more impressive, and it does again give me insight into my model id otherwise not have. But in some ways, if i were to use it long term, no. Because while i can push the button once and get the data it would take me an hour to do with my original simulator (and even then not have the in depth statistics on it), the amount of time it would take me to keep this thing up for longer than the final couple weeks of election season isn't really worth it.
If anything, I lose some intimacy the original simulator had, where i can see each individual outcome as it happens and more easily study the outcome to see why things went as they did. Now I get so many outcomes that i cant look at them all, and the data is presented in a format that it's hard to read what individual outcomes spat out actually do.
Even more so, I'm not fully convinced 1000 outcomes really tells me much more than your typical sample of 100 would. I am seeing how even among samples of 1000 the data varies by as much as you'd expect with a 3-4 point margin of error, with me getting outcomes that can range from harris narrowly winning, to outcomes roughly twice as divergent as the one I posted. I admit I literally did push it a few times to get a nice middle of the road outcome that's representative of where the median outcome is, but generally speaking, if I just did a sample of 100, I could get outcomes similarly. Heck, I have the small simulator still with the same data. I'll spend 5 minutes right now just recording your typical sample of 100 for comparison's sake:
I got 41 Harris outcomes, 57 Trump ones, and 2 ties. I know from the big simulator that outcome is a little more of an outlier in favor of trump compared to a random sample of 1000, but it's not that extreme. I got the equivalent of 52 Trump ones, 45 Harris ones, and 2 ties so it's about 4 points closer. But yeah. If I hit the button on the big simulator, some outcomes do that. Then some outcomes will be like 490 Harris ones, 480 Trump ones, and 30 ties or something. it varies.
Doing the 1000 simulator again I get 530 trump ones, 452 harris ones and 18 ties. I guess with a larger sample size, hitting it a few times, I do get more consistency than i get with a smaller one, but statistically, that's the differences between a small sample and a large sample in polling. Small has less error, big has more.
So...yeah. I guess 1000 IS better. And the simulator for mass simulations IS better. But is the difference worth the extra work? Probably not. And I get "good enough" results just spending 5 minutes on my much simpler and more robust small simulator that doesn't break whenever i touch anything with it. Or if it does break I can fix it in exponentially less time. And I'm already familiar with how it breaks when it does and I can fix it in like 2-5 minutes itself. There are tradeoffs.
I guess if you care about maximizing the quality of the product (as our workaholic culture often does), the big one is better even if more labor intensive, but I kind of think if you can get something "close enough" (like say 80-90% of the quality) with like 1/10 of the effort, that's probably more efficient.
So that's the tradeoff of society. We care about maximizing growth at all costs and do stuff that's super labor intensive in order to accomplish it, when we could just live a little simpler and work a whole lot less. I would argue current society is past the efficiency curve to some extent of work vs quality. Whereas society is obsessed with quality even if they need to throw tons of labor at it in order to make it work. Then they tout over how many jobs they create and how we have a society that keeps everyone working endlessly. Idk, I think it's nonsense.
Anyway, that's my rambling for the day about how society works too much.
No comments:
Post a Comment