Monday, June 17, 2024

Discussing how AMD "lies" in benchmarks and how this is nothing new

 So, this is gonna be a time where I briefly indulge the anti AMD circlejerk that I sometimes do with the CPU market. Hardware unboxed reported recently how AMD "lied" in their new benchmarks for zen 5. And I'm just gonna say it, AMD has always done this, and the fan base for AMD is a cult. 

And yes yes, I know I defend them a lot in the GPU market recently. I kinda gotta when nvidia is deciding to just go full monopoly and charge whatever they want, but let's be real. AMD arent the "good guys", they're the lesser evil. They're the democrats to Nvidia's republicans. Let's not glorify it (and intel is like the libertarian party in the GPU space, not significant and pretty dysfunctional). 

But they've been doing pretty decent in the CPU space. They even took the lead recently with the 7800X3D and have been kinda leaning toward being either competitive or ahead since they released the 5000 series back in 2020. I strongly considered buying AMD this time given their 7800X3D, but given I was buying from microcenter, and given the reviews on the AMD combos had a ton of negative reviews related to memory stability, I decided to back off and go intel again and got a 12900k. I know some people go "why buy intel"? Well, when I can get a 16 core processor for $200 and it gets the same gaming performance as the AMD 8 core model (7700x) at the same price, and the AMD CPUs are having issues, why not? Sure the 7800X3D is better, but it's not worth getting a lemon over. But I digress.

So...basically the offending benchmark had them comparing their new 5700 GT (basically a 8 core non X3D 5000 model) vs the 13700k, claiming they tied. The problem is, the 5700 GT is certainly gonna be worse than a 7700x, and the 7700x gets the same performance as my 12900k. And the 13700k is a faster more modern 12900k. So....why would now almost 2 gen old 8 core perform the same as a faster 16 core in gaming? Even ignoring the fact that games dont use that many cores, the 13700k should straight up have faster cores, as HW unboxed points out. Which is the thing, they the benchmark with like a RX 6600, ie the weakest competent GPU they got, and something that is like 1/4 as powerful as a RTX 4090. So they induced a GPU bottleneck to claim the results are the same. And yes, you get the same results, because you're measuring the CPU, you're measuring the GPU bottlenecking everything. 

People act like OMG this is the only time AMD has ever done this! No, they do this every time they release a product. I call it their "fun house mirror" benchmarks. They also cite massive performance gains on these kinds of slides only for reality to be fair more underwhelming.

Remember how the 7000 series was supposed to get 50% more performance per watt than the 6000 series in the GPU market, only for them to perform about the same per watt? (compare to the 6600 XT/6650 XT). 

 Or how about how early Zen was supposed to match the 7700k in gaming back in the day, but only when again under GPU bottlenecked scenarios? And then how people would be like "oh but this is realistic, who games at 720p?" Uh, first of all, that should be an option for people. Second of all, because when testing CPUs, you wanna test in a CPU bottlenecked scenario. Basically, the reason AMD liked to force these GPU bottlenecked comparisons were to hide the shortcomings of their hardware. 

They dont need to do this when they're winning. And they price their products accordingly. I mean, when they finally took the gaming performance crown with the 5000 series, they raised prices. Their 7000 series prices were also quite high out of the gate, and another reason i ended up going intel. I could only afford AM5 through a microcenter combo given how street prices are $250+ for a CPU, $200 for a motherboard, and another $100-200 on RAM, and when their combos had QA issues, again, give me the intel product that works. 

But suddenly with 9000 series, there's rumblings about how the 7000 X3D processors are faster for gaming and how they're gonna make cheaper MSRPs this time. Why are they doing this? Because Zen 5 probably isnt that good. And here's why. Even though they're reporting similar gains in IPC as Zen 3 and Zen 4, there were other gains outside of that with those. Zen 3 finally solved the latency issues that plagued Zen since its inception making it mediocre for gaming (thats when they jacked the prices up with the 5000 series, they knew what they had), and then they further improved on them with the X3D variants where those performed like a 7000 series CPU. And then the 7000 series could match 5000 X3D at a similar price point, because on top of IPC they basically switched to DDR5 RAM, which is faster than DDR4. And THAT always provides a sizeable increase in and of itself.

So Zen 3 had a 15%ish IPC increase + latency improvements + X3D, and Zen 4 had a 15% IPC increase + DDR5 improvements. Zen 5 is basically just that 15%ish IPC increase with no clock speed increase and no other major improvement to be had. So it's gonna be 10-20% faster than current gen and that's it. A pretty muted increase all things considered. I mean, we're going back to mid 2010s intel style increases there. So this is gonna be a relatively ho hum generation that we can all skip and sleep through, and they know it. So dont be fooled by marketing.

And especially dont be fooled by them comparing a remake of a 2 generation old processor from 4 years ago with a slight clock speed bump going up against a fricking modern i7. Dont get me wrong the 5700 and 5800 XTs or whatever are likely okay processors, but yeah. They're not as good. Keep in mind, a 5800X3D, which has the 20-30% performance jump over normal 5000 series processors, competes head to head with my 12900k. There's no fricking way a non X3D 5000 series processor is matching something as fast or faster as their 7000 series processors. It just ain't happening. 

Heck, I'll just say this. When it comes to the hardware market, NEVER EVER BUY INTO MARKETING. ALWAYS WAIT FOR REVIEWS FROM TRUSTED PROFESSIONALS WHO KNOW HOW TO BENCHMARK STUFF PROPERLY. Ya know, like hardware unboxed, gamers nexus, etc. Even then, read into their methodologies a bit. HW unboxed kinda showed me most benchmarkers tend to underscore the intel 12 series because they use slow RAM and crap, and that clock for clock and memory clock for memory clock, they're like 99% as fast as 13th/14th gen intel. It's literally all clock speed increases and memory speed increases that account for the difference, with MAYBE the extra L2 cache being relevant here and there, but yeah. What makes 13th and 14th gen faster than 12th gen is faster clock speeds and memory speeds. You get the same memory and you get a much smaller difference. You dont overclock and it probably will only be like a 8% difference. 

So yeah. Keep in mind that kind of nuance too, but all in all, read reviews, read their methodologies, see what they did. and go from there. Just dont fricking fall for marketing. Especially AMD marketing. They love to compensate for weak products with weird funhouse comparison benchmarks that arent technically wrong if you read what they did, but don't make any sense at all in practice.

No comments:

Post a Comment