Sign in

Sign in to confirm

Have you forgotten your password?

... or login with Facebook:

Don't have an AndroidPIT account yet? Sign up

Your Galaxy S4 isn't as good as you think: Samsung cheat on benchmarks

Kris Carlon

As if exploding batteries and chargers that burn down apartments weren't enough to tarnish the Samsung Galaxy S4's reputation, now we find out that it appears Samsung faked benchmark results for the device through some clever 'optimization' (code for cheating). So if you take benchmark tests as gospel and thought you were getting what you saw in those tests, you may have been a little hoodwinked.

Galaxy s4 camera
The Galaxy S4 promises more than it delivers./ © AndroidPIT

A very clever poster in the Beyond3D forums by the name of Andrei F. was doing some GPU overclock and voltage control tests on the S4 and made a surprising discovery: it seems Samsung 'optimized' their GPU and CPU settings to run at higher speeds, specifically when benchmark tests were run. This means the numbers you see on benchmark tests are higher than you can get yourself.

In fact, when running GPU tests, Andrei found that the 533 MHz GPU clock that had been widely reported as the S4 benchmark, was only able to hit those heights when a specific benchmark test was launched to assess it. Any other time, I repeat, any other time, the GPU clock never ticked beyond 480 MHz – that's a 10% overstatement of performance. What this means is the S4 GPU only ever runs at 480 MHz, because since when do we consider running tests on a phone to be the same as using it?

You can clearly see the different GPU clock speeds depending on which benchmark test is launched. / © AnandTech

What's worse is that he continued his testing and found basically the same thing with the CPU clock, but with a twist. Whereas with the GPU the higher frequency was only available to certain benchmark apps, with the CPU clock the 'optimization' was worked so whenever one of these benchmark tests was run, that the S4 would automatically pump up the power so it would appear the phone ran at a much higher processing level than it normally would. Kind of like puffing out your chest when a pretty girl walks by.

Basically, as soon as the benchmark test was launched, the S4 powers up the more powerful four of it's quad-core processors (in the Exynos 5 Octa device, the ARM Cortex A15 cores) and keeps them idling at nothing lower than 1.2 GHz, even if the device is just sitting there, doing nothing. Of course, when subjected to actual benchmarking, the device would then beef up further, and the overall average result would be tipped higher. Outside these benchmarks the results are dramatically lower, in many cases, a quarter of the speeds.

Screenshot 2013 07 30 09 37 06 678x452
This reveals the CPU runs higher than normal whenever the benchmark is launched. / © AnandTech

The same is true for Qualcomm S4's equipped with Snapdragon 600 cores. As AnandTech's Brian Klug reports: ''In these select benchmarks, the CPU is set to the maximum CPU frequency available at app launch and stays there for the duration, all cores are plugged in as well, regardless of load, as soon as the application starts.'' Again, Samsung is faking the performance of the S4 to make it post higher numbers than it normally would.

Note that Klug mentioned ''select benchmark tests.'' What this means is that for some strange reason, Samsung only 'optimized' the S4 for certain benchmark tests and not others. This loophole helped uncover this performance enhancing treachery. By only cheating with some benchmarking tests they left the door open to compare results against all benchmarks to identify the discrepancy. Rather sloppy work Samsung. If you're going to cheat, at least do it properly.

If Samsung had been more consistent with their cheating this may never have been discovered. Indeed it took this long for anyone to notice, and now only because of the 'select' choices of Samsung. Andrei F. happened to be delving into this area and, due to another oversight by Samsung, noticed that the GPU clock never hit 533 MHz. This oversight is more of a gimme from Samsung, however, whereby S4 owners can monitor the clock speeds of their device though an ADB command.

Samsung galaxy s4 mute
Is cheating really cheating if no one catches you? / © AndroidPIT

This provided the loose thread that before long unraveled the whole shebang. Why Samsung would cheat on benchmark tests is obvious - in some circles benchmark results are heavily relied upon to dictate what devices have better performance beyond simple hardware specs, and these results absolutely dictate purchasing behavior and tech blogging commentary.

But the strange part is that for the CPU at least, the performance is actually there – Samsung could only cheat because the S4 actually has the power available, they simply ensured the CPU performed at the highest rate when someone was looking. For the GPU however, only the benchmark tests can access the fabled 533 MHz, the rest of the time it is stuck at 480 MHz. This is the real issue at stake here: overbuild and under-perform.

As Brian Klug reported: ''Note that the CPU behavior is different from what we saw on the GPU side however. These CPU frequencies are available for all apps to use, they are simply forced to maximum (and in the case of Snapdragon, all cores are plugged in) in the case of these benchmarks. The 532MHz max GPU frequency on the other hand is only available to these specific benchmarks.''

Samsung allowed their ruse to be uncovered because not all benchmarks apps were 'optimized.' / © AnandTech

While a lot of this may not mean much to the average consumer, it does mean that we are being lied to. This essentially constitutes cheating at the mobile device Olympics, as the benchmarks tests set by various smartphones go on to be used as all manner of advertising and justification for what's worth your hard earned dollars. Fancy that, false advertising and companies lying to us to get our money!

So what does this mean? Not much, the S4 seems to be doing a good enough job on its own of destroying its reputation, this just adds a little more mud to its already debatable reputation. And most consumers won't care about a few numerical discrepancies in performance tests. The good news is that this kind of thing will be checked for more closely by testers and bloggers. So Samsung should take note that they will be caught and just keep focusing on producing solid handsets with legitimate good performance, rather than investing valuable time and money in the smoke and mirrors department.

[Update 4:15 Berlin time 31.07.2013]

No sooner had the first stone been cast than Samsung replied on the ''faking the benchmark'' story with the following statement:

Under ordinary conditions, the Galaxy S4 has been designed to allow a maximum GPU frequency of 533MHz. However, the maximum GPU frequency is lowered to 480MHz for certain gaming apps that may cause an overload, when they are used for a prolonged period of time in full-screen mode.

Meanwhile, a maximum GPU frequency of 533MHz is applicable for running apps that are usually used in full-screen mode, such as the S Browser, Gallery, Camera, Video Player and certain benchmarking apps, which also demand substantial performance.

The maximum GPU frequencies for the Galaxy S4 have been varied to provide optimal user experience for our customers, and were not intended to improve certain benchmark results.

Draw your own conclusions, but we can't help but notice their new claims mention nothing about the specific code that AnandTech uncovered that specifically states 'benchmark booster'...

Related Topics

From the forum. Join the discussion now!


Write new comment:
  • ljhaye Jul 31, 2013 Link

    Are you really surprised? They got convicted for copying the hell out of Apple so this is par for the course. In the mean time htc is disappearing for being original and honest.

  • User picture
    Kris Carlon Jul 31, 2013 Link

    Nice guys finish last. I really don't understand why HTC has crashed so hard, the HTC One is a great device with great specs, it just missed the equivalent advertising budget I guess.

  • ljhaye Jul 31, 2013 Link

    Samsung=android at 47.2% of all android devices htc stands no chance and should be gone in 18 months...

  • User picture
    Kris Carlon Jul 31, 2013 Link

    Yeah i'd be sad if Samsung squeezed HTC out of existence. HTC made the first smartphone I ever had and I still think they make good phones. I guess when the market is like this though you either dominate or die out.

  • Bojan M. Jul 31, 2013 Link

    OMG! I guess I won't be buying Samsung anymore. I can't stand this douchebaggery.

  • Sean Gomes Aug 1, 2013 Link

    The optimisation refers to the i9500 only. The i9505 doesn't have that enhancement. And is still faster than other phones.

  • jones Aug 1, 2013 Link

    good discovery, but more test should be done on other samsung devices like the note 8.0 and others, also someone should try and make a short note on how individuals can test run thier devices for authentication.

  • User picture
    Kris Carlon Aug 1, 2013 Link

    According to AnandTech this kind of thing has been happening with PCs forever. And I guess if you're putting a new car to the test you don't put crap fuel in it, but it's still a shame that we just get lied to all the time in every corner of our consumer lives. At least Android users are smart and tend to poke around and find these things out.

  • MWisBest Aug 2, 2013 Link

    This is an absolutely absurd accusation.

    The frequencies of the CPU and GPU will change according to the workload and available room under its TDP. A benchmarking app will completely max out the CPU and GPU, causing the frequencies to jack up and the extra cores to switch on. This is all controlled by the kernel, and the kernel isn't going to know what Android app is running and whether or not to jack up the speeds just because that app is running. While playing a game, the GPU frequency might not go up completely to the 532MHz mark because by doing so it might end up exceeding the chips TDP and end up drawing way too much power.

    If you really want, I'll pull out my Galaxy Nexus and (god forbid) my extremely old LG Vortex and show you it does exactly the same thing.

    Also, I'm not sure why you're confused HTC went downhill. They basically "disowned" the open source community, which was what used to set them apart from other brands. Samsung then stepped up and took the same spot HTC once had with the Android community. Samsung has started going downhill from where they once were for this too, and now Sony has really stepped up themselves, ending up having the first non-Nexus device to be included in the AOSP tree.

  • User picture
    Guy Devos Aug 2, 2013 Link

    So basically the S4 runs at max speed when an app like a benchmark needs it. I see no problem with that. Besides, NVIDEA and INTEL do this all the time.

  • Kay Zee Aug 4, 2013 Link

    Yeah I agree Moto X with dual core processor is much better than GS4