We use cookies on our websites. Information about cookies and how you can object to the use of cookies at any time or end their use can be found in our privacy policy.
Huawei cheats in benchmarks tests... so what?
Huawei P20 Pro Hardware Opinion 4 min read 9 comments

Huawei cheats in benchmarks tests... so what?

Update: Official statement from Huawei added

Cheating. The word reminds me of fielding too many players in a football game or using answer sheets in school. But even in the world of technology, there is a lot of cheating going on by companies. This time Huawei got caught... so what?

Anandtech's experts are considered smart and incorruptible when it comes to chips, processors and laboratory tests. Now, the tech heads have discovered that Huawei is boosting the performance of both its own and the Honor smartphones in benchmark tests - making the smartphone run faster than it does in everyday life if no benchmark app is recognized. Anandtech confronted Huawei directly with the findings, and the manufacturer does not deny the behavior. Quite the contrary, in fact.

Huawei provides an explanation, or let's call it an excuse, that everyone does it anyway, especially the competition in China. You can't score points with customers with honesty in this area. That may be true, but you shouldn't get caught. Take a look at what happened to Volkswagen after the exhaust emissions scandal in the automotive industry, for example. But at the end of the day, Huawei is right: These tricks are nothing new in the mobile industry, and even if no one has been conspicuous about them recently, it is an open secret that everyone cheats the benchmark tests.

Everybody cheats and, in the end, everyone loses

This whole saga reminds me a bit of professional sport. When so many athletes dope, as in the Tour de France for example, it ruins the reputation of the sport, but the competition between them is no less fair than it already is due to the varying resources or financial backing the teams have.

Just like in the Tour de France - and this is really just one example, I could have mentioned many other sports and events - where I find it a pity that the competition suffers from fraud and the sport comes into disrepute, I just don't really care about smartphone benchmarks anymore. There is no well-known manufacturer that doesn't try to present a good benchmark score. Huawei is anything but alone in this.

wikipit benchmarks2
Benchmarks are not the key to performance wisdom. / © AndroidPIT

Huawei says it always prioritizes the user experience

Huawei reached out to us to provide the following official statement on the story:

"Huawei always prioritizes the user experience rather than pursuing high benchmark scores – especially since there isn’t a direct connection between smartphone benchmarks and user experiences. Huawei smartphones use advanced technologies such as AI to optimize the performance of hardware, including the CPU, GPU and NPU.

"When someone launches a photography app or plays a graphically-intensive game, Huawei’s intelligent software creates a smooth and stable user experience by applying the full capabilities of the hardware, while simultaneously managing the device’s temperature and power efficiency. For applications that aren’t as power intensive like browsing the web, it will only allocate the resources necessary to deliver the performance that’s needed.

"In normal benchmarking scenarios, once Huawei’s software recognizes a benchmarking application, it intelligently adapts to “Performance Mode” and delivers optimum performance. Huawei is planning to provide users with access to “Performance Mode” so they can use the maximum power of their device when they need to.

"Huawei – as the industry leader – is willing to work with partners to find the best benchmarking standards that can accurately evaluate the user experience."

Without standards and control, tests are worth only half as much

Yes, of course, the tests give us an indication of how powerful a platform is. And yes, they allow for a certain level of comparability. But as long as the tests are not standardized and controlled in any form, they provide nothing more than an educated guess and are too easy to manipulate. They are important as just one of many indicators in a performance test but are by no means decisive. The scores should only to be considered rough guides. A few hundred points either way does not make much difference.

No matter how good the benchmark scores are, if a smartphone shows weaknesses in everyday life, customers and testers will definitely notice that. That's real life, and no matter how fabulous your stats are, it won't help you. Let them all cheat, we'll still find out whether the smartphones are fast or lame when we use them in real life. You can always rely on that.

What do you think about the relevance of benchmarks today? Let us know in the comments.

Via: Areamobile Source: Anandtech


Write new comment:
All changes will be saved. No drafts are saved when editing

  • If ANYONE buys a smartphone, automobile, toaster oven, microwave, TV based on some goofy "benchmark" test, then, well, that is YOUR mistake. Yeah, cheating on benchmarks goes on all the time, as does the "shot on x" promotions to show how good a smartphone camera is, but, they fail to tell you it was done with professional lighting, usually with an extra EXPENSIVE add on lens, post processing etc.

  • One more reason not to buy anything Huawei.

  • yep many have done it too, even samsung and htc in the past. oneplus, oppo etc. But as for experience with huawei devices they're pretty good, I'm considering the upcoming mate 20 pro for my next phone

    • I would to, but living in the USA, and not wanting to jack around with unlocking, rooting, flashing (I gave up on those years ago), I doubt Huawei will have much love for our market. Mate 2, 8 & 9 for me have been the best phones I've ever owned, even giving Huawei's poor track record of updates. Right now my daily driver is the Essential PH-1 which has been updated QUICKLY at the beginning of each month. Got the September patch on the 4 of this month.

      • essential ph 1 is a pretty interesting phone imo. Kinda sad it wasn't successful on the market though. :/

  • storm 9 months ago Link to comment

    It raises the question of where else are they lying about the product. RoHS? Toxins in the case? Radiation measurements?

    • Bastian Siewers
      • Admin
      • Staff
      9 months ago Link to comment

      Well, in that case you basically need to ask that question for nearly any manufacturer. Don't make the mistake and think that everyone else isn't cheating ;)

  • They all cheat once in awhile but Huawei & One plus have been the biggest offenders! Getting caught with your hands in the till is always suspect, Beware!