Huawei caught cheating in a benchmark test

Huawei has been caught artificially boosting the performance of some of their mobile phones while running 3DMark, the popular benchmarking app. UL, the company behind the benchmarking application, has delisted P20, P20 Pro, Nova 3 and Honor Play after they were found to be artificially boosting their performance while running the benchmark.

The behavior was first noticed by AnandTech and it was later confirmed by UL in a press release that Huawei was indeed cheating on the benchmark and its phones were detecting the running benchmark application to temporarily boost their performance. As a punishment, 3DMark has removed these phones’ rankings from its leaderboard and updated their listings on its website with a note that the phone’s “manufacturer has not complied with UL benchmark rules.”

Huawei has admitted to using AI to increase their performance based on certain intensive tasks which might explain how the benchmarking app experienced artificially increased performance. However, testing done by UL shows that a private version of 3DMark application experienced no such spikes in performance, throwing shade over Huawei’s claims of using an AI that is smart enough to recognize hardware usage to tweak performance automatically.

Huawei is not the first phone manufacturer to artificially enhance benchmark scores. Brands like Samsung have previously been found meddling with phone’s performance during the benchmarking process, which raises further questions about the authenticity and importance of benchmarking tools while judging a phone’s performance.

UPDATE: Huawei has released the following statement to clarify their position on this matter:

“Huawei always prioritizes the user experience rather than pursuing high benchmark scores – especially since there isn’t a direct connection between smartphone benchmarks and user experiences. Huawei smartphones use advanced technologies such as AI to optimize the performance of hardware including CPU, GPU, and NPU.

When someone launches a photography app or plays a graphically-intensive game, Huawei’s intelligent software creates a smooth and stable user experience by applying the full capabilities of the hardware, while simultaneously managing the devices’ temperature and power efficiency. For applications that aren’t as power intensive like browsing the web, it will only allocate the resources necessary to deliver the performance that’s needed. In normal benchmarking scenarios, once Huawei’s software recognizes a benchmarking application, it intelligently adapts to “Performance Mode” and delivers optimum performance. Huawei is planning to provide users with access to “Performance Mode” so they can use the maximum power of their device when they need to.

Huawei – as the industry leader – is willing to work with partners to find the best benchmarking standards that can accurately evaluate the user experience.”

Image Source: CNET