OnePlus manipulates popular Android benchmarks
XDA has discovered that OnePlus is manipulating benchmarks with its OnePlus 3 and 3T smartphones. That has been the case since the merger of Hydrogen OS and Oxygen OS. The cheating yields a few percentage points of profit. OnePlus promises to stop the manipulation.
XDA discovered the manipulation while testing the Snapdragon 821-soc on various devices. It was noticed that the chip in the OnePlus 3T continues to run at a higher clock speed after opening certain apps. The fast cores of the soc continue to run at a minimum of 1.29GHz after opening the benchmark apps and the economical cores do not go below 0.98GHz. This is also the case when the SOC is not taxed at all. The Xiaomi Mi Note 2 and Google Pixel XL, which have the same soc, did not show that behavior.
After further investigation, XDA discovered that the OnePlus device exhibits this behavior when starting specific software. From a ROM dump, the developers were able to determine that the manipulation is performed in Geekbench, AnTuTu, Androbench, Quadrant, Vellamo, and GFXBench.
With the collaboration of Geekbench developer Primate Labs, XDA was able to run a test to compare what the cheating yields. Primate Labs made an alternate version of Geekbench 4, called ‘Bob’s Mini Golf Putt’. By changing the app on a number of points, the software on the OnePlus phone no longer recognizes it as the benchmark. If this app is used on the OnePlus 3T, the soc will clock back to 0.31GHz if there is no processor load.
The profit that OnePlus makes from cheating is a maximum of five percent in Geekbench and that is only visible if many runs are performed in succession. After seventeen runs, the OnePlus 3T scores 4244 points with manipulation. Without manipulation, that is 4034 points.
According to XDA, the manipulation has been done in Hydrogen OS for a long time and OnePlus has been cheating on the OP3 and OP3T since that OS merged with Oxygen OS. That merger took place in September. Community builds that appeared after that and the Nougat release for the OnePlus devices therefore contain the adjustments that artificially increase the scores.
XDA has confronted OnePlus with the tampering and the smartphone manufacturer has said it will disable the mechanism in future OxygenOS builds. According to the manufacturer, it is a function that is also used in certain games, to make them run better. OnePlus will not change anything about this, because according to the manufacturer this benefits the user.
XDA criticizes that solution, because OnePlus optimizes its smartphones for specific apps, instead of in its entirety. It is possible that current games run better due to the functionality, but if new games are released, for which OnePlus makes no adjustments, that advantage will expire. According to XDA, this results in inconsistent performance of the phones.
XDA has also tested several other smartphones with Geekbench’s “Mini Golf” version to see if multiple smartphone manufacturers are cheating on the benchmark. That turned out not to be the case with devices from HTC, Xiaomi, Huawei, Honor, Google and Sony, among others.
XDA does say that it has found indications for manipulation in smartphones of ‘a number of other brands’. The developers are doing further research to prove that. In any case, the Meizu Pro 6 Plus, with Exynos 8890-soc from Samsung, shows very suspicious behavior.
In 2013, a fuss arose after it emerged that several major manufacturers of Android smartphones were manipulating benchmarks. That ball started rolling after it turned out that Samsung adjusted the clock speed of the Galaxy S4 in benchmarks.