Smartphone companies cheat benchmarks? Test of 22 devices (using Dontkillmyapp)

One of the features on which smartphone manufacturers battle hard is undoubtedly in the camera sector. But there is another area that sometimes goes a little muted, which is that of the benchmark. Many companies are committed to providing more extensive batteries and, at the same time, offering faster and faster-charging systems.

Yet there is another way to improve the benchmark of smartphones, and companies know it well: act on the software. Google in Android 6.0 Marshmallow introduced a system called Doze to manage the behavior of apps in the background. But not all apps make good use of this feature, and a poorly designed app could still consume too much energy for no reason, even when not in use. 

This is why smartphone manufacturers have often made more aggressive software than Google, with ad hoc solutions that close applications and processes faster and more aggressively than you’d expect. This allows you to save battery and to safeguard yourself from badly coded apps. Still, it also makes life more complicated for the more capable developers. They are faced with software that prevents their apps from working well.

The developers of Urbandroid and other teams decided to report it to Google after having many problems and working on various solutions to get around them. And then creating a tool that would numerically evaluate the behavior of the different Android-based operating systems. Thus Dontkillmyapp was born, an app that allows you to find out how aggressive your smartphone’s software is.

Once Dontkillmyapp started, the software remains in the background for a period of time (from 1 hour to 8 hours). It assesses how the software manages the presence of the app itself in memory and its running process. Then it gives reports of these data called Work and Main, indicating a percentage value (100% means that it has always worked for the duration of the test). In addition, it makes an additional test on alarms. The data that interests us, however, are the first two.

Dontkillmyapp on the below section shows a list of companies that behave “better” or “worse.” Check out the smartphones tested by version 1.5 on 22 different devices from 22 different companies. The tests conducted below were for both 1 hour and 8 hours. The results are very interesting, thanks to Emanuele Cisotti for conducting the experiment. Check out the report below, divided by brands in alphabetical order.

ASUS

As for ASUS, testing is done on the ROG Phone II, between the two top-of-the-range smartphones of the company. It has the most personalized software and a battery that has always guaranteed us excellent results in terms of the benchmark. We see that the Work / Main test reports 100%, which means the smartphone perfectly manages the apps in the background.

Google

The test on the Google Pixel 4 XL is not surprising: 100% for Work and Main. Not giving much importance to the voice Alarm, which, however, signaled in the test (and reported it for completeness) to be 100% on the Android 11 beta.

Honor

As for Honor, the test was conducted for Honor 9X Pro (without Google services). The result is not at all exciting. In the one hour test Work / Main is 3% / 1%, which (unique case) increases in the 8 hour test to 3% / 2%. The process and the app got closed soon after a few minutes once the phone has gone to standby.

• See also: Performance test on HONOR 30 Pro+ (Benchmark, Gaming, Power)

Huawei

Results similar to Honor (obviously) also for Huawei. We tried Huawei P40 Lite 5G, Huawei P30 Pro (with Google services), and Huawei P40 Pro +. The results show how the process and the app are closed quickly after the screen has been locked.

LG

For LG, the test conducted for the LG Velvet device. The smartphone showed a value of 100% in all tests, both from one hour and from 8 hours. Therefore, the implementation of the LG UI did not make any changes to the energy-saving on this smartphone.

Motorola

Dontkillmyapp has been tested on Motorola Edge and Motorola Edge +. Excellent results in both cases where Work / Main have always reported a value of 100%. No wonder, given that Motorola has always worked for software very close to Google. However, Congratulations to Motorola for these two Edge / Edge+, which achieve truly out-of-range benchmark values ​​without using any “tricks.”

OnePlus

OnePlus is the only company that has behaved differently from those who have achieved stable values ​​at 100%. And even for those who didn’t reach 10%. The 1-hour test, on both OnePlus 7 and OnePlus 8 Pro, showed a 100% full result. However, if the phone remains on standby for many hours, the software becomes more rigid, closing some processes and allowing it to be run again later, probably by “freezing” them. OnePlus 7 has a result of 37%, OnePlus 8 Pro of 71%. We tend to believe that the results do not depend on the phone but on the apps that run and create notifications or activities during the duration of the test.

OPPO

For Oppo, the test conducted for OPPO A72 and OPPO Find X2 Pro. In both cases, the system was very aggressive on the app and its process. The first in the 1h and 8h tests reported as Work / Main results only 11% / 6% and 4% / 1% (obviously the longer the time, the less proportionally it will remain active). For the top of the range the results are even lower: 4% / 4% and 1% / 1%.

realme

The OPPO subsidiary reports practically identical results. Realme 6, Realme X3 Superzoom and Realme X50 Pro 5G are the tested smartphones. In order the results were, 4% / 4% – 1% / 1% for Realme 6, 9% / 7% – 3% / 1% for Realme X3 Superzoom and 4% / 4% – 1% / 1% for Realme X5 Pro 5G.

Samsung

The Samsung smartphones we tested are Samsung Galaxy M31 and Samsung Galaxy S20 Ultra. The results were more than heartening with a nice 100% on the Work and Main for both tests. Dontkillmyapp points out, however, that some Samsung software releases still have a system of “freezing” the apps if they are not used for more than three days.

Sony

The Sony smartphone they tested is the brand new Sony Xperia 1 II. The results were 100% & 100%. There is no surprise, considering that the Japanese company implements software-based largely on stock Android.

Xiaomi

Surprisingly, Xiaomi manages to achieve a 100/100% score on the Work / Main test for all the devices tested: Xiaomi Mi Note 10 Lite and POCO F2 Pro. They had carried out the same test (always 100%) for Mi 10 Pro, but they didn’t include it since it was tested with version 1.4. Despite the MIUI interface very different from Android stock, Xiaomi smartphones, at least those officially sold in Europe, seem to have shaken off the obnoxious reputation for too aggressive software with apps.

Wiko

Let’s get close with Wiko, and it’s View 4 Lite. The almost totally stock implementation pays, and the “vote” is 100% / 100% on both tests and for both durations taken into consideration. Not bad considering the benchmark of these smartphones.

Conclusions

With the data in our possession, we can say that the Huawei / Honor and the OPPO / Realme are the most aggressive in closing the apps in the background. The only company to pose with an intermediate solution is OnePlus, which behaves normally when the smartphone is on standby.

However, when this state lasts for several hours, it still deals with blocking these processes. All the other companies (ASUS, Google, LG, Motorola, Samsung, Sony, Xiaomi, and Wiko) have instead obtained results in line with what should be.

However, this does not mean that OPPO and Huawei smartphones do not work well or should be avoided. In all our tests, we report problems with the apps in the background, and now we hardly ever encounter these problems with these manufacturers. Even those who own smartphones from these companies will be able to confirm what we say. This is because OPPO and Huawei still work well with their software to allow the most common apps to work without problems. Instead, they are behaving more aggressively with those that would consume too much energy.

However, this approach is not without risk. It may, in fact, happen that well-developed apps still suffer from operational problems due to early closure by the system.

Therefore, this test does not overturn any opinion on the various devices and manufacturers, but still highlights the behavior of their software, which is still correct for users to know before their purchase.