Proper way to tackle and resolve "Excessive network usage (background)"

Question

Problem Background

Currently, we have facing "Excessive network usage (background)" from Android Vital report. Last 30 days is 0.04%, but we're only Better than 9%

• Last 30 days - 0.04%
• Benchmark - Better than 9%

Since only better than 9% looks like a scary thing. We decide to look into this issue seriously.

The app is a note taking app (https://play.google.com/store/apps/details?id=com.yocto.wenote), which provides an optional feature - sync to cloud in background after the app close.

This is how we perform sync to cloud in background.

1. We use WorkManager.
2. In Application onPause, Schedule OneTimeWorkRequest, with constraint NetworkType.CONNECTED. The worker is scheduled to start with delay 8 seconds.
3. In case failure, we retry using BackoffPolicy.LINEAR, with delay time 1.5 hours.
4. The maximum number of retry is 1 time. That's mean, after the app close till the app re-open again. The maximum number of execution, of sync to cloud process is 2.
5. The size of data is vary, can be few KB till few hundred MB.

---

Additional information how we perform sync

1. We are using Google Drive REST API.
2. We are performing downloading of a zip file from Google Drive App Data folder, perform data merging in local, then zip, and re-upload the single zip file back to Google Drive App Data folder.
3. The zip file size can ranged from few KB, to few hundred MB. This is because our note taking app supports image as attachment.

---

Analysis

The only information we have is https://developer.android.com/topic/performance/vitals/bg-network-usage .

When an app connects to the mobile network in the background, the app wakes up the CPU and turns on the radio. Doing so repeatedly can run down a device's battery. An app is considered to be running in the background if it is in the PROCESS_STATE_BACKGROUND or PROCESS_STATE_CACHED state. ... ... ... Android vitals considers background network usage excessive when an app is sending and receiving a combined total of 50 MB per hour while running in the background in 0.10% of battery sessions.

1. We start the background sync job, 8 seconds after Application's onPause. During that period, will the app inside or outside PROCESS_STATE_BACKGROUND/PROCESS_STATE_CACHED? How can we avoid running inside PROCESS_STATE_BACKGROUND/PROCESS_STATE_CACHED?
2. What does it mean by "running in the background in 0.10% of battery sessions."? How can we avoid such?
3. Another assumption, is sync file is too large, and using too much data. Soon, we notice this assumption might not be true. We notice according to "Hourly mobile network usage (background)", the data size is from 0MB to 5MB.

---

Questions

My questions are

1. What is the actual root cause for such "Excessive network usage (background)" warning? How can we accurately find out the root cause.
2. How does other apps (Like Google Photo, Google Keep, Google Doc, ...) which perform background sync, tackle this problem?

Kopje

Actual root cause "Excessive network usage (background)"... you are probably transferring more data compared to the other applications/competition. Are you transferring any binary data like pictures, video, music, etc? How does other apps (Like Google Photo, Google Keep, Google Doc, ...)? There are no magic tricks. If you have to transfer huge amounts of data they use compression. Did you try to compress your traffic e.g. gzip? (developers.google.com/drive/api/v3/performance) – Cammack

We are using Google Drive REST API, which I believe gzip is enabled by default. https://mcmap.net/q/664669/-download-file-from-google-drive-via-rest-api-using-gzip-compression If you look at the screenshot, you will notice the hourly mobile data usage is 0 to 5 MB. Doesn't seem like a large data size to me. – Kopje

@CheokYanCheng Worst case scenario, 5MB per hour = 120MB per day = 3.7GB per month! Is there any chance you can switch to wifi only by default? Lots of apps only sync on wifi. – Petrolatum

@CheokYanCheng JakeSteam said it, 5MB per hour is too much. Maybe you should consider, if possible, to send only delta differences (like on incremental backup) – Cammack

@Cheok-Yan-Cheng If i get it correct, it is 7587 sessions of 0-5MB per hour. Even a 0.1Mb session is 0.1Mb x 7587= 758.7MB. That's huge. Try to concentrate on the sessions in the background. As you have not posted any code related. Can't help much further. Thanks. – Weiss

@VanshajDaga 7587 sessions is not per user, but total number of app users. – Kopje

Answer 1

For your first question, "Excessive network usage (background)" is triggered when:

... an app is sending and receiving a combined total of 50 MB per hour while running in the background in 0.10% of battery sessions. A battery session refers to the interval between two full battery charges.

Source

To identify what is causing this, try using Battery Historian to analyse your app's battery usage over time. For us, it helped identify a repeating wakelock we didn't intend to introduce.

Here's an example of the output, showing us that excessive BLE scanning is causing a major battery impact:

---

For your second question, WorkManager is likely what you are after, as you correctly identified. This allows you to schedule a task, as well as a window you'd like it to occur in. Using this allows the OS to optimise task scheduling for you, along with other app's jobs. For example, instead of 6 apps all waking the device up every 10 minutes for their hourly task, it can be scheduled to happen for all 6 apps at the same time, increasing the time spent in doze mode.

Notice the screenshot above includes a "JobScheduler Jobs" tab. After running an analysis you'll be able to see how your jobs are actually performing:

I've previously used Firebase JobDispatcher with great success (tutorial I wrote), which extends the OS' JobScheduler API and is ultimately similar.

I see you're using WorkManager now (Jetpack's version of JobDispatcher), but with 8 seconds there's no chance for the OS to optimise your jobs. Is there any capacity of scheduling them with a minimum of a few seconds, and as large a maximum as possible?

---

Further improvements

However, your current task scheduling setup may not be the root cause. Here's a few additional ideas that may provide the battery improvement you need. The usefulness of them will become clearer after you've run Battery Historian and identified the root cause:

1. Consider whether wifi-only is a feasible default / option for data syncing. You'll experience better battery usage, fewer network issues, and likely better customer satisfaction.

2. Why does a note taking app need to sync a few hundred MB? Can you perhaps just sync the note that has changed, instead of the entire list of notes every time?

Petrolatum

it does not only matter when it syncs, but how it syncs (if the comparison of timestamps happens on the server-side or on the client-side). – Stephanystephen

@MartinZeitler What do you mean by "how it syncs", sorry? I couldn't see much information about the sync requirements in the question. – Petrolatum

me neither, just often seen Firebase users complaining about the traffic it causes. the battery drain is just a consequence of excessive traffic generation. – Stephanystephen

@MartinZeitler Ah right, for Cloud Firestore / Realtime Database? Yeah, not sure what OP is using for the sync, since there's no information. Work Manager doesn't imply actually syncing with a Firebase service imo. – Petrolatum

well, the documentation at least hints for, that lacking indexes causes more traffic: firebase.google.com/docs/database/usage/optimize ...and this might apply to any noSQL backend, in general. inspecting/changing the way how it syncs appears the most leading. – Stephanystephen

@MartinZeitler Based on their other comments, they're using Google Drive's REST API, not any backend of their own / database. – Petrolatum

Drive has one option "only sync some folders to this computer", which then only takes the most recently used items into account. but this can only be set by the user, probably not by code... it still boils down to "sync it all" vs. "sync relevant items". – Stephanystephen

I have added relevant information, on how we perform data syncing. – Kopje

Hello @JakeLee .. I came here from google after searching for "BLE scanning". This is not a dev question, I just have the exact same constant ble scanning wakelock on my battery historian report. I'm not an app developer but I'm wondering if you have any advice on how to track down the offending app. I could disable one app at a time and read the historian report each time, but that would take days. Any advice? – Gustie