Android Paging 3: LoadType.APPEND returns null remote keys

Question

I've been trying to work out how to fix my issue with RemoteMediator's APPEND LoadType.

On an empty Room DB, here's how the LoadType flows: REFRESH -> PREPEND -> APPEND (remoteKeys = null, endOfPaginationReached = true)

With at least 10 rows for entity and remote keys table, here's how the LoadType flows: REFRESH -> PREPEND -> APPEND (remoteKeys = prev=null, next=2, endOfPaginationReached = false)

Clearly, my issue is on a fresh installed device (with empty Room DB), the user won't see more than 10 items because APPEND's state.lastItemOrNull() is returning null.

Here's my code so far:

private suspend fun getRemoteKeysForLastItem(state: PagingState<Int, MovieCache>): MovieRemoteKeys? {
    return state.lastItemOrNull()?.let { movie ->
        appDatabase.withTransaction {
            appDatabase.remoteKeysDao().remoteKeysByImdbId(movie.imdbId)
        }
    }
}

for my load() function:

val loadKey = when (loadType) {
            LoadType.REFRESH -> {
                val key = getRemoteKeysClosestToCurrentPosition(state)
                Timber.d("REFRESH key: $key, output: ${key?.nextKey?.minus(1)}")
                key?.nextKey?.minus(1) ?: 1
            }
            LoadType.PREPEND -> {
                Timber.d("PREPEND key requested")
                return MediatorResult.Success(true)
            }
            LoadType.APPEND -> {
                val key = getRemoteKeysForLastItem(state)
                Timber.d("APPEND key: $key")
                appDatabase.withTransaction {
                    val size = movieDao.movies().size
                    val remoteSize = remoteKeysDao.allKeys().size
                    Timber.d("APPEND DB size: $size, remote: $remoteSize")
                }
                key?.nextKey ?: return MediatorResult.Success(true)
            }
        }

Here's a sample logcat showing that APPEND is null

Leaving my app unable to scroll down even at least once!

Edit: Here is how I save my remote keys:

Obnoxious

Answer 1

Finally, this is how I managed to fix this issue, by relying on the remoteKeys on the DB than PagingState:

LoadType.APPEND -> {
//  val key = getRemoteKeysForLastItem(state) // Doesn't work. state returns NULL even Room has data for both remote keys and entity.
    val key = appDatabase.withTransaction {
        remoteKeysDao.allKeys().lastOrNull() // Workaround
    }
    key?.nextKey ?: return MediatorResult.Success(true)
}

Obnoxious

Can you please look into this query? #67952460 – Gretna

Hi Jim, your idea was revolutionary, literally had the same issue although I was using item keys instead of remote keys. I will post my answer below but couldn't come-up until I saw your approach few days ago. Thanks mate. – Genni

Answer 2

Instead of returning endOfPaginationReached = true from LoadType.APPEND when remote key is null, return endOfPaginationReached = key != null. Returning endOfPaginationReached = false means keep loading which will give you the remote key next time.

LoadType.APPEND -> {
    val key = getRemoteKeysForLastItem(state)
    Timber.d("APPEND key: $key")
    appDatabase.withTransaction {
        val size = movieDao.movies().size
        val remoteSize = remoteKeysDao.allKeys().size
        Timber.d("APPEND DB size: $size, remote: $remoteSize")
    }
    key?.nextKey ?: return MediatorResult.Success(key != null)
}

Cobnut

This should be the accepted answer. (key != null) fixes the issue for me. It is a super weird behavior that after updating the database LoadType.APPEND still returns null. – Pachisi

Answer 3

When working with RemoteMediator there are 2 approaches -> Item Keys or Remote Keys .

I faced the exact issue very well documented by @Jim Ovejera but this time when using items keys.

Because APPEND's state.lastItemOrNull() is returning null (triggering endOfPageReached = true) the very first time you populate the database, then you can't rely on the inbuilt state.lastItemOrNull().

The revolutionary idea that Jim came up with is to use the last item on the database instead to calculate the offset or the next item id to load.

    LoadType.APPEND -> {
        val lastItem = state.lastItemOrNull() //returns null 1st time
        val lastDbItemId = dao.selectLast()?.id
        
        if (lastDbItemId== null){ return 
MediatorResult.Success(endOfPaginationReached = true)}
                  
           
        lastDbItemId
                            }

If your offset is zero-based the above snippet works perfectly but you may need to use lastDbItemId.plus(1) if not.

The dao's selectLast() function is just a simple SQLite Query returning a nullable entity.

@Query("SELECT * FROM table_name WHERE id = (SELECT MAX(id) FROM table_name) ")
suspend fun selectLast: Entity?

My Entity Table looks like this:

@Entity(tableName = "table_name")
data class Entity(
@PrimaryKey(autoGenerate = false)
val id: Int? = null .....)

I noted setting autoGenerate=true creates discrepancies every time new data is loaded.

Genni