Android Paging 3 - experiencing flickers, glitches or jumps in position when scrolling & loading new pages

Question

Hello Guys im using Android Jetpack Paging library 3, I'm creating a news app that implements network + database scenario, and im following the codelab by google https://codelabs.developers.google.com/codelabs/android-paging , im doing it almost like in the codelab i almost matched all the operations shown in the examples https://github.com/android/architecture-components-samples/tree/main/PagingWithNetworkSample.

It works almost as it should...but my backend response is page keyed, i mean response comes with the list of news and the next page url, remote mediator fetches the data, populates the database, repository is set, viewmodel is set...

The problem is : when recyclerview loads the data , following happens:recyclerview flickers, items jump, are removed , added again and so on. I dont know why recyclerview or its itemanimator behaves like that , that looks so ugly and glitchy. More than that, when i scroll to the end of the list new items are fetched and that glitchy and jumping effect is happening again.

I would be very grateful if you could help me, im sitting on it for three days , thank you very much in advance.Here are my code snippets:

@Entity(tableName = "blogs")
data class Blog(
@PrimaryKey(autoGenerate = true)
val databaseid:Int,

@field:SerializedName("id")
val id: Int,
@field:SerializedName("title")
val title: String,

@field:SerializedName("image")
val image: String,

@field:SerializedName("date")
val date: String,

@field:SerializedName("share_link")
val shareLink: String,

@field:SerializedName("status")

val status: Int,

@field:SerializedName("url")
val url: String
) {
var categoryId: Int? = null
var tagId: Int? = null
 }

Here's the DAO

@Insert(onConflict = OnConflictStrategy.REPLACE)
suspend fun insertAll(blogs: List<Blog>)

 @Query("DELETE FROM blogs")
suspend fun deleteAllBlogs()

 @Query("SELECT * FROM blogs WHERE categoryId= :categoryId ORDER BY id DESC")
fun getBlogsSourceUniversal(categoryId:Int?): PagingSource<Int, Blog>

 @Query("SELECT * FROM blogs WHERE categoryId= :categoryId AND tagId= :tagId ORDER BY id DESC")
fun getBlogsSourceUniversalWithTags(categoryId:Int?,tagId:Int?): PagingSource<Int, Blog>

NewsDatabaseKt

abstract class NewsDatabaseKt : RoomDatabase() {

abstract fun articleDAOKt(): ArticleDAOKt
abstract fun remoteKeyDao(): RemoteKeyDao

companion object {


    @Volatile
    private var INSTANCE: NewsDatabaseKt? = null


    fun getDatabase(context: Context): NewsDatabaseKt =
        INSTANCE ?: synchronized(this) {
            INSTANCE ?: buildDatabase(context).also { INSTANCE = it }
        }


    private fun buildDatabase(context: Context) = 
   Room.databaseBuilder(context.applicationContext,
            NewsDatabaseKt::class.java,
            "news_database_kt")
            .build()
    }

RemoteMediator

    @ExperimentalPagingApi
   class BlogsRemoteMediator(private val categoryId: Int,
                      private val service: NewsAPIInterfaceKt,
                      private val newsDatabase: NewsDatabaseKt,
                      private val tagId : Int? = null ,
                      private val initialPage:Int = 1
    ) : RemoteMediator<Int, Blog>() {

override suspend fun initialize(): InitializeAction {
    
    return InitializeAction.LAUNCH_INITIAL_REFRESH
}

override suspend fun load(loadType: LoadType, state: PagingState<Int, Blog>): MediatorResult {
    try {
        val page = when (loadType) {
            REFRESH ->{ 
                initialPage
                
            }
            PREPEND -> {
                return MediatorResult.Success(endOfPaginationReached = true)}
            APPEND -> {
              
                val remoteKey = newsDatabase.withTransaction {
                    newsDatabase.remoteKeyDao().remoteKeyByLatest(categoryId.toString())
                }
                if(remoteKey.nextPageKey == null){
                    return MediatorResult.Success(endOfPaginationReached = true)
                }
                remoteKey.nextPageKey.toInt()
                }


            }


        val apiResponse =
                if(tagId == null) {
            service.getCategoryResponsePage(RU, categoryId, page.toString())
        }else{
            service.getCategoryTagResponsePage(RU,categoryId,tagId,page.toString())
        }
        val blogs = apiResponse.blogs
        val endOfPaginationReached = blogs.size < state.config.pageSize

        newsDatabase.withTransaction {
            // clear all tables in the database
            if (loadType == LoadType.REFRESH) {
              
                newsDatabase.remoteKeyDao().deleteByLatest(categoryId.toString())
                if(tagId == null) {
                    newsDatabase.articleDAOKt().clearBlogsByCatId(categoryId)
                }else {
                    newsDatabase.articleDAOKt().clearBlogsByCatId(categoryId,tagId)
                }
            }

            blogs.map {blog ->
                blog.categoryId = categoryId
                if(tagId != null) {
                    blog.tagId = tagId
                }
            }
        newsDatabase.remoteKeyDao().insert(LatestRemoteKey(categoryId.toString(),
        apiResponse.nextPageParam))
            newsDatabase.articleDAOKt().insertAll(blogs)

        }

        return MediatorResult.Success(
                endOfPaginationReached = endOfPaginationReached
        )
    } catch (exception: IOException) {
        return MediatorResult.Error(exception)
    } catch (exception: HttpException) {
        return MediatorResult.Error(exception)
    }

}

PagingRepository

 class PagingRepository(
    private val service: NewsAPIInterfaceKt,
    private val databaseKt: NewsDatabaseKt
    ){
    @ExperimentalPagingApi
 fun getBlogsResultStreamUniversal(int: Int, tagId : Int? = null) : Flow<PagingData<Blog>>{
    val pagingSourceFactory =  {
        if(tagId == null) {
            databaseKt.articleDAOKt().getBlogsSourceUniversal(int)

        }else databaseKt.articleDAOKt().getBlogsSourceUniversalWithTags(int,tagId)
    }
    return Pager(
            config = PagingConfig(
                    pageSize = 1
            )
            ,remoteMediator = 
            BlogsRemoteMediator(int, service, databaseKt,tagId)
            ,pagingSourceFactory = pagingSourceFactory
    ).flow
  }
}

BlogsViewmodel

class BlogsViewModel(private val repository: PagingRepository):ViewModel(){

private var currentResultUiModel: Flow<PagingData<UiModel.BlogModel>>? = null
private var categoryId:Int?=null

@ExperimentalPagingApi
fun getBlogsUniversalWithUiModel(int: Int, tagId : Int? = null): 
Flow<PagingData<UiModel.BlogModel>> {

    val lastResult = currentResultUiModel


    if(lastResult != null && int == categoryId){
        return lastResult
    }

    val newResult: Flow<PagingData<UiModel.BlogModel>> = 
     repository.getBlogsResultStreamUniversal(int, tagId)
            .map { pagingData -> pagingData.map { UiModel.BlogModel(it)}}
            .cachedIn(viewModelScope)

    currentResultUiModel = newResult
    categoryId = int
    return newResult
}

sealed class UiModel{
    data class BlogModel(val blog: Blog) : UiModel()
}

PoliticsFragmentKotlin

      @ExperimentalPagingApi
   class PoliticsFragmentKotlin : Fragment() {

     private lateinit var recyclerView: RecyclerView
     private lateinit var pagedBlogsAdapter:BlogsAdapter

     lateinit var viewModelKt: BlogsViewModel
     lateinit var viewModel:NewsViewModel

     private var searchJob: Job? = null

      @ExperimentalPagingApi
     private fun loadData(categoryId:Int, tagId : Int? = null) {

    searchJob?.cancel()
    searchJob = lifecycleScope.launch {
        

        viewModelKt.getBlogsUniversalWithUiModel(categoryId, tagId).collectLatest {
            pagedBlogsAdapter.submitData(it)
           
        }
     }
   }

    @ExperimentalPagingApi
override fun onCreateView(inflater: LayoutInflater, container: ViewGroup?,
                          savedInstanceState: Bundle?): View? {
    val view = inflater.inflate(R.layout.fragment_blogs, container, false)   
      viewModelKt = ViewModelProvider(requireActivity(),Injection.provideViewModelFactory(requireContext())).get(BlogsViewModel::class.java)

  viewModel = ViewModelProvider(requireActivity()).get(NewsViewModel::class.java)
 pagedBlogsAdapter = BlogsAdapter(context,viewModel)
  val decoration = DividerItemDecoration(context, DividerItemDecoration.VERTICAL)
   recyclerView = view.findViewById(R.id.politics_recyclerView)
   recyclerView.addItemDecoration(decoration)

    initAdapter()
    loadData(categoryId)
    initLoad()
 return view
}

       private fun initLoad() {
    lifecycleScope.launchWhenCreated {
        Log.d("meylis", "lqunched loadstate scope")
        pagedBlogsAdapter.loadStateFlow
                // Only emit when REFRESH LoadState for RemoteMediator changes.
                .distinctUntilChangedBy { it.refresh }
                // Only react to cases where Remote REFRESH completes i.e., NotLoading.
                .filter { it.refresh is LoadState.NotLoading }
                .collect { recyclerView.scrollToPosition(0) }
    }
}

  private fun initAdapter() {
    recyclerView.adapter = pagedBlogsAdapter.withLoadStateHeaderAndFooter(
            header = BlogsLoadStateAdapter { pagedBlogsAdapter.retry() },
            footer = BlogsLoadStateAdapter { pagedBlogsAdapter.retry() }
    )

    lifecycleScope.launchWhenCreated {
        pagedBlogsAdapter.loadStateFlow.collectLatest {
            swipeRefreshLayout.isRefreshing = it.refresh is LoadState.Loading
        }
    }

       pagedBlogsAdapter.addLoadStateListener { loadState ->
        // Only show the list if refresh succeeds.
        recyclerView.isVisible = loadState.source.refresh is LoadState.NotLoading
                // Show loading spinner during initial load or refresh.
        progressBar.isVisible = loadState.source.refresh is LoadState.Loading
        // Show the retry state if initial load or refresh fails.
        retryButton.isVisible = loadState.source.refresh is LoadState.Error

        // Toast on any error, regardless of whether it came from RemoteMediator or PagingSource
        val errorState = loadState.source.append as? LoadState.Error
                ?: loadState.source.prepend as? LoadState.Error
                ?: loadState.append as? LoadState.Error
                ?: loadState.prepend as? LoadState.Error
        errorState?.let {
            Toast.makeText(context, "\uD83D\uDE28 Wooops ${it.error}", Toast.LENGTH_LONG
            ).show()
        }
    }
}


     companion object {

    @JvmStatic
    fun newInstance(categoryId: Int, tags : ArrayList<Tag>): PoliticsFragmentKotlin {
        val args = Bundle()
        args.putInt(URL, categoryId)
        args.putSerializable(TAGS,tags)
        val fragmentKotlin = PoliticsFragmentKotlin()
        fragmentKotlin.arguments = args
        Log.d("meylis", "created instance")
        return fragmentKotlin
    }
}

BlogsAdapter

class BlogsAdapter(var context: Context?, var newsViewModel:NewsViewModel) : 
  PagingDataAdapter<BlogsViewModel.UiModel.BlogModel, RecyclerView.ViewHolder> 
   (REPO_COMPARATOR) {

private val VIEW = 10

override fun onCreateViewHolder(parent: ViewGroup, viewType: Int): RecyclerView.ViewHolder {
    return when (viewType) {
        VIEW -> MyViewHolder(LayoutInflater.from(parent.context).inflate(R.layout.card_layout, parent, false))
}
}

override fun onBindViewHolder(holder: RecyclerView.ViewHolder, position: Int) {
  
   val uiModel = getItem(position)
  
    if(uiModel == null){
        if(uiModel is BlogsViewModel.UiModel.BlogModel){(holder as MyViewHolder).bind(null)}
    }
      
        if(uiModel is BlogsViewModel.UiModel.BlogModel){(holder as 
         MyViewHolder).bind(uiModel.blog)}


}

override fun getItemViewType(position: Int): Int  {
    return VIEW
 }


companion object {
    private val REPO_COMPARATOR = object : DiffUtil.ItemCallback<BlogsViewModel.UiModel.BlogModel>() {
        override fun areItemsTheSame(oldItem: BlogsViewModel.UiModel.BlogModel, newItem: BlogsViewModel.UiModel.BlogModel): Boolean =
                oldItem.blog.title == newItem.blog.title
        override fun areContentsTheSame(oldItem: BlogsViewModel.UiModel.BlogModel, newItem: BlogsViewModel.UiModel.BlogModel): Boolean =
                oldItem == newItem
    }

}

MyViewHolder

class MyViewHolder(var container: View) : RecyclerView.ViewHolder(container) {
var cv: CardView
@JvmField
var mArticle: TextView
var date: TextView? = null
@JvmField
var time: TextView
@JvmField
var articleImg: ImageView
@JvmField
var shareView: View
var button: MaterialButton? = null
@JvmField
var checkBox: CheckBox

var progressBar: ProgressBar

private var blog:Blog? = null

init {
    cv = container.findViewById<View>(R.id.cardvmain) as CardView
    mArticle = container.findViewById<View>(R.id.article) as TextView
    articleImg = container.findViewById<View>(R.id.imgvmain) as ImageView
    //button = (MaterialButton) itemView.findViewById(R.id.sharemain);
    checkBox = container.findViewById<View>(R.id.checkboxmain) as CheckBox
    time = container.findViewById(R.id.card_time)
    shareView = container.findViewById(R.id.shareView)
    progressBar = container.findViewById(R.id.blog_progress)
}

fun bind(blog: Blog?){
    if(blog == null){
        mArticle.text = "loading"
        time.text = "loading"
        articleImg.visibility = View.GONE
    }else {
        this.blog = blog
        mArticle.text = blog.title
        time.text = blog.date

        if (blog.image.startsWith("http")) {
            articleImg.visibility = View.VISIBLE
            val options: RequestOptions = RequestOptions()
                    .centerCrop()
                    .priority(Priority.HIGH)

            GlideImageLoader(articleImg,
                    progressBar).load(blog.image, options)
        } else {
            articleImg.visibility = View.GONE
        }
    }

}
}

NewsApiInterface

interface NewsAPIInterfaceKt {

 @GET("sort?")
suspend fun getCategoryResponsePage(@Header("Language") language: String, @Query("category") 
categoryId: Int, @Query("page") pageNumber: String): BlogsResponse

@GET("sort?")
suspend fun getCategoryTagResponsePage(@Header("Language") language: String, 
@Query("category") categoryId: Int,@Query("tag") tagId:Int, @Query("page") pageNumber: String)
:BlogsResponse

     companion object {

    fun create(): NewsAPIInterfaceKt {
        val logger = HttpLoggingInterceptor()
        logger.level = HttpLoggingInterceptor.Level.BASIC


        val okHttpClient = UnsafeOkHttpClient.getUnsafeOkHttpClient()

        return Retrofit.Builder()
                .baseUrl(BASE_URL)
                .client(okHttpClient)
                .addConverterFactory(GsonConverterFactory.create())
                .build()
                .create(NewsAPIInterfaceKt::class.java)
    }
}

}

I have tried setting initialLoadSize = 1 But the problem still persists

EDIT: Thanks for your answer @dlam , yes, it does , my network API returns the list of results ordered by id. BTW, items do this jump when the application is run offline as well.

Videos when refreshing and loading online

online loading and paging

online loading and paging(2)

Videos when refreshing and loading offline

offline loading and refreshing

Thanks again, here is my gist link https://gist.github.com/Aydogdyshka/7ca3eb654adb91477a42128de2f06ea9

EDIT Thanks a lot to @dlam, when I set pageSize=10, jumping has disappeared...Then i remembered why i set pageSize=1 in the first place... when i refresh , 3 x pageSize of items are loaded, even if i overrided initialLoadSize = 10 , it still loads 3 x pageSize calling append 2x times after refresh , what could i be doing wrong, what's the correct way to only load first page when i refresh ?

Remotion

Does your network API return results ordered by id? If possible, can you share a video of the items jumping and also helpful would be the logs of pages loaded by PagingSource and RemoteMediator. You may want to dump that in a gist as it may be quite long. – Hexamerous

@dlam, Yes, my network response returns results ordered by id. And i also attached videos and gist link to my question – Remotion

Can you try making your pageSize larger? Or at least change initialPageSize. It should cover > viewport * 2, so in your case at least 8, maybe try pageSize = 10? When you refresh it replaces the list and resumes position by passing both lists to DiffUtil. By default, Room's PagingSource picks an index based on last bound position and offsets based on pageSize. In order for refresh to load around the right spot and animate in the update smoothly, pageSize must be sufficiently large. – Hexamerous

@dlam, yasss, it worked ) I set pageSize=10 , thank you very much, jumping has disappeared...Then i remembered why i set pageSize=1 in the first place... when i refresh , 3xpageSize of items are loaded even if i overrided initialLoadSize = 10 , it still loads 3 x pageSize calling append 2x times after refresh , what could i be doing wrong ? – Remotion

Could you submit your answer so i could accept it? – Remotion

Done, I should probably check SO more often than I do - thanks for the reminder, answered your follow-up as well :) – Hexamerous

If you look into PagingConfig source code it says that parameters pageSize and initialLoadSize could be ignored. So adjusting these 2 params are not 100% solution. Note: [initialLoadSize] is used to inform [PagingSource.LoadParams.loadSize], but is not enforced. A [PagingSource] may completely ignore this value and still return a valid initial [Page][PagingSource.LoadResult.Page]. – Hemminger

Answer 1

Just following up here from comments:

Setting pageSize = 10 fixes the issue.

The issue was with pageSize being too small, resulting in PagingSource refreshes loading pages that did not cover the viewport. Since source refresh replaces the list and goes through DiffUtil, you need to provide an initialLoadSize that is large enough so that there is some overlap (otherwise scroll position will be lost).

BTW - Paging loads additional data automatically based on PagingConfig.prefetchDistance. If RecyclerView binds items close enough to the edge of the list, it will automatically trigger APPEND / PREPEND loads. This is why the default of initialLoadSize is 3 * pageSize, but if you're still experiencing additional loads, I would suggest either adjusting prefetchDistance, or increasing initialLoadSize further.

Hexamerous

Hi i still have the problem with additional append load, when i set the initialLoadSize less than the page size , app crashes , when i set it to initialLoadSize = pageSize , it still appends 2 times , i tried different values for prefetchDistance but the problem still persists, what should i do ? Thanks for your help – Remotion

How many items are visible at a time on screen? Paging will automatically fetch items as recyclerview binds them. Page fetch is triggered by prefetchDistance based on most recent call to PagingDataAdapter.get(index). – Hexamerous

If you set initialLoadSize less than pageSize, the app is probably crashing because that's an invalid PagingConfig, did you read the exception message thrown? – Hexamerous

4 items are visible at a time on screen, i tried setting initialLoadSize bigger , and less than page but it crashes the app, i tried setting prefetch distance to 1 , but it still loads additional 2 pages – Remotion

I just want to load 1 page of data initially and get other pages as i scroll – Remotion

I searched for fatal errors , but there were none, the logcat didn't show any fatal errors – Remotion

What values are you setting in your PagingConfig exactly? Note that RV will typically bind at least 1 more item than visible and its not too easy to predict how many items are visible at once since Android phones come in many different screen sizes. You should really not set initialLoadSize less than pageSize, and you should use much larger values than pageSize = 1. I would try to see if you can get your network api to return more than 1 item per page as its going to lead to a lot of network requests. – Hexamerous

You cannot really force Paging to only load 1 page then wait for scrolling, because if there aren't enough items loaded to allow you to scroll then it can get stuck (never load any more pages). Paging opportunistically loads more pages based on prefetchDistance from last call to PagingDataAdapter.get(index) and it is not really feasible to change that. – Hexamerous

my network api returns 10 items at a time, then i save them to the Room database, config = PagingConfig( pageSize = 10, initialLoadSize = 10, prefetchDistance = 0, enablePlaceholders = true ) – Remotion

but i still see the additional load – Remotion

You cannot set prefetchDistance to 0, can you share your project directly? And again, you cannot really control how many items will be needed to fulfill the viewport between different devices, so you cannot guarantee a single page load. – Hexamerous

thank you very much for your time , i appreciate your help , i understood what you meant . I tried setting pagesize to the numbers of items visible on the screen , then additional page loading went away, i know this might be wrong , i would be very pleased if you could point me to the right direction of how can i achieve this – Remotion

@Remotion do you able to solve the issue where every refresh will show 4 items above the screen? – Baseburner

What do you mean by 4 items above the screen? Are you using compose LazyItems or regular RV adapter? Does this happen on initial load or after on subsequent page load / invalidates? If you want to post a separate question with your code or share your project with me I'm happy to help. I believe the original question is about manipulating pageSize to prevent additional loads, but it is not actually not all that feasible due to screen size / text size / density disparity – Hexamerous

How about the flickering issue? I am facing similar issue, but the flicker only happens on the first item upon adapter.refresh() – Estrade

Answer 2

config = PagingConfig(
                pageSize = PAGE_SIZE,
                enablePlaceholders = true,
                prefetchDistance = 3* PAGE_SIZE,
                initialLoadSize = 2*PAGE_SIZE,
            )

make sure enablePlaceholders is set to true and set the page size to around 10 to 20

Pinpoint

Knew the issue, but I was lost! Didn't know the optimum values. This is the best combination for almost any number of items in the screen. – Cairistiona

Answer 3

recyclerview flickers becouse from dao you get items not the same order it was responded from network. I will suggest you my solution. we will get items from database order by primary key, databaseid, descending. first of all delete autogenerated = true. we will set databaseid manualy, in same order we got items from network.

next lets edit remoteMediator load function.

when (loadType) {
            LoadType.PREPEND -> {
                blogs.map {
                    val databaseid = getFirstBlogDatabaseId(state)?.databaseid?:0
                    movies.forEachIndexed{
                            index, blog ->
                        blog.databaseid = roomId - (movies.size -index.toLong())
                    }
                }
            }
            LoadType.APPEND -> {
                val roomId = getLastBlogDatabaseId(state)?.databaseid ?:0
                blogs.forEachIndexed{
                        index, blog ->
                    blog.databaseid = roomId + index.toLong() + 1
                }
            }
            LoadType.REFRESH -> {
                blogs.forEachIndexed{
                    index, blog ->
                    blog.databaseid = index.toLong()
                }
            }
        }


private fun getFirstBlogDatabaseId(state: PagingState<Int, Blog>): Blog? {
    return state.pages.firstOrNull { it.data.isNotEmpty() }?.data?.firstOrNull()
}

private fun getLastBlogDatabaseId(state: PagingState<Int, Blog>): Blog? {
    return state.lastItemOrNull()
}

Sunlit