Infinite scroll on real-time data

Question

I am trying to implement infinite scrolling in a web application, records maybe added or deleted from the server, sorted in alphabetical order of their id, also the associated data can also change at any time and I have to show the latest data. I understand how infinite scrolling works and how to show the latest data for a fixed set of objects (in my case, polling on it repeatedly and dumping the data in view), but I am unable to understand how to integrate both. The API uses a cursor and sends me 20 records each time. Please help

Calicut

If you can understand what comes into view and what goes out of view, you can use FCM (Firebase Cloud Messaging) to subscribe and unsubscribe topics (posts) as it is designed to scale to billions of users. A post that is updated through the server would then send out to that "topic" and people could then see the post update if you put logic to update your data. – Colima

Answer 1

I think I understand what is your main concern. Infinite scrolling with real time updates is very hard to achieve. You have to think it thru and figure out what really you want to achieve. There are couple situations to consider, to simplify, let's assume we use a grid with rows:

1. Visible row has changed (edit)
2. A new row was added between visible rows (add)
3. An existing row was removed from visible rows (delete)
4. Loaded row has changed (edit)
5. A new row was added between previously loaded rows, but they're not visible - we scrolled down (add)
6. An existing row has been removed from loaded rows (delete)
7. Any change in not loaded row is trivial - as we still have to load it

I think the biggest problem here is to operate with rows which are loaded and aren't visible. Any change made won't be visible for a user. Since that's the case, have you considered virtual scrolling? So show only 10 rows and replace them when user scrolls up / down. If you really want to have infinite scrolling, the good way would be notifying users, that the data has been changed, and you have to re-render the whole infinite scrolling. In that case, user decides to refresh already loaded data. You can grab what's visible and try to compute what to show to him (to reflect the previous state with added/removed/edited rows)

Gagarin

Answer 2

Essentially you would need to setup the records in a scrollable container. Then use some jquery to scroll the container down as more rows are added from the API callout.

This is some javascript code that I have that scrolls a list down. The css will need to have the overflow property set to achieve this.

   function scrolllist(){


            //get the current width of the screen
            var width = document.body.clientWidth;
            var channelPanelWidth = 120;

            //get the height of the info container of the current video 
            var elem = document.getElementsByClassName("listInfoContainer")[0];


            $('#listContainer').animate({scrollLeft: nextScroll + "px"}, {duration: 1000, specialEasing: { width: 'linear', height: 'easeOutBounce'},
            complete: function (e) {

                //debug print
                //console.log("animation completed");
            }

        }


  #listContainer{
  display: inline-block;
  vertical-align: top;
  width: 196px;
  position: relative;
  overflow-y: scroll;
  overflow-x: hidden;
  height: 510px;
  left: 0px;

}

The nextscroll variable will control how much the box is scrolled. If you would like to give it the appearance of continuous scrolling, then you just need to adjust the duration and the nextscroll variables so that they haven't reached the end of the list by the time the next update comes in from the API.

Otiose

Answer 3

Socket.io or Google Cloud Messaging with Infinite Scroll

I'm coming here for ideas myself but consequently, when there are no answers to suite your needs you must improvise and come up with one yourself.

The typical infinite scrolling behavior

Doing infinite essentially is making queries on a paged set of data, fetching rows prior to showing them ahead of scroll momentum (up or down). I have this working perfectly in my web and ios/android.

To get real time per post

After doing this in socket.io and getting it to work just fine using "room" concepts, I've now decided to move to GCM Google Cloud Messaging (or now Firebase Cloud Messaging FCM) instead.

The approach

What I'm doing now is fetching 20 records at a time ahead of momentum and subscribing the user to 20 topics (posts) and as they move out of view I unsubscribe them from the topics not viewed behind momentum. If the momentum delta is too great, I disable this until it is within bounds as it makes no sense to subscribe/unsubscribe super quickly and could be considered as abuse to FCM.

Then, I have a hash lookup on the page/device for the posts I have loaded based on post id which matches the topics subscribed to and the post updates have the post id on the message packets so I can quickly update stats on the page / app based on this approach.

I was worried this would overload FCM but it seems to be amazingly scalable thus far. By the way, Firebase (anything) pretty much rocks but the FB DB is very limiting in terms of querying which is a bit disappointing so I used other cloud DBs.

Colima