How to save user session info when the user closes the website

Question

I'm trying to build a user replay session functionality for a website and I'm using rrweb library to do that.

What this library does is when recording: it captures all the events in the webpage and I'm able to save those events by storing them in an array and when I want to replay the session I'll just pass the array to the replay function and that function handles the session replay.

Currently for testing purposes I'm saving this array in my sessionStorage and every time a new event is emitted I'll just get the array then push that new event into it then save the array again in my sessionStorage like this:

rrweb.record({
    emit(event) {
        const sessions = JSON.parse(sessionStorage.getItem('sessions')) || [];
        sessions.push(event);
        sessionStorage.setItem('sessions', JSON.stringify(sessions));
    },
});

However, for production, instead of saving that array in my sessionStorage and then updating it every time a new event is emitted, I would like to save that array in my database and I want to call the function that would save the array to my database once either when the user logs out or when the user decides to close the website (like by pressing the X button).

The first part -when the user logs out- is pretty straight forward, I'll just add an eventListener on the logout button, it's the second part -when the user decides to close the website- that is giving me some headache.

I know that there is the beforeUnload event however after a quick search it became clear to me that it's unreliable so basically what I'm looking for is a reliable way to determine when the user closed my website so I could fire an async function that would save the array to my database.

Rhinoscopy

I hope you consider all the legal implications. The recording of such things might reveal personal information (based on the content they are editing) – Archaean

Answer 1

Here are my thoughts on what you can do to mitigate the unreliableness of the beforeunload event:

1. When the page is initially loaded get and save the current number of entries in your sessions array. Assuming we are talking about a single page that you wish to replay, this count could be saved as a JavaScript variable eventCount. Better yet would be to get the count from the server-side table in case for some reason not all the events recorded were successfully saved the last time the page was closed.
2. Arrange for a function checkEvents to be called every N seconds (you must decide how often you want to call this function) using the Window setInterval method. This function will look at the current event count (variable newCount) and if greater than the current eventCount value, then the Navigator sendBeacon method can be used to POST a request to the server passing all the events that have been added since the last call (i.e. JSON.stringify(sessions.slice(eventCount, newCount)) and when the request completes assign eventCount = newCount if newCount is > eventCount. Note that while the asynchronous sendBeacon request is running it is possible that new events have been generated and that's why we update the event count with newCount rather than the current size of the sessions array.
3. You will need to do a final sendBeacon request when the page is unloaded. Since we know that the beforeunload and unload events are not reliable, then we use the visibiltychanged event (if supported by the browser) and when the new visibility state is 'hidden' we update the server. The documentation discusses those user actions that cause this event to be fired (not just on closing the page). If, however, the browser does not support this event, then the pagehide event will have to be used.

It's not discussed within the documentation for the Navigator.sendBeacon method whether there can be multiple concurrent requests. Assuming there can be (it's an asynchronous call), there is slight possibility the user might decide to navigate away from the page or close it while a sendBeacon request is currently in progress because of the setInterval call. Then the server URL to which you post this request should probably lock the table while doing the insertions so that any subsequent POST to this URL will block until the previous one completes. Another solution I would propose if your table uses some sort of sequence number as the primary key would be to also pass to the server the starting array index of the first event being passed and the server would use that to explicitly set the sequence number for each event inserted. Then concurrent updates could run without having to lock the entire table.

Fix

Answer 2

Introduction

In such situations, you need to evaluate what the dangers are, how serious are those dangers, and what would be the minimal expectation that would be good-enough.

The problem-statement was that you want to save an array when the user logs out or when he or she navigates away. There are multiple topics we should cover and you can combine them according to your needs in order to get the best possible mix.

logout

This is the easiest part of the problem-space and I strongly suggest that you make this your first milestone. You will need to send your array to your server upon logout and then the server will save the array (or its changes) into your database.

So, you need to:

• be able to load the array from the database, pass it to the UI, and make it operational
• be able to modify the array each time when such a modification is needed
• be able to send the array to your server which will store it

Now, the load part should be pretty clear, it's likely already implemented. However, modification and sending needs a good plan already, because you do not want to implement the same thing twice.

You need to have a function that will run upon each modification and another one that will run upon each sending. And you will likely need to make the second one an AJAX request and trigger this AJAX request upon logout click and only log out when this save has been completed.

onbeforeunload

You said this is unreliable. So, let's see how is this unreliability manifesting: Reliability of onbeforeunload

Basically, under normal circumstances, it will run successfully, so if you embed the sending of your array into this, then it should be successfully sent. However, it's indeed possible that the browser or the OS would crash for some reason and you need to handle that as well.

sessionStorage

You can store the array in sessionStorage and whenever the user reloads the page while being in sessionStorage, then the array could be synced. But, of course, this comes with limitations, due to the fact that the user may use a different browser or maybe his session was destroyed. The latter problem may be handled by localStorage as long as the data is not confidential, but localStorage will not handle the former problem.

periodic saves

You can run setInterval, which periodically would call your send function (as long as there was some change to save), and, if this runs every 5 minutes, then the maximum data loss is of 5 minutes, which only happens if the browser/OS/comp has crashed, so it would be a very rare situation with minimalized damage.

saving every change

You can also decide to send all changes when they happen to the server, which may create some extra burden on your server, but if it's manageable, then you have huge benefits.

Imagine the case when a user logs in with a second browser while still being logged in with the first one. It is highly desirable to make sure that the second browser will get whatever changes were made in the first browser, even if both sessions are still alive

WebSocket

You could create a WebSocket connection that maintains a duplex channel between the browser and the server, allowing you to sync browser and server instantly and that would allow you to create multiple sessions and each session would be maintained real-time, without having you, as a user to worry about data syncing.

Conclusion

onbeforeunload is pretty much reliable, but in case of crashes, it will not work. There are work-arounds and maybe (probably) you want to save your data periodically, because in the case of crashes or other extreme scenarios, if there are no intermittent saves, the user may end up having to concede the loss of the work of multiple hours. So, save frequently, but pay attention to your server's limits.

Hydrology

The onbeforeunload fires every time the user changes a page including if its another page from the same website, do you know if its possible to have fire the function that will save to the db only once in case the user either changes website or closes tab/browser? – Rhinoscopy

@dan_pran the bad news is that the target is private information, so you will not be able to get that info from unbeforeunload. But on the server's side you could temporarily store the would-be-changes in memory or some session data or something and create a cron job or so that will periodically pick up the old changes (5 minutes old, for example) stored locally and store them in the database. So, if your user navigates away and quickly back, then you will store those changes only in a quick store rather than in the database and store them in db when they "mature". – Hydrology

@dan_pran that would reduce the number of requests you need to send to the server. Note, that tracking outgoing links from your site and using semaphores that would prevent unbeforeunload from posting the changes is not a good idea, because if the user's internet connection drops while using the next page, then you would end up with the changes not being posted. So, it's better to aim for a solution that sends the changes all the time when they make sense, but doing quick saves most of the time, not the heavy-lifting of saving them into a huge db. That could be done 5 minutes after last save. – Hydrology

Answer 3

For this kind of problem you should make a database schema to store this and fire a function that takes the event and save that events in the database as usual. And for another problem of the after closing website you may call a function like:

    window.onunload=()=>{
      //function to save the event in the database
}

May this will help you for you.

Carswell

For this kind of problem you should make a database schema to store this and fire a function that takes the event and save that events in the database as usual do you mean call the api that saves the array every time a new event is emitted?? if thats the case then I don't agree bcz it will be way too taxing to call that function every single time a new event is emitted – Rhinoscopy

that should be called whenever to save the data of the users – Carswell