Django - How to track if a user is online/offline in realtime?

Question

I'm considering to use django-notifications and Web Sockets to send real-time notifications to iOS/Android and Web apps. So I'll probably use Django Channels.

Can I use Django Channels to track online status of an user real-time? If yes then how I can achieve this without polling constantly the server?

I'm looking for a best practice since I wasn't able to find any proper solution.

UPDATE:

What I have tried so far is the following approach: Using Django Channels, I implemented a WebSocket consumer that on connect will set the user status to 'online', while when the socket get disconnected the user status will be set to 'offline'. Originally I wanted to included the 'away' status, but my approach cannot provide that kind of information. Also, my implementation won't work properly when the user uses the application from multiple device, because a connection can be closed on a device, but still open on another one; the status would be set to 'offline' even if the user has another open connection.

class MyConsumer(AsyncConsumer):

    async def websocket_connect(self, event):
        # Called when a new websocket connection is established
        print("connected", event)
        user = self.scope['user']
        self.update_user_status(user, 'online')

    async def websocket_receive(self, event):
        # Called when a message is received from the websocket
        # Method NOT used
        print("received", event)

    async def websocket_disconnect(self, event):
        # Called when a websocket is disconnected
        print("disconnected", event)
        user = self.scope['user']
        self.update_user_status(user, 'offline')

    @database_sync_to_async
    def update_user_status(self, user, status):
        """
        Updates the user `status.
        `status` can be one of the following status: 'online', 'offline' or 'away'
        """
        return UserProfile.objects.filter(pk=user.pk).update(status=status)

NOTE:

My current working solution is using the Django REST Framework with an API endpoint to let client apps send HTTP POST request with current status. For example, the web app tracks mouse events and constantly POST the online status every X seconds, when there are no more mouse events POST the away status, when the tab/window is about to be closed, the app sends a POST request with status offline. THIS IS a working solution, depending on the browser I have issues when sending the offline status, but it works.

What I'm looking for is a better solution that doesn't need to constantly polling the server.

Fostoria

Answer 1

Using WebSockets is definitely the better approach.

Instead of having a binary "online"/"offline" status, you could count connections: When a new WebSocket connects, increase the "online" counter by one, when a WebSocket disconnects, decrease it. So that, when it is 0, then the user is offline on all devices.

Something like this

@database_sync_to_async
def update_user_incr(self, user):
    UserProfile.objects.filter(pk=user.pk).update(online=F('online') + 1)

@database_sync_to_async
def update_user_decr(self, user):
    UserProfile.objects.filter(pk=user.pk).update(online=F('online') - 1)

Scenarist

Yes, your solution works. How I could track the "away" status? – Fostoria

If you defined "away" as "user has not performed any action on any of the connected devices for the past 5 minutes", you'd need to remember the time of the last action on the last device used. So just have a second UserProfile property where you put the timestamp of the last "action". Of course, it also depends on what you call a "user action". Posting text? Or moving the mouse (could use JavaScript to communicate the action via the open WebSocket to the backend)? – Scenarist

Thanks! You gave a good idea about how implement this. I'll count the number of connections as you suggested, also I'll created a Django middleware to store the last_seen timestamp. If the connection count is greater than 0 and last_seen has been set more than X seconds ago, the user will be away. – Fostoria

I'll wait for the bounty expiration before assigning you +50 to let other users share other answers – Fostoria

I tested properly the solution, it works pretty good except when the server goes down. The connections counter will be greater than 0 because it's not possible to decrease the counter. – Fostoria

You could reset the counters of all users when the Django application starts up. – Scenarist

I added UserProfile.objects.filter(pk=user.pk).update(online=0) out of the MyConsumer class but on the same file. Do you think is good to put there or there is a better place? – Fostoria

I've voted your answer because it seems to me an interesting approach for some projects, but I also think that this way you lose very important information with no benefits in return – Courson

Is there a reason connection status counts should be stored in the database? I don't see the need to persist it. Could the counts not be stored in a session variable or memcache? – Lawton

@BenDavis yes, unless you run more than one Django server behind a load balancer, and need to share the state. – Scenarist

How you are managing this in frontend, – Stereochromy

Answer 2

The best approach is using Websockets.

But I think you should store not just the status, but also a session key or a device identification. If you use just a counter, you are losing valuable information, for example, from what device is the user connected at a specific moment. That is key in some projects. Besides, if something wrong happens (disconnection, server crashes, etc), you are not going to be able to track what counter is related with each device and probably you'll need to reset the counter at the end.

I recommend you to store this information in another related table:

from django.db import models
from django.conf import settings


class ConnectionHistory(models.Model):
    ONLINE = 'online'
    OFFLINE = 'offline'
    STATUS = (
        (ONLINE, 'On-line'),
        (OFFLINE, 'Off-line'),
    )
    user = models.ForeignKey(
        settings.AUTH_USER_MODEL,
        on_delete=models.CASCADE
    )
    device_id = models.CharField(max_lenght=100)
    status = models.CharField(
        max_lenght=10, choices=STATUS,
        default=ONLINE
    )
    first_login = models.DatetimeField(auto_now_add=True)
    last_echo = models.DatetimeField(auto_now=True)

    class Meta:
        unique_together = (("user", "device_id"),)

This way you have a record per device to track their status and maybe some other information like ip address, geoposition, etc. Then you can do something like (based on your code):

@database_sync_to_async
def update_user_status(self, user, device_id, status):
    return ConnectionHistory.objects.get_or_create(
        user=user, device_id=device_id,
    ).update(status=status)

How to get a device identification

There are plenty of libraries do it like https://www.npmjs.com/package/device-uuid. They simply use a bundle of browser parameters to generate a hash key. It is better than use session id alone, because it changes less frencuently.

Tracking away status

After each action, you can simply update last_echo. This way you can figured out who is connected or away and from what device.

---

Advantage: In case of crash, restart, etc, the status of the tracking could be re-establish at any time.

Courson

How the device_id is determined? I mean from the scope variable we can get session cookie but not a device id. – Fostoria

There are libraries to achieve this (client side) like npmjs.com/package/device-uuid. They normally use a bundle of browser parameters (user_agent, etc) to build a hash. But as I said, you can also use session_id instead of device_id or even both. The session id is already unique per device, but unlike device_id will change after each re-login, so if you use session_id instead of device_id, your ConnectionHistory could get bigger and maybe you'll need to delete old records ocassionally. – Courson

Answer 3

My answer is based on the answer of C14L. The idea of counting connections is very clever. I just make some improvement, at least in my case. It's quite messy and complicated, but I think it's necessary

Sometimes, WebSocket connects more than it disconnects, for example, when it has errors. That makes the connection keep increasing. My approach is instead of increasing the connection when WebSocket opens, I increase it before the user accesses the page. When the WebSocket disconnects, I decrease the connection

in views.py

def homePageView(request):
    updateOnlineStatusi_goIn(request)
    # continue normal code
    ...


def updateOnlineStatusi_goIn(request):
    useri = request.user
    if OnlineStatus.objects.filter(user=useri).exists() == False:
        dct = {
            'online': False,
            'connections': 0,
            'user': useri
        }
        onlineStatusi = OnlineStatus.objects.create(**dct)
    else:
        onlineStatusi = OnlineStatus.objects.get(user=useri)

    onlineStatusi.connections += 1
    onlineStatusi.online = True
    onlineStatusi.save()

    dct = {
        'action': 'updateOnlineStatus',
        'online': onlineStatusi.online,
        'userId': useri.id,
    }
    async_to_sync(get_channel_layer().group_send)(
        'commonRoom', {'type': 'sendd', 'dct': dct})

In models.py

class OnlineStatus(models.Model):
    online = models.BooleanField(null=True, blank=True)
    connections = models.BigIntegerField(null=True, blank=True)
    user = models.OneToOneField(User, on_delete=models.CASCADE, null=True, blank=True)

in consummers.py

class Consumer (AsyncWebsocketConsumer):
    async def sendd(self, e): await self.send(json.dumps(e["dct"]))

    async def connect(self):
        await self.accept()
        await self.channel_layer.group_add('commonRoom', self.channel_name)


    async def disconnect(self, _):
        await self.channel_layer.group_discard('commonRoom', self.channel_name)
        dct = await updateOnlineStatusi_goOut(self)
        await self.channel_layer.group_send(channelRoom, {"type": "sendd", "dct": dct})


@database_sync_to_async
def updateOnlineStatusi_goOut(self):
    useri = self.scope["user"]

    onlineStatusi = OnlineStatus.objects.get(user=useri)
    onlineStatusi.connections -= 1

    if onlineStatusi.connections <= 0:
        onlineStatusi.connections = 0
        onlineStatusi.online = False
    else:
        onlineStatusi.online = True
    onlineStatusi.save()

    dct = {
        'action': 'updateOnlineStatus',
        'online': onlineStatusi.online,
        'userId': useri.id,
    }
        
    return dct

Aryan

if you use the switch in the database - online / offline, there will be a big load, no? – Forceful