PubNub or Pusher and storing data on my own server

Question

I have a mobile application where users can talk to each others via a chat (user to user only, not chatroom).

Today, everything is synchronous. We are considering switching the solution to something more "real time", maybe using PubNub or Pusher.

We are wondering about the best way to do it, given that our server will need to store every sent message.

What we have in mind :

• Each mobile app would create a channel using user Id as the channel name.
• This channel would be used for mobile app / server communication.

My problem is regarding the server, today we have a Nginx/PHP backend. We would like our server to be able to listen all user channels as we need to store users sent messages on our own server.

For user to user communication, we are thinking to create our own protocol inside the messages. Eg. if user 111 wants to send "hello" to user 222, he could publish "to:222 hello", which would be retrieved by the server.

The server, upon receiving this command would push "from:111 hello" to his own channel (which would be 222).

The problem I see with this design is that our server would need to open as many channels as we have total users in our database.

I do not see a better way to do it.

Peplum

maybe open channels for online users, and checking if user present to post the message, after that you can send, or store message to db, there are many ways of dealing with this. – Trapezius

Answer 1

As suggestion about how it can be done, try next thing:

1. subscribe clients on their own channels
2. subscribe server on some special channel with constant name
3. if client A want to send message to client B it should send it into channel at which server subscribed before. Maybe you will have to send it in special format, which will allow to identify sender and recipient.
4. Server parse formatted message from client A and send it into the channel at which channel B subscribed.

So basically server subscribed only on one channel and receive messages from it, parse and send into recipient's channel (you don't have to be subscribed on channel into which you want to send message).

Bloxberg

Thanks moonlight. Sounds a good idea. I think this is the one we are going to try. – Peplum

Answer 2

PubNub Chat with Message History

Good news: you can easily write a chat app with many different channels assigned to each users and also save message history using PubNub's Real-time Network - High Availability Globally Distributed - Storage Service. With this service you are able to selectively load the messages directly on the Mobile/Web Client Device from the nearest Data Center for past message history, but also you can load the messages into your own server using out Storage Retrieval API. Let's see how this works with the following chat app:

Chat with History JavaScript Source Code

Subscribe to your USER_ID Channel Name in order to receive messages from other users. Also load history from previous chats.

<script src="https://cdn.pubnub.com/pubnub.min.js"></script>
<script>(function(){

// INIT
var channel = 'USER_ID-123456';
var pubnub  = PUBNUB.init({
    subscribe_key : 'demo',
    publish_key   : 'demo'
});

// CHAT MESSAGE RECEIVER
function chat(message) {
    // process chat message here...
}

// LOAD HISTORICAL MESSAGES
pubnub.history({
    channel  : channel, // USER_ID Channel
    limit    : 50,      // Load Last 50 Messages
    callback : function(msgs) { pubnub.each( msgs[0], chat ) }
});

// PUBNUB REAL-TIME NETWORK HA-TCP STREAM CONNECTION
// FOR RECEIVING INCOMING CHAT MESSAGES
pubnub.subscribe({
    channel  : channel, // USER_ID Channel
    connect  : connect, // Connected - Ready to Receive Messages
    callback : chat     // Callback Processor
});

})();</script>

That's the basics of the chat app on the mobile/web client app. Now you can easily load/save messages to/from a Global provider. Next you'll want to load these messages on your server from PHP using the PubNub REST interface.

Loading Stored Message via REST API on PHP Backend

You'll use the REST interface to collect previously posted messages as needed from you PHP Backend server. You may actually not need this step since the data is stored on PubNub's global Real-time Network where your messages are replicated to many geographic regions for reliability and high read/write performance.

PubNub Storage/History V2 REST API Doc - https://gist.github.com/stephenlb/d53f4cc3a891c03b478e

REST Request

http://pubsub.pubnub.com/v2/history/sub-key/demo/channel/my_channel?count=5

REST Response

[["Pub1","Pub2","Pub3","Pub4","Pub5"],13406746729185766,13406746845892666]

You can also use the PubNub PHP SDK to help with some of the complexities. You can find the PubNub PHP SDK here: https://github.com/pubnub/php and load history with this example:

<?php
$pubnub = new Pubnub(
    "demo",  ## PUBLISH_KEY
    "demo",  ## SUBSCRIBE_KEY
    "",      ## SECRET_KEY
    false    ## SSL_ON?
);

$history_data = $pubnub->history(array(
    'channel' => $channel,
    'count'   => 100,
    'end'     => "13466530169226760"
));
?>

More Details about Storage REST API on PubNub

Please follow this link to dive further in depth with PubNub Storage API: https://gist.github.com/stephenlb/d53f4cc3a891c03b478e - This guide will help answer additional details regarding storage REST API.

More of a Full GUI Chat Client using History

The following is a group-chat which will help you get started, it is written using Bootstrap CSS Framework - https://github.com/pubnub/real-time-stocks/#simple-embedded-chat-application

Kinship

Thank you very much for your detailed reply. Regarding the storage on our own server, we actually need it because we do some processing on them and this needs to be done locally. One other thing is I think I've read that PubNub has a limit in terms of storage history per channel and limited in time as well. So it means we really need to store them on our server. In that case I think we could have a routine server side that would download messages from channels periodically. What are your thoughts ? – Peplum

Storage is unlimited. And you can download a dump of all messages anytime. gist.github.com/stephenlb/d53f4cc3a891c03b478e - This guide will help answer additional details regarding storage REST API. – Kinship

@PubNub what is the easiest way to download the dump of all messages? – Hitlerism

Hi @Hitlerism you will want to write a loop that pages over all your messages in the channel. gist.github.com/stephenlb/4f0aab24f1fc5cf250bef0d18f4322c4 Note that you must know what your channel names are. If you don't know what your channel names are then you can't get the data! We are also creating a new method for this later on. – Kinship

@PubNub thanks. Are there any libraries for that? I remember writing loop back in the day and it was a bit of a pain because of how the paging mechanism is implemented – Hitlerism