How to stream data over socket.io to client

Question

I have socket.io sending a basic object from server to client. This bit works fine.

Now want to send a stream from server to client, using event-stream (specifically the results of a block-chain query). I am getting unexpected results in the browser console..

    var io = require('socket.io')(server);
    var dsteem = require('dsteem')
    var es = require('event-stream') 
    var util = require('util')
    var client = new dsteem.Client('https://api.steemit.com')
    var stream = client.blockchain.getBlockStream()

/* This sends results to stdout, fine
io.on('connection', function(socket){
    stream.pipe(es.map(function(block, callback) {
        callback(null, util.inspect(block) + '\n') 
    })).pipe(process.stdout);

    // And this sends a simple object to the client 
    socket.emit('blockchainOps', {"Foo!":"Doo!"} );
});
*/

// Putting both together sends strange connection data to client
io.on('connection', function(socket){

        socket.emit('blockchainOps', function() {

            stream.pipe(es.map(function(block, callback) {
                callback(null, util.inspect(block) + '\n');
            }))
        })

    });

What I get in the client console appears to be some kind of TCP socket function,

ƒ (){if(!n){n=!0;var r=a(arguments);u("sending ack %j",r),e.packet({type:i.ACK,id:t,data:r})}}

Can anyone help me understand what's going on and what I'm doing wrong?

== EDIT UPDATE ==

As suggested in comments, I've tried socket.io-stream to augment event-stream.

var es = require('event-stream') 
var util = require('util')
var ss = require('socket.io-stream'); 
var stream = ss.createStream();

io.on('connection', function(socket){

ss(socket).emit('blockchainOps', stream, function(){


        client.blockchain.getBlockStream()
            .pipe(es.map(function(block, callback) {
                callback(null, util.inspect(block) + '\n') 
                }))
            .pipe(process.stdout)


}());

});

This time I get a socket object returned in the browser console which does not seem to be the stream data I was hoping for.

Superhighway

Answer 1

You're using socket.emit wrong, you're passing the ACK callback to the client instead of your stream. Have a look at socket.emit signature :socket.emit(eventName[, ...args][, ack]).

You probably want something like

socket.emit('blockchainOps', client.blockchain.getBlockStream());

However, I don't think plain socket io supports passing a Stream like that. To pipe a stream down to the client you could use socketio-stream. It would look like this:

var ss = require('socket.io-stream');
var stream = ss.createStream();

ss(socket).emit('blockchainOps', stream);
client.blockchain.getBlockStream().pipe(stream);

EDIT:

On the client, you should be able to read your stream like this:

    <script src="socket.io/socket.io.js"></script>
    <script src="socket.io-stream.js"></script>
    ...
    ss(socket).on('blockchainOps', function(stream) {
       var binaryString = "";

       stream.on('data', function(data) {
           for(var i=0;i<data.length;i++) {
                binaryString+=String.fromCharCode(data[i]);
           }                
       });

        stream.on('end', function(data) {
             console.log(binaryString);
             binaryString = "";
       });
    });

Afb

Isn't your top example what I'm already doing? In your second example, isn't socket.io-stream duplicative of event-stream? I've been unable to get either to pipe results of getBlockStream() into the socket. – Superhighway

They are for different purposes, socketio-stream allows you to pass down a stream socket.emit, whilst event-stream is just a wrapper around node streams. I've updated my answer, can you try it? Note that you have to pipe the blockchain stream to the stream you created via var stream = ss.createStream(); – Afb

Daniel, I'm willing to accept your answer, but how do I extract my data out of this IOStream object? I can't find anything buried in here that looks like the results of getBlockStream(); – Superhighway

I've posted an example of how you could read the Stream in the browser, bear in mind you'll need the socketio-stream client <script src="socket.io-stream.js"></script>. You can easily get it from a cdn cdnjs.com/libraries/socket.io-stream – Afb

I'll give this a try. Thanks for taking the time to look at this. – Superhighway

My data stream will not have an "end". How can I console.log() this in real time rather than waiting for a stream.on('end') to fire? – Superhighway

add a console.log to stream.on('data', function(data) { console.log(data); ... }) – Afb

I tried this but get nothing. Seems like stream has no child data object. – Superhighway

Let us continue this discussion in chat. – Afb

Answer 2

If anyone is looking for a working socket.io stream example

// server side
const { pipeline } = require('stream')
const server = require('http').Server().listen(8080)
const io = require('socket.io')(server)
const ss = require('socket.io-stream')

io.on('connection', (socket) => ss(socket).on('stream', (stream) => {
  pipeline(stream, process.stdout,  (err) => err && console.log(err))
}));


// client side
const client = require('socket.io-client')
const socket = client.connect('http://localhost:8080')

socket.on('connect', () => {
  const stream = ss.createStream()
  ss(socket).emit('stream', stream)
  pipeline(process.stdin, stream,  (err) => err && console.log(err))
});

Fillender

I was able to connect the client to the server using socket.io-client and socket.io-stream. This should have been the accepted answer. – Aguiar