Does Express.js support sending unbuffered progressively flushed responses?

Question

Perl's Catalyst framework permitts you to send an progressively flushed response over an open connection. You could for instance use write_fh() on Catalyst::Response. I've begun using Node.js, and I can't find how to do the equivalent.

If I want to send a big CSV file, on the order of 200 megs is there a way to do that without buffering the whole CSV file in memory? Granted, the client will timeout if you don't send data in a certain amount of time, so a promise would be nice if -- but is there anyway to do this?

When I try to do a res.send(text) in a callback, I get

Express
500 Error: This socket has been ended by the other party

And, it doesn't seem that Express.js supports an explicit socket.close() or anything of the ilk.

Here is an example,

exports.foo = function (res) {                          
  var query = client.query("SELECT * FROM naics.codes");
  query.on('row', function(row) {                       
    //console.log(row);                                 
    res.write("GOT A ROW");                             
  });                                                   
  query.on('end', function() {                          
    res.end();                                          
    client.end();                                       
  });                                                   
};                                                      

I would expect for that to send "GOT A ROW" out for each row, until the call to client.end() signifying completion.

Atop

How are you using the export? What results are you getting from that export? – Interlace

Export is in a model that is called by a route. It's like it doesn't get the copy of res that I send to foo(), I wonder if it is a threading error or something underneath. – Atop

Change exports.foo from function(res) to function(req, res). You are passing in the request instead of the response. – Interlace

@Interlace no difference, I was only sending it res from the route. this code comes from a db-interface that I keep my SQL in. I don't actually like even sending it res, but I do it out of necessity. – Atop

Answer 1

Express is built on the native HTTP module, which means res is an instance of http.ServerResponse, which inherits from the writable stream interface. That said, you can do this:

app.get('/', function(req, res) {
  var stream = fs.createReadStream('./file.csv');
  stream.pipe(res);

  // or use event handlers
  stream.on('data', function(data) {
    res.write(data);
  });

  stream.on('end', function() {
    res.end();
  });
});

The reason you can't use the res.send() method in Express for streams is because it will use res.close() automatically for you.

Interlace

I updated with an example. I'll try your code just for fun but your idea didn't work for me. – Atop