node.js http.request event flow - where did my END event go?

Question

I am working on a cunning plan that involves using node.js as a proxy server in front of another service.

In short:

1. Dispatch incoming request to a static file (if it exists)
2. Otherwise, dispatch the request to another service

I have the basics working, but now attempting to get the whole thing working with Sencha Connect so I can access all the kick-ass middleware provided.

All of the action happens in dispatchProxy below

connect(
  connect.logger(), 
  connect.static(__dirname + '/public'),
  (request, response) ->  
    dispatchProxy(request, response)
).listen(8000)

dispatchProxy = (request, response) ->  

  options = {host: host, port: port, method: request.method, headers: request.headers, path: request.url}

  proxyRequest = http.request(options, (proxyResponse) ->
    proxyResponse.on('data', (chunk) ->
     response.write(chunk, 'binary')
    )

    proxyResponse.on('end', (chunk) ->        
     response.end()
    )

    response.writeHead proxyResponse.statusCode, proxyResponse.headers    
  )

  request.on('data', (chunk) ->
    proxyRequest.write(chunk, 'binary')
  )

  # this is never triggered for GETs
  request.on('end', ->
    proxyRequest.end()
  )

  # so I have to have this here
  proxyRequest.end()

You will notice proxyRequest.end() on the final line above.

What I have found is that when handling GET requests, the END event of the request is never triggered and therefore a call to proxyRequest.end() is required. POST requests trigger both DATA and END events as expected.

So several questions:

• Is this call to proxyRequest.end() safe? That is, will the proxyResponse still be completed even if this is called outside of the event loops?

• Is it normal for GET to not trigger END events, or is the END being captured somewhere in the connect stack?

Patrolman

The finish event seems to work as suggested here – Erelong

Answer 1

The problem is less the end event and more the data event. If a client makes a GET requests, there's headers and no data. This is different from POST, where the requester is sending data, so the on("data") handler gets hit. So (forgive me for the JS example, I'm not that familiar with coffeescript):

var http = require('http');

// You won't see the output of request.on("data")
http.createServer(function (request, response) {
  request.on("end", function(){
    console.log("here");
  });
  request.on("data", function(data) {
    console.log("I am here");
    console.log(data.toString("utf8"));
  });
  response.writeHead(200, {'Content-Type': 'text/plain'});
  response.end('Hello World\n');
}).listen(8124);

console.log('Server running at http://127.0.0.1:8124/');

If I make a curl call to this server, the data event never gets hit, because the GET request is nothing more than headers. Because of this, your logic becomes:

// okay setup the request...
// However, the callback doesn't get hit until you
// start writing some data or ending the proxyRequest!
proxyRequest = http.request(options, (proxyResponse) ->
  // So this doesn't get hit yet...
  proxyResponse.on('data', (chunk) ->
   response.write(chunk, 'binary')
  )

  // and this doesn't get hit yet
  proxyResponse.on('end', (chunk) ->
   // which is why your response.on("end") event isn't getting hit yet        
   response.end()
  )

  response.writeHead proxyResponse.statusCode, proxyResponse.headers    
)

// This doesn't get hit!
request.on('data', (chunk) ->
  proxyRequest.write(chunk, 'binary')
)

// So this isn't going to happen until your proxyRequest
// callback handler gets hit, which hasn't happened because
// unlike POST there's no data in your GET request
request.on('end', ->
  proxyRequest.end()
)

// now the proxy request call is finally made, which
// triggers the callback function in your http request setup
proxyRequest.end()

So yes you're going to have to manually call proxyRequest.end() for GET requests due to the logic branching I just mentioned.

Girish

Using your example script above, the request.on("end") callback is triggered on GET, but the data event is not ... which makes me think something funky is happening inside the connect infrastructure ... – Patrolman

Answer 2

my experience is that request.on('end',) is not consistently called unless it's a POST. I suspect the event (of someone making a http.request) is over before the script gets a chance to detect it.

Trabzon

I'm wrestling with this very same issue. it's maddening. – Jordan