SSE events are not reaching client until res.end() is called

Question

The same code used to work before but i dont know why it is not working now.Please help.

My problem is that when i use SSE for real time data sharing.The data object which should be sent on res.write(data:${JSON.stringify(dataObject)}\n\n) is not being sent until res.end() is called and all the data is event streams are sent at once.

response.writeHead(200, {
    Connection: "keep-alive",
    "Content-Type": "text/event-stream",
    "Cache-Control": "no-cache",
    "Access-Control-Allow-Origin": '*',
    'Content-Encoding':'identity' // this with and without this
});

let syncher= setInterval(() => { 
    if(response.finished){ // If response ended the  interval is cleared
        clearInterval(syncher);
        return;
    }else{
        let dataToSend = getEventData(user,event);
        if(! dataToSend ){
            response.write('data:{close:true}');
            clearInterval(syncher);
            return;
        }
        response.write(`data:${JSON.stringify(dataToSend)}\n\n`);
        response.flushHeaders(); // Also tried with response.flush()
        if(dataToSend.close){
            delEventData(user,event);
            response.end();
        }
    }
}, 500);

The above code is in server side this also has on close listener to close the connection

const ev = new EventSource(conf.apiUrl+'/getStatus/'+ (userData.id || '') );
  let data = '';
  ev.onmessage = eventData=>{
    data = JSON.parse(eventData.data);
    if(!data){
      setState('progress '+data.completedSoFar)
      return;
    }
    if(!data.close){

    }else{
      if(data.success){
        console.log('Done Successfully')
      ev.close();
    }
  }

This is my client side code

I don't know why the event listener is not getting data stream while i searched the internet about this issue i only found that when compression middleware is used this issue occurs .I don't use any compression middleware in my app. I am using nodejs v11.4.0. I am guessing that when i am making eventsource request chrome is adding gzip encoding by default and node is using that to set response encoding header as gzip I tried to delete and replace it but did'nt work which is causing this issue??

Here is my request and response headers for eventSource request

Sorry for my grammar if i made any mistakes.

Thanks for help. Cheers!

Astrict

Answer 1

Nextjs is basically compressing your data to make it transmit faster. Unfortunately, this makes it so we can't see the data until we flush the cache (my guess is render behavior has changed because compression has changed the content). You can disable compression entirely by including compress: false in your next.config.

I found here that including the following header sidesteps the compression for a specific endpoint: res.setHeader("Cache-Control", "no-cache, no-transform");

Caution: This will increase bandwidth/resource usage! HTTP compression can reduce the size of your data by 70%.

Brasca

God bless you. This is really what I found!! – Pretension

Answer 2

After lot of debugging and research. The problem turned to be in the webpack-dev-server which compressed my responses. For more info refer https://github.com/facebook/create-react-app/issues/966

Astrict