Heroku sock=backend Server Request Interrupted for a Flask SocketIO application

Question

I have a flask-socketio app hosted on heroku with the following Procfile :

web: gunicorn --worker-class eventlet hello:app

Ever since I switched to socketio, the app has been behaving inconsistenly. Earlier the app would run for a while and then POST requests would start timing out.

Since yesterday night, I keep getting the error

sock=backend at=error code=H18 desc="Server Request Interrupted" method=GET path="/static/js/third-party/browser.js" host=deard.herokuapp.com request_id=725da6af-aa29-4293-a411-2c89977f1d4d fwd="216.165.95.0" dyno=web.1 connect=1ms service=36ms status=503 bytes=13811

I read the heroku error code description which says "The backend socket, belonging to your app’s web process was closed before the backend returned an HTTP response."

But I have no clue why this would be happening.

Any clue as to what I should be looking out for, will help me debug this.

I could share the code but its 300 lines long and since I don't know where the error is originating from, I am not sure if it would be helpful.

Curtsy

do you have the same problem (or any others) when you run the app locally? (with gunicorn) – Albarran

No it runs fine locally with gunicorn. – Curtsy

You could try changing the procfile worker class tag as: --worker-class socketio.sgunicorn.GeventSocketIOWorker. I am following this here. – Albarran

I had read your blog before I am using eventlet so would avoid changing to gevent. I tried installing gevent and changing the Procfile and gunicorn would not run. still figuring out whats wrong. – Curtsy

@Albarran No, your advice does not apply to this. The GeventSocketIOWorker class is used when you use gevent-socketio. But Flask-SocketIO does not use that package. – Mortar

@DennyGeorge any other errors in your heroku log? You can also run SocketIO with logging turned on. For that add logger=True, engineio_logger=True to your SocketIO constructor. – Mortar

@Miguel I had overcome this issue by adding -no-sendfile to my Procfile after which I started getting error where my websocket connection would keep timing out, which I later learnt (from reading something you wrote) that was caused by gunicorn 19.x. I downgraded to gunicorn 18.0 last night and everything worked fine. Now from this mornign onwards, I am getting the same "Server Request Interrupted" error. i tried adding the code you suggested into the constructor. I don't think there is any new error. I pasted the log here - gist.github.com/dennyabrain/cc7fd78d51cf184d6572 – Curtsy

That still only heroku logging, there is nothing from Flask-SocketIO. – Mortar

@Miguel, I added what you asked me to in the socketio constructor. Would you know if I have to do anything additional to enable it to go to the heroku logs? – Curtsy

@DennyGeorge as long as you see the output in your console when you run locally (like using foreman on honcho), then heroku should pick up the logs and include them in their own logs. – Mortar

@miguel, oh sorry. I thin i was running old code on heroku. Now the logs look like this - gist.github.com/dennyabrain/2ec31f551e3b6b087ccf I am new to websockets and so i might be way off here but is it possible that this is happening because in the logs i see my app is listening on port 45173 and I remember reading that socketio library will read on port 80. could that be a reason? – Curtsy

and also, I get a GET deard.herokuapp.com/static/js/third-party/react.min.js net::ERR_CONTENT_LENGTH_MISMATCH error on the frontend (chrome console) – Curtsy

I am experiencing the same issue on a Heroku app using Flask-SockteIO. In the Chrome console I see a net::ERR_CONTENT_LENGTH_MISMATCH and in the server logs I see error code H18 during what appears to be a request for a large JavaScript asset. – Vena

@DennyGeorge —> Did you guys ever resolve this? – Nyhagen

@Vena —> Did you guys ever resolve this? – Nyhagen

Answer 1

This kind of "socket stream error" is raised by Flask when you cancel or resend the same request while on the server side (flask) you are still under computation in order to render layout corresponding to the initial request.

So it may be relative to timeout or network Error handling on both side (server/client).

Some question :

How long the request computation is during on server side ? How do you handle network error exception on server and client side ? Does exist any timeout on flask or client side ?

It is also possible that a new request is sent before the previous one is over..

Please share some code relative to request processing.

Farinose

Answer 2

This worked for me:

from waitress import serve
# app.run(host='0.0.0.0', port=port) # <---- REMOVE THIS
# serve your flask app with waitress, instead of running it directly.
serve(app, host='0.0.0.0', port=port) # <---- ADD THIS

Collocate