Intermittent Error during WebSocket handshake: Unexpected response code: 400 on CloudBees

Question

I am running a websocket app on CloudBees - and I intermittently see:

Error during WebSocket handshake: Unexpected response code: 400

I have told it to use http 1.1 to allow Upgrades via:

bees app:proxy:update http_version=1.1

And it works, but I sometimes see the error (not always).

Corri

Answer 1

This is almost certainly due to not using https (SSL). Websocket over plain http is vulnerable to proxies in the middle (often transparent) operating at the http layer breaking the connection.

This is common on cellular networks, or office networks that may use multiple wireless connections with a proxy that spreads the http requests across connections.

The only way to avoid this is to use SSL all the time - this gives websocket the best chance of working.

Corri

Answer 2

Adding to Michael Neale's solution.

As stated there, Play doesn't support WSS natively, as of late October, 2013.

So simply switching to SSL wouldn't work.

Thankfully, when configuring an app to use SSL, Cloudbees sets up an Nginx server as a router, with the SSL endpoint being the router, so the workaround described there will work.

So, once you create a custom domain name and corresponding Cloudbees app alias, set up your SSL certificates in a Cloudbees router, and configure your app to be use that Cloudbees router, you'll be able to connect to the websockets.

But you'll have to force the URLs to be secure, since using the regular Play route resolvers won't work. They return ws://..., not wss://... websockets URLs.

Specifically, using the out-of-the-box Play Framework sample Scala Websocket Chat app as an example:

1. conf/routes defines:

   GET /room/chat controllers.Application.chat(username)
   

2. Application defines:

   def chat(username: String) = WebSocket.async[JsValue] { request => ChatRoom.join(username) }
   

3. and chatRoom.scala.js creates the web socket:

   var WS = window['MozWebSocket'] ? MozWebSocket : WebSocket 
   var chatSocket = new WS("@routes.Application.chat(username).webSocketURL()") 
   

That won't work, since @routes....webSocketURL() will return a ws://, not a wss:// url.

chatRoom.scala.js can be modified as follows to make it work regardless of whether it's running within an https:// or http:// page:

var WS = window['MozWebSocket'] ? MozWebSocket : WebSocket; 
var wsUrl = "@controllers.api.routes.PubSubController.chat(username).webSocketURL()"; 
if(window.location.protocol == "https:") wsUrl = wsUrl.replace("ws:","wss:"); 
var chatSocket = new WS(wsUrl);

Hope this helps.

Paragrapher

shouldn't the X-Forwarded-Proto header tell play that the connection was secure - and to use wss when writing urls? if it doesn't - that is a play bug I think. – Corri

Answer 3

If it is intermittent, it might be that your client library has some trouble forming a valid handshake after a time. It would informative to run Wireshark to capture HTTP requests containing Connection: Upgrade headers to validate that the handshake request is valid.

For ways on how this could happen, see subsection 4.2.1 of the WebSockets RFC 6455:

    The client's opening handshake consists of the following parts.  If
    the server, while reading the handshake, finds that the client did
    not send a handshake that matches the description below (note that as
    per [RFC2616], the order of the header fields is not important),
    including but not limited to any violations of the ABNF grammar
    specified for the components of the handshake, the server MUST stop
    processing the client's handshake and return an HTTP response with an
    appropriate error code (such as 400 Bad Request).

Strabismus

It could well be - the specific case that prompted this was someone in a remote office that happened to run over 2 cellular connections - with some proxy that would balance between them. SSL was in the works anyway - so it can help with that. I believe in this case it was chrome - no library outside of that used. – Corri

Having deployed a few public applications using WebSockets, wss:// is a requirement, especially for major cellular carrier networks in the USA. AT&T, Verizon, T-Mobile networks may not have WebSocket-handshake-aware intermediaries everywhere for every route. Chrome seems to have implemented the WebSocket specification pretty well, so I'd lean towards these network issues for sure. – Strabismus