Javax Websocket closing due to Illegal UTF-8 Sequence

Question

I'm writing a Websocket client in Java, using javax.websocket API, and org.glassfish.tyrus as the implementation.

Everything usually works, but sometimes, when I'm receiving very large strings, the connection closes with a mysterious 'Illegal UTF-8 Sequence' as the close reason.

log.info("Ws closed cuz: " 
   + reason.getCloseCode() + " , " 
   + reason.getReasonPhrase() + " , " 
   + reason.toString());

Output:

INFO: Ws closed cuz: NOT_CONSISTENT , Illegal UTF-8 Sequence ,
CloseReason[1007,Illegal UTF-8 Sequence]

I'm guessing that either the string was too large, or the string contained any characters which aren't UTF-8 compatible.

Is there a way to get any more info on the actual string / packet / frame which causes this issue? Or, if there's a way to tell tyrus to ignore any encoding issues and just pass me the raw string and let me handle it?

If not, is there another java websockets client which does the bare bones work of transmitting the strings over socket and doesn't do any validation, and just lets me handle the responses?

Appreciate any feedback.

Eshman

Parsing reason is an option? – Simple

can you debug the client and see which are offending bytes? – Goodlooking

I had this same problem with the ASP.NET websocket implementation. In my case, the problem turned out to be that the large string was split in the middle of a multi-byte character representing a scandinavian letter. See github.com/dotnet/corefx/issues/29834. There might be a similar with the Java client. – Firetrap

Answer 1

The following is just a guess.

(1) On the server side, the large string is split into one text frame and one or more following continuation frames. Technically, the original large string is converted into a byte array and then the byte array is split into multiple sub byte arrays. The sub arrays are set to frames one by one (= Each frame contains one sub byte array).

(2) Although there is no guarantee that each sub byte array is a valid UTF-8 sequence, validity check is performed either on the server side or on the client side. If so, it's a bug of Tyrus.

WebSocketListener of nv-websocket-client has callback methods in frame granularity such as onFrame, onTextFrame, onContinuationFrame and others (note that onTextMessage and onTextFrame are different), so you can examine the byte array of each frame there.

WebSocket websocket = new WebSocketFactory()
    .createSocket("ws://...")
    .addListener(new WebSocketAdapter() {
        @Override
        public void onFrame(WebSocket ws, WebSocketFrame frame) {
            // If the frame is a text frame with FIN bit cleared, or
            // if the frame is a continuation frame.
            if ((frame.isTextFrame() && frame.getFin() == false) ||
                frame.isContinuationFrame()) {
                // The payload of the frame. There is no guarantee
                // that this byte array is a valid UTF-8 sequence.
                byte[] payload = frame.getPayload();

                // Check whether the payload is a valid UTF-8 sequence
                // if you want to.
                checkPayload(payload);
            }
        }
    })
    .connect();

Why don't you use nv-websocket-client to examine what is happening in your WebSocket connection?

Lody

Yeah, setting a breakpoint, I saw that a Utf8DecodeException or something was being thrown by Tyrus / Glassfish, but couldn't see the exact frame which caused it. I switched to your library, and everything seems to be working well so far, using onTextMessage. Thanks – Eshman

Is there a way to set the exception handler for any exceptions that might occur in the WebsocketAdapter methods? – Eshman

WebSocketListener has some onXxxError() methods such as onFrameError and onSendError. Among such methods, onError() is a special one. It is always called before any other onXxxError() is called. For example, in the implementation of run() method of ReadingThread and WritingThread, Throwable is caught and onError() and onUnexpectedError() are called in this order. So, although there is no means to register a custom exception handler, you can handle all error cases in onError(). See WebSocketError for possible error cases. – Lody

where exactly do you specify WebsocketListener? – Eshman

WebSocketAdapter is an empty implementation of WebSocketListener. – Lody

It doesn't seem like exceptions triggered inside the onTextMessage method are caught by setting an onError method. Things like 'unexpected end of stream' are being caught, but if an NPE occurs within the onTextMessage method of the adapter, it silently fails and I have to usual manual logs / debugger stepping to find the cause. Any ideas? – Eshman

Oh, I understood what you had meant just now. The implementation (ListenerManager) ignores any Throwable raised from within onXxx methods. – Lody

Added handleCallbackError() method to WebSocketListener. Try nv-websocket-client version 1.9. – Lody

Answer 2

I just had the same error. After I changed from org.glassfish.tyrus version 1.1 to 2.0.0 it worked without any problems.

Infantry