UTF-8 encoding in Volley Requests

Question

In my Android app I am loading json data with a Volley JsonArrayRequest. The data were created by myself and I saved them with Sublime with UTF-8 encoding. When I get the Response and fill my ListView, the texts are not displayed correctly (umlauts). This is what my Request looks like:

JsonArrayRequest request = new JsonArrayRequest(targetUrl,
                new Response.Listener<JSONArray>() {
                    @Override
                    public void onResponse(final JSONArray response) {
                        try {
                            fillList(response);
                        } catch (JSONException e) {
                            e.printStackTrace();
                        }
                    }
                },
                new Response.ErrorListener() {
            @Override
            public void onErrorResponse(VolleyError error) {

            }
        });
        requestQueue.add(request);

When I load the exact same data with this method, all texts are displayed correctly:

final StringBuilder builder = new StringBuilder();
        final HttpClient client = new DefaultHttpClient();
        final HttpGet httpGet = new HttpGet(request);
        try {
            final HttpResponse response = client.execute(httpGet);
            final StatusLine statusLine = response.getStatusLine();
            int statusCode = statusLine.getStatusCode();
            if (statusCode == 200) {
                final HttpEntity entity = response.getEntity();
                final InputStream content = entity.getContent();
                final BufferedReader reader = new BufferedReader(new InputStreamReader(content));
                String line;
                while ((line = reader.readLine()) != null) {
                    builder.append(line);
                }
            } else {

            }
        } catch (ClientProtocolException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

So to me it seems like there is no problem with the encoding of my json file. How can I fix this encoding problem in Volley?

Bax

For what it's worth, I have used UTF-8 encoded JSON files and it has worked correctly. Are you setting the "Content-Type" header when you serve the file? Volley should read that and build it's response appropriately. – Calamine

No, I don't set this header. I didn't know that I have to do that. DoO you have an example for me? – Bax

Are you serving the JSON files on a server? Wherever you are processing the request for those files, you want to specify a return header with something like "Content-Type text/json charset=utf-8". – Calamine

I just uploaded the JSON files on my private webspace via FTP. – Bax

Answer 1

If you know that absolutely all of the files you are requesting will be in the UTF-8 format, which it sounds like you do, then you might consider forcing your Volley request to return UTF-8 formatted strings. You could accomplish this by subclassing the standard JSON request. Something like this:

public class Utf8JsonRequest extends JsonRequest<JSONObject> {
    ...
    @Override
    protected Response<JSONObject> parseNetworkResponse (NetworkResponse response) {
        try {
            String utf8String = new String(response.data, "UTF-8");
            return Response.success(new JSONObject(utf8String), HttpHeaderParser.parseCacheHeaders(response));
        } catch (UnsupportedEncodingException e) {
            // log error
            return Response.error(new ParseError(e));
        } catch (JSONException e) {
            // log error
            return Response.error(new ParseError(e));
        }
    }
}

Calamine

I see now that you are using a JsonArrayRequest. You could subclass that instead. I don't have the syntax handy, but it would be pretty similar. – Calamine

It works. And overriding parseNetworkResponse in my StringRequest also solved the problems I had with some XML Feeds I was loading. Thanks @xaevinx – Bax

Hey @Bax , how did this fix your problem? I tried extending both JsonArrayRequest and JsonRequest<JSONArray> and none worked. I still get "?????" as the response. When I go to the php service in Chrome json is displayed properly, however when debugging, when reading the received json it appears to be ???? so it should be Volley's fault again. It's strange that we have to override Volley's encoding as in JsonRequest class, it's using UTF8 - protected static final String PROTOCOL_CHARSET = "utf-8"; – Deadbeat

why HttpHeaderParser.parseCharset(response.headers) isn't working correctly? – Odawa

Answer 2

I have same problem like this and i solve it using UTF-8 charset.

String str = "";
try {
     str = new String(strFromService.getBytes("ISO-8859-1"), "UTF-8");
} catch (UnsupportedEncodingException e) {

 e.printStackTrace();
}

String decodedStr = Html.fromHtml(str).toString();

I hope this will work for you

Bick

Answer 3

donot use try{} catch in the onResponse block, that is giving some problem in my code , rather than that you can implement like this .

@Override 
onResponse(String s) {

s= fixEncoding(s);
Toast.makeToast(this,s,Toast.LENGTH_LONG).show();

}

and i think you will get the required result

 public static String fixEncoding(String response) {
            try {
                byte[] u = response.toString().getBytes(
                        "ISO-8859-1");
                response = new String(u, "UTF-8");
            } catch (UnsupportedEncodingException e) {
                e.printStackTrace();
                return null;
            }
            return response;
        }

Showman

Answer 4

This is worked as a charm for me,, I just created a static method from @Muhammad Naeem's answer,, Thanks Muhammed..

public static String fixEncodingUnicode(String response) {
    String str = "";
    try {
        str = new String(response.getBytes("ISO-8859-1"), "UTF-8");
    } catch (UnsupportedEncodingException e) {

        e.printStackTrace();
    }

    String decodedStr = Html.fromHtml(str).toString();
    return  decodedStr;
}

Dukedom

Answer 5

Add this line of code inside response of volley :

 public void onResponse(String response) 
    {

     if (response != null) 
      {
        response=new String(response.getBytes("ISO-8859-1"), "UTF-8");
      }
   }

Wabble

Answer 6

another approach: you can encode your whole response by this line of code.

newStr = URLDecoder.decode(URLEncoder.encode(oldStr, "iso8859-1"),"UTF-8");

I have encoded whole response string into UTF-8 as I know volley default encode method is iso8859-1

Dumah

Answer 7

To specify in server part set utf-8 in content type like "text/html;charset=utf-8"

Giana

Answer 8

I had the same problem earlier two days ago and i have tried every thing my friends just said and it`s the same problem and i tried so meany things .. my file is written in php and using volley to get the data as json i followed these steps to solve the problem ..

1. before executing the query in your database write this query

   mysqli_query($this->connection,"set_NAMES_utf8");

2. use this header in your file if it`s php

   header('Content-Type: application/json ; charset=utf-8 ');

3. while you are echoing the json according to php there is a solve for this problem use this special charachter in json_encode($json,JSON_UNESCAPED_UNICODE)

   i hope it helped

Cassation

Answer 9

I just use this code line in my public void onResponse(){} method and It's work pretty well.

          if (response != null){
                try {
                    response=new String(response.getBytes("ISO-8859-1"), "UTF-8");

                } catch (UnsupportedEncodingException e) {
                    e.printStackTrace();
                }
            }

Little

Answer 10

StringRequest request = new StringRequest(URL, new Response.Listener<String>() {
        @Override
        public void onResponse(String response) {
            if (response != null){
                dialog.dismiss();
                try {
                    response=new String(response.getBytes("ISO-8859-1"), "UTF-8");
                    JSONArray jsonArray = new JSONArray(response);
                    parseArray(jsonArray);

                } catch (JSONException | UnsupportedEncodingException e) {
                    e.printStackTrace();
                }
            }

        }
    }

Pave