PHP output JSON Web Service charset UTF-8 error

Question

I am hosting a web service in JSON output by PHP.

I have Hebrew data set in DB and I am posting this as an output to Web service.

When I post the data initially it output the result as follows:

JSON:

{
    "tasklist": [
        {
            "customerID": "9936",
            "name": "טר ×רמה ×™×–×•× ×•×‘×™× ×•×™ בע"מ",
            "cargo":"×ברר",
            "destination":"מכר", 
            "quantity":"1.000000",
            "startdate":"03/01/201300: 00: 00" 
        }
               ]
}

But this "×ברר" can be readable by Android/Iphone parser and convert it to original Hebrew. But i faced Error in "name": "טר ×רמה ×™×–×•× ×•×‘×™× ×•×™ בע"מ",. where " is in between the string so the JSON is not valid and shows error!

To Over come this issue I used UTF-8 to convert "×ברר" this to Hebrew "נברר". But in this case too the problem remains same:

PHP:

header('Content-type: text/html; charset=UTF-8');

JSON:

{
    "tasklist": [
        {
            "customerID": "9936",
            "name": "טר ארמה יזום ובינוי בע"מ",
            "cargo":"נברר",
            "destination":"מכר",
            "quantity":"1.000000",
            "startdate":"03/01/201300: 00: 00"
        }
                ]
}

But still the problem remains:

Also in some case I am getting this � because of using UTF-8

"name":"מחצבות כפר גלעדי-חומרי מ�"

• How can I overcome this issue?
• Is there any other specific encode I need to use?

Note: The data cannot be changes in Database The solution should be while output to JSON.

How the data stored in DB is shown below:

name

מחצבות כפר גלעדי-חומרי מ×

My PHP Script which output JSON:

<?php

//My DB connection and Query comes here

$jsontext = '{"tasklist":[';
while($row = mysql_fetch_array($queryExe)){
$jsontext .= '{"customerID":"'.$row['AUTO_ID'].'",';
$jsontext .='"name":"'.$row['Customer_Name'].'",';
$jsontext .='"cargo":"'.$row['Type_of_Cargo'].'",';
$jsontext .='"destination":"'.$row['Destination'].'",';
$jsontext .='"quantity":"'.$row['Quantity'].'",';
$jsontext .='"startdate":"'.$row['startdate'].'"},';
}
$jsontext = substr_replace($jsontext, '', -1); // to get rid of extra comma
$jsontext .= "]}";

header('Content-type: text/html; charset=UTF-8');

//Output the final JSON
echo $jsontext;

?>

Thank you for your help in advance!

Was the question clear? to understand my issue.

Caryl

how are you creating your json? I would do the following: 1. read values from db, 2. convert each value to utf8 using utf8_encode, 3. put it into a php array and finally 4. do json_encode and echo – Isahella

@Isahella I have posted my PHP script have a look at it. I am not encoding initially only while output to web service I change the header! – Caryl

setting the header is not enough. You need utf8_encoded strings. You get them by using utf8_encode if your db-field is not utf8. – Isahella

@Isahella my DB field is utf8_general_ci – Caryl

This has to do with mysql but there's no tag about it. I supose many people end up here looking for: header('Content-Type: application/json'); or header("Content-type: application/json; charset=utf-8"); – Gigantean

Answer 1

If your db-field is utf8 you should fist do:

mysql_query("SET NAMES 'utf8'");

You should always do the 'SET NAMES...' before inserting your data, too. Be sure that you really stored utf8 encoded strings!

then do your query:

mysql_query($your_query);

$array = array("tasklist"=>array());    

while($row = mysql_fetch_array($queryExe)){
        $a = array();
        $a["customerID"] = $row['AUTO_ID'];
        $a["name"] = $row['Customer_Name'];
        $a["cargo"] = $row['Type_of_Cargo'];
        $a["destination"] = $row['Destination'];
        $a["quantity"] = $row['Quantity'];
        $a["startdate"] = $row['startdate'];
        $array["tasklist"][] = $a;
}

header("Content-type: application/json; charset=utf-8");

echo json_encode($array);
exit();

i've made the experience that these is not enough when the servers default charset is for example iso. In that case i need to do the following in my .htaccess:

AddDefaultCharset utf-8

Isahella

can you check I have added the JSON output for your code. near name "x" it cannot be parsed in Android / iphone? – Caryl

{ "tasklist": [ { "customerID": "9936", "name": "×ר ×ר×× ×××× ×××× ×× ××¢\"×", "cargo": "× ×רר", "destination": "××ר", "quantity": "1.000000", "startdate": "03/01/2013 00:00:00" } ] } – Caryl

hmmm the documentation of utf8_encode says that data shoud be an ISO-8859-1 string. I dont think that your string is such. you should tell us what the encoding of your db-field is. I would recommend utf8 and in that case you should SET NAMES 'utf8' like @BigToach sayed. In that case you don't need to utf8_encode with php. Hope it helps. – Isahella

my DB field is utf8_general_ci I even tried BigToach method but still same error.. – Caryl

Answer 2

You should change your code to use json_encode. You need to pass it properly utf8 encoded data.

If you are using MySQL you can try running the following before your query to get your data.

SET NAMES 'utf8';

You can also look into using utf8_encode.

Wil

BigToach mysql_set_charset('utf8'); still same issue :( any idea? – Caryl

Did you try changing your code to use json_encode instead of generating it yourself as well? – Wil

Yes.. I did as per the another answer and mingled your code in it.. but same error.. JSON validation error only in names. – Caryl

Answer 3

From http://www.php.net/manual/en/function.json-encode.php#100565

That said, quotes " will produce invalid JSON, but this is only an issue if you're using json_encode() and just expect PHP to magically escape your quotes. You need to do the escaping yourself.

May be you can replace " with \" , i guess it will solve the issue.

Source : PHP JSON String, escape Double Quotes for JS output

Oder