htmlspecialchars remove the value inside the array?

Question

So I'm uploading a csv file. One of the column have a single quote.

145 Test St'

The array is

(
    [0] => 2
    [1] => 20
    [2] => 145 Test St�
    [3] => Test City
    [4] => 1455
    [5] => 919749797
)

As you can see. Instead of single quote ( ' ), it becomes �.

From here, I use htmlspecialchars($row), which gives the result.

(
    [0] => 2
    [1] => 20
    [2] => 
    [3] => Test City
    [4] => 1455
    [5] => 919749797
)

First question, why ( ' ) becomes ( � ) ?

Second question, why after using htmlspecialchars(), the value disappear?

Third question, How can I retain the ( ' ) ?

Thanks for those who can answer.

EDIT:

  $row  = array_map('str_getcsv', file($_FILES['file']['tmp_name']));
        $csv  = Array();
        $head = $row[0];
        $col  = count($row[0]);
        unset($row[0]);

        pre($row[1]);

        $row[1] = array_map('htmlentities', $row[1]);

        pre($row[1]);

EDIT:

pre() is a function I created that works like

<pre></pre>.

EDIT:

I've look at the CSV file using file --mime at the terminal. It's charset is unknown 8-bit. I convert the CSV file to UTF-8 by doing a save as. After that I manage to upload the CSV file successfully. The problem is on the encoding of the CSV file.

Is it possible to convert the file into UTF-8?

Higgler

the file is utf8?, or you read file use utf8? can you show your code? thanks – Conchoidal

@Conchoidal what file? the csv? – Higgler

yes csv file, and can you show your read file code? – Conchoidal

Sorry, but how to know if a csv file is utf8? I'll update my question wait.. – Higgler

can you paste the definition of pre()? otherwise is hard to answer to your question :) – Unbelief

Sorry!! haha. pre() is actually a function I created. It works similar with <pre> to see the value of array easily. But don't mind it. – Higgler

you can check this https://mcmap.net/q/385823/-utf-8-problems-while-reading-csv-file-with-fgetcsv – Conchoidal

Answer 1

According to php.net's htmlspecialchars page :

"If the input string contains an invalid code unit sequence within the given encoding an empty string will be returned, unless either the ENT_IGNORE or ENT_SUBSTITUTE flags are set."

So the solution is: use "$variable = htmlspecialchars( $string, ENT_IGNORE);" You can create a function with "htmlspecialchars" and array map that function like this -

function specialchars($string){
    return htmlspecialchars( $string, ENT_IGNORE);
}


$row  = array_map('str_getcsv', file($_FILES['file']['tmp_name']));
$csv  = Array();
$head = $row[0];
$col  = count($row[0]);
unset($row[0]);
pre($row[1]);
$row[1] = array_map('specialchars', $row[1]);
pre($row[1]);

Foley

This delete the single quote. I need to retain the single quote. – Higgler

I've guessed your php (which read the csv file) file is not utf-8 formatted and it is still using ISO-8859-1 format instead. Convert your php file as UTF-8 without BOM with your prior code. If it it will not works then let me know. – Foley