How to use strtr() to translate multibyte/accented/diacritic characters?

Question

Does anyone have a multibyte variant of the strtr() function?

Example of desired usage:

Example:
$from = 'ľľščťžýáíŕďňäô'; // these chars are in UTF-8
$to   = 'llsctzyairdnao';

// input - in UTF-8
$str  = 'Kŕdeľ ďatľov učí koňa žrať kôru.';
$str  = mb_strtr( $str, $from, $to );

// output - str without diacritic
// $str = 'Krdel datlov uci kona zrat koru.';

Rabe

I dont have an exact example at hand, but it is always worth to have a look at the user comments on phps documentation page: us3.php.net/strtr it seems there are people that already had the same problem. Maybe one of them posted the solution already there. – Import

Answer 1

I believe strtr is multi-byte safe, either way since str_replace is multi-byte safe you could wrap it:

function mb_strtr($str, $from, $to)
{
  return str_replace(mb_str_split($from), mb_str_split($to), $str);
}

Since there is no mb_str_split function you also need to write your own (using mb_substr and mb_strlen), or you could just use the PHP UTF-8 implementation (changed slightly):

function mb_str_split($str) {
    return preg_split('~~u', $str, null, PREG_SPLIT_NO_EMPTY);;

}

However if you're looking for a function to remove all (latin?) accentuations from a string you might find the following function useful:

function Unaccent($string)
{
    return preg_replace('~&([a-z]{1,2})(?:acute|cedil|circ|grave|lig|orn|ring|slash|th|tilde|uml|caron);~i', '$1', htmlentities($string, ENT_QUOTES, 'UTF-8'));
}

echo Unaccent('ľľščťžýáíŕďňä'); // llsctzyairdna
echo Unaccent('Iñtërnâtiônàlizætiøn'); // Internationalizaetion

Assizes

There is no function mb_str_split – Import

This doesn't work correctly for all strings. For example echo mb_strtr("a", 'a'.unichr(769), "b"); will display b, while I would expect a since unichr(769) is not in the original string. – Dion

@BertR: That's the correct behavior of [mb_]strtr: a is replaced by b and unichr(769) is ignored since there's no corresponding character in $to. – Assizes

@Alix Axel: You are correct. I checked the documentation of strtr: "If from and to have different lengths, the extra characters in the longer of the two are ignored. The length of str will be the same as the return value's. " – Dion

A minor improvement: add "caron" in the regular expression like so: ... slash|th|tilde|uml|caron); .... Now texts such as 'Škoda' will also be unaccented. – Occlusion

@bilygates: Very nice, thank you! I don't remember caron coming up when I was looking into this. – Assizes

@bilygates: I went to check if caron diacritics were supported by PHP and it still doesn't show up on my get_html_translation_table(HTML_ENTITIES) call. What version of PHP are you using? And can you see &...caron; entities in that lookup table? If not, htmlentities() should never encode those diacritics, which renders the additional regex lookup/replace useless. – Assizes

@Alix Axel: Use get_html_translation_table(HTML_ENTITIES, ENT_QUOTES, 'UTF-8') with the argument for UTF-8 encoding and you will see Š and š. I'm using PHP 5.3.12. Cheers! – Occlusion

@bilygates: Yeah, that must be it, I'm using PHP 5.3.2 and the encoding argument was only added in PHP 5.3.4. Thanks for getting back to me. – Assizes

Answer 2

function mb_strtr($str,$map,$enc){
$out="";
$strLn=mb_strlen($str,$enc);
$maxKeyLn=1;
foreach($map as $key=>$val){
    $keyLn=mb_strlen($key,$enc);
    if($keyLn>$maxKeyLn){
        $maxKeyLn=$keyLn;
    }
}
for($offset=0; $offset<$strLn; ){
    for($ln=$maxKeyLn; $ln>=1; $ln--){
        $cmp=mb_substr($str,$offset,$ln,$enc);
        if(isset($map[$cmp])){
            $out.=$map[$cmp];
            $offset+=$ln;
            continue 2;
        }
    }
    $out.=mb_substr($str,$offset,1,$enc);
    $offset++;
}
return $out;
}

Ingredient

Answer 3

Probably using str_replace is a good solution. An alternative:

<?php
header('Content-Type: text/plain;charset=utf-8');

function my_strtr($inputStr, $from, $to, $encoding = 'UTF-8') {
        $inputStrLength = mb_strlen($inputStr, $encoding);

        $translated = '';

        for($i = 0; $i < $inputStrLength; $i++) {
                $currentChar = mb_substr($inputStr, $i, 1, $encoding);

                $translatedCharPos = mb_strpos($from, $currentChar, 0, $encoding);

                if($translatedCharPos === false) {
                        $translated .= $currentChar;
                }
                else {
                        $translated .= mb_substr($to, $translatedCharPos, 1, $encoding);
                }
        }

        return $translated;
}


$from = 'ľľščťžýáíŕďňä'; // these chars are in UTF-8
$to   = 'llsctzyairdna';

// input - in UTF-8
$str  = 'Kŕdeľ ďatľov učí koňa žrať kôru.';

print 'Original: ';
print chr(10);
print $str;

print chr(10);
print chr(10);

print 'Tranlated: ';
print chr(10);
print my_strtr( $str, $from, $to);

Prints on my machine using PHP 5.2:

Original: 
Kŕdeľ ďatľov učí koňa žrať kôru.

Tranlated: 
Krdel datlov uci kona zrat kôru.

Import

Answer 4

strtr() has two valid signatures for receiving its parameters.

The way that you have implemented strtr() performs byte-by-byte translations -- this is obviously inappropriate for your multibyte characters.

$from = 'ľľščťžýáíŕďňäô'; // these chars are in UTF-8
$to   = 'llsctzyairdnao';

$str  = 'Kŕdeľ ďatľov učí koňa žrať kôru.';
echo strtr($str, $from, $to);
// Kd�deyn y�atynov uyaa� kod�a dnradr ka�ru.

---

The correct implementation is to feed the function an associative array of characters to translate -- this is the multibyte-safe way. (Demo)

$trans = [
    'ľ' => 'l',
    'š' => 's',
    'č' => 'c',
    'ť' => 't',
    'ž' => 'z',
    'ý' => 'y',
    'á' => 'a',
    'í' => 'i',
    'ŕ' => 'r',
    'ď' => 'd',
    'ň' => 'n',
    'ä' => 'a',
    'ô' => 'o',
];
echo strtr($str, $trans);
// Krdel datlov uci kona zrat koru.

---

It should also be noted that there are libraries and native functions developed to handle such a task.

• Remove diacritics from a string
• How to handle diacritics (accents) when rewriting 'pretty URLs'
• How to Convert Special Chars to Standard Chars?

Matronna