EDITED (to show new clues): there are a possible solution using str_word_count() with PHP v5.1!
function my_word_count($str, $myLangChars="àáãâçêéíîóõôúÀÁÃÂÇÊÉÍÎÓÕÔÚ") {
return str_word_count($str, 0, $myLangChars);
}
but not is 100% because I try to add to $myLangChars \xC2\xAD (the SHy - SOFT HYPHEN character), that must be a word component in any language, and it not works (see).
Another, not so fast, but complete and flexible solution (extracted from here), based on PCRE library, but with an option to mimic the str_word_count() behaviour on non-valid-UTF8:
/**
* Like str_word_count() but showing how preg can do the same.
* This function is most flexible but not faster than str_word_count.
* @param $wRgx the "word regular expression" as defined by user.
* @param $triggError changes behaviour causing error event.
* @param $OnBadUtfTryAgain when true mimic the str_word_count behaviour.
* @return 0 or positive integer as word-count, negative as PCRE error.
*/
function preg_word_count($s,$wRgx='/[-\'\p{L}\xC2\xAD]+/u', $triggError=true,
$OnBadUtfTryAgain=true) {
if ( preg_match_all($wRgx,$s,$m) !== false )
return count($m[0]);
else {
$lastError = preg_last_error();
$chkUtf8 = ($lastError==PREG_BAD_UTF8_ERROR);
if ($OnBadUtfTryAgain && $chkUtf8)
return preg_word_count(
iconv('CP1252','UTF-8',$s), $wRgx, $triggError, false
);
elseif ($triggError) trigger_error(
$chkUtf8? 'non-UTF8 input!': "error PCRE_code-$lastError",
E_USER_NOTICE
);
return -$lastError;
}
}
(TEMPLATE ANSWER) help for bounty!
(this is not an answer, is a help for bounty, because I can not edit neither to duplicate the question)
We want to count "real-world words" in a UTF-8 latim text.
FOR BOUNTY, WE NEED:
- a function that comply the
asserts below and is faster than str_word_count;
- or
str_word_count working with SHy character (how to?);
- or
preg_word_count working faster (using preg_replace? word-separator regular expression?).
ASSERTS
Supose that a "multibyte safe" function my_word_count() exists, then the following asserts must be true:
assert_options(ASSERT_ACTIVE, 1);
$text = "1,2,3,4=0 (1 2 3 4)=0 (... ,.)=0 (2.5±0.1; 0.5±0.2)=0";
assert( my_word_count($text)==0 ); // no word there
$text = "(one two,three;four)=4 (five-six se\xC2\xADven)=2";
assert( my_word_count($text)==6 ); // hyphen merges two words
$text = "(um±dois três)=3 (àáãâçêéíîóõôúÀÁÃÂÇÊÉÍÎÓÕÔÚ)=1";
assert( my_word_count($text)==4 ); // a UTF8 case
$text = "(ÍSÔ9000-X, ISÔ 9000-X, ÍSÔ-9000-X)=6"; //Codes are words?
assert( my_word_count($text)==6 ); // suppose no: X is another word
