How to get correct list position in multi-byte string using preg_match [duplicate]

Question

I am currently matching HTML using this code:

preg_match('/<\/?([a-z]+)[^>]*>|&#?[a-zA-Z0-9]+;/u', $html, $match, PREG_OFFSET_CAPTURE, $position)

It matches everything perfect, however if I have a multibyte character, it counts it as 2 characters when giving back the position.

For example the returned $match array would give something like:

array
  0 => 
    array
      0 => string '<br />' (length=6)
      1 => int 132
  1 => 
    array
      0 => string 'br' (length=2)
      1 => int 133

The real number for the <br /> match is 128, but there are 4 multibyte characters, so it's giving 132. I really thought adding the /u modifier would make it realize what's going on, but no luck there.

Baroness

If you're curious what I'm using this for: #1194000 – Baroness

Does [this][1] helps? [1]: https://mcmap.net/q/270524/-get-multibyte-character-count-before-match-with-preg_match-preg_offset_capture-parameter-is-unhelpfully-counting-bytes – Parenteral

Yeah it's helping. Surprised these threads didn't come up in the suggestions when I was asking – Baroness

I actually can correct the position manually by counting the number of mb characters before the point I'm at in my function. I just can't figure a good regex for all "standard" characters on an English keyboard. – Baroness

Maybe you can use https://mcmap.net/q/393647/-utf-8-characters-in-preg_match_all-php-duplicate "change the encoding to UTF-32 before and then divide the position by 4". – Foxed

@Foxed I was able to get it to work based on that link, without dividing somehow. Posting the answer – Baroness

Answer 1

I looked at this suggestion from @Qtax:

UTF-8 characters in preg_match_all (PHP)

And for some more reference, this bug surfaced while using this: Truncate text containing HTML, ignoring tags

The gist of the change is this:

$orig_utf = 'UTF-8';
$new_utf  = 'UTF-32';

mb_regex_encoding( $new_utf );

$html     = mb_convert_encoding( $html, $new_utf, $orig_utf );
$end_char = mb_convert_encoding( $end_char, $new_utf, $orig_utf );


mb_ereg_search_init( $html );

$pattern = '</?([a-z]+)[^>]*>|&#?[a-zA-Z0-9]+;';
$pattern = mb_convert_encoding( $pattern, $new_utf, $orig_utf );

while ( $printed < $limit && $tag_match = mb_ereg_search_pos( $pattern, $html ) ) {

  $tag_position = $tag_match[0]/4;
  $tag_length   = $tag_match[1];
  $tag          = mb_substr( $html, $tag_position, $tag_length/4, $new_utf );
  $tag_name     = preg_replace( '/[\s<>\/]+/', '', $tag );

  // Print text leading up to the tag.
  $str = mb_substr($html, $position, $tag_position - $position, $new_utf );

  .......

} 

Also in reference to the truncate HTML page, there are other neccessary changes:

$first_char = mb_substr( $tag, 0, 1, $new_utf );

if ( $first_char == mb_convert_encoding( '&', $new_utf ) ) {
  ...
}

My text editor is UTF-8 so if I was comparing the 32 to my file's ampersand, it wouldn't work.

Baroness

Answer 2

If you need quick fix and don't care about speed:

$mb_pos = mb_strlen( substr($string, 0, $pos) );

Gae

Answer 3

Have you looked into http://www.php.net/manual/en/function.mb-ereg.php ?

Pindaric

It doesn't have the same options I need: ( string $pattern , string $subject [, array &$matches [, int $flags = 0 [, int $offset = 0 ]]] ) vs ( string $pattern , string $string [, array $regs ] ) – Baroness