How to find overlapping matches with a regexp? [duplicate]

Question

>>> match = re.findall(r'\w\w', 'hello')
>>> print match
['he', 'll']

Since \w\w means two characters, 'he' and 'll' are expected. But why do 'el' and 'lo' not match the regex?

>>> match1 = re.findall(r'el', 'hello')
>>> print match1
['el']
>>>

Pentangular

Lookahead – Coin

Answer 1

findall doesn't yield overlapping matches by default. This expression does however:

>>> re.findall(r'(?=(\w\w))', 'hello')
['he', 'el', 'll', 'lo']

Here (?=...) is a lookahead assertion:

(?=...) matches if ... matches next, but doesn’t consume any of the string. This is called a lookahead assertion. For example, Isaac (?=Asimov) will match 'Isaac ' only if it’s followed by 'Asimov'.

Stromboli

But I don't understand why it advances to the next letter if it's inside the positive lookahead assertion. Could you explain, please? – Vial

@Vial I guess it's due to group capturing (braces around \w\w). The actual match is still an empty string, whereas group 1 is filled with \w\w (as you can test at regex101.com). So I believe it captures it in a group, but doesn't advance past it because the match is zero-length. And python's re.findall will print captured groups docs.python.org/3/library/re.html#re.findall – Smalto

Answer 2

You can use the new Python regex module, which supports overlapping matches.

>>> import regex as re
>>> match = re.findall(r'\w\w', 'hello', overlapped=True)
>>> print match
['he', 'el', 'll', 'lo']

Veliz

Answer 3

Except for zero-length assertion, character in the input will always be consumed in the matching. If you are ever in the case where you want to capture certain character in the input string more the once, you will need zero-length assertion in the regex.

There are several zero-length assertion (e.g. ^ (start of input/line), $ (end of input/line), \b (word boundary)), but look-arounds ((?<=) positive look-behind and (?=) positive look-ahead) are the only way that you can capture overlapping text from the input. Negative look-arounds ((?<!) negative look-behind, (?!) negative look-ahead) are not very useful here: if they assert true, then the capture inside failed; if they assert false, then the match fails. These assertions are zero-length (as mentioned before), which means that they will assert without consuming the characters in the input string. They will actually match empty string if the assertion passes.

Applying the knowledge above, a regex that works for your case would be:

(?=(\w\w))

Pendant

Answer 4

Am no regex expert but I would like to answer my similar question.

If you want to use a capture group with the lookahead:

example regex: (\d)(?=.\1)

string: 5252

this will match the first 5 as well as the first 2

The (\d) is to make a capture group, (?=\d\1) is to match any digit followed by the capture group 1 without consuming the string, thus allow overlapping

Goodill