Regex matching between two strings?

Question

I can't seem to find a way to extract all comments like in following example.

>>> import re
>>> string = '''
... <!-- one 
... -->
... <!-- two -- -- -->
... <!-- three -->
... '''
>>> m = re.findall ( '<!--([^\(-->)]+)-->', string, re.MULTILINE)
>>> m
[' one \n', ' three ']

block with two -- -- is not matched most likely because of bad regex. Can someone please point me in right direction how to extract matches between two strings.

---

Hi I've tested what you guys suggested in comments.... here is working solution with little upgrade.

>>> m = re.findall ( '<!--(.*?)-->', string, re.MULTILINE)
>>> m
[' two -- -- ', ' three ']
>>> m = re.findall ( '<!--(.*\n?)-->', string, re.MULTILINE)
>>> m
[' one \n', ' two -- -- ', ' three ']

thanks!

Terminable

anything between the [] is a single character so (-->) will not look for that grouping is part of the problem... – Erg

re.findall('<!--(.*?)-->', string, re.DOTALL) should do. You don't need ^\(-->) here, because the question mark makes it non-greedy. – Studnia

You look like you're looking for just the words? If so, what's wrong with m = re.findall('[\w]+', string, re.MULTILINE)? Also, string is a really bad name for a, um, string. – Elly

Answer 1

this should do the trick

 m = re.findall ( '<!--(.*?)-->', string, re.DOTALL)

Extravert

In case anyone is wondering, the "re.DOTALL" flag makes the dot (.) to match any character, including newline. The (.*?) captures the text inside the parenthesis, and .*? means that "non-greedy" version of .* (i.e. capture the shortest possible matches). – Brazil

If <!-- and --> should be part of the resulting list items, the capturing parentheses should be removed - re.findall ( '<!--.*?-->', string, re.DOTALL) – Martamartaban

Answer 2

In general, it is impossible to do arbitrary matching between two delimiters with a regular grammar.

Specifcally, if you allow nesting,

<!-- how do you deal <!-- with nested --> comments? -->

you'll run in to issues. So, while you may be able to solve this specific problem with a regular expression, any regular expression that you write will be able to be broken by some other strange nesting of comments.

To parse arbitrary comments, you'll need to move on to a method of parsing context free grammars. A simple method to do so is to use a pushdown automaton.

Shadowy

I don't think nested comments are all that common. Kinda defeats the point of commenting if anything inside it is processed? – Lithograph

And it looks like they're not possible in HTML. #443286 I'm going to leave this here, because I think it's important to recognize, but I don't expect any upvotes. – Shadowy

Finite state machines cannot parse context free grammars - you could use Pushdown automatons. – Twobit

@JamesThiele Ahhhhh, of course. I've edited the answer to reflect this – Shadowy