Possible to retrieve an arbitrary unordered set of named groups in one swoop with Python's re module?

Question

This is super handy for some problems:

>>> re.search('(?P<b>.b.).*(?P<i>.i.)', 'abcdefghijk').groupdict()
{'i': 'hij', 'b': 'abc'}

But what if I don't know what order to expect ahead of time?

[update]

Eg, say I have an input variable containing some unknown order of characters and it just so happens that 'b' comes after 'i'. I want to still be able to reference the groups for '.b.' and '.i.' without having to order my regex according to their order in the input var. So, I wish I could do something like this but I don't know if it's possible:

>>> re.search('(?P<b>.b.)|(?P<i>.i.)', unknown_order_alphabet_str).groupdict()
{'i': 'hij', 'b': 'abc'}

[end update]

I've searched around and racked my brain a bunch but can't generate any good leads. Guessing this functionality wouldn't exist because probably the only way for re to do this is to scan the entire string once for each group (which of course I could do in a loop instead) but I thought I'd see what the stackoverflow brain had to say about it.

Thanks for your help,
Josh

Pillbox

Answer 1

Use a vertical bar ("or") in the RE pattern, and finditer to get all match objects of interest: each will have a groupdict with None as the value for groups not involved in that match, and you can "merge" the dicts as you prefer.

For example:

import re

def mergedgroupdict(pattern, thestring):
  there = re.compile(pattern)
  result = {}
  for mo in there.finditer(thestring):
    d = mo.groupdict()
    for k in d:
      if k not in result and d[k] is not None:
        result[k] = d[k]
  return result

this uses a merge strategy which is just to pick the first actual match for each named group in the pattern. Now for example

>>> mergedgroupdict('(?P<b>.b.)|(?P<i>.i.)', 'abcdefghijk')
{'i': 'hij', 'b': 'abc'}
>>> mergedgroupdict('(?P<b>.b.)|(?P<i>.i.)', 'abcdefghijk'[::-1])
{'i': 'jih', 'b': 'cba'}

presumably as you desire, if I interpret your question correctly.

Depreciable

There is a way which doesn't require finditer nor dictionary merging. See my answer. – Eastlake

Answer 2

>>> [m.groupdict() for m in re.finditer('(?P<b>.b.)|(?P<i>.i.)', 'abcdefghijk')]
[{'i': None, 'b': 'abc'}, {'i': 'hij', 'b': None}]

Seems to work fine, though if you have many groups checking which one isn't None might get tedious.

This finds all .b. and all .i. matches in the string. If you wanted to be sure it one found one of each you will have to check that manually, too.

Castellan

There is a way which doesn't require finditer nor dictionary merging. See my answer. – Eastlake

Answer 3

The closest I can get is this:

>>> [match.groupdict() for match in re.finditer('(?P<b>.b.)|(?P<i>.i.)', 'abcdefghijk')]
[{'i': None, 'b': 'abc'}, {'i': 'hij', 'b': None}]

How you combine the dictionaries then depends on whether you're expecting more than one match. If you only want one match each, you could do:

>>> results = {}
>>> for match in re.finditer('(?P<b>.b.)|(?P<i>.i.)', 'abcdefghijk'):
...     results.update(dict((k,v) for k, v in match.groupdict().iteritems() if v is not None))
... 
>>> results
{'i': 'hij', 'b': 'abc'}

Or for multiple matches:

>>> results = defaultdict(lambda: [])
>>> for match in re.finditer('(?P<b>.b.)|(?P<i>.i.)', 'abcdefghijkabcdefghijk'):
...     for k, v in match.groupdict().iteritems():
...         if v is not None:
...             results[k].append(v)
... 
>>> results
defaultdict(<function <lambda> at 0x7f53d0992c08>, {'i': ['hij', 'hij'], 'b': ['abc', 'abc']})

Melissiamelita

There is a way which doesn't require finditer nor dictionary merging. See my answer. – Eastlake

Answer 4

Here is a way which doesn't require finditer nor dictionary merging:

>>> pat = re.compile(r'(?:.*?(?:(?P<b>.b.)|(?P<i>.i.))){2}')

>>> pat.search('abcdefghijk').groupdict()
{'i': 'hij', 'b': 'abc'}

>>> pat.search('aicdefghbjk').groupdict()
{'i': 'aic', 'b': 'hbj'}

This is assuming each one of the characters b and i appears exactly once in your string , otherwise:

• If one of the characters might be missing, you can use {,2} instead of {2}.
• If one of the characters appears more than once, the search will retrieve the first two appearances of either of them (for example it can find b twice and not find i at all).

Eastlake

Answer 5

Here is a late comer to the game in one swipe, which is readable for beginners too:

>>> dict([(name, re.search(pattern, "abcdefghijk").group())
          for name, pattern in {"b": ".b.", "i": ".i"}.items()])  
{'b': 'abc', 'i': 'hij'}

Lodging