38

Ruby's regular expressions have a feature called atomic grouping (?>regexp), described here, is there any equivalent in Python's re module?

1
  • (?>...) new in 3.11
    – dugres
    Dec 7, 2022 at 8:18

5 Answers 5

53

Python does not directly support this feature, but you can emulate it by using a zero-width lookahead assert ((?=RE)), which matches from the current point with the same semantics you want, putting a named group ((?P<name>RE)) inside the lookahead, and then using a named backreference ((?P=name)) to match exactly whatever the zero-width assertion matched. Combined together, this gives you the same semantics, at the cost of creating an additional matching group, and a lot of syntax.

For example, the link you provided gives the Ruby example of

/"(?>.*)"/.match('"Quote"') #=> nil

We can emulate that in Python as such:

re.search(r'"(?=(?P<tmp>.*))(?P=tmp)"', '"Quote"') # => None

We can show that I'm doing something useful and not just spewing line noise, because if we change it so that the inner group doesn't eat the final ", it still matches:

re.search(r'"(?=(?P<tmp>[A-Za-z]*))(?P=tmp)"', '"Quote"').groupdict()
# => {'tmp': 'Quote'}

You can also use anonymous groups and numeric backreferences, but this gets awfully full of line-noise:

re.search(r'"(?=(.*))\1"', '"Quote"') # => None

(Full disclosure: I learned this trick from perl's perlre documentation, which mentions it under the documentation for (?>...).)

In addition to having the right semantics, this also has the appropriate performance properties. If we port an example out of perlre:

[nelhage@anarchique:~/tmp]$ cat re.py
import re
import timeit


re_1 = re.compile(r'''\(
                           (
                             [^()]+           # x+
                           |
                             \( [^()]* \)
                           )+
                       \)
                   ''', re.X)
re_2 = re.compile(r'''\(
                           (
                             (?=(?P<tmp>[^()]+ ))(?P=tmp) # Emulate (?> x+)
                           |
                             \( [^()]* \)
                           )+
                       \)''', re.X)

print timeit.timeit("re_1.search('((()' + 'a' * 25)",
                    setup  = "from __main__ import re_1",
                    number = 10)

print timeit.timeit("re_2.search('((()' + 'a' * 25)",
                    setup  = "from __main__ import re_2",
                    number = 10)

We see a dramatic improvement:

[nelhage@anarchique:~/tmp]$ python re.py
96.0800571442
7.41481781006e-05

Which only gets more dramatic as we extend the length of the search string.

2
  • 1
    This is still true with Python's "re" library, but it is worth noting that the "regex" library does provide both atomic grouping and possessive quantification. There are cases where "pseudo-atomic grouping" provides a minimal speedup, but pales in comparison with using regex (both "plain" and with a possessive quantifier)
    – grjash
    Apr 2, 2021 at 2:23
  • I've also found a case where the use of "pseudo-atomic grouping" seems appropriate, but it actually runs significantly slower: ^\w+: on a very long string of word characters that ends with a colon
    – grjash
    Apr 2, 2021 at 2:42
16

According to this table, the answer is no. A RFE was created to add it to Python 3, but was declined in favor of the new regex module, which supports it:

>>> import regex
>>> regex.match('"(?>.*)"', '"Quote"')
>>> regex.match('"(.*)"', '"Quote"')
<_regex.Match object at 0x00C6F058>

Note: regex is also available for Python 2.

3
  • 1
    the regex module that you've linked is the same regex module from the python bug that you've linked i.e., there is no support in Python 3.3 stdlib
    – jfs
    Nov 27, 2012 at 4:36
  • @J.F.Sebastian Oh, ok! I thought Python 3 still used re module, since I only ever used Python 2 I wasn't sure, and assumed it referred to the same module. (Edit: NVM that, just saw what was wrong: I was looking at this RFE, which was closed as a duplicate of the one I mentioned in the answer. I'll update it)
    – mgibsonbr
    Nov 27, 2012 at 4:40
  • now, that you've changed the link, my previous comment doesn't make sense. For the record the links were pypi.python.org/pypi/regex and bugs.python.org/issue2636
    – jfs
    Nov 27, 2012 at 4:51
9

As of March 21, 2022, about 10 years after this thread was created, Python has finally added atomic grouping (and possessive matching) in the standard library re module in Python 3.11.0a7: commit link. So now, all the answers to this thread might be rendered obsolete; before, only the third-party regex module has atomic grouping and possessive matching. Now, it is already built-in to Python's re module.

0

It would seem not.

http://www.regular-expressions.info/atomic.html

Atomic grouping is supported by most modern regular expression flavors, including the JGsoft flavor, Java, PCRE, .NET, Perl and Ruby.

You can emulate the non-capturing-ness of them by using non-capturing groups, (?:RE), but if I'm reading it right, that still won't give you the optimization benefits.

-2

from http://docs.python.org/2/library/re.html#regular-expression-syntax

(?P<name>...)

Similar to regular parentheses, but the substring matched by the group is accessible within the rest of the regular expression via the symbolic group name name. Group names must be valid Python identifiers, and each group name must be defined only once within a regular expression. A symbolic group is also a numbered group, just as if the group were not named. So the group named id in the example below can also be referenced as the numbered group 1.

For example, if the pattern is (?P[a-zA-Z_]\w*), the group can be referenced by its name in arguments to methods of match objects, such as m.group('id') or m.end('id'), and also by name in the regular expression itself (using (?P=id)) and replacement text given to .sub() (using \g).

(?P=name)

Matches whatever text was matched by the earlier group named name.
1
  • 4
    Named matches are not the same thing as atomic grouping. Nov 27, 2012 at 4:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.