Skip to content

re.IGNORECASE | re.LOCALE: a negated character set matches the other case #156444

Description

@serhiy-storchaka

Under re.IGNORECASE | re.LOCALE, [bc] matches b'B', but so does [^bc].

import re

W = re.IGNORECASE | re.LOCALE
print(re.fullmatch(rb'[bc]', b'B', W))
print(re.fullmatch(rb'[^bc]', b'B', W))
<re.Match object; span=(0, 1), match=b'B'>
<re.Match object; span=(0, 1), match=b'B'>

Expected: the second line is None, which is what 3.6 and earlier print.

charset_loc_ignore() in Modules/_sre/sre_lib.h tests the whole set once per locale case and returns true if either test matches. For a set carrying NEGATE the negation has to be applied outside that disjunction: b'B' matches because 'B' itself is not in {b, c}, though its lowercase is.

A one-member set escapes this, since it compiles to NOT_LITERAL_LOC_IGNORE and compares against both cases correctly, so [^b] is fine and two members are needed to see it.

IN_LOC_IGNORE and charset_loc_ignore() were added in 3.7 by bpo-30215 (898ff03), which moved locale case folding from compile time to match time. 3.2–3.6 are correct, 3.7 through main are affected.

The set-difference fusion new in 3.16 hits the same limitation from another direction: it rewrites A(?<![B]) into a single NEGATE-bearing set, which then compiles to IN_LOC_IGNORE. See gh-155985 and GH-155993, which avoids the fusion under these flags.

Linked PRs

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13bugs and security fixes3.14bugs and security fixes3.15pre-release feature fixes, bugs and security fixes3.16new features, bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytopic-regextype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions