Support POSIX character classes in bracket expressions - #130
Open
dngr2 wants to merge 2 commits into
Open
Conversation
Git's wildmatch supports POSIX character classes inside bracket expressions
-- '[[:digit:]]', '[[:alpha:]]', '[[:space:]]', etc. pathspec did not: the
inner '[:...:]' confused the closing-bracket scan (the ']' that ends the
class was taken as the end of the whole expression), and even when delimited,
Python's 're' has no POSIX class syntax, so '[[:digit:]]' compiled to a set of
the literal characters '[', ':', 'd', 'i', 'g', 't' (with a 'Possible nested
set' warning) and matched nothing sensible.
The result: a .gitignore rule like '[[:upper:]]*.tmp' matched files in Git but
not in pathspec, so tools built on pathspec (e.g. via GitIgnoreSpec) silently
disagreed with Git about which files are ignored.
Skip '[:...:]' as a unit when finding the closing bracket, and expand each
POSIX class to the equivalent ASCII range (matching Git's wildmatch in the C
locale). Negated ('[![:digit:]]') and mixed ('[[:alpha:]0-9]') forms work too.
Adds tests covering the translation and matching.
Owner
|
Thanks for the pull request. I'll review this along with the similar #128. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Git's
wildmatchsupports POSIX character classes inside bracket expressions —[[:digit:]],[[:alpha:]],[[:space:]],[[:upper:]], etc. pathspec does not, for two reasons:[[:digit:]]for the]that ends the expression, it stops at the]that closes the inner[:digit:], so the expression is mis-delimited.rehas no POSIX class syntax. Even delimited,[[:digit:]]is handed toreunchanged, which reads it as a set of the literal characters[ : d i g t(raising aFutureWarning: Possible nested set) — matching nothing sensible.Result: every POSIX class is broken. Against
git check-ignore(the reference), all 12 classes diverge:A
.gitignorerule like[[:upper:]]*.tmptherefore matches files in Git but not in pathspec, so tools built on pathspec (Black, pre-commit, pip, GitPython viaGitIgnoreSpec) silently disagree with Git about which files are ignored. This affects both thegitwildmatchandgitignorefactories (they share the translation).The fix
In
_translate_segment_glob:[:...:]as a unit when finding the closing bracket, so the inner]isn't mistaken for the end.wildmatchin the C locale ([:digit:]→0-9,[:alpha:]→A-Za-z,[:punct:]→ the punctuation ranges, etc.).Negated (
[![:digit:]],[^[:digit:]]) and mixed ([[:alpha:]0-9],[[:alnum:]_]) forms are handled. Non-class brackets ([a-z],[!a-z],[]a]) are unchanged.Verification
test_15_posix_character_classasserts both the translated regex and match behavior; it fails on the current code and passes with the fix.git check-ignoreover all 12 POSIX classes (plus negated/mixed forms) and a 1,500-case fuzz of random bracket/wildcard patterns — 0 POSIX-class divergences remain.