Skip to content

Match glob characters literally in completion and quote them for the shell - #1812

Merged
tleonhardt merged 9 commits into
mainfrom
escape_get_exes_in_path
Nov 17, 2026
Merged

tleonhardt merged 9 commits into
mainfrom
escape_get_exes_in_path

Conversation

@kmvanbrunt

@kmvanbrunt kmvanbrunt commented Nov 14, 2026 •

Copy link
Copy Markdown
Member

This continues the work of #1811 by adding the fix to get_exes_in_path().

It started as that one fix and grew during review. It now makes file path and shell command completion treat every glob character literally, quotes completions which a shell would otherwise expand, and fixes how PATH is searched.

Changes

Literal matching of glob characters

  • get_exes_in_path() passes its search path through glob.escape(), so executable names and PATH directories containing square brackets are matched literally. This is the same fix fix: complete literal bracketed file paths #1811 made to path_complete().
  • Removed the guard which made path_complete() and get_exes_in_path() return nothing when the search text contained * or ?. Now that the search text is escaped, the guard no longer protected anything. It only made those two characters behave differently from every other special character. A file named my*file.txt can now be completed.
  • A leading ~ is still special in path_complete(). The docstring and docs say so.

Quoting of completions which are passed to a shell

  • Tab completion now adds quotes when a match has a character which is special to a shell, if what is being completed is passed to one. Completing !ls my* gives !ls "my*file.txt".
  • Without this, removing the guard would have been a hazard. !rm my* + Tab used to complete nothing. It would have completed to an unquoted my*file.txt, which the shell expands to every matching file.
  • This only happens in the two places where cmd2 knows a shell is the consumer: the arguments of the shell command (including !) and everything after a pipe. Everywhere else, only a match with a space is quoted, exactly as before. A command which takes its raw argument and uses path_complete() therefore does not start receiving quotes for a name like notes[2026].txt.
  • Those two places mark their results through Cmd._for_shell(), which sets a private Completions._quote_for_shell field that Cmd._perform_completion() checks. The field is excluded from comparisons.
  • The characters are chosen for the platform's shell:
    • POSIX: a space, a quote character, *, ?, [, {, }, (, ), <, >, |, &, ;, $, a backtick, a backslash, and a leading # or =.
    • Windows: the characters cmd.exe documents as requiring quotes, other than ~.
  • Double quotes are used unless a match has a $, a backtick, a backslash, or a double quote in it. Those get single quotes, since a POSIX shell still interprets them inside double quotes. Windows always gets double quotes.
  • A macro which runs a shell command is not covered. Its arguments are only quoted for a space.
  • The quotes go around the whole argument, so when several matches are offered and only one needs quotes, they all get them. This is how quoting for spaces already worked. An executable named [ is common, so completing a shell command which could match it quotes every match. A bare [ has to count because zsh rejects it as a bad pattern.

PATH handling

  • get_exes_in_path() and find_editor() no longer skip PATH entries which are symbolic links to directories. Both now read PATH through one private helper, _get_path_dirs().
  • The entries are used as written and are not resolved. os.path.realpath() can raise for an entry which can't be reached, and find_editor() runs when cmd2 is imported. Executable names are de-duplicated, so the only cost is searching a directory twice when it is listed under two names (e.g. /bin -> /usr/bin).
  • The Windows editor list now names edit.exe instead of edit. The other entries already had their extensions, and a file literally named edit could never be found.

Documentation and tests

  • New Special Characters in Paths section in the completion docs, and CHANGELOG entries for each user-visible change.
  • New tests cover:
    • get_exes_in_path(), shell_cmd_complete(), and path_complete() with names and directories containing [, *, and ?
    • which completions get an opening quote and which quote character is used, including that edit, run_script, file redirection, raw-argument commands, choices, and alias names are left alone
    • symbolic link PATH entries for get_exes_in_path() and find_editor()
    • a removed working directory and PATH entries which can't be resolved
    • find_editor(): the order of preference, an unset PATH, and that the current directory is not searched
  • The tests which need * or ? in a file name, a symbolic link, or a removed working directory are skipped on Windows.
  • The helper which creates fake executables moved to tests/conftest.py as make_exes().

Behavior changes worth a look

  • Completions for shell arguments and piped commands are now quoted when they have a shell-special character. Completions everywhere else are unchanged.
  • The default editor may change on Windows. A machine with Microsoft Edit installed should now default to edit, which is what 91e5bc2 intended. This has not been confirmed on a Windows machine which has Edit installed.

Follow-up

Testing

  • make check, make test (3064 passed, 3 skipped on macOS), and make docs-test pass locally.
  • CI runs on Linux, macOS, and Windows for Python 3.11 through 3.15t.
  • Checked in a real terminal (tmux) that completing !ls my* and help | grep -c . fi inserts the quoted name and that the shell then sees only that file, and that edit fi and help > fi complete file(1).txt without quotes.

@codecov

codecov Bot commented Nov 14, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.75%. Comparing base (05128bf) to head (80c8ed7).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1812   +/-   ##
=======================================
  Coverage   99.75%   99.75%           
=======================================
  Files          23       23           
  Lines        6632     6647   +15     
=======================================
+ Hits         6616     6631   +15     
  Misses         16       16           
Flag Coverage Δ
unittests 99.75% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment thread cmd2/utils.py Resolved
Comment thread CHANGELOG.md Resolved
Comment thread cmd2/utils.py Resolved
Comment thread tests/test_utils.py Resolved
Comment thread cmd2/utils.py Resolved
@tleonhardt tleonhardt added the bug label Nov 14, 2026
…TH entries

Removed the guard which made path_complete() and get_exes_in_path() return
nothing when the search text contained '*' or '?'. Since the search text now
goes through glob.escape(), those characters are matched literally like any
other, so names containing them can be completed.

get_exes_in_path() and find_editor() no longer skip PATH entries which are
symbolic links to directories. get_exes_in_path() resolves each entry so a
directory reachable through more than one entry is only searched once.

Documented that completion results are literal names which are not escaped
for a shell, and added tests for shell_cmd_complete() with bracketed names
and for PATH directories containing glob characters.
Tab completion now adds quotes to a match containing '*', '?', or '[', as it
already did for a match containing a space. With the wildcard guard gone,
completing '!rm my*' to an unquoted 'my*file.txt' would have let the shell
expand it to every matching file.

get_exes_in_path() no longer raises when PATH has an empty or relative entry
and the current working directory has been removed. Only absolute entries are
resolved with realpath().

find_editor() now uses shutil.which() instead of walking PATH by hand. As a
side effect it can find 'edit' on Windows, which the old search never could
because it did not try any file extensions.

Documented that a leading '~' is the exception to literal matching, promoted
the 'Special Characters in Paths' docs section to H2, and moved the test
helper which creates fake executables into conftest.py.
@tleonhardt tleonhardt changed the title Fixed get_exes_in_path() when files or dirs contain brackets Match glob characters literally in completion and quote them for the shell Nov 16, 2026
The glob-character quoting added in the previous commit applied to every
completer. That broke commands which preserve quotes when one of their
choices had a glob character, left a stray quote when completing an alias
with a glob character in its name, and quoted every shell command when
an executable named '[' was among the matches.

path_complete() and shell_cmd_complete() now mark their results, and only
marked results are quoted for a shell. The check covers the characters
which are special to the platform's shell rather than only glob characters,
a '[' only counts when a ']' follows it, and single quotes are used on
POSIX when a name has a character which a shell still expands within
double quotes. All other completions are quoted only for a space, as
before.

find_editor() goes back to searching the PATH directories itself.
shutil.which() also searched the current directory on Windows and fell
back to a default path when PATH was not set. The Windows editor list now
names edit.exe so it can still be found. find_editor() and
get_exes_in_path() share one helper for reading PATH.
Quoting every path_complete() result for shell-special characters reached
commands which never involve a shell. A command taking its raw argument,
with path_complete as its completer, received the quotes literally for a
name like notes[2026].txt and could no longer open the file.

The quoting is now decided where cmd2 knows a shell is the consumer: the
arguments of the shell command and everything after a pipe. Everywhere else
only a match with a space is quoted, as before.

Within that scope a '[' is special even with no ']' after it, since zsh
rejects it as a bad pattern, a leading '=' is special on POSIX, and the
Windows list follows the characters cmd.exe documents as requiring quotes.
The private Completions._quote_for_shell field no longer takes part in
comparisons.
os.path.realpath() can raise for a PATH entry which can't be reached, such
as one on an unavailable network share on Windows. Since find_editor() runs
when cmd2 is imported, that could keep cmd2 from being imported. The PATH
entries are now used as written. Executable names were already
de-duplicated, so the only cost is searching a directory twice when it is
listed under two names.

The places which mark completions as being passed to a shell now share one
helper, Cmd._for_shell().

Corrected the docs for how the quote character is chosen when several
matches are offered, and documented that a macro which runs a shell
command is not quoted for the shell.
@tleonhardt
tleonhardt merged commit ff372de into main Nov 17, 2026
32 checks passed
@tleonhardt
tleonhardt deleted the escape_get_exes_in_path branch November 17, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants