fix(sessions): decode git worktree paths as UTF-8 - #1225
Open
MohammedAlkindi wants to merge 1 commit into
Open
Conversation
git worktree list --porcelain emits paths as raw UTF-8, but the subprocess call decoded them with the locale default. On a stock Windows install that is cp1252, which misreads a non-ASCII worktree path in one of two ways. If the path's UTF-8 contains 0x81, 0x8D, 0x8F, 0x90 or 0x9D, the five bytes cp1252 leaves undefined, the decode raises UnicodeDecodeError inside subprocess's reader thread. That never reaches the caller, which sees returncode 0 with stdout None, so the existing guard returns an empty list and every worktree in the repo is lost, ASCII ones included. Otherwise the bytes decode to mojibake and the returned path does not resolve on disk. Measured on Windows with a Cyrillic worktree: 0 of 2 entries returned before, 2 of 2 after.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
git worktree list --porcelainemits paths as raw, unquoted UTF-8, but_get_worktree_pathsdecodes them with the locale default. On a stock Windows install that is cp1252.Before (Windows 11, cp1252, real repo with a Cyrillic worktree):
plus a
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81traceback on stderr from a subprocess reader thread.After, same repo: 2 of 2 paths returned, both resolve on disk, no traceback.
Two distinct failure modes, both fixed by the same line
cp1252 leaves exactly five bytes undefined —
0x81 0x8D 0x8F 0x90 0x9D:returncode == 0withstdout is None; the existingif result.returncode != 0 or not result.stdout: return []guard turns that into an empty list. One such worktree hides every worktree in the repo, ASCII ones included. This covers all hiragana (U+3040–307Fencode asE3 81 xx), Cyrillic likeU+0441(D1 81), andU+200DZWJ (E2 80 8D), which appears in every emoji-family sequence.é→é,日→æ—¥, no exception.So a
caféworktree degrades quietly while anあworktree takes the whole repo's worktree list with it. Both are wrong; both are fixed by pinning the decode.Reach:
_get_worktree_pathshas six call sites backinglist_sessions,get_session_info,fork_session,rename_sessionandtag_session. This affects every legacy Windows codepage — cp1252, cp932, cp936, cp1251.Why UTF-8 specifically
sys.getfilesystemencoding()isutf-8on Windows (PEP 529) and on POSIX, soencoding="utf-8"is the codecos.fsdecodealready uses for these paths. It also matches the surrounding code, which normalizes the same strings withunicodedata.normalize("NFC", ...)and uses explicitencoding="utf-8"for file I/O throughoutsessions.py.One trade-off I want to surface rather than leave for review
There is a narrow configuration where this is a regression: POSIX with a legacy 8-bit locale (e.g.
LC_ALL=en_US.ISO-8859-1) and a non-ASCII worktree path. Today that decodes to mojibake which round-trips throughos.fsencode, so the paths resolve and the lookup works; with a pinned UTF-8 decode it raises instead. It cannot surface to a caller — all six call sites already wrap this inexcept Exception, andUnicodeDecodeErroris aValueError— so the result degrades to[], which is the same outcome Windows has today. I verified nothing escapes throughlist_sessionsorget_session_info.errors="surrogateescape"would close that window, and I deliberately did not use it: it returns strings containing lone surrogates, which resolve on disk but raise on any subsequent encode (JSON, logging to a UTF-8 stream). Given these paths flow onward into session lookup, a loud failure that the callers already swallow seemed safer than a value that breaks somewhere else. Happy to switch if you'd rather haveerrors="surrogateescape", orencoding=sys.getfilesystemencoding(), errors="surrogateescape"which measured correct in every locale I tried.Testing
_get_worktree_pathscurrently has no test coverage — everylist_sessionstest passesinclude_worktrees=False. The added test pins the decode viamonkeypatch, so it is deterministic on every platform. That matters here: hosted CI runs a UTF-8 locale, including thewindows-latestleg, so a purely behavioural test would pass without the fix and guard nothing.Verified with the test in place and the source change reverted, it fails for the right reason:
Full suite, Windows,
trioinstalled: 1477 passed, 14 skipped with the change; upstreammainat22795feis 1476 passed, 14 skipped. Same run three times each, no variance.ruff check,ruff format --check(68 files) andmypy src/ scripts/(33 files) are all clean before and after.Note on #1188
#1188 rewrites this same call from
subprocess.run(toexecutable.run(, so there will be a one-line textual conflict whichever lands first. It does not change the decoding, andexecutable.runforwards**kwargs, soencoding="utf-8"composes with it either way. Glad to rebase on top of it if you'd prefer that order.AI assistance: I used Claude Code for this investigation. Verified myself on my own Windows 11 machine rather than taken on trust: the before/after worktree counts against a real git repo with a Cyrillic worktree; that the exception is swallowed into
returncode 0/stdout None; the per-character split between the raising and mojibake failure modes; that the new test fails with the fix reverted and passes with it; the full-suite, ruff and mypy results above; and thatgit worktree list --porcelainemits raw unescaped UTF-8 under bothcore.quotepathsettings, which the whole report depends on.