Skip to content

gh-123018: Keep the libedit history file header when truncating - #157165

Merged
gpshead merged 1 commit into
python:mainfrom
gpshead:fix-editline-tests
Sep 8, 2026
Merged

gh-123018: Keep the libedit history file header when truncating#157165
gpshead merged 1 commit into
python:mainfrom
gpshead:fix-editline-tests

Conversation

@gpshead

@gpshead gpshead commented Sep 8, 2026

Copy link
Copy Markdown
Member

libedit's history_truncate_file() keeps the last N lines of the file, which drops the "_HiStOrY_V2_" header line that its own write_history() emits and that its read_history() requires. So on a libedit build, readline.write_history_file() or readline.append_history_file() after readline.set_history_length() produced a file that readline.read_history_file() rejected with EINVAL.

Under the libedit emulation, truncate the file ourselves and keep the header, resolving the default "~/.history" the same way libedit does. Apple's libedit fork already preserves the header, so the workaround is not compiled on macOS.

Comment thread Modules/readline.c
static int
_py_libedit_history_truncate_file(const char *filename, int nlines)
{
static const char cookie[] = "_HiStOrY_V2_\n";

@gpshead gpshead Sep 8, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

over in #121160 the earlier choice was not to do a workaround because it seemed fragile. yes this PR intentionally decides the other way. libedit is not likely to change or a fix to it become widely distributed any time soon. apple patched theirs rather than push on upstream. if netbsd libedit does adopt a new history format version, this marker won't match and this function basically becomes a no-op that does nothing.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like someone did at least file a NetBSD bug this year https://gnats.netbsd.org/60322

libedit's history_truncate_file() keeps the last N lines of the file,
which drops the "_HiStOrY_V2_" header line that its own write_history()
emits and that its read_history() requires.  So on a libedit build,
readline.write_history_file() or readline.append_history_file() after
readline.set_history_length() produced a file that
readline.read_history_file() rejected with EINVAL.

Under the libedit emulation, truncate the file ourselves and keep the
header, resolving the default "~/.history" the same way libedit does.
Apple's libedit fork already preserves the header, so the workaround is
not compiled on macOS.
@gpshead
gpshead enabled auto-merge (squash) September 8, 2026 05:58
@gpshead
gpshead merged commit 2638785 into python:main Sep 8, 2026
54 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

@bedevere-app

bedevere-app Bot commented Sep 8, 2026

Copy link
Copy Markdown

GH-157172 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Sep 8, 2026
@miss-islington-app

Copy link
Copy Markdown

Sorry, @gpshead, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 26387858990674f8cd20566f89c700fa1de5d829 3.13

@bedevere-app

bedevere-app Bot commented Sep 8, 2026

Copy link
Copy Markdown

GH-157173 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Sep 8, 2026
@whitequark

Copy link
Copy Markdown

Yay! Thank you.

gpshead added a commit that referenced this pull request Sep 9, 2026
GH-157165) (#157173)

gh-123018: Keep the libedit history file header when truncating (GH-157165)

libedit's history_truncate_file() keeps the last N lines of the file,
which drops the "_HiStOrY_V2_" header line that its own write_history()
emits and that its read_history() requires.  So on a libedit build,
readline.write_history_file() or readline.append_history_file() after
readline.set_history_length() produced a file that
readline.read_history_file() rejected with EINVAL.

Under the libedit emulation, truncate the file ourselves and keep the
header, resolving the default "~/.history" the same way libedit does.
Apple's libedit fork already preserves the header, so the workaround is
not compiled on macOS.
(cherry picked from commit 2638785)

Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs backport to 3.13 bugs and security fixes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants