Skip to content

Unix: lock key buffers in memory and exclude them from core dumps - #1968

Open
flatstik wants to merge 2 commits into
veracrypt:masterfrom
flatstik:fix-1958-key-memory
Open

flatstik wants to merge 2 commits into
veracrypt:masterfrom
flatstik:fix-1958-key-memory

Conversation

@flatstik

@flatstik flatstik commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1958.

SecureBuffer allocations get whole pages of their own, locked with mlock and excluded from core dumps (MADV_DONTDUMP, MADV_NOCORE on FreeBSD). Separate pages let a freed buffer be unlocked without affecting another one. Memory locks are not inherited by fork, so the FUSE service locks the live buffers again once FUSE has put it in the background. If RLIMIT_MEMLOCK does not allow locking, the buffers are used unlocked as before.

Key material can also be copied outside SecureBuffer, so the core service and the FUSE service additionally set RLIMIT_CORE to 0 and, on Linux, clear the dumpable flag, which a core_pattern pipe does not ignore.

Volume::WriteSectors keeps a plain Buffer, still erased when freed: its data is encrypted in place right after the copy, and locking and unlocking its pages on every write reduced write throughput by 19 to 60 % in a benchmark (x86_64, AES-NI, 1 MiB to 4 KiB writes).

Tested:

  • OpenWrt 25.12.6 mipsel (FUSE3, root), 1.26.30 against 1.26.30 with this change: VmLck 0 kB, core limit unlimited and a 38 MB core file after SIGSEGV, against VmLck 24 kB, core limit 0 and no core file.
  • Tests/test_linux_fuse_service_memory.py (new, run in the Linux workflow): passes on ubuntu-24.04 and ubuntu-26.04, https://github.com/flatstik/VeraCrypt/actions/runs/40497963940.
  • Not built or tested on macOS and FreeBSD.

The new test adds lines next to test_mount_helper.py in .github/workflows/build-linux.yml, as do the other PRs split from #1963, so whichever is merged later conflicts there; I will rebase it.

Split from #1963, with #1967 (#1959), #1966 (#1960), #1964 (#1961), #1965 (#1962).

SecureBuffer only erased its memory when freed. The FUSE service that
holds the master keys had no locked memory and could be core dumped,
and a core file of it contained both halves of the master key.

Give each SecureBuffer whole pages of its own, lock them with mlock and
exclude them from core dumps (MADV_DONTDUMP, or MADV_NOCORE on FreeBSD).
Separate pages let a freed buffer be unlocked without affecting another
one. Memory locks are not inherited by fork, so the FUSE service locks
the live buffers again once FUSE has started it in the background. If
RLIMIT_MEMLOCK does not allow locking, the buffers are used unlocked as
before.

Key material can also be copied outside SecureBuffer, so the core
service and the FUSE service additionally disable core dumps: they set
RLIMIT_CORE to 0 and, on Linux, clear the dumpable flag, which a
core_pattern pipe does not ignore.

Volume::WriteSectors encrypts each write in a buffer that is encrypted
in place right after the copy. Allocating it as a SecureBuffer would
lock and unlock its pages on every write and cut write throughput by
about 20 to 60 percent depending on the write size, so it becomes a
Buffer that is still erased when freed.

Fixes veracrypt#1958
Tests/test_linux_fuse_service_memory.py checks locked memory and no
core file for the FUSE service against the installed package in the
Linux workflow.
@flatstik
flatstik force-pushed the fix-1958-key-memory branch from 002cc09 to 49f4b39 Compare October 29, 2026 10:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

key memory is not locked and the volume service can be core dumped

1 participant