Skip to content

Keep a hot journal if the key does not fit it, add a crash test - #314

Merged
utelle merged 3 commits into
utelle:mainfrom
SchwarzDigits:test/crash-recovery
Nov 17, 2026
Merged

utelle merged 3 commits into
utelle:mainfrom
SchwarzDigits:test/crash-recovery

Conversation

@mtrossbach

@mtrossbach mtrossbach commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Data loss with aes128cbc/aes256cbc after a crash and a wrong key.

If a crash leaves a hot journal and the application opens the database with a wrong key first (for example a mistyped passphrase), the journal is finalized without restoring anything: the open fails with "file is not a database", but the journal is gone, and with the right key rows are missing and integrity_check reports errors. With a cipher without authentication, the wrong key decrypts the journal records into garbage without an error; SQLite sees a record whose checksum does not match, takes it as the end of the journal (pager_playback_one_page returns SQLITE_DONE) and finalizes the journal. mcHotJournalGuard only covers the case without a key. Ciphers with authentication (chacha20, sqlcipher, aegis, ascon128) are not affected; the decryption error keeps the journal. The same happens, though, after an interrupted change from such a cipher to aes256cbc if the new key is tried first. 2.6.1 and 2.7.1 behave the same.

Before SQLite plays back a hot journal, mcHotJournalKeyCheck now decrypts the first record with the current key and compares the checksum as SQLite does (pager_cksum). SQLite writes the journal magic and the record count only after the records have been synced, and changes the database file only after that, so the first record of such a journal is complete and matches with the right key. If it does not match, the access is refused with SQLITE_NOTADB, as without a key, and the journal is kept for an attempt with the right key. Journals without a record count (0xffffffff: no sync or SQLITE_IOCAP_SAFE_APPEND) are passed through as before.

test/crashtest.c (CI step "Crash tests", about 7 seconds):

a VFS below the Multiple Ciphers VFS keeps the files in memory and counts every write, sync, truncate and delete. An operation runs again and again, and in run k the k-th call fails: as a crash, as a torn write (half of the data arrives), as a power loss (every file falls back to its last sync), or as an I/O error. A new connection must then find the database complete in the old or in the new state, also after first trying without a key, with a wrong key and, after a key change, with the other key; the state must stay the same on a second open, no file may contain the plaintext of the rows, and if an I/O error was injected and the operation reported success, the database must be in the new state. The default selection covers all ciphers with journal modes DELETE and WAL, page size 4096, and transaction, rekey and cipher change; crashtest full adds TRUNCATE and PERSIST, page size 1024, decryption and VACUUM (about 91000 runs, 80 seconds). An interrupted decryption of a sqlcipher database can only be rolled back with the old salt (cipher_salt); the test accepts this and reports the count.

Testing (macOS arm64, flags from Makefile.am):

  • With the fix: default and full selection pass, the other CI tests pass as well; ASan and UBSan are clean.
  • On e65527d without the fix: 173 failures in the default selection: aes128cbc/aes256cbc after an open with a wrong key or with the other key of an interrupted rekey, and chacha20 after an interrupted change to aes256cbc when the new (aes256cbc) key is tried first.
  • On v2.6.1 the test also finds the problems behind A crash during PRAGMA rekey can leave a database that opens with neither key #291 to A PRAGMA rekey that fails after spilling pages leaves a database no key opens #294 and the data loss after an I/O error during a rekey.
  • With real files (database and journal copied during an open transaction whose pages were already spilled to the file), e65527d and 2.7.1 lose 342 of 3000 rows with aes256cbc after an open with a wrong key; with the fix the journal is kept and all rows are back with the right key.

mtrossbach and others added 3 commits November 17, 2026 08:40
With a wrong key, a cipher without authentication (aes128cbc, aes256cbc)
decrypts the records of a hot journal into garbage without an error. SQLite
then sees a record whose checksum does not match, takes it as the end of
the journal and finalizes it without having restored anything: the
database stays half changed, and the journal is gone. An application that
opens the database with a mistyped passphrase after a crash loses data.

Before SQLite plays back a hot journal, mcHotJournalKeyCheck decrypts the
first record with the current key and compares the checksum as SQLite does.
SQLite writes the journal magic and the record count only after the
records have been synced, so the first record of such a journal is
complete. If the checksum does not match, the access is refused with
SQLITE_NOTADB, as without a key, and the journal is kept.
test/crashtest.c runs transactions, rekeys, cipher changes, decryption and
VACUUM on databases kept in memory by a VFS below the Multiple Ciphers VFS.
In run k the k-th write, sync, truncate or delete fails: as a process crash,
with a torn write, as a power loss (files fall back to their last sync), or
as an I/O error. A new connection must then find the database complete in
the old or in the new state, also after a first attempt without a key or
with a wrong key, and no file may contain plaintext rows.

The default selection runs in a few seconds; 'full' runs all ciphers,
journal modes, page sizes and operations.
@utelle
utelle merged commit 07b3c48 into utelle:main Nov 17, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants