Skip to content

[#874] Grant the offline tools a read-only JDBC transaction instead of refusing it - #880

Open
vharseko wants to merge 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:issues/874-offline-tools-jdbc-readonly
Open

[#874] Grant the offline tools a read-only JDBC transaction instead of refusing it#880
vharseko wants to merge 1 commit into
OpenIdentityPlatform:masterfrom
vharseko:issues/874-offline-tools-jdbc-readonly

Conversation

@vharseko

Copy link
Copy Markdown
Member

Problem

export-ldif, verify-index and backendstat open a backend read-only when the server is not already holding it open — a stopped server, or a disabled backend. JDBCStorage refused to hand out a write transaction in that mode, and RootContainer.open() always asks for one, so all three tools failed on a JDBC backend before they read anything:

The database environment could not be opened: This storage is read-only.
  at JDBCStorage$WriteableTransactionTransactionImpl.<init>(JDBCStorage.java:453)
  at JDBCStorage.write(JDBCStorage.java:260)
  at RootContainer.open(RootContainer.java:135)
  at BackendImpl.newRootContainer(BackendImpl.java:987)

RootContainer.open() asks for a write transaction even in READ_ONLY mode because that is where it opens the compressed schema and the entry containers. What it actually does through that transaction in read-only mode is only openTree(name, false), read, openCursor and getRecordCountPersistentCompressedSchema.load() gets shouldCreate = false, EntryContainer.open() passes shouldCreate = accessMode.isWriteable() down, and DefaultIndex.afterOpen() already guards its own write with createOnDemand.

Only the offline case is affected. The online task path reuses the root container the running server holds, which is READ_WRITE, so an online export-ldif task works.

Change

The mode check moves from the constructor of WriteableTransactionTransactionImpl to the mutating operationsopenTree(..., true), clearTree, deleteTree, put, update and delete. That is the shape the other three storages of this server already have; JDBC was the only outlier of the four:

Storage Read-only write transaction
PDBStorage ReadOnlyStorageImplopenTree(createOnDemand) and the mutators throw
JEStorage ReadOnlyTransactionImpl — same
CASStorage TransactionImpl.checkReadOnly() per operation
JDBCStorage refused in the constructor ← this PR

The mode is captured once per transaction rather than read per operation: the access mode of the storage is mutable state — ImporterImpl reopens it READ_WRITE under its caller — and a transaction has to keep the mode it was created with. It also drives isReadOnly, so a cursor such a transaction opens refuses delete() as well; PDBStorage.ReadOnlyStorageImpl.openCursor() delegates to its writeable implementation and leaves that hole open.

VLVIndex no longer writes from its constructor. It upgraded an untrusted index of an empty backend to trusted there, regardless of the access mode, so the same three tools would still fail on an empty backend carrying a VLV index — on PDB and JE too, not only JDBC. The write moves to afterOpen(txn, createOnDemand), guarded exactly as DefaultIndex.afterOpen() already guards its own. EntryContainer.open() and the VLV add-listener both call open(txn, true) right after constructing the index, and the listener reads isTrusted() only after that call, so the upgrade still happens where it did.

Tests

testReadOnly() caught none of this, and could not: it expects a ReadOnlyStorageException, and a storage that fails the open throws one too — from RootContainer.open() rather than from the write the test means to be checking. It was green on the broken code for the wrong reason, and after this change it finally exercises the put it was written for.

  • testOfflineToolsOpenBackendReadOnly (PluggableBackendImplTestCase, so it covers pdb / jeb / jdbc / cassandra at once) — closes the backend and runs export-ldif and verify-index against it, which is exactly the offline path.
  • testReadOnlyTransactionReadsButRefusesWrites (backends/jdbc/TestCase) — a read-only transaction serves openTree(..., false), read, openCursor and getRecordCount, and refuses openTree(..., true), put, update, delete, deleteTree and cursor.delete(); a re-open afterwards asserts nothing reached the database.

Verified that both tests fail without the fix: on PostgreSQL the suite goes 38/40 with exactly those two red, carrying the stack trace above.

All suites pass locally, no skips:

Suite Result
PostgreSQL (JDBC) 40/40
MySQL (JDBC) 40/40
MS SQL (JDBC) 40/40
Oracle (JDBC) 40/40
Cassandra 39/39
PDB + JE + PDBStorageTest + OnDiskMergeImporterTest 102/102

(The counts above are from the branch as first written; rebased onto master the JDBC suites lose the one test that belongs to #867, and PgSql + PDB + JE re-run at 109/109.)

Out of scope

Two neighbouring gaps this does not close, both worth their own issue:

  • JDBCStorage.listTrees() returns the JVM-local tree2table memo rather than the tables in the database, so backendstat list-raw-dbs / dump-raw-db see only the trees whose names happened to be hashed in this process. CASStorage.listTrees() returns an empty set outright (TODO). Recovering the real mapping needs the tree name stored on the table, which is what Stamp JDBC backend tables with their tree name and refresh optimizer statistics after import #866 adds.
  • A backend that was never opened read-write has no tables, so an offline export of it fails with a raw "relation does not exist" instead of exporting nothing. PDBStorage and JEStorage have ReadOnlyEmpty* implementations for that case; JDBC has no analogue.

Fixes #874

@vharseko vharseko added bug jdbc java Pull requests that update java code tests Test suites: fixing, enabling, un-disabling labels Aug 19, 2026
@vharseko
vharseko requested a review from maximthomas August 19, 2026 11:44
…ransaction instead of refusing it

export-ldif, verify-index and backendstat open a root container of their own,
in READ_ONLY mode, whenever the backend is not already open - a stopped server
or a disabled backend. RootContainer.open() asks the storage for a write
transaction even in that mode, since that is where it opens the compressed
schema and the entry containers, so JDBCStorage refusing to construct one
failed all three tools before they read anything.

The mode check moves from the constructor of WriteableTransactionTransactionImpl
to the mutating operations - openTree(..., true), clearTree, deleteTree, put,
update and delete. That is the shape the other three storages of this server
already have: PDBStorage.ReadOnlyStorageImpl, JEStorage.ReadOnlyTransactionImpl
and CASStorage.checkReadOnly(). The mode is captured once per transaction, since
ImporterImpl reopens the storage READ_WRITE under its caller, and it also drives
isReadOnly, so a cursor such a transaction opens refuses delete() as well.

VLVIndex upgraded an untrusted index of an empty backend to trusted from its
constructor, regardless of the access mode. That write moves to
afterOpen(txn, createOnDemand), guarded exactly as DefaultIndex.afterOpen()
already guards its own: it would otherwise fail the same three tools on an empty
backend carrying a VLV index, on PDB and JE too.

testReadOnly() caught none of this - it expects a ReadOnlyStorageException, and a
storage that fails the open throws one too, from RootContainer.open() rather than
from the write it means to be checking. testOfflineToolsOpenBackendReadOnly now
runs export-ldif and verify-index against a closed backend for every pluggable
backend, and testReadOnlyTransactionReadsButRefusesWrites asserts that the JDBC
read-only transaction serves openTree(..., false), read, openCursor and
getRecordCount while refusing every mutation.
@vharseko
vharseko force-pushed the issues/874-offline-tools-jdbc-readonly branch from e48d4d1 to 23d63bf Compare August 20, 2026 10:36
@vharseko

Copy link
Copy Markdown
Member Author

Rebased onto master (0b9c0f6), which had moved on under the JDBC backend since this branch was cut (#886 catalog lookup, #866 table stamping, #867 SQL Server upsert).

Conflicts and how they were resolved:

  • JDBCStorage.delete(): master's statement (where h="+hashParam(con)+", the cast that keeps the primary key seekable on SQL Server) with this branch's checkReadOnly() in front of it.
  • jdbc/TestCase.java: both imports kept (Importer from master, ReadOnlyStorageException from here).

Also checked that the write path master added meanwhile - commentTable() in openTree(createOnDemand) - sits behind checkReadOnly(), so a read-only open still stamps nothing.

mvn -pl opendj-server-legacy test-compile passes. The tests of this PR need a database and are left to CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug java Pull requests that update java code jdbc tests Test suites: fixing, enabling, un-disabling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Offline export-ldif, verify-index and backendstat cannot open a JDBC backend

2 participants