Repository navigation
sbx-mgr: callers use the store's own handle and transaction types - #570
Conversation
73cc7b6 to
a2ec835
Compare
a2ec835 to
20e4085
Compare
Review — 2 findingsNo correctness findings. I checked that Low
Verified against |
the-gigi-pplx
left a comment
There was a problem hiding this comment.
Approved. 2 non-blocking findings in the review comment above.
20e4085 to
bc51f2f
Compare
sbx-mgr's node-local SQLite store, `src/db`, now defines the types its
callers hold, so nothing outside it names rusqlite:
- `db::SharedConnection` replaces `Arc<Mutex<rusqlite::Connection>>`. It
wraps the same mutex, so the jobs worker shares it (`background_worker`),
and `lock()` returns a `ConnectionGuard` with `Mutex::lock`'s poisoning.
`connection_or_signal_failure` keeps its semantics on the new handle.
- `db::Connection`, a `repr(transparent)` view (a private
`RefCastCustom` view, so the crate stays `forbid(unsafe_code)` and
publishes no `RefCast` impl), is what every `db` function takes. Its
raw accessor and statement methods, `Error::raw`, and the row helpers
that take or return rusqlite values are private to `db` in production
builds, and the first two groups are crate-visible only under
`cfg(test)`. The jobs worker the store builds returns common-jobs' own
`PruneTerminalJobsError` from `try_prune_terminal_jobs_once`, with the
same `Display`. So production code outside `db` cannot reach a rusqlite
value; the only rusqlite type left in the store's public API is
`db::Error`'s inbound `From<rusqlite::Error>`.
- `db::Transaction::{immediate, deferred}` wraps rusqlite's transaction and
derefs to the connection.
- `db::Error` is a `thiserror` `#[error(transparent)]` wrapper around
rusqlite's error, so `Display` and `source` are unchanged wherever
clients and operators see them. Every `db` function and error enum
returns or wraps it; callers test it with `is_no_rows` and
`is_invalid_query` instead of matching rusqlite variants.
- Opening and migrating the database move into `db` (`db::open`,
`db::migrate`, `db::OpenError`, transparent over the shared helpers'
error), and the cleanup executor's job-store calls go through
`db::cleanup_jobs`.
Tests outside the store get raw SQL through the test-visible statement
methods on `db::Connection` and a small `#[cfg(test)]` `db::fixture`
re-export (`params!`, trace types, `Value`, `Type`, `StatementStatus`),
plus test-only error predicates. No behavior changes.
The `sbxmgr_sqlite` baseline is now empty and `no_new_entries` is
required for it; the `sbxmgr_sqlite_schema`
baseline lists only the database migrator, which milestone 3 moves onto
the store's migration entry point.
bc51f2f to
00a2036
Compare
Summary
Milestone 2 of 3, stacked on #561. It concerns sbx-mgr's node-local SQLite database (
sandboxesand every other sbx-mgr table), not the control-plane Postgres. The store,crates/libs/sbx/mgr/server/src/db, now defines every type its callers hold. No code outsidedb/names rusqlite, and thesbxmgr_sqliteratchet baseline is empty.The store's handle types (
db/handle.rs)db::SharedConnectionArc<Mutex<rusqlite::Connection>>Arc<Mutex<rusqlite::Connection>>, because common-jobs'BackgroundWorkermust share that mutex (SharedConnection::background_worker).lock()returnsLockResult<ConnectionGuard>withMutex::lock's poisoning: a poisoned lock still yields its guard through thePoisonError, whose text is unchanged.is_poisoned()delegates.db::ConnectionGuardMutexGuard<'_, rusqlite::Connection>db::Connection.db::Connectionrusqlite::Connectiondbfunction takes as&Connection. It is#[repr(transparent)]over rusqlite's connection.#[derive(RefCastCustom)]generates only a privateview(&rusqlite::Connection) -> &Connection; there is no publicRefCastimpl. The crate keeps#![forbid(unsafe_code)].raw(),execute,prepare,prepare_cachedandquery_roware private todbin production builds (see below).db::Transaction::{immediate, deferred}rusqlite::Transaction::new_unchecked(.., Immediate),unchecked_transaction(),transaction()Connection, and rolls back on drop.commit/rollbackreturndb::Result.db::Error/db::Resultrusqlite::Error/rusqlite::Resultthiserror,#[error(transparent)]overrusqlite::Error, soDisplayandsource()are byte-for-byte the same. Callers useis_no_rows()/is_invalid_query()instead of matching rusqlite variants.Error::raw()is private todbin production builds. The constructorsinvalid_query,no_rows,poisonedandto_sql_conversionbuild the same rusqlite variants as before, so their texts are identical.db::OpenErrorpplx_space_common_svc::sql::sqlite::SqliteErrorinrun.rsEvery
dbfunction anddberror enum now takes these types and returns or wrapsdb::Error. Row decoders insidedbkeeprusqlite::Result, since rusqlite's row callbacks require it.Opening and migrating move into
dbrun.rs's migration moves todb::migrate()/db::migrate_at(path), with the same migrators in the same order.db::open()keeps the same open helper, pragmas and statement-cache capacity.RunSbxMgrError::{Migrate, OpenDatabase}keep their messages ("failed to migrate the sandbox-manager database: {0}", "failed to open the sandbox-manager database: {0}"). Their{0}and source chain are unchanged becauseOpenErroris transparent.test_conn,create_test_schema*) and theserialize-based schema image were already indb.Callers
db/is converted:Arc<Mutex<rusqlite::Connection>>/&rusqlite::Connectionspellings;service/mod.rs,activation_support.rsandgrpc_lifecycle/teardown_suspend.rs;connection_or_signal_failuretakes&SharedConnectionand keeps its semantics exactly: on a poisoned lock it logs, notifies the supervised outbox worker and returnsDatabaseMutexPoisoned.main's code since the original base is converted the same way. The retired-teardown release claim indb/eviction.rs([backup] refuse teardown-suspend backups from retired executions and release their runtimes #331) passesConnection::raw()to the job store from insidedb, and collects its artifact rows intodb::Resultas the other readers do.main's new tests takedb::Connection, bind withdb::fixture::params!and passconn.raw()to the job store:grpc_lifecycle/retired_teardown_tests.rs,service/tests/node_manager_transport_tests.rsanddb/tests/stopped_artifact_pin.rs.service/cleanup_executor.rsno longer importspplx_space_common_jobs::db. Its reset, get, claim, yield and retry calls go through newdb::cleanup_jobsfunctions (reset_running,reset_inactive_running,get_job,mark_running,yield_running,retry_running) that wrap the same job-store calls.What production code outside
dbcan reachThe boundary, exactly. In production builds (
not(test)), code outsidedbgets no rusqlite value from the store's types or from the jobs worker the store builds:db:Connection::raw(),Error::raw(), the statement methodsexecute,prepare,prepare_cachedandquery_row,Connection::from_rawandError::decodearepub(in crate::db). So are the row-level helpers that take or return rusqlite values:timestamps::EpochSeconds::decode,timestamps::diagnostic_textandinventory_read::check_row. The ref-cast view is private todb/handle.rs.SharedConnection::background_workerbuilds common-jobs'BackgroundWorkerinsidedbfrom the privateArc<Mutex<rusqlite::Connection>>and returns only the worker. The worker's one public method that returned a rusqlite value,try_prune_terminal_jobs_once, now returns common-jobs' ownPruneTerminalJobsError. That type is#[error(transparent)]over a privaterusqlite::Errorfield and has noFromimpl, so itsDisplay(logged bysnapshot_upload_jobs.rs) is byte-identical and no rusqlite value comes out.snapshot_upload_jobs.rsis that method's only caller outside common-jobs.db::Error's#[from] rusqlite::Errorconversion, an inboundFromimpl that hands nothing out. This PR keeps everything in one crate, socfg(test)widens these methods to the whole crate; milestone 3 replaces that with the store'stest-supportfeature.Under
cfg(test), the six raw/statement methods above arepub(crate)for fixtures and tripwires. One macro inhandle.rsdeclares each method twice, once per cfg, so both visibilities share one body. I checked this by addingconn.raw(),error.raw()andconn.prepare(..)to a production module:cargo check --librejects all three with E0624.Tests
Exactly these items are test-only (
#[cfg(test)],pub(crate)):Connection::{open_in_memory, open_at, pragma_update, pragma_query_value, busy_timeout, trace_v2, execute_batch};SharedConnection::try_lock;Error::{is_database_busy, conversion_failure, into_raw};db::fixture, which re-exportsparams!, the trace types,Value,TypeandStatementStatus;test_conn,create_test_schema*,seed_historical_lifecycle_rows);#[cfg(test)]before this PR:complete_row_intent_cleanup{,_with_receipt},clear_cleanup_intent,list_requested_cleanup_recovery_candidate_ids,mark_quarantinedandscan_quarantine_recovery_candidates;route_activation_claims::claim_exists;mark_{stopping,deleting}_superseding_activation(and their private helper),mark_{starting,restoring}_quarantined,rollback_{starting,restoring}andupdate_state_from_states.The six methods above,
raw,execute,prepare,prepare_cached,query_rowandError::raw, are test-visible butdb-private in production.is_autocommit()ispub(crate)in every build, because it returns only abool.Tests that call the shared job store directly pass
Connection::raw().run/tests.rs' worker-drain test now builds its worker overdb::test_conn()(the full schema, which includesjobs) instead of a hand-migrated common-schema connection. No test changed what it asserts.Counts (outside
db/)rusqlite::/use rusqlitelines, productionrusqlite::/use rusqlitelines, tests[pattern_ratchet.sbxmgr_sqlite]baseline[pattern_ratchet.sbxmgr_sqlite_schema]baselineCounts are code lines, comments excluded, as archcheck counts them.
file_size: no cap raised. Caps are lowered where files shrank, and one is tightened:db/auto_lifecycle_delete_receipts.rs1158→1149;db/sandboxes.rs1547→1541;db/state_transitions.rs1016→1012;run.rs1101→1091;service/cache_cleanup.rs1455→1452. This one tightens the cap to the file's current size; the file did not shrink in this PR.service/auth_lookup.rskeeps 1221 anddb/cleanup_episodes.rskeeps 1833:maingrew both past their caps (to 1222 and 1845), and this PR brings them back to exactly those caps.Hidden observable changes
tracingevent moved between modules.sql!locators:sql!statements indbfiles this PR edits keep their module paths. Theirline!()values shift with the edits above them. No statement changed text otherwise.#[from]/#[source]fields that heldrusqlite::Errornow holddb::Error, whoseDisplayandsource()are rusqlite's.source().downcast_ref::<rusqlite::Error>()would now seedb::Error. Nothing in the workspace downcasts that way.file!()-derived values are involved.Review round 2
rusqlite::Errorthroughtry_prune_terminal_jobs_oncePruneTerminalJobsError(transparentDisplay, privaterusqlite::Errorfield, noFrom).git grepfinds one caller outside common-jobs,snapshot_upload_jobs.rs. It logs it with%error, so the log text is unchanged.prune_terminal_jobs_oncelogs the sameDisplayas before.EpochSeconds::decode,diagnostic_textandcheck_rowarepub(in crate::db); all their callers are indb/. A sweep of everypub/pub(crate)signature and field indb/, including names imported from rusqlite under short aliases, finds only#[cfg(test)]items and private tuple fields.#[cfg(test)].Review round 1
#[derive(RefCast)]publishes the raw type#[derive(RefCastCustom)]with a private#[ref_cast_custom] fn view. There is noRefCastimpl, andforbid(unsafe_code)holds.Connection::raw()andError::raw()reachable from productionpub(in crate::db)outsidecfg(test)andpub(crate)under it. So areexecute,prepare,prepare_cachedandquery_row, which return rusqlite statements. A production probe fails with E0624.due_job_ids' doc comment displaceddue_job_ids.no_new_entriesadvisory with an empty baselinerequired, and the comment is fixed.cache_cleanup.rsis described as tightened; the test-only surface is listed exactly.Rebased onto #561's new head (round 2). The only conflict was the
sbxmgr_sqlite_schemabaseline, which now lists the migrator at 5 lines.Rebased with the stack onto
main825b786. Conflicts:Cargo.lock(regenerated); the baseline (file_sizecaps recomputed,node_manager_commandskept); anddb/local_artifacts.rs, wheremainmaderow_to_local_artifactpub(super)fordb/eviction.rsand this PR made it returnrusqlite::Result. It now does both.Rebased with the stack onto
mainc4be884, without conflicts. The caps are unchanged.Decisions
ref-cast'sRefCastCustomrather thanunsafe, and the raw mutex kept. The jobs worker must share sbx-mgr's mutex, so the guard derefs arusqlite::Connectionthat has to be viewed asdb::Connection.ref-cast1.0.27, already inCargo.lock, does that safely underforbid(unsafe_code).RefCastCustomkeeps the view private, where#[derive(RefCast)]would have published<Connection as RefCast>::From = rusqlite::Connection. It is added as a workspace dependency;ci/rust-artifact-inputs.jsonis unchanged.db-private in production, test-visible in tests. The compiler, notcargo arch, keeps production code outsidedbfrom reaching a raw connection, statement or error. Milestone 3 moves the same split onto the store crate'stest-supportfeature.sbxmgr_sqlite.no_new_entriesis required. The baseline is empty, so no new entry is allowed. The rule's comment now calls its patterns a line-based tripwire.Validation
bash ci/scripts/run-rust-checks.sh fmt(includes the Rust error policy)bash ci/scripts/run-rust-checks.sh clippy(CI's clippy mode, including the workspace--all-targets --all-featurespass)cargo arch --base-baseline <main> --base-architecture <main>file_sizedrift and the schema table'sno_new_entries)SPACE_CI_REQUIRE_TEST_DEPS=1 cargo nextest run --locked -p pplx-space-sbx-mgr -p pplx-space-db-migrate -p pplx-space-common-jobs --retries 0#[ignore]transfer_directoryUID test, not a dependency skipcargo test --locked -p pplx-space-archcheckcargo clippy --locked -p pplx-space-sbx-mgr -p pplx-space-sbxmgr-svc -p pplx-space-db-migrate --lib --bins -- -D warningsAfter the
c4be884rebase, fmt, clippy,cargo archagainstmainand the archcheck tests re-ran on every head of the stack and passed. Nextest ran once, on #586's headd9f6b99:SPACE_CI_REQUIRE_TEST_DEPS=1 cargo nextest run --locked --all-features -p pplx-space-sbx-mgr -p pplx-space-sbx-mgr-store -p pplx-space-db-migrate -p pplx-space-sqlite -p pplx-space-db-test-fixtures --retries 0gave 1808 passed, 0 failed (server 1367, store 366, migrator 53, sqlite 12, test fixtures 10). The 1 skip is the root-only#[ignore]transfer_directoryUID test. Kind was not re-run, because the rebase changed no code.🤖 Generated with Claude Code