feat: Allow toggling event flag during auto-migration for empty tables by Ludv1gL · Pull Request #4875 · clockworklabs/SpacetimeDB

Ludv1gL · 2026-04-22T23:02:03Z

Summary

Currently, toggling #[spacetimedb::table(event)] on an existing table
fails auto-migration with AutoMigrateError::ChangeTableEventFlag and
forces a manual migration. This PR allows the flip in either direction
as a live auto-migration step when the table has zero committed
rows. Non-empty tables fail with an actionable error telling the user
to clear the table first. Clients are disconnected on the flip because
the change is observable to subscribers (event tables have no committed
state; v2 subscribers see a different message variant; v1 subscribers
stop seeing updates).

Changes

crates/datastore/src/locking_tx_datastore/tx_state.rs: new
PendingSchemaChange::TableAlterEventFlag(TableId, bool) variant
storing the old is_event value for rollback.
crates/datastore/src/locking_tx_datastore/committed_state.rs:
rollback branch restoring the old value on the live schema.
crates/datastore/src/locking_tx_datastore/mut_tx.rs: new
alter_table_event_flag(table_id, is_event) mirroring
alter_table_access — dual-writes to the tx + commit table schemas
via with_mut_schema_and_clone, and to st_event_table via
insert_via_serialize_bsatn / delete_col_eq. Idempotent no-op
early-returns before pushing a pending change. New
delete_st_event_table_row helper.
crates/datastore/src/locking_tx_datastore/replay.rs: new hooks on
ST_EVENT_TABLE_ID insert/delete that flip is_event on the
referenced user-table's cached schema, mirroring
reschema_table_for_st_table_update. This is load-bearing for cold
replay across the flip — without it, a snapshot predating the flip
plus commitlog replay would leave the cached schema stale, and
post-flip inserts would incorrectly land in committed state.
crates/core/src/db/relational_db.rs: thin
alter_table_event_flag wrapper analogous to alter_table_access.
crates/schema/src/auto_migrate.rs: event_ok error branch
replaced with AutoMigrateStep::ChangeEventFlag +
plan.ensure_disconnect_all_users(). Removed the now-unused
AutoMigrateError::ChangeTableEventFlag variant. Replaced
test_change_event_flag_rejected with three new tests asserting the
plan shape in both directions and confirming no orphan sub-object
steps.
crates/schema/src/auto_migrate/formatter.rs /
termcolor_formatter.rs: new format_change_event_flag +
EventFlagChangeInfo mirroring format_change_access.
crates/core/src/db/update.rs: new ChangeEventFlag handler with
an O(1) row-count precheck via table_row_count_mut before any
mutation. Fails with a clear message if the table has data.

Safety

Transaction safety: the precheck (row count) and all three writes
(st_event_table row, tx table schema, commit table schema) run in
the same MutTx. No window for concurrent inserts between check and
flip.
Rollback: TableAlterEventFlag stores the pre-flip flag value so
failed transactions revert is_event on the live schema via the
existing rollback_pending_schema_change path. Idempotent flips do
not push a pending change and thus require no rollback work.
Replay correctness: the new st_event_table reschema hook is the
piece with no prior analog. st_event_table was already in the
commitlog, but the existing reschema path only covered st_table and
st_column, so flipping is_event mid-life would have been invisible
to cold replay. Added hooks in both replay_insert and
replay_delete_by_rel call the same helper, reusing the existing
Self::read_table_id(row) pattern (no new unsafe code).
Client contract: flipping event changes what subscribers see —
v1 subscribers stop receiving updates (event tables aren't in default
subscriptions); v2 subscribers receive a different message variant
(TableUpdateRows::EventTable). ensure_disconnect_all_users forces
reconnection so clients observe a consistent state.

Example error output

Cannot change `event` flag on table `my_table`: table contains data.
Clear the table's rows (e.g. via a reducer) before toggling the
`event` annotation.

Test plan

cargo test -p spacetimedb-datastore --features test — 87 pass,
including four new alter_table_event_flag_* tests covering both
directions, rollback, and the idempotent no-op case
cargo test -p spacetimedb-schema — 103 pass, including three new
plan-shape tests (both directions + "no orphan sub-object steps")
cargo test -p spacetimedb-core — 192 pass, including two new
end-to-end update-execution tests (empty succeeds, non-empty fails)
cargo clippy -p spacetimedb-datastore -p spacetimedb-schema -p spacetimedb-core --tests clean
All 10 pre-existing event-table tests still pass
Verified live on a development deployment — flipped the
event flag on an empty user table via spacetime publish; clients
disconnected as expected, subsequent inserts correctly routed to the
commitlog only.

Centril · 2026-04-23T10:13:34Z

+        assert_matches!(
+            tx.pending_schema_changes(),
+            &[PendingSchemaChange::TableAlterEventFlag(t, false)] if t == table_id
+        );


Repeated 3x; Let's make a function out of this: check_table_event_flag_altered(&tx, table_id, state: bool)

Done in ead511c3f4. The new check_table_event_flag_altered(datastore, tx, table_id, expected_is_event) helper in test_helpers.rs bundles the schema is_event check with the st_event_table row-presence check. The 4 test_alter_table_event_flag_* tests each call it in lieu of the inline pattern.

By this, I meant a function that checks pending_schema_changes. This function should be named check_table_event_flag_altered. The one you made in test_helpers doesn't do what its name suggests.

Currently, toggling `#[spacetimedb::table(event)]` on an existing table fails with `AutoMigrateError::ChangeTableEventFlag` and requires a manual migration. This PR allows the flip in either direction as a live auto-migration step when the table has zero committed rows. Non-empty tables fail with an actionable error guiding the user to clear the table first. Clients are disconnected on the flip because the change is observable to subscribers (event tables have no committed state). - **`tx_state.rs`**: New `PendingSchemaChange::TableAlterEventFlag` variant storing the old `is_event` value for rollback. - **`committed_state.rs`**: Rollback branch restores the old value on the live schema. - **`mut_tx.rs`**: New `alter_table_event_flag` — dual-write to the tx + commit table schemas and to `st_event_table`. Idempotent no-op early returns before pushing a pending change. New `delete_st_event_table_row` helper using the existing `delete_col_eq` utility (hits the unique btree index on col 0). - **`replay.rs`**: New hook on `ST_EVENT_TABLE_ID` insert/delete mirrors `reschema_table_for_st_table_update` — flips `is_event` on the referenced user-table's cached schema during replay. This is load-bearing for cold replay across the flip point. - **`relational_db.rs`**: Thin `alter_table_event_flag` wrapper. - **`auto_migrate.rs`**: `event_ok` error branch replaced with `AutoMigrateStep::ChangeEventFlag` + `ensure_disconnect_all_users`. Removed dead `AutoMigrateError::ChangeTableEventFlag` variant. - **`formatter.rs` / `termcolor_formatter.rs`**: New `format_change_event_flag` mirroring `format_change_access`. - **`update.rs`**: New `ChangeEventFlag` handler with an O(1) row-count precheck before any mutation. - **Transaction safety**: Precheck (row count) and all three writes (st_event_table, tx schema, commit schema) run in the same `MutTx`. - **Rollback**: `TableAlterEventFlag` stores the old flag value so failed txs revert `is_event` on the live schema. Idempotent flips do not push a pending change. - **Replay correctness**: Without the replay hook, cold replay from a pre-migration snapshot would miss the schema flip and post-migration inserts would silently land in committed state. The hook mirrors the existing `st_table`/`st_column` pattern. - **Client contract**: Flipping `event` changes observability — v1 subscribers stop seeing updates; v2 subscribers see a different message variant. `ensure_disconnect_all_users` forces reconnection. ``` Cannot change `event` flag on table `my_table`: table contains data. Clear the table's rows (e.g. via a reducer) before toggling the `event` annotation. ``` - [x] `cargo test -p spacetimedb-datastore --features test` — 87 pass (including 4 new `alter_table_event_flag` tests) - [x] `cargo test -p spacetimedb-schema` — 103 pass (including 3 new `change_event_flag` plan tests) - [x] `cargo test -p spacetimedb-core` — 192 pass (including 2 new empty/non-empty integration tests) - [x] `cargo clippy -p spacetimedb-datastore -p spacetimedb-schema -p spacetimedb-core --tests` clean - [x] Pre-existing event-table tests still pass (10 tests)

Address all 16 actionable items from the review — deferred item D1 (moving the row-count precheck into `AutoMigratePrecheck::CheckTableEmpty`) is not addressed here and will follow in a separate PR as agreed. - `schema/auto_migrate.rs`: merge `test_change_event_flag_produces_step_{non_to_event,event_to_non}` into a single test with a closure called twice; make the sub-object-steps test use the same closure pattern instead of duplicating the old/new builder with an implicit `false` vs explicit `true`. - `datastore/replay.rs`: assert `num_rows()==0` inside the `reschema_table_for_st_event_table_update` hook. This enforces the feature's core invariant during cold replay. - `datastore/committed_state.rs`: same assertion on the `TableAlterEventFlag` rollback arm. - `datastore/mut_tx.rs`: extract `insert_st_event_table_row(table_id)` shared by `create_table` and `alter_table_event_flag`. - `datastore/locking_tx_datastore/test_helpers.rs` (new): host the cross-crate-reachable `assert_is_event_state`, `st_event_table_has_row`, and `check_table_event_flag_altered` helpers. Gated by the existing `test` feature. Re-exported from `locking_tx_datastore::mod`. - `datastore/datastore.rs`: replace the 4 `test_alter_table_event_flag_*` tests' inline is-event + st_event_table-row-presence duplication with `check_table_event_flag_altered` (was repeated 3x by the reviewer's count; actual count after deduplication drops to 5 call sites via the helper). Add `TxData` assertions on the successful commits proving the schema change materializes as an `st_event_table` insert/delete and does not touch the user-table row data. Swap `is_empty()` / `assert_matches!(&[])` for `assert_eq!(tx.pending_schema_changes(), [])`. - `core/update.rs`: cleaner v10 builder via `.with_event(is_event)` chain; extract `setup_events_table` returning `TableId`; move the row insert into a separate tx in the non-empty-fails test; replace `.any(|c| matches!(...))` with `assert_matches!([pat, ..])`; replace `.is_empty()` with `assert_eq!([])`; adopt the shared `assert_is_event_state` helper from `spacetimedb-datastore`.

Ludv1gL · 2026-04-23T20:48:10Z

Thanks for the review! Addressed all 16 actionable items — the deferred AutoMigratePrecheck::CheckTableEmpty refactor (your update.rs:298 comment) is sitting out of scope for this PR per our offline agreement and will follow in a separate PR.

Rebased onto current upstream/master (includes #4850 replay extraction, #4666 primary-key migration, etc.). The branch is now two commits:

SHA	Subject
`667dfd5401`	`feat: Allow toggling event flag during auto-migration for empty tables` (the original, re-based)
`ead511c3f4`	`refactor(event-flag): apply Centril 2026-04-23 review feedback` (this round)

Summary of changes in `ead511c3f4`

schema/auto_migrate.rs — merged the two test_change_event_flag_produces_step_* tests into one that calls a build = |is_event: bool| ... closure twice. test_change_event_flag_does_not_produce_orphan_sub_object_steps uses the same closure pattern with an explicit .with_event(is_event) in both branches (no more implicit-false asymmetry).
datastore/replay.rs — assert_eq!(table.num_rows(), 0) inside the reschema_table_for_st_event_table_update hook. Enforces the feature's empty-table invariant on cold replay.
datastore/committed_state.rs — same assertion on the TableAlterEventFlag rollback arm.
datastore/mut_tx.rs — extracted insert_st_event_table_row(table_id) shared by create_table and alter_table_event_flag.
datastore/locking_tx_datastore/test_helpers.rs (new) — assert_is_event_state, st_event_table_has_row, check_table_event_flag_altered. Gated by #[cfg(any(test, feature = "test"))], re-exported from locking_tx_datastore::mod so spacetimedb-core's test module can import it.
datastore/datastore.rs — the 4 test_alter_table_event_flag_* tests now use check_table_event_flag_altered (was the 3× repeated inline pattern). Successful commits now assert TxData reflects exactly the st_event_table insert/delete and leaves the user-table row data untouched. is_empty() / assert_matches!(&[]) → assert_eq!(tx.pending_schema_changes(), []).
core/update.rs — cleaner v10 builder (.with_event(is_event) chain); extracted setup_events_table() returning TableId; insert moved to a separate tx in the non-empty-fails test; .any(|c| matches!(...)) → assert_matches!([pat, ..]); .is_empty() → assert_eq!([]); adopt the shared assert_is_event_state helper.

Validation

cargo test -p spacetimedb-datastore --features test --lib — 87 pass (incl. all 4 test_alter_table_event_flag_*)
cargo test -p spacetimedb-schema — 102 pass (incl. 3 refactored event-flag tests)
cargo test -p spacetimedb-core --lib -- update — 16 pass (incl. both change_event_flag_*)
cargo clippy --tests on touched crates — clean
cargo build -j32 -p spacetimedb-standalone --release — clean

Per-thread replies incoming with the specific commit SHA for each item.

Centril

Getting there but still some things I'd like to change.

Centril · 2026-04-24T14:27:46Z

+        assert_matches!(
+            tx.pending_schema_changes(),
+            [PendingSchemaChange::TableAlterEventFlag(t, false), ..] if *t == table_id


We can use assert_eq! here instead, it's simpler.

Centril · 2026-04-24T14:30:57Z

+            .iter_by_col_eq_mut_tx(
+                tx,
+                ST_EVENT_TABLE_ID,
+                ColList::from(StEventTableFields::TableId.col_id()),


This doesn't seem like it was actually done.

Centril · 2026-04-24T14:32:48Z

-
    #[test]
    fn test_alter_table_event_flag_non_event_to_event() -> ResultTest<()> {
-        // Create a non-event table.


Comment was useful, please bring it back.

Centril · 2026-04-24T14:32:52Z

        let (datastore, tx, table_id) = setup_table()?;
        commit(&datastore, tx)?;

-        // Flip `is_event` from `false` to `true`.


Comment was useful, please bring it back.

Centril · 2026-04-24T14:36:38Z

+        assert_matches!(
+            tx.pending_schema_changes(),
+            &[PendingSchemaChange::TableAlterEventFlag(t, false)] if t == table_id
+        );


By this, I meant a function that checks pending_schema_changes. This function should be named check_table_event_flag_altered. The one you made in test_helpers doesn't do what its name suggests.

Centril · 2026-04-24T14:37:21Z

+            tx_data.inserts_for_table(ST_EVENT_TABLE_ID).map(<[_]>::len),
+            Some(1),


Please assert the contents of the row too.

Centril · 2026-04-24T14:38:41Z

-        );
+        check_table_event_flag_altered(&datastore, &tx, table_id, true);

-        // Flip `is_event` from `true` to `false`.


Useful, please keep it.

Centril · 2026-04-24T14:39:15Z

+            tx_data.deletes_for_table(ST_EVENT_TABLE_ID).map(<[_]>::len),
+            Some(1),


Please assert the row contents here too.

Centril · 2026-04-24T14:39:44Z

    #[test]
    fn test_alter_table_event_flag_non_event_to_event() -> ResultTest<()> {
-        // Create a non-event table.
+        use crate::locking_tx_datastore::test_helpers::check_table_event_flag_altered;


Could be imported once for the test module instead of in each function.

Third round of review fixes. All 9 actionable comments addressed, no incidental cleanup. - `test_helpers.rs`: - `st_event_table_has_row`: drop the `ColList::from(...).col_id()` wrapping and pass `StEventTableFields::TableId` directly (the `iter_by_col_eq_mut_tx` `cols: impl Into<ColList>` + blanket `impl<C: Into<ColId>> From<C> for ColList` handle the conversion). Matches the idiom in `system_tables.rs` (`with_primary_key`, `with_unique_constraint`, `btree(...)`). - Rename: the previous bundled `check_table_event_flag_altered` (schema + `st_event_table` row-presence check) is replaced by a version that asserts `tx.pending_schema_changes()` contains exactly one `TableAlterEventFlag(table_id, !state)` entry — matching the repeated assertion pattern that Centril asked to factor out in round 2. - `datastore.rs` tests: - Hoist the `test_helpers::{assert_is_event_state, check_table_event_flag_altered, st_event_table_has_row}` import to the module level (was duplicated inside each of the 4 tests). - Restore the per-step `// Create a non-event table.` / `// Flip is_event from false to true.` / `// After rollback...` narrative comments and assertion messages that were lost in the previous refactor. - Replace the `inserts_for_table(ST_EVENT_TABLE_ID).map(<[_]>::len) == Some(1)` length-only check (and its `deletes_for_table` counterpart) with an exact row-content assertion against `ProductValue::from(StEventTableRow { table_id })`. - `core/db/update.rs`: - Swap the inline `assert_matches!(tx.pending_schema_changes(), [PendingSchemaChange::TableAlterEventFlag(t, false), ..] if *t == table_id)` for a call to the shared `check_table_event_flag_altered` helper (addresses both the "assert_eq is simpler" suggestion and the round-2 request that the helper live in `crates/datastore` and be used from `update.rs`). Validation: - `cargo test -p spacetimedb-datastore --features test --lib test_alter_table_event_flag` — 4/4 pass - `cargo test -p spacetimedb-core --lib change_event_flag` — 2/2 pass - `cargo clippy -p spacetimedb-datastore --features test --tests` — clean Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Ludv1gL · 2026-04-24T20:17:14Z

New commit fb321ac85b addresses all 9 items from this round — no other changes, scoped tightly per your feedback.

Thread	Fix
`update.rs:659` (`assert_eq!` simpler)	Replaced `assert_matches!` with a call to the new `check_table_event_flag_altered` helper (uses `assert_eq!` internally), which also addresses your earlier "this function should live in `crates/datastore` and be used in `update.rs` too" from round 2.
`datastore.rs:3198` (Outdated — "doesn't seem like it was actually done")	Applied in `test_helpers.rs`: `ColList::from(StEventTableFields::TableId.col_id())` → `StEventTableFields::TableId` (the `cols: impl Into<ColList>` plus blanket `From<C: Into<ColId>>` handle the conversion directly; matches `system_tables.rs:623-625`).
`datastore.rs:3168` (useful comment)	`// Create a non-event table.` restored.
`datastore.rs:3172` (useful comment)	`// Flip is_event from false to true.` restored.
`datastore.rs:3166` (Outdated — helper misnamed)	Replaced: `check_table_event_flag_altered(tx, table_id, state: bool)` now asserts exactly `tx.pending_schema_changes() == [PendingSchemaChange::TableAlterEventFlag(table_id, !state)]` via `assert_eq!`. The previous bundled schema+row helper is gone; sites that need those use `assert_is_event_state` + `st_event_table_has_row` directly.
`datastore.rs:3174` (assert row contents)	`inserts_for_table(ST_EVENT_TABLE_ID)` now compared against `Some(&[ProductValue::from(StEventTableRow { table_id })][..])` instead of only `.map(<[_]>::len) == Some(1)`.
`datastore.rs:3227` (useful message)	"event table should have a row in `st_event_table`" assertion message restored in the event→non-event test.
`datastore.rs:3206` (assert row contents)	Same row-content fix as `:3174` for `deletes_for_table`.
`datastore.rs:3154` (import once)	`use …::{assert_is_event_state, check_table_event_flag_altered, st_event_table_has_row};` hoisted to the top of `pub(crate) mod tests` — removed the per-function imports.

Validated: datastore event-flag tests 4/4 pass, core change_event_flag_* tests 2/2 pass, cargo clippy clean.

Centril self-requested a review April 23, 2026 09:07

Centril self-assigned this Apr 23, 2026

Centril added enhancement New feature or request release-any To be landed in any release window labels Apr 23, 2026

Centril requested changes Apr 23, 2026

View reviewed changes

Ludv1gL added 2 commits April 23, 2026 22:26

Ludv1gL force-pushed the feat/auto-migrate-event-flag-empty-table branch from 40af039 to ead511c Compare April 23, 2026 20:47

Centril requested changes Apr 24, 2026

View reviewed changes

		tx_data.inserts_for_table(ST_EVENT_TABLE_ID).map(<[_]>::len),
		Some(1),

		tx_data.deletes_for_table(ST_EVENT_TABLE_ID).map(<[_]>::len),
		Some(1),

Conversation

Ludv1gL commented Apr 22, 2026

Summary

Changes

Safety

Example error output

Test plan

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Ludv1gL commented Apr 23, 2026

Summary of changes in ead511c3f4

Validation

Uh oh!

Centril left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Ludv1gL commented Apr 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Summary of changes in `ead511c3f4`