refactor(persistence)!: run the web database on WebAssembly - #2941
Draft
renefloor wants to merge 7 commits into
Draft
refactor(persistence)!: run the web database on WebAssembly#2941renefloor wants to merge 7 commits into
renefloor wants to merge 7 commits into
Conversation
Moving `stream_chat_persistence` onto `package:drift/wasm.dart` needs two files served by the consuming app, `sqlite3.wasm` and `drift_worker.js`. From drift 2.34.2 onward a single drift release publishes both, which removes a version-matching hazard: `sqlite3.wasm` has to match the `package:sqlite3` version drift resolves, and `sqlite3` is only a transitive dependency here (drift constrains it to `^3.1.5`), so it can float without us noticing. Taking both files from one release makes that impossible to get wrong. `package:web` becomes a direct dependency of `stream_chat_persistence` because its web implementation uses `window.localStorage` directly, and `depend_on_referenced_packages` requires it to be declared. It is already in the tree as a drift transitive dependency. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Groundwork for the WebAssembly web database; nothing uses these yet. `StreamChatPersistenceWebOptions` is the public knob for where an app serves `sqlite3.wasm` and `drift_worker.js`. Both default to a path relative to the document's base href, so apps served from a sub-path work without configuration, and a CDN or custom asset directory can be pointed at explicitly. It is grouped as a value class rather than added as more `webXxx` constructor parameters because `WasmDatabase.open` has several more options we may want to expose later, and following `StreamHttpClientOptions` in `stream_chat`. `web_storage_diagnostics.dart` holds every judgement call about the storage drift picked: which log level it deserves, and what the message says. On the web drift chooses between five storage implementations based on browser support, two of which cannot be trusted — `unsafeIndexedDb` can be corrupted by a second tab, and `inMemory` persists nothing at all — so the choice has to be reported rather than assumed. Both files are deliberately free of web-only imports and take plain strings rather than drift's enums. `package:drift/wasm.dart` imports `dart:js_interop`, so a signature naming `WasmStorageImplementation` could not even be imported from a VM test; keeping the decisions here makes them testable with `flutter test`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replaces `package:drift/web.dart`, which drift has deprecated and put in bugfix-only mode, with `package:drift/wasm.dart`. This also fixes persistence on WebAssembly builds. `shared_db.dart` selected the web implementation with `if (dart.library.html)`, which is false under dart2wasm, so those apps silently resolved to `unsupported_db.dart` and threw `UnsupportedError`. The condition could not simply be switched to `dart.library.js_interop` while the old backend was in place, because `package:drift/web.dart` imports `dart:html` and that would have turned a runtime error into a compile failure. With the dependency replaced, one web implementation now serves dart2js and dart2wasm alike. BREAKING: web apps must serve `sqlite3.wasm` and `drift_worker.js` from their `web/` folder in place of `sql-wasm.js` and `sql-wasm.wasm`, and drop the `sql-wasm.js` script tag from `index.html`. The legacy sql.js cache cannot be read by the new backends, so the cache is refilled from the API once; its two `localStorage` keys are removed on connect to reclaim the space, since `localStorage` is capped near 5 MB. BREAKING: `webUseExperimentalIndexedDb` is removed in favour of `webOptions`. It named `DriftWebStorage.indexedDbIfSupported`, which does not exist in the new backend, so a deprecated shim could only have silently done nothing. drift now probes the browser and picks the most reliable storage itself, which is what the flag was approximating. Two failure modes needed handling, because drift reports neither in a way a caller can act on: - A missing `drift_worker.js` is swallowed into `MissingBrowserFeature.workerError` and drift falls back to a database that answers every query and persists nothing. That is now logged at SEVERE, naming the file and the URI it was looked up at. - A missing `sqlite3.wasm` lets `WasmDatabase.open` resolve successfully and fails on the first query instead, inside a worker, mentioning neither the file nor where it was sought. Opening is therefore forced with one `SELECT 1` so the failure lands at `connect` with both. Because that eager open can also fail on a cache damaged by, say, a killed tab, an unopenable database is discarded and reopened once before giving up. The local database is only a cache, so this costs one extra sync rather than blocking sign-in, while a genuine setup problem still fails loudly because it fails the retry the same way. `SharedDB.constructDatabase` gains a `Logger` so the storage report reaches the handler the integrator already configured through `logHandlerFunction`, instead of a second logger they cannot control. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The WebAssembly backend loads a different pair of files than sql.js did, so every web app in the repo swaps `sql-wasm.js` and `sql-wasm.wasm` for `sqlite3.wasm` and `drift_worker.js`. Both new files come from the same drift release. That pairing matters: `sqlite3.wasm` must match the `package:sqlite3` version drift resolves, and `drift_worker.js` must match drift itself, whose changelog repeatedly notes protocol changes that require an updated worker. Taking both from one release is the only combination guaranteed to be compatible, so `scripts/fetch_web_assets.sh` pins the drift version in one place and downloads them into all three `web/` directories. It is wired up as `melos run web:assets`, following `scripts/generate.sh` and `gen:openapi`, so bumping drift is two commands rather than six manual downloads and the pin is reviewable in a diff. The files stay committed, as the sql.js ones were, so `flutter build web` and CI keep working with no extra step and a fresh clone never hits the missing-asset error. `.gitignore` needs new negations to match: the repo ignores `*.js` and `*.wasm` wholesale and listed the sql.js paths as exceptions. Without this the new assets are silently untracked and the sample apps break for anyone who clones. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Drops the `sql-wasm.js` script tag from all three web apps. Nothing replaces it: `drift_worker.js` is started by drift as a worker at runtime and must not be loaded into the page, and `sqlite3.wasm` is fetched by the worker. The `stream_chat_persistence` and `stream_chat_flutter` examples were also still on the pre-`flutter_bootstrap.js` template, which hardcodes `<script src="main.dart.js">`. That silently defeats `--wasm`: the build emits `main.dart.wasm`, the page loads the dart2js output instead, and the app appears to work while never running the WebAssembly it just compiled. Both now use the current bootstrap, which also removes the deprecated service worker registration Flutter warns about on every build. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README's "Flutter Web" section was doubly stale: it documented sql.js and showed a pre-`flutter_bootstrap.js` `index.html`. It now covers the two files to copy and where to get them, states plainly that nothing is added to `index.html`, and explains what each browser actually ends up with. That last part is the non-obvious bit. Which storage drift can use depends on whether the app is cross-origin isolated, because `opfsLocks` needs `SharedArrayBuffer`, which is only exposed with the `Cross-Origin-Opener-Policy` and `Cross-Origin-Embedder-Policy` headers. Without them Chrome and Safari fall back to IndexedDB behind a shared worker. Everything works either way, so the headers are documented as a recommendation rather than a requirement. Also adds the v11 migration guide entry — Symbol Map rows, a Quick Reference row and a Feature Area section — as that guide requires breaking changes to do in the same PR, and the CHANGELOG entries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
(cherry picked from commit 96a8098)
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueThanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces
package:drift/web.dart, which drift has deprecated and put in bugfix-only mode, withpackage:drift/wasm.dart— and fixes persistence on WebAssembly builds along the way.shared_db.dartselected the web implementation withif (dart.library.html), which is false under dart2wasm, so those apps silently resolved tounsupported_db.dartand threwUnsupportedError. The condition could not simply be switched todart.library.js_interopwhile the old backend was in place, becausepackage:drift/web.dartimportsdart:htmland that would have turned a runtime error into a compile failure. With the dependency replaced, one web implementation now serves dart2js and dart2wasm alike.Note
Based on
v11. The last commit cherry-picks thedart.library.js_interopfixes forstream_chatandstream_chat_flutter, which landed onmasteras #2940 but are not onv11— without them a wasm build still dies inmain()before persistence is reached. Expect that commit to drop out whenmastermerges intov11.Breaking changes
Web apps must serve two different files.
sqlite3.wasmanddrift_worker.jsreplacesql-wasm.jsandsql-wasm.wasminweb/, and thesql-wasm.jsscript tag comes out ofindex.html— nothing replaces it, since drift starts the worker itself at runtime. Not a new kind of work: the README already required hosting the sql.js pair.webUseExperimentalIndexedDbis removed in favour ofwebOptions. It namedDriftWebStorage.indexedDbIfSupported, which does not exist in the new backend, so a deprecated shim could only have silently done nothing. drift now probes the browser and picks the most reliable storage itself.The web cache is refilled once. Legacy sql.js storage cannot be read by the new backends. Its two
localStoragekeys are removed on connect to reclaim the space, sincelocalStorageis capped near 5 MB. Themoor_databasesIndexedDB store used by the old experimental flag is deliberately left alone — that name is shared with any other database an app opened through drift's legacy web backend.Two failure modes that needed handling
drift reports neither in a way a caller can act on, and both were traced through
wasm_setup.dart:drift_worker.jsis swallowed intoMissingBrowserFeature.workerError, and drift falls back to a database that answers every query and persists nothing. A smoke test cannot tell the difference. Now logged atSEVERE, naming the file and the URI it was looked up at.sqlite3.wasmletsWasmDatabase.openresolve successfully and fails on the first query instead — inside a worker, mentioning neither the file nor where it was sought, at whatever DAO call happens first. Opening is therefore forced with oneSELECT 1so the failure lands atconnectwith both.Because that eager open can also fail on a cache damaged by a killed tab, an unopenable database is discarded and reopened once before giving up. The local database is only a cache, so this costs one extra sync rather than blocking sign-in — while a genuine setup problem still fails loudly, because it fails the retry the same way.
What each browser actually gets
Which storage drift can use depends on cross-origin isolation, because
opfsLocksneedsSharedArrayBuffer. Everything works either way, so the headers are documented as a recommendation rather than a requirement:Testing
Every judgement call — which log level a storage mode deserves, what each message says — lives in
web_options.dartandweb_storage_diagnostics.dart, which are free of web-only imports and take plain strings rather than drift's enums. That is deliberate:package:drift/wasm.dartimportsdart:js_interop, so a signature namingWasmStorageImplementationcould not be imported from a VM test at all. Both files are at 100% coverage; package coverage is 97.88% against the 95 gate.web_db.dartkeeps// coverage:ignore-file— the browser probe cannot run underflutter test, andflutter test --coveragedoes not support--platform, so browser tests would contribute nothing to the lcov CI reads.Verified by hand, since no CI job builds web:
flutter build webandflutter build web --wasmfor the persistence example;flutter build webforsample_appand the UI example.sharedIndexedDb, no errors, full write path. Persistence proven across restarts — cold runcids=0, thencids=1with 15 cached messages read back, 2.1 MB of IndexedDB on disk.opfsLocksatINFOwith no error, confirming the with-headers row above.opfsLocksdatabase with an injected first-attempt failure — warning logged, database discarded, second open succeeded, ~27 ms, no throw.sample_appboots and renders fully under dart2wasm with the cherry-picked fixes in place.Not verified: Firefox and Safari, so their rows remain reasoned from drift's probe conditions rather than observed.
🤖 Generated with Claude Code