Repository navigation
Let a host cancel all running JavaScript - #33
Conversation
sqlitejs_cancel_all() stops the JavaScript running in every connection: each call fails with an "interrupted" error, even code that catches every exception. JavaScript that begins afterwards runs normally. Android cannot otherwise stop a runaway function: it cancels a query with a progress handler, which SQLite never calls while a function runs, and Room 2.7 and later pass no cancellation signal at all. A global epoch, read when the outermost call into JavaScript begins and compared by the existing interrupt handler, makes the cancel reach every runtime without a list of connections. The interrupt handler is now installed even when sqlite3_is_interrupted() is unavailable (SQLite before 3.41), so cancelling still works there. When the cancel stops a call, the handler also interrupts that call's connection with sqlite3_interrupt(), so its statement stops too. A collation cannot report an error: without this, a sort went on calling it, each call beginning after the cancel, and never ended. Add to the Cancel all test a runaway collation that one cancel stops; it hung without this. The test's worker cancels only once the JavaScript under test has called started(), an SQL function, since a cancel that came first would leave the JavaScript running. If that has not happened within 10 seconds, the worker fails the test with a message instead of cancelling. Like the Interrupt test, a watchdog ends the program if the JavaScript cannot be stopped. Only a call into JavaScript is cancelled, not the JavaScript a connection runs while it is set up: a cancel then made the connection fail to load the extension (SQLITE_NOMEM), and an app's query on a new pool connection failed. The Cancel all test now opens 200 connections while another thread cancels without pause; without this, the first failed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw
Since 1.4.2 every result set still open when the outermost call into JavaScript returns is ended there, which runs the final of each window or aggregate function its statement was computing. Each final enters JavaScript as a new outermost call, so js_enter() took the cancel epoch of that moment and a cancel during the call no longer stopped it. On SQLite 3.41 and later the interrupt the cancel left on the connection does; before 3.41 nothing did, and a final that never ends hung the statement for good. The finals that ending a call's result sets runs now keep that call's epoch, so a cancel during the call, or during the ending, stops each of them at QuickJS's next interrupt check: one that never ends is stopped and the statement fails with an interrupt, though a short final can still run to its end. Reading on from those result sets throws "interrupted", as after sqlite3_interrupt() on 3.41 and later, rather than the message for a result set that outlived its call, whether the cancel came before the call returned or while its result sets were ended. js_end_rowsets() marks the latter after its loop, by the pass that ended them, since a final it runs can free result sets of the list. Before 3.41, an sqlite3_interrupt() that no cancel came with still reads as the result set outliving its call: nothing shows it. Test: Cancel all now covers both cases on any SQLite. A call that returns normally keeps two result sets of window functions whose finals never end, and the first final starts the cancel: both finals stop and both result sets read on as "interrupted". And a cancel that lands as a call returns, from SQL inside it, stops the long final that ending its result set runs, so the call fails and the final records nothing. Without this change the first fails with the "closed" message, and the second returns OK with the final run to its end. Upstream's suite passes 17 of 17, also under ASan, LeakSanitizer and UBSan. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw
The DLL exports only what the generated .def file lists, so the test
program could not link sqlitejs_cancel_all() on Windows ("undefined
reference" in the windows-x86_64 job). The .def file now lists it with
the other public functions; it is the only export list in the project.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw
- js_leave no longer tests the cancel epoch: the block after the loop in js_end_rowsets already marks a cancelled call's result sets interrupted. - Say that only JavaScript the extension runs while it initialises a connection (globaljs_init, depth 0) is exempt from a cancel; stored functions loaded by js_init_table run as a call and can be cancelled. - Initialise rowset end_pass to 0 where a rowset is created. - Document sqlitejs_cancel_all in sqlitejs.h. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw
Patch 0002 (rebuilt from upstream PR sqliteai/sqlite-js#33) now: - drops the redundant cancelled term from js_leave: the block after the loop in js_end_rowsets marks a cancelled call's result sets interrupted; - says only JavaScript the extension runs while it initialises a connection is exempt from a cancel, not js_init_table's stored functions; - initialises a rowset's end_pass where it is created; - documents sqlitejs_cancel_all in sqlitejs.h. upstream/ is regenerated with vendor.sh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw
|
Merged, and released in 1.4.3. Thank you — Your four commits (c52018c…f0f6216) went in unmodified through #34: the merge commit on #34 adds two commits on top. One is a gap I found while testing the cancel against a connection pool, which is the case this feature exists for: the exemption covered the JavaScript run while the extension loads, but the functions Loading a stored function is now setup too: The other commit documents in I ran your branch against |
Upstream merged sqliteai/sqlite-js#33, our patches 0001 (cancel) and 0002 (a cancelled call's finals and result sets) unmodified, and released them in 1.4.3 with one fix of the maintainer's: a cancel no longer stops a stored function's code while js_init_table(1) loads it, as it already spared the JavaScript that loads the extension. Before, a cancel in flight left the connection without that function. Only that call is exempt: JavaScript the code runs through SQL is still cancelled. - upstream/ is plain 1.4.3; patches/ is gone. vendor.sh and test.sh apply any patches/ there may be later (nullglob). - The glue still loads a stored function up to three times, for the case 1.4.3 still cancels. glue_test.c: new test 20 checks that a cancel during a function's own load code does not stop it (fails on 1.4.2 with the two patches); the w and h tests now cancel JavaScript that the load code runs through SQL (they fail with one try). - README, SqliteJs.cancelAll() javadoc: a cancel interrupts the connection, so any other statement in progress on it fails too, and it spares loading stored functions. The README now says the 1.4.2 build passed on the phone and the 1.4.3 build has not run on one yet. test.sh: upstream's suite 17 of 17; glue test 50 of 50 normally and as on SQLite 3.40, and the same under ASan, LeakSanitizer and UBSan. vendor.sh --check passes. NDK 29 builds for both ABIs: no warnings, the same four exports, version string 1.4.3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw
|
Thank you for merging this, for releasing it in 1.4.3, and for the fix in #34. Exempting the Our Android build now vendors 1.4.3 with no local patches (in review), and its tests pass against the release, including a new one that checks a cancel no longer stops a stored function's own load code. This reply originated with AI: Claude, Anthropic's AI model, wrote it for App Dev For All. Contact: davidschachter@appdevforall.org |
* ADFA-7082: Vendor sqlite-js 1.3.7 with our patches sqlite-js 1.3.7 (Apache-2.0, with QuickJS/quickjs-ng amalgamated in libs/quickjs.c, MIT), only the files the AAR build uses, with patches/ applied: 0001-0008 are the Termux package's (terminal-packages packages/sqlite-js), 0009 makes each call measure its own thread's stack, and 0010 adds sqlitejs_cancel_all(). 0001-0009 are sent upstream as sqliteai/sqlite-js #11-#19. vendor.sh rebuilds upstream/ and the license notices from the release (checked against its SHA-256) and patches/, or with --check tells whether they match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Build the sqlite-js AAR for apps in subprojects:sqlite-js The AAR the JavaScript-in-SQLite project template depends on: libsqlitejs.so for arm64-v8a, armeabi-v7a and x86_64 (minSdk 26), and SqliteJs: cancelAll(), libraryPath(), ENTRY_POINT (loads the functions stored in js_functions on every connection) and ENTRY_POINT_FOR_DEFINING. sqlitejs_android.c, ours, holds the JNI and that entry point; the version script exports only the entry points and JNI_OnLoad. Nothing in CoGo depends on the module. AndroidModuleConf leaves it without the v7/v8 flavors, like :plugin-api, since one AAR serves every ABI, and without core-library desugaring, like :logsender, so apps need not enable it. Tested in a trial app (requery 3.49.0, Room 2.8.4) on a Galaxy Note20 Ultra as a 64-bit and a 32-bit process: scalar and aggregate from DAOs, deep recursion on pool threads, 32 parallel queries under WAL, and a runaway query stopped by cancelAll() within 5 ms. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Release the sqlite-js AAR to sqlite-js-latest A manual workflow, like release-plugin-api.yml: checks the vendored sqlite-js, builds the AAR from main, and replaces the moving sqlite-js-latest release, from which dev-assets fetches it into the template that uses it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Build the sqlite-js AAR for arm64-v8a and armeabi-v7a only Apps built with Code on the Go run on ARM phones, so the AAR no longer carries x86_64. ADR 0005 says so. CLAUDE.md's ABI section now names the two modules without v7/v8 flavors, as ARCHITECTURE.md and ADR 0005 do (review of PR #2244, F07). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Run no stored code by default; test sqlite-js on pull requests From the reviews of PR #2244 (independent F02, F03, F05, F06; CodeRabbit): - ENTRY_POINT (sqlite3_sqlitejs_init, also the name SQLite derives from libsqlitejs.so) now loads sqlite-js alone. The old entry point loaded the functions stored in js_functions, which runs their code, so any database file the app opened ran its stored JavaScript at once. Loading them is now ENTRY_POINT_FOR_TRUSTED_DATABASE (sqlite3_sqlitejs_trusted_init), for databases the app made. ENTRY_POINT_FOR_DEFINING is gone: ENTRY_POINT does that job. - The trusted entry point loads each stored function on its own, so one that fails is logged and skipped instead of dropping those after it. - cancelAll()'s Javadoc and the README say it stops JavaScript in every query, so call it only for a known runaway and repeat it until the query ends. - test.sh runs upstream's test suite with patches/, and glue_test.c, the first test of sqlitejs_android.c. test-sqlite-js.yml runs vendor.sh --check and test.sh on pull requests for this module. glue_test.c passes 14 of 14; against the old glue 5 fail (stored functions loaded by ENTRY_POINT, functions after a broken one dropped). The AAR builds and exports the two entry points, sqlite3_js_init and JNI_OnLoad. Not run on a device. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Update sqlite-js-latest in place, out of latest-release selection release-sqlite-js.yml deleted the release and its tag before making the new one, so a failed upload left dev-assets nothing to fetch. It now uploads the new assets over the old ones, putting the old ones back if that fails, then moves the tag and the notes. The release is created and kept with --latest=false, so it never becomes the repository's latest release (CodeRabbit, PR #2244). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Sort the QuickJS copyright lines the same on every machine vendor.sh built LICENSE.quickjs with sort -u in the machine's locale, so the new test-sqlite-js workflow's vendor.sh --check failed on the runner: en_US.UTF-8 and C order "2017" and "2017-2018" differently. It now sorts with LC_ALL=C, and upstream/LICENSE.quickjs and the META-INF notice are regenerated; only the order of their copyright lines changes. --check passes under C, C.UTF-8 and en_US.UTF-8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Log stored functions that cannot be listed; guard the tag move From CodeRabbit's review of the previous commits: - The trusted entry point returned silently when it could not list the stored functions (busy, out of memory, an odd table), opening the connection with some or none of them. Each failure is now logged; the connection still opens. It first looks js_functions up in sqlite_master, so a database without stored functions makes SQLite log no "no such table" error on each connection, as it did. - release-sqlite-js.yml moves the tag and edits the notes in the same guarded sequence as the upload. If any step fails, the previous assets and tag are put back. glue_test.c passes 16 of 16; the two new tests fail on the previous glue. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Vendor the reviewed sqlite-js patches Patches 0002, 0003, 0004 and 0010 are regenerated after three reviews: the independent reviews of terminal-packages #6 and of this PR, and CodeRabbit's review of this PR. - 0002: garbage is collected before the memory limit fills. Running out of memory through small allocations reports "out of memory". - 0003: the README says TEMP views and triggers can call the js_* functions. - 0004: db.exec throws for a buffer it cannot read instead of binding NULL. A typed array's own offset and length are used, not properties a script defines. - 0010: a cancel also interrupts the cancelled call's statement, so a runaway JavaScript collation stops. Before, nothing could stop one. The other patches change only in their headers and line numbers. upstream/ is regenerated with vendor.sh, and vendor.sh --check passes. SqliteJs.cancelAll() and the README say a cancel interrupts the statement too. Tested: - test.sh: upstream's suite passes 16 of 16, and the glue test 16 of 16. - In the work repo, the same suite passes under ASan and UBSan with leak checking, and each new test fails without its fix. - The AAR builds for arm64-v8a and armeabi-v7a. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Put back both the assets and the tag if a release update fails From CodeRabbit's review: under bash -e, a failed restore of the previous assets stopped the rollback before it put the tag back, so the tag could name the new build while the release held the old one. Each restore is now attempted, reported if it fails, and the step still fails. Checked with gh stubbed out: when the update fails and the asset restore fails, the tag restore still runs and the step exits 1. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Find js_functions in any case; sync the cancel test; doc caveats From CodeRabbit's review of the vendored patches: - The trusted entry point looks up js_functions with COLLATE NOCASE, as SQLite resolves a table name, so a table named JS_FUNCTIONS loads. A new glue test, now 18 of 18, fails without it. - Patch 0010's Cancel all test cancels once the JavaScript under test has called started(), an SQL function, instead of after a fixed 200 ms. A slow start can no longer end the test at its alarm. It still hangs without the fix it tests. - SqliteJs.cancelAll() and the README now say two more things. An interrupted INSERT, UPDATE or DELETE in an explicit transaction rolls the whole transaction back. And a cancel reaches SQL that JavaScript runs through db.exec only when that SQL returns to JavaScript. Tested: - test.sh: upstream's suite passes 16 of 16, also under ASan and UBSan in the work repo, and the glue test 18 of 18. - vendor.sh --check passes. - The AAR builds with the new lookup for arm64-v8a and armeabi-v7a. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Let a re-run repair a release left without its assets A run that stops during `gh release upload --clobber` can leave an asset missing. The next run then failed at the backup download ("no assets match the file pattern") before it uploaded anything, so re-running could not repair the release. The step now backs up only the assets that exist and puts the previous ones back only when it has both. Tested with a stub gh over a fake release: both assets missing now ends with the new pair and the tag moved (before: exit 1 at the download); the other five cases are unchanged. Checked against gh 2.46: download fails only when no pattern matches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Vendor sqlite-js main with our changes merged, one patch left Upstream merged our patches 0001 to 0009 (sqliteai/sqlite-js #11 to #19) on 2026-11-12, with fixes of its own. One fixes a bug in our 0002: a memory limit lowered under memory that is garbage left the connection out of memory until the limit was raised again (ef06558). upstream/ is now sqlite-js main at c8ae85a, the same source as the Termux package 1.3.7-p20261112, with one patch: 0001, sqlitejs_cancel_all(), our old 0010 rebased. Its test now fails with a message when the JavaScript under test has not started within 10 seconds, instead of cancelling anyway, and uses upstream's watchdog thread instead of alarm(). The patch is offered upstream in sqliteai/sqlite-js issue 20. vendor.sh, test.sh and test-sqlite-js.yml take the commit (COMMIT, URL) instead of the release tag. vendor.sh --check passes; test.sh passes: upstream's suite 16 of 16 with Cancel all, glue_test 18 of 18. The upstream suite with the patch also passes under ASan + UBSan; without the patch's sqlite3_interrupt() its collation case runs until the watchdog, and with started() never called the worker fails with its message. The native library builds with NDK 29 for arm64-v8a and armeabi-v7a, with no warnings in sqlitejs.c or sqlitejs_android.c, and exports the same four symbols. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Vendor the sqlite-js 1.4.1 release Upstream released 1.4.0 and 1.4.1 on 2026-11-12, after c8ae85a. Both contain it, and 1.4.1 differs from it in src/ and libs/ only in SQLITE_JS_VERSION, so js_version() now reports 1.4.1. vendor.sh, test.sh and test-sqlite-js.yml take the release's archive again (VERSION, SHA256, URL). Patch 0001 is unchanged but for its context. The scripts now take the archive's one top directory whatever its name (tar --strip-components=1). They named it after COMMIT, so the README's "point URL at its tag again" would have failed: a tag's archive unpacks to sqlite-js-<tag> (independent review). The README says how to vendor a release, and a commit between releases. vendor.sh --check passes on the 1.4.1 archive (SHA-256 d45d394f...3078). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Let a connection open, and its stored functions load, during a cancel A cancelAll() that landed while another thread opened a connection made that connection fail to load sqlite-js (SQLITE_NOMEM), so a Room query on a new pool connection failed (independent review). Setting up the runtime runs JavaScript of its own, which the interrupt handler stopped. Patch 0001 now cancels only JavaScript inside a call (js_enter), and its Cancel all test opens 200 connections while another thread cancels without pause: the first one failed before. A cancel that lands while the trusted entry point loads a stored function stops that function's code the same way, and the function was then missing from the connection for good. load_stored_function now tries such a load again, up to three tries in all; its stored code runs again each time. glue_test.c stores a function whose code cancels the first time it loads: it loads on the second try, and with one try it is missing. The README and SqliteJs's Javadoc say both. test.sh passes: upstream's suite 16 of 16, glue_test 22 of 22. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Patch two stack-bound crashes and two garbage cases 0002, after CodeRabbit's review: code that called itself without end crashed the process instead of failing with "Maximum call stack size exceeded" in two cases. - On a thread with less than 64 KB of stack left, js_enter kept the connection's 1 MB size rather than the room it measured. It now keeps any measurement, and 1 rather than 0, which turns the check off. - On the main thread, RLIMIT_STACK was counted from where the first measurement was taken. Loaded 2 MB deep, the extension allowed 10.2 MB of an 8 MB stack. It is now counted from the stack's top, as the kernel counts it. 0003, after the independent review of terminal-packages #6: garbage could leave an allocation that fits in the memory limit failing on every try. After 40 MB of cyclic garbage, new ArrayBuffer(30 MB) failed under the 64 MB default. And after a limit was lowered to 32 MB above the garbage, new ArrayBuffer(16 MB) failed. The extension now collects after any "out of memory" error and whenever a limit is set. Each patch adds tests to upstream's suite that fail without it. The suite passes 16 of 16 with all three patches, also under ASan and UBSan, and test.sh's glue_test 22 of 22. The NDK 29 build for arm64-v8a and armeabi-v7a has no warnings outside QuickJS and exports the same four symbols. Both fixes are offered upstream. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Let a re-run replace a release left as a draft gh release create makes a draft and publishes it once its assets are uploaded. A first run that stopped during that upload left a draft: the next run found it, skipped the create, and failed fetching its tag, which a draft does not have yet. Every re-run then failed (independent review). The step now deletes a draft release, which dev-assets cannot fetch from, and creates the release again. Tested with a stub gh over a fake release, eight cases. The draft case failed with a 404 before, and now ends with the new assets and tag. The other seven are unchanged: no release, normal, upload fails, tag update fails, both assets missing, one missing, and upload fails with both missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * style: spotless reformat, no functional change Spotless wraps the cancelAll() Javadoc paragraph, which the previous commits lengthened. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: End the result sets an interrupted call leaves open CodeRabbit: SQLite keeps an interrupt in effect while any statement of the connection runs, and a db.exec result set that JavaScript keeps open is a running statement. After cancelAll() stopped JavaScript that kept one in a global, every later statement on that connection failed with "interrupted", until the connection closed. A stored function whose code kept one could not load on any of its three tries. sqlite3_interrupt() does the same, without our cancel patch. Patch 0004 lists each connection's result sets. When the outermost call into JavaScript returns on an interrupted connection, it finalizes those still open. Reading on from one that JavaScript kept throws "interrupted". Reproduced on the host, with a cancel and with sqlite3_interrupt(): every later statement failed before, and all run after. Upstream's Interrupt test interrupts code that keeps a result set, in a global and as garbage, and fails without the patch. glue_test.c stores a function whose code keeps one and cancels the first time it loads. It loads on its second try, and the connection runs SQL afterwards; without the patch, those three checks fail. test.sh passes: upstream's suite 16 of 16, also under ASan and UBSan, and glue_test 26 of 26. The NDK 29 build for arm64-v8a and armeabi-v7a has no warnings outside QuickJS and exports the same four symbols. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Make ending result sets safe when it runs JavaScript, and on old SQLite CodeRabbit, on 0004: - 0005: finalizing a result set that is computing a JavaScript window function runs the function's final, whose js_leave() ended the result sets again on the interrupted connection, finalizing the same statement twice. Under ASan that is a use after free in sqlite3VdbeMemFinalize, reproduced on the host. Each statement is now taken from its result set before it is finalized, the list is walked again from its head after each one, and a nested call does nothing. Upstream's Interrupt test now keeps a result set computing a window function; it fails under ASan without 0005. - 0006: on SQLite before 3.41, which has no sqlite3_is_interrupted(), 0004 never ran, though a cancel still interrupts the connection. The interrupt handler now records the interrupt it sends. Built to report SQLite 3.40, a cancel of JavaScript that keeps a result set left every later statement failing before 0006, and not after. test.sh passes: upstream's suite 16 of 16, also under ASan and UBSan, and glue_test 26 of 26. The NDK 29 build for both ABIs has no warnings outside QuickJS and exports the same four symbols. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Stop what ending result sets runs; count the interrupts db.exec sees CodeRabbit, two more cases on SQLite before 3.41, both in patch 0007: - Ending result sets after a cancel can run a window or aggregate function's final, which begins after the cancel, so nothing stopped it: a final that never ended hung the cancelled query. The interrupt handler now stops any JavaScript while result sets are being ended. - A direct sqlite3_interrupt() that reached a db.exec statement's SQL, with another result set kept open, went unseen. A result set whose step returns SQLITE_INTERRUPT now records it. Checked with the patched source built to report SQLite 3.40: the endless final hung before and ends after, and a direct interrupt during db.exec's SQL, caught or not, left later statements failing before and not after. Still unseen before 3.41, and said in the README: an interrupt that never reaches the SQL that JavaScript runs. Android's cancelling never calls sqlite3_interrupt(), and cancelAll() is covered. test.sh passes: upstream's suite 16 of 16, also under ASan and UBSan, and glue_test 26 of 26. The NDK 29 build for both ABIs has no warnings outside QuickJS and exports the same four symbols. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Answer the independent review of patches 0004 to 0007 - 0009: a result set's SQLITE_INTERRUPT now counts as an interrupt only before SQLite 3.41. A progress handler that stops a statement, as Android's cancelling does, gives the same code and interrupts nothing else, yet it ended the result sets earlier calls kept, on any SQLite. - 0008: the Interrupt test's "garbage" result set was freed by reference counting, so the case passed without 0004. It is now a reference cycle, and fails on 1.4.1 without 0004. Also pushed to sqliteai/sqlite-js#24. - Tests for 0006, 0007 and 0009, which had none: glue_test.c cancels JavaScript that keeps a result set, then a kept window function whose final never ends, then has a progress handler stop a statement. test.sh runs it twice, the second time with sqlitejs.c built to report SQLite 3.40. Without 0009 the progress-handler check fails; without 0007 the 3.40 run hangs until the test's 30 s alarm; without 0006 seven checks fail in the 3.40 run. - README: a cancel ends every result set open on the connection, not only the stopped call's. And on any SQLite, an sqlite3_interrupt() that lands while no JavaScript runs leaves the connection failing while JavaScript keeps a result set, so finish result sets before a call returns. Reproduced on 3.49: every later statement fails, and sqlite3_close returns SQLITE_BUSY. test.sh passes: upstream's suite 16 of 16, also under ASan and UBSan, and glue_test 34 of 34 in both runs. The NDK 29 build for both ABIs has no warnings outside QuickJS and exports the same four symbols. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Test a direct interrupt during db.exec with a result set kept CodeRabbit: the case that 0007's record of a step's SQLITE_INTERRUPT is for had no automated test. glue_test.c now keeps a result set, has another thread call sqlite3_interrupt() while db.exec runs a query that never ends otherwise, and checks that the connection runs SQL and that the kept result set was ended. In the run as SQLite 3.40, those checks fail without the record. The checks that a kept result set was ended now tell an ended one (caught in JavaScript: "ended: interrupted") from a connection that is still interrupted, whose error is the same word. test.sh passes: upstream's suite 16 of 16, and glue_test 37 of 37 in both runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Vendor the sqlite-js 1.4.2 release Upstream released 1.4.2 with our stack, garbage-collection and result-set patches merged (sqliteai/sqlite-js #22, with two maintainer commits, #23 and #24) and with a fix for the use after free we reported (#25). That fix also changes behaviour: a db.exec result set no longer outlives the call into JavaScript that created it, and reading one afterwards throws "The result set was closed when the call into JavaScript that created it returned". So patches 0002 to 0009 go: - 0002 to 0005 and 0008 are upstream, by patch-id identical except for 0002, which has the maintainer's two commits on top. - 0006, 0007 (its second half) and 0009 only told interrupts apart for result sets kept across calls, which 1.4.2 no longer allows. 0001 (sqlitejs_cancel_all, issue 20) is rebased: the same changes, with new context in the struct. A new 0002 keeps what 0006 and the first half of 0007 did for a cancel on SQLite before 3.41: ending a cancelled call's result sets runs the finals of the window and aggregate functions they were computing, each a call that begins after the cancel, which before 3.41 nothing else stops; and those result sets read on as "interrupted". It is narrowed to a cancelled call, since in 1.4.2 result sets are also ended on every normal return. The next commit updates the tests for 1.4.2. vendor.sh --check passes on the 1.4.2 archive (SHA-256 1aa4c566...cf24). The libs/ and license files are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Test sqlite-js 1.4.2's result-set rule and patch 0002 glue_test.c, for 1.4.2: - The progress-handler case kept a result set across calls, which 1.4.2 no longer allows. It now checks, within one call, that a statement a progress handler stopped (as Android's cancelling does) leaves the call's other result set readable, on any SQLite, and that the connection runs SQL afterwards. - A direct sqlite3_interrupt() before SQLite 3.41 shows nowhere, so its kept result set reads as one that outlived its call; on 3.41 and later, as interrupted. - The cleanup that read a kept result set after its call is gone: it now throws. test.sh and the README name patch 0002 for the run as on SQLite 3.40, and the README lists the two patches, states the 1.4.2 rule, and says the 1.4.2 build has not been run on a phone yet. test.sh: upstream's suite 17 of 17 (its 16 and 0001's Cancel all), and glue_test 37 of 37 both normally and as on SQLite 3.40. Without patch 0002, the normal run still passes 37 of 37, and the 3.40 run fails as expected: the kept result set after a cancel reads "The result set was closed ..." rather than "interrupted", and the window function whose final never ends hangs until the 30 s alarm stops the test. Both ABIs build with NDK 29, and libsqlitejs.so exports only JNI_OnLoad and the three entry points. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Keep a call's cancel for the finals its result sets run From the independent review of 3a5cb53 and 464e6e7 (no critical, high or medium findings; F01 to F05 low or nits). Patch 0002 told a cancelled call apart only once, in js_leave(), and each final that ending its result sets ran entered JavaScript as a new call with a new cancel epoch. So, before SQLite 3.41: - a cancel that landed while the result sets were ended stopped only the final then running, and the next final that never ended hung the statement (F01); - a cancel that landed just before the call returned let the call succeed with its finals skipped, on any SQLite (F02). Patch 0002 now keeps the call's epoch while its result sets are ended (js_enter), so a cancel during the call or the ending stops every final, and a late cancel fails the statement with an interrupt. The ending_cancelled flag and its interrupt-handler check are gone; the patch is 8 lines. upstream/ is regenerated with vendor.sh. glue_test.c, 44 tests (was 37): - 31: a cancelled call that keeps two result sets whose window finals never end (F03's case); - 33-34: the cancel lands during the first final of a call that returned normally (F01); - 36-37: a cancel that lands as the call returns fails it (F02); - 43: after a progress handler's stop, the kept result set reads as outliving its call, not as interrupted (F04; it passes without the fix too, as a guard against counting such a stop as an interrupt). test.sh: upstream's suite 17 of 17; the glue test 44 of 44 normally and as on SQLite 3.40. With the previous 0002, test 37 fails in the normal run and the 3.40 run hangs at test 34; with 0001 alone, test 37 fails, and the 3.40 run fails test 29 and hangs at test 31. README: what 0002 does now, and what the glue test covers (F05). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Say what a late cancel does to a call's finals From the independent review of 5b9b062 (no code defects; tests catch what they're named for and are not flaky in 120 runs on 2 cores and 16 on 1; two low findings and a nit): - F01: "those result sets read on as interrupted" holds only when the cancel came before the call returned. js_leave() decides that once, before ending them, so a cancel that lands while they are ended stops their finals but leaves the "closed" message. - F02: a cancel stops each final at QuickJS's next interrupt check. A final that never ends is stopped and the statement fails, but a short final can still run to its end, so "rather than skipping its finals" overstated it. The patch 0002 header and README say so; 5b9b062's message overstated both in the same way. Patch 0002's code is unchanged. - F03: test 36's final records that it ran, and a new test 38 checks that the late cancel stopped it before it did; test 37 is renamed to say its final is a long one. test.sh: upstream's suite 17 of 17; the glue test 45 of 45 normally and as on SQLite 3.40. With 0001 alone, tests 37 and 38 fail (the final ran to its end) and the 3.40 run fails 29 and hangs at 31; with the previous 0002, test 37 fails and the 3.40 run hangs at 34. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Read result sets as interrupted when the cancel lands while they are ended From CodeRabbit's review of 5b5cd20 (4570522172), the same gap the independent review noted as F01: js_leave() decides once, before ending a call's result sets, whether they read on as "interrupted". A cancel that landed while their finals ran stopped the finals but left the "closed when the call ... returned" message. Patch 0002 now has js_end_rowsets() number its passes and mark each result set with the pass that ended it. After the loop, where nothing runs JavaScript and so no final can have freed a listed result set, it marks that pass's result sets interrupted if the cancel epoch has moved. The README and the patch header say the message is "interrupted" whether the cancel came before the call returned or during the ending. glue_test.c: new test 36 reads both result sets after the cancel that lands during the ending, and expects "interrupted". test.sh: upstream's suite 17 of 17; the glue test 46 of 46 normally and as on SQLite 3.40, and the same under AddressSanitizer and UBSan. With the previous 0002, test 36 fails in both runs with the "closed" message. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Address review of the sqlite-js cancel patches Patch 0002 (rebuilt from upstream PR sqliteai/sqlite-js#33) now: - drops the redundant cancelled term from js_leave: the block after the loop in js_end_rowsets marks a cancelled call's result sets interrupted; - says only JavaScript the extension runs while it initialises a connection is exempt from a cancel, not js_init_table's stored functions; - initialises a rowset's end_pass where it is created; - documents sqlitejs_cancel_all in sqlitejs.h. upstream/ is regenerated with vendor.sh. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Vendor the sqlite-js 1.4.3 release, with no patches Upstream merged sqliteai/sqlite-js#33, our patches 0001 (cancel) and 0002 (a cancelled call's finals and result sets) unmodified, and released them in 1.4.3 with one fix of the maintainer's: a cancel no longer stops a stored function's code while js_init_table(1) loads it, as it already spared the JavaScript that loads the extension. Before, a cancel in flight left the connection without that function. Only that call is exempt: JavaScript the code runs through SQL is still cancelled. - upstream/ is plain 1.4.3; patches/ is gone. vendor.sh and test.sh apply any patches/ there may be later (nullglob). - The glue still loads a stored function up to three times, for the case 1.4.3 still cancels. glue_test.c: new test 20 checks that a cancel during a function's own load code does not stop it (fails on 1.4.2 with the two patches); the w and h tests now cancel JavaScript that the load code runs through SQL (they fail with one try). - README, SqliteJs.cancelAll() javadoc: a cancel interrupts the connection, so any other statement in progress on it fails too, and it spares loading stored functions. The README now says the 1.4.2 build passed on the phone and the 1.4.3 build has not run on one yet. test.sh: upstream's suite 17 of 17; glue test 50 of 50 normally and as on SQLite 3.40, and the same under ASan, LeakSanitizer and UBSan. vendor.sh --check passes. NDK 29 builds for both ABIs: no warnings, the same four exports, version string 1.4.3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * style: spotless rewrap of the cancelAll() javadoc, no functional change Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Answer the independent review of the 1.4.3 move No correctness defects; two nits: - F01: two glue-test comments still credited "patch 0002" for what sqlite-js 1.4.3 now does. - F02: the v test assumed cancel_calls was still 0; it now resets it, as the w and h tests do. Glue test 50 of 50 normally and as on SQLite 3.40. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Record the phone run of the 1.4.3 build On the Note20, with an AAR built from 0c2e603: the trial app passes 7 of 7 as a 64-bit and as a 32-bit process, on a new and an existing database, and the template (dev-assets #79) stops its runaway query 11 to 12 ms after Cancel in its 64-bit and 32-bit debug builds and its release build. The README said the 1.4.3 build had not run on a phone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Answer CodeRabbit's full review of the sqlite-js module - release-sqlite-js.yml: when the release is missing but its tag is left behind, move the tag to the commit being built before creating the release, since gh release create reuses an existing tag and ignores --target. - glue_test.c: a worker thread that cannot start stops the run (exit 2), as a database that cannot open does, instead of returning SQLITE_ERROR, which the cancel checks accepted as a pass. - test.sh: upstream's test main leaves its "Memory" (leak) check out of its exit code, so any FAILED line in its output fails the run too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw * ADFA-7082: Take only a 404 for a missing sqlite-js release or tag The release step took any failed lookup for a missing release, so a passing API error could move the live release's tag before gh release create failed, and a failed tag lookup skipped the move the last commit added. exists() now answers 1 only for HTTP 404 and stops the job on any other error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Closes #20. This is the patch that issue offers, rebased on
mainafter 1.4.2, plus a second commit that 1.4.2's result-set lifetime calls for.1. Let the host cancel all running JavaScript.
sqlitejs_cancel_all()stops the JavaScript running in every connection. Each call fails with "interrupted", even code that catches every exception, and JavaScript that begins afterwards runs normally.Android has no other way to stop a runaway function:
How it works:
js_init_tablerun as a call, so a cancel can stop them.2. End a cancelled call's result sets as interrupted, and stop their finals. Since 1.4.2, ending a call's result sets at its return runs the finals of the window or aggregate functions they were computing. Each final entered JavaScript as a new outermost call and took the epoch of that moment, so the cancel no longer reached it. On SQLite 3.41 and later, the interrupt the cancel left stops it; before 3.41 nothing did, and a final that never ends hung the statement.
The finals now keep the call's epoch, and those result sets read on as "interrupted" rather than "closed when the call ... returned":
js_end_rowsets()marks them after its loop, by the pass that ended them, since a final it runs can free result sets of the list; before SQLite 3.41 that is the only way they read so.3. Export
sqlitejs_cancel_allfrom the Windows DLL. The DLL exports only what the generated.deffile lists, so the test program could not link the new function on Windows; the.deflist, the only export list in the project, now includes it.4. Address review (an independent review; no defects found):
js_leaveis upstream's again, since the marking above covers a cancel before the return too; the setup exemption above is stated precisely;end_passis initialised; and the header documentssqlitejs_cancel_all.Tests. Cancel all covers:
Each check fails without its change; the two new ones fail on the first commit alone. The suite passes 17 of 17, also under ASan, LeakSanitizer and UBSan, and this pull request's CI passes on all 14 platforms. Code on the Go's Android library carries these two commits as patches on 1.4.2. A further test there runs the same cases as if on SQLite 3.40.
This pull request originates with AI: it was written by Claude (Anthropic's model) for App Dev For All. The human contact is David Schachter, davidschachter@appdevforall.org.