diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 62828449ad..b8e89f6acd 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -144,7 +144,7 @@ Local builds get `documentation.db` from dev-assets instead, which is updated by > **Persistence policy (authoritative):** new relational/queryable persistence uses **Room** (`@Entity` + DAO + `RoomDatabase` with explicit migrations, provided via Koin). Non-relational settings use the **filesystem/preferences (DataStore)**. **Raw SQLite is the exception, not the default** — see [ADR 0001](docs/adr/0001-prefer-room-for-persistence.md). > -> **Recent Projects** is the reference example of the default: `app/src/main/java/com/itsaky/androidide/roomData/recentproject/` (`RecentProjectRoomDatabase`, `@Database version = 4` with migrations 1→4; `RecentProjectDao`; the `RecentProject` `@Entity` → table `recent_project_table`). It's provided via Koin in `di/AppModule.kt` and consumed by `RecentProjectsViewModel`, `ProjectInfoBottomSheet`, and `ProjectCreationManager` directly, and by `MainActivity`/`EditorHandlerActivity` indirectly through `RecentProjectRepository` (`repositories/RecentProjectRepository.kt`) -- kept behind that interface, rather than injecting the DAO into those two Activities directly, per this section's own UI -> ViewModel -> Repository -> data source layering. +> **Recent Projects** is the reference example of the default: `app/src/main/java/com/itsaky/androidide/roomData/recentproject/` (`RecentProjectRoomDatabase`, `@Database version = 6` with migrations 1→6; the `DebuggerEndpoint` `@Entity` → table `debugger_endpoint_table` pairs an app with its project through a foreign key to `recent_project_table.location`; `RecentProjectDao`; the `RecentProject` `@Entity` → table `recent_project_table`). It's provided via Koin in `di/AppModule.kt` and consumed by `RecentProjectsViewModel`, `ProjectInfoBottomSheet`, and `ProjectCreationManager` directly, and by `MainActivity`/`EditorHandlerActivity` indirectly through `RecentProjectRepository` (`repositories/RecentProjectRepository.kt`) -- kept behind that interface, rather than injecting the DAO into those two Activities directly, per this section's own UI -> ViewModel -> Repository -> data source layering. > > **Raw SQLite is allowed only when** the database is prebuilt and opened read-only, the data is performance/allocation-critical and needs granular schema control, or the schema is shared across a process/component boundary. Current exceptions: symbol indexing (`lsp/indexing/SQLiteIndex.kt`), tooltips (`idetooltips/ToolTipManager.kt`), in-app/plugin help (`plugin-manager/.../documentation/PluginDocumentationManager.kt` — the one *writer*, inserting plugin-contributed rows into a schema owned elsewhere), the shared compression-dictionary loader (`common/.../utils/DocumentationCompression.kt`), and documentation serving (`common/.../documentation/DocumentationContentSource.kt`, the one read pipeline behind both the in-process WebView transport and `app/.../localWebServer/WebServer.kt`). The `androidx.room:*` strings in `editor`'s `GroovyAutoComplete` are autocomplete suggestions for the *user's* code, not CoGo persistence. > diff --git a/app/src/main/AndroidManifest.xml b/app/src/main/AndroidManifest.xml index 134ac9e1f1..f42ebf7b95 100755 --- a/app/src/main/AndroidManifest.xml +++ b/app/src/main/AndroidManifest.xml @@ -360,6 +360,18 @@ + + + + + + + McpHttp -> McpProtocol -> DatabaseTools -> Debug - `engine/`: `DebugEngine` is what the server needs from the extension, `NativeDebugEngine` implements it over JNI, and `EventDecoder` turns the event buffer's bytes into `Event`s. - `mcp/`: `McpHttp` is the HTTP/1.1 listener, `McpProtocol` is JSON-RPC and the MCP session handling, `DatabaseTools` is the tools: `list_databases`, `list_connections`, `get_schema`, `query`, `explain_query_plan`, `get_stats`, `integrity_check`, `get_events`. - `DebuggerStartup` is a `ContentProvider` with the highest `initOrder`, declared in the AAR's manifest. It runs before the app's own providers and `Application.onCreate`, so the extension is registered before the app opens a database. It installs the extension synchronously in `onCreate` (so an app that opens its database in `Application.onCreate` is still seen) and starts the server on a background thread. Everything after the debuggable check is wrapped in `catch (Throwable)`: a missing requery library (`UnsatisfiedLinkError`) means the debugger is disabled and logged, never a crash of the host app. +- `EndpointBroadcast` runs once the server listens, on every process start (the port changes). It sends an explicit broadcast, action `com.itsaky.androidide.SQLITE_DEBUGGER_ENDPOINT`, to `com.itsaky.androidide` with `FLAG_INCLUDE_STOPPED_PACKAGES`, carrying the token, the port and a `PendingIntent` that identifies the app. The `PendingIntent` is immutable (on API 23 and later; the flag does not exist before) and wraps an intent that is never sent: Code on the Go reads its `creatorPackage`, which the framework sets and another app cannot forge, as the app's package name. The library manifest declares `` for Code on the Go. A failure to send is logged and never thrown. Before the first send it registers a receiver at run time for `org.appdevforall.sqlitedebugger.ANNOUNCE` (exported, as Code on the Go is another app; `RECEIVER_EXPORTED` on API 33 and later), and sends the same token and port again each time it receives one. Code on the Go sends that request when it pairs the app, because it drops a report from an app it has not paired: an app that was already running then, installed some other way or started before its project was opened, would otherwise stay unknown until its process restarted. The receiver is not in the manifest, so a request never starts an app that is not running, and is not authenticated (ADFA-7131 D8): the reply goes only to Code on the Go. +- On the Code on the Go side, `DebuggerEndpointReceiver` (exported) validates the token and port and resolves the sender, then `DebuggerEndpointRepository` stores them in `debugger_endpoint_table` of the Recent Projects database, keyed by application ID. A row exists only for an app Code on the Go installed or launched (`LaunchAppAction` and `ProjectHandlerActivity` record the pairing); a broadcast from any other app stores nothing. When `ProjectHandlerActivity` installs a build, it notes the PackageInstaller session ID with the project and the APK's application ID before the session is committed. `InstallationResultReceiver` takes the result app-wide, so a result that arrives while the editor is stopped still counts: on success it moves the app to that session's project (Quick Build's installs are never noted, so they never pair); on failure it drops that session's note, as `ApkInstaller`'s `onCommitFailed` does when staging or committing the session throws, since no result follows. Only the same session's result uses a note, however long the install prompt stayed open, so two projects with the same application ID, such as two made from one template, each pair their own installs. The post-install launch does not wait for the write; the announce request above brings the app's report once the row exists. An install Code on the Go did not start from a project (an APK opened from the file tree), or one whose result arrives after Code on the Go's process restarted (notes are kept in memory), is not paired at install; `LaunchAppAction` pairs it at the next launch. After each pairing, `DebuggerEndpointAnnounce` sends the app the announce request above. The row is deleted with its project (foreign key, ON DELETE CASCADE). ### The event record @@ -43,13 +45,12 @@ The layout of `sqlitedbg_event_record` lives in `sqlitedbg.h`. The JNI layer han - **Host C tests of the extension:** `subprojects/sqlite-debugger/test.sh [--update-fixture] [AMALGAMATION_TARBALL]`. It builds SQLite 3.49.1 from the amalgamation as requery builds it (`SQLITE_THREADSAFE=2`), checks its headers against `sqlite-js/upstream/libs`, and runs `src/test/c/sqlitedbg_test.c` under AddressSanitizer with UBSan, then under ThreadSanitizer. It needs gcc, curl, tar and sha256sum, and downloads the tarball unless you pass one. - **The fixture:** `src/test/resources/.../engine/events.bin` and `events.tsv` are written by the C test and read by `EventDecoderTest`. Without `--update-fixture`, `test.sh` checks the committed pair against the current record layout, so a layout change fails until you regenerate it and update `EventDecoder.kt`. - **JNI test:** when a JDK is on PATH, `test.sh` also builds `libsqlitedbg.so` as it ships (no `SQLITE_CORE`) against a requery-like `libsqlite3x.so` and runs `src/test/jni/JniQueryTest.java` through `NativeBridge`: paging, the total-bytes budget, per-cell truncation, bound types, one statement per call, the deadline, refused writes and transactions, and the rollback of a transaction left open. It uses Java stubs of `NativeBridge` and `DebugEngineException` instead of the Kotlin, so a JDK is all it needs; `test.sh` checks the native names match `NativeBridge.kt`. Otherwise it is skipped with a message. CI runs `test.sh` in `.github/workflows/test-sqlite-debugger.yml` for changes under this module. -- **JVM tests:** `flox activate -d flox/local -- ./gradlew :subprojects:sqlite-debugger:testDebugUnitTest`. They cover `EventDecoder`, `McpProtocol`, `McpHttp` and `DatabaseTools` (with `FakeDebugEngine`). +- **JVM tests:** `flox activate -d flox/local -- ./gradlew :subprojects:sqlite-debugger:testDebugUnitTest`. They cover `EventDecoder`, `McpProtocol`, `McpHttp`, `DatabaseTools` (with `FakeDebugEngine`) and `EndpointBroadcast`, its announce receiver included. `.github/workflows/test-sqlite-debugger.yml` runs `test.sh` (the C and JNI tests) on pull requests that touch the module or the SQLite headers. No workflow runs the JVM tests. ## Not done yet -- Broadcasting the port and token to Code on the Go (ADFA-7137). `DebuggerStartup` only keeps them in `DebuggerStartup.port` and `endpointToken` and logs the port, never the token. - Gradle plugin wiring and the build-time rewrite that adds the library to debug builds (ADFA-7138). - The AI-Agent-MCP side that connects to the server (ADFA-7139). - A device run of the current code. The device results on ADFA-7135 (statements captured on both ABIs, the overhead) predate the review fixes. diff --git a/subprojects/sqlite-debugger/build.gradle.kts b/subprojects/sqlite-debugger/build.gradle.kts index 8223239452..025dab6224 100644 --- a/subprojects/sqlite-debugger/build.gradle.kts +++ b/subprojects/sqlite-debugger/build.gradle.kts @@ -34,6 +34,10 @@ android { targetCompatibility = JavaVersion.VERSION_1_8 } + testOptions { + unitTests.isIncludeAndroidResources = true + } + buildFeatures { viewBinding = false buildConfig = false @@ -67,4 +71,6 @@ dependencies { testImplementation(libs.org.json) testImplementation(libs.tests.junit) testImplementation(libs.tests.google.truth) + testImplementation(libs.tests.robolectric) + testImplementation(libs.tests.androidx.test.core) } diff --git a/subprojects/sqlite-debugger/src/main/AndroidManifest.xml b/subprojects/sqlite-debugger/src/main/AndroidManifest.xml index 18de3c738f..c9caebc849 100644 --- a/subprojects/sqlite-debugger/src/main/AndroidManifest.xml +++ b/subprojects/sqlite-debugger/src/main/AndroidManifest.xml @@ -1,6 +1,10 @@ + + + + diff --git a/subprojects/sqlite-debugger/src/main/java/org/appdevforall/sqlitedebugger/DebuggerStartup.kt b/subprojects/sqlite-debugger/src/main/java/org/appdevforall/sqlitedebugger/DebuggerStartup.kt index a2605df810..8779b35a39 100644 --- a/subprojects/sqlite-debugger/src/main/java/org/appdevforall/sqlitedebugger/DebuggerStartup.kt +++ b/subprojects/sqlite-debugger/src/main/java/org/appdevforall/sqlitedebugger/DebuggerStartup.kt @@ -2,6 +2,7 @@ package org.appdevforall.sqlitedebugger import android.content.ContentProvider import android.content.ContentValues +import android.content.Context import android.content.pm.ApplicationInfo import android.database.Cursor import android.net.Uri @@ -35,7 +36,7 @@ class DebuggerStartup : ContentProvider() { engine.install() thread(name = "sqlite-debugger-start", isDaemon = true) { try { - startServer(engine, databaseDir) + startServer(context, engine, databaseDir) } catch (t: Throwable) { Log.e(TAG, "could not start the server", t) } @@ -47,6 +48,7 @@ class DebuggerStartup : ContentProvider() { } private fun startServer( + context: Context, engine: NativeDebugEngine, databaseDir: File, ) { @@ -58,6 +60,7 @@ class DebuggerStartup : ContentProvider() { port = http.boundPort endpointToken = token Log.i(TAG, "listening on 127.0.0.1:${http.boundPort}") + EndpointBroadcast.sendAndListen(context, token, http.boundPort) } override fun query( @@ -91,7 +94,7 @@ class DebuggerStartup : ContentProvider() { companion object { private const val TAG = "SqliteDebugger" - /** Set once the server listens; the broadcast to Code on the Go (ADFA-7137) reads them. */ + /** Set once the server listens. */ @Volatile var port: Int = 0 private set diff --git a/subprojects/sqlite-debugger/src/main/java/org/appdevforall/sqlitedebugger/EndpointBroadcast.kt b/subprojects/sqlite-debugger/src/main/java/org/appdevforall/sqlitedebugger/EndpointBroadcast.kt new file mode 100644 index 0000000000..2dc8dff2e0 --- /dev/null +++ b/subprojects/sqlite-debugger/src/main/java/org/appdevforall/sqlitedebugger/EndpointBroadcast.kt @@ -0,0 +1,103 @@ +package org.appdevforall.sqlitedebugger + +import android.app.PendingIntent +import android.content.BroadcastReceiver +import android.content.Context +import android.content.Intent +import android.content.IntentFilter +import android.os.Build +import android.util.Log + +/** Tells Code on the Go which port and token this process's server uses (ADFA-7137). */ +internal object EndpointBroadcast { + private const val TAG = "SqliteDebugger" + + // ACTION, EXTRA_TOKEN, EXTRA_PORT and EXTRA_SENDER repeat the constants of the same names in Code on the + // Go's DebuggerEndpointValidation (:app), which this library cannot depend on. EndpointBroadcastTest here + // and DebuggerEndpointContractTest there pin the same values. + const val ACTION = "com.itsaky.androidide.SQLITE_DEBUGGER_ENDPOINT" + const val COGO_PACKAGE = "com.itsaky.androidide" + const val EXTRA_TOKEN = "token" + const val EXTRA_PORT = "port" + const val EXTRA_SENDER = "sender" + + // Code on the Go sends this when it pairs the app; its DebuggerEndpointAnnounce (:app) repeats the value, and + // the same two tests pin it. + const val ANNOUNCE_ACTION = "org.appdevforall.sqlitedebugger.ANNOUNCE" + private const val IDENTITY_ACTION = "org.appdevforall.sqlitedebugger.IDENTITY" + + /** + * Sends the endpoint now, and again each time Code on the Go asks with [ANNOUNCE_ACTION]. Code on the Go drops + * a report from an app it has not paired, so an app already running when it pairs would otherwise stay unknown + * to it until the process restarts. Listens first, so a request that arrives before the first send is not lost. + */ + fun sendAndListen( + context: Context, + token: String, + port: Int, + ) { + listen(context, token, port) + send(context, token, port) + } + + /** + * Registered at run time, not in the manifest, so it lives as long as this process's server and a request never + * starts the app. Exported, because Code on the Go is another app, and not authenticated (ADFA-7131 D8): the + * reply goes only to Code on the Go, so a request from any other app learns nothing. Never throws. + */ + private fun listen( + context: Context, + token: String, + port: Int, + ) { + try { + val receiver = + object : BroadcastReceiver() { + override fun onReceive( + context: Context, + intent: Intent, + ) { + if (intent.action == ANNOUNCE_ACTION) send(context, token, port) + } + } + val filter = IntentFilter(ANNOUNCE_ACTION) + if (Build.VERSION.SDK_INT >= 33) { + context.registerReceiver(receiver, filter, Context.RECEIVER_EXPORTED) + } else { + context.registerReceiver(receiver, filter) + } + } catch (e: Exception) { + Log.w(TAG, "could not listen for Code on the Go's requests to send the endpoint again", e) + } + } + + /** Never throws: the app must run whether or not Code on the Go is there to hear this. */ + fun send( + context: Context, + token: String, + port: Int, + ) { + try { + // Code on the Go reads creatorPackage, which the framework sets and no other app can forge. + // The wrapped intent is explicit and never sent; nothing handles its action. + val flags = if (Build.VERSION.SDK_INT >= 23) PendingIntent.FLAG_IMMUTABLE else 0 + val sender = + PendingIntent.getBroadcast( + context, + 0, + Intent(IDENTITY_ACTION).setPackage(context.packageName), + flags, + ) + val intent = + Intent(ACTION) + .setPackage(COGO_PACKAGE) + .addFlags(Intent.FLAG_INCLUDE_STOPPED_PACKAGES or Intent.FLAG_RECEIVER_FOREGROUND) + .putExtra(EXTRA_TOKEN, token) + .putExtra(EXTRA_PORT, port) + .putExtra(EXTRA_SENDER, sender) + context.sendBroadcast(intent) + } catch (e: Exception) { + Log.w(TAG, "could not send the endpoint to Code on the Go", e) + } + } +} diff --git a/subprojects/sqlite-debugger/src/test/java/org/appdevforall/sqlitedebugger/EndpointBroadcastTest.kt b/subprojects/sqlite-debugger/src/test/java/org/appdevforall/sqlitedebugger/EndpointBroadcastTest.kt new file mode 100644 index 0000000000..e62f973594 --- /dev/null +++ b/subprojects/sqlite-debugger/src/test/java/org/appdevforall/sqlitedebugger/EndpointBroadcastTest.kt @@ -0,0 +1,69 @@ +package org.appdevforall.sqlitedebugger + +import android.app.Application +import android.app.PendingIntent +import android.content.Intent +import android.os.Looper +import androidx.test.core.app.ApplicationProvider +import com.google.common.truth.Truth.assertThat +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.RobolectricTestRunner +import org.robolectric.Shadows.shadowOf +import org.robolectric.annotation.Config + +/** + * The literals below are Code on the Go's contract: the same values are pinned by + * DebuggerEndpointContractTest in :app (DebuggerEndpointValidation). Change both together. + */ +@RunWith(RobolectricTestRunner::class) +class EndpointBroadcastTest { + @Test + fun `the constants match the receiver in Code on the Go`() { + assertThat(EndpointBroadcast.ACTION).isEqualTo("com.itsaky.androidide.SQLITE_DEBUGGER_ENDPOINT") + assertThat(EndpointBroadcast.COGO_PACKAGE).isEqualTo("com.itsaky.androidide") + assertThat(EndpointBroadcast.EXTRA_TOKEN).isEqualTo("token") + assertThat(EndpointBroadcast.EXTRA_PORT).isEqualTo("port") + assertThat(EndpointBroadcast.EXTRA_SENDER).isEqualTo("sender") + assertThat(EndpointBroadcast.ANNOUNCE_ACTION).isEqualTo("org.appdevforall.sqlitedebugger.ANNOUNCE") + } + + @Test + fun `send broadcasts the endpoint to Code on the Go with the sender identity`() { + val app = ApplicationProvider.getApplicationContext() + + EndpointBroadcast.send(app, "tok-en_1", 5555) + + val intent = shadowOf(app).broadcastIntents.single() + assertThat(intent.action).isEqualTo("com.itsaky.androidide.SQLITE_DEBUGGER_ENDPOINT") + assertThat(intent.`package`).isEqualTo("com.itsaky.androidide") + assertThat(intent.getStringExtra("token")).isEqualTo("tok-en_1") + assertThat(intent.getIntExtra("port", 0)).isEqualTo(5555) + @Suppress("DEPRECATION") + val sender = intent.getParcelableExtra("sender") + assertThat(sender).isNotNull() + assertThat(sender!!.creatorPackage).isEqualTo(app.packageName) + } + + // 28 and 33 run both registerReceiver calls: RECEIVER_EXPORTED exists from 33. + @Test + @Config(sdk = [28, 33]) + fun `an announce request sends the endpoint again with the same token and port`() { + val app = ApplicationProvider.getApplicationContext() + EndpointBroadcast.sendAndListen(app, "tok-en_1", 5555) + assertThat(endpointBroadcasts(app)).hasSize(1) + + app.sendBroadcast(Intent("org.appdevforall.sqlitedebugger.ANNOUNCE").setPackage(app.packageName)) + shadowOf(Looper.getMainLooper()).idle() + + val sent = endpointBroadcasts(app) + assertThat(sent).hasSize(2) + val again = sent.last() + assertThat(again.`package`).isEqualTo("com.itsaky.androidide") + assertThat(again.getStringExtra("token")).isEqualTo("tok-en_1") + assertThat(again.getIntExtra("port", 0)).isEqualTo(5555) + } + + private fun endpointBroadcasts(app: Application) = + shadowOf(app).broadcastIntents.filter { it.action == "com.itsaky.androidide.SQLITE_DEBUGGER_ENDPOINT" } +}