Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions docs/docs/configure/warehouses.md
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,26 @@ If you're already authenticated via `gcloud`, omit `credentials_path`:
| Field | Required | Description |
|-------|----------|-------------|
| `path` | No | Database file path. Omit or use `":memory:"` for in-memory |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Align the optional-path documentation with the new rejection

The warehouse documentation still tells users that path is optional and may be omitted for an in-memory DuckDB connection, but requireStorePath() now rejects every omitted path. A user following this table with { "type": "duckdb" } therefore gets a missing-path error instead of the documented in-memory database; either mark the field required and remove the omission guidance or preserve the documented fallback.

Useful? React with 👍 / 👎.

| `create` | No | Create the database file if it is missing (default: `false`) |

!!! warning "The store must already exist"
Connecting never creates the database. If the file is missing, the connection
fails with an error naming the path it looked for — an empty database would
otherwise answer every query with no rows and no error, which reads as a
successful query against an empty warehouse. Set `"create": true` only when
you intend this store to be created.

!!! note "How a relative `path` is resolved"
A relative `path` is resolved once, when the config is loaded, against the
directory that declares it:

- `~/.altimate-code/connections.json` → resolved against `~/.altimate-code`
- `<project>/.altimate-code/connections.json` → resolved against `<project>`
- `ALTIMATE_CODE_CONN_*` environment variables → resolved against the project root

It is never resolved against the current working directory, so `--dir` cannot
re-point an existing connection at a different file. Absolute paths are always
safest.

!!! note "Concurrent access"
DuckDB does not support concurrent write access to the same file. If another process holds a write lock, Altimate Code automatically retries the connection in **read-only** mode so you can still query the data. A clear error message is shown if read-only access also fails.
Expand Down Expand Up @@ -443,10 +463,16 @@ If you're already authenticated via `gcloud`, omit `credentials_path`:
|-------|----------|-------------|
| `path` | No | Database file path. Omit or use `":memory:"` for in-memory |
| `readonly` | No | Open in read-only mode (default: `false`) |
| `create` | No | Create the database file if it is missing (default: `false`) |

!!! note
SQLite uses Bun's built-in `bun:sqlite` driver. WAL journal mode is enabled automatically for writable databases.

!!! warning "The store must already exist"
As with DuckDB, connecting never creates the database, and a relative `path`
resolves against the directory of the config that declares it — not the current
working directory. See the DuckDB section above for the full rules.

## SQL Server

```json
Expand Down
5 changes: 5 additions & 0 deletions docs/docs/drivers.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,11 @@ MongoDB supports server versions 3.6 through 8.0. Queries use MQL (MongoDB Query
|--------|--------------|
| File | `path: "./my-database.sqlite"` |

For both file-backed drivers, the database must already exist — connecting never
creates it. Pass `create: true` to create it deliberately. A relative `path`
resolves against the directory of the config that declares it, not the current
working directory. See [Warehouses](configure/warehouses.md#duckdb) for the full rules.

## SSH Tunneling

Connect through a bastion host by adding SSH config to any connection:
Expand Down
8 changes: 7 additions & 1 deletion packages/drivers/src/duckdb.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
* DuckDB driver using the `duckdb` package.
*/

import { assertStoreExists, requireStorePath } from "./file-store"
import type { ConnectionConfig, Connector, ConnectorResult, ExecuteOptions, SchemaColumn } from "./types"
import { loadOptionalDriver } from "./resolve"

Expand All @@ -10,7 +11,9 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
duckdb = await loadOptionalDriver("duckdb", "duckdb")
duckdb = duckdb.default || duckdb

const dbPath = (config.path as string) ?? ":memory:"
// altimate_change start — a missing path must fail loudly, not become :memory:
const dbPath = requireStorePath(config, "DuckDB")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: When an intentional unnamed in-memory connection omits path, requireStorePath now fails before DuckDB opens. Update the in-memory callers to pass path: ":memory:", or provide an explicit unnamed-memory option, so the E2E suite does not report a false skip.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/drivers/src/duckdb.ts, line 15:

<comment>When an intentional unnamed in-memory connection omits `path`, `requireStorePath` now fails before DuckDB opens. Update the in-memory callers to pass `path: ":memory:"`, or provide an explicit unnamed-memory option, so the E2E suite does not report a false skip.</comment>

<file context>
@@ -11,7 +11,9 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
 
-  const dbPath = (config.path as string) ?? ":memory:"
+  // altimate_change start — a missing path must fail loudly, not become :memory:
+  const dbPath = requireStorePath(config, "DuckDB")
+  // altimate_change end
   let db: any
</file context>

// altimate_change end
let db: any
let connection: any

Expand Down Expand Up @@ -48,6 +51,9 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {

return {
async connect() {
// altimate_change start — never conjure an empty store on open
assertStoreExists(config, dbPath, "DuckDB")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: For DuckDB, a file: URI path bypasses the new 'never conjure an empty store' guard entirely. isLocalFilePath("file:/data/wh.duckdb") returns false (it matches the two-or-more-char scheme regex), so assertStoreExists returns without checking existence, and new duckdb.Database(dbPath) silently creates a missing store — the exact silent-creation bug this PR fixes. The scheme exclusion is correct for extension schemes like md: and s3:// where DuckDB errors on an unknown scheme, but file: is a built-in scheme DuckDB opens and creates, so it cannot be left to the driver. Existence-check the path extracted from a file: URI, as the PR's own relativeFileUriPath does for SQLite.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/drivers/src/duckdb.ts, line 53:

<comment>For DuckDB, a `file:` URI path bypasses the new 'never conjure an empty store' guard entirely. `isLocalFilePath("file:/data/wh.duckdb")` returns false (it matches the two-or-more-char scheme regex), so `assertStoreExists` returns without checking existence, and `new duckdb.Database(dbPath)` silently creates a missing store — the exact silent-creation bug this PR fixes. The scheme exclusion is correct for extension schemes like `md:` and `s3://` where DuckDB errors on an unknown scheme, but `file:` is a built-in scheme DuckDB opens and creates, so it cannot be left to the driver. Existence-check the path extracted from a `file:` URI, as the PR's own `relativeFileUriPath` does for SQLite.</comment>

<file context>
@@ -48,6 +49,9 @@ export async function connect(config: ConnectionConfig): Promise<Connector> {
   return {
     async connect() {
+      // altimate_change start — never conjure an empty store on open
+      assertStoreExists(config, dbPath, "DuckDB")
+      // altimate_change end
       // altimate_change start — retry with read-only on lock errors
</file context>

// altimate_change end
// altimate_change start — retry with read-only on lock errors
const tryConnect = (accessMode?: string): Promise<any> =>
new Promise<any>((resolve, reject) => {
Expand Down
96 changes: 96 additions & 0 deletions packages/drivers/src/file-store.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
/**
* Shared guards for file-backed drivers (DuckDB, SQLite).
*
* Both engines create an empty database when asked to open a file that does
* not exist. For a warehouse connection that is never what the caller wants:
* a mistyped or mis-resolved path then yields a working connector over an
* empty database, so every query succeeds and returns nothing. An agent handed
* that result reports "no tables" instead of an error.
*
* Opening a store is therefore read-or-fail by default. Creation is opt-in via
* `create: true`, which the tools that deliberately materialize a local store
* (local test scratch databases, schema sync targets) pass explicitly.
*/

import * as fs from "fs"
import type { ConnectionConfig } from "./types"

/**
* Whether `dbPath` names a file on the local filesystem, and so can be
* existence-checked before the driver opens it.
*
* Only the exact string `:memory:` is an in-memory database. Both engines
* treat `:memory:named` — and any other colon-prefixed name — as an ordinary
* (if oddly named) file: DuckDB really does write a file called
* `:memory:named`, so those must stay inside the guard. An empty path is
* DuckDB's in-memory database and SQLite's anonymous temporary one; neither
* touches disk.
*
* A scheme-qualified target is not a local file: MotherDuck (`md:`), object
* storage (`s3://`), DuckLake, and any other scheme a DuckDB extension
* provides. Those are left to the driver, which reports an unknown scheme as a
* missing-extension error rather than silently creating anything. The pattern
* requires two or more characters before the colon so a Windows drive letter
* (`C:\data\wh.duckdb`) stays a path.
*/
export function isLocalFilePath(dbPath: string): boolean {
if (dbPath === "" || dbPath === ":memory:") return false
if (/^[a-zA-Z][a-zA-Z0-9+.-]+:/.test(dbPath)) return false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: When SQLite is configured with a missing absolute file: URI, this predicate classifies the local target as non-local, so assertStoreExists skips the existence check and bun:sqlite creates an empty database. Treat absolute/local file: URIs as filesystem paths for the guard, or normalize them to a filesystem path before checking while retaining remote-scheme exemptions.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/drivers/src/file-store.ts, line 38:

<comment>When SQLite is configured with a missing absolute `file:` URI, this predicate classifies the local target as non-local, so `assertStoreExists` skips the existence check and `bun:sqlite` creates an empty database. Treat absolute/local `file:` URIs as filesystem paths for the guard, or normalize them to a filesystem path before checking while retaining remote-scheme exemptions.</comment>

<file context>
@@ -0,0 +1,88 @@
+ */
+export function isLocalFilePath(dbPath: string): boolean {
+  if (dbPath === "" || dbPath === ":memory:") return false
+  if (/^[a-zA-Z][a-zA-Z0-9+.-]+:/.test(dbPath)) return false
+  return true
+}
</file context>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1: On SQLite, FILE:warehouse.db and foo:warehouse.db are literal filenames, not URI schemes, but this shared classifier marks them non-local. Make classification engine-aware so only actual remote DuckDB schemes bypass local path resolution.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/drivers/src/file-store.ts, line 38:

<comment>On SQLite, `FILE:warehouse.db` and `foo:warehouse.db` are literal filenames, not URI schemes, but this shared classifier marks them non-local. Make classification engine-aware so only actual remote DuckDB schemes bypass local path resolution.</comment>

<file context>
@@ -0,0 +1,114 @@
+ */
+export function isLocalFilePath(dbPath: string): boolean {
+  if (dbPath === "" || dbPath === ":memory:") return false
+  if (/^[a-zA-Z][a-zA-Z0-9+.-]+:/.test(dbPath)) return false
+  return true
+}
</file context>

return true
}

/**
* The store path a file-backed connection names, or a loud failure.
*
* Both drivers used to read `(config.path as string) ?? ":memory:"`. That turns
* ANY failure to carry a path — a config the registry never loaded, a field
* under the wrong name, a lookup that fell through — into a successful
* connection over an empty in-memory database. Every query then returns no rows
* and no error, which reads as a healthy warehouse that happens to be empty.
*
* It is a worse failure than creating a store on disk: a stray file can at
* least be found afterwards, whereas an in-memory database leaves nothing
* behind to explain the empty answer. `:memory:` remains available, but only
* when a caller asks for it by name.
*/
export function requireStorePath(config: ConnectionConfig, engine: string): string {
const value = config.path
if (typeof value === "string" && value !== "") return value
throw new Error(
Comment on lines +58 to +59

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the DuckDB availability probe runnable

When the DuckDB addon is installed, probeDuckDB() in packages/opencode/test/altimate/drivers-e2e.test.ts:71-95 still calls connect({ type: "duckdb" }); this new rejection is caught and reported as duckdbAvailable = false, causing every test.skipIf(!duckdbAvailable) DuckDB E2E test to be skipped even though the driver is available. Update the probe and the other in-memory fixtures in that suite to pass path: ":memory:" explicitly so CI continues exercising DuckDB.

Useful? React with 👍 / 👎.

`${engine} connection is missing its "path". A file-backed warehouse must name its database explicitly — ` +
`falling back to an in-memory database would answer every query with no rows and no error, ` +
`which is indistinguishable from a healthy but empty warehouse. ` +
`Set "path" to the database file, or to ":memory:" if a throwaway empty database is genuinely what you want.`,
)
}

/** Whether the caller explicitly opted in to creating the store. */
export function allowsCreate(config: ConnectionConfig): boolean {
return config.create === true
}

/**
* Throw unless the store is safe to open: it already exists, the caller opted
* in to creating it, or the path is not a local file at all.
*
* @param engine Human-readable engine name used in the error message.
* @param allowCreate Whether this open will actually create the store. Defaults
* to the config's `create` flag; a driver passes it explicitly when its own
* options can veto creation — SQLite never creates a read-only connection.
*/
export function assertStoreExists(
config: ConnectionConfig,
dbPath: string,
engine: string,
allowCreate: boolean = allowsCreate(config),
): void {
if (allowCreate) return
if (!isLocalFilePath(dbPath)) return
if (fs.existsSync(dbPath)) return

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: fs.existsSync() also returns true when dbPath is a directory, so a wrong path pointing at a directory bypasses the guard's clear error and surfaces a generic engine error later. Check it is a file (fs.statSync(...).isFile()) before treating the path as present.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/drivers/src/file-store.ts, line 55:

<comment>fs.existsSync() also returns true when dbPath is a directory, so a wrong path pointing at a directory bypasses the guard's clear error and surfaces a generic engine error later. Check it is a file (fs.statSync(...).isFile()) before treating the path as present.</comment>

<file context>
@@ -0,0 +1,62 @@
+): void {
+  if (allowCreate) return
+  if (!isLocalFilePath(dbPath)) return
+  if (fs.existsSync(dbPath)) return
+  throw new Error(
+    `${engine} database file not found: "${dbPath}". ` +
</file context>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: If a DuckDB file is removed after this check but before new duckdb.Database, DuckDB recreates it and queries silently see an empty store. Use an atomic no-create open mode for non-create connections, or reject and clean up a file created during the race.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/drivers/src/file-store.ts, line 64:

<comment>If a DuckDB file is removed after this check but before `new duckdb.Database`, DuckDB recreates it and queries silently see an empty store. Use an atomic no-create open mode for non-`create` connections, or reject and clean up a file created during the race.</comment>

<file context>
@@ -0,0 +1,71 @@
+): void {
+  if (allowCreate) return
+  if (!isLocalFilePath(dbPath)) return
+  if (fs.existsSync(dbPath)) return
+  throw new Error(
+    `${engine} database file not found: "${dbPath}". ` +
</file context>

throw new Error(
`${engine} database file not found: "${dbPath}". ` +
`Opening a warehouse connection never creates the database — an empty store would answer every query with no rows. ` +
`Check the "path" in your connection config (relative paths resolve against the config file's directory, not the current directory), ` +
`or pass "create": true if this store is meant to be created.`,
)
}
3 changes: 3 additions & 0 deletions packages/drivers/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ export type { Connector, ConnectorResult, SchemaColumn, ConnectionConfig } from
// Re-export config normalization
export { normalizeConfig, sanitizeConnectionString } from "./normalize"

// Re-export file-backed store guards
export { allowsCreate, assertStoreExists, isLocalFilePath, requireStorePath } from "./file-store"

// Re-export driver connect functions
export { connect as connectPostgres } from "./postgres"
export { connect as connectSnowflake } from "./snowflake"
Expand Down
17 changes: 15 additions & 2 deletions packages/drivers/src/sqlite.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,32 @@
*/

import { Database } from "bun:sqlite"
import { allowsCreate, assertStoreExists, requireStorePath } from "./file-store"
import type { ConnectionConfig, Connector, ConnectorResult, ExecuteOptions, SchemaColumn } from "./types"

export async function connect(config: ConnectionConfig): Promise<Connector> {
const dbPath = (config.path as string) ?? ":memory:"
// altimate_change start — a missing path must fail loudly, not become :memory:
const dbPath = requireStorePath(config, "SQLite")
// altimate_change end
let db: Database | null = null

return {
async connect() {
const isReadonly = config.readonly === true
// altimate_change start — never conjure an empty store on open.
// A read-only connection never creates, so `create: true` cannot excuse a
// missing file there; the guard is told the effective decision.
const willCreate = !isReadonly && allowsCreate(config)
assertStoreExists(config, dbPath, "SQLite", willCreate)
Comment thread
cubic-dev-ai[bot] marked this conversation as resolved.
db = new Database(dbPath, {
readonly: isReadonly,
create: !isReadonly,
// `create` alone is no longer implied by "not readonly", so the
// read-write flag has to be explicit — bun:sqlite rejects an options
// object that sets no open flag at all.
readwrite: !isReadonly,
create: willCreate,
Comment on lines +29 to +30

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve creation flags for SQLite memory URIs

When path is a valid SQLite URI-backed in-memory or temporary database, such as file::memory:?cache=shared, file:name?mode=memory, or file:, willCreate is false by default and the driver passes create: false; Bun/SQLite requires the create open flag for these targets and rejects them with unable to open database file. These paths are deliberately exempted from the filesystem existence guard, so enable the SQLite create flag for known non-file memory/temporary URI forms without permitting an ordinary missing disk file to be created.

Useful? React with 👍 / 👎.

})
// altimate_change end
if (!isReadonly) {
db.exec("PRAGMA journal_mode = WAL")
}
Expand Down
6 changes: 3 additions & 3 deletions packages/drivers/test/driver-security.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ describe("DuckDB driver", () => {
}))

const { connect } = await import("../src/duckdb")
const connector = await connect({ type: "duckdb", path: "/tmp/test.duckdb" })
const connector = await connect({ type: "duckdb", path: "/tmp/test.duckdb", create: true })
await connector.connect()
expect(connectAttempts).toBe(2) // First failed, second succeeded in READ_ONLY
// The retry must specifically request READ_ONLY — two attempts alone don't
Expand Down Expand Up @@ -262,7 +262,7 @@ describe("DuckDB driver", () => {
}))

const { connect } = await import("../src/duckdb")
const connector = await connect({ type: "duckdb", path: "/tmp/sync-lock.duckdb" })
const connector = await connect({ type: "duckdb", path: "/tmp/sync-lock.duckdb", create: true })
await connector.connect()
expect(connectAttempts).toBe(2)
expect(await connector.execute("SELECT 1")).toMatchObject({ columns: ["ok"], rows: [[1]], row_count: 1 })
Expand All @@ -287,7 +287,7 @@ describe("DuckDB driver", () => {
}))

const { connect } = await import("../src/duckdb")
const connector = await connect({ type: "duckdb", path: "/tmp/corrupt.duckdb" })
const connector = await connect({ type: "duckdb", path: "/tmp/corrupt.duckdb", create: true })
await expect(connector.connect()).rejects.toThrow("catalog is corrupt")
})
})
Expand Down
Loading
Loading