Skip to content

warehouse_test times out on healthy local DuckDB stores, and the failure looks like a bad connection #1197

Description

@anandgupta42

warehouse_test fails on local DuckDB stores with Timed out opening DuckDB database, and the failure is indistinguishable from a bad connection.

The failure

Seven warehouse_test calls against local DuckDB stores, from six separate compiled-binary processes, all returned:

Failed to connect to warehouse '<name>'.
Error: Error: Timed out opening DuckDB database "<abs path>/workspace/warehouse.duckdb"

Durations, from the callers' own timestamps: 2061, 2002, 2005, 2005, 2004, 2003, 2004 ms. Six of seven within 5ms of exactly 2000. That is a fixed deadline firing, not I/O contention — contention scatters.

The stores were healthy. In the same processes, moments later, python3 -c "import duckdb" opened the same 1.3MB files and queried them successfully. The store was never the problem; our driver's own deadline was.

Why 2000ms is not enough

packages/drivers/src/duckdb.ts rejects the open after a hard-coded 2000ms with no way to raise it. That is not a performance budget. DuckDB dispatches the open to the libuv threadpool, so the wait covers queueing behind every other threadpool user in the process — fs, dns, crypto — not just DuckDB's own work. A busy agent process can push a healthy open past it. When it fires, the driver rejects and closes the handle that was about to succeed, so the caller has no way to recover.

A second, live path to the identical symptom

The "open callback fired synchronously" sentinel is undefined:

let pendingOpen: Error | null | undefined      // undefined = "not fired yet"
const onOpen = (err) => { if (!instance) { pendingOpen = err; return }  }

if (pendingOpen !== undefined) onOpen(pendingOpen)

undefined is exactly what a success callback invoked with no arguments passes. Such a callback is recorded, the replay guard rejects it as "not fired yet", the promise never settles, and the open fails on the deadline — producing this exact string, deterministically, at exactly 2000ms.

The second defect: the failure is silent

warehouse_test reports a driver that will not load and a wrong password with the same words:

Connection 'x': FAILED

So an infrastructure fault reads as a configuration fault, or — to a model, or to anyone reading a transcript — as the task simply not working. That is the more expensive half of this bug: it lets broken infrastructure be scored as a capability result.

Two further faults found while reproducing

  1. duckdb is missing from trustedDependencies in the root package.json, so bun install never runs its node-pre-gyp install lifecycle script and lib/binding/duckdb.node is never fetched. On such a tree every DuckDB call fails with DuckDB driver not installed. Run: npm install duckdb even though the package is present. Reproduced 0/7 in a fresh worktree, and verified in both directions in a scratch install. drivers-e2e.test.ts already carries a comment working around this symptom.

  2. wrapDuckDBError discards the only actionable part of a lock error. DuckDB's message names the PID and executable holding the conflicting lock; the wrapper replaced it with a generic summary.

Not this issue

PR #1122 fixes driver resolution (a bare await import("duckdb") resolving against bunfs in a compiled binary). That is a different failure: I compiled a probe with the production Bun.build options and ran it from a cwd with no node_modules, and resolution failure surfaces as DuckDB driver not installed in 0-2ms, never as a timeout. The two are disjoint.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions