Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 27 additions & 24 deletions native/kernel/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,33 +61,36 @@ platform — each containing a single `.node` binary. `native/kernel/index.js`
(the napi-rs router) `require()`s the package matching the consumer's
`process.platform` / `process.arch` at load time.

> **M0 status:** these per-platform packages are **not yet published**, so
> they are intentionally **not** declared in the driver's
> `optionalDependencies`. (npm refuses an `npm ci` against a pinned
> dependency it cannot resolve from the registry, so declaring an
> unpublished package would break every install.) Until they ship, the
> binding is produced locally via `npm run build:native` (which copies
> `index.<triple>.node` into this directory). Once the packages are
> published, add `@databricks/databricks-sql-kernel-<triple>` back to
> `optionalDependencies` — npm then installs only the matching one.

## Supported platforms (M0)

M0 targets a **single** triple: **`linux-x64-gnu`** (package
`@databricks/databricks-sql-kernel-linux-x64-gnu`, once published).

On every other platform (macOS, Windows, linux-arm64, linux-x64-musl
/ Alpine, …) the kernel binding is simply absent: `KernelNativeLoader`
returns `undefined` from `tryGet()` / throws a structured
`MODULE_NOT_FOUND` hint from `get()`, and the driver continues to use
the Thrift backend exclusively. This is expected, not a regression —
additional triples are added to `optionalDependencies` as the kernel
CI starts publishing them in later milestones.
> **Status:** the per-platform packages are published on npm (kernel
> `0.2.0`) and declared in the driver's `optionalDependencies`, pinned to
> that exact version. `npm install` resolves only the package matching the
> consumer's `process.platform` / `process.arch`; the others are skipped
> (that is what `optionalDependencies` tolerates), so installing on an
> unsupported platform does not fail. Local development can still override
> the installed binary with `npm run build:native`, which copies a freshly
> built `index.<triple>.node` into this directory — the router prefers that
> local file over the installed package.

## Supported platforms

The driver declares all eight published triples in `optionalDependencies`:

- `linux-x64-gnu`, `linux-arm64-gnu`
- `linux-x64-musl`, `linux-arm64-musl`
- `darwin-x64`, `darwin-arm64`
- `win32-x64-msvc`, `win32-arm64-msvc`

On any other platform (e.g. linux-arm gnueabihf, riscv64, s390x, FreeBSD)
the kernel binding is simply absent: `KernelNativeLoader` returns
`undefined` from `tryGet()` / throws a structured `MODULE_NOT_FOUND` hint
from `get()`, and the driver continues to use the Thrift backend
exclusively. This is expected, not a regression — additional triples are
added to `optionalDependencies` as the kernel CI starts publishing them.

## Supply-chain note

The unpublished triple names (`@databricks/databricks-sql-kernel-darwin-arm64`,
`…-win32-x64-msvc`, etc.) referenced by the router are **not**
The triple names not yet built/published (`…-linux-arm-gnueabihf`,
`…-riscv64-gnu`, etc.) referenced by the router boilerplate are **not**
squat-able: `@databricks` is a Databricks-owned npm scope, and npm
only allows org members to publish under a scope it owns. A third
party therefore cannot register `@databricks/databricks-sql-kernel-*` and have
Expand Down
184 changes: 184 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 9 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,14 @@
"winston": "^3.8.2"
},
"optionalDependencies": {
"lz4": "^0.6.5"
"lz4": "^0.6.5",
"@databricks/databricks-sql-kernel-linux-x64-gnu": "0.2.0",
"@databricks/databricks-sql-kernel-linux-arm64-gnu": "0.2.0",
"@databricks/databricks-sql-kernel-linux-x64-musl": "0.2.0",
"@databricks/databricks-sql-kernel-linux-arm64-musl": "0.2.0",
"@databricks/databricks-sql-kernel-darwin-x64": "0.2.0",
"@databricks/databricks-sql-kernel-darwin-arm64": "0.2.0",
"@databricks/databricks-sql-kernel-win32-x64-msvc": "0.2.0",
"@databricks/databricks-sql-kernel-win32-arm64-msvc": "0.2.0"
}
}
77 changes: 77 additions & 0 deletions tests/e2e/kernel/select-one-e2e.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
// Copyright (c) 2026 Databricks, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import { expect } from 'chai';
import { DBSQLClient } from '../../../lib';
import { ConnectionOptions } from '../../../lib/contracts/IDBSQLClient';
import { InternalConnectionOptions } from '../../../lib/contracts/InternalConnectionOptions';

/**
* Canonical `SELECT 1` round-trip over the kernel backend.
*
* Unlike `execution-e2e.test.ts` (which exercises the operation
* lifecycle but does not read rows), this asserts the full pipeline
* end to end — including result fetch — when the kernel backend is
* selected via `useKernel: true`:
*
* DBSQLClient.connect({ useKernel: true })
* → KernelBackend → napi binding (loaded from the published
* @databricks/databricks-sql-kernel-<triple> optional dependency)
* → live warehouse → inline Arrow result → fetchAll()
*
* This is the smoke test for "the released driver consumes the npm
* kernel package and a query actually returns a value", matching the
* manual verification used to validate the optionalDependencies wiring.
*
* **Gating:** requires the pecotesting secrets exported in the shell.
* If any is missing, the suite is skipped so machines without
* provisioned credentials don't flap.
*/
describe('kernel SELECT 1 end-to-end (result fetch)', function e2eSuite() {
const host = process.env.DATABRICKS_PECOTESTING_SERVER_HOSTNAME;
const path = process.env.DATABRICKS_PECOTESTING_HTTP_PATH;
const token = process.env.DATABRICKS_PECOTESTING_TOKEN_PERSONAL;

// Live-warehouse round-trips can take a few seconds through warm-up.
this.timeout(60_000);

before(function gate() {
if (!host || !path || !token) {
// eslint-disable-next-line no-invalid-this
this.skip();
}
});

it('runs SELECT 1 via useKernel and fetches the row', async () => {
const client = new DBSQLClient();

await client.connect({
host: host as string,
path: path as string,
token: token as string,
useKernel: true,
} as ConnectionOptions & InternalConnectionOptions);

const session = await client.openSession();
const operation = await session.executeStatement('SELECT 1 AS one');

const rows = (await operation.fetchAll()) as Array<Record<string, unknown>>;
expect(rows).to.be.an('array').with.length(1);
expect(Number(rows[0].one)).to.equal(1);

await operation.close();
await session.close();
await client.close();
});
});
Loading