Commit 21df925
authored
ci(sea): Kernel E2E workflow — build napi binding from KERNEL_REV + run tests/e2e/sea on a live warehouse (#418)
* ci(sea): add Kernel E2E workflow — build napi binding from KERNEL_REV, run tests/e2e/sea on a live warehouse
Brings the SEA-backend end-to-end gate onto main (it previously lived only in the
stacked sea-kernel-rev-pin branch). Mirrors the Python connector's kernel-e2e.yml.
What it does:
- Reads the pinned kernel SHA from KERNEL_REV, checks the kernel out via a GitHub
App token (private repo), runs `npm run build:native` to compile the napi
binding into native/sea/ — so the binary matches KERNEL_REV exactly.
- Asserts the committed native/sea/index.{d.ts,js} contract matches the rebuilt
binding (catches drift between the pin and the committed router/types).
- Smoke-checks the binding loads, then runs `tests/e2e/sea/**` against the
azure-prod warehouse (maps the secrets onto the DATABRICKS_PECOTESTING_* vars
the SEA suite gates on).
Gate semantics (same pattern as the Thrift integration + Python kernel-e2e):
- Plain PR events post a synthetic-success "Kernel E2E" check so PRs that don't
touch SEA aren't blocked; the real suite runs in the merge queue.
- The `kernel-e2e` label triggers a preview run on the PR and is auto-removed on
push (a labelled run can't be re-triggered with unreviewed code).
- merge_group runs the real gate when SEA-relevant files changed, auto-passes
otherwise.
All referenced pieces already exist on main: the setup-jfrog composite (npm + the
inline cargo proxy config), KERNEL_REV, the build:native script, the committed
native/sea contract, and tests/e2e/sea/.
One-time admin setup required before this gates anything (documented in the
workflow header): the `kernel-e2e` label, the INTEGRATION_TEST_APP_ID /
INTEGRATION_TEST_PRIVATE_KEY GitHub App (allowlisted for databricks-sql-kernel),
and the azure-prod environment exposing DATABRICKS_HOST /
TEST_PECO_WAREHOUSE_HTTP_PATH / DATABRICKS_TOKEN.
Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
* ci(sea): inject E2E_* vars so tests/e2e/sea actually runs (config.ts process.exit fix)
The tests/e2e/sea suite reads creds from two sources: most tests read
DATABRICKS_PECOTESTING_* directly (already injected), but e2e-smoke.test.ts
imports tests/e2e/utils/config.ts, which process.exit(1)s at module load if any
E2E_* var is unset — aborting the whole mocha run before it starts. Mirror
main.yml's e2e-test job mapping (E2E_HOST/PATH/ACCESS_TOKEN + CATALOG/SCHEMA/
VOLUME/TABLE_SUFFIX) so the suite loads and runs.
Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
* ci(sea): cache ~/.npm in kernel-e2e (mirror main.yml) so npm ci doesn't hit unreachable dev proxy
Two package-lock entries (flatbuffers etc.) resolve to npm-proxy.dev.databricks.com,
which the protected runner can't reach (ECONNRESET). main.yml's e2e job only
succeeds because it caches ~/.npm — those tarballs load from the warm cache, never
fetched. kernel-e2e had no such cache, so npm ci failed deterministically.
Add the identical actions/cache step (same key) before npm ci. The driver has a
single root package-lock.json and the kernel checkout adds none, so
hashFiles('**/package-lock.json') matches main.yml's key exactly → restores main's
warmed cache. Confirmed end-to-end up to this point: App token + private-kernel
checkout + JFrog now all pass; npm ci was the only remaining failure.
Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
* ci(sea): run SEA e2e under nyc so ts-node/register loads .ts (fix ESM-loader race)
Bare `mocha` has no ts-node registration, so it loaded the .ts specs via the ESM
dynamic-import path and crashed with ERR_INTERNAL_ASSERTION (... not yet fully
loaded ... Promise.all) before any test ran. main.yml's e2e job avoids this by
running `nyc mocha` — nyc.config.js's `require: ['ts-node/register']` installs the
CommonJS ts-node hook, loading specs synchronously. Mirror that: invoke
`nyc ... mocha --config tests/e2e/.mocharc.js <glob>`, keeping the glob at argv
index 4 (the mocharc derives spec from process.argv.slice(4)).
Everything else in the pipeline is now confirmed green: App-token + private-kernel
checkout, ~/.npm cache, npm ci, build:native, the KERNEL_REV contract-drift assert,
and the binding smoke-check. This was the last failing step.
Co-authored-by: Isaac
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
---------
Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>1 parent 03936ec commit 21df925
1 file changed
Lines changed: 415 additions & 0 deletions
0 commit comments