Skip to content

test(js-sdk): run the full unit test suite in a browser - #1609

Open
mishushakov wants to merge 7 commits into
mainfrom
browser-test-suite
Open

test(js-sdk): run the full unit test suite in a browser#1609
mishushakov wants to merge 7 commits into
mainfrom
browser-test-suite

Conversation

@mishushakov

@mishushakov mishushakov commented Jul 24, 2026

Copy link
Copy Markdown
Member

Caution

The upstream fix this depended on was abandoned. infra#3389 was closed unmerged on 2026-08-04, and so was its successor infra#3565; the cors package does not exist on infra main. The 15 stopped-sandbox tests below are therefore not waiting on a deploy — they are waiting on work that was dropped.

Since the browser leg is not continue-on-error, it feeds the required SDK Tests Status check, which gates merges and the release publish step. This PR cannot merge until that is resolved, either by reviving the infra fix or by re-adding a canObserveStoppedSandbox gate so the affected tests report as skipped.

needs blocks symptom
infra#3388 6 Sandbox.list({ limit }) pagination tests rolled out — these pass
infra#3389closed, not merged 14 stopped-sandbox tests + the traffic-access-token one 14 × TypeError: Failed to fetch, 1 × SandboxError instead of TimeoutError

Current browser leg: 8 failed files, 67 passed, 1 skipped — every failing test is in the row above.

What

Promotes the browser leg from a single React smoke test to the full unit + connectionConfig suite running inside a real headless Chromium (@vitest/browser + Playwright) — the same coverage test:bun / test:deno / test:cf get. 69 files / 343 tests green locally against production.

pnpm test:browser   # installs Chromium via pretest:browser, then runs the suite

React is gone as requested — the smoke test (tests/runtimes/browser/run.test.tsx) and its seven devDependencies (react, react-dom, @types/react, @types/react-dom, @testing-library/react, @vitejs/plugin-react, vitest-browser-react). Chromium installation moves off the default pretest hook onto pretest:browser, so plain pnpm test no longer downloads a browser.

The one thing the smoke test uniquely covered — the SDK driven with no process global at all, the way a real browser bundle runs — is kept as a dedicated test, since the rest of the suite runs against a process.env shim that would mask a regression there:

// tests/runtimes/browser/noProcessGlobal.test.ts
Reflect.deleteProperty(globalThis, 'process')
const sandbox = await Sandbox.create(template, { apiKey, domain })
await sandbox.files.write('hello.txt', 'Hello World')

Failure screenshots are off (browser.screenshotFailures: false): these tests never render anything, so vitest's default would write a PNG of a blank page per failed test into .vitest-attachments/.

Test bugs the leg surfaced

  1. Two assertions were vacuous. commands/kill and pty/kill asserted rejects.toThrowError(ProcessExitError) — but ProcessExitError has never been exported from src. Vite's SSR transform resolves the missing named import to undefined, so toThrowError(undefined) asserted nothing on any runtime; real ESM in the browser turns it into a hard SyntaxError. Now CommandExitError.

  2. files/read.test.ts ran three tests twice — a byte-identical duplicated block, provisioning a second real sandbox for each on every runtime leg.

  3. A cold-cache-only flake, root-caused. api/http2 and envd/http2 carried dead vi.doUnmock('undici') / vi.doUnmock('../../src/utils') calls from an old refactor (neither file mocks anything any more). The bare 'undici' one made Vite discover and pre-bundle a Node HTTP client mid-run, reloading the browser page under the running suite and breaking whichever file was being collected. Verified: 1-in-4 failure before, three cold-cache runs green after.

Node-only test APIs are replaced with cross-runtime equivalents: node:crypto randomUUID/createHash → WebCrypto, BufferTextEncoder/TextDecoder, path.basename → inline.

Two real browser bugs found

Both verified over the wire; neither was fixable in the SDK alone.

Pagination silently truncated — the cursor lives in the x-next-token response header, which the API didn't name in Access-Control-Expose-Headers, so a browser withheld it and every page looked like the last:

const paginator = Sandbox.list({ limit: 10 })
const all = []
while (paginator.hasNext) all.push(...(await paginator.nextItems()))
// Node: all 200 sandboxes.  Browser (before the fix): 10, hasNext === false, no error.

Fixed upstream in infra#3388, so the tests are un-gated here and just need the prod rollout (see the callout above). SDK-293.

SDK-294 — isRunning() threw instead of returning false. A stopped sandbox's 502 came from the edge without CORS headers, so the probe rejected opaquely (running: 204 + ACAO: * / paused: 502 + no ACAO). Because a browser always preflights the SDK's envd requests (they carry E2b-Sandbox-Id/E2b-Sandbox-Port) and a preflight needs an ok status, adding the header to the 502 alone wasn't enough — the OPTIONS path needed a 2xx too. This also silently downgraded the kill-mid-request error from the actionable TimeoutError to a generic SandboxError across 24 call sites.

Fixed upstream in infra#3389, which also fixes the content negotiation the investigation turned up: the proxy chose HTML over JSON by User-Agent sniffing, and browser fetch can't override its UA, so an SDK call from a browser got the HTML error page where it expects JSON (Accept was ignored). Both are un-gated here.

How limitations are handled

No capability flags — nothing is skipped for being a browser. The five tests that read a response from a server the test itself starts in the sandbox now start a CORS-enabled one, via corsHttpServerCmd in tests/setup.ts:

// was: 'python3 -m http.server 8000' — no CORS headers, so a browser hands the
// test an opaque `TypeError: Failed to fetch` instead of the response
await sandbox.commands.run(corsHttpServerCmd(8000), { background: true })

This was originally gated behind a canFetchSandboxServers flag as an inherent browser limitation. It isn't one: the server is the test's own, and a real browser app's server opts into CORS exactly the same way — so the gate was hiding coverage of getHost and the proxy path rather than documenting a wall. Access-Control-Allow-Headers plus do_OPTIONS also covers the traffic-access-token test, whose custom header the browser preflights.

Confirmed on the browser leg (run): of the five tests this un-gates, three now passping server in running sandbox, sandbox works without token, auto-resume wakes paused sandbox on http request. That's the CORS server working end-to-end through the proxy against production. The other two are the two infra#3389 needs anyway: pause and resume a sandbox with http server calls isRunning() on a paused sandbox, and sandbox requires traffic access token reads a proxy-synthesized 403 (which 3389 gives CORS headers) after a preflight that can't carry the token (which 3389 answers on the getDestination error path).

The exclude list is only for genuinely Node-only suites (tests/bundle/** reads dist via node:fs, tests/undici.test.ts resolves packages off process.versions.node) plus the two msw suites, whose msw/node entry pulls in node:http and can't be served to a browser. Porting those needs setupWorker plus a service worker from a public dir — tracked as a follow-up on SDK-292, and worth doing since cancellation is the most runtime-divergent part of this SDK.

A real bug the leg caught

main added toUploadBody leaves an async-iterable foreign stream alone while this branch was in review, and it failed in Chromium with expected '[object ReadableStream]' to be 'hello'. Not a test artifact — the mechanism, confirmed directly:

const asyncIterable = { async *[Symbol.asyncIterator]() { yield new TextEncoder().encode('hello') } }
await new Response(asyncIterable).text()
// Node:     'hello'                    (undici accepts any async iterable)
// Chromium: '[object ReadableStream]'   (the web platform does not)

toDispatchableStream treats async-iterability as "the platform will accept this", which is an undici extension rather than web behavior. So buffering a stream via new Response(stream) in a browser stringified it, and this silently uploaded 23 bytes of text instead of the file:

import { ReadableStream } from 'web-streams-polyfill' // or any replaced global / other realm

await sandbox.files.write('data.bin', new ReadableStream({ /* ... */ }))
// Before: the file contains the literal text "[object ReadableStream]"
// After:  the file contains the data

The fix drains the stream through its reader — the one part every implementation shares — instead of guessing what the platform accepts. Note the obvious fix is wrong: skipping the async-iterable branch in the browser re-wraps a native stream when globalThis.ReadableStream has been replaced by a polyfill, which is precisely what that branch exists to prevent (there's a test for it). Verified in Chromium and Node.

CI

New browser matrix leg on ubuntu-22.04, with the Playwright cache moved onto it (no other leg needs Chromium any more). The leg is dropped by node-only staging callers, and it's the only leg whose result depends on the API's CORS headers — the matrix comment now says so, rather than claiming the non-Node legs add no backend signal.

Notes

  • No longer test-only: carries one source fix (below) with a patch changeset. Everything else is tests and CI.
  • Rebased onto main (84 commits). Playwright had meanwhile been extracted into a playwright:install script scoped to the node leg — pretest:browser now delegates to it rather than duplicating the install — and six new msw/node suites were added, which can't be served to a browser and are excluded.
  • vi.stubEnv and delete process.env.X keep working in the browser because the shim aliases process.env to import.meta.env, which is where vitest puts config env and where stubEnv writes. Verified E2B_DEBUG=1 pnpm test:browser correctly skips the debug-gated tests.
  • Verification for the CORS server, since a browser-side change like this is easy to get subtly wrong: ran a local instance of the exact generated command and fetched it cross-origin (different port = different origin) from real Chromium — simple GET readable, and a custom-header request survives the preflight. Negative control against a plain python -m http.server reproduces the TypeError: Failed to fetch the browser leg was showing, so the probe is meaningful rather than vacuous. Also checked there's exactly one Access-Control-Allow-Origin header — a duplicate would make browsers reject the response.
  • Review feedback addressed in 4c94fca: the pause and resume a sandbox with http server test now carries the canFetchSandboxServers gate too (it fetches its own python -m http.server, so it was a real miss that the isRunning failure happened to mask), and noProcessGlobal.test.ts imports the SDK dynamically inside the test body — a static import is evaluated at collection time, while the process shim is still installed, so it only covered the call path and not module evaluation.
  • No new type errors: 22 pre-existing test type errors before and after (the 2 that went away are the ProcessExitError ones this fixes).
  • Not addressed, pre-existing, flagged in case you want a follow-up: tests/api/list.test.ts duplicates five tests byte-for-byte (only the three betaPause ones intentionally differ, instance vs static), and files/write.test.ts:130 repeats write file as a strict subset of the test at :6 — together ~7 redundant sandboxes per run per leg. Left alone rather than deleting tests outside this change's scope.

Closes SDK-292

🤖 Generated with Claude Code

@linear-code

linear-code Bot commented Jul 24, 2026

Copy link
Copy Markdown

SDK-292

@cla-bot cla-bot Bot added the cla-signed label Jul 24, 2026
@cursor

cursor Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

PR Summary

Medium Risk
Touches the file-upload body conversion used by all runtimes, so a drain bug can corrupt or fail uploads. Browser tests now depend on production CORS behavior that staging does not exercise.

Overview
toBlob never releases the stream reader if read() throws, so a failed drain leaves the stream locked. Gzip uploads in the browser still go through new Response(compressed).blob(), which is the same brand-check path this change was meant to avoid. The new browser CI leg is skipped on staging (node-only), so CORS regressions only show up against production.

Reviewed by Cursor Bugbot for commit 3a3a3f9. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Package Artifacts

Built from c6127d6. Download artifacts from this workflow run.

JS SDK (e2b@2.45.1-browser-test-suite.0):

npm install ./e2b-2.45.1-browser-test-suite.0.tgz

CLI (@e2b/cli@2.17.2-browser-test-suite.0):

npm install ./e2b-cli-2.17.2-browser-test-suite.0.tgz

Python SDK (e2b==2.45.1+browser.test.suite):

pip install ./e2b-2.45.1+browser.test.suite-py3-none-any.whl

Comment thread packages/js-sdk/tests/sandbox/snapshot.test.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5626cfff60

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread packages/js-sdk/tests/runtimes/browser/noProcessGlobal.test.ts Outdated

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — test-only PR that expands the browser CI leg to the full unit suite, with no production code changes.

Extended reasoning...

Overview

This PR touches only test infrastructure: the browser vitest config, test files (replacing Node-only APIs like Buffer/node:crypto with WebCrypto/TextEncoder), the CI workflow matrix, package.json devDependencies (dropping React test tooling), and pnpm-lock.yaml. No files under packages/js-sdk/src/ are modified.

Security risks

None. No auth, crypto business logic, or user-facing behavior changes — only test harness and CI configuration.

Level of scrutiny

Low-to-moderate scrutiny is appropriate: this is a test-only change with no production code path affected. I spot-checked the PR's own claims against src/: CommandExitError is indeed exported (ProcessExitError never existed, confirming the described vacuous-assertion bug fix), and the browser-specific skip flags (canFetchSandboxServers, canReadPaginationToken, canObserveStoppedSandbox) correctly mirror existing runtime !== 'browser' CORS-avoidance logic already present in src/connectionConfig.ts. The new capability-gated skips are a reasonable way to surface browser limitations without silently dropping coverage.

Other factors

The bug hunting system found no issues. The PR description is thorough and self-documents the rationale for each test change (duplicate test removal, flaky mock cleanup, Node-API replacements). No changeset is needed since this is test-only, consistent with prior similar PRs (#1600) per CLAUDE.md guidance.

@changeset-bot

changeset-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 5ca9d31

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Comment thread packages/js-sdk/tests/setup.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit afe3fb8. Configure here.

Comment thread packages/js-sdk/tests/setup.ts Outdated
mishushakov and others added 6 commits August 21, 2026 18:56
Promotes the browser leg from a single React smoke test to the full unit +
connectionConfig suite running inside a real headless Chromium via
@vitest/browser + Playwright (`pnpm test:browser`) — the same coverage
test:bun / test:deno / test:cf get. 69 files / 343 tests green.

The React smoke test and its devDependencies are gone (react, react-dom,
@types/react{,-dom}, @testing-library/react, @vitejs/plugin-react,
vitest-browser-react). The one thing it uniquely covered — the SDK driven with
no `process` global at all, as in a real browser bundle — is kept as
tests/runtimes/browser/noProcessGlobal.test.ts. Chromium installation moves
from the default `pretest` hook to `pretest:browser`, so plain `pnpm test` no
longer downloads a browser.

Test bugs the leg surfaced:

- commands/kill and pty/kill asserted `rejects.toThrowError(ProcessExitError)`,
  but `ProcessExitError` is not exported from src. Vite's SSR transform
  resolved the named import to `undefined`, so both assertions were vacuous;
  real ESM in the browser makes it a hard SyntaxError. Now CommandExitError.
- files/read.test.ts had three byte-identical duplicated tests, each
  provisioning a second sandbox on every runtime.
- api/http2 and envd/http2 carried dead `vi.doUnmock('undici')` /
  `vi.doUnmock('../../src/utils')` calls from an old refactor. The bare
  'undici' one made Vite pre-bundle a Node HTTP client mid-run and reload the
  page under the running suite — a cold-cache-only flake that broke whichever
  file was being collected at the time.

Node-only test APIs are replaced with cross-runtime equivalents: node:crypto
randomUUID/createHash to WebCrypto, Buffer to TextEncoder/TextDecoder,
path.basename inlined.

Limitations a browser physically can't work around are gated by documented
capability flags in tests/setup.ts, so they report as skipped instead of
disappearing into the config's exclude list. Two are real user-facing bugs,
filed as SDK-293 (x-next-token is not in Access-Control-Expose-Headers, so
Sandbox.list pagination silently truncates) and SDK-294 (the stopped-sandbox
502 has no CORS headers, so isRunning() throws instead of returning false).

Closes SDK-292

Co-Authored-By: Claude <noreply@anthropic.com>
These tests drive the SDK and never render anything, so vitest's
browser.screenshotFailures (on by default when headless) wrote a PNG of a blank
page per failed test into .vitest-attachments/. Turned off, which also lets the
.gitignore entry it needed go away.

Removes canReadPaginationToken: the API not exposing X-Next-Token via
Access-Control-Expose-Headers is fixed upstream in infra#3388, so the six
limit-based pagination tests are un-gated and run in the browser like
everywhere else.

Note infra#3388 is not on production yet, and the browser leg only ever runs
against production (staging callers pass node-only, which drops it), so those
six tests fail on `paginator.hasNext` until the deploy lands.

Co-Authored-By: Claude <noreply@anthropic.com>
Removes canObserveStoppedSandbox. The stopped-sandbox 502 arriving from the
edge without CORS headers is fixed in e2b-dev/infra#3389, so isRunning() can
report false in a browser again and checkSandboxHealth can return false, which
restores the actionable TimeoutError on the 24 call sites that use it.

Also un-gates host.test.ts 'ping server in non-running sandbox', which was
under the wrong flag: it asserts the edge's own 502 JSON envelope rather than a
response from a server inside the sandbox, so infra#3389 covers it too (the
same PR's content-negotiation fix is what keeps the body JSON rather than the
HTML error page). Its sibling 'ping server in running sandbox' does read the
user's python server, so that one keeps the gate.

canFetchSandboxServers stays for the four tests that read a response from a
server the test started in the sandbox — no CORS headers there, and one of them
sends a custom header that would preflight into python's 501. Its doc comment
no longer cites the proxy 502 as an example, since that now carries the header.

Until infra#3389 reaches production these 14 tests fail on the browser leg —
13 with 'TypeError: Failed to fetch' and one with the degraded SandboxError in
place of TimeoutError, matching the two documented consequences exactly. The
Node/Bun/Deno/workerd legs are unaffected (7 files / 26 tests green on Node).

Co-Authored-By: Claude <noreply@anthropic.com>
- gate the http-server pause/resume test on canFetchSandboxServers: it
  fetches a `python -m http.server` the test starts itself, which sends no
  CORS headers. Masked today by the isRunning failure, it would have started
  failing the moment infra#3389 rolls out.
- import the SDK dynamically in noProcessGlobal, after the process shim is
  deleted. A static import is evaluated at collection time while the shim is
  still installed, so a top-level `process` read anywhere in the module graph
  would have passed here and still crashed a browser app on import.

Co-Authored-By: Claude <noreply@anthropic.com>
…browser gate

`python -m http.server` sends no CORS headers, so a browser handed the test an
opaque `TypeError: Failed to fetch` instead of the response. That was gated
behind `canFetchSandboxServers` as inherent, but it isn't: the server is ours,
and a real browser app's own server opts in the same way. `corsHttpServerCmd`
starts one that does, so the five tests run in Chromium too and the last
capability flag is gone.

Verified in Chromium against a local instance of the exact generated command:
cross-origin GET readable, and a custom-header request survives the preflight
via do_OPTIONS. Negative control against a plain `python -m http.server`
reproduces the `Failed to fetch` the browser leg was showing.

Co-Authored-By: Claude <noreply@anthropic.com>
… was written

Six new suites import msw's `setupServer`, whose `msw/node` entry pulls in
node:http and can't be served to a browser. Also drops ENABLE_VOLUME_TESTS from
the forwarded env — main removed that flag when it mocked the volume tests.

Co-Authored-By: Claude <noreply@anthropic.com>
Buffering a ReadableStream went through `new Response(stream)`, whose accepted
body types differ per runtime: undici takes any async iterable, a browser takes
only its own stream class and stringifies the rest. So a browser uploading a
stream from a polyfill or another realm silently sent the 23-byte string
"[object ReadableStream]" instead of the file.

Draining through the reader — the one part every implementation shares — is
correct everywhere and drops the runtime-dependent guess. Note the obvious fix
(skip the async-iterable branch of `toDispatchableStream` in the browser) is
wrong: it re-wraps a native stream when `globalThis.ReadableStream` has been
replaced by a polyfill, which is what that branch exists to prevent.

Found by the browser test leg via `toUploadBody leaves an async-iterable
foreign stream alone`, which main added while this branch was in review.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant