Skip to content

fix(providers): refuse lease allocation when the daemon holds other provider credentials - #3207

Merged
thymikee merged 3 commits into
mainfrom
claude/provider-credential-fingerprint
Oct 5, 2026
Merged

thymikee merged 3 commits into
mainfrom
claude/provider-credential-fingerprint

Conversation

@thymikee

@thymikee thymikee commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

Summary

A local daemon keeps its startup env while connect checks the shell's, so a stale daemon could act on old credentials, such as creating a billed Limrun instance for an attach-mode shell. Lease allocation now refuses that before anything is created:

Error (INVALID_ARGS): The running daemon holds different limrun credentials than this shell.
hint: Stop it with agent-device daemon stop --state-dir '<dir>', then rerun the command …
  • On a local lease_allocate, the CLI sends a versioned hash of the values each provider's own reader uses (Limrun, BrowserStack), never the values themselves. It goes over the socket and local HTTP. The daemon compares it with the hash of its startup env.
  • Boundaries: the check runs only on new allocations; leases already held are not re-checked. No hash is sent from a shell without credentials, or to remote, proxy or cloud daemons. A daemon with an HTTP auth hook treats every caller as remote and skips the check. AWS Device Farm is excluded.
  • The CLI loads the fingerprint module only for lease_allocate.

30 files, 814 gross lines; help and docs updated. An empty BrowserStack variable now counts as missing.

Validation

At 3fccd8615c, rebased on main after #3208 and #3209: pnpm check:affected --run passes (5,044 related tests, eager-closure budgets, daemon-wire-compat). Remaining minor: changing only LIMRUN_REGION, or the other platform's variables, also refuses (fails closed, as documented). No live run with real keys.

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.07 MB 5.07 MB +2.8 kB
Package (unpacked) 5.07 MB 5.07 MB +2.8 kB
Package (download) 1.52 MB 1.52 MB +1.2 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.8 ms 27.0 ms +0.2 ms
CLI --help 83.5 ms 85.6 ms +2.1 ms

@github-actions

github-actions Bot commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-05 14:11 UTC

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 27 files

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/provider-credential-fingerprint.ts Outdated
Comment thread src/daemon/server/http-server.ts
Comment thread website/docs/docs/browserstack.md Outdated
Comment thread src/provider-credential-fingerprint.ts Outdated
Comment thread src/provider-credential-fingerprint.ts Outdated
Comment thread website/docs/docs/limrun.md Outdated
Comment thread src/commands/schema/cli-help.ts Outdated
Comment thread src/daemon/handlers/lease.ts Outdated
Comment thread test/wire-compat/ledger.json Outdated
@thymikee

thymikee commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

Reviewed 27dd43a. The refusal logic itself looks right, but the Coverage check fails on this diff, so it needs a change before merge.

The eager-closure test in scripts/__tests__/eager-closure-budgets.test.ts fails on two edges this PR adds. providers.ts now imports AppError, which grows the providers subpath from 1 to 3 modules. src/cli.ts now reaches provider-limrun-credentials.ts through daemon-client.ts and the fingerprint module, which takes it from 295 to 297. Please make the gate pass without raising its budgets. The rule: providers.ts stays a leaf that only holds BROWSERSTACK_CREDENTIAL_VARIABLES, and the CLI loads the fingerprint module only for lease_allocate. Moving requireBrowserStackCredentials next to its consumers in provider-definitions.ts and loading the fingerprint lazily in sendToDaemon would do it. The Coverage run should then pass.

Not blocking: no test pins that the daemon digests its own startup env at daemon-runtime.ts:450, so reading {} there would still type-check. One startDaemonRuntime-level assertion would cover it. You can take or leave this.

Could the BrowserStack reader and its trimming rule live in one function that both the consumer and the fingerprint call? That would close the trim thread by construction and keep providers.ts a leaf. I also looked for a smaller owner than the per-request field. Putting the digest in daemon.json and checking it on the stale-daemon path would decide before any RPC. It writes a credential digest to disk and needs a restart policy, so I do not think it is smaller.

On the open threads, two P1s from cubic-dev-ai still apply: trim vs raw key mismatch and auth-hook strip on a local daemon. One P2 still applies: constructor provider lookup throws. Four P3 threads still apply. You can resolve two P2s: remote daemon wording does not apply because the help text says these providers only use local profiles, and fingerprint as authentication does not apply because it is only a staleness check and a caller that sends none already proceeds with the daemon's credentials.

I did not run the eager-closure test locally. I took the failure from the CI log and the diff's import edges. There was no live run with real Limrun or BrowserStack keys. The refusal fires before any provider call, and the socket and HTTP sendToDaemon test covers the client-to-daemon route, so I did not require one. The end-to-end connect limrun plus open refusal under a stale daemon is still unobserved. I also did not check whether a Limrun runtime registers in a daemon started without credentials, which decides whether the "no credentials" refusal or the provider-not-available refusal fires first.

No conflicts. Before merge, the eager-closure gate must pass, and the two held P1 threads need an answer. For the second, either cover the local auth-hook HTTP case or document it.

@thymikee

thymikee commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

Addressed in a0ac012. pnpm check:affected --run passes locally, including eager-closure-budgets.

  • Eager closure: providers.ts is a leaf again. It holds the BrowserStack variable names and a non-throwing readBrowserStackCredentials, with no imports. requireBrowserStackCredentials moved into provider-definitions.ts and is exported from the package index, which the CLI callers already load. sendToDaemon imports the fingerprint module only for a local lease_allocate, so src/cli.ts is back at its merge-base count.
  • Trim P1: closed by construction, as you suggested. The fingerprint hashes the values each provider's reader returns: BrowserStack's single reader, and Limrun's trimmed values through readLimrunCredentialValues.
  • Auth-hook P1: documented. An auth-hook daemon already treats every HTTP caller as remote (it strips developerDir and refuses path installs) and runs on its operator's credentials, so it skips the comparison. The strip comment, the help rule and the ledger rationale say so.
  • constructor P2: the provider lookup is now a Map.
  • P3s: fixed the docs wording, "rerun the command", the refusal text for a daemon without credentials (it now records none), and the ledger rationale. I replied to and resolved the two P2s you marked as not applicable.
  • Daemon env wiring: a new test in daemon-runtime-interactor-composition.test.ts boots a daemon with BrowserStack credentials and sends lease allocations over HTTP. A mismatch is refused and a match is not. It fails if the daemon reads {}.
  • Reachability: a Limrun daemon without credentials doesn't register the runtime, so the provider-not-available refusal fires first. The "no credentials" lease test therefore uses BrowserStack, which always registers.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 17 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/daemon/server/daemon-runtime-interactor-composition.test.ts
@thymikee

thymikee commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

The earlier findings at 27dd43a are fixed at a0ac012. The Coverage job that failed then now passes, because the change removes the providers.ts AppError edge and the static fingerprint import in daemon-client.ts. All 21 checks pass at this head, and I found no code problems in the delta. There are no conflicts.

Not blocking: verifyBrowserStack in src/cli/connection/connect-provider-adapters.ts:140 now calls requireBrowserStackCredentials. A profile missing a BrowserStack credential now fails with INVALID_ARGS and no reconnect hint, where it used to fail with COMMAND_FAILED and the hint to reconnect. An empty value is now refused too. You can note this in the PR body or rethrow the old error shape, or leave it as is.

On the open threads: the Cubic P2 about the test spreading process.env still applies (#3207 (comment)). Add AGENT_DEVICE_HTTP_AUTH_HOOK: '' to the env in daemon-runtime-interactor-composition.test.ts so a runner with the hook set does not strip the fingerprint and fail the test. These threads do not apply and you can resolve them. They are fixed at this head: #3207 (comment) (shared credential reader), #3207 (comment) (map lookup), #3207 (comment) (docs wording), #3207 (comment) (help text), #3207 (comment) (missing-credentials message) and #3207 (comment) (ledger rationale). These are benign by design: #3207 (comment) (stripping is now documented contract), #3207 (comment) (these providers use local profiles only) and #3207 (comment) (the fingerprint is a staleness check, not authentication).

I did not run the eager-closure budget test locally. That result rests on green CI and on the import edges I read in the diff. I also did not run connect plus open against a stale daemon with real Limrun or BrowserStack keys. The refusal fires before any provider call, and the startDaemonRuntime HTTP test covers the daemon composition, so I am not asking for that run.

Once the Cubic P2 is answered or applied, this is ready for a human review.

@thymikee
thymikee force-pushed the claude/provider-credential-fingerprint branch from a0ac012 to 9b3c7ca Compare October 5, 2026 05:57
@thymikee

thymikee commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

Rebased on main and addressed at 9b3c7ca. pnpm check:affected --run passes (4,984 related tests).

  • Rebase: squashed the branch into one commit and rebased onto main. The only conflict was in limrun.md, beside the fix(limrun): keep the owner's reverse mappings on attached Android instances #3204 port-reverse paragraph; both paragraphs are kept.
  • Cubic P2 (auth hook in the test env): fixed in the shared hermetic setup instead of one test. hermetic-env-setup.ts now clears AGENT_DEVICE_HTTP_AUTH_HOOK and _EXPORT along with the daemon URL and token. With the hook exported, the startup test fails without the scrub and passes with it.
  • verifyBrowserStack: it keeps the shared reader, but restores the old COMMAND_FAILED "profile missed …" error and reconnect hint. An empty value now counts as missing, as noted in the PR body.
  • I resolved the threads you listed. No review threads remain open.

@thymikee

thymikee commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

The earlier findings from a0ac012 are now fixed at 9b3c7ca, and I found no new problems. The ambient hook is cleared for all tests in the shared test setup, the credential fingerprint reads through the same reader every consumer uses, and the connect error path and docs now match the behavior.

CI is green: 21/21 checks pass at 9b3c7ca. The changes since the last review touch only the test setup, one connect error path and docs, and none of that overlaps a check route. I know of no conflicts. Nothing else needs to change before human review.

On the open Cubic threads, no P1 or P2 thread still applies. These are fixed at this head and can be resolved: #3207 (comment) (ambient hook in tests), #3207 (comment) (fingerprint credential reader), #3207 (comment) (reader map lookup), #3207 (comment) (limrun docs wording), #3207 (comment) (help wording), #3207 (comment) (missing-credentials message) and #3207 (comment) (ledger rationale). These are intended behavior and can be resolved too. #3207 (comment): a hook-configured daemon serves remote callers on its operator's credentials, so stripping the fingerprint is intended and documented. #3207 (comment): lease allocation from a shell goes through local daemons, and the docs say remote daemons are not checked. #3207 (comment): the digest is a staleness check for local callers and grants no authentication.

I read the code but did not run two things. I did not run the startup composition test with AGENT_DEVICE_HTTP_AUTH_HOOK exported, so the claim that it fails without the scrub rests on reading restrictRemoteHttpRequest. I also did not run a live connect plus lease_allocate against a stale daemon with real Limrun or BrowserStack keys. The refusal fires before any provider call, so that run is not needed.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 5, 2026
@thymikee
thymikee added this pull request to stack #3223 October 5, 2026 08:59
…rovider credentials

A local daemon keeps the environment it started with, while connect checks
the shell's. A stale daemon could act on old credentials, for example
creating a billed Limrun instance for an attach-mode shell. On a local
lease_allocate the CLI now sends a versioned hash of the values each
provider's own reader uses (Limrun, BrowserStack), over the socket and local
HTTP; the daemon compares it with the hash of its startup environment and
refuses with INVALID_ARGS, reason provider-credentials-changed, and a hint
naming its state dir. No hash is sent from a shell without credentials or to
remote, proxy, or cloud daemons; a daemon with an HTTP auth hook treats every
caller as remote. AWS Device Farm is excluded.
…HTTP auth hook in tests

verifyBrowserStack reads credentials through the shared reader again but keeps
its COMMAND_FAILED 'profile missed' error and reconnect hint. The hermetic test
setup now also clears AGENT_DEVICE_HTTP_AUTH_HOOK and _EXPORT, so a host with a
hook configured cannot turn a local daemon test into a remote one.
DaemonRequestMeta and toLeaseDaemonRequest now carry both the retainOnClose
fields from #3208 and providerCredentialFingerprint; their acks keep both
rationales and the digest of the merged declarations.
@thymikee
thymikee force-pushed the claude/provider-credential-fingerprint branch from 9b3c7ca to 3fccd86 Compare October 5, 2026 11:18
@thymikee

thymikee commented Oct 5, 2026

Copy link
Copy Markdown
Member Author

The earlier review at 9b3c7ca found no code problems, and the rebased head 3fccd86 still has none. The new changes only touch the lease handler and test/wire-compat/ledger.json, and they keep the rule that lease allocation is refused when the daemon holds other provider credentials.

All 21 checks pass at 3fccd86. That includes the wire-compat gate, which runs the route this change touched. I did not recompute the sha256 digests or rerun the lease handler tests locally after the rebase, so I relied on CI for both.

Nothing else is needed from you. The PR is ready for human review.

@thymikee
thymikee merged commit d0deeb8 into main Oct 5, 2026
21 checks passed
@thymikee
thymikee deleted the claude/provider-credential-fingerprint branch October 5, 2026 14:11
amankansal-lt added a commit to LambdaTest/agent-device that referenced this pull request Oct 5, 2026
Brings in callstack#3207 (refuse lease allocation when the daemon holds other
provider credentials) and callstack#3222.

Conflicts:
- provider-definitions.ts: keep main's requireBrowserStackCredentials and
  drop its local request helpers, which this branch moved to
  webdriver-utils.ts; requireEnv is no longer used here but stays exported
  from provider-webdriver/plugin for TestMu.
- cli-help.ts: keep the TestMu AI provider and credential text and add
  main's daemon credential comparison line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
amankansal-lt added a commit to LambdaTest/agent-device that referenced this pull request Oct 6, 2026
…dentials

A provider plugin can declare agentDevicePlugin.credentialVariables in its
manifest. providerCredentialFingerprint falls back to that declaration for
providers without a built-in reader, so the client (shell env) and the
daemon (startup env) both fingerprint plugin credentials without loading
plugin code. TestMu declares LT_USERNAME and LT_ACCESS_KEY, which brings
it under the provider-credentials-changed refusal from callstack#3207.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
thymikee added a commit that referenced this pull request Oct 7, 2026
* refactor(provider-webdriver): share hub upload and app-reference helpers

BrowserStack's app upload, install adapter, --provider-app resolution,
session-details URL artifacts, and orientation check were written for
one vendor. A second hosted Appium hub needs the same mechanics with a
different form field, reference scheme, and response shape, so move
them into shared helpers and have BrowserStack use them:

- webdriver-utils.ts: postHubAppUpload, createHubUploadApp,
  resolveHubAppReference, appFileUploadForm, asRecord,
  readProviderJsonBody, and requireProviderDeviceOrientation.
- artifact-results.ts: urlArtifactFromDetails.
- browserstack.ts: resolveBrowserStackAppReference, moved out of
  provider-definitions.ts.

Two small BrowserStack behaviour changes come with the shared code:

- An upload response that is not JSON (a gateway error page, an empty
  body) now fails with a typed COMMAND_FAILED that carries the HTTP
  status, instead of a raw JSON SyntaxError.
- The http(s) scheme of a --provider-app URL is matched
  case-insensitively, so HTTPS://... is passed through to the hub rather
  than being treated as a local path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(provider-webdriver): bound and type session-details lookups

The BrowserStack session-details lookup behind `artifacts` had no
deadline, so a stalled API call could hang the command indefinitely. A
transport failure or a body that was not JSON surfaced as an untyped
fetch or SyntaxError, and a JSON array passed the object check and was
read as session details.

Add fetchProviderSessionDetails to webdriver-utils.ts and use it for
BrowserStack. It sends basic auth with a 15 second deadline and reports
every failure as COMMAND_FAILED: a timeout or network error with a retry
hint and the original error as its cause, and a non-2xx answer or a body
that is not a JSON object with the HTTP status and the parsed response.

Connection verification gets the same treatment through
fetchProviderVerificationJson, which BrowserStack now uses in place of
its private fetch. Behaviour is unchanged: 401/403 is UNAUTHORIZED with
a credential hint, any other HTTP failure points at the provider's
service status, and a transport failure points at network access. A
new test pins the two non-credential hints. sameOsVersion moves
alongside it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(provider-webdriver): add TestMu AI emulator and simulator provider

Add `testmu`, a hosted WebDriver provider for TestMu AI (formerly
LambdaTest) virtual devices: Android emulators and iOS simulators behind
the TestMu AI Appium hub. It follows the BrowserStack shape:
`connect testmu` verifies and saves a local profile, and `open` creates
the hosted session. The service hostnames still carry the lambdatest.com
domain.

connect
- Reads LT_USERNAME and LT_ACCESS_KEY, the variables TestMu AI SDKs use.
- Checks the device and OS version against the public virtual-device
  catalog. The match is exact because the hub rejects `18` for a device
  listed as `18.0`; the error lists the versions the device offers.
- Uses the authenticated app listing as the credential check, and looks
  an lt:// id up in the list for the session's runtime (`emulator` or
  `simulator`). An id that is not listed is reported as configured,
  since TestMu AI validates it at session creation.
- Verifies against TESTMU_API_ENDPOINT when it is set, as the runtime
  does, and composes catalog and listing URLs with URL so a base or
  override that carries a query keeps it.
- Never creates a session.

Sessions
- Standard Appium keys stay `appium:`-prefixed; everything vendor-
  specific goes in `lt:options`, merged per key with any configured
  `lt:options`. `isRealMobile: false` and `w3c: true` are applied last,
  so configuration cannot move the session to another device pool or
  off the W3C dialect agent-device speaks.
- `appiumVersion` is sent only when --provider-appium-version pins one;
  otherwise TestMu AI starts its default server for the device.
- Orientation, geo-location, timezone, Appium version, language, and
  locale map onto `lt:options` through a table. The BrowserStack-only
  network-profile, custom-network, and no-resign flags are refused by
  flag name at connect and at session preparation, instead of being
  silently dropped.

Apps
- --provider-app takes an lt:// id, an http(s) URL, or a local path. The
  hub only accepts lt:// references, so a URL is handed to the upload
  API to fetch (`storage=url`) and a local file is uploaded.
- An unzipped iOS `.app` directory is rejected before any request, with
  a hint to zip it.
- Only a well-formed lt:// reference or app id in the upload response
  counts as success.

Artifacts
- Read from the session-details API through the shared bounded lookup.
  A 404 reads as pending until TestMu AI publishes the details.
  `console_logs_url` is the device log on virtual devices.
- Session video, Appium, network, and command logs, screenshots, and
  the dashboard link are returned once at least one artifact URL exists.

TESTMU_WEBDRIVER_ENDPOINT, TESTMU_APP_UPLOAD_ENDPOINT, and
TESTMU_API_ENDPOINT redirect the endpoints.

The session, upload, verification, and device-feature modules are loaded
with dynamic import, so the package entry's eager module graph does not
grow. The CLI reaches the device-feature checks through a new
`./testmu-device-features` subpath export. A .fallowrc entry covers the
exports that are read only through the dynamic import. `connect testmu`
is listed in the connect usage, the remote help topic, and the artifacts
provider description.

Co-authored-by: gautam-jain-dev <gautamj@lambdatest.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(provider-webdriver): support TestMu AI real devices

TestMu AI serves real devices and virtual devices from the same Appium
hub; `lt:options.isRealMobile` selects the pool. Add
`--provider-device-type real|virtual` to choose it. The default is
`virtual`, so existing `testmu` profiles and sessions keep their current
behaviour.

The value is a cloud provider profile field like the others. It is
defined in contracts as PROVIDER_DEVICE_TYPES and carried through the
lease_allocate projection, request overrides, the remote-config schema,
`connect`, the CLI request flags, and the session doctor's provider
keys. It reaches session preparation from a `connect testmu` profile and
from `client.leases.allocate({ providerDeviceType })`.

For `real`, TestMu AI sessions:
- Set `isRealMobile: true`. It is applied after any configured
  `lt:options`, so configuration cannot switch pools.
- Upload through the real-device upload API. It has its own override,
  TESTMU_REAL_DEVICE_APP_UPLOAD_ENDPOINT; TESTMU_APP_UPLOAD_ENDPOINT
  keeps redirecting virtual-device uploads only. An `.app` directory is
  refused with a hint to pass a signed .ipa.
- Verify against the real-device catalog
  (`capability/generator?isVirtualDevice=false`). It lists devices
  directly under the platform key rather than under `app.devices`. The
  OS version must still match exactly, and real iOS devices are listed
  by major version, such as `18`. A catalog response without the
  expected pool shape fails with a typed error.
- Look an lt:// id up in the real-device app list (`type=android` or
  `type=ios`), the same way virtual uploads are looked up under
  `emulator` or `simulator`.

BrowserStack, AWS Device Farm, and Limrun refuse the flag at connect.
BrowserStack and AWS Device Farm also refuse it at session preparation,
which the typed client and hand-written profiles reach without
`connect`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(provider-webdriver): upload the archive of materialized iOS builds

Install from a remote source materializes an iOS build by extracting the
`.app` bundle from a zipped simulator build or an .ipa. The WebDriver
deployment runtime handed that extracted `.app` directory to the
provider's uploader. Hosted upload APIs take a file, not a directory, so
the upload could not succeed.

When the materialized artifact is an iOS `.app` extracted from a `.zip`
or `.ipa` archive, upload the archive it came from. Every other case
uploads the installable path as before: no archive, an archive of
another type, or an Android build. A provider without an uploader still
installs the extracted bundle path. The bundle id and launch target
hints are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: add the TestMu AI device cloud guide

Add a TestMu AI guide next to the BrowserStack and AWS Device Farm ones
and link it from the sidebar. It covers credentials and `connect`, the
exact device and OS version match, app references and uploads, device
features, real devices with `--provider-device-type real`, the CLI and
Node.js client workflows, artifacts, and endpoint overrides.

List TestMu AI among the device clouds in the README, the device clouds
overview, the client API page, and the command reference.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(daemon): start a lease's TTL when its allocation completes

The registry records a lease, and stamps its expiry, before the lease
lifecycle provider allocates the session behind it. Hosted providers can
spend 30-80 seconds creating that session, so a lease on the default
one-minute inactivity window was expired, or nearly so, by the time its
client received it: the next command failed with "Lease is not active",
release reported nothing to release, and the paid provider session was
left running. The expiry sweep could also reap the lease while the
provider was still allocating.

Hold a lease work pass for the duration of the provider allocation, the
same protection admitted request work gets. The lease cannot expire
underneath the allocation, and ending the pass while the requester is
still waiting renews the lease for its own TTL from that moment. The
response now carries the renewed lease. A requester that hung up still
protects nothing, and its allocation is released as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(provider-webdriver): declare the profile fields each provider consumes

Lease allocation forwards every provider profile field, but only some
routes checked which ones a provider could act on. The connect builders
and the BrowserStack/AWS session preparation refused flags they did not
own, while the Limrun lease runtime and a remote-config profile skipped
that check. A typed client asking Limrun for providerDeviceType 'real'
was therefore given a simulator without an error.

Each lease provider now declares, for every Cloud provider profile field,
whether it consumes or refuses it. The declaration is a total record over
CloudProviderProfileFields, so adding a field fails to build until every
provider decides. One refusal, derived from that declaration, runs at
lease preparation (the WebDriver session manager for BrowserStack, AWS
Device Farm and TestMu AI, and Limrun allocation) and at connect, so
connect, leases.allocate and remote-config profiles all fail the same
way with INVALID_ARGS naming the flag and the provider.

This replaces the per-provider TestMu-only, TestMu-unsupported and
BrowserStack-only checks, the hand-copied complement of the BrowserStack
feature table, and the ./testmu-device-features package subpath. AWS
Device Farm now also refuses the hub-only app, OS version, project and
build flags it never read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(provider-webdriver): let the Apple materializer name the uploadable file

The WebDriver deployment runtime guessed what to upload for a materialized
iOS build from file extensions: an extracted .app with a .zip or .ipa
archive uploaded that archive. The archive it saw is the outermost one the
source arrived as, so a URL .zip wrapping an .ipa uploaded the wrapper zip
instead of the .ipa, and a zip holding a .app.tar.gz uploaded a zip with no
app bundle at its top.

Materialization now records the archive an installable was extracted from
directly, and the Apple materializer declares the file a hosted provider
uploads: the .ipa itself, or the zip a simulator .app was extracted from.
It names nothing when no such file exists. The deployment runtime uploads
the declared file and otherwise the installable path, with no inference of
its own.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(provider-webdriver): tighten app-reference, endpoint and upload input handling

Several inputs reached a hosted provider in a shape it could not use, or
failed with an untyped error:

- An empty `lt://` passed as a TestMu AI app reference and reached session
  creation. A reference is now validated against the app-id grammar, and
  an invalid one is INVALID_ARGS on connect and at session preparation.
- `LT://APP1` or `BS://id` was treated as a file path. App schemes now
  match case-insensitively and are canonicalized to the spelling the hub
  accepts.
- A directory given as `--provider-app` reached the upload and failed
  with a raw EISDIR; it is now INVALID_ARGS with a hint.
- A missing path given straight to the TestMu AI upload failed with a
  bare ENOENT; it now gets the same typed refusal as a directory.
- Session-detail URLs were built by string concatenation, so a query on
  TESTMU_API_ENDPOINT or the BrowserStack details endpoint swallowed the
  route. Routes are now appended to the URL path, keeping the query.
- A whitespace-only artifact URL was reported as a ready artifact; it is
  now treated as absent, and URLs are trimmed.
- A 2xx connection-verification answer that is not JSON was reported as
  a network failure. It is now COMMAND_FAILED with the HTTP status, and
  each hub supplies its own service-status hint.

The connect help and the TestMu AI guide now say what connect checks for
`--provider-app` and what is only validated when the session is created.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(provider-webdriver): exercise in-flight deadlines and aborts for real

The BrowserStack session-details timeout test threw a prebuilt
TimeoutError from the fetch stub, so it passed even with the deadline
removed. The fetch now stays pending until the signal the lookup armed
aborts, and the test checks that signal is the 15 second deadline.

The TestMu AI upload cancellation test aborted before the upload reached
fetch, so it never covered a request in flight. It now waits for the
stub to start before aborting.

The connect suites shared a copied connectWithGeneratedProviderProfile
helper; it now lives in the shared test utilities.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(provider-webdriver): refuse profile fields on a repeat allocation too

The daemon hands a run's repeat lease_allocate the lease it already
holds, and both the WebDriver runtime and the Limrun runtime returned
the live session for a known lease before checking the request's
profile fields. A second allocation on the same run that set a refused
field, such as providerDeviceType on BrowserStack, therefore succeeded.

Both runtimes now refuse before reusing a live session. Because a
refusal of that repeat request is a provider allocate failure, the lease
handler no longer releases a lease it reused when allocate throws; doing
so would end the run's lease and leave the session the first allocation
created without an owner.

The runtime's profileFields option is now required, and each provider
definition passes its own declaration into createRuntime, so a provider
that forgets to wire its declaration fails to compile instead of
silently refusing nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(provider-webdriver): validate and canonicalize app references at connect

Connect only checked that an lt:// or bs:// reference had something after
the scheme, so `lt://a b` and `lt://a/b` were saved and failed only when
the hub created the session. Both hubs' reference grammars now live in
the light providers module, and connect, session preparation and the
TestMu AI upload reader all validate against them.

Connect also verified the raw spelling typed on the command line: the
generated profile stored `lt://APP1` for `LT://APP1`, but the flags
handed to verification were the raw CLI flags laid over the profile, so
the uploaded-app lookup missed and reported a local artifact. The
canonical reference now wins in the flags verification reads.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(provider-webdriver): refuse a directory before any hosted upload

A materialized build that names no uploadable file falls back to its
installable path, which for iOS is the extracted .app directory. The
deployment runtime handed that to the hub's uploader, and BrowserStack's
read it as a file and failed with a raw EISDIR.

The deployment runtime now refuses a directory before calling any hub's
uploader, with INVALID_ARGS naming the provider and path and a hint to
zip the .app or pass the .ipa, .apk, or .aab.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(cli): refuse a non-URL install-from-source source with a typed error

install-from-source sends its positional as a URL source. A local path
reached the iOS trusted-host check, where `new URL()` threw a TypeError
that surfaced as an untyped UNKNOWN "Invalid URL".

The CLI now refuses a positional that is not an http(s) URL with
INVALID_ARGS and points at `install <app> <path>` for local builds, and
the trusted-host check reports an unparsable source as INVALID_ARGS
instead of throwing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat: support provider-owned plugin connections and WebDriver adapters

* chore(gates): expose the optional WebDriver plugin configuration

* feat: load TestMu from its separately installed provider plugin

* chore(gates): declare TestMu plugin build dependencies and analysis entrypoints

* fix: retain plugin tests through development-only package exports

* chore(gates): track TestMu packaging fixtures and workspace test dependencies

* chore(gates): smoke the optional packed WebDriver plugin SDK

* perf: load the shared WebDriver engine on demand

* chore(gates): bound the optional plugin SDK declarations

* fix: consolidate TestMu plugin helper imports

* fix(testmu): trim credentials and log URLs, canonicalize lt:// everywhere

Whitespace-only LT_USERNAME/LT_ACCESS_KEY now fail as missing in the
runtime, matching connect. Verification of a hand-authored profile,
upload responses, and app-list ids share one case-insensitive lt://
canonicalization, and a blank console_logs_url falls back to
device_logs_url.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(testmu): export credentials in help and note the fixed app listing

The help flow's bare assignments did not reach the next agent-device
process. The endpoint overrides do not move the app listing connect
uses for the credential check, so say so instead of implying a private
deployment is fully redirectable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(kernel): require a real Error behind the AppError brand

A plain object carrying the global brand symbol passed instanceof
AppError and left normalizeError with no code or message. Package copies
share the realm's Error.prototype, so the plugin-copy case still holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): refuse a primitive factory result as an invalid plugin

'webDriver' in <primitive> threw a raw TypeError past the loader's
INVALID_ARGS validation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(plugins): archive into a fresh zip, not over a stale one

zip -r updates an existing archive in place, so leftover entries from an
earlier attempt would ride along in the upload.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(connect): look plugin capabilities up in the caller's environment

isConnectProviderName already took an env; connectionProviderCapabilities
read process.env, so under another plugin home the two disagreed and a
plugin provider fell back to the builtin capability shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(kernel): type the AppError brand lookup on a narrowed Error

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): reserve provider ids from one list

assertUniquePluginProviders merged RESERVED_PLUGIN_PROVIDERS into
whatever its callers passed, and the daemon passed the overlapping
DEFAULT_PROVIDER_RUNTIME_REQUIRED_IDS. Every caller now passes
RESERVED_PLUGIN_PROVIDERS, which takes the WebDriver ids from
CLOUD_WEBDRIVER_PROVIDERS, and a test pins every bundled runtime id
into it. plugins add/update no longer loads the provider runtimes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): name the WebDriver factory result check

Keeps instantiateProviderPlugin under the Fallow complexity threshold
after the object guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(fallow): stop suppressing statically imported TestMu helpers

connection.ts imports readTestMuDeviceFeatureFields and
readTestMuDeviceType statically, so Fallow sees them; only the lazily
read buildTestMuDeviceFeatureCapabilities still needs the exemption.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: allocate through a WebDriver plugin and assert typed ownership

The WebDriver plugin fixture now declares its profile fields and the
test allocates a session through the shared engine, refusing a field it
marks refused before any request. The refused repeat allocation also
asserts session-1 still resolves to the live lease, and the Limrun
refusal checks details.provider and details.flags instead of copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(connect): refuse a WebDriver plugin's refused fields before resolve

Builtin connect routes run rejectRefusedProviderProfileFields against
their declaration; the plugin route left it to the plugin. A
{ webDriver } plugin already hands core a total profileFields
declaration, so the connect route now applies it before the plugin's
resolve. Raw runtime plugins carry no declaration and still validate
their own flags.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(testing): say when to run the packed provider plugin check

check-provider-plugin.mjs needs pnpm and a built core, which the Node
22.12 package lane cannot run and whose script body CI credits
verbatim, so it is a manual pre-push check for plugin package changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: document the plugin package check in its script, not testing.md

testing.md is over the agent-guidance byte budget with the note, so the
how and when now sit in the script header.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* revert: let resolve and allocate own plugin profile-field refusal

Reverts 62e814c. The maintainer judged the connect-route check
redundant: plugin resolve and lease allocation both refuse those fields,
and #3168 owns profile-field admission.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* revert: keep the AppError brand check as it was

Reverts c553ed7 and aab5684. Plugins are trusted in-process code, so
a forged brand is not a threat the maintainer wants handled here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* revert: drop the plugin host archive fix with the apple host block

Reverts 1d52862; ProviderPluginHost.apple is removed next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): remove the unused apple block from the plugin host

No plugin calls host.apple, and archiveDirectory copied the Limrun
dependency's zip helper. A later plugin that needs Apple packaging can
share one extracted helper with Limrun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(webdriver): one owner for lease-flag and credential readers

requireRequest, requireRequestPlatform, requireFlag, readFlag and
requireEnv move from provider-definitions.ts into webdriver-utils.ts and
are exported from the provider-webdriver/plugin subpath; the TestMu
plugin drops its copies. requireEnv keeps the trim-empty rule, so a
whitespace-only BrowserStack credential now also fails as missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(testmu): import session and device-feature modules statically

connection.ts already imports them statically, so the dynamic imports
in plugin.ts saved nothing; the Fallow exemption for the lazy read goes
with them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build(testmu): typecheck the plugin against core source, not dist

CI typechecks before it builds, so agent-device/plugins and
agent-device/plugins/webdriver resolved to dist types that did not
exist yet and ProviderPluginHost became any. Both tsconfigs map the two
specifiers to src/sdk, as examples/sdk does; provider-testmu maps
through its workspace link so the files stay outside its rootDir.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(fallow): credit the testmu plugin's type-only agent-device link

With the tsconfig mapping to core source, Fallow resolves the plugin's
agent-device/plugins imports to src/sdk files and reported the
devDependency unused, which a clean checkout surfaced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(testmu): assert only Limrun's device-type refusal at connect

Main's refusal names a field's aliases too, so the combined message this
test matched no longer exists, and main already covers Limrun refusing
the OS version. Keep the part this PR adds: --provider-device-type is
refused by Limrun.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(providers): share the lower-case scheme rule for hub app references

canonicalTestMuAppReference repeated the rule canonicalBrowserStackAppReference
applies: a hub matches only the lower-case scheme, so BS:// and LT:// are
rewritten to bs:// and lt://. Both now call canonicalSchemeReference from the
dependency-free provider-webdriver/providers subpath, so neither eager
closure grows. Behaviour is unchanged: BrowserStack still returns undefined
without the scheme, and TestMu still returns the value as given.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(provider-webdriver): export the plugin's webdriver-utils readers in one block

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(providers): refuse a lease when the daemon holds other plugin credentials

A provider plugin can declare agentDevicePlugin.credentialVariables in its
manifest. providerCredentialFingerprint falls back to that declaration for
providers without a built-in reader, so the client (shell env) and the
daemon (startup env) both fingerprint plugin credentials without loading
plugin code. TestMu declares LT_USERNAME and LT_ACCESS_KEY, which brings
it under the provider-credentials-changed refusal from #3207.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(plugins): split manifest connection and credential checks

Keeps readPluginManifest under the fallow complexity threshold, and types
the manifest fixture's version so the typecheck passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs: explain installation of the TestMu package

* chore(gates): align TestMu package with workspace releases

* docs: explain daemon restart after plugin installation

* docs: show the complete plugin connection manifest

* chore(gates): deduplicate the merged package boundary fixture import

---------

Co-authored-by: Anurag Sharma <38938168+anurag-lambdatest@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant