Repository navigation
chore: sync with tester-army/e2e main and use the @agent-device/testmu plugin - #6
Merged
amankansal-lt merged 83 commits intoOct 12, 2026
Conversation
…-army#894) Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
Six test files created a temp root and never deleted it, so a run left the whole set in tmpdir: e2e-step-cache, e2e-flush, e2e-flush-derived, e2e-rekeyed, e2e-trace-cache, e2e-sessions, e2e-globs and crosscheck, up to 2 GB of orphans on this machine before anyone noticed. last-run.test.ts had the subtler leak: one module-level `dir` that `reportFile` overwrote, so a case calling it twice dropped the first root on the floor and no afterEach could see it. It now keeps the roots in a list. Measured with the delta of `os.tmpdir()` entries before and after the suites, which is the only gate that sees this: `pnpm --filter e2e run test:unit` and `pnpm --filter @e2e-dev/web run test:unit` both leave 0 behind, and each file leaks its own count when its cleanup is reverted.
…army#902) * fix(init): retain MCP server environment and client settings Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com> * test(init): preserve top-level settings during stale repair Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com> --------- Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com>
* fix(web): infer row headers beside table data cells Signed-off-by: Rudy Celekli <rudy@gradiahq.com> * docs(screen): qualify the scope-less header exception Signed-off-by: Rudy Celekli <rudy@gradiahq.com> --------- Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
…ted models at config load (tester-army#916)
…-army#929) Every engine minor re-keyed all committed recordings, so each `chore: version packages` merge failed the strict web agent job with REPLAY_STALE on main (tester-army#825 after tester-army#774). The key keeps the engine's name and SPI version; a release of the same engine is caught at replay, where each action re-finds its node. The 30 web benchmark entries move to their new hashes by a re-key verified against the old ones.
…er-army#907) Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com> Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
…er-army#898) Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
…er-army#899) Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com> Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
…th (tester-army#968) The Video snippet took src as a prop. Mintlify never rewrites a custom component's props, so /images/bug-bash.mp4 resolved to the site root and 404'd on e2e.tester.army, where the docs live under /docs. The pages now write a native <video> inside the wrapper, which Mintlify resolves like an <img>; the wrapper keeps the frame, the caption, and reduced-motion-aware autoplay.
…reference (tester-army#969) Reorder the page by what a user does (run, write a goal, read the results, sign in, raise the budget) and rewrite it in a plain, simplified-English voice. The verdict table, per-step budgets, stuck rule, severity scale, artifact directory, and notices move to the e2e explore section of the CLI reference; the credentials paragraph moves to the security page, which linked to the removed anchor. Also corrects two claims: with no credentials the explorer signs in with made-up ones, and the artifact directory digest covers the target and agent too.
Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com> Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
…ester-army#904) Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com> Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
…r-army#905) Signed-off-by: Rudy Celekli <47457359+rudycelekli@users.noreply.github.com> Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
…ble smol machines microVMs (tester-army#956)
* feat(locator): add dblclick as an alias of doubleTap Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * chore(changeset): release the dblclick alias as a minor Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Rudy Celekli <rudy@gradiahq.com>
tester-army#1001) The smol page icon pointed at the full-color card logo in /images/integrations/, a white tile the sidebar filter in style.css never touches. Add a transparent black silhouette under /images/icons/, cropped and thickened to match the other marks, and document the page icon rules in the authoring-docs skill.
…ng-docs and ship-pr (tester-army#1004) * docs(skills): add a shortening pass to authoring-docs Measure the page against its sidebar neighbors, then cut what the reader doesn't act on, merge what is said twice, and fold what only some readers need. When asked how to shorten a page, show the plan with savings and a recommendation before editing. Lists the claims a shortening pass tends to break: a caveat cut from its claim, a merged sentence that is wrong, a leftover fact parked under the nearest heading. * docs(skills): make the shortening pass the first, mandatory rule in authoring-docs Move it above Scope, mark it important, and require it on every page written or edited, new pages and small edits included. The commit checklist opens with it. * docs(skills): run the authoring-docs shortening pass on touched pages in ship-pr * docs(skills): address review of the shortening pass Scope the pass to the touched sections on a PR that isn't about docs, keep the skeleton's 2-4 Next cards and a bulleted Before you start, shorten before the fact comparison in a rewrite, and name every cut fact in the PR body. ship-pr runs the pass before the checks and links to it instead of restating it.
… rendered row; mobile benchmark re-recorded, Android strict (tester-army#998)
…h the OpenAI Decisions API (tester-army#965)
…s reference (tester-army#1008) The guide answers what most readers ask: how secrets stay out of the model, what can still leak, and what runs with your permissions. Every rule moves to docs/reference/security.mdx. smol host-loopback access moves to the smol page's warning. Fixes facts that no longer matched src/: defineTool errors are INVALID_CONFIG, the CI cache default skips a cache.store, start-screen replay matches routes, and outbound connections now list login, models, init, and the smol image pull.
The footer reads "Built by TesterArmy | Powered by Mintlify" on one line, GitHub alone on the left; on a phone the Mintlify credit takes a second, right-aligned line. Mintlify's footer takes no markup, so the credit is a website social restyled in style.css with the brand kit's mark and wordmark (images/footer/). Each image is two url() layers, one path for mint dev and one for production's /docs prefix, since Mintlify inlines the stylesheet into the page.
…ll (tester-army#1019) * docs(agent-steps): rewrite with the authoring-docs skill * docs(agents): rewrite with the authoring-docs skill * docs(api-testing): rewrite with the authoring-docs skill * docs(authentication): rewrite with the authoring-docs skill INVALID_CONFIG also fires for a password shorter than 6 characters. * docs(browser): rewrite with the authoring-docs skill * docs(bug-bash): rewrite with the authoring-docs skill * docs(cache): rewrite with the authoring-docs skill * docs(ci): rewrite with the authoring-docs skill Rerun folders are rerun-<n>/, not rerun-1/. Merge exit codes into Gate pull requests. * docs(ci/bitrise): rewrite with the authoring-docs skill * docs(ci/codemagic): rewrite with the authoring-docs skill * docs(ci/eas-workflows): rewrite with the authoring-docs skill * docs(ci/github-actions): rewrite with the authoring-docs skill * docs(coding-agents): rewrite with the authoring-docs skill * docs(core-concepts): rewrite with the authoring-docs skill * docs(debugging): rewrite with the authoring-docs skill * docs(decision-models): rewrite with the authoring-docs skill A point tap costs three decision calls and one text call, not four. * docs(example-projects): rewrite with the authoring-docs skill * docs(executors): rewrite with the authoring-docs skill * docs(explore): rewrite with the authoring-docs skill * docs(github): rewrite with the authoring-docs skill * docs(index): rewrite with the authoring-docs skill * docs(integrations/eas): rewrite with the authoring-docs skill * docs(integrations/index): rewrite with the authoring-docs skill * docs(integrations/kernel): rewrite with the authoring-docs skill * docs(integrations/smol): rewrite with the authoring-docs skill * docs(migrate/cypress): rewrite with the authoring-docs skill * docs(migrate/detox): rewrite with the authoring-docs skill * docs(migrate/maestro): rewrite with the authoring-docs skill Fold the command map tables into accordions. * docs(migrate/playwright): rewrite with the authoring-docs skill List use and testMatch among the Playwright keys e2e rejects. Fold the mapping tables into accordions. * docs(migrate/selenium): rewrite with the authoring-docs skill * docs(mobile): rewrite with the authoring-docs skill Fix the Android installApp troubleshooting entry: agent-device 0.21.22 parses the manifest itself and searches ~/Library/Android/sdk on macOS. * docs(models): rewrite with the authoring-docs skill * docs(quickstart): rewrite with the authoring-docs skill * docs(security): rewrite with the authoring-docs skill Name smol beside Kernel and EAS Simulators among the integrations that make outbound connections. * docs(subscriptions): rewrite with the authoring-docs skill npx e2e login with no provider opens a picker. * docs(telemetry): rewrite with the authoring-docs skill Add the source-checkout, unwritable-preferences, and CI opt-out rules. * docs(tools): rewrite with the authoring-docs skill Config load fails only when an entry sets both tools and a custom executor. * docs(visual-testing): rewrite with the authoring-docs skill * docs(web): rewrite with the authoring-docs skill Fold the Next.js dev-origins section, merge the basicAuth examples, and fix the reuseExisting row: in CI e2e ignores it and only a server that already answers is APP_ALREADY_RUNNING. Drop the unrelated agents block from the command config example. * docs(writing-an-engine): rewrite with the authoring-docs skill Remove the origin-check claim for location; PR tester-army#290 removed origin policy. * docs(models): keep process.loadEnvFile() as a way to load the pulled OIDC token * docs(mobile): restore the imports in the several-devices and agent-tools configs * docs(agent-steps): shorten Prose words 841 -> 764. * docs(agents): shorten Prose words 789 -> 675. * docs(api-testing): shorten Prose words 270 -> 266. * docs(authentication): shorten Prose words 1000 -> 849. * docs(browser): shorten Prose words 1934 -> 1697. * docs(bug-bash): shorten Prose words 615 -> 585. * docs(cache): shorten Prose words 2793 -> 2374. * docs(ci): shorten Prose words 1338 -> 1182. * docs(ci/bitrise): shorten Prose words 321 -> 306. * docs(ci/codemagic): shorten Prose words 259 -> 250. * docs(ci/eas-workflows): shorten Prose words 431 -> 394. * docs(ci/github-actions): shorten Prose words 544 -> 453. * docs(coding-agents): shorten Prose words 841 -> 735. * docs(core-concepts): shorten Prose words 686 -> 620. * docs(debugging): shorten Prose words 1826 -> 1604. * docs(decision-models): shorten Prose words 1003 -> 942. * docs(example-projects): shorten Prose words 347 -> 288. * docs(executors): shorten Prose words 1297 -> 1114. * docs(explore): shorten Prose words 451 -> 413. * docs(github): shorten Prose words 723 -> 626. * docs(index): shorten Prose words 387 -> 333. * docs(integrations/eas): shorten Prose words 1293 -> 1119. * docs(integrations/index): shorten Prose words 174 -> 163. * docs(integrations/kernel): shorten Prose words 794 -> 620. * docs(integrations/smol): shorten Prose words 924 -> 855. * docs(migrate/cypress): shorten Prose words 957 -> 890. * docs(migrate/detox): shorten Prose words 1073 -> 1021. * docs(migrate/maestro): shorten Prose words 2464 -> 2246. * docs(migrate/playwright): shorten Prose words 4835 -> 4546. * docs(migrate/selenium): shorten Prose words 933 -> 887. * docs(mobile): shorten Prose words 2912 -> 2595. * docs(models): shorten Prose words 1187 -> 1048. * docs(quickstart): shorten Prose words 980 -> 946. * docs(security): shorten Prose words 587 -> 559. * docs(subscriptions): shorten Prose words 694 -> 651. * docs(telemetry): shorten Prose words 2158 -> 1999. * docs(tools): shorten Prose words 460 -> 397. * docs(visual-testing): shorten Prose words 583 -> 566. * docs(web): shorten Prose words 1262 -> 1104. * docs(writing-an-engine): shorten Prose words 1731 -> 1522. * docs(agent-steps): address review findings * docs(agents): address review findings * docs(authentication): address review findings * docs(cache): address review findings * docs(ci): address review findings * docs(coding-agents): address review findings * docs(debugging): address review findings * docs(example-projects): address review findings * docs(executors): address review findings * docs(index): address review findings * docs(integrations/eas): address review findings * docs(migrate/maestro): address review findings * docs(subscriptions): address review findings * docs(telemetry): address review findings * docs(tools): address review findings * docs(web): address review findings * docs(writing-an-engine): address review findings
Conflicts resolved by keeping upstream's text and re-adding the TestMu AI device and browser entries: the integrations list and nav (after smol and Limrun), the browser provider table, the hosted devices section, the setup reference, the integration peer note, and the build and test scripts. Upstream moved the outbound connections list to docs/reference/security.mdx, so the TestMu AI CDP hub row moves there. pnpm-lock.yaml is upstream's, regenerated with pnpm install for packages/testmu. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
agent-device 0.21.22's plugin loader accepts only a factory that returns
a native provider runtime ({ runtime, platformModule }). The published
@agent-device/testmu plugin returns a WebDriver plugin ({ webDriver,
connection }) and declares credentialVariables, which the loader learned
in 0.21.25. @e2e-dev/testmu must share @e2e-dev/mobile's agent-device, so
mobile moves to 0.21.25. @e2e-dev/limrun pins the same version as mobile
and moves with it; @e2e-dev/testmu's peer floor and dev copy follow.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The engine closes a worker's agent-device session when it retires the worker, and closing the session ended the lease and with it the TestMu AI session while the run still held the device. Allocate with retainOnClose, so the lease ends only on release, expiry, or daemon shutdown, and refuse (releasing it) a lease the daemon does not say it keeps. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lugin agent-device ships TestMu AI as a plugin since 0.21.25, so the setup no longer overrides @e2e-dev/mobile's agent-device to an unpublished build: install agent-device at mobile's version and run `agent-device plugins add @agent-device/testmu`, with a daemon restart for one already running. A daemon without the plugin grants a testmu lease without starting a session and the first command fails with DEVICE_NOT_FOUND; the docs name that symptom, since the client does not expose whether a plugin served the allocation. The @e2e-dev/mobile peer starts at 0.11.1, the release that moves to agent-device 0.21.25. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
14 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This brings LambdaTest/e2e
mainup to date with tester-army/e2emain(78 commits), clearing tester-army#772's conflict. It also moves@e2e-dev/testmuonto the published TestMu AI plugin,@agent-device/testmu@0.21.25, now that callstack/agent-device#3118 has merged.Please merge with "Create a merge commit", not squash or rebase, so
mainfast-forwards and tester-army#772 picks up upstream's history.71176d57). Nine files conflicted:CONTRIBUTING.md,docs/browser.mdx,docs/docs.json,docs/integrations/index.mdx,docs/mobile.mdx,docs/security.mdx,package.json,pnpm-lock.yamlandskills/e2e/references/setup.md. Upstream's text wins in each, and only the TestMu AI device and browser entries are re-added. The outbound-connections row follows upstream's move todocs/reference/security.mdx. The lockfile is regenerated.38494793). agent-device 0.21.22's plugin loader only accepts{ runtime, platformModule }, but@agent-device/testmu@0.21.25returns{ webDriver, connection }, which only 0.21.25's loader handles. So@e2e-dev/mobilemoves to agent-device 0.21.25, and so does@e2e-dev/limrun, because it must share mobile's copy. A patch changeset explains the bump.d58b42da,bd2ec29c).testmu()allocates withretainOnClose: true(feat(daemon): keep a caller-owned lease through session close with retainOnClose callstack/agent-device#3208). e2e closes a session when it retires a worker, and without this, closing ends the TestMu AI session mid-run.@e2e-dev/limrunalready handles this the same way.3bbb8e80). The overrides and the version placeholder are replaced with installing agent-device 0.21.25 and runningagent-device plugins add @agent-device/testmu. The docs also cover restarting the daemon and the error you get when the plugin is missing.Validation
pnpm install --frozen-lockfilepasses.pnpm checkpasses: lint, dead code, typecheck, error codes, peer ranges, install scripts, docs and broken links.@e2e-dev/testmu: 109 passed@e2e-dev/mobile(on agent-device 0.21.25): 236 passed@e2e-dev/web: 543 passed, 1 skipped@e2e-dev/limrun: 14 passedgit merge-treeagainst tester-armymain(c807f948) is clean.🤖 Generated with Claude Code