Fix Frontend testing documentation drift

docs/developer-guide/160-frontend-testing.md

@@ -12,7 +12,7 @@ All frontend testing code is situated under the `ui` directory.
 
 We use different Playwright configurations for `app` and `component` testing.
 
-- `ui/test/app.config.ts`: App test configuration
+- `ui/test/app.config.cts`: App test configuration
 - `ui/test/component.config.ts`: Component test configuration
 
 We do this because we modify the base configuration that comes with Playwright so that component testing works, however this configuration is incompatible with app testing thus we use 2 configurations.
@@ -21,7 +21,7 @@ We do this because we modify the base configuration that comes with Playwright s
 
 Both the `app` and `component` tests depend on the `@openremote/test` package which includes shared fixtures, configurations and our Playwright component testing plugin.
 
-The `shared` fixtures in the test package are meant for general test utilities like intercepting requests. Besides this there are also shared component only test fixtures under `CtShared`.
+The `shared` fixture is available to every test. Apps get a `Shared` instance with general test utilities like intercepting requests. Components get a `CtShared` instance, which adds utilities for the component test page, such as serving the icon fonts and translations that the manager provides in a running app.
 
 Each project that needs testing should configure its own Playwright configuration file which must reuse the above-mentioned configurations.
 
@@ -34,7 +34,7 @@ The app tests are used to test the app UI (End-to-End).
 #### Configuration
 
 - **Target:** Any app in the `ui/app/*` directory.
-- **Worker Scope:** Single worker (to avoid tests interfering with one-another).
+- **Worker Scope:** A single worker for the whole app config (to avoid tests interfering with one-another).
 - **Code reuse:** Apps may include a `fixtures` directory with test and data fixtures, and reuse fixtures from components they depend on.
 - **Setup & Teardown:** App test projects should depend on `*.setup.ts` and `*.cleanup.ts` project files to provision realm(s), user(s) and collect authentication states for more robust and performant tests.
 
@@ -45,12 +45,10 @@ function createAppSetupAndTeardown(app) {
       name: `setup ${app}`,
       testMatch: "**/*.setup.ts",
       teardown: `cleanup ${app}`,
-      worker: 1,
     },
     {
       name: `cleanup ${app}`,
       testMatch: "**/*.cleanup.ts",
-      worker: 1,
     },
   ];
 }
@@ -75,14 +73,24 @@ Assuming you have set up your [development tooling](010-preparing-the-environmen
 
 1. Create a playwright configuration file `playwright.config.ts` in your component / app directory.
 
-Playwright configuration file contents:
+Playwright configuration file contents for an app:
 
 ```ts
-import defineConfig from "@openremote/test/<app|component>.config";
+import { defineAppConfig } from "@openremote/test/app.config";
 
-export default defineConfig(__dirname);
+export default defineAppConfig(__dirname);
 ```
 
+Or for a component:
+
+```ts
+import { defineCtConfig } from "@openremote/test/component.config";
+
+export default defineCtConfig(__dirname);
+```
+
+Both derive the project name from the directory you pass, so the argument is always `__dirname`.
+
 2. Add the corresponding `test` script to the `package.json` file in your component / app directory.
 
 |           | test script (in package.json)                                        |
@@ -92,13 +100,25 @@ export default defineConfig(__dirname);
 
 3. Add the `npmTest` Gradle task to the `build.gradle` file in the component / app directory so that the CI/CD pipeline knows to run your tests.
 
+For a component:
+
 ```groovy
 tasks.register('npmTest', Exec) {
     dependsOn getYarnInstallTask()
     commandLine npmCommand("yarn"), "run", "test"
 }
 ```
 
+For an app, which installs only the workspaces the test run needs and forwards extra Playwright arguments through the `args` property:
+
+```groovy
+tasks.register('npmTest', Exec) {
+    dependsOn ":ui:test:installTestDeps", ":ui:test:clean"
+    commandLine npmCommand("yarn"), "run", "test"
+    args((findProperty('args')?.toString()?.tokenize()) ?: [])
+}
+```
+
 4. Install the required Playwright browser(s):
 
 ```sh
@@ -124,10 +144,12 @@ Then include the following boilerplate for app tests:
 ```ts
 import { test } from "@openremote/test";
 
-test("My app test", async ({ myApp }) => {
+test("My app test", async ({ page, shared }) => {
 })
 ```
 
+The `test` function from `@openremote/test` only provides Playwright's own fixtures plus `shared`. App specific fixtures like `myApp` come from extending it, see [Reusing test code](#reusing-test-code).
+
 Or the following for component tests:
 
 ```ts
@@ -145,7 +167,9 @@ ct("My component test", async ({ mount }) => {
 ```
 
 :::note
+
 You must import a component by its alias `@openremote/*`. Relative paths will cause issues. The downside of alias imports is that they refer to the transpiled TypeScript (in the `lib` directory), which is why the component test script includes `npx tsc -b` and needs manual rebuilding. However you can still live reload the component changes indirectly by running `npm run serve` on the manager app.
+
 :::
 
 Playwright uses [`locators`](https://playwright.dev/docs/locators) to find elements in the DOM. It's crucial to know the different types of locators to be able to write tests that are robust and to avoid flaky behavior.
@@ -171,7 +195,9 @@ export class AssetsPage implements BasePage {
 ```
 
 :::note
+
 In case you want to reuse certain non-project specific fixtures across multiple projects you can add your fixture to the `shared` fixtures in the `@openremote/test` package under `ui/test/fixtures/shared.ts`. If you want to reuse component specific fixtures in tests for a parent component or an app, simply import the fixtures and add them through the `extend` method.
+
 :::
 
 Finally extend the `test` function:
@@ -210,24 +236,25 @@ test("My app test", async ({ assetsPage }) => {
 
 ### Running the test
 
-The best way to run and debug your tests in Playwright is by using the [Playwright UI mode](https://playwright.dev/docs/test-ui-mode) feature.
+The `npmTest` task runs in every package that registers it, so the directory you point Gradle at decides which tests run:
 
-You may consider adding a Gradle task to the `build.gradle` file in the component / app directory to open it:
-
-```groovy
-tasks.register('npmTestUI', Exec) {
-    dependsOn getYarnInstallTask()
-    commandLine npmCommand("yarn"), "run", "test", "--ui"
-}
+```sh
+./gradlew npmTest                    # every frontend test
+./gradlew -p ui/component npmTest    # all component tests
+./gradlew -p ui/app npmTest          # all app tests
 ```
 
-Then run it with:
+App packages forward extra Playwright arguments through the `args` property, which is how CI shards the app test run:
 
 ```sh
-gradle ui:component:or-<my-component>:npmTestUI
+./gradlew -p ui/app/manager npmTest -Pargs="--shard=1/4"
 ```
 
-Or simply run `npm test -- --ui` in the component / app directory.
+While writing or debugging a test, run Playwright directly from the component / app directory instead. [UI mode](https://playwright.dev/docs/test-ui-mode) gives you the watch mode, trace viewer and time travel debugging that the Gradle tasks do not:
+
+```sh
+npm test -- --ui
+```
 
 ### Best practices
 
@@ -247,15 +274,21 @@ Please read the [Playwright Best practices](https://playwright.dev/docs/best-pra
 The most useful feature Playwright provides when it comes to writing frontend tests is [UI mode](https://playwright.dev/docs/test-ui-mode). Once the Playwright UI is launched you can select which projects you want to see and run.
 
 :::tip
+
 The UI includes a locator tab, which allows you to click an element in the test preview to easily get a locator of an element.
+
 :::
 
 :::warning
+
 Sometimes the locators Playwright provides are susceptible to flaky behavior, it is important to understand the DOM structure of the UI to get the most effective locators.
+
 :::
 
 :::tip
+
 In some cases you may face a situation where the UI needs to load first, before you can run an action. You can use `await selector.waitFor()` to ensure the element you want to interact with is visible.
+
 :::
 
 In case you want to see how Playwright runs in a headed browser you can add the `--headed` argument.