Build/Test Tools: Run the post list E2E specs in parallel.

tests/e2e/README.md

@@ -1 +1 @@
-# E2E TestsEnd-To-End (E2E) tests for WordPress.## Running the testsThe e2e tests require a production-like environment to run. By default, they will assume an environment is available at `http://localhost:8889`, with username `admin` and password `password`.If you don't already have an environment ready, you can set one up by following [these instructions](https://github.com/WordPress/wordpress-develop/blob/master/README.md).Then you can launch the tests by running:```npm run test:e2e```which will run the test suite using a headless browser.If your environment has a different url, username or password to the default, you can provide the base URL, username and password like this:```WP_BASE_URL=http://mycustomurl WP_USERNAME=username WP_PASSWORD=password npm run test:e2e```**DO NOT run these tests in an actual production environment, as they will delete all your content.**For debugging purposes, you might want to follow the test visually. You can do so by running the tests in an interactive mode:```npm run test:e2e -- --ui```[UI Mode](https://playwright.dev/docs/test-ui-mode) let's you explore, run and debug tests with a time travel experience complete with watch mode.All test files are loaded into the testing sidebar where you can expand each file and describe block to individually run, view, watch and debug each test.You can also run a single test file separately:```npm run test:e2e tests/e2e/specs/hello.test.js```## ProjectsThe suite has three Playwright projects, defined in `playwright.config.js`:* `all`: every spec, one at a time on one worker. `npm run test:e2e` runs this project.* `parallel`: specs that can share the site with other tests. Spec files run at the same time on several workers, one for every two CPU cores. Tests inside one file still run in order. Run it with `npm run test:e2e -- --project=parallel`.* `serial`: specs that break the site for other tests. They run one at a time on one worker. Run it with `npm run test:e2e -- --project=serial`.CI runs `parallel` and `serial` in separate jobs.New specs go in `parallel` by default. Add a spec to `SERIAL_SPECS` if it makes the site unusable for other tests. If it changes content or settings that other tests use, for example with `requestUtils.deleteAllPosts()`, follow [Parallel safety](#parallel-safety).Before you open a pull request with a new spec, run its project a few times with `--shuffle`. Playwright then runs the spec files in a random order, so a spec that needs another spec to run first, or that breaks a spec that runs after it, fails:```npm run test:e2e -- --project=parallel --shuffle```Playwright prints the seed of each run, for example `shuffle seed 317753389`. To run the same order again, give the seed: `--shuffle 317753389`.## Parallel safetySpecs in `parallel` share one site. A test that changes shared data can break a test in another file that runs at the same time. To make a spec safe, use the first of these that works:1. Scope the data. Let the test create its own data, for example a post with a unique title, and read only that data.2. If the test must change shared data, such as a site setting, give it a lock. Do this only if every spec that writes the same data holds the same lock.3. If you cannot lock every spec that writes the data, add the spec to `SERIAL_SPECS`.A lock is a name. Playwright never runs two tests that hold the same lock name at the same time. A lock only schedules tests. It does not isolate data: a test without the lock can still change the data while a locked test runs.The lock names are in `utils/locks.js`. Import the constant, so that all specs use the same name. Playwright gives no error for a lock name that no other test uses, or for an import name that is not in `utils/locks.js`:```import { GENERAL_SETTINGS } from '../utils/locks';test.describe( 'Settings -> General', { lock: GENERAL_SETTINGS }, () => {} );```A form save writes every field on the page, not only the field that the test changed. A spec that saves the General Settings form (`options-general.php`) must hold `GENERAL_SETTINGS`. A spec in `parallel` that changes a General Settings option in a different way, for example with `siteSettings` or `siteOptions`, must also hold it.## Logged-out pagesMost specs run logged in as the admin user. There are two ways to run a test logged out:* To make the `page` of a whole `describe` group logged out, add `test.use( { storageState: { cookies: [], origins: [] } } )` to the group. See `specs/dashboard.test.js`.* To use a logged-out page next to the logged-in `page` in one test, import `test` and `expect` from `../fixtures` and use the `loggedOutPage` fixture. Its browser context has no cookies, and it closes when the test ends. Unlike `page`, it resolves a URL that starts with `/` against the host only, not the full `WP_BASE_URL`, so it does not work with a site in a subdirectory. See `specs/cache-control-headers-directives.test.js`.## FixturesTo create posts, pages or users, to upload attachments, or to change a setting, an option, or a must-use plugin, import `test` and `expect` from `../fixtures`, and use one of the fixtures below. Each fixture cleans up when the test ends, also if the test fails. If the cleanup fails, the test fails.* `siteSettings`: reads and changes the settings of `/wp/v2/settings` through REST, with `get()`, `remember( keys )` and `set( patch )`. It restores only the keys passed to `remember` or `set`. See `specs/privacy.test.js`.* `siteOptions`: reads and changes options through WP-CLI, with `get( names )`, `remember( names )` and `set( values )`. It reads and restores the stored `gmt_offset`, not the value calculated for a named timezone. It restores only the keys passed to `remember` or `set`. See `specs/general-settings-invalid-timezone.test.js`.* `muPlugin`: writes a must-use plugin with `write( fileName, php )`, and deletes the file when the test ends. Call `write` before the test visits a page. See `specs/fatal-error-handler.test.js`.* `postFactory`: creates posts with `create( data )` and pages with `createPage( data )`, and force-deletes them when the test ends. `uniqueTitle( prefix )` adds a UUID to the prefix. See `specs/edit-posts.test.js`.* `attachments`: uploads a file with `upload( absolutePath )`, records an attachment that the browser uploaded with `track( id )`, and deletes them when the test ends. See `specs/media-new-client-side-upload.test.js`.* `userFactory`: creates users with `create( { roles, ...data } )`, which returns `{ user, username, password }`. For a user that the test creates through a form, call `adopt( username )` before the form is submitted. When the test ends, it deletes each user and the user's auto-drafts. To log in as the user, import `signIn( page, username, password )` from `../fixtures/user`. See `specs/users.test.js`.If the test changes a setting or an option through a form, first call `remember` with an array of the keys that the form changes. The fixture then reads the current value, without a write, and restores it when the test ends. `remember` returns the current values.A fixture goes in `fixtures/` only when 2 or more spec files use it. A fixture that only one spec uses stays in that spec.## Documentation* Block Editor Handbook end to end testing overview: https://developer.wordpress.org/block-editor/contributors/code/testing-overview/#end-to-end-testing* Gutenberg e2e-test-utils-playwright package API docs: https://github.com/WordPress/gutenberg/tree/trunk/packages/e2e-test-utils-playwright* Playwright API docs: https://playwright.dev/docs (the version we are using is indicated in the `@wordpress/scripts` package: https://github.com/WordPress/gutenberg/blob/trunk/packages/scripts/package.json)
+# E2E TestsEnd-To-End (E2E) tests for WordPress.## Running the testsThe e2e tests require a production-like environment to run. By default, they will assume an environment is available at `http://localhost:8889`, with username `admin` and password `password`.If you don't already have an environment ready, you can set one up by following [these instructions](https://github.com/WordPress/wordpress-develop/blob/master/README.md).Then you can launch the tests by running:```npm run test:e2e```which will run the test suite using a headless browser.If your environment has a different url, username or password to the default, you can provide the base URL, username and password like this:```WP_BASE_URL=http://mycustomurl WP_USERNAME=username WP_PASSWORD=password npm run test:e2e```**DO NOT run these tests in an actual production environment, as they will delete all your content.**For debugging purposes, you might want to follow the test visually. You can do so by running the tests in an interactive mode:```npm run test:e2e -- --ui```[UI Mode](https://playwright.dev/docs/test-ui-mode) let's you explore, run and debug tests with a time travel experience complete with watch mode.All test files are loaded into the testing sidebar where you can expand each file and describe block to individually run, view, watch and debug each test.You can also run a single test file separately:```npm run test:e2e tests/e2e/specs/hello.test.js```## ProjectsThe suite has three Playwright projects, defined in `playwright.config.js`:* `all`: every spec, one at a time on one worker. `npm run test:e2e` runs this project.* `parallel`: specs that can share the site with other tests. Spec files run at the same time on several workers, one for every two CPU cores. Tests inside one file still run in order. Run it with `npm run test:e2e -- --project=parallel`.* `serial`: specs that break the site for other tests. They run one at a time on one worker. Run it with `npm run test:e2e -- --project=serial`.CI runs `parallel` and `serial` in separate jobs.New specs go in `parallel` by default. Add a spec to `SERIAL_SPECS` if it makes the site unusable for other tests. If it changes content or settings that other tests use, for example with `requestUtils.deleteAllPosts()`, follow [Parallel safety](#parallel-safety).Before you open a pull request with a new spec, run its project a few times with `--shuffle`. Playwright then runs the spec files in a random order, so a spec that needs another spec to run first, or that breaks a spec that runs after it, fails:```npm run test:e2e -- --project=parallel --shuffle```Playwright prints the seed of each run, for example `shuffle seed 317753389`. To run the same order again, give the seed: `--shuffle 317753389`.## Parallel safetySpecs in `parallel` share one site. A test that changes shared data can break a test in another file that runs at the same time. To make a spec safe, use the first of these that works:1. Scope the data. Let the test create its own data, for example a post with a unique title, and read only that data.2. If the test must change shared data, such as a site setting, give it a lock. Do this only if every spec that writes the same data holds the same lock.3. If you cannot lock every spec that writes the data, add the spec to `SERIAL_SPECS`.A lock is a name. Playwright never runs two tests that hold the same lock name at the same time. A lock only schedules tests. It does not isolate data: a test without the lock can still change the data while a locked test runs.The lock names are in `utils/locks.js`. Import the constant, so that all specs use the same name. Playwright gives no error for a lock name that no other test uses, or for an import name that is not in `utils/locks.js`:```import { GENERAL_SETTINGS } from '../utils/locks';test.describe( 'Settings -> General', { lock: GENERAL_SETTINGS }, () => {} );```A form save writes every field on the page, not only the field that the test changed. A spec that saves the General Settings form (`options-general.php`) must hold `GENERAL_SETTINGS`. A spec in `parallel` that changes a General Settings option in a different way, for example with `siteSettings` or `siteOptions`, must also hold it.## Logged-out pagesMost specs run logged in as the admin user. There are two ways to run a test logged out:* To make the `page` of a whole `describe` group logged out, add `test.use( { storageState: { cookies: [], origins: [] } } )` to the group. See `specs/dashboard.test.js`.* To use a logged-out page next to the logged-in `page` in one test, import `test` and `expect` from `../fixtures` and use the `loggedOutPage` fixture. Its browser context has no cookies, and it closes when the test ends. Unlike `page`, it resolves a URL that starts with `/` against the host only, not the full `WP_BASE_URL`, so it does not work with a site in a subdirectory. See `specs/cache-control-headers-directives.test.js`.## FixturesTo create posts, pages or users, to upload attachments, or to change a setting, an option, or a must-use plugin, import `test` and `expect` from `../fixtures`, and use one of the fixtures below. Each fixture cleans up when the test ends, also if the test fails. If the cleanup fails, the test fails.* `siteSettings`: reads and changes the settings of `/wp/v2/settings` through REST, with `get()`, `remember( keys )` and `set( patch )`. It restores only the keys passed to `remember` or `set`. See `specs/privacy.test.js`.* `siteOptions`: reads and changes options through WP-CLI, with `get( names )`, `remember( names )` and `set( values )`. It reads and restores the stored `gmt_offset`, not the value calculated for a named timezone. It restores only the keys passed to `remember` or `set`. See `specs/general-settings-invalid-timezone.test.js`.* `muPlugin`: writes a must-use plugin with `write( fileName, php )`, and deletes the file when the test ends. Call `write` before the test visits a page. See `specs/fatal-error-handler.test.js`.* `postFactory`: creates posts with `create( data )` and pages with `createPage( data )`, and force-deletes them when the test ends. `uniqueTitle( prefix )` adds a UUID to the prefix. See `specs/edit-posts.test.js`.* `attachments`: uploads a file with `upload( absolutePath )`, records an attachment that the browser uploaded with `track( id )`, and deletes them when the test ends. See `specs/media-new-client-side-upload.test.js`.* `userFactory`: creates users with `create( { roles, ...data } )`, which returns `{ user, username, password }`. For a user that the test creates through a form, call `adopt( username )` before the form is submitted. When the test ends, it deletes each user and all of the user's posts. To log in as the user, import `signIn( page, username, password )` from `../fixtures/user`. See `specs/users.test.js`.If the test changes a setting or an option through a form, first call `remember` with an array of the keys that the form changes. The fixture then reads the current value, without a write, and restores it when the test ends. `remember` returns the current values.A fixture goes in `fixtures/` only when 2 or more spec files use it. A fixture that only one spec uses stays in that spec.## Documentation* Block Editor Handbook end to end testing overview: https://developer.wordpress.org/block-editor/contributors/code/testing-overview/#end-to-end-testing* Gutenberg e2e-test-utils-playwright package API docs: https://github.com/WordPress/gutenberg/tree/trunk/packages/e2e-test-utils-playwright* Playwright API docs: https://playwright.dev/docs (the version we are using is indicated in the `@wordpress/scripts` package: https://github.com/WordPress/gutenberg/blob/trunk/packages/scripts/package.json)

tests/e2e/fixtures/user.js

@@ -43,18 +43,26 @@ async function findUserId( requestUtils, username ) {
 }
 
 async function deleteUser( requestUtils, id ) {
-	// User deletion only trashes posts; permanently delete this user's Quick Draft auto-drafts first.
-	const drafts = await requestUtils.rest( {
-		path: '/wp/v2/posts',
-		params: { author: id, status: 'auto-draft', per_page: 100 },
-	} );
-	for ( const draft of drafts ) {
-		await requestUtils.rest( {
-			method: 'DELETE',
-			path: `/wp/v2/posts/${ draft.id }`,
-			params: { force: true },
+	// User deletion only trashes posts, so permanently delete them first. `any` skips `trash` and `auto-draft`.
+	// Always fetch the first page, because each delete moves later posts onto it.
+	let posts;
+	do {
+		posts = await requestUtils.rest( {
+			path: '/wp/v2/posts',
+			params: {
+				author: id,
+				status: 'any,trash,auto-draft',
+				per_page: 100,
+			},
 		} );
-	}
+		for ( const post of posts ) {
+			await requestUtils.rest( {
+				method: 'DELETE',
+				path: `/wp/v2/posts/${ post.id }`,
+				params: { force: true },
+			} );
+		}
+	} while ( posts.length );
 	try {
 		await requestUtils.rest( {
 			method: 'DELETE',

tests/e2e/playwright.config.js

@@ -23,11 +23,8 @@ const SERIAL_SPECS = [
 	'gutenberg-plugin.test.js',
 	'plugins.test.js',
 	'themes.test.js',
-	// These delete all posts or media, including other workers' content.
-	'cache-control-headers-directives.test.js',
-	'dashboard.test.js',
-	'edit-posts.test.js',
-	'empty-trash-restore-trashed-posts.test.js',
+	// This deletes all posts, including other workers' content.
+	'empty-trash.test.js',
 ];
 
 const config = defineConfig( {

tests/e2e/specs/cache-control-headers-directives.test.js

@@ -4,20 +4,17 @@
 import { test, expect } from '../fixtures';
 
 test.describe( 'Cache Control header directives', () => {
-	test.beforeAll( async ( { requestUtils } ) => {
-		await requestUtils.deleteAllPosts();
-	});
-
 	test(
 		'No private directive present in cache control when user not logged in.',
-		async ( { loggedOutPage, admin, editor}
+		async ( { loggedOutPage, postFactory }
 		) => {
-		await admin.createNewPost( { title: 'Hello World' } );
-		await editor.publishPost();
-
-		await admin.visitAdminPage( '/' );
+		const post = await postFactory.create( {
+			title: postFactory.uniqueTitle( 'Cache control' ),
+			status: 'publish',
+		} );
+		const { pathname, search } = new URL( post.link );
 
-		const response = await loggedOutPage.goto( '/hello-world/' );
+		const response = await loggedOutPage.goto( pathname + search );
 		const responseHeaders = response.headers();
 
 		const cacheControl = responseHeaders[ 'cache-control' ] ?? '';

tests/e2e/specs/dashboard.test.js

@@ -5,8 +5,19 @@ import { test, expect } from '../fixtures';
 import { signIn } from '../fixtures/user';
 
 test.describe( 'Quick Draft', () => {
-	test.beforeEach( async ({ requestUtils }) => {
-		await requestUtils.deleteAllPosts();
+	// Recent Drafts and the author-filtered post list show only this user's posts.
+	test.use( { storageState: { cookies: [], origins: [] } } );
+	let userId;
+
+	test.beforeEach( async ( { page, userFactory } ) => {
+		const { user, username, password } = await userFactory.create( {
+			roles: [ 'administrator' ],
+		} );
+		userId = user.id;
+		await signIn( page, username, password );
+		await expect(
+			page.getByRole( 'heading', { name: 'Dashboard', exact: true } )
+		).toBeVisible();
 	} );
 
 	test( 'should allow Quick Draft to be created with Title and Content', async ( {
@@ -57,7 +68,7 @@ test.describe( 'Quick Draft', () => {
 		).toBeVisible();
 
 		// Check that the new draft appears in the Posts page.
-		await admin.visitAdminPage( '/edit.php' );
+		await admin.visitAdminPage( '/edit.php', `author=${ userId }` );
 
 		await expect(
 			page
@@ -92,7 +103,7 @@ test.describe( 'Quick Draft', () => {
 		).toHaveText( 'Cannot create a draft post with empty title and content.' );
 
 		// Check that no new draft appears in the Posts page.
-		await admin.visitAdminPage( '/edit.php' );
+		await admin.visitAdminPage( '/edit.php', `author=${ userId }` );
 
 		await expect(
 			page.getByRole( 'cell', { name: 'No posts found.' } )
@@ -132,7 +143,7 @@ test.describe( 'Quick Draft', () => {
 		).toHaveText( 'Quick Draft test title' );
 
 		// Check that the new draft appears in the Posts page.
-		await admin.visitAdminPage( '/edit.php' );
+		await admin.visitAdminPage( '/edit.php', `author=${ userId }` );
 
 		await expect(
 			page
@@ -181,7 +192,7 @@ test.describe( 'Quick Draft', () => {
 		).toBeVisible();
 
 		// Check that the new draft appears in the Posts page.
-		await admin.visitAdminPage( '/edit.php' );
+		await admin.visitAdminPage( '/edit.php', `author=${ userId }` );
 
 		await expect(
 			page