From 8b9c856da4642afc5f85d75dbe6b2b42fa956551 Mon Sep 17 00:00:00 2001 From: Marcos Sevilla Date: Thu, 10 Sep 2026 17:41:36 +0200 Subject: [PATCH] docs: document exclude-optimization Documents the `--exclude-optimization` option and the `test.optimization` map in `very_good.yaml`, including how exclusion globs are matched and that excluded files run as their own suites. Co-Authored-By: Claude Opus 5 (1M context) --- site/docs/commands/test.md | 28 +++++++- site/docs/configuration.md | 127 +++++++++++++++++++++++++------------ 2 files changed, 112 insertions(+), 43 deletions(-) diff --git a/site/docs/commands/test.md b/site/docs/commands/test.md index 505fc3d8d..bf18a8a0c 100644 --- a/site/docs/commands/test.md +++ b/site/docs/commands/test.md @@ -17,8 +17,9 @@ very_good test [arguments] -r, --recursive Run tests recursively for all nested packages. --[no-]optimization Whether to apply optimizations for test performance. Automatically disabled when --platform is specified. - Add the `skip_very_good_optimization` tag to specific test files to disable them individually. + Add the `skip_very_good_optimization` tag to specific test files to disable them individually, or use --exclude-optimization to exclude them by path. (defaults to on) + --exclude-optimization= A glob which will be used to exclude matching test files from the optimized bundle (e.g. 'test/integration'). Excluded files still run, as their own test suites. Can be passed multiple times. -j, --concurrency The number of concurrent test suites run. (defaults to "4") -t, --tags Run only tests associated with the specified tags. @@ -109,6 +110,28 @@ By default, all tests run with optimizations enabled; use the `--no-optimization import 'package:test/test.dart'; ``` +To exclude tests by path instead of tagging each file, pass one or more globs to `--exclude-optimization`, or list them under `optimization.exclude` in [`very_good.yaml`](../configuration.md): + +```sh +very_good test --exclude-optimization test/integration +``` + +```yaml +# very_good.yaml +test: + optimization: + exclude: + - test/integration +``` + +Globs are matched against each test file's path relative to the package root, and they match everything nested underneath what they name, so `test/integration` excludes every test in that directory. See [`optimization`](../configuration.md#optimization) for the glob syntax details. + +Excluded files are left out of the optimized bundle but still run, each as its own test suite. Because they run as their own suites, their file-level `@Tags` are honored again, so combining `--exclude-optimization test/integration` with `--exclude-tags integration` skips a suite annotated with `@Tags(['integration'])`. + +Prefer narrow globs. Every excluded file becomes its own suite with its own VM startup, so excluding hundreds of files gives up most of what the optimization buys you. + +Exclusions only matter while optimization is on. `--no-optimization`, `--platform`, `--update-goldens`, and targeting specific test files all disable optimization entirely, which makes the globs a no-op. + ### Configuring defaults with `very_good.yaml` To avoid repeating flags every time you run `very_good test` locally or on CI, you may create a `very_good.yaml` file at the root of your project. The `test` section accepts the same names as the CLI flags in snake_case (e.g. `--min-coverage` becomes `min_coverage`). Values from `very_good.yaml` are used as defaults; anything you pass on the command line takes precedence. @@ -123,6 +146,9 @@ test: dart_define: - FLAVOR=development file_reporter: json:reports/tests.json + optimization: + exclude: + - test/integration ``` With the file above, running `very_good test` behaves the same as running `very_good test --min-coverage 100 --exclude-coverage '**/*.g.dart' --report-on lib/ --dart-define=FLAVOR=development`. You can still override any of these values on the command line, for example `very_good test --min-coverage 90` to lower the coverage threshold for a single run. diff --git a/site/docs/configuration.md b/site/docs/configuration.md index aa7c1ebdf..f421bee2c 100644 --- a/site/docs/configuration.md +++ b/site/docs/configuration.md @@ -36,8 +36,9 @@ the command line without editing the file. Each top-level key in `very_good.yaml` maps to a Very Good CLI command. Every field mirrors a CLI flag using `snake_case` (e.g. `--min-coverage` becomes -`min_coverage`). Unrecognized keys cause the CLI to exit with a configuration -error. +`min_coverage`); `test.optimization` is the one field that also accepts a map, +grouping `--optimization` and `--exclude-optimization` together. Unrecognized +keys cause the CLI to exit with a configuration error. ```yaml # very_good.yaml @@ -58,7 +59,10 @@ Defaults for [`very_good test`](commands/test.md). ```yaml test: coverage: true - optimization: false + optimization: + enabled: true + exclude: + - test/integration concurrency: 8 tags: my-tag exclude_coverage: '**/*.g.dart' @@ -82,27 +86,63 @@ test: file_reporter: json:reports/tests.json ``` -| Field | Type | Notes | -| ----------------------- | ------------------ | ----------------------------------------------------------------------------- | -| `coverage` | `bool` | Whether to collect coverage information. | -| `optimization` | `bool` | Whether to apply optimizations for test performance. | -| `concurrency` | `int` | Positive integer. The number of concurrent test suites run. | -| `tags` | `string` | Run only tests associated with the specified tags. | -| `exclude_coverage` | `string` | A glob that excludes matching files from coverage. | -| `exclude_tags` | `string` | Run only tests that do not have the specified tags. | -| `min_coverage` | `number` | Between `0` and `100`. Enforces a minimum coverage percentage. | -| `show_uncovered` | `bool` | Whether to show uncovered lines when coverage is below 100%. | -| `collect_coverage_from` | `imports` \| `all` | Whether to collect coverage from imported files only or all files. | -| `update_goldens` | `bool` | Whether `matchesGoldenFile()` calls should update the golden files. | -| `fail_fast` | `bool` | Whether to stop running tests after the first failure. | -| `dart_define` | `string` \| `list` | Additional `--dart-define` values. | -| `dart_define_from_file` | `string` \| `list` | Paths of `.json` or `.env` files with `--dart-define-from-file` values. | -| `platform` | `string` | The platform to run tests on (`chrome`, `vm`, `android`, `ios`). | -| `report_on` | `string` \| `list` | File paths to report coverage information to. | -| `run_skipped` | `bool` | Whether to run skipped tests instead of skipping them. | -| `flavor` | `string` | The flavor to build for testing. | -| `timeout` | `int` | Positive integer (seconds). Maximum time tests may run before being killed. | -| `file_reporter` | `string` | Additional file reporter as `:` (e.g. `json:reports/tests.json`). | +| Field | Type | Notes | +| ----------------------- | ------------------ | ----------------------------------------------------------------------------------------- | +| `coverage` | `bool` | Whether to collect coverage information. | +| `optimization` | `bool` \| `map` | Whether to apply optimizations for test performance. See [`optimization`](#optimization). | +| `concurrency` | `int` | Positive integer. The number of concurrent test suites run. | +| `tags` | `string` | Run only tests associated with the specified tags. | +| `exclude_coverage` | `string` | A glob that excludes matching files from coverage. | +| `exclude_tags` | `string` | Run only tests that do not have the specified tags. | +| `min_coverage` | `number` | Between `0` and `100`. Enforces a minimum coverage percentage. | +| `show_uncovered` | `bool` | Whether to show uncovered lines when coverage is below 100%. | +| `collect_coverage_from` | `imports` \| `all` | Whether to collect coverage from imported files only or all files. | +| `update_goldens` | `bool` | Whether `matchesGoldenFile()` calls should update the golden files. | +| `fail_fast` | `bool` | Whether to stop running tests after the first failure. | +| `dart_define` | `string` \| `list` | Additional `--dart-define` values. | +| `dart_define_from_file` | `string` \| `list` | Paths of `.json` or `.env` files with `--dart-define-from-file` values. | +| `platform` | `string` | The platform to run tests on (`chrome`, `vm`, `android`, `ios`). | +| `report_on` | `string` \| `list` | File paths to report coverage information to. | +| `run_skipped` | `bool` | Whether to run skipped tests instead of skipping them. | +| `flavor` | `string` | The flavor to build for testing. | +| `timeout` | `int` | Positive integer (seconds). Maximum time tests may run before being killed. | +| `file_reporter` | `string` | Additional file reporter as `:` (e.g. `json:reports/tests.json`). | + +#### `optimization` + +`optimization` accepts either a boolean or a map. A boolean is shorthand for +`enabled`, so `optimization: false` and `optimization: {enabled: false}` are +equivalent. + +```yaml +test: + optimization: + enabled: true + exclude: + - test/integration + - test/**/serial_*_test.dart +``` + +| Field | Type | Notes | +| --------- | ------------------ | ------------------------------------------------------------------------------------------ | +| `enabled` | `bool` | Whether to apply optimizations for test performance. Mirrors `--optimization`. | +| `exclude` | `string` \| `list` | Globs of test files to keep out of the optimized bundle. Mirrors `--exclude-optimization`. | + +Globs are matched against each test file's path relative to the package root +and match everything nested underneath what they name, so `test/integration` +excludes every test in that directory. `**` matches zero or more directories, +so `test/**/serial_*_test.dart` matches both `test/serial_a_test.dart` and +`test/nested/serial_a_test.dart`. Matching is case-sensitive on every +platform. + +Excluded files still run, each as its own test suite, which also means their +file-level `@Tags` are honored again. See +[Skip optimization for specific tests](commands/test.md#skip-optimization-for-specific-tests). + +With `--recursive`, the closest `very_good.yaml` applies to every package in the +run and each glob is matched against the package currently being tested, so +`test/integration` excludes that directory in every package. Per-package +exclusions are not expressible. ### `create` @@ -135,7 +175,10 @@ Defaults for [`very_good dart test`](commands/test.md). The fields mirror dart: test: coverage: true - optimization: false + optimization: + enabled: true + exclude: + - test/integration concurrency: 8 tags: my-tag exclude_coverage: '**/*.g.dart' @@ -153,23 +196,23 @@ dart: file_reporter: json:reports/tests.json ``` -| Field | Type | Notes | -| ----------------------- | ------------------ | ----------------------------------------------------------------------------- | -| `coverage` | `bool` | Whether to collect coverage information. | -| `optimization` | `bool` | Whether to apply optimizations for test performance. | -| `concurrency` | `int` | Positive integer. The number of concurrent test suites run. | -| `tags` | `string` | Run only tests associated with the specified tags. | -| `exclude_coverage` | `string` | A glob that excludes matching files from coverage. | -| `exclude_tags` | `string` | Run only tests that do not have the specified tags. | -| `min_coverage` | `number` | Between `0` and `100`. Enforces a minimum coverage percentage. | -| `show_uncovered` | `bool` | Whether to show uncovered lines when coverage is below 100%. | -| `collect_coverage_from` | `imports` \| `all` | Whether to collect coverage from imported files only or all files. | -| `fail_fast` | `bool` | Whether to stop running tests after the first failure. | -| `platform` | `string` | The platform to run tests on (`chrome`, `vm`). | -| `report_on` | `string` \| `list` | File paths to report coverage information to. | -| `run_skipped` | `bool` | Whether to run skipped tests instead of skipping them. | -| `check_ignore` | `bool` | Whether to respect coverage ignore comments (e.g. `// coverage:ignore-line`). | -| `file_reporter` | `string` | Additional file reporter as `:` (e.g. `json:reports/tests.json`). | +| Field | Type | Notes | +| ----------------------- | ------------------ | ----------------------------------------------------------------------------------------- | +| `coverage` | `bool` | Whether to collect coverage information. | +| `optimization` | `bool` \| `map` | Whether to apply optimizations for test performance. See [`optimization`](#optimization). | +| `concurrency` | `int` | Positive integer. The number of concurrent test suites run. | +| `tags` | `string` | Run only tests associated with the specified tags. | +| `exclude_coverage` | `string` | A glob that excludes matching files from coverage. | +| `exclude_tags` | `string` | Run only tests that do not have the specified tags. | +| `min_coverage` | `number` | Between `0` and `100`. Enforces a minimum coverage percentage. | +| `show_uncovered` | `bool` | Whether to show uncovered lines when coverage is below 100%. | +| `collect_coverage_from` | `imports` \| `all` | Whether to collect coverage from imported files only or all files. | +| `fail_fast` | `bool` | Whether to stop running tests after the first failure. | +| `platform` | `string` | The platform to run tests on (`chrome`, `vm`). | +| `report_on` | `string` \| `list` | File paths to report coverage information to. | +| `run_skipped` | `bool` | Whether to run skipped tests instead of skipping them. | +| `check_ignore` | `bool` | Whether to respect coverage ignore comments (e.g. `// coverage:ignore-line`). | +| `file_reporter` | `string` | Additional file reporter as `:` (e.g. `json:reports/tests.json`). | ### `packages.get`