diff --git a/src/content/cookbook/testing/integration/introduction.md b/src/content/cookbook/testing/integration/introduction.md index fb5265ba7f3..93e565d1d3b 100644 --- a/src/content/cookbook/testing/integration/introduction.md +++ b/src/content/cookbook/testing/integration/introduction.md @@ -16,7 +16,38 @@ To perform these tasks, use *integration tests*. Integration tests verify the behavior of the complete app. This test can also be called end-to-end testing or GUI testing. -The Flutter SDK includes the [integration_test][] package. +## Testing frameworks + +Flutter supports two main approaches for integration testing: + +### `integration_test` package + +The Flutter SDK includes the [`integration_test`][integration_test] package. +Tests written with `integration_test` can perform the following tasks: + +* Run on the target device. + To test multiple Android or iOS devices, use Firebase Test Lab. +* Run from the host machine with `flutter test integration_test`. +* Use `flutter_test` APIs. This makes integration tests + similar to writing [widget tests][]. + +However, `integration_test` can't interact with native platform UI, +such as permission dialogs, notifications, or the contents of +platform views. + +### `patrol` package + +[Patrol][] is an open-source testing framework that builds on top +of Flutter's testing capabilities. It extends `integration_test` +with native interaction support. Patrol lets you: + +* Interact with native platform UI, such as permission dialogs, + notifications, and WebViews. +* Access device-level features like toggling Wi-Fi + or adjusting system settings. +* Write more concise tests using [patrol finders][]. + +To learn more, visit the [Patrol documentation][]. ## Terminology @@ -30,30 +61,27 @@ The Flutter SDK includes the [integration_test][] package. If you run your app in a web browser or as a desktop application, the host machine and the target device are the same. -## Dependent package +## Getting started -To run integration tests, add the `integration_test` package -as a dependency for your Flutter app test file. +To use `integration_test`, add it as a dependency for your +Flutter app test file. To migrate existing projects that use `flutter_driver`, consult the [Migrating from flutter_driver][] guide. -Tests written with the `integration_test` package -can perform the following tasks. - -* Run on the target device. - To test multiple Android or iOS devices, use Firebase Test Lab. -* Run from the host machine with `flutter test integration_test`. -* Use `flutter_test` APIs. This makes integration tests - similar to writing [widget tests][]. +To use Patrol, follow the [Patrol setup guide][]. ## Use cases for integration testing The other guides in this section explain how to use integration tests to validate [functionality][] and [performance][]. +[patrol finders]: {{site.pub-pkg}}/patrol_finders [functionality]: /testing/integration-tests/ -[performance]: /cookbook/testing/integration/profiling/ [integration_test]: {{site.repo.flutter}}/tree/main/packages/integration_test [Migrating from flutter_driver]: /release/breaking-changes/flutter-driver-migration +[Patrol]: {{site.pub-pkg}}/patrol +[Patrol documentation]: https://patrol.leancode.co/ +[Patrol setup guide]: https://patrol.leancode.co/getting-started +[performance]: /cookbook/testing/integration/profiling/ [widget tests]: /testing/overview#widget-tests diff --git a/src/content/testing/overview.md b/src/content/testing/overview.md index 771aa0ea56e..2e89dece20a 100644 --- a/src/content/testing/overview.md +++ b/src/content/testing/overview.md @@ -96,12 +96,24 @@ such as iOS Simulator or Android Emulator. The app under test is typically isolated from the test driver code to avoid skewing the results. +The Flutter SDK includes the [`integration_test`][] package. +However, this package can't interact with native platform UI, +such as permission dialogs, notifications, or platform views. +For apps that need native interactions, you can use [Patrol][], +an open-source framework that extends Flutter's testing +capabilities with native platform support. + For more information on how to write integration tests, see the [integration testing page][]. +[`integration_test`]: {{site.repo.flutter}}/tree/main/packages/integration_test +[Patrol]: {{site.pub-pkg}}/patrol + ### Recipes {:.no_toc} - [Integration testing concepts](/cookbook/testing/integration/introduction) +- [Write and run an integration test](/testing/integration-tests) +- [Write and run a Patrol test](/testing/patrol-tests) - [Measure performance with an integration test](/cookbook/testing/integration/profiling) ## Continuous integration services @@ -119,13 +131,15 @@ integration services, see the following: * [Test Flutter apps on Travis][] * [Test Flutter apps on Cirrus][] * [Codemagic CI/CD for Flutter][] +* [Codemagic CI/CD for Patrol][] * [Flutter CI/CD with Bitrise][] [code coverage]: https://en.wikipedia.org/wiki/Code_coverage [Codemagic CI/CD for Flutter]: https://blog.codemagic.io/getting-started-with-codemagic/ +[Codemagic CI/CD for Patrol]: https://docs.codemagic.io/integrations/patrol-integration/ [Continuous delivery using fastlane with Flutter]: /deployment/cd#fastlane [Flutter CI/CD with Bitrise]: https://devcenter.bitrise.io/en/getting-started/quick-start-guides/getting-started-with-flutter-apps [Test Flutter apps on Appcircle]: https://blog.appcircle.io/article/flutter-ci-cd-github-ios-android-web# [Test Flutter apps on Cirrus]: https://cirrus-ci.org/examples/#flutter [Test Flutter apps on Travis]: {{site.flutter-blog}}/test-flutter-apps-on-travis-3fd5142ecd8c -[integration testing page]: /testing/integration-tests +[integration testing page]: /cookbook/testing/integration/introduction diff --git a/src/content/testing/patrol-tests/index.md b/src/content/testing/patrol-tests/index.md new file mode 100644 index 00000000000..9969edd815a --- /dev/null +++ b/src/content/testing/patrol-tests/index.md @@ -0,0 +1,373 @@ +--- +title: Write and run a Patrol test +description: Learn how to write and run integration tests with Patrol +--- + +## Introduction + +[Patrol][] is an open-source testing framework for Flutter that +extends the capabilities of `integration_test` with native +platform interaction support. With Patrol, you can: + +* Interact with native platform UI, such as permission dialogs, + notifications, and WebViews. +* Access device-level features like toggling Wi-Fi + or adjusting system settings. +* Write more concise tests using [patrol finders][]. +* Iterate faster with hot restart during test development. + +This guide walks you through setting up, writing, +and running a Patrol test using the same Counter App example +from the [integration test guide][]. + +## Set up Patrol + +For complete setup instructions, including installing the Patrol CLI, +adding dependencies, and configuring native platform integration, +see the [Patrol documentation][]. + +## Create a test app + +This guide uses the same Counter App from the +[integration test guide][]. +If you haven't already, follow the steps in that guide to +[create the Counter App][create app] +with a `Key` added to the floating action button. + +## Create the test files + +1. Create a directory named `patrol_test` in your project root. +1. Add a new file named `app_test.dart` in that directory. + +The resulting directory tree should resemble the following: + +```plaintext +counter_app/ + lib/ + main.dart + patrol_test/ + app_test.dart +``` + +## Write a Patrol test + +Open `patrol_test/app_test.dart` and add the following code: + +```dart title="patrol_test/app_test.dart" +import 'package:counter_app/main.dart'; +import 'package:patrol/patrol.dart'; + +void main() { + patrolTest('tap on the floating action button, verify counter', ($) async { + // Load app widget. + await $.pumpWidgetAndSettle(const MyApp()); + + // Verify the counter starts at 0. + await $('0').waitUntilVisible(); + + // Tap the floating action button. + await $(#increment).tap(); + + // Verify the counter increments by 1. + await $('1').waitUntilVisible(); + }); +} +``` + +This example highlights the differences from a standard +`integration_test` test: + +* **`patrolTest`** replaces `testWidgets` and sets up the + Patrol test environment. +* **`$`** is the `PatrolIntegrationTester`, which provides + access to both widget and native interactions. +* **`$.pumpWidgetAndSettle`** loads the app and waits for + all animations to complete. +* **`$(#increment)`** finds a widget by its `Key` using + Patrol's concise finder syntax. + You can also find widgets by type (`$(FloatingActionButton)`) + or by text (`$('some text')`). + +:::warning +When initializing your app inside a Patrol test, +don't call `runApp()` or `WidgetsFlutterBinding.ensureInitialized()` +directly - Patrol handles this for you. +For the full list of requirements, see +[Initializing app inside a test][]. +::: + +## Patrol finders + +Patrol provides its own finder system that is more concise +and readable than Flutter's default `find.*` API. +Instead of verbose calls like `find.byKey(const Key('login'))`, +Patrol lets you write `$(#login)`. Finders also support +chaining, so you can locate widgets nested inside other +widgets without complex predicates. + +You can find widgets in several ways: + +| Syntax | Description | +|---|---| +| `$('text')` | Find by text content | +| `$(Type)` | Find by widget type | +| `$(#key)` | Find by `Key` | +| `$(Icon)` | Find by icon | + +{:.table .table-striped} + +### Chaining finders + +Nest finders to target widgets within other widgets: + +```dart +// Find 'Subscribe' text inside a ListView +await $(ListView).$('Subscribe').tap(); + +// Find a widget by key inside a ListTile containing specific text +await $(ListTile).containing('Activated').$(#learnMore).tap(); +``` + +### Common actions + +```dart +// Tap on a widget +await $('Subscribe').tap(); + +// Tap on the third occurrence +await $('Subscribe').at(2).tap(); + +// Enter text +await $(#cityTextField).enterText('Warsaw, Poland'); + +// Scroll to a widget and tap +await $('Subscribe').scrollTo().tap(); + +// Wait for a widget to become visible +await $('Log in').waitUntilVisible(); +``` + +For the full finder API, see the [patrol finders][] package. + +## Platform interactions + +Patrol lets you interact with platform features +directly from your Dart test code through the `$.platform` API. + +### Example: Testing a location permission request + +This example extends the Counter App with a button that +requests location permission, and a Patrol test that +handles the native permission dialog. + +First, add the [`geolocator`][geolocator] package to your app: + +```console +$ flutter pub add geolocator +``` + +For Android, add the required permission to your `android/app/src/main/AndroidManifest.xml`: + +```xml + +``` + +For iOS, add the required permission to your `ios/Runner/Info.plist`: + +```xml +NSLocationWhenInUseUsageDescription +This app needs location permission to show your location. +``` + +Then, add a location permission button to your app: + +```dart title="lib/main.dart" +import 'package:geolocator/geolocator.dart'; + +// ... inside _MyHomePageState ... + +String _permissionStatus = 'unknown'; + +Future _requestLocationPermission() async { + final permission = await Geolocator.requestPermission(); + setState(() { + _permissionStatus = permission.name; + }); +} + +// ... inside build method, add to the Column children ... + +ElevatedButton( + key: const Key('requestLocation'), + onPressed: _requestLocationPermission, + child: const Text('Request location'), +), +Text('Permission: $_permissionStatus'), +``` + +Now, write a Patrol test that handles the native +permission dialog: + +```dart title="patrol_test/permission_test.dart" +import 'package:counter_app/main.dart'; +import 'package:patrol/patrol.dart'; + +void main() { + patrolTest('grants location permission', ($) async { + await $.pumpWidgetAndSettle(const MyApp()); + + // Tap the button that triggers the permission request. + await $(#requestLocation).tap(); + + // Handle the native permission dialog. + if (await $.platform.mobile.isPermissionDialogVisible()) { + await $.platform.mobile.grantPermissionWhenInUse(); + } + + // Verify that the permission was granted. + await $('Permission: whileInUse').waitUntilVisible(); + }); +} +``` + +### Other platform actions + +Beyond permissions, the `$.platform` API provides access to +many other platform features: + +```dart +// Interact with notifications +await $.platform.mobile.openNotifications(); +await $.platform.mobile.tapOnNotificationByIndex(0); + +// Navigate the device +await $.platform.mobile.pressHome(); + +// Platform-specific actions +await $.platform.android.pressBack(); +await $.platform.ios.swipeBack(); +``` + +For detailed information on all available platform interactions, +see the [Patrol native automation documentation][]. + +For more examples of native interaction tutorials, +see the [Patrol feature guide][]. + +## Run Patrol tests + +```console +$ patrol test -t patrol_test/app_test.dart +``` + +### Test on an Android device + +Make sure that you followed all the Android setup steps from the +[setup guide][Patrol documentation]. + +Connect an Android device or start an emulator, then run: + +```console +$ patrol test -t patrol_test/app_test.dart +``` + +--- + +### Test on an iOS device + +Connect an iOS device or start a simulator, then run: + +```console +$ patrol test -t patrol_test/app_test.dart +``` + +To target a specific iOS version on the simulator, +use the `--ios` flag (for example, `--ios=17.5`). +If omitted, `latest` is used. +This flag only works with the iOS simulator and is supported by both +`test` and `develop` commands. + +:::note +Running on a physical iOS device requires additional setup. +See [Physical iOS devices setup][] for more information. +::: + +--- + +### Test in a web browser + +Patrol supports web testing using [Playwright][]. +You need [Node.js][] installed before running web tests. + +```console +$ patrol test --device chrome -t patrol_test/app_test.dart +``` + +To run headless (for CI environments): + +```console +$ patrol test --device chrome -t patrol_test/app_test.dart --web-headless true +``` + +--- + +### Development mode with hot restart + +For faster iteration while writing tests, +use the `develop` command: + +```console +$ patrol develop -t patrol_test/app_test.dart +``` + +In this mode, press `r` in the terminal to re-run tests +without rebuilding the entire app. + +:::note +`patrol develop` is not supported on web due to +[a Flutter issue](https://github.com/flutter/flutter/issues/175318). +::: + +--- + +### Test results + +After a test run, Patrol generates native test reports: + +* **Android**: An HTML report is saved to + `build/app/reports/androidTests/connected/debug/index.html`. + Note that this path may differ depending on the build flavor used. + For more details, see the [Patrol native test reports documentation][]. +* **iOS**: An `.xcresult` bundle is saved to + `build/ios_results_.xcresult`. + Open it in Xcode to view logs and test recordings. + +For more details on logs and configuration options, +see the [Patrol logs documentation][]. + +--- + +## Learn more + +* [Patrol documentation][] +* [patrol package on pub.dev][Patrol] +* [patrol_finders package on pub.dev][patrol finders] +* [Patrol on GitHub][] + +[create app]: /testing/integration-tests#create-a-new-app-to-test +[patrol finders]: {{site.pub-pkg}}/patrol_finders +[geolocator]: {{site.pub-pkg}}/geolocator +[Initializing app inside a test]: https://patrol.leancode.co/documentation#initializing-app-inside-a-test +[integration test guide]: /testing/integration-tests +[Node.js]: https://nodejs.org/ +[Patrol]: {{site.pub-pkg}}/patrol +[Patrol documentation]: https://patrol.leancode.co/documentation +[Patrol Firebase Test Lab documentation]: https://patrol.leancode.co/documentation/integrations/firebase-test-lab +[Patrol logs documentation]: https://patrol.leancode.co/documentation/logs +[Patrol native test reports documentation]: https://patrol.leancode.co/documentation/logs#native-test-reports +[Patrol feature guide]: https://patrol.leancode.co/feature-guide +[Patrol native automation documentation]: https://patrol.leancode.co/documentation/native/usage +[Patrol supported platforms]: https://patrol.leancode.co/documentation/supported-platforms +[Physical iOS devices setup]: https://patrol.leancode.co/documentation/physical-ios-devices-setup +[Patrol on GitHub]: {{site.github}}/leancodepl/patrol +[Playwright]: https://playwright.dev/ diff --git a/src/content/testing/testing-plugins.md b/src/content/testing/testing-plugins.md index fe2a32956c5..ec2cf48fa40 100644 --- a/src/content/testing/testing-plugins.md +++ b/src/content/testing/testing-plugins.md @@ -42,11 +42,21 @@ and look in the indicated directories. code that needs to run in a browser. These are often the most important tests for a plugin. - However, Dart integration tests can't interact with native UI, - such as native dialogs or the contents of platform views. + However, Dart integration tests using `integration_test` can't + interact with native UI, such as native dialogs or the contents + of platform views. See the `example/integration_test` directory for an example. +* Dart [Patrol][] integration tests. + Patrol combines access to the Flutter runtime with the + ability to interact with native components. + If your plugin requires testing native UI interactions + (for example, permission dialogs or platform views), + Patrol lets you do so entirely in Dart. + + To learn more, visit the [Patrol documentation][]. + * Native unit tests. Just as Dart unit tests can test the Dart portions of a plugin in isolation, native unit tests can @@ -84,13 +94,16 @@ such as [Espresso][] or [XCUITest][], enables tests that interact with both native and Flutter UI elements, so can be useful if your plugin can't be tested without native UI interactions. - +Alternatively, [Patrol][] lets you write these tests in Dart +while still being able to interact with native UI elements. [Espresso]: {{site.repo.packages}}/tree/main/packages/espresso [GoogleTest]: {{site.github}}/google/googletest [integration tests]: /cookbook/testing/integration/introduction [JUnit]: {{site.github}}/junit-team/junit4/wiki/Getting-started [mocked in tests]: /testing/plugins-in-tests#mock-the-platform-channel +[Patrol]: {{site.pub-pkg}}/patrol +[Patrol documentation]: https://patrol.leancode.co/ [plugin-tests]: /packages-and-plugins/developing-packages#step-1-create-the-package-1 [unit tests]: /cookbook/testing/unit/introduction [widget tests]: /cookbook/testing/widget/introduction @@ -190,10 +203,10 @@ Some extra considerations for plugin testing: try to have at least one integration test of each platform channel call. -* If some flows can't be tested using integration - tests—for example if they require interacting with - native UI or mocking device state—consider writing - "end to end" tests of the two halves using unit tests: +* If some flows can't be tested using `integration_test`—for + example if they require interacting with native UI or mocking + device state—consider using [Patrol][] for native UI interactions, + or writing "end to end" tests of the two halves using unit tests: * Native unit tests that set up the necessary mocks, then call into the method channel entry point diff --git a/src/data/sidenav.yml b/src/data/sidenav.yml index 392c6502608..de0c12aa9a3 100644 --- a/src/data/sidenav.yml +++ b/src/data/sidenav.yml @@ -514,6 +514,8 @@ permalink: /cookbook/testing/integration/introduction - title: Write and run an integration test permalink: /testing/integration-tests + - title: Write and run a Patrol test + permalink: /testing/patrol-tests - title: Profile an integration test permalink: /cookbook/testing/integration/profiling - title: Test a plugin