e4c726ab89
## `integration_ui_keyboard_resize` flakiness reduction This PR addresses flakiness in the `integration_ui_keyboard_resize` test (issue #161300) and slightly improves its performance. **Issue summary:** The `integration_ui_keyboard_resize` test occasionally fails because the device keyboard does not reliably open at the start of the test. This PR mitigates one cause of this flakiness, though other potential underlying issues may still exist. See #161300 for a detailed investigation. **Changes and rationale:** 1. **Improved keyboard detection logic (flakiness reduction):** * The core change moves `await driver.tap(defaultTextField);` *inside* the loop that checks for layout changes. This ensures the test repeatedly attempts to open the keyboard *while* waiting for the expected layout shift, significantly improving reliability. * **Local testing results (MacBook Pro M1 + Pixel 8 Pro):** * **Original implementation:** Consistent failures within the first 15 iterations. * **This PR:** Ran consistently for at least 300 iterations (out of 2000) before any failure, with some runs exceeding 900 iterations. 2. **`enableTextEntryEmulation` configuration improvement:** * The `keyboard_resize.dart` app is *exclusively* used for testing and requires `enableTextEntryEmulation: false` for proper device keyboard interaction. * This PR sets `enableTextEntryEmulation: false` directly within the app's `enableFlutterDriverExtension` setup. This simplifies the test setup, avoids cross-thread communication, and prevents the app from appearing broken when run independently (as manual text input wouldn't work otherwise). 3. **UI enhancements for manual testing:** * Added `SafeArea` and `InputDecoration` to `keyboard_resize.dart`. This ensures the `TextField` is fully visible and usable during manual testing (it was previously obscured by the Android status bar). This improves the developer experience when debugging the app directly. 4. **Optimized polling for speed:** * Reduced the polling interval and increased the number of polling iterations (maintaining the overall timeout). This change resulted in a ~2-second speed improvement per test iteration during local testing. **Further flakiness improvement discussion and ideas** See the discussion at https://github.com/flutter/flutter/issues/161300 **Testing and reproduction:** To reproduce the original flakiness and verify the fix (reproducibility may vary based on setup; see https://github.com/flutter/flutter/issues/161300#issuecomment-2618952501): 1. **Disable automatic reboots:** Comment out the `await checkForRebootRequired();` line in9e273d5e6e/dev/devicelab/lib/framework/framework.dart (L240-L243)2. **Disable automatic retries:** Set `static const int retryNumber = 0;` in9e273d5e6e/dev/devicelab/lib/framework/cocoon.dart (L56-L57)3. **Install `hyperfine`:** This is a command-line benchmarking tool (e.g., `brew install hyperfine` on macOS). 4. **Connect a device/emulator:** Connect a physical Android device or start an emulator. 5. **Run repeated tests:** Use `hyperfine` to run the test multiple times and capture the output: ```bash hyperfine -r 100 'dart bin/test_runner.dart test -t integration_ui_keyboard_resize > log.txt' ```
Automated Flutter integration test suites
Each suite consists of either a complete Flutter app and a flutter_driver
specification that drives tests from the UI, or a native app that is meant to
integrate with Flutter for testing.
Intended for use with devicelab tests.
If you want to run a driver test locally, to debug a problem with a test, you can use this command from the appropriate subdirectory:
flutter drive -t <test> --driver <driver>
For example:
flutter drive -t lib/keyboard_resize.dart --driver test_driver/keyboard_resize_test.dart
New tests require new CI runner
Adding code to this directory will not automatically cause it to be run by any already existing ci tooling. This directory is intentinally a "choose your own adventure" piece of tooling.