**Three things**
Re-lands #154374
New: fix `platform_channels_benchmarks` to print the "done" key. Updated notes for the microbenchmark parser. There are no other users of `microbenchmarks.readJsonResults`.
Re-Re-land: Uninstall microbenchmarks before running them.
Flakes in https://github.com/flutter/flutter/issues/153828 stem from adb saying the app isn't installed, but then failing to install wtih -r. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys.
Potential solution https://github.com/flutter/flutter/issues/153828
Re-land Make things go fast
Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.
Reverts: flutter/flutter#154374
Initiated by: jtmcdole
Reason for reverting: A different benchmark was using the microbenchmark parser and timing out.
Original PR Author: jtmcdole
Reviewed By: {zanderso}
This change reverts the following previous change:
Two things:
**Re-land**: Uninstall microbenchmarks before running them.
Flakes in #153828 stem from adb saying the app isn't installed, but then failing to install wtih `-r`. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys.
Potential solution #153828
**Make things go fast**
Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.
Two things:
**Re-land**: Uninstall microbenchmarks before running them.
Flakes in #153828 stem from adb saying the app isn't installed, but then failing to install wtih `-r`. Several other tests uninstall the app before trying to run it. Previous fix called uninstall between tests, but iOS takes 12 to 13 seconds to perform uninstall / install, which timed out the test. Just uninstall the one time since we only care about any lingering apps with different keys.
Potential solution #153828
**Make things go fast**
Instead of installing 21 different compilations of the same app to get results; compile and run them together. Locally on Mac+iOS, this should takes ~3 minutes instead of ~15 minutes.
Contributes to https://github.com/flutter/flutter/issues/130459
It adds a test for
- `examples/api/lib/rendering/sliver_grid/sliver_grid_delegate_with_fixed_cross_axis_count.0.dart`
- `examples/api/lib/rendering/sliver_grid/sliver_grid_delegate_with_fixed_cross_axis_count.1.dart`
I also fixed a mistake in the documentation
Reverts: flutter/flutter#154183
Initiated by: zanderso
Reason for reverting: microbenchmarks timing out on CI.
Original PR Author: jtmcdole
Reviewed By: {andrewkolos}
This change reverts the following previous change:
Flakes in #153828 stem from adb saying the app isn't installed, but then failing to install wtih `-r`. Several other tests uninstall the app before trying to run it.
Potential corrects #153828
Flakes in #153828 stem from adb saying the app isn't installed, but then
failing to install wtih `-r`. Several other tests uninstall the app
before trying to run it.
Potential corrects #153828
Reverts: flutter/flutter#154044
Initiated by: zanderso
Reason for reverting: Version does not exist.
Original PR Author: flutter-pub-roller-bot
Reviewed By: {fluttergithubbot}
This change reverts the following previous change:
This PR was generated by `flutter update-packages --force-upgrade`.
This was partially fixed in flutter/flutter#153787, which added just the extension-safe simulator dSYM, but we also bundle:
* arm64 release dSYM
* arm64 release extension-safe dSYM
* arm64_x86_64 simulator dSYM
In a followup engine patch, we'll stop bundling both the regular and extension-safe simulator dSYMs, but this gets the test passing at tip-of-tree.
Issue: https://github.com/flutter/flutter/issues/116493
*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*
*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
CustomScrollViewExampleApp
Part of https://github.com/flutter/flutter/issues/130459
The test is checking:
- if all crucial Widgets are initially visible
- if IconButton click will expand existing SliverList
- if IconButton click and mouse scroll will reveal additional SliverList
A pending change to the flutter_drone LUCI recipe is using a logging library that sets the FLUTTER_TEST_OUTPUTS_DIR environment variable (see https://flutter.googlesource.com/recipes/+/b111cd3ef2297e66905cc48c2cfedce0cf1ba46f)
flutter_driver uses FLUTTER_TEST_OUTPUTS_DIR as its output directory if it is set. run_web_tests invokes flutter_driver and needs to look for outputs in that directory.
Adds usage of SystemChrome.setAPplicationSwitcherDescription to update the page title for the AutoComplete demo page.
[Before screenshot: example AutoComplete](https://screenshot.googleplex.com/8DpESj4QrQA2YrW)
[After screenshot: example AutoComplete](https://screenshot.googleplex.com/6AdfSA8a8VwUaXM)
Fixes b/338056490
*Replace this paragraph with a description of what this PR is changing or adding, and why. Consider including before/after screenshots.*
*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
A recent change to the flutter_drone recipe is setting FLUTTER_TEST_OUTPUTS_DIR in the environment (see https://flutter.googlesource.com/recipes/+/b111cd3ef2297e66905cc48c2cfedce0cf1ba46f)
If FLUTTER_TEST_OUTPUTS_DIR is set, then perf_tests will use an output path based on that variable. perf_tests_test needs to use the same logic when looking for the expected output file.
Not so long ago I remember a very informal conversation that went something like this:
> @matanlurey: I wish I could pass `--dev` or something to `flutter` to run from source.
>
> @christopherfujino: I get what you want, but I don't want to overload the tool with more dev-only things. I would consider a script like `flutter-dev` that does that thing, though.
>
> @matanlurey: Cool, I might send a PR!
So uh, here it is 6-9 months later. Suggestions welcome.
Contributes to https://github.com/flutter/flutter/issues/130459
It adds a test for
- `examples/api/lib/widgets/single_child_scroll_view/single_child_scroll_view.0.dart`
- `examples/api/lib/widgets/single_child_scroll_view/single_child_scroll_view.1.dart`
I also fixed a mistake in the documentation
Reverts: flutter/flutter#152487
Initiated by: gmackall
Reason for reverting: I forgot that I need to override the compileSdkVersion in the AGP 8.0 [instance of this test](ef9cd32f5a/dev/devicelab/bin/tasks/android_java17_dependency_smoke_tests.dart (L19))
Original PR Author: gmackall
Reviewed By: {reidbaker}
This change reverts the following previous change:
Updates `compileSdk`, `targetSdk`, and `ndk` versions (former 2 to latest, latter to the version of the ndk we are hosting on CIPD).
Summary of changes:
- Updates mentioned template values
- `compileSdk` 35 requires AGP 8.0+, so updated to 8.1 in many places.
- This also necessitated Gradle upgrades in most places
- This also necessitated moving the `package` xml attribute to the AGP `namespace` field in a couple places (test + template).
- Some tests use the output of `flutter create` but then use intentionally lower AGP versions. [I downgraded the `compileSdk` in these tests.](fee34fd61a)
- [Stopped lockfile generation](82324a2570) script from hitting the `hello_world` example because it uses `.kts` gradle files.
- One test needed [some Gradle options we had already added to templates](6aa187b4b6).
Updates `compileSdk`, `targetSdk`, and `ndk` versions (former 2 to latest, latter to the version of the ndk we are hosting on CIPD).
Summary of changes:
- Updates mentioned template values
- `compileSdk` 35 requires AGP 8.0+, so updated to 8.1 in many places.
- This also necessitated Gradle upgrades in most places
- This also necessitated moving the `package` xml attribute to the AGP `namespace` field in a couple places (test + template).
- Some tests use the output of `flutter create` but then use intentionally lower AGP versions. [I downgraded the `compileSdk` in these tests.](fee34fd61a)
- [Stopped lockfile generation](82324a2570) script from hitting the `hello_world` example because it uses `.kts` gradle files.
- One test needed [some Gradle options we had already added to templates](6aa187b4b6).
When attempting to release the Flutter 3.24.1 hotfix, Flutter framework post submits failed due to the following error:
```
âââ¡ERROR #1âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â UNEXPECTED ERROR!
â Exception: Found unexpected binary in cache: /Volumes/Work/s/w/ir/x/w/flutter/bin/cache/artifacts/engine/ios-release/extension_safe/Flutter.xcframework/ios-arm64_x86_64-simulator/dSYMs/Flutter.framework.dSYM/Contents/Resources/DWARF/Flutter
â #0 verifyExist (file:///Volumes/Work/s/w/ir/x/w/flutter/dev/bots/suite_runners/run_verify_binaries_codesigned_tests.dart:150:12)
â <asynchronous suspension>
â #1 verifyCodesignedTestRunner (file:///Volumes/Work/s/w/ir/x/w/flutter/dev/bots/suite_runners/run_verify_binaries_codesigned_tests.dart:28:3)
â <asynchronous suspension>
â #2 _runFromList (file:///Volumes/Work/s/w/ir/x/w/flutter/dev/bots/utils.dart:601:5)
â <asynchronous suspension>
â #3 main (file:///Volumes/Work/s/w/ir/x/w/flutter/dev/bots/test.dart:125:5)
â <asynchronous suspension>
â
â The test.dart script should be corrected to catch this error and call foundError().
â Some tests are likely to have been skipped.
ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
```
I've updated the tests to include the listed artifact.
### Open questions
1. Is it possible that the test failed only on the first binary?
2. Do we expect a need to add the remainder of the [entitlement changes](c9b9d5780d)?
3. I am under the assumption that the bots pull from master and are not branch specific, is this a correct assumption?
CC @christopherfujino @cbracken
