In "build tests", the `properties` of the build specify the
configuration of the coordinator, and the `drone_dimensions` specify the
configuration of the devicelab bot. This PR should therefore fix the
infra errors on the `Linux_build_test` tests caused by my last PR at
https://github.com/flutter/flutter/pull/152756. (Unfortunately, a simple
revert won't work since the MotoG4s have already been removed from the
Linux hosts.)
Sets up tests that verify we can build a fresh counter app across our Gradle/AGP/Kotlin support range.
Post submit only, because the suite takes ~30 minutes to run, and I expect it to be _somewhat_ rare that we break only one of these versions (and therefore it doesn't get caught by existing presubmits).
For currently unknown reasons, this test is failing at a rate >10% but only in presubmit. This is blocking many engine -> framework rolls, requiring a lot of manual intervention from the sheriff.
https://github.com/flutter/flutter/issues/150642
These tests aren't actually new, they are old tests that had to be broken out to a new shard in https://github.com/flutter/flutter/pull/151433/. So they've already been passing for some time, and are still passing on their new shard, so they should be safe to mark as not `bringup:true`.
Reverts: flutter/flutter#151608
Initiated by: zanderso
Reason for reverting: Overeager cleanup of unused device spec.
Original PR Author: zanderso
Reviewed By: {christopherfujino, jonahwilliams}
This change reverts the following previous change:
Next step of https://github.com/flutter/flutter/issues/148085
This is a very common usage of ad so we want to make sure it's performant.
From the video, it scrolls quite smoothly, but we want to see the numbers, and keep monitoring it in case of regression.
https://github.com/flutter/flutter/assets/41930132/c7811c15-ac07-4989-a8a9-3c128e08cbe0
*List which issues are fixed by this PR. You must list at least one issue. An issue is not required if the PR fixes something trivial like a typo.*
Fixes https://github.com/flutter/flutter/issues/150230
*If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].*
Reverts: flutter/flutter#150969
Initiated by: goderbauer
Reason for reverting: Failing test in https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8743574743030691569/+/u/run_android_obfuscate_test/stdout
Original PR Author: gmackall
Reviewed By: {christopherfujino, reidbaker}
This change reverts the following previous change:
After the land of https://github.com/flutter/engine/pull/53592, there is some log spam:
```
e: /Users/mackall/.gradle/caches/transforms-3/c1e137371ec1afe9bc9bd7b05823752d/transformed/fragment-1.7.1/jars/classes.jar!/META-INF/fragment_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
e: /Users/mackall/.gradle/caches/transforms-3/d86c7cb1c556fe1655fa56db671c649c/transformed/jetified-activity-1.8.1/jars/classes.jar!/META-INF/activity_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
...
```
I think this is harmless, but still annoying. Upgrading the AGP version fixes it. To be honest, I don't know why (I expected the Kotlin version would do it). But after https://github.com/flutter/flutter/pull/146307, our tests have been running on AGP/Gradle 8.1/8.3 for a while, so it makes sense to upgrade anyways.
In a follow up PR:
1. Also upgrade the tests that were left behind in https://github.com/flutter/flutter/pull/146307, as I think removal of discontinued plugins paved the way here.
After the land of https://github.com/flutter/engine/pull/53592, there is some log spam:
```
e: /Users/mackall/.gradle/caches/transforms-3/c1e137371ec1afe9bc9bd7b05823752d/transformed/fragment-1.7.1/jars/classes.jar!/META-INF/fragment_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
e: /Users/mackall/.gradle/caches/transforms-3/d86c7cb1c556fe1655fa56db671c649c/transformed/jetified-activity-1.8.1/jars/classes.jar!/META-INF/activity_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
...
```
I think this is harmless, but still annoying. Upgrading the AGP version fixes it. To be honest, I don't know why (I expected the Kotlin version would do it). But after https://github.com/flutter/flutter/pull/146307, our tests have been running on AGP/Gradle 8.1/8.3 for a while, so it makes sense to upgrade anyways.
In a follow up PR:
1. Also upgrade the tests that were left behind in https://github.com/flutter/flutter/pull/146307, as I think removal of discontinued plugins paved the way here.
This suite typically takes almost 30 minutes on CI, and some recent
engine->framework rolls failed because the tests exceeded the current 30
minute timeout.
On Linux hosts, there are currently 5 mokey's in staging and 7 in prod.
On Linux hosts, there is currently 1 MotoG4 in staging and 11 in prod.
This PR shifts the ~5 tests that have been running on mokey's for awhile
now from staging to prod, and shifts about half of the MotoG4 benchmarks
from MotoG4's in prod to mokey's in staging.
Following this change, we can disconnect half of the Linux/MotoG4's in
the prod pool and replace them with mokey's in the prod pool.
Part of https://github.com/flutter/flutter/issues/148085
We never made much use of the a02s's, so we can easily free up some
hosts for more mokey devices by eagerly shifting some of the tests on
a02s to mokey.
The test on MotoG4 has lots of flaky timeouts, so this PR moves it to
bringup to stop turning the tree red. This PR also adds the test on
Mokey in anticipation of removing the MotoG4 devices and to see if it
might be flaky there as well.
The macOS devicelab machines are not physically configured for benchmarking at a 60 Hz refresh rate. With Flutter vsync wired up via CVDisplayLink (rather than a hardcoded 60 Hz rate as previously), the bot shows 0 monitors configured and syncs at 30 Hz. This is not a meaningful target to measure against but consumes devicelab CPU cycles.
Given that there is currently no plan in place to acquire and configure a consistent pool of physical benchmarking desktop machines, disabling this test to reduce load on the bots.
See https://github.com/flutter/engine/pull/51210 for a discussion of the decision to move forward with wiring up vsync from macOS and ignore this benchmark.
For future archaeologists, I don't think it's worth removing all the plumbing in the Dart code (in [dev/devicelab/lib/tasks/perf_tests.dart](9d47055640/dev/devicelab/lib/tasks/perf_tests.dart (L963))) that supports running this benchmark on macOS. If we ever acquire hardware suitable for benchmarking with the appropriate HDMI dongle to simulate a 60Hz 4k display, for example, it would be reasonable to re-enable this benchmark.
