Towards https://github.com/flutter/flutter/issues/161333.
This adds the initial `.ci.yaml` configuration (marked `bringup: true`),
without changing the existing shard to avoid the problems we ran into
last time when I renamed the shard atomically, as well as piping for
providing context in which the test will run (Vulkan or OpenGLES), which
can't be introspected in the _running_ app.
In a follow-up PR I'll:
- Edit `AndroidManifest.xml` before the suite, setting `ImpellerBackend`
appropriately
- Move the bringup nodes to presubmit
- Delete the existing (legacy) node
The goal here is to have a great standalone `android_engine_test` suite
that [replaces
`scenario_app/android`](https://github.com/flutter/flutter/pull/160992).
No test is _functionally_ changed in this PR, but overview of changes:
- Finished renaming the suite `android_engine_tests` instead of
`flutter_driver_android`
- Added instructions and an environment variable for local generation of
golden-files (`UPDATE_GOLDENS=1`)
- Added explanations of the individual tests, where they live, and how
to run them locally
- Added a hybrid-composition (HC, not TLHC, which is already tested)
test
- Renamed "other_smiley" to "surface_texture_smiley" (and renamed the
original to "surface_producer_smiley")
- Removed unnecessary ".android" suffix (we will not run this on
anything but Android)
- Added a `tool/deflake.dart` to run a test suite 10x (or custom) times
locally to try and determine flakiness
After this PR, I'll add flags to let you control variants and name the
screenshots accordingly, i.e.:
- API v34 or v35
- OpenGLES or Vulkan (will require an `AndroidManifest.xml` edit during
the test instrumentation)
The arm64 Macs should have more memory than the Linux machines that are
available in the devicelab. The analyzer benchmark spawns a Dart
analysis server process that has a peak memory consumption of around
7GB, which may be activating the OOM killer on the Linux bots.
See https://github.com/flutter/flutter/issues/161306
Random seed 349875783 is failing half the time.
```
flutter run -v -d IR65S859OJ4HAY6P --profile lib/benchmark_collection.dart --dart-define=seed=349875783
```
I'm concerned that this sometimes passes, which points to some hidden
timing bugs.
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
"name": "Mac_benchmark flutter_gallery_macos__compile"
}
-->
Issue link: https://github.com/flutter/flutter/issues/160495
Co-authored-by: Chris Bracken <chris@bracken.jp>
This auto-formats all *.dart files in the repository outside of the
`engine` subdirectory and enforces that these files stay formatted with
a presubmit check.
**Reviewers:** Please carefully review all the commits except for the
one titled "formatted". The "formatted" commit was auto-generated by
running `dev/tools/format.sh -a -f`. The other commits were hand-crafted
to prepare the repo for the formatting change. I recommend reviewing the
commits one-by-one via the "Commits" tab and avoiding Github's "Files
changed" tab as it will likely slow down your browser because of the
size of this PR.
---------
Co-authored-by: Kate Lovett <katelovett@google.com>
Co-authored-by: LongCatIsLooong <31859944+LongCatIsLooong@users.noreply.github.com>
I fixed the autoroller in https://github.com/flutter/flutter/pull/160059
but forgot to mark the shard non-bring-up. Note, it's currently failing
in post-submit, but only because bringUp shards run in the staging pool,
which does not have access to the GitHub access token necessary for the
roller to operate.
<!-- start_original_pr_link -->
Reverts: flutter/flutter#159756
<!-- end_original_pr_link -->
<!-- start_initiating_author -->
Initiated by: gmackall
<!-- end_initiating_author -->
<!-- start_revert_reason -->
Reason for reverting: Made the tree red due to some std out.
<!-- end_revert_reason -->
<!-- start_original_pr_author -->
Original PR Author: gmackall
<!-- end_original_pr_author -->
<!-- start_reviewers -->
Reviewed By: {reidbaker}
<!-- end_reviewers -->
<!-- start_revert_body -->
This change reverts the following previous change:
In preparation for changing engine builds to be unstripped by default
https://github.com/flutter/engine/pull/52852, which will allow us to
make progress towards resolving
https://github.com/flutter/flutter/issues/60240.
Tricks AGP in to downloading the NDK when building a flutter app (that
uses the FGP, which to my knowledge is all ways of building flutter
apk/aab/aar).
I want to follow this up by modifying the tool to search for the log
line that the NDK is missing (making it throw an error in that case) as
a safeguard, because that would be the last line of defense before we
accidentally build a bloated app. The safeguard won't work for add to
app, from what I understand, because while they use the FGP (so they
should be forced to be download the NDK) they don't invoke the flutter
tool, and therefore won't invoke the custom error handling of
[`gradle_errors.dart`](https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/android/gradle_errors.dart)
Some details
1. Respects existing values for `externalNativeBuild.cmake.path` for
apps that actually use it.
2. Silences some warnings that would otherwise appear for add to app
builds or builds that manually invoke gradle:
```
C/C++: CMake Warning (dev) in CMakeLists.txt:
C/C++: No project() command is present. The top-level CMakeLists.txt file must
C/C++: contain a literal, direct call to the project() command. Add a line of
C/C++: code such as
C/C++: project(ProjectName)
C/C++: near the top of the file, but after cmake_minimum_required().
C/C++: CMake is pretending there is a "project(Project)" command on the first
C/C++: line.
C/C++: This warning is for project developers. Use -Wno-dev to suppress it.
C/C++: CMake Warning:
C/C++: Manually-specified variables were not used by the project:
C/C++: CMAKE_EXPORT_COMPILE_COMMANDS
C/C++: CMAKE_LIBRARY_OUTPUT_DIRECTORY
C/C++: CMAKE_RUNTIME_OUTPUT_DIRECTORY
```
3. Our ci installs the NDK at an abnormal place that AGP can't find
without help. I've modified all the `build.gradle`s that we have checked
in to point to the pre-installed path. **But some of our tests make a
new app from the templates, and those tests will now start downloading
the NDK** (as they won't be able to find it at it's current path from
templates). We could resolve this by actually fixing
https://github.com/flutter/flutter/issues/136666. This would be a very
significant lift from what I understand - we rely on this hardcoding in
a lot of places in our infra.
Fixes https://github.com/flutter/flutter/issues/155576
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
<!-- end_revert_body -->
Co-authored-by: auto-submit[bot] <flutter-engprod-team@google.com>
In preparation for changing engine builds to be unstripped by default
https://github.com/flutter/engine/pull/52852, which will allow us to
make progress towards resolving
https://github.com/flutter/flutter/issues/60240.
Tricks AGP in to downloading the NDK when building a flutter app (that
uses the FGP, which to my knowledge is all ways of building flutter
apk/aab/aar).
I want to follow this up by modifying the tool to search for the log
line that the NDK is missing (making it throw an error in that case) as
a safeguard, because that would be the last line of defense before we
accidentally build a bloated app. The safeguard won't work for add to
app, from what I understand, because while they use the FGP (so they
should be forced to be download the NDK) they don't invoke the flutter
tool, and therefore won't invoke the custom error handling of
[`gradle_errors.dart`](https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/android/gradle_errors.dart)
Some details
1. Respects existing values for `externalNativeBuild.cmake.path` for
apps that actually use it.
2. Silences some warnings that would otherwise appear for add to app
builds or builds that manually invoke gradle:
```
C/C++: CMake Warning (dev) in CMakeLists.txt:
C/C++: No project() command is present. The top-level CMakeLists.txt file must
C/C++: contain a literal, direct call to the project() command. Add a line of
C/C++: code such as
C/C++: project(ProjectName)
C/C++: near the top of the file, but after cmake_minimum_required().
C/C++: CMake is pretending there is a "project(Project)" command on the first
C/C++: line.
C/C++: This warning is for project developers. Use -Wno-dev to suppress it.
C/C++: CMake Warning:
C/C++: Manually-specified variables were not used by the project:
C/C++: CMAKE_EXPORT_COMPILE_COMMANDS
C/C++: CMAKE_LIBRARY_OUTPUT_DIRECTORY
C/C++: CMAKE_RUNTIME_OUTPUT_DIRECTORY
```
3. Our ci installs the NDK at an abnormal place that AGP can't find
without help. I've modified all the `build.gradle`s that we have checked
in to point to the pre-installed path. **But some of our tests make a
new app from the templates, and those tests will now start downloading
the NDK** (as they won't be able to find it at it's current path from
templates). We could resolve this by actually fixing
https://github.com/flutter/flutter/issues/136666. This would be a very
significant lift from what I understand - we rely on this hardcoding in
a lot of places in our infra.
Fixes https://github.com/flutter/flutter/issues/155576
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [ ] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
---------
Co-authored-by: Gray Mackall <mackall@google.com>
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
"name": "Mac_benchmark flutter_view_macos__start_up"
}
-->
Issue link: https://github.com/flutter/flutter/issues/159540
Co-authored-by: Chris Bracken <chris@bracken.jp>
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
"name": "Mac_benchmark hello_world_macos__compile"
}
-->
Issue link: https://github.com/flutter/flutter/issues/159542
Co-authored-by: Chris Bracken <chris@bracken.jp>
Update to the latest definitions of emulator configurations.
Specifically to past https://crrev.com/c/5908894 which added `gpu_mode:
"swangle_indirect"` to the api 35 proto. This should help with emulator
stability on versions past android 13.
Related to b/371020223
Related: https://github.com/flutter/engine/pull/56313
Research notes / my findings for how this system works at
go/flutter-android-ci-emulator
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I signed the [CLA].
- [x] All existing and new tests are passing.
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
"name": "Mac_x64 hot_mode_dev_cycle_macos_target__benchmark"
}
-->
Issue link: https://github.com/flutter/flutter/issues/158568
It would be nice to have somewhere to iterate and experiment with what we can do to either fix frequent timeout problems we have with `flutter build apk` (across platforms, to be clear, though I've just started with Linux) or get more information on why the crashes/timeouts happen.
Open to other ways to doing this (though preferably _not_ LED).