Use the PlatformViewController2 to register a platform view, allow the
dart side platform view logic to opt into this new platform view. Wires
up an integration test with android_engine_test.
A few changes in this PR (could have been split up, but that would take
an extra 4 hours - 2 days in CI time):
- Removed the old `flutter_driver_android_test` shards and references
- Override `AndroidManifest.xml` per backend, forcing that backend to be
used (no fallbacks) or it fails
- Bumps the Android emulator to 35 due to the use of magic strings, I
think
The check for the Impeller backend actually being used is the honor
system right now, as a bug in `flutter drive`
(https://github.com/flutter/flutter/issues/162087) prevents me from
seeing error output, meaning that it's impossible to be sure a
particular Impeller backend is used (or understand why a crash happens),
so I guess I'll work on that next.
Due to https://github.com/flutter/flutter/issues/162088, we were
accidentally running our Vulkan tests as OpenGLES 🤦🏼 .
Towards https://github.com/flutter/flutter/issues/161333.
This adds the initial `.ci.yaml` configuration (marked `bringup: true`),
without changing the existing shard to avoid the problems we ran into
last time when I renamed the shard atomically, as well as piping for
providing context in which the test will run (Vulkan or OpenGLES), which
can't be introspected in the _running_ app.
In a follow-up PR I'll:
- Edit `AndroidManifest.xml` before the suite, setting `ImpellerBackend`
appropriately
- Move the bringup nodes to presubmit
- Delete the existing (legacy) node
The goal here is to have a great standalone `android_engine_test` suite
that [replaces
`scenario_app/android`](https://github.com/flutter/flutter/pull/160992).
No test is _functionally_ changed in this PR, but overview of changes:
- Finished renaming the suite `android_engine_tests` instead of
`flutter_driver_android`
- Added instructions and an environment variable for local generation of
golden-files (`UPDATE_GOLDENS=1`)
- Added explanations of the individual tests, where they live, and how
to run them locally
- Added a hybrid-composition (HC, not TLHC, which is already tested)
test
- Renamed "other_smiley" to "surface_texture_smiley" (and renamed the
original to "surface_producer_smiley")
- Removed unnecessary ".android" suffix (we will not run this on
anything but Android)
- Added a `tool/deflake.dart` to run a test suite 10x (or custom) times
locally to try and determine flakiness
After this PR, I'll add flags to let you control variants and name the
screenshots accordingly, i.e.:
- API v34 or v35
- OpenGLES or Vulkan (will require an `AndroidManifest.xml` edit during
the test instrumentation)
Towards https://github.com/flutter/flutter/issues/161261 (before I add
more tests).
This is a 1:1 move, with massaging of
`run_flutter_driver_android_tests.dart` to be more automated as tests
are added.
/cc @reidbaker
Now that "auto" is not supported anymore, it makes more sense to make
this dart define false by default. And there's no need to pass
`FLUTTER_WEB_AUTO_DETECT=false` anymore.
This auto-formats all *.dart files in the repository outside of the
`engine` subdirectory and enforces that these files stay formatted with
a presubmit check.
**Reviewers:** Please carefully review all the commits except for the
one titled "formatted". The "formatted" commit was auto-generated by
running `dev/tools/format.sh -a -f`. The other commits were hand-crafted
to prepare the repo for the formatting change. I recommend reviewing the
commits one-by-one via the "Commits" tab and avoiding Github's "Files
changed" tab as it will likely slow down your browser because of the
size of this PR.
---------
Co-authored-by: Kate Lovett <katelovett@google.com>
Co-authored-by: LongCatIsLooong <31859944+LongCatIsLooong@users.noreply.github.com>
Fixes https://github.com/flutter/flutter/issues/156456
There is a known long running test that has been bumping up against the default timeout. This adjusts the perTestTimeout for the devicelab tests to add 15 seconds to stop this from flaking.
The test is question: 82ebb74c64/dev/devicelab/test/run_test.dart (L76)
`test/run_test.dart: run.dart script prints a message after a few seconds when failing to connect (this test takes >10s)`
This updates the codesigning test to account for iOS and macOS binaries in the artifact cache that are _expected_ to not be codesigned.
In https://github.com/flutter/engine/pull/54414 we started bundling dSYM (debugging symbols) within Flutter.xcframework, a requirement for App Store verification using Xcode 16.
We did the same for macOS in https://github.com/flutter/engine/pull/54696.
Unlike the framework dylib, dSYM contents are not directly codesigned (though the xcframework containing them is).
Issue: https://github.com/flutter/flutter/issues/154571
This was partially fixed in flutter/flutter#153787, which added just the extension-safe simulator dSYM, but we also bundle:
* arm64 release dSYM
* arm64 release extension-safe dSYM
* arm64_x86_64 simulator dSYM
In a followup engine patch, we'll stop bundling both the regular and extension-safe simulator dSYMs, but this gets the test passing at tip-of-tree.
Issue: https://github.com/flutter/flutter/issues/116493
A pending change to the flutter_drone LUCI recipe is using a logging library that sets the FLUTTER_TEST_OUTPUTS_DIR environment variable (see https://flutter.googlesource.com/recipes/+/b111cd3ef2297e66905cc48c2cfedce0cf1ba46f)
flutter_driver uses FLUTTER_TEST_OUTPUTS_DIR as its output directory if it is set. run_web_tests invokes flutter_driver and needs to look for outputs in that directory.
When attempting to release the Flutter 3.24.1 hotfix, Flutter framework post submits failed due to the following error:
```
âââ¡ERROR #1âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â UNEXPECTED ERROR!
â Exception: Found unexpected binary in cache: /Volumes/Work/s/w/ir/x/w/flutter/bin/cache/artifacts/engine/ios-release/extension_safe/Flutter.xcframework/ios-arm64_x86_64-simulator/dSYMs/Flutter.framework.dSYM/Contents/Resources/DWARF/Flutter
â #0 verifyExist (file:///Volumes/Work/s/w/ir/x/w/flutter/dev/bots/suite_runners/run_verify_binaries_codesigned_tests.dart:150:12)
â <asynchronous suspension>
â #1 verifyCodesignedTestRunner (file:///Volumes/Work/s/w/ir/x/w/flutter/dev/bots/suite_runners/run_verify_binaries_codesigned_tests.dart:28:3)
â <asynchronous suspension>
â #2 _runFromList (file:///Volumes/Work/s/w/ir/x/w/flutter/dev/bots/utils.dart:601:5)
â <asynchronous suspension>
â #3 main (file:///Volumes/Work/s/w/ir/x/w/flutter/dev/bots/test.dart:125:5)
â <asynchronous suspension>
â
â The test.dart script should be corrected to catch this error and call foundError().
â Some tests are likely to have been skipped.
ââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
```
I've updated the tests to include the listed artifact.
### Open questions
1. Is it possible that the test failed only on the first binary?
2. Do we expect a need to add the remainder of the [entitlement changes](c9b9d5780d)?
3. I am under the assumption that the bots pull from master and are not branch specific, is this a correct assumption?
CC @christopherfujino @cbracken
Closes https://github.com/flutter/flutter/issues/152325.
This PR is large due to generate `flutter create --platforms android`. A quick summary:
- Moves the integration test from `packages/flutter_driver/test` to `dev/integration_tests`
- Created a sample Flutter app that draws a blue rectangle
- Forked a subset of `package:flutter_goldens` that will work on the standalone Dart VM
- Forked a subset of `goldens.dart` (from `flutter_test`) to `src/native/goldens.dart` (i.e. `matchesGoldenFile`)
This ... works locally, but as usual I have no idea if it will work on Skia Gold so let's roll some dice.
Turns out just supporting the right value for `kDebugMode` was a lot simpler than I thought. Debug builds used to never go through the build system code path when using `flutter run`, but now that we have wasm this can occur with the run command.
This should address https://github.com/flutter/flutter/issues/148850
Reverts: flutter/flutter#150969
Initiated by: goderbauer
Reason for reverting: Failing test in https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8743574743030691569/+/u/run_android_obfuscate_test/stdout
Original PR Author: gmackall
Reviewed By: {christopherfujino, reidbaker}
This change reverts the following previous change:
After the land of https://github.com/flutter/engine/pull/53592, there is some log spam:
```
e: /Users/mackall/.gradle/caches/transforms-3/c1e137371ec1afe9bc9bd7b05823752d/transformed/fragment-1.7.1/jars/classes.jar!/META-INF/fragment_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
e: /Users/mackall/.gradle/caches/transforms-3/d86c7cb1c556fe1655fa56db671c649c/transformed/jetified-activity-1.8.1/jars/classes.jar!/META-INF/activity_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
...
```
I think this is harmless, but still annoying. Upgrading the AGP version fixes it. To be honest, I don't know why (I expected the Kotlin version would do it). But after https://github.com/flutter/flutter/pull/146307, our tests have been running on AGP/Gradle 8.1/8.3 for a while, so it makes sense to upgrade anyways.
In a follow up PR:
1. Also upgrade the tests that were left behind in https://github.com/flutter/flutter/pull/146307, as I think removal of discontinued plugins paved the way here.
After the land of https://github.com/flutter/engine/pull/53592, there is some log spam:
```
e: /Users/mackall/.gradle/caches/transforms-3/c1e137371ec1afe9bc9bd7b05823752d/transformed/fragment-1.7.1/jars/classes.jar!/META-INF/fragment_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
e: /Users/mackall/.gradle/caches/transforms-3/d86c7cb1c556fe1655fa56db671c649c/transformed/jetified-activity-1.8.1/jars/classes.jar!/META-INF/activity_release.kotlin_module: Module was compiled with an incompatible version of Kotlin. The binary version of its metadata is 1.8.0, expected version is 1.6.0.
...
```
I think this is harmless, but still annoying. Upgrading the AGP version fixes it. To be honest, I don't know why (I expected the Kotlin version would do it). But after https://github.com/flutter/flutter/pull/146307, our tests have been running on AGP/Gradle 8.1/8.3 for a while, so it makes sense to upgrade anyways.
In a follow up PR:
1. Also upgrade the tests that were left behind in https://github.com/flutter/flutter/pull/146307, as I think removal of discontinued plugins paved the way here.
The previous expression `success || line.contains('--- TEST FAILED ---')` made it such that as soon as success is reported, the test would send "q" on every line of stdout, which is wrong. This fixes the condition such that "q" is sent once, only when `--- TEST SUCCEEDED ---` or `--- TEST FAILED ---` is printed.
Refactor the framework + test harness test suites in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing tests
The refactor of both suites included in this PR because they both depended on util functions and variables that also needed to be refactored.
Part of https://github.com/flutter/flutter/issues/145482
Refactor framework coverage tests in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing framework coverage tests
Part of https://github.com/flutter/flutter/issues/145482
Move from `test.dart` into new `runner_utils.dart` file:
- shard calculation related methods + `shuffleSeed` logic
- env variable flag calculation
- runner wrapper methods `runDartTest` and `_runFromList`
Create web tests suite class containing:
- web long running tests
- runWebHtmlUnitTests
- runWebCanvasKitUnitTests
- runWebSkwasmUnitTests
- the associate helper methods with the above
This pull request aims for improved readability, based on issue #146600.
```dart
// before
Set<Color> _distinctVisibleColors() {
final Set<Color> distinctVisibleColors = <Color>{};
if (top.style != BorderStyle.none) {
distinctVisibleColors.add(top.color);
}
if (right.style != BorderStyle.none) {
distinctVisibleColors.add(right.color);
}
if (bottom.style != BorderStyle.none) {
distinctVisibleColors.add(bottom.color);
}
if (left.style != BorderStyle.none) {
distinctVisibleColors.add(left.color);
}
return distinctVisibleColors;
}
// after
Set<Color> _distinctVisibleColors() {
return <Color>{
if (top.style != BorderStyle.none) top.color,
if (right.style != BorderStyle.none) right.color,
if (bottom.style != BorderStyle.none) bottom.color,
if (left.style != BorderStyle.none) left.color,
};
}
```
Most of the repo should be covered in this PR (aside from `flutter_tools/`, since there was a lot going on in there).
This adds support for adding the `--wasm` flag to `flutter run` and `flutter drive`
* Emits errors if you attempt to use the skwasm renderer without the `--wasm` flag
* Emits errors if you try to use `--wasm` when not using a web device
* Uses the skwasm renderer by default if you pass `--wasm` and no `--web-renderer`
Refactor verify codesigned tests in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing verify codesigned tests
Part of https://github.com/flutter/flutter/issues/145482
Refactor fuchsia_precache suite in order to reduce testing logic in test.dart and allow for later implementing package:test onto the existing fuchsia_precache tests
Part of https://github.com/flutter/flutter/issues/145482
