_Run integration tests for the project_ is not an accurate statement, as
there is nothing inherently about `flutter_driver` that is for
integration tests (for example, it could be just plain automation,
benchmarking, etc).
In addition, clarifies what the most common two arguments might be and
their defaults.
Co-authored-by: Andrew Kolos <andrewrkolos@gmail.com>
Closes https://github.com/flutter/flutter/issues/161449.
~3 LOC, with 203 lines of tests (including an e2e integration test that
it actually works).
Feedback welcome!
(The reason I'm working on this is the ability to pass environment
variables makes it much easier and less hacky to make
`android_engine_test` configurable, i.e. have different expected outputs
for OpenGLES/Vulkan, compare screenshots locally for deflaking, etc).
This gets us closer to enabling [hot
reload](https://github.com/dart-lang/sdk/issues/54934) on the web as
this format is a prerequisite. Historically, we added support for the
DDC module format only to enable hot reload, but that format is not
feasible for the goal, so we added the DDC library bundle format. The
DDC library bundle format is currently represented as the combination of
the `ddc` module format and `canary`. We no longer need to support the
old DDC module format.
- Adds build artifacts to build the SDKs for this format (but only in
sound mode as unsound is unsupported), and removes said artifacts for
the DDC module format.
- Update artifact maps and constants to add the new format and remove
the old format.
- Adds handling of the `canaryFeatures` flag.
- Update dwds to 24.3.0 and use the new
`FrontendServerDdcLibraryBundleStrategyProvider`.
- Add bootstrap code for the new format. Kept DDC module format
bootstrap code as it's used internally.
- Updates tests.
I ran `spinning_square` with the new module format to verify that it can
run.
## Pre-launch Checklist
- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [X] I signed the [CLA].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [ ] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [X] All existing and new tests are passing.
imperative apply has been deprecated since #139690
part of #121541
Doing this will make things easier for us during the conversion of
`flutter.groovy` into Kotlin.
Closes https://github.com/flutter/flutter/issues/160218.
Basically, replaces `String?` with `sealed class TestCompilerResult {}`,
and ensures `errorMessage` is propogated.
We'll be using this path now for _all_ integration tests (not just for
web-specific things), so I'd like to get error messages.
Closes https://github.com/flutter/flutter/issues/160219.
This hard-coded package configuration is no longer strictly correct
as-of Dart 3.6.0; a pub _workspace_
(https://dart.dev/tools/pub/workspaces) can appear at a higher-level
than a package, and if the package is part of the workspace, tooling is
expected to (automatically) find `.dart_tool/package_config.json` at a
higher-level.
For example, the _engine_ sub-repo uses a
[workspace](9fd5bddc65/engine/src/flutter/pubspec.yaml (L82)),
which means that, for example,
`%ENGINE%/tools/engine_tool/.dart_tool/package_config.json` will _never_
exist (it will be at `%ENGINE%/.dart_tools/package_config.json`.
As currently defined, the test-golden comparator interface will fail
with a package that uses workspaces. By removing the flag (and letting
automatic `--packages` resolution occur), I _believe_ the problem is
automatically resolved (but I'll let CI prove that for us).
Now that we're in a monorepo, it makes sense to use the in-tree engine
by default when using `--local-engine` or `--local-web-sdk` rather than
looking for a sibling directory like we used to.
This PR rolls in a number of breaking changes from dart-lang/native:
* `BuildMode` is no longer part of the protocol, so Flutter no longer
passes it in.
* This means all code dealing with the name conflict between
`native_assets_cli.BuildMode` and `flutter_tools.BuildMode` has been
cleaned up.
* Also, the logs no longer mention the build mode.
* The tests still exercise both modes, because linking only happens in
release mode.
* `OS` is no longer part of the main protocol, but of the "code"
"protocol extension".
* The code now aligns more with `OS?` being nullable in a bunch of
places, since it is nullable if there's no code assets.
* The OS-specific config is nested in an object per OS.
* `CCompilerConfig`s fields are non-nullable now.
* So instead of passing an object with nullable fields around, a null
instead of the object is returned in various places.
* `FileSystem` is now passed in to the native assets builder.
This PR contains no feature changes.
This PR will need to be followed up by restricting what environment
variables are passed in (similar to
https://github.com/dart-lang/native/pull/1764), I will do this in a
follow up PR.
Tests:
* All existing features should be covered by existing tests.
Work towards https://github.com/flutter/flutter/issues/143299.
Work towards https://github.com/flutter/flutter/issues/160043.
---
This PR implements, end-to-end, support for `matchesGoldenFile` when (a)
running with `package:integration_test` (b) on a device, such as an
Android emulator, Android device, iOS simulator, or iOS device, where
the _runner_ of a test file does not have process and local-file system
access.
There are multiple parts to this PR; I could make it smaller than 1K
lines, but the bulk of that is tests, and it would mean landing PRs that
are incomplete and unused, which does not seem useful - so instead here
is a quick overview of the PR's contents - questions/feedback welcome,
and I am willing to break code out or land incremental refactors if
requested.
1. Augmented `flutter_platform.dart` (used for iOS and Android), similar
to
[`flutter_web_platform.dart`](1398dc7eec/packages/flutter_tools/lib/src/test/flutter_web_platform.dart (L117-L128)),
now creates and uses
[`test_golden_comparator.dart`](https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/test/test_golden_comparator.dart)
to proxy calls (coming from the VM service protocol) for golden-file
updates and comparisons to a `flutter_tester` process. A full
explanation of how (or why) it works this way is too hard to include
here, but see https://github.com/flutter/flutter/pull/160215 for more
details.
1. Added `VmServiceProxyGoldenFileComparator`, which is a currently
unused (outside of a single e2e test) comparator that forwards calls to
`compare` and `update` to the VM service protocol (of which, the other
side of this is implemented above, in `flutter_platform.dart`. The idea
is that this comparator would be used automatically when running in an
integration test on a device that requires it (similar to how web works
today), but that is **not** wired up yet and requires additional work in
`flutter_tools`.
1. Added two unit tests (of both the client and server), and a full
e2e-test using it to run `matchesGoldenFile`.
Now that "auto" is not supported anymore, it makes more sense to make
this dart define false by default. And there's no need to pass
`FLUTTER_WEB_AUTO_DETECT=false` anymore.
dart_style 3.0.1 comes with some minor style fixes:
https://github.com/dart-lang/dart_style/blob/main/CHANGELOG.md#301
This PR applies this fixes in bulk across the repository. (Otherwise,
the next person touching these files would have been the one updating
them to the new format by running the formatter).
Work around: https://github.com/flutter/flutter/issues/160689.
I locally verified that we have not regressed what this was testing
(that just invoking `//flutter/bin/dart` will not build the flutter
tool).
In the future a hook may be invoked multiple times with different
`supportedAssetTypes` (soon to be renamed to `buildAssetTypes`).
The hook should only emit those asset types that are in
`supportedAssetTypes` - anything else is an error. Right now flutter
happens to invoke hooks only with `Code` asset types, but more asset
types are coming, for which this PR is a preparation for.
The integration test framework that waits for transitions and
(optionally) takes actions on transitions allows to match patterns.
If one uses a RegExp pattern than the framework only checks whether a
line contains the given RegExp pattern.
If one uses a String pattern it matches it exactly.
=> We add a `Barrier.contains()` and `Multiple.contains()` that allow
matching a line with if it contains the String (just like in RegExp)
=> This makes tests simpler as they don't have to know about the exact
padding of progres bar etc. Those may be irrelevant for the purpose of
the integration test and only complicate it.
This auto-formats all *.dart files in the repository outside of the
`engine` subdirectory and enforces that these files stay formatted with
a presubmit check.
**Reviewers:** Please carefully review all the commits except for the
one titled "formatted". The "formatted" commit was auto-generated by
running `dev/tools/format.sh -a -f`. The other commits were hand-crafted
to prepare the repo for the formatting change. I recommend reviewing the
commits one-by-one via the "Commits" tab and avoiding Github's "Files
changed" tab as it will likely slow down your browser because of the
size of this PR.
---------
Co-authored-by: Kate Lovett <katelovett@google.com>
Co-authored-by: LongCatIsLooong <31859944+LongCatIsLooong@users.noreply.github.com>
Part of https://github.com/flutter/flutter/issues/102983.
`<FlutterManifest>.generateSyntheticPackage` _never_ meant generate a
synthetic package 😒, it only meant "we _might_ need to generate a
synthetic package because localizations are being generated and the
default, unless otherwise specified, is to generate a synthetic
package".
Renamed as `generateLocalizations` and added some strategic TODOs in
places where removing the `package:flutter_gen` feature
(https://github.com/flutter/flutter/issues/102983) will allow us to
cleanup this erroneous code and technical debt.
Simplified a bit code (just a refactor) in the process, and fixes a bug
that `flutter packages get` would generate internationalization files
even if `flutter: generate: true` was not present in `pubspec.yaml` that
was revealed as part of fixing this up.
/cc @sigurdm.
Without this change, the order that tests run matter and it's easy to
add tests without remembering this flag.
(There are existing tests that forgot it too, they just happen to work
if shuffled in a way that, well, works)
Towards https://github.com/flutter/flutter/issues/160257.
I intentionally skipped `create_test.dart`, as that requires updating
the generated template which uses l10n, a feature changing. I'll do that
in the "big bang" PR that finally enables the feature to avoid getting
us into a bad state.
Part of https://github.com/flutter/flutter/issues/160043, makes it
easier to add https://github.com/flutter/flutter/pull/160131.
This PR has no functional changes to any of the code, but does refactor
both the code and tests:
- Makes a number of always non-null but not migrated to non-null
properties, well, not-null
- Creates two concrete methods (`update` and `compare` versus a
positional nullable boolean)
- Uses type signatures instead of `String?` to explain the possible
results of the methods
- Renames the mysterious `shellPath` variable to `flutterTesterBinPath`
- Expands and rewrites internally-facing doc comments
- Moves `WebRenderer` environment variable setting to
`flutter_web_platform.dart`
- Makes the tests have less duplication, and check for update/compare
cases
After this PR, I can use it in the non-web branch of the Flutter tool
without any hacks or TODOS :)
/cc @eyebrowsoffire (trivial web refactoring), @camsim99 (changes being
made to tool).
When running `dart format` over these lines the `// ignore` ended up on
a line where it wasn't properly ignoring the lint. This adjusts the
placement of `// ignore`s so they will continue to ignore the right
thing even after the code is auto formatted.
I am hoping that if we do this now the large PR that formats the entire
repo will go in smoother without manual intervention.
