Reverts flutter/flutter#136851
Initiated by: CaseyHillers
This change reverts the following previous change:
Original Description:
One of the reasons gathering coverage information is expensive is that we have to force compile every function in the libraries we're interested in. Without this, functions that haven't been invoked (so haven't been compiled) won't have any line number information, so the coverage tool doesn't know which lines to add to the list of misses. In flutter's case, the test infra spawns many VMs, and each of these needs to recompile all those libraries.
To fix this, we need a way of skipping force compilation for libraries we've already seen in previous tests, without losing the information about which lines in each library are coverable. So I [added](https://github.com/dart-lang/coverage/pull/466) the `coverableLineCache` to `coverage.collect` in package:coverage v1.7.0. This cache starts out empty, but fills up with lists of all the lines that are coverable for every library as coverage is gathered. package:coverage can then tell the VM not to force compile any libraries in this cache (using `getSourceReport`'s `librariesAlreadyCompiled` param). So the first test suite will still have to compile everything, but subsequent test suites will be much faster.
This speeds up coverage collection significantly, for large test suites:
| Running flutter/packages/flutter tests... | Time | Overhead |
| --- | --- | --- |
| without coverage | 8:53 | - |
| with coverage | 20:25 | 130% |
| with `coverableLineCache` | 12:21 | 40% |
Bug: https://github.com/flutter/flutter/issues/100751
One of the reasons gathering coverage information is expensive is that we have to force compile every function in the libraries we're interested in. Without this, functions that haven't been invoked (so haven't been compiled) won't have any line number information, so the coverage tool doesn't know which lines to add to the list of misses. In flutter's case, the test infra spawns many VMs, and each of these needs to recompile all those libraries.
To fix this, we need a way of skipping force compilation for libraries we've already seen in previous tests, without losing the information about which lines in each library are coverable. So I [added](https://github.com/dart-lang/coverage/pull/466) the `coverableLineCache` to `coverage.collect` in package:coverage v1.7.0. This cache starts out empty, but fills up with lists of all the lines that are coverable for every library as coverage is gathered. package:coverage can then tell the VM not to force compile any libraries in this cache (using `getSourceReport`'s `librariesAlreadyCompiled` param). So the first test suite will still have to compile everything, but subsequent test suites will be much faster.
This speeds up coverage collection significantly, for large test suites:
| Running flutter/packages/flutter tests... | Time | Overhead |
| --- | --- | --- |
| without coverage | 8:53 | - |
| with coverage | 20:25 | 130% |
| with `coverableLineCache` | 12:21 | 40% |
Bug: https://github.com/flutter/flutter/issues/100751
Fixes https://github.com/flutter/flutter/issues/130277
This PR does two things:
1. introduce a hidden `flutter build _preview` command, that will build a debug windows desktop app and copy it into the SDK's binary cache. This command is only intended to be run during packaging.
2. introduce a new device type, called `PreviewDevice`, which relies on the prebuilt desktop debug app from step 1, copies it into the target app's assets build folder, and then hot reloads their dart code into it.
Fixes https://github.com/flutter/flutter/issues/134279.
Changes:
* Moves all tests of `--dart-define-from-file` behavior from `build_bundle_test.dart` and `assemble_test.dart` to `flutter_command_test.dart`.
* Deletes a duplicate test of malformed JSON detection behavior.
* Renames the `useDartDefineFromFileOption` method of `FlutterCommand` to `_usesDartDefineFromFileOption`. This 1) makes the name more consistent with the other `uses*Option` methods and 2) hides the method since it is not used outside of the file.
* Renames several tests to better articulate what is under test and what the expected result is.
* Adds a test for the case where a `.env` file with a malformed line is provided to `--dart-define-from-file`.
Adds support for a new --web-header option to flutter run.
Creates a workaround for https://github.com/flutter/flutter/issues/127902
This PR allows adding additional headers for the flutter run web server. This is useful to add headers like Cross-Origin-Embedder-Policy and Cross-Origin-Opener-Policy without the use of a proxy server. These headers are required enable advanced web features. This approach provides flexibility to the developer to make use of the feature as they see fit and is backward-compatible. One tradeoff is that it increases the surface area to support for future changes to the flutter web server.
https://github.com/flutter/flutter/issues/127902 is not fully addressed by this change. The solution for that task will be more opinionated. This PR creates a general-purpose workaround for anyone who needs a solution sooner while the bigger solution is developed.
This is the final change needed to address https://github.com/flutter/flutter/issues/133467
This allows us to use the platform dill that is built by the engine when compiling apps. This also fixes the `--local-web-sdk` flag when compiling to wasm (which previously didn't work without some serious tweaking of the engine build output).
Part of:
- https://github.com/flutter/flutter/issues/128251
Currently, when we want to use the analytics instance from `package:unified_analytics`, we are just grabbing it from globals. However, with the legacy analytics instance, there are some things we check to return a no-op version of the instance.. for example, if we are running on bots or a non standard branch, we use a no-op instance
This PR uses the same previous checks for the new analytics instance
* This is completely broken on the Impeller renderer, see: https://github.com/flutter/flutter/issues/135052
* Even on the Skia renderer, this gives a software rasterized screenshot which will absolutely look different from a native rendering screenshot.
I plan to remove this functionality from the engine.
This implements https://github.com/flutter/flutter/issues/132654#issuecomment-1738221257, namely:
Make `Chromium.close` more robust:
* Send `SIGTERM` and wait up to 5 seconds, if the process exited, great! Return from the function.
* If the process has not exited, then send a `SIGKILL`, which is a much firmer way to exit a process. Same as before, wait up to 5 seconds, if the process exited, great! Return from the function.
* If it still hasn't exited then give up trying to exit Chromium, just print a warning to the console and return from the function.
Bonus: a few nullability fixes and extra `-v` logging.
Fixes https://github.com/flutter/flutter/issues/132654
Resolves#128046.
Adds a services API that allows flutter app developers to write app code that determines `--flavor` the app was built with.
This is implemented by having the tool adding the value of `--flavor` to its list of dart environment declarations, which will be available to the app at run time. Specifically,`FLUTTER_APP_FLAVOR` is set. I chose this implementation for its simplicity. There is some precedent for this, but only for web ([example](cd2f3f5e78/packages/flutter_tools/lib/src/runner/flutter_command.dart (L1231))).
Support for FFI calls with `@Native external` functions through Native assets on Windows. This enables bundling native code without any build-system boilerplate code.
For more info see:
* https://github.com/flutter/flutter/issues/129757
### Implementation details for Windows.
Mainly follows the design of https://github.com/flutter/flutter/pull/134031.
Specifically for Windows in this PR is the logic for finding the compiler `cl.exe` and environment variables that contain the paths to the Windows headers `vcvars.bat` based on `vswhere.exe`.
So there appears to be a race situation between the flutter CLI and Xcode. In the CLI, we update the `CONFIGURATION_BUILD_DIR` in the Xcode build settings and then tell Xcode to install, launch, and debug the app. When Xcode installs the app, it should use the `CONFIGURATION_BUILD_DIR` to find the bundle. However, it appears that sometimes Xcode hasn't processed the change to the build settings before the install happens, which causes it to not be able to find the bundle.
Fixes https://github.com/flutter/flutter/issues/135442
---
Since it's a timing issue, there's not really a consistent way to test it.
I was able to confirm that it works, though, by using the following steps:
1. Create a flutter project
2. Open the project in Xcode
3. `flutter clean`
4. `flutter run --profile -v`
If I saw a print line `stderr: CONFIGURATION_BUILD_DIR: build/Debug-iphoneos`, that means it first found the old and incorrect `CONFIGURATION_BUILD_DIR` before updating to the the new, so I was able to confirm that it would wait until it updated.
Reland of #134031. (Reverted in #135069.) Contains the fix for b/301051367 together with cl/567233346.
Support for FFI calls with `@Native external` functions through Native assets on Linux. This enables bundling native code without any build-system boilerplate code.
For more info see:
* https://github.com/flutter/flutter/issues/129757
### Implementation details for Linux.
Mainly follows the design of https://github.com/flutter/flutter/pull/130494.
Some differences are:
* Linux does not support cross compiling or compiling for multiple architectures, so this has not been implemented.
* Linux has no add2app.
The assets copying is done in the install-phase of the CMake build of a flutter app.
CMake requires the native assets folder to exist, so we create it also when the feature is disabled or there are no assets.
### Tests
This PR adds new tests to cover the various use cases.
* packages/flutter_tools/test/general.shard/linux/native_assets_test.dart
* Unit tests the Linux-specific part of building native assets.
It also extends various existing tests:
* packages/flutter_tools/test/integration.shard/native_assets_test.dart
* Runs (incl hot reload/hot restart), builds, builds frameworks for Linux and flutter-tester.
Adds warning to `flutter create` command that checks if detected Java version is compatible with the template AGP and template Gradle versions. If a developer is building for Android and their Java version is incompatible with either the AGP or Gradle versions that Flutter currently supports by default for new Flutter projects, then
- a warning will show noting the incompatibility and
- steps will be shown to fix the issue, the recommended option being to configure a new compatible Java version given that Flutter knows we can support the template Gradle/AGP versions and updating them manually may be risky (feedback on this approach would be greatly appreciated!)
Given that the template AGP and Gradle versions are compatible, this PR assumes that the detected Java version may only conflict with one of the template AGP or Gradle versions because:
- the minimum Java version for a given AGP version is less than the maximum Java version compatible for the minimum Gradle version required for that AGP version (too low a Java version will fail AGP compatibility test, but not Gradle compatibility).
- the maximum Java version compatible with minimum Gradle version for a given AGP version is higher than minimum Java version required for that AGP version (too high a Java version will fail Gradle compatibility test, but not AGP compatibility test).
Fixes https://github.com/flutter/flutter/issues/130515 in the sense that `flutter create foo`; `cd foo`; `flutter run` should always be successful.
Reverts flutter/flutter#134031
context: b/301051367
Looked at the error message from the broken TAP target, but seems like the failure might be non trivial to resolve. Would it be okay if we revert this for now while it is being triaged?
This PR modifies the web build slightly to create an `AssetManifest.json`, that is a JSON(base64)-encoded version of the `AssetManifest.bin` file.
_(This should enable all browsers to download the file without any interference, and all servers to serve it with the correct headers.)_
It also modifies Flutter's `AssetManifest` class so it loads and uses said file `if (kIsWeb)`.
### Issues
* Fixes https://github.com/flutter/flutter/issues/124883
### Tests
* Unit tests added.
* Some tests that run on the Web needed to be informed of the new filename, but their behavior didn't have to change (binary contents are the same across all platforms).
* I've deployed a test app, so users affected by the BIN issue may take a look at the PR in action:
* https://dit-tests.web.app
Support for FFI calls with `@Native external` functions through Native assets on Linux. This enables bundling native code without any build-system boilerplate code.
For more info see:
* https://github.com/flutter/flutter/issues/129757
### Implementation details for Linux.
Mainly follows the design of https://github.com/flutter/flutter/pull/130494.
Some differences are:
* Linux does not support cross compiling or compiling for multiple architectures, so this has not been implemented.
* Linux has no add2app.
The assets copying is done in the install-phase of the CMake build of a flutter app.
CMake requires the native assets folder to exist, so we create it also when the feature is disabled or there are no assets.
### Tests
This PR adds new tests to cover the various use cases.
* packages/flutter_tools/test/general.shard/linux/native_assets_test.dart
* Unit tests the Linux-specific part of building native assets.
It also extends various existing tests:
* packages/flutter_tools/test/integration.shard/native_assets_test.dart
* Runs (incl hot reload/hot restart), builds, builds frameworks for Linux and flutter-tester.
Speeds up the native assets target in the backend by
1. changing other targets `gen_dart_plugin_registrant` and
`release_unpack_ios` to do async I/O,
2. not reparsing the package config, and
3. not calling `dart pub deps --json` for 0 or 1 packages (fixed
package:native_assets_builder).
* https://github.com/flutter/flutter/issues/134427
```
[ +2 ms] native_assets: Starting due to {}
[ +2 ms] Skipping target: gen_localizations
[ +1 ms] gen_dart_plugin_registrant: Starting due to {InvalidatedReasonKind.inputChanged: The following inputs have updated contents: /Users/dacoharkes/flt/engine/flutter/examples/hello_world/.dart_tool/package_config_subset}
[ +33 ms] gen_dart_plugin_registrant: Complete
[ +107 ms] release_unpack_ios: Complete
[ +60 ms] Writing native_assets.yaml.
[ +7 ms] Writing /Users/dacoharkes/flt/engine/flutter/examples/hello_world/.dart_tool/flutter_build/be2692bbfbc0b9a27fcd2422d52354c6/native_assets.yaml done.
[ ] native_assets: Complete
```
->
```
[ +4 ms] native_assets: Starting due to {}
[ ] Skipping target: gen_localizations
[ +1 ms] gen_dart_plugin_registrant: Starting due to {InvalidatedReasonKind.inputChanged: The following inputs have updated contents: /Users/dacoharkes/flt/engine/flutter/examples/hello_world/.dart_tool/package_config_subset}
[ +31 ms] Writing native_assets.yaml.
[ +8 ms] Writing /Users/dacoharkes/flt/engine/flutter/examples/hello_world/.dart_tool/flutter_build/f9451a65a465bfab70d004e21d6cc1d6/native_assets.yaml done.
[ +1 ms] native_assets: Complete
```
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [ ] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] All existing and new tests are passing.
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
Fixes https://github.com/flutter/flutter/issues/134566.
Prior to this fix, `ShutdownHooks` were run in the private helper
function `_exit()` defined in the `package:flutter_tools/runner.dart`
library. Independent of this, the tool had signal handling logic that
traps SIGINT and SIGTERM. However, these handlers called `exit()` from
`dart:io`, and didn't run these hooks.
This PR moves the `_exit()` private helper to
`package:flutter_tools/src/base/process.dart` and renames it to
`exitWithHooks()`, so that it can be called by the signal handlers in
`package:flutter_tools/src/base/signals.dart`.
Xcode uses the CONFIGURATION_BUILD_DIR build setting to determine the location of the bundle to build and install. When launching an app via Xcode with the Xcode debug workflow (for iOS 17 physical devices), temporarily set the CONFIGURATION_BUILD_DIR to the location of the bundle so Xcode can find it.
Also, added a Xcode Debug version of the `microbenchmarks_ios` integration test since it uses `flutter run --profile` without using `--use-application-binary`.
Fixes https://github.com/flutter/flutter/issues/134186.
Support for FFI calls with `@Native external` functions through Native assets on MacOS and iOS. This enables bundling native code without any build-system boilerplate code.
For more info see:
* https://github.com/flutter/flutter/issues/129757
### Implementation details for MacOS and iOS.
Dylibs are bundled by (1) making them fat binaries if multiple architectures are targeted, (2) code signing these, and (3) copying them to the frameworks folder. These steps are done manual rather than via CocoaPods. CocoaPods would have done the same steps, but (a) needs the dylibs to be there before the `xcodebuild` invocation (we could trick it, by having a minimal dylib in the place and replace it during the build process, that works), and (b) can't deal with having no dylibs to be bundled (we'd have to bundle a dummy dylib or include some dummy C code in the build file).
The dylibs are build as a new target inside flutter assemble, as that is the moment we know what build-mode and architecture to target.
The mapping from asset id to dylib-path is passed in to every kernel compilation path. The interesting case is hot-restart where the initial kernel file is compiled by the "inner" flutter assemble, while after hot restart the "outer" flutter run compiled kernel file is pushed to the device. Both kernel files need to contain the mapping. The "inner" flutter assemble gets its mapping from the NativeAssets target which builds the native assets. The "outer" flutter run get its mapping from a dry-run invocation. Since this hot restart can be used for multiple target devices (`flutter run -d all`) it contains the mapping for all known targets.
### Example vs template
The PR includes a new template that uses the new native assets in a package and has an app importing that. Separate discussion in: https://github.com/flutter/flutter/issues/131209.
### Tests
This PR adds new tests to cover the various use cases.
* dev/devicelab/bin/tasks/native_assets_ios.dart
* Runs an example app with native assets in all build modes, doing hot reload and hot restart in debug mode.
* dev/devicelab/bin/tasks/native_assets_ios_simulator.dart
* Runs an example app with native assets, doing hot reload and hot restart.
* packages/flutter_tools/test/integration.shard/native_assets_test.dart
* Runs (incl hot reload/hot restart), builds, builds frameworks for iOS, MacOS and flutter-tester.
* packages/flutter_tools/test/general.shard/build_system/targets/native_assets_test.dart
* Unit tests the new Target in the backend.
* packages/flutter_tools/test/general.shard/ios/native_assets_test.dart
* packages/flutter_tools/test/general.shard/macos/native_assets_test.dart
* Unit tests the native assets being packaged on a iOS/MacOS build.
It also extends various existing tests:
* dev/devicelab/bin/tasks/module_test_ios.dart
* Exercises the add2app scenario.
* packages/flutter_tools/test/general.shard/features_test.dart
* Unit test the new feature flag.
Sometimes `ios-deploy` loses connection to the device after installing, starting debugserver, and launching. This is shown with an error message like:
```
Process 579 exited with status = -1 (0xffffffff) lost connection
```
This happens frequently in our CI system: https://github.com/flutter/flutter/issues/120808
Usually in CI, on retry it'll work and pass - so this is an attempt to retry without failing the test first. It's not guaranteed to fix since we're unable to recreate this error locally.
â¦on file
The deeplink validation tool will become an static app so it can't no longer access vm services.
The goal will be then to turn them into flutter analyze command similar to `flutter analyze --android --[options]` that static app can use on.
This pr only removes vm services and turn the api to dump a output file instead of printing everything to stdout.
