This PR introduces a `NativeAssetsManifest.json` next to the
`AssetManifest.bin` and `FontManifest.json`. This removes the need for
embedding the native assets mapping inside the kernel file and enables
decoupling native assets building and bundling from the kernel
compilation in flutter tools. This means `flutter run` no longer does a
dry run of `hook/build.dart` hooks.
(It also means all isolate groups will have the same native assets.
However, since Flutter does not support `Isolate.spawnUri` from kernel
files, this is not a regression.)
Implementation details:
* g3 is still using kernel embedding.
https://github.com/flutter/flutter/pull/142016 introduced an argument to
embed a `native_assets.yaml` inside `flutter attach` and `flutter run`
(the outer flutter process), but it is not used in `flutter assemble`
(the inner process when doing `flutter run`). So, those arguments need
to still be respected. However, all other logic related to embedding a
yaml encoding in the kernel file has been removed.
* All dry-run logic has been removed. 🎉
* The `KernelSnapshot` target no longer depends on the
`InstallCodeAssets` target. Instead, the various OS-specific
"BundleAsset" targets now depend on the `InstallCodeAssets` target. The
`InstallCodeAssets` invokes the build hooks and produces the
`NativeAssetsManifest.json`. The various "BundleAsset" commands
synchronize the `NativeAssetsManifest.json` to the app bundle.
* `InstallCodeAssets` produces a `native_assets.json`, which is renamed
to `NativeAssetsManifest.json` in the various "Bundle" targets. This
means that all unit tests of the "Bundle" targets now need to create
this file. (Similar to how `app.dill` is expected to exist because
`KernelSnapshot` is a dependency of the "Bundle" targets.)
* Because dynamic libraries need to be code signed (at least on iOS and
MacOS), the bundling of the dylibs is _not_ migrated to reuse
`_updateDevFS` (which is used for ordinary assets). Only the 2nd and 3rd
invocation of `flutter assemble` from `xcodebuild` has access to the
code signing identity.
Relevant tests:
* test/integration.shard/isolated/native_assets_test.dart - runs
`flutter run` with native assets including hot restart and hot reload.
TODO:
* Undo engine-roll in this PR after engine has rolled in.
Issue:
* https://github.com/flutter/flutter/issues/154425
Related PRs:
* https://dart-review.googlesource.com/c/sdk/+/388161
* https://github.com/flutter/engine/pull/56727
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
This PR attempts to improve clarity of androids section of `flutter
doctor -v` output by providing explicit information about which JDK is
being used and how to configure a different one if needed.
### Before
```console
• Java binary at: /Users/user/Applications/Android Studio Ladybug Feature Drop 2024.2.2 Canary 2.app/Contents/jbr/Contents/Home/bin/java
```
### After
1. When JDK is from Android Studio:
```console
• Java binary at: /Users/users/Applications/Android Studio Ladybug Feature Drop 2024.2.2 Canary 2.app/Contents/jbr/Contents/Home/bin/java
This is the JDK bundled with latest Android Studio installation
To manually set a custom JDK path, use: `flutter config --jdk-dir="path/to/jdk"`
```
2. When JDK is from JAVA_HOME env variable:
```console
• Java binary at: /Users/user/Applications/Android Studio Ladybug Feature Drop 2024.2.2 Canary 2.app/Contents/jbr/Contents/Home/bin/java
This JDK is specified by JAVA_HOME environment variable
To manually set a custom JDK path, use: `flutter config --jdk-dir="path/to/jdk"`
```
3. When path to JDK is set in flutter config:
```console
• Java binary at: /Users/user/Applications/Android Studio Ladybug Feature Drop 2024.2.2 Canary 2.app/Contents/jbr/Contents/Home/bin/java
This JDK was found in system PATH
To change current JDK, run: `flutter config --jdk-dir="path/to/jdk"`
```
4. When java binary is found in PATH (as fallback)
```console
• Java binary at: /Users/user/Applications/Android Studio Ladybug Feature Drop 2024.2.2 Canary 2.app/Contents/jbr/Contents/Home/bin/java
This JDK is specified in Flutter configuration
To manually set a custom JDK path, use: `flutter config --jdk-dir="path/to/jdk"`
```
### Motivation
I think it's described in
https://github.com/flutter/flutter/issues/153156#issuecomment-2336814991.
TLDR; many developers struggle with Java-related issues and more verbose
doctor's output will (presumably) improve DX in that part.
fixes#153156
## Pre-launch Checklist
- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
---------
Co-authored-by: Andrew Kolos <andrewrkolos@gmail.com>
Killing the flutter_tools parent process may leak child processes
spawned by the tools.
Also wait for the message indicating that DevTools has started before
stopping flutter_tools. If DevTools has not fully launched, then there
may be a race between DevTools startup and flutter_tools shutdown.
Fixes https://github.com/flutter/flutter/issues/159154
Speculative fix for https://github.com/flutter/flutter/issues/157640.
A few observations:
- I was a bit paranoid about indented files meaning something was parsed
incorrectly
- I removed `android.enableR8=true` (not used elsewhere)
- I removed `android.experimental.enableNewResourceShrinker=true` (not
used elsewhere)
- I matched the rest of the `jvmargs` used in the standard template file
... let's hope this does good things?
See https://github.com/flutter/flutter/issues/159154.
See https://github.com/flutter/flutter/pull/159169.
Before this PR, it appeared we were accidentally leaking (keeping
active) `flutter_tester` instances (or any test device) after `flutter
run` completion, even if the runner was not explicitly detached. I
_think_ this is a bug, but I'll check with the tools team and possibly
@jonahwilliams before finalizing this.
/cc @jason-simmons
---------
Co-authored-by: Andrew Kolos <andrewrkolos@gmail.com>
Closes https://github.com/flutter/flutter/issues/158560.
I believe but am not sure as of
https://github.com/flutter/flutter/pull/159170 merging, many process
flakes that were consuming memory and in turn, making Gradle
particularly sensitive to timing out or being killing by the OS for
low-memory, have been rectified.
It is possible there are additional problems, but they aren't visible at
the moment.
I'd like to re-enable these and keep tracking their stability.
The dart-lang/native repository contains a `Target` class that is almost
not needed anymore. The remaining uses are mainly due to kernel asset
mapping (which we may be able to remove in the future).
This PR removes usages of that `Target` (in favor of `Architecture`)
class in most places in flutter tools.
This makes the code also cleaner because we no longer have an implicit
assumption that
a `List<Target>` all belong to the same operating system.
Towards https://github.com/flutter/flutter/issues/51421.
```sh
flutter_tools % dart test test/integration.shard/break_on_framework_exceptions_test.dart
02:38 +29: All tests passed!
54692 ttys003 0:00.02 /opt/homebrew/bin/zsh -il
```
Requires https://github.com/flutter/flutter/pull/159115 for the process
cleanup to work properly, but this is safe to land as-is, otherwise we
still accumulate `flutter` processes over and over as each test case
runs which is not WAI.
Closes https://github.com/flutter/flutter/issues/20949.
Signals (such as SIGTERM or SIGKILL) end up flowing through
`exitWithHooks`, which in turn, after running hooks, call `exit().` That
means, as a result, any `try { } finally { }` guarded execution may
_not_ run, which happens to also be how `flutter_tester` instances are
cleaned up if they have not terminated.
This PR adds in-progress `flutter_tester` runs (or any platform
`flutter_platform` supports) to the shutdown hooks, guaranteeing that
the finalizers (which in turn, kill the process) are _always_ executed
as long as either the test completes, _or_ `exitWithHooks` is called.
The existing integration tests (`integration.shard/test_test.dart`)
still pass as well.
It's possible that the tool can be in the process of shutting down,
which could result in the temp directory being deleted after the
shutdown hooks run before we check if `output` exists. If this happens,
we shouldn't crash but just carry on as if no devices were found as the
tool will exit on its own.
Fixes https://github.com/flutter/flutter/issues/141892
---------
Co-authored-by: Andrew Kolos <andrewrkolos@gmail.com>
Towards https://github.com/flutter/flutter/issues/159000 as part of
debugging issues such as
https://github.com/flutter/flutter/issues/158560.
On a `bringup: true` shard, run a number of `flutter build apk` tests in
succession to try and routinely trigger timeouts and crashes, so we can
test other hypotheses on how to fix this problem (i.e. potentially
around increasing memory, changing daemon configuration, aggressively
killing processes, etc).
Currently the `NativeAsset` target in flutter tools is responsible for
two things:
* performing the dart build (in the app as well as all transitive pub
dependencies)
* taking output (shared libraries) from this build and copying them
around
This intermingling of responsibilities leads to more complex code and
potentially unnecessary work: If the source code changed (e.g. `.c`
files change) we have to run the dart build again. But doing so may
result in the same shared libraries (e.g. adding comments to the `.c`
code). Currently we're going to copy the shared libraries despite them
having not changed, which then may cause upstream things to be dirtied
(if it's based on timestamp of files) and re-built.
Instead this PR splits this `NativeAsset` into the two orthogonal pieces
* `DartBuild` target that is responsible for the dart build
* `InstallCodeAssets` that is responsible for copying shared libraries
to the right place and producing a `native_assets.yaml`.
This decoupling is also preparation for a future where a dart build can
produce other kinds of assets (e.g. data assets) and is used in the web
build as well. (The web build would use `DartBuild` but not
`InstalCodeAssets`).
After the dart build is done, the flutter tool has to bundle the
produced shared libraries, which it does that by copying them around.
Though the code assumed that all code assets are shared libraries to be
bundled, whereas in fact one can have code assets without any actual
code (ones that are installed on the target system already or artificial
code assets whose symbols get resolved from executable / process).
=> Using non-bundled code assets currently results in null pointer
exceptions and/or cast errors.
=> We update the copy code to only operate on code assets that have a
shared library to bundle.
We also update the copy routines by removing copy&past'ed - but slightly
different - printing code into the shared caller function.
In release builds linking of native assets is enabled. The build step is
only a temprary step, it's output is given to the link step which then
returns all final assets (effectively a map-reduce system). Assets that
aren't sent to a specific linker could be conceptually viewed as sent to
a linker that will emit it's input as-is.
=> The code currently took output of build & link step and therefore
accumulated assets that aren't explicitly sent to a linker twice.
=> This led to performing work twice for those (e.g. copying them twice)
This PR changes this such that if linking mode is enabled, we only rely
on the output of the link phase.
That in return means many tests that mock the native asset builds need
to be updated to mock the output of the link phase.
This started happening after moving DDS to launch from `dart
development-service` rather than `DartDevelopmentService` (see
33b402d24c2131dcf97e272ec3d59c0249762e7f) . This state error was
originally meant to be thrown when some string parsing failed, but is
currently wrapping the `DartDevelopmentServiceException`.
Fixes https://github.com/flutter/flutter/issues/158537
This adds support for web 1.x.x to be included in the plugin template, when creating a web plugin.
I don't have an issue referenced. Related to https://github.com/dart-lang/web/issues/316
I was poking around in logger.dart when I noticed a few classes have unnecessary exposed member variables. This PR reduces visibility of these in the pursuit of making these classes slightly easier to grok.
<details>
<summary> Pre-launch checklist </summary>
</details>
Cleans up https://github.com/flutter/flutter/pull/155800. In summary, `ResidentRunner`/`FlutterDevice` have branching behavior around logging that depends on the type of `DeviceLogReader` on the `FlutterDevice` instance. Let's instead move this behavior to the `DeviceLogReader` implementations.
My apologies for the large diff. Much of this is a refactor that was a bit too difficult to separate into its own commits.
Here are the main two changes
* Replaces the mutable `connectedVmService` field on the `DeviceLogReader` class with a new method `provideVmService`. This serves largely the same purpose as the mutable field, but it allows for asynchronous code. This is where we put the logic that used to exist in `FlutterDevice.tryInitLogReader`.
* Removes the `tryInitLogReader` method from `FlutterDevice`. This method served to set the `appPid` field on the `FlutterDevice`'s `DeviceLogReader` instance. This was only used in the case of Android to filter out logs unrelated to the flutter app coming from the device, so we can move this logic to `AdbLogReader`'s implementation of `provideVmService`.
Closes https://github.com/flutter/flutter/issues/158012.
This is (effectively) a user-facing NOP, which is exchanging an
on-by-default command-line argument (`--implicit-pubspec-resolution`)
for an off-by-default global feature flag
(`explicit-package-dependencies`). It matches the mental model better,
is less painstaking to maintain and feed throughout, and will be easier
to globally flip on/off in a future PR.
---------
Co-authored-by: Andrew Kolos <andrewrkolos@gmail.com>
It would be nice to have somewhere to iterate and experiment with what we can do to either fix frequent timeout problems we have with `flutter build apk` (across platforms, to be clear, though I've just started with Linux) or get more information on why the crashes/timeouts happen.
Open to other ways to doing this (though preferably _not_ LED).
In the future, it will be possible for Swift Package Manager to be enabled on one but not all platforms (see https://github.com/flutter/flutter/issues/151567#issuecomment-2455941279).
This moves the `usesSwiftPackageManager` property from the platform-agnostic `Project` to the platform-specific `IosProject` and `MacOSProject`.
This will allow the `IosProject` and `MacOSProject` to return different values for `usesSwiftPackageManager` in the future. For now, both of these projects will always return the same value.
Part of https://github.com/flutter/flutter/issues/151567
Closes https://github.com/flutter/flutter/issues/157629.
Avoids failures in the style:
```txt
00:37 +4597 ~3 -1: /Users/matanl/Developer/flutter/packages/flutter_tools/test/commands.shard/permeable/build_appbundle_test.dart: analytics logs success [E]
Expected: contains Event:<{"eventName":"flutter_command_result","eventData":{"commandPath":"create","result":"success","commandHasTerminal":false,"maxRss":147013632}}>
Actual: [
Event:{"eventName":"command_usage_values","eventData":{"workflow":"create","commandHasTerminal":false,"createProjectType":"app","createAndroidLanguage":"kotlin","createIosLanguage":"swift"}},
Event:{"eventName":"flutter_command_result","eventData":{"commandPath":"create","result":"success","commandHasTerminal":false,"maxRss":143261696}},
Event:{"eventName":"timing","eventData":{"workflow":"flutter","variableName":"create","elapsedMilliseconds":527,"label":"success"}},
Event:{"eventName":"command_usage_values","eventData":{"workflow":"appbundle","commandHasTerminal":false,"buildAppBundleTargetPlatform":"android-arm,android-arm64,android-x64","buildAppBundleBuildMode":"release"}},
Event:{"eventName":"flutter_command_result","eventData":{"commandPath":"appbundle","result":"success","commandHasTerminal":false,"maxRss":147013632}},
Event:{"eventName":"timing","eventData":{"workflow":"flutter","variableName":"appbundle","elapsedMilliseconds":10,"label":"success"}}
]
Which: does not contain Event:<{"eventName":"flutter_command_result","eventData":{"commandPath":"create","result":"success","commandHasTerminal":false,"maxRss":147013632}}>
```
I am guessing <https://api.flutter.dev/flutter/dart-io/ProcessInfo/maxRss.html> is not guaranteed to be stable while running, and as a result is not. This stabilizes the value for a test that I believe does not care about getting the "real" value anyway.
