Previously we were appending Flutter's fork of third_party/pyyaml to the Python sys.path. In the case where a bot image or local install already has pyyaml installed elsewhere on the path, ours will fail to be picked up. Instead prepend to the path.
Issue spotted by jsimmons@.
Related: https://github.com/flutter/engine/pull/54001
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This many explicit calls are all but useless unless you are doing bringup. And we do that in unopt modes anyway. Having this on in even the debug and profile opt modes was turning out to be pretty egregious.
Reverts: flutter/engine#54019
Initiated by: jonahwilliams
Reason for reverting: does not work on CI emulators
Original PR Author: jonahwilliams
Reviewed By: {chinmaygarde}
This change reverts the following previous change:
Fixes https://github.com/flutter/flutter/issues/147533
This works now that the backpressure issue is fixed. Maybe the wait semaphore didn't work, but the CPU wait on the fence does.
This is only ever used for gen_snapshot_arm64 where the value will only
ever be 'clang_x64'. If we were to migrate builds to arm64 hosts, the
gen_snapshot_x64 target would be broken as the script stands today.
This flag is never passed by recipe infra or outside callers, and can
thus be safely inlined. The same refactoring was performed for
generating universal gen_snapshots in #53954.
A followup patch will update the location where this gen_snapshot binary
is generated (and adds an arm64 host binary build), and the merged
universal gen_snapshot_${target_cpu} will be written to the root output
directory.
This is minor pre-factoring to simplify the followup patch, which
migrates to universal binaries for iOS.
Issue: https://github.com/flutter/flutter/issues/101138
Issue: https://github.com/flutter/flutter/issues/69157
The `//flutter/build/archives:artifacts` target is used to build a zip
archive (artifacts.zip) of host tools such as flutter_tester, the Dart
kernel compiler, the impellerc shader compiler, and other tooling that
is bundled in debug-mode host builds.
This moves the `!is_android` to the definition of
`build_engine_artifacts`. This is required because of the way that we
produce 32-bit arm gen_snapshot for Android on Windows hosts, which
relies on the regular x64 host toolchain due to us having no 32-bit arm
toolchain for Windows. As such, `current_toolchain == host_toolchain` on
that platform.
```
build_engine_artifacts =
flutter_build_engine_artifacts &&
(current_toolchain == host_toolchain ||
(is_linux && !is_chromeos && current_cpu != "arm") || is_mac || is_win)
```
On iOS builds, we don't have this issue since `current_toolchain` will
be one of:
* `//build/toolchain/mac:ios_clang_arm`
* `//build/toolchain/mac:ios_clang_arm_sim`
* `//build/toolchain/mac:ios_clang_x64_sim`
Whereas `host_toolchain` will be one of:
* `//build/toolchain/mac:clang_arm64`
* `//build/toolchain/mac:clang_x64`
This patch also adds documentation to clarify the purpose of this target
and where related artifacts are produced so that future readers don't
need to do a deep dive into our build plumbing to figure this out.
While the target itself is primarily intended for producing host
binaries, one target binary (gen_snapshot) is bundled into the same
archive bundle as the host tools. This should be refactored such that
just like iOS and Android, they are bundled into their own
target-platform-specific archive, and the tool code accordingly updated
to pull these down into the appropriate cache directory.
As a side-note, on macOS we do rely on this archive target for the host
tools, but the bundled gen_snapshot is unused -- instead, one produced
by the //flutter/sky/tools/create_macos_gen_snapshots.py script used.
This should be fixe in a followup patch.
Related: https://github.com/flutter/flutter/issues/38935
Identified while trying to resolve:
Issue: https://github.com/flutter/flutter/issues/101138
Issue: https://github.com/flutter/flutter/issues/69157
## Pre-launch Checklist
- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I signed the [CLA].
- [X] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
Fixes https://github.com/flutter/flutter/issues/148139
Block on the CPU rather than immediately beginning encoding. Seems to fix the synchronization issues that we knew about. We need to find a way to use AHB swapchains, as it seems like correct synchronization via either SurfaceView or a SurfaceControl hierarchy will require exposing our rendering as a SurfaceTransaction.
Reverts: flutter/engine#54002
Initiated by: kevmoo
Reason for reverting: the issue in google 3 has been fixed.
Original PR Author: auto-submit[bot]
Reviewed By: {fluttergithubbot}
This change reverts the following previous change:
Reverts: flutter/engine#54000
Initiated by: chingjun
Reason for reverting: Broke Google Testing.
See b/352191023 for more details.
Original PR Author: kevmoo
Reviewed By: {jonahwilliams}
This change reverts the following previous change:
Fixes https://github.com/flutter/flutter/issues/151679
Reverts: flutter/engine#54000
Initiated by: chingjun
Reason for reverting: Broke Google Testing.
See b/352191023 for more details.
Original PR Author: kevmoo
Reviewed By: {jonahwilliams}
This change reverts the following previous change:
Fixes https://github.com/flutter/flutter/issues/151679
We were using a very old version of this library that broke with
Python3.12. This updates to a Python 3.12 compatible SHA.
The specific breakage at the former commit was that
`collections.Hashable` was deprecated and moved to
`collections.abc.Hashable` in Python 3.7. It was removed in Python 3.8.
For anyone on newer versions of Python3, such as Python 3.9.6 which
ships with macOS 15 Sequoia, this is broken.
The error on versions of Python newer than 3.8 is:
```
________ running 'python3 src/flutter/tools/pub_get_offline.py' in '/Users/chris/Developer/flutter/engine'
Traceback (most recent call last):
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 167, in <module>
sys.exit(main())
^^^^^^
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 157, in main
if not package_uses_workspace_resolution(package):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/pub_get_offline.py", line 76, in package_uses_workspace_resolution
return yaml.safe_load(pubspec_file).get('resolution') == 'workspace'
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/__init__.py", line 75, in load
return loader.get_single_data()
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 37, in get_single_data
return self.construct_document(node)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 46, in construct_document
for dummy in generator:
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 398, in construct_yaml_map
value = self.construct_mapping(node)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 204, in construct_mapping
return super().construct_mapping(node, deep=deep)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/chris/Developer/flutter/engine/src/flutter/tools/../third_party/pyyaml/lib3/yaml/constructor.py", line 126, in construct_mapping
if not isinstance(key, collections.Hashable):
^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'collections' has no attribute 'Hashable'
Error: Command 'python3 src/flutter/tools/pub_get_offline.py' returned non-zero exit status 1 in /Users/chris/Developer/flutter/engine
```
## Pre-launch Checklist
- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [ ] I listed at least one issue that this PR fixes in the description
above.
- [X] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I signed the [CLA].
- [X] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
The Linux embedder had a number of handlers for Flutter channels that
were named plugin/handler/manager. Rename these all to handler to be
consistent and reduce confusion with Flutter plugins (which these don't
use the infrastructure for).
Our treatment of the interaction between offset and transform was incorrect. Modified our platform view unit tests to cover more cases of nested offsets and transforms.
Converts several AIKS golden tests to use DisplayList as the mechanism.
In order to convert some of the tests, new factory methods were added to DlColor and tested with new unit tests (an earlier golden test conversion PR had a version of this as well).
Also, a new DisplayList record op was created for ClipOval to handle the AIKS clipping golden tests, but this new recording op is not used from Flutter `ui` code (no plumbing to call it from `lib/ui/painting` or to convert any other DisplayList call to use the new record). An earlier attempt to add the new recording op caused a large number of golden changes upstream so this version will only be used for internal tests and support to use it from apps will follow in more targeted PRs to better manage golden changes. This PR should not result in any changes to goldens outside of internal engine tests.
In #53524, we started producing universal `gen_snapshot_arm64` and `gen_snapshot_x64` binaries. This migrates away from bundling `gen_snapshot` binaries with x64-only host architecture to universal binaries that bundle both x64 and arm64 host architectures.
Also adds a dependency on `flutter/lib/snapshot:create_macos_gen_snapshots` to the profile build, where it was previously missing. Presumably, `gen_snapshot` was being built as a side-effect of one of the other targets.
Issue: https://github.com/flutter/flutter/issues/101138
Issue: https://github.com/flutter/flutter/issues/69157
[C++, Objective-C, Java style guides]: https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
This patch updates `create_macos_gen_snapshots.py` to take the path to
the input `gen_snapshot` binaries rather than just the path to the
buildroot, and hardcoding the binary names and buildroot-relative paths
internally.
This has a couple advantages:
* Those grepping the source for gen_snapshot to see where it's bundled
will immediately find it in the recipe JSON as a parameter to this
script.
* In future (the next patch I send out) we can pass different input
paths that point to gen_snapshot binaries that are universal binaries.
The current binaries are single-architecture x64 host binaries.
This is a refactoring with no semantic change, therefore no tests have
changed.
Background
----------
`create_macos_gen_snapshot.py` is used to:
* Generate a named copy of gen_snapshot for the specified target
architecture: e.g. gen_snapshot_arm64, gen_snapshot_x64
* Strip bitcode, if present. Today, bitcode is no longer enabled by
default.
* Create a zip archive containing the (two) gen_snapshot binaries and
bundle with an entitlements.txt file.
Issue: https://github.com/flutter/flutter/issues/151848
This is refactoring to support:
Issue: https://github.com/flutter/flutter/issues/101138
Issue: https://github.com/flutter/flutter/issues/69157
## Pre-launch Checklist
- [X] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [X] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [X] I read and followed the [Flutter Style Guide] and the [C++,
Objective-C, Java style guides].
- [X] I listed at least one issue that this PR fixes in the description
above.
- [X] I've dealt with this script more than any one person should ever
have to deal with such a thing in one lifetime.
- [ ] I added new tests to check the change I am making or feature I am
adding, or the PR is [test-exempt]. See [testing the engine] for
instructions on writing and running engine tests.
- [X] I updated/added relevant documentation (doc comments with `///`).
- [X] I signed the [CLA].
- [X] All existing and new tests are passing.
If you need help, consider asking for advice on the #hackers-new channel
on [Discord].
<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#overview
[Tree Hygiene]: https://github.com/flutter/flutter/wiki/Tree-hygiene
[test-exempt]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/wiki/Style-guide-for-Flutter-repo
[C++, Objective-C, Java style guides]:
https://github.com/flutter/engine/blob/main/CONTRIBUTING.md#style
[testing the engine]:
https://github.com/flutter/flutter/wiki/Testing-the-engine
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/wiki/Tree-hygiene#handling-breaking-changes
[Discord]: https://github.com/flutter/flutter/wiki/Chat
