firka/flutter
firka/flutter
5
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
flutter/dev/bots
History
auto-submit[bot] 21766a4f9f
Reverts "Support conditional bundling of assets based on --flavor" (#139787) 
Reverts flutter/flutter#132985
Initiated by: christopherfujino
This change reverts the following previous change:
Original Description:
Provides support for conditional bundling of assets through the existing `--flavor` option for `flutter build` and `flutter run`. Closes https://github.com/flutter/flutter/issues/21682. Resolves https://github.com/flutter/flutter/issues/136092

## Change
Within the `assets` section pubspec.yaml, the user can now specify one or more `flavors` that an asset belongs to. Consider this example:

```yaml
# pubspec.yaml
flutter:
  assets:
    - assets/normal-asset.png
    - path: assets/vanilla/ice-cream.png
      flavors: 
        - vanilla
    - path: assets/strawberry/ice-cream.png
      flavors:
        - strawberry
```

With this pubspec,
* `flutter run --flavor vanilla` will not include `assets/strawberry/ice-cream.png` in the build output.
* `flutter run --flavor strawberry` will not include `assets/vanilla/ice-cream.png`.
* `flutter run` will only include `assets/normal-asset.png`.

## Open questions

* Should this be supported for all platforms, or should this change be limited to ones with documented `--flavor` support (Android, iOS, and (implicitly) MacOS)? This PR currently only enables this feature for officially supported platforms.

## Design thoughts, what this PR does not do, etc.

### This does not provide an automatic mapping/resolution of asset keys/paths to others based on flavor at runtime.

The implementation in this PR represents a simplest approach. Notably, it does not give Flutter the ability to dynamically choose an asset based on flavor using a single asset key. For example, one can't use `Image.asset('config.json')` to dynamically choose between different "flavors" of `config.json` (such as `dev-flavor/config.json` or `prod-flavor/config.json`). However, a user could always implement such a mechanism in their project or in a library by examining the flavor at runtime.

### When multiple entries affect the same file and 1) at least one of these entries have a `flavors` list provided and 2) these lists are not equivalent, we always consider the manifest to be ambiguous and will throw a `ToolExit`. 

<details>
For example, these manifests would all be considered ambiguous:

```yaml
assets:
  - assets/
  - path: assets/vanilla.png
    flavors: 
      - vanilla

assets:
  - path: assets/vanilla/
    flavors: 
      - vanilla
  - path: assets/vanilla/cherry.png
     flavor:
      - cherry

# Thinking towards the future where we might add glob/regex support and more conditions other than flavor:
assets:
  - path: assets/vanilla/**
    flavors:
      - vanilla
  - path: assets/**/ios/**
    platforms: 
       - ios

# Ambiguous in the case of assets like "assets/vanilla/ios/icon.svg" since we 
# don't know if flavor `vanilla` and platform `ios` should be combined using or-logic or and-logic.
```

See [this review comment thread](https://github.com/flutter/flutter/pull/132985#discussion_r1381909942) for the full story on how I arrived at this decision.
</details>

### This does not support Android's multidimensional flavors feature (in an intuitive way)

<details>

Conder this excerpt from a Flutter project's android/app/build.gradle file:

```groovy
android {
    // ...

    flavorDimensions "mode", "api"

    productFlavors {
        free {
            dimension "mode"
            applicationIdSuffix ".free"
        }

        premium {
            dimension "mode"
            applicationIdSuffix ".premium"
        }

        minApi23 {
            dimension "api"
            versionNameSuffix "-minApi23"
        }

        minApi21 {
            dimension "api"
            versionNameSuffix "-minApi21"
        }
    }
}
```

In this setup, the following values are valid `--flavor` are valid `freeMinApi21`, `freeMinApi23`, `premiumMinApi21`, and `premiumMinApi23`. We call these values "flavor combinations". Consider the following from the Android documentation[^1]:

> In addition to the source set directories you can create for each individual product flavor and build variant, you can also create source set directories for each combination of product flavors. For example, you can create and add Java sources to the src/demoMinApi24/java/ directory, and Gradle uses those sources only when building a variant that combines those two product flavors.
> 
> Source sets you create for product flavor combinations have a higher priority than source sets that belong to each individual product flavor. To learn more about source sets and how Gradle merges resources, read the section about how to [create source sets](https://developer.android.com/build/build-variants#sourcesets).

This feature will not behave in this way. If a user utilizes this feature and also Android's multidimensional flavors feature, they will have to list out all flavor combinations that contain the flavor they want to limit an asset to:

```yaml
assets:
  - assets/free/
    flavors:
      - freeMinApi21
      - freeMinApi23
```

This is mostly due to a technical limitation in the hot-reload feature of `flutter run`. During a hot reload, the tool will try to update the asset bundle on the device, but the tool does not know the flavors contained within the flavor combination (that the user passes to `--flavor`). Gradle is the source of truth of what flavors were involved in the build, and `flutter run` currently does not access to that information since it's an implementation detail of the build process. We could bubble up this information, but it would require a nontrivial amount of engineering work, and it's unclear how desired this functionality is. It might not be worth implementing.

</details>

See https://flutter.dev/go/flavor-specific-assets for the (outdated) design document. 

<summary>Pre-launch Checklist</summary>

</details>

[^1]: https://developer.android.com/build/build-variants#flavor-dimensions
2023-12-08 06:40:28 +00:00
..
prepare_package
Refactor prepare_package.dart (#139277)
2023-11-30 19:25:50 +00:00
test
Refactor prepare_package.dart (#139277)
2023-11-30 19:25:50 +00:00
accept_android_sdk_licenses.sh
Do not swallow the error from the licenses acceptance (#121243)
2023-02-22 08:55:08 -08:00
allowlist.dart
Upgrade leak_tracker and remove some deps in allow list. (#137806)
2023-11-03 06:20:47 -07:00
analysis_options.yaml
Enable more lints (#91642)
2021-10-14 22:03:03 -07:00
analyze_snippet_code.dart
Enable private field promotion for dev (#134480)
2023-09-12 18:29:00 +00:00
analyze.dart
Analyze against using Stopwatches in the framework (#138507)
2023-11-29 08:49:35 -06:00
browser.dart
Enable avoid_redundant_argument_values lint (#91409) (#91462)
2021-10-08 09:25:14 -07:00
check_code_samples.dart
Write Tests for API Examples of cupertino_text_field.0, data_table.0, icon_button.2 & ink_well.0 (#139258)
2023-11-30 20:07:00 +00:00
codelabs_build_test.sh
…
docs.sh
Bump Dartdoc to 8.0.2 (#139780)
2023-12-07 17:28:32 -08:00
post_process_docs.dart
Deprecate these old APIs (#116793)
2023-04-06 19:53:50 +00:00
prepare_package.dart
Refactor prepare_package.dart (#139277)
2023-11-30 19:25:50 +00:00
pubspec.yaml
Reverts "Support conditional bundling of assets based on --flavor" (#139787)
2023-12-08 06:40:28 +00:00
README.md
Update links and surrounding text for new main-api docs (#138602)
2023-11-17 22:27:53 +00:00
run_command.dart
Migrate customer_testing to sharded tests. (#138659)
2023-11-28 00:41:53 +00:00
service_worker_test.dart
Implement switch expressions in dev/ (#139048)
2023-11-28 23:40:28 +00:00
serviceaccount.enc
…
test.dart
Migrate docs_test to shard. (#139282)
2023-11-30 21:29:41 +00:00
tool_subsharding.dart
Parser machine logs (#118707)
2023-02-02 20:04:06 +00:00
unpublish_package.dart
Implement switch expressions in dev/ (#139048)
2023-11-28 23:40:28 +00:00
utils.dart
Do not use colors in output if running from CI. (#122645)
2023-04-04 18:41:47 +00:00

README.md

Flutter's Build Infrastructure

This directory exists to support building Flutter on our build infrastructure.

Flutter build results are available at:

Flutter infra requires special permissions to retrigger builds on the build dashboard. File an infra ticket to request permission.

The Cirrus-based bots run the test.dart script for each PR and submission. This does testing for the tools, for the framework, and (for submitted changes only) rebuilds and updates the main branch API docs staging site. For tagged dev and beta builds, it also builds and deploys the gallery app to the app stores. It is configured by the .cirrus.yml.

The build dashboard includes post-commit testing run on physical devices. See //dev/devicelab for more information.

LUCI (Layered Universal Continuous Integration)

A set of infra scripts run on Windows, Linux, and Mac machines. The configuration for how many machines and what kind are managed internally by Google. File an infra ticket to request new machine types to be added. Both of these technologies are highly specific to the LUCI project, which is the successor to Chromium's infra and the foundation to Flutter's infrastructure.

Prerequisites

To work on this infrastructure you will need:

  • depot_tools
  • Python package installer: sudo apt-get install python-pip
  • Python coverage package (only needed for training_simulation): sudo pip install coverage

To run prepare_package.dart locally:

  • Make sure the depot_tools is in your PATH. If you're on Windows, you also need an environment variable called DEPOT_TOOLS with the path to depot_tools as value.
  • Run gsutil.py config (or python3 %DEPOT_TOOLS%\gsutil.py on Windows) to authenticate with your auth token.
  • Create a local temp directory. cd into it.
  • Run dart [path to your normal Flutter repo]/dev/bots/prepare_package.dart --temp_dir=. --revision=[revision to package] --branch=[branch to deploy to] --publish.
  • If you're running into gsutil permission issues, check with @Hixie to make sure you have the right push permissions.

Editing a recipe

Flutter has several recipes depending on the test. The recipes share common actions through recipe_modules. Searching the builder config in infra will indicate the recipe used for a test.

Recipes are just Python with some limitations on what can be imported. They are documented by the luci/recipes-py GitHub project.

The typical cycle for editing a recipe is:

  1. Check out the recipes project using git clone https://flutter.googlesource.com/recipes.
  2. Make your edits (probably to files in //recipes/recipes).
  3. Update the tests. Run recipes.py test train to update the existing expected output to match the new output. Verify completely new test cases by altering the GenTests method of the recipe. The recipe is required to have 100% test coverage.
  4. Run led get-builder 'luci.flutter.staging:BUILDER_NAME' | led edit -pa git_ref='refs/pull/<PR number>/head' | led edit -pa git_url='https://github.com/flutter/<repo>' | led edit-recipe-bundle | led launch, where BUILDER_NAME is the builder name (e.g. Linux Engine), and git_ref/git_url is the ref/url of the intended changes to build.
    • If led fails, ensure that your depot_tools checkout is up to date.
  5. To submit a CL, you need a local branch first (git checkout -b [some branch name]).
  6. Upload the patch (git commit, git cl upload), and open the outputted URL to the CL.
  7. Use "Find owners" to get reviewers for the CL

Android Tools

The Android SDK and NDK used by Flutter's Chrome infra bots are stored in Google Cloud. During the build, a bot runs the download_android_tools.py script that downloads the required version of the Android SDK into dev/bots/android_tools.

To check which components are currently installed, download the current SDK stored in Google Cloud using the download_android_tools.py script, then dev/bots/android_tools/sdk/tools/bin/sdkmanager --list. If you find that some components need to be updated or installed, follow the steps below:

How to update Android SDK on Google Cloud Storage

  1. Run Android SDK Manager and update packages $ dev/bots/android_tools/sdk/tools/android update sdk Use android.bat on Windows.

  2. Use the UI to choose the packages you want to install and/or update.

  3. Run dev/bots/android_tools/sdk/tools/bin/sdkmanager --update. On Windows, run sdkmanager.bat instead. If the process fails with an error saying that it is unable to move files (Windows makes files and directories read-only when another process is holding them open), make a copy of the dev/bots/android_tools/sdk/tools directory, run the sdkmanager.bat from the copy, and use the --sdk_root option pointing at dev/bots/android_tools/sdk.

  4. Run dev/bots/android_tools/sdk/tools/bin/sdkmanager --licenses and accept the licenses for the newly installed components. It also helps to run this command a second time and make sure that it prints "All SDK package licenses accepted".

  5. Run upload_android_tools.py -t sdk $ dev/bots/upload_android_tools.py -t sdk

How to update Android NDK on Google Cloud Storage

  1. Download a new NDK binary (e.g. android-ndk-r10e-linux-x86_64.bin)

  2. cd dev/bots/android_tools $ cd dev/bots/android_tools

  3. Remove the old ndk directory $ rm -rf ndk

  4. Run the new NDK binary file $ ./android-ndk-r10e-linux-x86_64.bin

  5. Rename the extracted directory to ndk $ mv android-ndk-r10e ndk

  6. Run upload_android_tools.py -t ndk $ cd ../.. $ dev/bots/upload_android_tools.py -t ndk

Flutter codelabs build test

The Flutter codelabs exercise Material Components in the form of a demo application. The code for the codelabs is similar to, but distinct from, the code for the Shrine demo app in Flutter Gallery.

The Flutter codelabs build test ensures that the final version of the Material Components for Flutter Codelabs can be built. This test serves as a smoke test for the Flutter framework and should not fail. If it does, please address any issues in your PR and rerun the test. If you feel that the test failing is not a direct result of changes made in your PR or that breaking this test is absolutely necessary, escalate this issue by submitting an issue to the MDC-Flutter Team.

Unpublishing published archives

Flutter downloadable archives are built for each release by our continuous integration systems using the prepare_package.dart script, but if something goes very wrong, and a release is published that wasn't intended to be published, the unpublish_package.dart script may be used to remove the package or packages from the channels in which they were published.

For example To remove a published package corresponding to the git hash d444a455de87a2e40b7f576dc12ffd9ab82fd491, first do a dry run of the script to see what it will do:

$ dart ./unpublish_package.dart --temp_dir=/tmp/foo --revision d444a455de87a2e40b7f576dc12ffd9ab82fd491

And once you've verified the output of the dry run to be sure it is what you want to do, run:

$ dart ./unpublish_package.dart --confirm --temp_dir=/tmp/foo --revision d444a455de87a2e40b7f576dc12ffd9ab82fd491

and it will perform the actions. You will of course need to have access to the cloud storage server and have gsutil installed to perform this operation. Only runs on Linux or macOS systems.

See dart ./unpublish_package.dart --help for more details.

Once the package is unpublished, it will not be available from the website for download, and will not be rebuilt (even though there is a tagged revision in the repo still) unless someone forces the packaging build to run again at that revision to rebuild the package.