firka/flutter
firka/flutter
5
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update readme for LUCI (#29023)

Browse Source 
* update readme for LUCI

* Update README.md
This commit is contained in:
Dan Field 2019-03-08 09:25:13 -08:00 committed by GitHub
parent da7649586a
commit 4be4830c2f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 57 additions and 57 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

114
dev/bots/README.md
View File

@ -5,41 +5,37 @@ This directory exists to support building Flutter on our build infrastructure.
The results of such builds are viewable at: The results of such builds are viewable at:
* https://cirrus-ci.com/github/flutter/flutter/master * https://cirrus-ci.com/github/flutter/flutter/master
- Testing done on PRs and submitted changes on GitHub. - Testing done on PRs and submitted changes on GitHub.
* https://build.chromium.org/p/client.flutter/console * https://ci.chromium.org/p/flutter/
- Additional testing and processing done after changes are submitted. - Additional testing and processing done after changes are submitted.
The Chromium infra bots do not allow forcing new builds from outside The LUCI infra requires permissions to retrigger or schedule builds. Contact
the Google network. Contact @eseidelGoogle or another Google member of @kf6gpe or another Google member of the Flutter team if you need to do that.
the Flutter team if you need to do that.
The [Cirrus](https://cirrus-ci.org)-based bots run the The [Cirrus](https://cirrus-ci.org)-based bots run the [`test.dart`](test.dart)
[`test.dart`](test.dart) script for each PR and submission. This does script for each PR and submission. This does testing for the tools, for the
testing for the tools, for the framework, and (for submitted changes framework, and (for submitted changes only) rebuilds and updates the master
only) rebuilds and updates the master branch API docs branch API docs [staging site](https://master-docs.flutter.dev/).
[staging site](https://master-docs-flutter-io.firebaseapp.com). For tagged dev and beta builds, it also builds and deploys the gallery app to
For tagged dev and beta builds, it also builds and deploys the gallery the app stores. It is configured by the [.cirrus.yml](/.cirrus.yml).
app to the app stores. It is configured by the
[.cirrus.yml](/.cirrus.yml).
We also have post-commit testing with actual devices, in what we call We also have post-commit testing with actual devices, in what we call our
our [devicelab](../dev/devicelab/README.md). [devicelab](../dev/devicelab/README.md).
## Chromium infra bots ## LUCI (Layered Universal Continuous Intergration)
This part of our infrastructure is broken into two parts. A buildbot A [set of recipes](https://chromium.googlesource.com/chromium/tools/build.git/+/master/scripts/slave/recipes/flutter)
master specified by our are run on Windows, Linux, and Mac machines. The configuration for how many
[builders.pyl](https://chromium.googlesource.com/chromium/tools/build.git/+/master/masters/master.client.flutter/builders.pyl) machines and what kind are managed internally by Google. Contact @kf6gpe or
file, and a [set of another Google member of the Flutter team if you suspect changes are needed
recipes](https://chromium.googlesource.com/chromium/tools/build.git/+/master/scripts/slave/recipes/flutter) there. Both of these technologies are highly specific to the [LUCI](https://github.com/luci)
which we run on that master. Both of these technologies are highly project, which is the successor to Chromium's infra. We're just borrowing some
specific to Google's Chromium project. We're just borrowing some of of their infrastructure.
their infrastructure.
### Prerequisites ### Prerequisites
To work on this infrastructure you will need: To work on this infrastructure you will need:
- [depot_tools](http://www.chromium.org/developers/how-tos/install-depot-tools) - [depot_tools](https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.html#_setting_up)
- Python package installer: `sudo apt-get install python-pip` - Python package installer: `sudo apt-get install python-pip`
- Python coverage package (only needed for `training_simulation`): `sudo pip install coverage` - Python coverage package (only needed for `training_simulation`): `sudo pip install coverage`
@ -58,7 +54,8 @@ To run prepare_package.dart locally:
### Getting the code ### Getting the code
The following will get way more than just recipe code, but it _will_ get the recipe code: The following will get way more than just recipe code, but it _will_ get the
recipe code:
```bash ```bash
mkdir chrome_infra mkdir chrome_infra
@ -82,7 +79,7 @@ and
- build/scripts/slave/recipes/flutter/flutter.py - build/scripts/slave/recipes/flutter/flutter.py
- build/scripts/slave/recipes/flutter/engine.py - build/scripts/slave/recipes/flutter/engine.py
Recipes are just Python. They are Recipes are just Python with some limitations on what can be imported. They are
[documented](https://github.com/luci/recipes-py/blob/master/doc/user_guide.md) [documented](https://github.com/luci/recipes-py/blob/master/doc/user_guide.md)
by the [luci/recipes-py github project](https://github.com/luci/recipes-py). by the [luci/recipes-py github project](https://github.com/luci/recipes-py).
@ -90,33 +87,30 @@ The typical cycle for editing a recipe is:
1. Make your edits (probably to files in 1. Make your edits (probably to files in
`//chrome_infra/build/scripts/slave/recipes/flutter`). `//chrome_infra/build/scripts/slave/recipes/flutter`).
1. Update the tests. Run `build/scripts/slave/recipes.py test train` to update 2. Update the tests. Run `build/scripts/slave/recipes.py test train` to update
existing expected output to match the new output. Verify completely new test existing expected output to match the new output. Verify completely new test
cases by altering the `GenTests` method of the recipe. cases by altering the `GenTests` method of the recipe. The recipe is required
The recipe is required to have 100% test coverage. to have 100% test coverage.
1. Run `build/scripts/slave/recipes.py run flutter/<repo> slavename=<slavename> 3. Run `build/scripts/slave/recipes.py run flutter/<repo> slavename=<slavename>
mastername=client.flutter buildername=<buildername> buildnumber=1234` where `<repo>` is one mastername=client.flutter buildername=<buildername>
of `flutter` or `engine`, and `slavename` and `buildername` can be looked up buildnumber=1234` where `<repo>` is one of `flutter` or `engine`, and
from the *Build Properties* section of a [recent `slavename` and `buildername` can be looked up from the *Build Properties*
section of a [recent
build](https://build.chromium.org/p/client.flutter/one_line_per_build). build](https://build.chromium.org/p/client.flutter/one_line_per_build).
1. To submit a CL, you need a local branch first (`git checkout -b [some branch name]`). 4. To submit a CL, you need a local branch first (`git checkout -b [some branch name]`).
1. Upload the patch (`git commit`, `git cl upload`) and send it to someone in 5. Upload the patch (`git commit`, `git cl upload`) and send it to someone in
the `recipes/flutter/OWNERS` file for review. the `recipes/flutter/OWNERS` file for review.
### Editing the client.flutter buildbot master ### The infra config repository
Flutter uses Chromium's fancy The [flutter/infra](https://github.com/flutter/infra) repository contains
[builders.pyl](https://chromium.googlesource.com/infra/infra/+/master/doc/users/services/buildbot/builders.pyl.md) configuration files for the dashboard, builder groups, scheduling, and
master generation system. Chromium hosts 100s (if not 1000s) of buildbot individual builders. Edits to this may require changes other internal Google
masters and thus has lots of infrastructure for turning them up and down. repositories - e.g., to change the operating system or number of machines. If
Eventually all of buildbot is planned to be replaced by other infrastructure, you want to do that, reach out to @kf6gpe or another member of the Google team.
but for now flutter has its own client.flutter master.
You would need to edit client.flutter's master in order to add slaves (talk to Each configuration file in that repository has a link in the top comments to a
@eseidelGoogle), add builder groups, or to change the html layout of schema that describes available properties.
https://build.chromium.org/p/client.flutter. Carefully follow the [builders.pyl
docs](https://chromium.googlesource.com/infra/infra/+/master/doc/users/services/buildbot/builders.pyl.md)
to do so.
### Future Directions ### Future Directions
@ -130,11 +124,13 @@ tried, but it's not quite ready.
### Android Tools ### Android Tools
The Android SDK and NDK used by Flutter's Chrome infra bots are stored in Google Cloud. During the build a bot runs the The Android SDK and NDK used by Flutter's Chrome infra bots are stored in Google
`download_android_tools.py` script that downloads the required version of the Android SDK into `dev/bots/android_tools`. Cloud. During the build a bot runs the `download_android_tools.py` script that
downloads the required version of the Android SDK into `dev/bots/android_tools`.
To check which components are currently installed, download the current SDK stored in Google Cloud using the To check which components are currently installed, download the current SDK
`download_android_tools.py` script, then `dev/bots/android_tools/sdk/tools/bin/sdkmanager --list`. If you find that some stored in Google Cloud using the `download_android_tools.py` script, then
`dev/bots/android_tools/sdk/tools/bin/sdkmanager --list`. If you find that some
components need to be updated or installed, follow the steps below: components need to be updated or installed, follow the steps below:
#### How to update Android SDK on Google Cloud Storage #### How to update Android SDK on Google Cloud Storage
@ -145,13 +141,17 @@ components need to be updated or installed, follow the steps below:
2. Use the UI to choose the packages you want to install and/or update. 2. Use the UI to choose the packages you want to install and/or update.
3. Run `dev/bots/android_tools/sdk/tools/bin/sdkmanager --update`. On Windows, run `sdkmanager.bat` instead. If the 3. Run `dev/bots/android_tools/sdk/tools/bin/sdkmanager --update`. On Windows,
process fails with an error saying that it is unable to move files (Windows makes files and directories read-only run `sdkmanager.bat` instead. If the process fails with an error saying that
when another process is holding them open), make a copy of the `dev/bots/android_tools/sdk/tools` directory, run it is unable to move files (Windows makes files and directories read-only
the `sdkmanager.bat` from the copy, and use the `--sdk_root` option pointing at `dev/bots/android_tools/sdk`. when another process is holding them open), make a copy of the
`dev/bots/android_tools/sdk/tools` directory, run the `sdkmanager.bat` from
the copy, and use the `--sdk_root` option pointing at
`dev/bots/android_tools/sdk`.
4. Run `dev/bots/android_tools/sdk/tools/bin/sdkmanager --licenses` and accept the licenses for the newly installed 4. Run `dev/bots/android_tools/sdk/tools/bin/sdkmanager --licenses` and accept
components. It also helps to run this command a second time and make sure that it prints "All SDK package licenses the licenses for the newly installed components. It also helps to run this
command a second time and make sure that it prints "All SDK package licenses
accepted". accepted".
5. Run upload_android_tools.py -t sdk 5. Run upload_android_tools.py -t sdk