firka/flutter
firka/flutter
5
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

update readme for LUCI (#29023)

Browse Source 
* update readme for LUCI

* Update README.md
This commit is contained in:
Dan Field 2019-03-08 09:25:13 -08:00 committed by GitHub
parent da7649586a
commit 4be4830c2f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 57 additions and 57 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

114
dev/bots/README.md
View File

@ -5,41 +5,37 @@ This directory exists to support building Flutter on our build infrastructure.
The results of such builds are viewable at:
* https://cirrus-ci.com/github/flutter/flutter/master
- Testing done on PRs and submitted changes on GitHub.
* https://build.chromium.org/p/client.flutter/console
* https://ci.chromium.org/p/flutter/
- Additional testing and processing done after changes are submitted.
The Chromium infra bots do not allow forcing new builds from outside
the Google network. Contact @eseidelGoogle or another Google member of
the Flutter team if you need to do that.
The LUCI infra requires permissions to retrigger or schedule builds. Contact
@kf6gpe or another Google member of the Flutter team if you need to do that.
The [Cirrus](https://cirrus-ci.org)-based bots run the
[`test.dart`](test.dart) script for each PR and submission. This does
testing for the tools, for the framework, and (for submitted changes
only) rebuilds and updates the master branch API docs
[staging site](https://master-docs-flutter-io.firebaseapp.com).
For tagged dev and beta builds, it also builds and deploys the gallery
app to the app stores. It is configured by the
[.cirrus.yml](/.cirrus.yml).
The [Cirrus](https://cirrus-ci.org)-based bots run the [`test.dart`](test.dart)
script for each PR and submission. This does testing for the tools, for the
framework, and (for submitted changes only) rebuilds and updates the master
branch API docs [staging site](https://master-docs.flutter.dev/).
For tagged dev and beta builds, it also builds and deploys the gallery app to
the app stores. It is configured by the [.cirrus.yml](/.cirrus.yml).
We also have post-commit testing with actual devices, in what we call
our [devicelab](../dev/devicelab/README.md).
We also have post-commit testing with actual devices, in what we call our
[devicelab](../dev/devicelab/README.md).
## Chromium infra bots
## LUCI (Layered Universal Continuous Intergration)
This part of our infrastructure is broken into two parts. A buildbot
master specified by our
[builders.pyl](https://chromium.googlesource.com/chromium/tools/build.git/+/master/masters/master.client.flutter/builders.pyl)
file, and a [set of
recipes](https://chromium.googlesource.com/chromium/tools/build.git/+/master/scripts/slave/recipes/flutter)
which we run on that master. Both of these technologies are highly
specific to Google's Chromium project. We're just borrowing some of
their infrastructure.
A [set of recipes](https://chromium.googlesource.com/chromium/tools/build.git/+/master/scripts/slave/recipes/flutter)
are run on Windows, Linux, and Mac machines. The configuration for how many
machines and what kind are managed internally by Google. Contact @kf6gpe or
another Google member of the Flutter team if you suspect changes are needed
there. Both of these technologies are highly specific to the [LUCI](https://github.com/luci)
project, which is the successor to Chromium's infra. We're just borrowing some
of their infrastructure.
### Prerequisites
To work on this infrastructure you will need:
- [depot_tools](http://www.chromium.org/developers/how-tos/install-depot-tools)
- [depot_tools](https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.html#_setting_up)
- Python package installer: `sudo apt-get install python-pip`
- Python coverage package (only needed for `training_simulation`): `sudo pip install coverage`
@ -58,7 +54,8 @@ To run prepare_package.dart locally:
### Getting the code
The following will get way more than just recipe code, but it _will_ get the recipe code:
The following will get way more than just recipe code, but it _will_ get the
recipe code:
```bash
mkdir chrome_infra
@ -82,7 +79,7 @@ and
- build/scripts/slave/recipes/flutter/flutter.py
- build/scripts/slave/recipes/flutter/engine.py
Recipes are just Python. They are
Recipes are just Python with some limitations on what can be imported. They are
[documented](https://github.com/luci/recipes-py/blob/master/doc/user_guide.md)
by the [luci/recipes-py github project](https://github.com/luci/recipes-py).
@ -90,33 +87,30 @@ The typical cycle for editing a recipe is:
1. Make your edits (probably to files in
`//chrome_infra/build/scripts/slave/recipes/flutter`).
1. Update the tests. Run `build/scripts/slave/recipes.py test train` to update
2. Update the tests. Run `build/scripts/slave/recipes.py test train` to update
existing expected output to match the new output. Verify completely new test
cases by altering the `GenTests` method of the recipe.
The recipe is required to have 100% test coverage.
1. Run `build/scripts/slave/recipes.py run flutter/<repo> slavename=<slavename>
mastername=client.flutter buildername=<buildername> buildnumber=1234` where `<repo>` is one
of `flutter` or `engine`, and `slavename` and `buildername` can be looked up
from the *Build Properties* section of a [recent
cases by altering the `GenTests` method of the recipe. The recipe is required
to have 100% test coverage.
3. Run `build/scripts/slave/recipes.py run flutter/<repo> slavename=<slavename>
mastername=client.flutter buildername=<buildername>
buildnumber=1234` where `<repo>` is one of `flutter` or `engine`, and
`slavename` and `buildername` can be looked up from the *Build Properties*
section of a [recent
build](https://build.chromium.org/p/client.flutter/one_line_per_build).
1. To submit a CL, you need a local branch first (`git checkout -b [some branch name]`).
1. Upload the patch (`git commit`, `git cl upload`) and send it to someone in
4. To submit a CL, you need a local branch first (`git checkout -b [some branch name]`).
5. Upload the patch (`git commit`, `git cl upload`) and send it to someone in
the `recipes/flutter/OWNERS` file for review.
### Editing the client.flutter buildbot master
### The infra config repository
Flutter uses Chromium's fancy
[builders.pyl](https://chromium.googlesource.com/infra/infra/+/master/doc/users/services/buildbot/builders.pyl.md)
master generation system. Chromium hosts 100s (if not 1000s) of buildbot
masters and thus has lots of infrastructure for turning them up and down.
Eventually all of buildbot is planned to be replaced by other infrastructure,
but for now flutter has its own client.flutter master.
The [flutter/infra](https://github.com/flutter/infra) repository contains
configuration files for the dashboard, builder groups, scheduling, and
individual builders. Edits to this may require changes other internal Google
repositories - e.g., to change the operating system or number of machines. If
you want to do that, reach out to @kf6gpe or another member of the Google team.
You would need to edit client.flutter's master in order to add slaves (talk to
@eseidelGoogle), add builder groups, or to change the html layout of
https://build.chromium.org/p/client.flutter. Carefully follow the [builders.pyl
docs](https://chromium.googlesource.com/infra/infra/+/master/doc/users/services/buildbot/builders.pyl.md)
to do so.
Each configuration file in that repository has a link in the top comments to a
schema that describes available properties.
### Future Directions
@ -130,11 +124,13 @@ tried, but it's not quite ready.
### Android Tools
The Android SDK and NDK used by Flutter's Chrome infra bots are stored in Google Cloud. During the build a bot runs the
`download_android_tools.py` script that downloads the required version of the Android SDK into `dev/bots/android_tools`.
The Android SDK and NDK used by Flutter's Chrome infra bots are stored in Google
Cloud. During the build a bot runs the `download_android_tools.py` script that
downloads the required version of the Android SDK into `dev/bots/android_tools`.
To check which components are currently installed, download the current SDK stored in Google Cloud using the
`download_android_tools.py` script, then `dev/bots/android_tools/sdk/tools/bin/sdkmanager --list`. If you find that some
To check which components are currently installed, download the current SDK
stored in Google Cloud using the `download_android_tools.py` script, then
`dev/bots/android_tools/sdk/tools/bin/sdkmanager --list`. If you find that some
components need to be updated or installed, follow the steps below:
#### How to update Android SDK on Google Cloud Storage
@ -145,13 +141,17 @@ components need to be updated or installed, follow the steps below:
2. Use the UI to choose the packages you want to install and/or update.
3. Run `dev/bots/android_tools/sdk/tools/bin/sdkmanager --update`. On Windows, run `sdkmanager.bat` instead. If the
process fails with an error saying that it is unable to move files (Windows makes files and directories read-only
when another process is holding them open), make a copy of the `dev/bots/android_tools/sdk/tools` directory, run
the `sdkmanager.bat` from the copy, and use the `--sdk_root` option pointing at `dev/bots/android_tools/sdk`.
3. Run `dev/bots/android_tools/sdk/tools/bin/sdkmanager --update`. On Windows,
run `sdkmanager.bat` instead. If the process fails with an error saying that
it is unable to move files (Windows makes files and directories read-only
when another process is holding them open), make a copy of the
`dev/bots/android_tools/sdk/tools` directory, run the `sdkmanager.bat` from
the copy, and use the `--sdk_root` option pointing at
`dev/bots/android_tools/sdk`.
4. Run `dev/bots/android_tools/sdk/tools/bin/sdkmanager --licenses` and accept the licenses for the newly installed
components. It also helps to run this command a second time and make sure that it prints "All SDK package licenses
4. Run `dev/bots/android_tools/sdk/tools/bin/sdkmanager --licenses` and accept
the licenses for the newly installed components. It also helps to run this
command a second time and make sure that it prints "All SDK package licenses
accepted".
5. Run upload_android_tools.py -t sdk