1242 Commits

Author SHA1 Message Date
Jonah Williams
5bcb3073f4
[devicelab] remove flaky designation from flutter_driver_android_test (#157300)
https://github.com/flutter/engine/pull/55944 should have fixed all flakes and rolled into framework.
2024-10-21 22:27:52 +00:00
Gray Mackall
3baf9d4c60
Make android dependency version test suite run on changes to templates (#157196)
We have a group of tests that verifies that we can build a flutter app across the range of AGP, Kotlin, and Gradle versions we claim to support. They currently only run on postsubmit. 

This pr aims to make them run in presubmit as well, but only when changes are made to the templates (not just android templates, see https://github.com/flutter/flutter/pull/157196#discussion_r1807055791 as to why).
2024-10-21 15:36:45 +00:00
Flutter GitHub Bot
384f0cc6d3
Marks Mac flavors_test_macos to be flaky (#156944)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac flavors_test_macos"
}
-->
Issue link: https://github.com/flutter/flutter/issues/156943
2024-10-19 01:28:15 +00:00
Zachary Anderson
4ed8136610
Move analyzer_benchmark to the devicelab (#157127)
We want this benchmark to run in the devicelab rather than on a VM so that it will give consistent results across runs.

First run (results at the bottom):
https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8733793239434768065/+/u/run_analyzer_benchmark/stdout
2024-10-17 23:03:10 +00:00
Qun Cheng
23a721f9e9
Mark Linux_android_emu_34 flutter_driver_android_test bringup: true (#157069)
Mark `Linux_android_emu_34 flutter_driver_android_test` bringup: true

Related issue: https://github.com/flutter/flutter/issues/156903
2024-10-16 17:58:01 -07:00
Gray Mackall
d1d9954c45
Support host android apps with kts gradle files for add to app (#156502)
Allows applying of `include_flutter.groovy` via the `apply from:` syntax, which allows using a host app that is using the Gradle Kotlin DSL (the default these days when creating an Android app in AS).

Explanation: The `include_flutter.groovy` script is currently not able to be called by Kotlin gradle files, because it is [intended to be invoked with the following lines](https://docs.flutter.dev/add-to-app/android/project-setup#depend-on-the-modules-source-code):
```
setBinding(new Binding([gradle: this]))                                // new
evaluate(new File(                                                     // new
    settingsDir.parentFile,                                            // new
    'flutter_module/.android/include_flutter.groovy'                   // new
))
```

`setBinding` isn't part of the Kotlin gradle DSL, and there isn't (that I can find) an easy Kotlin equivalent. If this binding isn't set, the reference to `gradle` in `include_flutter.groovy` is wrong, which breaks the script.

This PR modifies `include_flutter.groovy` to also support being invoked through the standard way of invoking a script via the Gradle Groovy/Kotlin DSLs, which is `apply from:` (or it's slightly different Kotlin syntax). The start of the script identifies which of the two approaches is being used by checking if the binding is set, and then initializes some variables differently depending on the case.

If we land this, I believe we should update the example Gradle files for both the `kts` and `groovy` cases to prefer the `apply from` syntax as I think this is the syntax most developers would be more familiar with already seeing in their Gradle files.
2024-10-16 22:47:27 +00:00
Andrew Kolos
dacfa2d452
increase sharding of Windows build_tests (#156763)
Fixes https://github.com/flutter/flutter/issues/156762

<details>

<summary> Pre-launch checklist </summary> 

</details>
2024-10-14 22:52:24 +00:00
Jonah Williams
56e2a9d380
[devicelab] remove raster cache benchmarks. (#156600)
Death to the raster cache!
2024-10-11 19:46:21 +00:00
Andrew Kolos
514cf62d03
remove bringup from Windows tool_integration_tests_* (#156179)
Follow up to
* https://github.com/flutter/flutter/pull/156121

Removes bringup status from these tasks.
2024-10-04 19:49:14 +00:00
Andrew Kolos
58e0e00c63
further shard the Windows tool_integration_tests* targets (#156121)
Helps with https://github.com/flutter/flutter/issues/155012

Will remove `bringup` statuses after this merges.
2024-10-03 17:13:28 +00:00
Flutter GitHub Bot
4342887826
Marks Windows_mokey native_assets_android to be flaky (#156064)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Windows_mokey native_assets_android"
}
-->
Issue link: https://github.com/flutter/flutter/issues/156063
2024-10-02 13:35:26 +00:00
Christopher Fujino
3dc1e81526
mark {Linux,Windows} tool_integration_tests_* non-bringup (#155773)
Follow-up to https://github.com/flutter/flutter/pull/155631

These have not failed except one flake from gradle exhausting the java heap.
2024-10-02 02:17:06 +00:00
Gray Mackall
dec2091c03
Turn the packages roller bot back on (#155842)
Turns the roller bot back on, as it should have been fixed by https://github.com/flutter/flutter/pull/155794/.
2024-09-27 19:43:37 +00:00
Christopher Fujino
eb42a01302
mark linux packages autoroller bringup: true (#155705)
Unblock the tree while I investigate
https://github.com/flutter/flutter/issues/155704
2024-09-25 12:38:19 -07:00
Flutter GitHub Bot
538e7424d0
Marks Linux build_aar_module_test to be unflaky (#155349)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux build_aar_module_test"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20build_aar_module_test%22).
This test can be marked as unflaky.
2024-09-25 18:44:07 +00:00
Christopher Fujino
3d14246cee
increase both linux & windows tool integration test shards (#155631)
Fixes both:

- https://github.com/flutter/flutter/issues/155621
- https://github.com/flutter/flutter/issues/155317
2024-09-25 17:15:14 +00:00
Chris Bracken
0cd170798c
iOS: update provisioning profile for 2024-2025 cert (#155101)
Tells our iOS bots to use the provisioning profile stored in the flutter_internal/mac/mobileprovision CIPD packages tagged `version:to_2025`.

This CIPD package contains an updated provisioning profile which supports both the current development signing certificate expiring in October 2024, and the updated signing cert expiring in August 2025.

CIPD packages can be seen at:
https://chrome-infra-packages.appspot.com/p/flutter_internal/mac/mobileprovision

This is a reland of #155052, except without the removal of the CIPD dependency from the devicelab bots. After landing the original, many devicelab bots (and only devicelab bots) started failing with signing errors. The devicelab bots *should* be getting their certs/provisioning profile via the devicelab Salt configuration server (Googlers, see: go/flutter-salt) but it's possible some tests are fetching it from the CIPD package instead. This new patch simply updates the CIPD package version wherever it was already present. We can attempt to remove the CIPD package dependency from devicelab bots in a followup PR, and switch those over to using the profile installed via Salt (these are the same profile).

Required for: https://g-issues.chromium.org/issues/366034566
Issue: https://github.com/flutter/flutter/issues/152888
2024-09-12 18:15:58 +00:00
auto-submit[bot]
50a190caec
Reverts "iOS: update provisioning profile for 2024-2025 cert (#155052)" (#155059)
Reverts: flutter/flutter#155052
Initiated by: zanderso
Reason for reverting: The tree is red on this change: https://logs.chromium.org/logs/flutter/buildbucket/cr-buildbucket/8737044980814837569/+/u/run_flutter_gallery_ios__compile/stdout.
Original PR Author: cbracken

Reviewed By: {christopherfujino, bdero}

This change reverts the following previous change:
Tells our iOS bots to use the provisioning profile stored in the flutter_internal/mac/mobileprovision CIPD packages tagged `version:to_2025`.

This CIPD package contains an updated provisioning profile which supports both the current development signing certificate expiring in October 2024, and the updated signing cert expiring in August 2025.

Also removes the CIPD dependency for devicelab tasks. On devicelab bots, the cert and provisioning profile are deployed via Salt, not via CIPD. We only need this dependency on the Chromium bots.

CIPD packages can be seen at:
https://chrome-infra-packages.appspot.com/p/flutter_internal/mac/mobileprovision

Required for: https://g-issues.chromium.org/issues/366034566

Issue: https://github.com/flutter/flutter/issues/152888
2024-09-12 01:47:46 +00:00
Chris Bracken
8bb8a03fa9
iOS: update provisioning profile for 2024-2025 cert (#155052)
Tells our iOS bots to use the provisioning profile stored in the flutter_internal/mac/mobileprovision CIPD packages tagged `version:to_2025`.

This CIPD package contains an updated provisioning profile which supports both the current development signing certificate expiring in October 2024, and the updated signing cert expiring in August 2025.

Also removes the CIPD dependency for devicelab tasks. On devicelab bots, the cert and provisioning profile are deployed via Salt, not via CIPD. We only need this dependency on the Chromium bots.

CIPD packages can be seen at:
https://chrome-infra-packages.appspot.com/p/flutter_internal/mac/mobileprovision

Required for: https://g-issues.chromium.org/issues/366034566

Issue: https://github.com/flutter/flutter/issues/152888
2024-09-12 00:26:17 +00:00
Flutter GitHub Bot
d4bfa2fd14
Marks Linux build_tests_3_5 to be unflaky (#154993)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux build_tests_3_5"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20build_tests_3_5%22).
This test can be marked as unflaky.
2024-09-11 17:59:39 +00:00
Flutter GitHub Bot
6e46ee8d6d
Marks Linux build_tests_5_5 to be unflaky (#154995)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux build_tests_5_5"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20build_tests_5_5%22).
This test can be marked as unflaky.
2024-09-11 17:51:06 +00:00
Flutter GitHub Bot
d8de38f10d
Marks Linux build_tests_1_5 to be unflaky (#154991)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux build_tests_1_5"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20build_tests_1_5%22).
This test can be marked as unflaky.
2024-09-11 16:52:29 +00:00
Flutter GitHub Bot
e662a9bafe
Marks Linux build_tests_2_5 to be unflaky (#154992)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux build_tests_2_5"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20build_tests_2_5%22).
This test can be marked as unflaky.
2024-09-11 16:52:27 +00:00
Gray Mackall
fadbf44471
Fix java version used by build_aar_module_test (#154967)
Used the wrong syntax, whoops

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md

Co-authored-by: Gray Mackall <mackall@google.com>
2024-09-11 10:17:25 -04:00
Gray Mackall
6fd2b9f88b
Remove scheduler: luci from new build_aar_module_test (#154945)
This option was deprecated (/removed?) in the time since the test was originally disabled.

See https://github.com/flutter/flutter/issues/109712, https://github.com/flutter/flutter/pull/109714
2024-09-10 20:35:07 +00:00
Gray Mackall
f964f15dcb
Fix flutter build aar for modules that use a plugin (#154757)
https://github.com/flutter/flutter/pull/151675 bumped module templates to AGP 8.1.

In doing so, I tried to work around a behavior change [that was new in AGP 8.0](https://developer.android.com/build/releases/past-releases/agp-8-0-0-release-notes):
> AGP 8.0 creates no SoftwareComponent by default. Instead AGP creates SoftwareComponents only for variants that are configured to be published using the publishing DSL.

by using AGP's publishing DSL to define which variants to publish in the module's ephemeral gradle files:
```
android.buildTypes.all {buildType ->
    if (!android.productFlavors.isEmpty()) {
        android.productFlavors.all{productFlavor ->
            android.publishing.singleVariant(productFlavor.name + buildType.name.capitalize()) {
                withSourcesJar()
                withJavadocJar()
            }
        }
    } else {
        android.publishing.singleVariant(buildType.name) {
            withSourcesJar()
            withJavadocJar()
        }
    }
}
```

The problem is that this doesn't get applied to the plugin projects used by the module, so if a module uses any plugin it breaks. This PR fixes that by applying similar logic, but to each project (not just the module's project).

Tested manually with https://github.com/gmackall/GrayAddToApp, and also re-enabled an old test that tested this use case as a part of the PR.

Fixes: https://github.com/flutter/flutter/issues/154371
2024-09-10 16:42:22 +00:00
Christopher Fujino
92697d3ecc
Improve tracing and fix packages_autoroller (#154841)
Reverts flutter/flutter#154555
Re-lands of https://github.com/flutter/flutter/pull/154441 and https://github.com/flutter/flutter/pull/154369

New changes in this PR:

1. extend timeout of the `Linux packages_autoroller` shard (see step 14 in this example [LED build](https://ci.chromium.org/ui/p/flutter/builders/prod.shadow/Linux%20packages_autoroller/7/infra))
2. use the managed framework repo as the working directory in the `framework.streamDart()` helper function--we were generating gradle lockfiles in the wrong directory

A LED build with this code generated the following gradle lockfile update: d939981cd2
2024-09-09 23:31:21 +00:00
Kate Lovett
132dc8f82d
Revert #154339, remove bringup on linux customer testing (#154372)
Reverts increased timeouts on customer testing from #154339, also removes bringup state from https://github.com/flutter/flutter/issues/154293.
This was related to https://github.com/flutter/flutter/issues/154251, which will be fixed by https://github.com/flutter/tests/pull/406 (so we should land that first).
2024-09-03 19:36:56 +00:00
Christopher Fujino
28a9124304
Add Android SDK to Linux packages_autoroller (#154462)
Fixes https://github.com/flutter/flutter/issues/154151

This target now requires the Android toolchain because it may
re-generate gradle lockfiles.
2024-09-03 11:23:49 -07:00
Yegor
9502cb1576
increase Linux build_test shard count from 3 to 5 (#154444)
Fixes https://github.com/flutter/flutter/issues/154443
2024-08-30 15:05:55 -07:00
Zachary Anderson
d01b9a4f8c
Increase timeout for customer_testing test step (#154339)
Related: https://github.com/flutter/flutter/issues/154251

Successful runs are bumping right up against the timeout

See https://ci.chromium.org/ui/p/flutter/builders/try/Linux%20customer_testing/84496/overview which succeeded in 29 minutes vs. https://ci.chromium.org/ui/p/flutter/builders/try/Linux%20customer_testing/84497/overview which failed with a timeout at 30 minutes.
2024-08-29 15:34:25 +00:00
Matan Lurey
32e29fc39b
Temporarily mark Linux customer_testing as bringup: true. (#154297)
See https://github.com/flutter/flutter/issues/154293.
2024-08-28 16:09:05 -07:00
Flutter GitHub Bot
4f37ef14ef
Marks Mac_mokey run_release_test to be flaky (#153831)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac_mokey run_release_test"
}
-->
Issue link: https://github.com/flutter/flutter/issues/153830
2024-08-21 19:37:11 +00:00
Zachary Anderson
88123277fc
Remove android stack_size_test (#153695)
This test expects to run on 32-bit hardware, but the devicelab no longer
has 32-bit Android hardware. In particular, the test contains hand-coded
32-bit arm assembly for reading the stack pointer from Dart code via
`dart:ffi`.

This test was added in response to a framework change that caused stack
frames to be larger than expected while building widgets, leading apps
to crash with StackOverflow exceptions unexpectedly. Since then (>3
years ago) this test has not prevented any similar issue, so I believe
deleting it rather than fixing it is a better use of resources.
2024-08-19 12:25:35 -07:00
Kate Lovett
6a28048935
Update goldctl git revision (#153710)
This updates the version of goldctl we checkout for testing.
The recent update makes goldctl more resilient to I/O timeouts.
2024-08-19 18:09:25 +00:00
Zachary Anderson
69ff9eae7b
Move Android API level 35 emulator tests to staging (#153568)
These same tests but running against API level 34 emulators are remaining as presubmit tests. The API level 35 tests should stay in staging while the emulator stabilizes. So we can collect flake data and continue to debug. 

Related: https://github.com/flutter/flutter/issues/153445
2024-08-16 15:43:13 +00:00
Matan Lurey
af815f0d9b
Roll avd_cipd_verison to latest to use the crashreport tool. (#153520)
Work towards https://github.com/flutter/flutter/issues/153445.

I can do a similar set of PRs on flutter/packages and flutter/engine for consistency.

/cc @johnmccutchan @reidbaker @zanderso
2024-08-15 21:58:31 +00:00
Zachary Anderson
cc13cd1f11
Retry Linux_android_emu tests in presubmit like they are in postsubmit (#153369)
To help keep rolls unblocked while the flakiness of these tests is
investigated.
2024-08-13 10:23:30 -07:00
Zachary Anderson
6200318586
Shift API level 34 emulator tests to prod (#153366)
These are passing consistently
2024-08-13 09:42:30 -07:00
Zachary Anderson
b12d86157e
Move windows mokey tests to prod (#153203)
For https://github.com/flutter/flutter/issues/148085
2024-08-10 04:08:08 +00:00
Zachary Anderson
f43ae3b1ff
Also run Linux_android_emu tests on API level 34 image (#153175)
These VMs are relatively inexpensive, so there's no reason not to
continue running on the API level 34 image while we work out how to deal
with the apparent instability in the 35 images.

Related: https://github.com/flutter/flutter/issues/152684
2024-08-09 15:23:12 -07:00
Zachary Anderson
857addcb31
Shift tests on moto g4 on Windows hosts to mokey in staging (#153167)
For https://github.com/flutter/flutter/issues/148085
2024-08-09 09:23:56 -07:00
Zachary Anderson
8b4b6ae91e
Shift Linux_android_emu tests from staging to prod (#153110)
These tests were moved to staging when they were flaking at a high rate.
I landed a couple of changes to fix unrelated issues that were masking
the underlying cause. However, these tests are no longer flaking at the
same rate in staging as they were in prod. So, I am moving these back to
prod and will monitor the flake rate.

Related https://github.com/flutter/flutter/issues/152684
2024-08-08 13:08:17 -07:00
Zachary Anderson
5eba89e5ed
Move Android tests with macOS host from staging to prod (#153113)
Part of https://github.com/flutter/flutter/issues/148085
2024-08-08 20:03:14 +00:00
Jenn Magder
ede2e27054
Revert "Marks Mac channels_integration_test to be flaky" (#153044)
Reverts flutter/flutter#151882

Flake rate is back under the threshold https://github.com/flutter/flutter/issues/151881#issuecomment-2272996113

Closes https://github.com/flutter/flutter/issues/151881
2024-08-07 19:49:51 +00:00
Flutter GitHub Bot
c4c84caf6e
Marks Linux android_java17_dependency_smoke_tests to be unflaky (#153011)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux android_java17_dependency_smoke_tests"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20android_java17_dependency_smoke_tests%22).
This test can be marked as unflaky.
2024-08-07 18:53:57 +00:00
Flutter GitHub Bot
6945ec0a8c
Marks Linux android_java11_dependency_smoke_tests to be unflaky (#153010)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Linux android_java11_dependency_smoke_tests"
}
-->
The test has been passing for [50 consecutive runs](https://data.corp.google.com/sites/flutter_infra_metrics_datasite/flutter_check_test_flakiness_status_dashboard/?p=BUILDER_NAME:%22Linux%20android_java11_dependency_smoke_tests%22).
This test can be marked as unflaky.
2024-08-07 18:50:06 +00:00
Flutter GitHub Bot
521b0b327c
Marks Mac integration_ui_test_test_macos to be flaky (#152213)
<!-- meta-tags: To be used by the automation script only, DO NOT MODIFY.
{
  "name": "Mac integration_ui_test_test_macos"
}
-->
Issue link: https://github.com/flutter/flutter/issues/152212
2024-08-06 21:24:53 +00:00
Zachary Anderson
a88e321c5b
Move Linux_build_test tests from staging to prod (#152877)
These have been migrated to mokey and have been consistently passing in staging.
2024-08-05 16:58:47 +00:00
Zachary Anderson
ae66107a5c
Mark Linux_android_emu tests bringup: true (#152867)
Due to https://github.com/flutter/flutter/issues/152684

This PR is just eagerly doing what the flake bot would do on Wednesday.

FYI @matanlurey @reidbaker
2024-08-05 09:28:00 -07:00