697 Commits

Author SHA1 Message Date
Yegor
1f8501f0a4
web benchmark choose renderer explicitly (#73582) 2021-01-08 12:04:03 -08:00
jensjoha
5bc725bea8
[dev] Don't use await for on stdout and stdin; pass local engine argument (#72136)
* [dev] Don't use await for on stdout and stdin; pass local engine argument

1) Don't use await for on stdout followd by await for on stderr.
   This can cause nothing to happen. E.g. if one didn't do something
   like `flutter pub upgrade` dev/automated_tests and the version in the
   old one was pre-nnbd (but flutter itself use lots of nnbd stuff) lots
   of errors would be emitted, but because we basically only listen to
   stdout nothing will happen (deadlock once stderr buffer runs out).
   Note that it still awaits the exit-code below.
2) Pass --local-engine (if given) so there won't be any crashes because
   of that.
2021-01-08 08:12:36 +01:00
Jenn Magder
3082a2806e
build ios-framework simulator slices for profile/release (#73378) 2021-01-06 12:12:23 -08:00
nturgut
272b23cabd
[web] make sure tests and benchmarks use correct backend - add autodetect false (#73047)
* add autodetect false

* fix typo
2020-12-28 10:55:53 -08:00
Casey Hillers
849784e262
[devicelab] Add results path flag to test runner (#72765) 2020-12-22 16:45:38 -08:00
nturgut
14206399c3
make sure tests and becnhmarks use correct backend (#72628) 2020-12-21 09:55:23 -08:00
xster
ca82c572b7
Revert "let NOTICES be double gzip wrapped to reduce on-disk installed space (#71899)" (#72438) 2020-12-16 10:58:19 -08:00
xster
4ed3432e8f
let NOTICES be double gzip wrapped to reduce on-disk installed space (#71899) 2020-12-15 13:29:14 -08:00
Michael Goderbauer
78cbfbff42
Revert "Migrate devicelab to package:vm_service (#71882)" (#72092)
This reverts commit 9d1d0cfeb50a123712c5425778ed704d119435b6.
2020-12-10 12:56:02 -08:00
Dan Field
9d1d0cfeb5
Migrate devicelab to package:vm_service (#71882) 2020-12-10 12:43:04 -08:00
Casey Hillers
2565062df3
git branch flag (#71886) 2020-12-08 15:53:05 -08:00
Emmanuel Garcia
d35dd9f8a6
Allow flavors and build types when using plugins (#71738) 2020-12-07 09:23:04 -08:00
Jenn Magder
d4150d3f6d
Adopt Flutter.xcframework in tool (#71495) 2020-12-04 13:57:11 -08:00
Yegor
7efd7afdf8
Handle the glibc bug that Chrome occasionally hits (#71705) 2020-12-04 12:21:37 -08:00
Kate Lovett
1c244da28d
Add verbose flag to hot mode tests (#71610) 2020-12-02 16:47:35 -06:00
Jonah Williams
1e0628001e
[devicelab] dont use set modified for hot reload bench (#70965) 2020-11-20 13:02:28 -08:00
Jonah Williams
9424474793
[devicelab] update hot reload benchmark to be more representative (#70643)
update hot reload benchmark to be more representative
2020-11-18 14:54:01 -08:00
Jim Graham
61460d255a
preserve detailFiles entry in TaskResult JSON (#70668)
* preserve detailFiles entry in TaskResult JSON
2020-11-17 16:05:54 -08:00
Casey Hillers
a228a17c57
[devicelab] LUCI builder flag (#70702) 2020-11-17 11:28:03 -08:00
Yuqian Li
f7811ea541
e2e perf tests don't have CPU/GPU/memory metrics (#70439)
Address comment in https://github.com/flutter/flutter/pull/66386/files#r522671103
2020-11-13 11:16:50 -08:00
Emmanuel Garcia
b18e2a1cc4
Bump Gallery version (#70130) 2020-11-09 13:05:56 -08:00
Jonah Williams
c985fd35cd
[devicelab] re-land cocoon auto-restart (#69710)
Due to #69707 the SkSL tests need the process killing logic still.
2020-11-03 13:20:40 -08:00
Jonah Williams
ea3551b058
Revert "[devicelab] reboot attached devices after 30 test runs (#69447)" (#69703)
This reverts commit 406ca0bc59dc4dcaaa618cd50d889c2d7095456c.
2020-11-03 11:13:19 -08:00
Yegor
9e5d22d101
Say [APP] instead of [Gallery] in web benchmarks (#69624) 2020-11-03 10:13:07 -08:00
Jonah Williams
406ca0bc59
[devicelab] reboot attached devices after 30 test runs (#69447) 2020-11-03 10:10:55 -08:00
Greg Spencer
c0ef94780c
Fix spelling errors (#69451)
My yearly pass to fix spelling errors in the API dartdocs.
2020-10-31 03:12:27 -07:00
Jenn Magder
bb495ba90c
Do not fail if average_memory_usage is not recorded (#69339) 2020-10-29 16:09:50 -07:00
Jonah Williams
31c96b5d83
[devicelab] do not fail test if CPU measurement is not recorded (#69268) 2020-10-29 09:32:34 -07:00
Jonah Williams
18341efa2f
[devicelab] retry startup of start up test (#69272) 2020-10-29 09:19:07 -07:00
Jonah Williams
83d4351e0c
[devicelab] work around occasional null values for devtools (#69273)
* [devicelab] work around occasional null values for devtools

* Update perf_tests.dart
2020-10-29 09:18:38 -07:00
Jenn Magder
13896b3bd1
Decrease device discovery timeout from 10 to 5 seconds (#69266) 2020-10-28 19:43:19 -07:00
Jenn Magder
135aab9cf5
Increase device run perf test timeout (#69260) 2020-10-28 18:29:04 -07:00
Jonah Williams
2c12a8d917
[devicelab] remove twc enabled test and uncaught image test (#68518) 2020-10-24 13:19:51 -07:00
Casey Hillers
ce3ce20b14
[devicelab] Upload git branch (#68541) 2020-10-23 08:02:02 -07:00
Jonah Williams
37042fdb08
[flutter_tools] remove iOS screenshot on failure functionality (#68650)
* [flutter_tools] remove iOS screenshot on failure functionality

* remove all screenshot code
2020-10-22 13:34:07 -07:00
Jonah Williams
ab20ae782a
[devicelab] uinstall during memory test (#68548) 2020-10-20 09:49:50 -07:00
Jonah Williams
938fae9c99
remove --machine flag (#68538) 2020-10-19 17:06:28 -07:00
Jonah Williams
a152c0f429
[devicelab] reland: reduce iterations of startup test, use application binary (#68532)
These devicelab tests are incredibly flaky. Currently they will build & install the same application 15 times. This causes the temp storage to fill up on android, and has a good chance of flaking on iOS due to install issues.

The change from 3 to 15 increased total test time for 2 -> 12 minutes on iOS, or roughly 5 more test equivalents for every single iOS test. Reduce the iteration count back to 5

Uninstall the app after each run so temp storage does not fill up.
2020-10-19 16:51:28 -07:00
Jonah Williams
4a3e2a1af5
Revert "[devicelab] reduce iterations, uninstall at end, and use --application-binary in all startup tests (#68516)" (#68531)
This reverts commit a4c2075a45f956baa018016e3593d6af84290c6d.
2020-10-19 15:05:46 -07:00
Jonah Williams
8262f808ba
Revert "[devicelab] fix flutter gallery app name (#68528)" (#68530)
This reverts commit 0f69f8e53b397a963df73627b396caa235761b60.
2020-10-19 15:05:10 -07:00
Jonah Williams
0f69f8e53b
[devicelab] fix flutter gallery app name (#68528) 2020-10-19 14:48:05 -07:00
Jonah Williams
a4c2075a45
[devicelab] reduce iterations, uninstall at end, and use --application-binary in all startup tests (#68516)
These devicelab tests are incredibly flaky. Currently they will build & install the same application 15 times. This causes the temp storage to fill up on android, and has a good chance of flaking on iOS due to install issues.

The change from 3 to 15 increased total test time for 2 -> 12 minutes on iOS, or roughly 5 more test equivalents for every single iOS test. Reduce the iteration count back to 5

Uninstall the app after each run so temp storage does not fill up.
2020-10-19 14:16:04 -07:00
Casey Hillers
1c35091a13
[devicelab] Cocoon client (#68333) 2020-10-19 13:12:04 -07:00
Jonah Williams
e8dc7a2e17
[devicelab] update hostonly tests to use flutter directly or precache deps (#68487)
Fixes #67370

By running most of these executions through flutter, we get the benefit of the flutter error handling and precaching. IN the test where this is not feasible, call pre-cache directly.
2020-10-19 13:01:50 -07:00
Jonah Williams
03db9bf918
[devicelab] migrate defines test to DriverTest (#68407)
Removes one-off handling for defines test and forces it to run through the DriverTest interface
2020-10-19 12:28:43 -07:00
Jonah Williams
223df0c351
[devicelab] split end to end tests into driver targets (#68410)
Split the integration_ui target into individual driver tests. Remove disabled screenshot tests
2020-10-19 12:28:13 -07:00
Jonah Williams
e422d5c724
[flutter_tools] reland: --no-android-gradle-daemon in devicelab (#68491)
Remove devicelab specific code for shutting down gradle daemon, add --android-gradle-daemon option to build/run/drive`. Avoids need for un-tested devicelab specific handler. There are also some feature requests for this, so 2 birds one stone.

Example:

flutter build apk --no-android-gradle-daemon will pass --no-daemon on to gradle
2020-10-19 10:27:58 -07:00
Jonah Williams
ed5482de19
Revert "[flutter_tools] add --android-gradle-daemon option, use in devicelab (#68409)" (#68489)
This reverts commit ddab72e79f72b7896d8f62b2062d46c764c224f6.
2020-10-19 09:33:52 -07:00
Jonah Williams
ddab72e79f
[flutter_tools] add --android-gradle-daemon option, use in devicelab (#68409)
Remove devicelab specific code for shutting down gradle daemon, add --android-gradle-daemon option to build/run/drive`. Avoids need for un-tested devicelab specific handler. There are also some feature requests for this, so 2 birds one stone.

Example:

flutter build apk --no-android-gradle-daemon will pass --no-daemon on to gradle
2020-10-19 09:12:21 -07:00
Alexandre Ardhuin
c0176c9ee5
use_is_even_rather_than_modulo (#68301) 2020-10-16 20:20:00 +02:00