139 Commits

Author SHA1 Message Date
Danny Tuppeny
55d70506ed
Mark module_test_ios as not-flaky (#22626)
The only reds on the dashboard for this test are on rows that have many failures across the run; this does not appear to be at all flaky.
2018-10-03 21:03:25 +01:00
Mikkel Nygaard Ravn
a600fe7f13 Support materializing Flutter module host app on iOS (#21276)
* Prototype

* Fix paths to Flutter library resources

* Invoke pod install as necessary for materialized modules

* Add devicelab test for module use on iOS

* Remove debug output

* Rebase, reame materialize editable

* Add devicelab test editable iOS host app

* Removed add2app test section
2018-09-25 15:21:13 -04:00
Ian Hickson
9f359aeee2
Remove all "flaky" annotations from devicelab (#21895)
These tests don't seem particularly more flaky than ony others any more.
2018-09-17 13:02:34 -07:00
Jonah Williams
c6e8a513f0
Add android semantics integration test to device lab (#20971) 2018-09-06 12:31:18 -07:00
amirh
146b906bbb
Mark the embedded_android_views_test as non flaky. (#20962)
Was marked as flaky as that's our policy for assing new device lab
tests. It's been consistently green so marking as non flaky.
2018-08-23 19:01:28 -07:00
amirh
1f31c3b354
Run the embedded Android views integration test on the device lab. (#20841) 2018-08-23 10:26:00 -07:00
Jonah Williams
f0e80933b0
Mark memory_nav as flaky (#20930) 2018-08-22 17:34:29 -07:00
Jonah Williams
452f255e95
undo changes to devicelab capabilities (#20536) 2018-08-13 13:26:32 -07:00
Jonah Williams
0892ace91e
agent capability doesnt work as expected (#20529) 2018-08-13 10:19:47 -07:00
Jonah Williams
cf88359ae4
restrict analysis task to faster machine (#20446) 2018-08-11 08:22:38 -07:00
liyuqian
e770685a4b
Mark complex_layout_scroll_perf__timeline_summary as non-flaky (#20272) 2018-08-08 10:25:36 -07:00
Greg Spencer
e60087a1a7
Remove Travis configuration from flutter/flutter (#20288)
This removes the final traces of Travis and Appveyor from the Flutter tree.

I've updated the documentation and fixed a couple of places where scripts look for Travis, and eliminated the dart tools runningOnTravis function (which was unused anyhow).

There are places in the flutter script that used to look for the environment variable TRAVIS. We actually do want to continue to detect that we're running on Travis there, since in the plugins repo we still use Travis (for the moment). In any case, it's OK, because the CI environment variable is set on all of the CI bots (Cirrus, Travis, and Appveyor).

FastLane doesn't have a setup_cirrus equivalent to setup_travis, but it actually doesn't matter there either, since it doesn't do Travis-specific things, and it also looks for the CI environment variable.
2018-08-07 13:41:33 -07:00
liyuqian
b6cd38d8df
Add tiles_scroll_perf__timeline_summary to devicelab (#20105) 2018-08-02 16:47:58 -07:00
Ryan Macnak
091da9c7d3 Revert "mark flaky benchmarks as flaky (#19492)"
This reverts commit 7a9dd709c6235153dd7500da65fb30dfbe76e560.

Bug: https://github.com/flutter/flutter/issues/19490
2018-08-01 14:15:22 -07:00
Chris Bracken
c0294e3a02
Eliminate Dart 1 benchmarks (#19430)
Flutter app development has been Dart 2 only for a significant amount of
time, and we're about to remove all support for Dart 1.
2018-07-18 15:56:51 -07:00
Michael Goderbauer
7a9dd709c6 mark flaky benchmarks as flaky (#19492) 2018-07-18 11:11:39 -07:00
Mikkel Nygaard Ravn
ca0e144f9f
Mark non-flaky tests as such (#18927) 2018-06-29 10:55:31 +02:00
Danny Tuppeny
e87b83bfd8 Mark flutter_gallery_instrumentation_test as flaky (#18887)
See #18879.
2018-06-27 15:48:22 -07:00
Ian Hickson
3ecd4301bf
Test performance of "flutter test" (#18810) 2018-06-26 17:34:25 -07:00
Ian Hickson
73498a54bc
Make flaky test as such. (#18815) 2018-06-25 19:25:01 -07:00
Mikkel Nygaard Ravn
d4db748047
Add2app devicelab test (#18795) 2018-06-25 22:37:47 +02:00
Chris Bracken
7213a7805a
Mark flutter_attach_test flaky (#18801)
Introduced in d248725e1509603551fc4e70e532c190743cd4ab, but has been red
since landing. Marking flaky rather than reverting since it's the only
failure.
2018-06-25 11:04:51 -07:00
Florian Loitsch
d248725e15
Flutter attach (#18677)
Add `flutter attach` command.

This command allows Flutter to connect to applications that haven't been launched using `flutter run`.
2018-06-25 15:33:42 +02:00
Michael Goderbauer
ebb174b0bf
Add size benchmark for hello_world (#18522) 2018-06-16 00:08:33 -07:00
Ian Hickson
d1cc8b6de8
Clean up output of "flutter run --release" (#18380)
(second attempt)
2018-06-14 12:30:20 -07:00
Chris Bracken
2ae48845a8
Revert elimination of Dart 1 (#18460)
fuchsia_tester.dart still assumes Dart 1. Previously, it ran tests directly
from source, flutter_platform.dart automatically runs a kernel compile when
operating in Dart 2 mode, but this assumes a functional Dart SDK is available
in the artifacts directly, and fuchsia_tester.dart mocks out the artifacts
directory with an empty temp dir.

Remaining work is:
1. Get the frontend server building as a dependency on Fuchsia.
2. Patch fuchsia_tester.dart to use a valid Dart SDK and frontend server.

This also reverts migration to Dart 2 typedef syntax.

This reverts commit 6c56bb2. (#18362)
This reverts commit 3daebd0. (#18316)
2018-06-13 12:46:39 -07:00
Chris Bracken
3daebd059c
It's time to #deleteDart1 (again) (#18316)
* It's time to #deleteDart1 (#18293)

Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.

Eliminates --preview-dart-2 / --no-preview-dart-2 support.

* Fix indentation, remove no longer necessary .toList()

* Only push udpated kernel if >0 invalidated srcs
2018-06-08 15:49:24 -07:00
Chris Bracken
9495df4bc3
Revert "It's time to #deleteDart1 (#18293)" (#18313)
Several benchmarks started failing during kernel compile:
1. Timeouts:
   * commands_test
   * hot_mode_dev_cycle__benchmark
   * run_machine_concurrent_hot_reload
   * service_extensions_test

2. Kernel compiler crash:
   * hot_mode_dev_cycle_linux__benchmark
   * routing_test

3. Unexpected termination:
   * hot_mode_dev_cycle_win__benchmark

This reverts commit ed63e7082ac89ea9851ae645ae69842da0c6150c.
2018-06-08 12:43:11 -07:00
Chris Bracken
ed63e7082a
It's time to #deleteDart1 (#18293)
Eliminates support for Dart 1 in flutter_tools, and drops our Dart 1
benchmarks. All commands now run in Dart 1 mode only.

Eliminates --preview-dart-2 / --no-preview-dart-2 support.
2018-06-08 11:04:57 -07:00
Ian Hickson
944f0a4cc1
Revert "Clean up output of "flutter run --release" (#18049)" (#18184)
This reverts commit 4f1b66044c075dee734d3a426a77d54b96971d0c.
2018-06-04 16:17:08 -07:00
Ian Hickson
4f1b66044c
Clean up output of "flutter run --release" (#18049) 2018-06-04 15:22:51 -07:00
Mikkel Nygaard Ravn
f986048094
Mark flaky test as such (#18019) 2018-05-30 12:54:46 +02:00
Todd Volkert
70772ea5a0
Remove flaky marker on some tests. (#17882)
They've been mostly green since https://github.com/flutter/flutter/pull/17812
2018-05-24 13:57:52 -07:00
Chris Bracken
efc657aabf
Mark 32-bit iOS gallery transitions test unflaky (#17662)
Remaining issue was fixed in 580c844c2fe37b680c0355b6b035e56b27da18df.
2018-05-16 11:57:06 -07:00
Chris Bracken
0cbda7d68d
Mark 32-bit iOS gallery startup test unflaky (#17660) 2018-05-16 11:13:44 -07:00
Chris Bracken
20512d458e
Add 32-bit iOS gallery startup test (#17653) 2018-05-16 09:29:10 -07:00
Sarah Zakarias
a3e71bf11f
Add startup test for Platform View example (#17645) 2018-05-16 13:24:47 +02:00
Chris Bracken
52246c048d
Add 32-bit iOS transition perf integration test (#17582) 2018-05-14 15:05:38 -07:00
Todd Volkert
33dd44bc98
Mark test as flaky (#17486) 2018-05-10 14:35:53 -07:00
Chris Bracken
4509b15d2d
Mark integration_ui flaky (#17319)
This test fails consistently on mac2 and mac3 with the attached Moto G4
devices but passes consistently on other machines.

Adding a delay of 1s right after driver.connect() in setUpAll() causes
it to pass on the machines in question, which suggests a race condition.
Specifically it looks like connect returns the moment Flutter Driver
identifies that the isolate is up and running, but empirically it looks
like we start running the first test before the UI is actually up. This
triggers a failure wherein we start looking for elements before they're
onstage.

Link to viewport.dart:213 at HEAD:
b2b4665926/packages/flutter/lib/src/widgets/viewport.dart (L213)

Stack trace:
FlutterDriver waitFor should find text "present"

```
  DriverError: Error in Flutter application: Uncaught extension error while executing waitFor: NoSuchMethodError: The getter 'visible' was called on null.
  Receiver: null
  Tried calling: visible
  #0      Object.noSuchMethod (dart:core/runtime/libobject_patch.dart:46:5)
  #1      _ViewportElement.debugVisitOnstageChildren. (package:flutter/src/widgets/viewport.dart:213:36)
  #2      WhereIterator.moveNext (dart:_internal/iterable.dart:439:11)
  #3      Iterable.forEach (dart:core/iterable.dart)
  #4      _ViewportElement.debugVisitOnstageChildren (package:flutter/src/widgets/viewport.dart:214:8)
  #5      _DepthFirstChildIterator._reverseChildrenOf (package:flutter_test/src/all_elements.dart:54:15)
  #6      _DepthFirstChildIterator.moveNext (package:flutter_test/src/all_elements.dart:45:19)
  #7      CachingIterable._fillNext (package:flutter/src/foundation/basic_types.dart:252:27)
  #8      _LazyListIterator.moveNext (package:flutter/src/foundation/basic_types.dart:279:21)
  #9      WhereIterator.moveNext (dart:_internal/iterable.dart:438:22)
  #10     CachingIterable._fillNext (package:flutter/src/foundation/basic_types.dart:252:27)
  #11     _LazyListIterator.moveNext (package:flutter/src/foundation/basic_types.dart:279:21)
  #12     Iterable.isEmpty (dart:core/iterable.dart:449:33)
  #13     Iterable.isNotEmpty (dart:core/iterable.dart:456:27)
  #14     FlutterDriverExtension._waitForElement. (package:flutter_driver/src/extension/extension.dart:215:51)
  #15     FlutterDriverExtension._waitUntilFrame (package:flutter_driver/src/extension/extension.dart:197:19)
  #16     FlutterDriverExtension._waitForElement (package:flutter_driver/src/extension/extension.dart:215:11)

  #17     FlutterDriverExtension._waitFor (package:flutter_driver/src/extension/extension.dart:286:11)

  #18     FlutterDriverExtension.call (package:flutter_driver/src/extension/extension.dart:168:51)

  #19     BindingBase.registerServiceExtension. (package:flutter/src/foundation/binding.dart:370:32)
```

Removes a previous hack that no longer appears to help (adding a 1
second delay in setUpAll() does seem to work around this issue though).
2018-05-06 18:00:35 -07:00
Todd Volkert
ae62a85699
Mark flaky tests as flaky (#17004) 2018-04-26 07:55:57 -07:00
Yegor
a90a850462
bump benchmark timeouts (#16817) 2018-04-20 13:57:06 -07:00
Mikkel Nygaard Ravn
29eebcbe88
Remove flaky flag from non-flaky test (#16075) 2018-03-29 22:02:29 +02:00
Todd Volkert
31049a5a73
Mark test as non-flaky now that it's been fixed (#16033) 2018-03-28 10:58:48 -07:00
Hans Muller
3fd0423a11
Marked flutter_gallery_instrumentation_test flaky (#15891) 2018-03-23 18:16:44 -07:00
Alexander Aprelev
21ad09fe57
With dart2 being default, move preview_dart_2 tests to dart1. (#15667)
* With dart being default, move preview_dart_2 tests to dart1.

* Swap out defaults for preview-dart-2 option in tests. Make dart1 explicit

* Move suffix around

* Rename files
2018-03-19 12:11:18 -07:00
Mikkel Nygaard Ravn
04de8c0a43
Adjust flaky flags to reality (#15608)
Get to green build.
2018-03-16 11:17:23 +00:00
Mikkel Nygaard Ravn
a61592793c Silence broken test (#15446)
* Silence broken test

* Mark test as flaky instead
2018-03-12 12:02:58 -07:00
Danny Tuppeny
465929f71e Remove flaky flag from all tests that are green for the whole dashboard
Plus run_machine_concurrent_hot_reload which isn't all green, but is
green since the fix was pushed.
2018-03-12 17:03:21 +00:00
Todd Volkert
5dca13dea5
Remove flaky marker from android_sample_catalog_generator (#14923) 2018-02-27 10:30:45 -08:00