120a01ccd2
Re-sets two jvmargs that were getting cleared because we set a value for `-Xmx`. Could help with https://github.com/flutter/flutter/issues/142957. Copied from comment here https://github.com/flutter/flutter/issues/142957: >Two random things I ran into while looking into this that might help: > >1. Gradle has defaults for a couple of the jvmargs, and setting any one of them clears those defaults for the others (bug here https://github.com/gradle/gradle/issues/19750). This can cause the "Gradle daemon to consume more and more native memory until it crashes", though the bug typically has a different associated error. It seems worth it to re-set those defaults. >2. There is a property we can set that will give us a heap dump on OOM ([-XX:HeapDumpOnOutOfMemoryError](https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/clopts001.html)) Mostly just a find and replace from `find . -name gradle.properties -exec sed -i '' 's/\-Xmx4G/-Xmx4G\ \-XX:MaxMetaspaceSize=2G\ \-XX:+HeapDumpOnOutOfMemoryError/g' {} \;`, with the templates and the one test that writes from a string replaced by hand. I didn't set a value for `MaxMetaspaceSize` in the template files because I want to make sure this value doesn't cause problems in ci first (changes to the templates are essentially un-revertable for those who `flutter create` while the changes exist).
Macrobenchmarks
Performance benchmarks use either flutter drive or the web benchmark harness.
Mobile benchmarks
Cull opacity benchmark
To run the cull opacity benchmark on a device:
flutter drive --profile -t test_driver/run_app.dart --driver test_driver/cull_opacity_perf_test.dart
Results should be in the file build/cull_opacity_perf.timeline_summary.json.
More detailed logs should be in build/cull_opacity_perf.timeline.json.
Cubic bezier benchmark
To run the cubic-bezier benchmark on a device:
flutter drive --profile -t test_driver/run_app.dart --driver test_driver/cubic_bezier_perf_test.dart
Results should be in the file build/cubic_bezier_perf.timeline_summary.json.
More detailed logs should be in build/cubic_bezier_perf.timeline.json.
Backdrop filter benchmark
To run the backdrop filter benchmark on a device: To run a mobile benchmark on a device:
flutter drive --profile -t test_driver/run_app.dart --driver test_driver/[test_name]_test.dart
Results should be in the file build/[test_name].timeline_summary.json.
More detailed logs should be in build/[test_name].timeline.json.
The key [test_name] can be:
animated_placeholder_perfbackdrop_filter_perfcolor_filter_and_fade_perfcubic_bezier_perfcull_opacity_perffading_child_animation_perfimagefiltered_transform_animation_perfmulti_widget_construction_perfpicture_cache_perfpost_backdrop_filter_perfsimple_animation_perftextfield_perffullscreen_textfield_perf
E2E benchmarks
(On-going work)
E2E-based tests are driven independent of the host machine. The following tests are E2E:
cull_opacity_perf.dartmulti_widget_construction_perf
These tests should be run by:
flutter drive --profile -t test/[test_name]_e2e.dart --driver test_driver/e2e_test.dart
Web benchmarks
Web benchmarks are compiled from the same entry point in lib/web_benchmarks.dart.
How to write a web benchmark
Create a new file for your benchmark under lib/src/web. See bench_draw_rect.dart
as an example.
Choose one of the two benchmark types:
- A "raw benchmark" records performance metrics from direct interactions with
dart:uiwith no framework. This kind of benchmark is good for benchmarking low-level engine primitives, such as layer, picture, and semantics performance. - A "widget benchmark" records performance metrics using a widget. This kind of benchmark is good for measuring the performance of widgets, often together with engine work that widget-under-test incurs.
- A "widget build benchmark" records the cost of building a widget from nothing. This is different from the "widget benchmark" because typically the latter only performs incremental UI updates, such as an animation. In contrast, this benchmark pumps an empty frame to clear all previously built widgets and rebuilds them from scratch.
For a raw benchmark extend RawRecorder (tip: you can start by copying
bench_draw_rect.dart).
For a widget benchmark extend WidgetRecorder (tip: you can start by copying
bench_simple_lazy_text_scroll.dart).
For a widget build benchmark extend WidgetBuildRecorder (tip: you can start by copying
bench_build_material_checkbox.dart).
Pick a unique benchmark name and class name and add it to the benchmarks list
in lib/web_benchmarks.dart.
How to run a web benchmark
Web benchmarks can be run using flutter run in debug, profile, and release
modes, using either the HTML or the CanvasKit rendering backend. Note, however,
that running in debug mode will result in worse numbers. Profile mode is useful
for profiling in Chrome DevTools because the numbers are close to release mode
and the profile contains unobfuscated names.
Example:
cd dev/benchmarks/macrobenchmarks
# Runs in profile mode using the HTML renderer
flutter run --web-renderer=html --profile -d web-server lib/web_benchmarks.dart
# Runs in profile mode using the CanvasKit renderer
flutter run --web-renderer=canvaskit --profile -d web-server lib/web_benchmarks.dart
You can also run all benchmarks exactly as the devicelab runs them:
cd dev/devicelab
# Runs using the HTML renderer
../../bin/cache/dart-sdk/bin/dart bin/run.dart -t bin/tasks/web_benchmarks_html.dart
# Runs using the CanvasKit renderer
../../bin/cache/dart-sdk/bin/dart bin/run.dart -t bin/tasks/web_benchmarks_canvaskit.dart
Frame policy test
File test/frame_policy.dart and its driving script test_driver/frame_policy_test.dart
are used for testing fullyLive
and benchmarkLive
policies in terms of its effect on WidgetTester.handlePointerEventRecord.