Relands #136851, which was rolled back in #137121
package:coverage has been rolled, so the breakages should be fixed.
Also, in this reland I've changed the `coverableLineCache` parameter to
be optional, which is safer.
Reverts flutter/flutter#136851
Initiated by: CaseyHillers
This change reverts the following previous change:
Original Description:
One of the reasons gathering coverage information is expensive is that we have to force compile every function in the libraries we're interested in. Without this, functions that haven't been invoked (so haven't been compiled) won't have any line number information, so the coverage tool doesn't know which lines to add to the list of misses. In flutter's case, the test infra spawns many VMs, and each of these needs to recompile all those libraries.
To fix this, we need a way of skipping force compilation for libraries we've already seen in previous tests, without losing the information about which lines in each library are coverable. So I [added](https://github.com/dart-lang/coverage/pull/466) the `coverableLineCache` to `coverage.collect` in package:coverage v1.7.0. This cache starts out empty, but fills up with lists of all the lines that are coverable for every library as coverage is gathered. package:coverage can then tell the VM not to force compile any libraries in this cache (using `getSourceReport`'s `librariesAlreadyCompiled` param). So the first test suite will still have to compile everything, but subsequent test suites will be much faster.
This speeds up coverage collection significantly, for large test suites:
| Running flutter/packages/flutter tests... | Time | Overhead |
| --- | --- | --- |
| without coverage | 8:53 | - |
| with coverage | 20:25 | 130% |
| with `coverableLineCache` | 12:21 | 40% |
Bug: https://github.com/flutter/flutter/issues/100751
One of the reasons gathering coverage information is expensive is that we have to force compile every function in the libraries we're interested in. Without this, functions that haven't been invoked (so haven't been compiled) won't have any line number information, so the coverage tool doesn't know which lines to add to the list of misses. In flutter's case, the test infra spawns many VMs, and each of these needs to recompile all those libraries.
To fix this, we need a way of skipping force compilation for libraries we've already seen in previous tests, without losing the information about which lines in each library are coverable. So I [added](https://github.com/dart-lang/coverage/pull/466) the `coverableLineCache` to `coverage.collect` in package:coverage v1.7.0. This cache starts out empty, but fills up with lists of all the lines that are coverable for every library as coverage is gathered. package:coverage can then tell the VM not to force compile any libraries in this cache (using `getSourceReport`'s `librariesAlreadyCompiled` param). So the first test suite will still have to compile everything, but subsequent test suites will be much faster.
This speeds up coverage collection significantly, for large test suites:
| Running flutter/packages/flutter tests... | Time | Overhead |
| --- | --- | --- |
| without coverage | 8:53 | - |
| with coverage | 20:25 | 130% |
| with `coverableLineCache` | 12:21 | 40% |
Bug: https://github.com/flutter/flutter/issues/100751
PR provides a new option to the `test` command to include coverage info of specified packages.
It helps collecting coverage info in test setups where test code lives in separate packages or for multi-package projects.
At present, only current package is included to the final report.
Usage:
Consider an app with two packages: `app`, `common`.
Some of the tests in `app` use (indirectly) code that is located in `common`. When running with `--coverage` flag, that code is not included in the coverage report by default. To include `common` package in report, we can run:
```sh
flutter test --coverage --coverage-package app --coverage-package common
```
Note that `--coverage-package` accepts regular expression.
Fixes https://github.com/flutter/flutter/issues/79661
Fixes https://github.com/flutter/flutter/issues/101486
Fixes https://github.com/flutter/flutter/issues/93619
Speedup coverage test runs by using (new) coverage handle with caching.
On a `flutter test --coverage` run on `packages/flutter` the runtime goes from ~828 seconds to ~617 seconds, or a ~25% reduction in time spent (testing without coverage takes ~236 seconds so the overhead of `--coverage` in this case goes from ~592 seconds to ~381 seconds, or a ~35% reduction).
In https://github.com/flutter/flutter/pull/103771, we rolled
dependencies in Flutter, which triggered an update of package:coverage
to v1.3.1. The new version includes
https://github.com/dart-lang/coverage/pull/370 in which two deprecations
landed:
* The `Resolver` default constructor was deprecated and replaced with
the `Resolver.create` static factory method, which unfortunately
happens to be async.
* The `packagesPath` parameter to `HitMap.parseJson`, which takes the
path to the `.packages` file of the package for which coverage is to
be collected, was deprecated. This parameter was replaced with
`packagePath` in https://github.com/dart-lang/coverage/pull/370 which
was part of the overall deprecation of the .packages file in Dart
itself https://github.com/dart-lang/sdk/issues/48272. The overall goal
being that end-user code shouldn't need to know about implementation
details such as whether dependency information is stored in a
.packages file or a package_info.json file, but rather use the
package_config package to obtain the package metadata and perform
other functions such as resolving its dependencies to filesystem
paths. packagesPath was replaced by packagePath, which takes the path
to the package directory itself. Internally, package:coverage then
uses package_config to do the rest of the package/script URI
resolution to filesystem paths.
This migrates off the deprecated `packagesPath` parameter to the
replacement `packagePath` paramter.
Issue: https://github.com/flutter/flutter/issues/103830
* Use libraryFilters flag to speed up coverage collection
* Allow libraryNames to be null
* Unconditionally enable the reportLines flag
* Fix analysis errors
Roll dependendencies
This rolls depdendencies to latest using
flutter update-packages --force-upgrade
This change includes three code changes:
* Removes charcode from the dependencies allowlist since it no longer
appears in the transitive closure of dependencies of the flutter,
flutter_test, flutter_driver, flutter_localizations, and
integration_test packages.
* Uses Resolver.create instead of the deprecated Resolver constructor.
The default Resolver constructor has been deprecated in favour of the
static Resolver.create() factory function, which unfortunately happens
to be async. Propagated the async-ness up the chain.
This change was partially reverted and the deprecation ignored in this
patch until package:coverage can be rolled internally at Google.
* Eliminates the use of the deprecated packagesPath parameter to
HitMap.parseJson. This parameter was deprecated and replaced with
packagePath in https://github.com/dart-lang/coverage/pull/370 which
was part of the overall deprecation of the .packages file in Dart
itself https://github.com/dart-lang/sdk/issues/48272. The overall goal
being that end-user code shouldn't need to know about implementation
details such as whether dependency information is stored in a
.packages file or a package_info.json file, but rather use the
package_config package to obtain the package metadata and perform
other functions such as resolving its dependencies to filesystem
paths. packagesPath was replaced by packagePath, which takes the path
to the package directory itself. Internally, package:coverage then
uses package_config to do the rest of the package/script URI
resolution to filesystem paths.
This change was partially reverted and the deprecation ignored in this
patch until package:coverage can be rolled internally at Google.
This is a pre-update prior to updating flutter_template_images in
https://github.com/flutter/flutter/pull/103739
Issue: https://github.com/flutter/flutter/issues/103371
Issue: https://github.com/flutter/flutter/issues/103775
Issue: https://github.com/flutter/flutter/issues/103830
When re-applying the partially-reverted changes to code coverage,
we'll need to patch host_entrypoint.dart internally to await the Future
that we'll be returning rather than a non-async value.
https://github.com/dart-lang/coverage - 0.14.0 added the ability to be able to ignore coverage depending on comments.
// coverage:ignore-line to ignore one line.
// coverage:ignore-start and // coverage:ignore-end to ignore range of lines inclusive.
// coverage:ignore-file to ignore the whole file.
