The other answer by @Fah helped be find the solution that I ended up using. Also, there are great many posts around the web describing how people have integrated JaCoCo into their projects in various ways.
The solutions are mostly similar, but all a little different because they fit into different projects. And none of them worked directly for my setup. So while the problem really isn't very complicated, it's important to understand how the pieces fit together in order to make the right adjustments for your particular project. The following is my attempt as a Gradle noob at explaining all these pieces from what I've gathered while investigating this:
Fundamentally, the solution breaks down to
Enable coverage instrumentation/collection in the unit test runner.
Define one or more Gradle tasks of type JacocoReport (which is defined by the jacoco plugin?). This task will collect the execution data produced by the test runner into a report and therefore depends on a task for actually running the tests.
Configure the task(s) with the correct locations of source, target, execution files, etc.
Add any additional JaCoCo customizations.
As mentioned in the question, the Android Gradle Plugin (AGP) already includes JaCoCo support. This means that we don't need to explicitly enable the jacoco plugin. It looks like the used version can be configured using
android {
jacoco {
version = "..."
}
}
This blog post discusses various shortcomings on AGP's JaCoCo support and proposes a Gradle plugin (in Kotlin) to address them.
1. Enable coverage instrumentation
AGP supports this directly by setting enableUnitTestCoverage on the appropriate build type:
android {
...
defaultConfig {
...
debug {
...
enableUnitTestCoverage true
}
}
There also is the testCoverageEnabled (deprecated combination of enableUnitTestCoverage and enableAndroidTestCoverage) option which additionally creates tasks that we can't use when running in CI without an actual device as described in the question.
2. Create Gradle task
Gradle tasks are defined with the tasks.create method. A task has a name, a type, and a set of other tasks that it depends on. In this case this means something like
tasks.create(name: "unitTestCoverageReport", type: JacocoReport, dependsOn: "testLocalDebugUnitTest") {
// ... task implementation ...
}
where testLocalDebugUnitTest is an existing task that runs the unit tests with coverage collection enabled. Use ./gradlew tasks | grep -i test to list all available tasks related to testing.
The task can be defined in the gradle.build file of the module that needs it (what I did), or it can be defined in a separate file to be applied from the toplevel module as in @fah's answer or imported explicitly as in some of the posts listed below.
By virtue of this being written in a full programming language, there are many possibilities for setting up abstractions around this construct in whatever way makes sense for the project at hand:
This example shows how JaCoCo may be integrated into a monorepo-like project, i.e. a project with multiple Gradle modules. It defines a generic JaCoCo Gradle module that can be imported into the individual build.gradle files to add a debugCoverage task for each of them (assuming that they have a variant "debug"). It then adds a dummy module jacoco with a single task allDebugCoverage that collects the coverage data produced by debugCoverage from all the other modules (assuming that they have been run first). The JaCoCo Report Aggregation Plugin looks like a possible alternative solution.
This example creates a task for each variant.
This example creates it for each flavor.
With the task unitTestCoverageReport being defined, it may be run using ./gradlew unitTestCoverageReport. Dependent tasks will be automatically be executed (transitively) first.
3. Configure Gradle task
As mentioned above, the task implementation sets up the configuration of JaCoCo report generator. As also seen in @Fah's answer, the relevant fields are:
reports: block for configuring which kinds of reports to generate.
sourceDirectories.from: The location of the original source code files. Optional, but allows the reporter to map bytecode instructions back to the original sources in order to include them into the report with coverage annotations.
classDirectories.from: The compiled class files without instrumentation. Execution data references instructions in these classes, so they're necessary for locating the actual instructions.
executionData.from: The location of "execution data", i.e. the coverage data collected by the unit test runner when running the tests against classes instrumented by the JaCoCo agent.
So while one just has to set these paths correctly, this can be challenging when the build process is poorly documented and you don't have good understanding of the underlying structure. In my case it involved a combination of guessing, searching the build directory for files with specific extensions, and reading other examples. I have for example not found any official documentation explaining that compiled Kotlin classes go into ${buildDir}/tmp/kotlin-classes.
Despite being named "directories", all guides I've seen pass the entire directory tree using the fileTree function with a bunch of excludes. For my Kotlin Android project, the following configuration suffices:
tasks.create(name: "unitTestCoverageReport", type: JacocoReport, dependsOn: "testLocalDebugUnitTest") {
group = "Verification" // existing group containing tasks for generating linting reports etc.
description = "Generate Jacoco coverage reports for the 'local' debug build."
reports {
// human readable (written into './build/reports/jacoco/unitTestCoverageReport/html')
html.enabled = true
// CI-readable (written into './build/reports/jacoco/unitTestCoverageReport/unitTestCoverageReport.xml')
xml.enabled = true
}
// Execution data generated when running the tests against classes instrumented by the JaCoCo agent.
// This is enabled with 'enableUnitTestCoverage' in the 'debug' build type.
executionData.from = "${project.buildDir}/outputs/unit_test_code_coverage/localDebugUnitTest/testLocalDebugUnitTest.exec"
// Compiled Kotlin class files are written into build-variant-specific subdirectories of 'build/tmp/kotlin-classes'.
classDirectories.from = "${project.buildDir}/tmp/kotlin-classes/localDebug"
// To produce an accurate report, the bytecode is mapped back to the original source code.
sourceDirectories.from = "${project.projectDir}/src/main/java"
}
I think the XML report is mostly useful for further processing like uploading the report to coveralls.
Some guides wrap the task definition in project.afterEvaluate. I didn't find a need for this and don't know enough about Gradle to understand what difference it makes.
4. Additional JaCoCo customizations
To collect coverage data for tests executed by for example RobolectricTestRunner, the following snippet must be added as well:
tasks.withType(Test) {
jacoco.includeNoLocationClasses true
jacoco.excludes = ['jdk.internal.*']
}
This post suggests using the testOptions block to set configurations like these.
Finally, the jacoco plugin also includes a configuration jacocoTestCoverageVerification for failing the build if certain coverage metrics aren't met.
GitHub Actions
The following example workflow runs the Gradle task and uploads the generated HTML report as a workflow artifact using madrapps/jacoco-report:
name: Build and test with coverage
on: ...
env:
java_version: "17"
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up JDK
uses: actions/setup-java@v3
with:
java-version: "${{env.java_version}}"
distribution: temurin
cache: gradle # caches dependencies (https://github.com/actions/setup-java#caching-packages-dependencies)
- name: Build project and run unit tests
run: ./gradlew test unitTestCoverageReport --no-daemon
- name: Add coverage comment to PR
uses: madrapps/[email protected]
with:
paths: "${{github.workspace}}/app/build/reports/jacoco/unitTestCoverageReport/unitTestCoverageReport.xml"
token: "${{secrets.GITHUB_TOKEN}}"
- name: Store HTML coverage report
uses: actions/upload-artifact@v3
with:
name: coverage-report
path: "${{github.workspace}}/app/build/reports/jacoco/unitTestCoverageReport/html/"
registeringa task is more efficient thancreating#53654690 – Tanked