How to upload test reports of Kotlin sources to Coveralls?

Question

I want to upload my Jacoco test report to Coveralls automatically after my Travis build finishes. It works for Java, but how to configure it for Kotlin?

Error message

I can generate a Jacoco test report locally and on Travis, but when Travis tries to submit to coveralls it fails with message

> Task :coveralls
No source file found on the project: "kotlin-template-project"
With coverage file: /home/travis/build/myname/myreponame/build/reports/jacoco/test/jacocoTestReport.xml

Google links me to the Gradle plugin implementation which shows where it throws this message, which tells me (I think) that the Jacoco report file is found but not the source files which coveralls apparently needs.

What I tried

Hence, I tried pointing the coveralls task to my source files, in all of these ways:

coveralls {
    sourceDirs += allprojects.sourceSets.main.allSource.srcDirs.flatten()
    sourceDirs += files(sourceSets.main.kotlin.srcDirs).files.absolutePath
    project.extensions.coveralls.sourceDirs += project.sourceSets.main.kotlin.srcDirs
    sourceDirs += ['src/main/kotlin']
    jacocoReportPath = 'build/reports/jacoco/test/jacocoTestReport.xml'
    sourceDirs += ['src/test/kotlin']
    sourceDirs += ["${projectDir}/src/main/kotlin"]
}

I also tried adding sourceSets project.sourceSets.main to the jacocoTestReport task.

Project setup

My minimal build.gradle file:

plugins {

    id 'org.jetbrains.kotlin.jvm' version '1.2.50'
    id 'java' // Required by at least JUnit.

    // Test coverage
    id 'jacoco'

    // Upload jacoco coverage reports to coveralls
    id 'com.github.kt3k.coveralls'  version '2.8.2'
}

dependencies {
    compile 'org.jetbrains.kotlin:kotlin-stdlib-jdk8'

    // JUnit 5
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.2.0'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.2.0'
    testRuntime 'org.junit.platform:junit-platform-console:1.2.0'

    // Kotlintest
    testCompile 'io.kotlintest:kotlintest-core:3.1.6'
    testCompile 'io.kotlintest:kotlintest-assertions:3.1.6'
    testCompile 'io.kotlintest:kotlintest-runner-junit5:3.1.6'

    // Spek
    testCompile 'org.jetbrains.spek:spek-api:1.1.5'
    testRuntime 'org.jetbrains.spek:spek-junit-platform-engine:1.1.5'
}

repositories {
    mavenCentral()
    mavenLocal()
    jcenter()
}

test {
    // Enable JUnit 5 (Gradle 4.6+).
    useJUnitPlatform()

    // Always run tests, even when nothing changed.
    dependsOn 'cleanTest'

    // Show test results.
    testLogging {
        events "passed", "skipped", "failed"
    }
}

// Test coverage reporting
jacocoTestReport {
    // Enable xml for coveralls.
    reports {
        html.enabled = true
        xml.enabled = true
        xml.setDestination(file("${buildDir}/reports/jacoco/test/jacocoTestReport.xml"))
    }
}

coveralls {
    sourceDirs += ['src/main/kotlin']
    jacocoReportPath = 'build/reports/jacoco/test/jacocoTestReport.xml'
}

Related issues

• Essentially the same issue is on https://github.com/kt3k/coveralls-gradle-plugin/issues/32 but the solution is to set sourceDirs and jacocoReportPath which I already have tried.
• At https://github.com/kt3k/coveralls-gradle-plugin/issues/39 and https://github.com/kt3k/coveralls-gradle-plugin/issues/63 it is suggested to add sourceDirs += ['src/main/kotlin'] which sounds sensible but doesn't help. Same for, from the first link, sourceDirs = files(sourceSets.main.kotlin.srcDirs).files.absolutePath.
• From https://github.com/kt3k/coveralls-gradle-plugin/issues/77 the solution is project.extensions.coveralls.sourceDirs += project.sourceSets.main.kotlin.srcDirs which I tried.

• The question Kotlin code coverage in CI pipeline is phrased rather generally, but a comment points to discuss.kotlinlang.org where someone shows a way to improve the Jacoco result regarding kotlin, and the answer links to the Jacoco Gradle plugin which I use and works: when I run the jacocoTestReport task a report is generated in build/reports/jacoco/test/, both in xml and html.

• The question Kotlin Test Coverage is also phrased general and answered with an unnecessarily complex build file from which I learned nothing new.

• The question Measure test coverage for Kotlin code? claims that the Jacoco report does not work, but for me this is not the case as I said.
• There are similar questions for Java, like Tool for java code coverage on GitHub but for me when I use Java it all works fine.

PS Actually I want to use the Gradle Kotlin DSL, but since nobody seems to use it I'm asking this question for Gradle. But in the end I want this question solved for the Kotlin DSL as well.

Amorphism

Answer 1

Had a similar experience with a variety of QA products not supporting or only partially supporting Kotlin codebases. Tried submitting support PRs to a couple of projects to no avail.

In the end ended up going with Coveralls and contributed a Kotlin focused plugin for the platform

https://github.com/nbaztec/coveralls-jacoco-gradle-plugin

Usage

Include the plugin in your build.gradle.kts (similar for build.gradle files):

plugins {
    jacoco
    id("com.github.nbaztec.coveralls-jacoco")
}

Then set the environment variable COVERALLS_REPO_TOKEN to the token from your Coveralls page.

Now you can use the coverallsJacoco task to publish a coverage report.

For more information and usage in CI, see https://github.com/nbaztec/coveralls-jacoco-gradle-plugin

Apteral

After I tried to use this plugin to submit the unitTest report I generated for my Kotlin project, I experienced some issues. More concretely I didn't manage to provide a correct path to my Kotlin source code sets on reportSourceSets after trying many different options. And if I leave this field blank i.e. "", the compiler will complain with a ClassCastException: java.lang.String cannot be cast to java.io.File Do you have an example on how to provide this path correctly @Apteral ? – Veinule

it's supposed to be a Collection<File> so [file("src/main/java"), ...] / listOf(file("src/main/kotlin"), ...) should be used docs might help, I'll perhaps add more examples on option usage – Apteral

Answer 2

[edit August 2020] @nbaztec wrote a plugin to support Kotlin, please see his answer.

---

Old answer:

Kotlin is not supported by Coveralls, see for example this open isse that was mentioned in the question as well (in the question it was also mentioned that the workaround presented there does not work): https://github.com/kt3k/coveralls-gradle-plugin/issues/77

Solution: try Codecov.io instead. Install it to GitHub using the Marketplace and add to your .travis.yml

after_success:
  - bash <(curl -s https://codecov.io/bash)

Then commit and push, done!

You can view the result (after the build finished) at https://codecov.io/gh/githubaccountname/reponame

Amorphism

Was in the same boat running from coverage to coverage yet no luck with Kotlin. Ended up contributing to coveralls with a Kotlin focused plugin: docs.coveralls.io/kotlin – Apteral

Answer 3

Not an answer, but in case anyone else is struggling with nbaztec like me, I want to give an alternative that worked for me: https://github.com/kt3k/coveralls-gradle-plugin

And besides what is in README.md, I needed this detail in build.gradle:

coveralls {
    sourceDirs += ['src/main/kotlin']
    jacocoReportPath "${buildDir}/reports/jacoco/report.xml"
}

Kenaz

Interesting, as the question indicates that using this plugin like that did not work. But maybe it has been updated and it works now. Thanks for the answer, in any case. – Amorphism