Kotlin Test Coverage

Question

Does anyone know if a good test coverage tool (preferably Gradle plugin) exists for Kotlin? I've looked into JaCoCo a bit, but it doesn't seem to reliably support Kotlin.

Modesta

I'm using Jacoco and it works well enough for me. Are you experiencing specific issues, or it has to be 100% effective and accurate in order for you to consider it? I don't rely on a coverage tool that heavily to tell me if something is adequately tested, but perhaps you're in a different environment. To answer your question, I don't think anyone's written something specifically for Kotlin. – Pauperize

@Pauperize If you got it working, maybe you could give an answer with an example build.gradle where it works with Kotlin. I have spent a lot of time trying to get it to work and so far only failed. Neither have I found anyone else that got it working either. – Reiterate

Added answer. Please review and let me know if you have questions – Pauperize

Related post: https://mcmap.net/q/591627/-kotlin-code-coverage-in-ci-pipeline/8583692 – Okechuku

Answer 1

As requested, here is an example build.gradle that uses Kotlin, and incorporates both Jacoco and Sonarqube integration, produces a jar and sources, and ties in Detekt for static analysis.

I had to manually add a couple things as my work build has jacoco applied by an in-house plugin.

plugins {
    id 'org.jetbrains.kotlin.jvm' version '1.2.10'
    id 'org.jetbrains.kotlin.plugin.spring' version '1.2.10'
    id 'org.springframework.boot' version '1.5.9.RELEASE'
    id 'io.spring.dependency-management' version '1.0.4.RELEASE'
    id 'io.gitlab.arturbosch.detekt' version '1.0.0.RC6'
    id "org.sonarqube" version "2.6.2".  
}

apply plugin: 'jacoco'

ext {
    springBootVersion = '1.5.9.RELEASE'
    springCloudVersion = 'Dalston.SR4'
    kotlinVersion = '1.2.10'
    detektVersion = '1.0.0.RC6'.  
}

//======================= Project Info =============================================
group = 'group'
version = '0.14'

dependencyManagement {
    imports {
        mavenBom("org.springframework.boot:spring-boot-starter-        parent:$springBootVersion") 
        mavenBom "org.springframework.cloud:spring-cloud-dependencies:$springCloudVersion"
    }
}

repositories {
    jcenter()
}

//======================= Dependencies =============================================
dependencies {
    // Version MUST be explicitly set here or else dependent projects will not be able to build as Gradle will not know
    // what version to use
    compile "org.jetbrains.kotlin:kotlin-stdlib-jdk8:$kotlinVersion"

    compile 'org.springframework.boot:spring-boot-starter-web'
    compile('org.springframework.ws:spring-ws-support') {
        exclude(module: 'javax.mail')
    }
    compile 'org.springframework.boot:spring-boot-starter-actuator'

    // Spring security
    compile 'org.springframework.security:spring-security-web'
    compile 'org.springframework.security:spring-security-config'
    compile 'javax.servlet:javax.servlet-api'
    compile 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml'
    compile('org.apache.httpcomponents:httpclient')
    compile 'com.nimbusds:nimbus-jose-jwt:4.23'
}

//======================= Tasks =============================================
defaultTasks 'build'

tasks.bootRepackage.enabled = false

tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).all {
    kotlinOptions {
        jvmTarget = 1.8
        freeCompilerArgs = ["-Xjsr305=strict"]
    }
}

compileJava.options.encoding = 'UTF-8'

test.testLogging.exceptionFormat = 'full'

// ********************************

ext.coverageExclusions = [
    // Configuration
    'com.bns.pm.config.*',
    // data classes
    'com.bns.pm.domain.*',
    // Account Service domain objects
    'com.bns.pm.account.domain.*',
    // Other items
    'com.bns.pm.exceptions.DataPowerFaultException.Companion',
    'com.bns.pm.controllers.ServiceExceptionHandler*',
    'com.bns.pm.service.callback.DPWebServiceMessageCallback',
    'com.bns.pm.service.HealthCheckService',
    'com.bns.pm.util.SystemPropertiesUtilKt',
]

check.dependsOn jacocoTestCoverageVerification
jacocoTestCoverageVerification {
    violationRules {
        rule {
            element = 'CLASS'
            // White list
            excludes = coverageExclusions
            limit {
                minimum = 0.70
            }
        }
    }
}

jacocoTestReport {
    description 'Generates Code coverage report. Fails build if it does not meet minimum coverage.'

    reports {
        xml.enabled = true    //XML required by coveralls and for the     below coverage checks
        html.enabled = true
        csv.enabled = false
    }

    def reportExclusions = coverageExclusions.collect {
        it.replaceAll('\\.', '/') + (it.endsWith('*') ? '' : '*')
    }
    afterEvaluate {
        classDirectories = files(classDirectories.files.collect {
            fileTree(dir: it, excludes: reportExclusions)
        })
    }
}

test.finalizedBy jacocoTestReport

afterEvaluate {
    sonarqube {
        properties {
            property 'sonar.jacoco.reportPath', "${buildDir}/jacoco/test.exec"
            property "detekt.sonar.kotlin.config.path", "detekt.yml"
            property 'sonar.java.binaries', "$projectDir/build/classes/kotlin"
            property 'sonar.coverage.exclusions', coverageExclusions.collect {
                '**/' + it.replaceAll('\\.', '/') + (it.endsWith('*') ? '' : '*')
            }
        }
    }
}

// Ensure source code is published to Artifactory
// Have to redefine publishing with new name as Accelerator Plugin already defined mavenJava
task sourceJar(type: Jar) {
    from sourceSets.main.allSource
    classifier 'sources'
}

publishing {
    publications {
        mavenJava2(MavenPublication) {
            from components.java
            artifact(sourceJar) {
                classifier = 'sources'
            }
        }
    }
}

artifactory {
    contextUrl = "${artifactory_contextUrl}"
    publish {
        repository {
            repoKey = "${artifactory_projectRepoKey}"
            username = "${artifactory_user}"
            password = "${artifactory_password}"
            maven = true
        }
        defaults {
            publications('mavenJava2')
        }
    }
}

artifactoryPublish {
    dependsOn jar
}

detekt {
    version = detektVersion
    profile("main") {
        input = "$projectDir/src"
        config = "$projectDir/detekt.yml"
        filters = ".*/resources/.*,.*/tmp/.*"
        output = "$project.buildDir/reports/detekt"
    }
}
check.dependsOn detektCheck

Pauperize

JaCoCo incorrectly reports some lines as "partially covered" instead of "fully covered" for @Autowired lateinit variables for example. See #45525779. And no, I don't have a suggestion for a better tool to use with gradle and in a CI job for Kotlin test coverage. I'm also looking... – Susian

Agreed. Jacoco is the best we have available right now, but it isn't 'amazing'. There are a number of Kotlin things it can't determine coverage on because it works on byte code. inline functions are a big one. – Pauperize

Answer 2

Good news: there is a new kotlinx-kover Gradle plugin, compatible with JaCoCo and IntelliJ.

plugins {
     id("org.jetbrains.kotlinx.kover") version "0.5.0"
}

Once applied, the plugin can be used out of the box without additional configuration.

Watch its YouTube announcement video and also track its roadmap from this youtrack issue.
As said in the video, it solves the problem with inline functions and more.

Okechuku