plugin-under-test-metadata.properties not created by Gradle TestKit when running tests in IDEA

Question

I am using Gradle 3.3 and trying to test a custom plugin with JUnit and Gradle TestKit. In plugin's build.gradle I have

version '0.1'

apply plugin: 'groovy'
apply plugin: 'java-gradle-plugin'

sourceCompatibility = 1.8

repositories {
    mavenCentral()
}

dependencies {
    compile project(':codegen-core')
    compile localGroovy()
    testCompile 'junit:junit:4.12'
}

The test is

package com.huawei.odmf.codegen.gradle

import org.gradle.testkit.runner.BuildResult
import org.gradle.testkit.runner.GradleRunner
import org.gradle.testkit.runner.TaskOutcome
import org.junit.Before
import org.junit.Rule
import org.junit.Test
import org.junit.rules.TemporaryFolder

import static org.junit.Assert.*

class TestOdmfCodegenPlugin {

    @Rule 
    public final TemporaryFolder testProjectDir = new TemporaryFolder()

    private File buildFile
    private File assetsDir

    @Before
    void setUp() {
        buildFile = testProjectDir.newFile("build.gradle")
        assetsDir = testProjectDir.newFolder("src", "main", "assets")
    }

    @Test
    void testPlugin() {
        buildFile << """
          plugins {
          id 'com.huawei.odmf'
        }

        apply plugin: 'com.android.application'

        odmf {
            modelFile 'odmf.xml'
        }
        """

        BuildResult result = GradleRunner.create().
                withProjectDir(testProjectDir.root).
                withArguments(OdmfCodegenPlugin.taskName).
                withPluginClasspath().
                build()

        // assertions
    }
}

Under src/main/resources/META-INF/gradle-plugins/com.huawei.odmf.properties I have

implementation-class=com.huawei.odmf.codegen.gradle.OdmfCodegenPlugin

This seems to be all that's required according to https://docs.gradle.org/current/userguide/test_kit.html#sub:test-kit-automatic-classpath-injection and the automaticClasspathInjectionQuickstart sample.

However, this test fails at withPluginClasspath() (EDIT: when run in IDEA; it works from command line) with the following stack trace (to my understanding, plugin-under-test-metadata.properties is supposed to be created by java-gradle-plugin automatically):

org.gradle.testkit.runner.InvalidPluginMetadataException: Test runtime classpath does not contain plugin metadata file 'plugin-under-test-metadata.properties'

  at org.gradle.testkit.runner.internal.PluginUnderTestMetadataReading.readImplementationClasspath(PluginUnderTestMetadataReading.java:44)
  at org.gradle.testkit.runner.internal.PluginUnderTestMetadataReading.readImplementationClasspath(PluginUnderTestMetadataReading.java:37)
  at org.gradle.testkit.runner.internal.DefaultGradleRunner.withPluginClasspath(DefaultGradleRunner.java:146)
  at org.gradle.testkit.runner.internal.DefaultGradleRunner$withPluginClasspath$0.call(Unknown Source)
  at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
  at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
  at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
  at com.huawei.odmf.codegen.gradle.TestOdmfCodegenPlugin.testPlugin(TestOdmfCodegenPlugin.groovy:40)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:497)
  at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
  at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
  at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
  at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
  at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
  at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
  at org.junit.rules.RunRules.evaluate(RunRules.java:20)
  at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
  at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
  at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
  at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
  at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
  at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
  at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
  at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
  at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
  at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
  at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
  at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:51)
  at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
  at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)

What am I missing?

Opia

As far as I remember you don't need to call withPluginClasspath() explicitly. Try without calling it explicitly. If it works, tell me, then I make this an answer. – Elder

@Elder Unfortunately, it doesn't: "Plugin [id: 'com.huawei.odmf'] was not found in any of the following sources" – Opia

@Elder It doesn't work from command line either, but it turns out that the version which does call withPluginClasspath() does. – Opia

Ah, ok, interesting. Unfortunately the repo of that plugin is deleted or private. – Elder

Yes, that's unfortunate. I also can't switch to 4.0 and check if the problem is still there, so I don't know if I should report the bug to Gradle. – Opia

@Elder Even weirder, the plugin is actually intended to fix a different bug which was fixed in 2.14: discuss.gradle.org/t/… – Opia

Same problem in 5.4.1 – Pompeii

Answer 1

plugins {
    id "org.jetbrains.gradle.plugin.idea-ext" version "0.4.2"
}

task fixIdeaPluginClasspath {
    doFirst {
        configure(tasks.pluginUnderTestMetadata) {
            def ideaClassesPath = project.buildDir.toPath().resolveSibling("out").resolve("production")
            def newClasspath = pluginClasspath as List
            newClasspath.add(0, ideaClassesPath)
            pluginClasspath.setFrom(newClasspath)
        }
    }
}
pluginUnderTestMetadata.mustRunAfter(fixIdeaPluginClasspath)

idea.project.settings {
    taskTriggers {
        beforeBuild fixIdeaPluginClasspath, pluginUnderTestMetadata
    }
}

This works with IDEA 2019.1 (and may very well work with earlier versions as well).

This utilizes JetBrains' own gradle plugin for configuring IDEA settings to execute both pluginUnderTestMetadata and the custom fixIdeaPluginClasspath before each build (the latter will only run from within the IDEA, not when running native gradle).

The first task -- pluginUnderTestMetadata -- makes sure to create the properties file, and is executed by native Gradle as well.

The second task -- fixIdeaPluginClasspath -- fixes another bug with how IDEA executes tests: The classpath generated by pluginUnderTestMetadata will only contain reference to the "$projectDir/build" directory, which is not where IDEA outputs its compiled classes; hence, you will not see changes you've made in the plugin code that were compiled by IDEA, but only those compiled by native gradle. What it does then is to prepend the IDEA classes directory to the classpath. At first I also tried removing the "$projectDir/build" reference(s), but gradle then didn't like it complaining about plugin namespace problems (too voodoo for me).

Thanks to @krzychu for pointing out pluginUnderTestMetadata (on an earlier answer's comment).

Defalcate

I've also opened a bug for them: youtrack.jetbrains.com/issue/IDEA-206158 – Defalcate

Answer 2

After determining the problem was in IDEA only, I've found https://plugins.gradle.org/plugin/com.palantir.idea-test-fix and added

plugins {
  id "com.palantir.idea-test-fix" version "0.1.0"
}

to the beginning of build.gradle for the plugin subproject. It fixes the problem.

In this post (Russian), I found another solution: in Settings-> Build-> Build Tools->Gradle->Runner, select Gradle Test Runner instead of Platform Test Runner, then delete the test's run/debug configuration before running it again.

Opia

I had the same problem with IntelliJ 2017.2. I couldn't get the com.palantir.idea-test-fix to work, but the trick to change to the gradle test runner worked great! – Tangled

BTW, this issue occurred for me while trying to use the vanilla gradle plugin example documented here: docs.gradle.org/current/userguide/… – Tangled

Problem still occurs with IDEA 2017.2.5 and Gradle 4.3. Changing the runner and deleting the run configurations fixed it, thanks! – Almaraz

Described solutions didn't solve the problem for me, but running ./gradlew pluginUnderTestMetadata did. – Lassitude

In IDEA 2020.2 only way to fix it was to go in Settings>Build>Gradle>YOUR_PROJECT>Run tests using and select Gradle (Default) – Digestion

Answer 3

In Gradle 4, I was trying to use gradleTestKit() and got this error.

• adding the below plugin fixed it.

plugins {
    id 'java-gradle-plugin'
}

Shutin

Agreed, I got the same error when I used gradleTestKit without the above line, when I searched google, I found your question. I know StackOverflow doesn't like irrelevant answers so we really can't answer side questions :( – Shutin

This is indeed the correct answer: docs.gradle.org/current/userguide/… – Broughton

Answer 4

This might be happening due to pluginUnderTestMetadata not being executed before the tests. That's what generates the metadata, which your execution is complaining about.

A way to fix this is to add that task as a requirement for your tests:

tasks.withType<Test> {
    dependsOn("pluginUnderTestMetadata")
}

Some test executors might not catch this before executing tests. As a workaround you can depend on that when compiling your test classes:

tasks.named("testClasses") {
    dependsOn("pluginUnderTestMetadata")
}

Attrition

Answer 5

For build.gradle.kts (kotlin) you can use:

    val createClasspathManifest = tasks.create("createClasspathManifest") {
        val outputDir = file("$buildDir/$name")
    
        inputs.files(sourceSets.main.get().runtimeClasspath)
        inputs.files(project(":core").sourceSets.main.get().runtimeClasspath)
    
        outputs.dir(outputDir)
        doLast {
            outputDir.mkdirs()
    
            val joined = sourceSets.main.get().runtimeClasspath.joinToString("\n") +
                    project(":core").sourceSets.main.get().runtimeClasspath.joinToString("\n")
    
            file("$outputDir/plugin-classpath.txt").writeText(joined)
        }
    }
    
    dependencies {
        testApi(files(createClasspathManifest))
    }

Weeper