Guide for Testing Gradle Scripts

Question

What are the best practices for testing Gradle Scripts?

I currently unit test my ant scripts with antunit, but I'm looking to migrate to Gradle. I can only find articles on testing Java code from Gradle or Groovy, but nothing on testing the Gradle tasks that I create or testing Groovy in general. Is there an equivalent of antunit for Gradle? Has anyone played with a BDD framework (like cucumber) to test Gradle scripts?

For instance, I currently have the following Ant Target

<target name="dist-bin" depends="build" description="creates a zip distribution of the current build">
    <zip destfile="build/TIBant-bin.zip">
        <zipfileset dir="src/ant" includes="**" />
        <zipfileset dir="test" includes="**" prefix="test" />
        <zipfileset dir="build" includes="TIBant.jar" />
        <zipfileset dir="build" includes="TIBant-*.html" />
        <zipfileset dir="build/examples/XMLtoProperties"
                    includes="XMLtoProperties.html"
                    prefix="examples/XMLtoProperties" />
        <zipfileset dir="lib" includes="**" prefix="lib" />
        <zipfileset dir="src/xslt" includes="**" excludes="test/**,userguide.xslt" prefix="lib/xslt" />
        <zipfileset dir="." includes="copyright.html,LICENSE.txt" />
        <zipfileset dir="examples"
                    includes="**"
                    excludes="**/build/**,**/config/default.properties"
                    prefix="examples" />
    </zip>
</target>

As you can imagine, it's pretty easy to break this when refactoring the project, so I have the following antunit test to check it.

<target name="test-dist-bin">
    <delete file="build/TIBant-bin.zip" />
    <au:assertFalse message="Bin dist still present">
        <available file="build/TIBant-bin.zip" />
    </au:assertFalse>
    <antcall target="dist-bin" />
    <au:assertTrue message="Bin dist not created">
        <available file="build/TIBant-bin.zip" />
    </au:assertTrue>
    <delete dir="build/testBinDist" />
    <au:assertFalse message="TestBinDist still present">
        <available file="build/testBinDist" />
    </au:assertFalse>
    <mkdir dir="build/testBinDist" />
    <unzip src="build/TIBant-bin.zip" dest="build/testBinDist" />
    <au:assertFalse message="config dir present">
        <available file="build/testBinDist/config/default.properties" />
    </au:assertFalse>
    <au:assertTrue message="Ant Macros missing">
        <available file="build/testBinDist/tibant.xml" />
    </au:assertTrue>
    <au:assertTrue message="Engine Stopper Jar missing">
        <available file="build/testBinDist/TIBant.jar" />
    </au:assertTrue>
    <au:assertTrue message="Ant-contrib-missing">
        <available file="build/testBinDist/lib/ant-contrib-1.0b3.jar" />
    </au:assertTrue>
    <au:assertTrue message="ant-unit missing">
        <available file="build/testBinDist/lib/ant-antunit-1.2.jar" />
    </au:assertTrue>
    <au:assertTrue message="Copyright missing">
        <available file="build/testBinDist/copyright.html" />
    </au:assertTrue>
    <au:assertTrue message="License missing">
        <available file="build/testBinDist/LICENSE.txt" />
    </au:assertTrue>
    <au:assertFalse message="Src present">
        <available file="build/testBinDist/src/java/org/windyroad/tibant/EngineStopper.jar" />
    </au:assertFalse>
    <au:assertTrue message="example missing">
        <available file="build/testBinDist/examples/SimpleProject/src/bw/example/Build/example.archive" />
    </au:assertTrue>
    <au:assertFalse message="example has build files">
        <available file="build/testBinDist/examples/SimpleProject/build/*" />
    </au:assertFalse>
    <au:assertFalse message="example has default config file">
        <available file="build/testBinDist/examples/SimpleProject/config/default.properties" />
    </au:assertFalse>
    <property name="doc.file"
              location="build/testBinDist/TIBant-User-Guide.html" />
    <au:assertTrue message="doc missing: ${doc.file}">
        <available file="${doc.file}" />
    </au:assertTrue>
    <au:assertTrue message="xslt missing">
        <available file="build/testBinDist/lib/xslt/configure-ear.xslt" />
    </au:assertTrue>
    <subant target="run-quick-tests">
        <fileset dir="build/testBinDist" includes="build.xml" />
    </subant>
</target>

which is called by the following snippet to produce a nice xml report

                <au:antunit failonerror="@{failonerror}">
                    <propertyset>
                        <propertyref name="config.filename" />
                    </propertyset>
                    <path>
                        <pathelement location="@{antunit}" />
                    </path>
                    <au:plainlistener logLevel="info" />
                    <au:xmllistener todir="build" loglevel="info" />
                </au:antunit>

I understand how to migrate dist-bin to gradle, but I'm not sure what's the right way to migrate test-dist-bin and the au:antunit call.

Homothallic

Answer 1

Gradle 3.x test Toolkit available! Please, check out userguide here: https://docs.gradle.org/current/userguide/test_kit.html

The correctness of the logic can then be verified by asserting the following, potentially in combination:

• The build's output;
• The build's logging (i.e. console output);
• The set of tasks executed by the build and their results (e.g. FAILED, UP-TO-DATE etc.).

copy-pasted example:

import org.gradle.testkit.runner.BuildResult;
import org.gradle.testkit.runner.GradleRunner;
import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Collections;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;

import static org.gradle.testkit.runner.TaskOutcome.*;

public class BuildLogicFunctionalTest {
    @Rule public final TemporaryFolder testProjectDir = new TemporaryFolder();
    private File buildFile;

    @Before
    public void setup() throws IOException {
        buildFile = testProjectDir.newFile("build.gradle");
    }

    @Test
    public void testHelloWorldTask() throws IOException {
        String buildFileContent = "task helloWorld {" +
                                  "    doLast {" +
                                  "        println 'Hello world!'" +
                                  "    }" +
                                  "}";
        writeFile(buildFile, buildFileContent);

        BuildResult result = GradleRunner.create()
            .withProjectDir(testProjectDir.getRoot())
            .withArguments("helloWorld")
            .build();

        assertTrue(result.getOutput().contains("Hello world!"));
        assertEquals(result.task(":helloWorld").getOutcome(), SUCCESS);
    }

    private void writeFile(File destination, String content) throws IOException {
        BufferedWriter output = null;
        try {
            output = new BufferedWriter(new FileWriter(destination));
            output.write(content);
        } finally {
            if (output != null) {
                output.close();
            }
        }
    }
}

Childbed

Answer 2

I think what tom meant is a way to test his own written gradle tasks, right? If you have written a custom gradle task by extending DefaultTask and you put it in the buildSrc folder of your project, you can add a junit/spock/whatever based test class to test your task implementation. Gradles own build provides a good example for that. have a look at

https://github.com/gradle/gradle/blob/master/buildSrc/src/test/groovy/org/gradle/build/docs/dsl/source/ExtractDslMetaDataTaskTest.groovy

This is a spock specification, that tests the ExtractDslMetaDataTask which was specifically written for griddles own build. The ExtractDslMetaDataTask is located at:

https://github.com/gradle/gradle/blob/master/buildSrc/src/main/groovy/org/gradle/build/docs/dsl/source/ExtractDslMetaDataTask.groovy

To add assertions to your build script "adhoc tasks" like your example above you can use the groovy power assertion.

an example: if you have a a very simple task like this in your script:

task writeFoo << {
    file("$buildDir/foo.txt").text =  "bar"
}

you can either modify the task itself to add an assertion:

task writeFoo << {
    file("$buildDir/foo.txt").text =  "bar"
    assert file("$buildDir/foo.txt).isFile()
}

or you add a dedicated test task to your script

task testWriteFoo(dependsOn: writeFoo) << {
    assert file("$buildDir/foo.txt).isFile()
    assert file("$buildDir/foo.txt).text == "bar"
}

remember that you can use the full power of the groovy language in your build scripts.

There are plans to have an integrated testing toolkit in gradle to support build script authors on testing their custom build logic. have a look at:

http://forums.gradle.org/gradle/topics/testing_toolkit_for_custom_build_logic

Fixed

No extending DefaultTask, just a plain old Gradle task (which Ant refers to as targets). Example added above. – Homothallic

I've updated my answer to show an example of using assertions in your build script – Fixed

Sigh. Thanks for the answer and the update with an example, but if asserts are all there is, then I've got to say I'm a little disappointed. – Homothallic

I have to admit, that I'm not aware of the features that antunit provides. In the snippet you listed here I just see assertions. What are you missing in detail? – Fixed

The xml output. The ability to either fail fast on the first test failure or run till completion and report the results for all the tests. Capture of log output for a given test. Fixtures (set-up and tear-down before each test). But most critically of all, automated test discovery. I can just point antunit to a set of files and it will run all the antunit tests within those files (admittedly the logic is pretty basic; any target starting with test is considered a test). – Homothallic

Answer 3

As long as you apply the plugin groovy and your tests sit under src/test/groovy there's no additional configuration needed to run them. The same is true for BDD tests with Spock for example. If you want to read more about Gradle's testing capabilities check out the book Building and Testing with Gradle. It covers testing with JUnit, TestNG, Spock, Geb and EasyB.

Novocaine