How to generate Java classes from Avro schemas as part of a Gradle build?

Question

For Maven there is an official Avro plugin to generate Java classes from Avro schemas.

However, for Gradle there exists no official plugin.

There is davidmc24/gradle-avro-plugin, but it is no longer maintained and it is looking for a maintainer.

How can I generate Java classes from Avro schemas as part of a Gradle build?

Watertight

The gradle avro plugin apparently found a new maintainer in the form of the Apache Avro project according to this post of davicmc24: github.com/davidmc24/gradle-avro-plugin/discussions/… – Havoc

Answer 1

I have now created a simple Gradle task that generates the Avro Java classes.

import org.apache.avro.tool.SpecificCompilerTool

buildscript {
    dependencies {
        // Add the Avro code generation to the build dependencies so that it can be used in a Gradle task.
        classpath group: 'org.apache.avro', name: 'avro-tools', version: '1.11.1'
    }
}

plugins {
    // some project plugins
    id 'application'
}

def avroSchemasDir = "src/main/avro"
def avroCodeGenerationDir = "build/generated-main-avro-java"

// Add the generated Avro Java code to the Gradle source files.
sourceSets.main.java.srcDirs += [avroCodeGenerationDir]

dependencies {
    // some project dependencies
}

tasks.register('avroCodeGeneration') {
    // Define the task inputs and outputs for the Gradle up-to-date checks.
    inputs.dir(avroSchemasDir)
    outputs.dir(avroCodeGenerationDir)
    // The Avro code generation logs to the standard streams. Redirect the standard streams to the Gradle log.
    logging.captureStandardOutput(LogLevel.INFO);
    logging.captureStandardError(LogLevel.ERROR)
    doLast {
        // Run the Avro code generation.
        new SpecificCompilerTool().run(System.in, System.out, System.err, List.of(
                "-encoding", "UTF-8",
                "-string",
                "-fieldVisibility", "private",
                "-noSetters",
                "schema", "$projectDir/$avroSchemasDir".toString(), "$projectDir/$avroCodeGenerationDir".toString()
        ))
    }
}

tasks.withType(JavaCompile).configureEach {
    // Make Java compilation tasks depend on the Avro code generation task.
    dependsOn('avroCodeGeneration')
}

This task can be added to any Gradle build script.

Watertight

how did you solve 'unable to resolve class SpecificCompilerTool' ? – Wojak

Answer 2

The plugin project README was updated in December 2023 to state that the Apache Avro project is taking over the Gradle plugin.

This project is no longer maintained. Its code has been donated to the Apache Avro project, which will be handling releases going forward.

It is probably therefore reasonable to treat the plugin like an official plugin, even though as of this writing (December 2023) there hasn't been an Apache Avro release of the plugin.

---

To specifically answer the question about how to generate the Java classes from the Gradle build, the usage section of the plugin project README has details with an example:

Usage

Add the following to your build files. Substitute the desired version based on CHANGES.md.

settings.gradle:

pluginManagement {
    repositories {
        gradlePluginPortal()
        mavenCentral()
    }
}

build.gradle:

plugins {
    id "com.github.davidmc24.gradle.plugin.avro" version "VERSION"
}

Additionally, ensure that you have an implementation dependency on Avro, such as:

repositories {
    mavenCentral()
}
dependencies {
    implementation "org.apache.avro:avro:1.11.0"
}

If you now run gradle build, Java classes will be compiled from Avro files in src/main/avro. Actually, it will attempt to process an "avro" directory in every SourceSet (main, test, etc.)

Bette

Answer 3

I ran into this problem too while my demand was to generate dto into a certain package in src/main/java (Springboot 3.3.2, java21 graalvm, gradle8.9 ).

{
  "type": "record",
  "name": "DemoDTO",
  "namespace": "com.****.sp.domain.dto",
  "fields":[
    {"name": "param0", "type": "string"},
    {"name": "param1", "type": "string"},
    {"name": "param2", "type": "string"}
  ]
}

Configurations from davidmc24/gradle-avro-plugin may still adds up, though it truly is not maintained anymore =/ (seems like it should take a while before a apache release)

Here is my solution:

1. Create avro config package under src/main/avro (default file path referenced when generating).
2. Add the plugin into build.gradle that allows you to do some custom stuff but don't keep the basic plugin for it would generate avro classes as well in the default avro output directory($buildDir/generated-main-avro-java) when building, which finally causes duplicate class error during compileJava.

plugins {
    id 'java'
    id 'org.springframework.boot' version '3.3.2'
    id 'io.spring.dependency-management' version '1.1.6'
    id 'org.graalvm.buildtools.native' version '0.10.2'
    id 'com.github.davidmc24.gradle.plugin.avro-base' version '1.3.0'
//  id 'com.github.davidmc24.gradle.plugin.avro' version '1.3.0'
}

3. Simply copy-paste those configuration then let the gradle build

import com.github.davidmc24.gradle.plugin.avro.GenerateAvroJavaTask

apply plugin: "java"
apply plugin: "com.github.davidmc24.gradle.plugin.avro-base"

def generateAvro = tasks.register("generateAvro", GenerateAvroJavaTask) {
    source("src/main/avro")
    outputDir = file("src/main/java")
}

tasks.named("compileJava").configure {
    source(generateAvro)
}

4. Build successful =u=

Dissolute