Apache AVRO with Rest
Asked Answered
C

2

11

I am evaluating using Apache AVRO for my Jersey REST services. I am using Springboot with Jersey REST.

Currently I am accepting JSON as input which are converted to Java Pojos using the Jackson object mapper.

I have looked in different places but I cannot find any example that is using Apache AVRO with a Jersey end point.

I have found this Github repository (https://github.com/FasterXML/jackson-dataformats-binary/) which has Apache AVRO plugin.

I still cannot find any good example as how to integrate this. Has anyone used Apache AVRO with Jersey? If yes, is there any example I can use?

Cranwell answered 26/8, 2017 at 18:23 Comment(2)
Some useful links : github.com/keedio/avro-schema-repo/tree/master/client/src/main/… pastebin.com/DNS8xntGArsyvarsy
Did you take a look at the test folder: github.com/FasterXML/jackson-dataformats-binary/tree/master/… ?Schreck
S
13

To start , two things need to happen:

  1. You need to develop a custom ObjectMapper after the fashion of the Avro schema format
  2. You need to supply that custom ObjectMapper to Jersey.

That should look something like this:

@Provider
public class AvroMapperProvider implements ContextResolver<ObjectMapper> {

    final AvroMapper avroMapper = new AvroMapper();

    @Override
    public ObjectMapper getContext(Class<?> type) {
        return avroMapper;
    }
}

Configure your application to use Jackson as the message handler:

public class MyApplication extends ResourceConfig {
    public MyApplication() {
         super(JacksonFeature.class,AvroMapperProvider.class);
    }
}

Alternatively, you can implement a custom MessageBodyReader and MessageBodyWriter that allows you to directly process the payloads on the way in and out:

public class AvroMessageReader implements MessageBodyReader<Person> {

    AvroSchema schema;

    final AvroMapper avroMapper = new AvroMapper();

    public AvroMessageReader(){
        schema = avroMapper.schemaFor(Person.class); //generates an Avro schema from the POJO class.
    }

    @Override
    public boolean isReadable(Class<?> type, Type type1, Annotation[] antns, MediaType mt) {
        return type == Person.class; //determines that this reader can handle the Person class.
    }

    @Override
    public Person readFrom(Class<Person> type, Type type1, Annotation[] antns, MediaType mt, MultivaluedMap<String, String> mm, InputStream in) throws IOException, WebApplicationException {
        return avroMapper.reader(schema).readValue(in);
    }

}

Here, we generate an avro schema from a hypothetical Person class. The JAX-RS runtime will select this reader based on the response from isReadable.

You can then inject the MessageBodyWorkers component into your service implementation class:

@Path("app")
public static class BodyReaderTest{

    @Context
    private MessageBodyWorkers workers;

    @POST
    @Produces("avro/binary")
    @Consumes("avro/binary")
    public String processMessage() {

        workers.getMessageBodyReader(Person.class, Person.class, new Annotation[]{}, MediaType.APPLICATION_JSON_TYPE);
    }
 }

To answer your last comment: Setting the mime type on your handler to the recommended avro/binary ought to do it.

Schreibman answered 4/9, 2017 at 8:50 Comment(5)
Thanks. What about the produce and consume in the rest end point. I have @Produces(MediaType.APPLICATION_JSON) @Consumes(MediaType.APPLICATION_JSON)Cranwell
Not sure I understand your question, but the Provider example I've provided here doesn't interfere with the service implementation class @Makky. See if my update provides more claritySchreibman
I think I understand what you were referring to now: setting the mime type? See the update @CranwellSchreibman
@Schreibman For serializing do we need to provide an implmenation of MessageBodyWriter, same as the MessageBodyReader you have above? I didn't get the part about injecting MessageBodyWorkers in the Service class, can I inject it in my controller so that it automaticaly marshalls and unmarshalls POJO(and registered Avro schemas) to Avro?Factual
@Schreibman If method 1, how do I setup Spring Boot application to use Jackson as the message handler? I added the AvroMapperProvider. And will this allow for automatic serialization/deserialization? Or should I choose method 2 MessageBodyWorker for it?Factual
L
2

There is a comprehensive demo (which I wrote) on how to use avro in a JAX-RS REST service at. The JAX-RS message body readers and writers for avro are implemented at and they do support avro binary, json, idiomatic json, csv where applicable. They do provide full support for schema evolution and projections (via the http accept header). There is a list of articles that explain in more detail the demonstrated concepts at. Also this demo project runs live on GKE at, you can browse at the openapi at. Avro is being used in the project for everything, for logs, for metrics, for profiling.

Lelialelith answered 4/2, 2020 at 10:52 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.