Convert json to dynamically generated protobuf in Java

Question

Given the following json response:

{
    "id" : "123456",
    "name" : "John Doe",
    "email" : "[email protected]"
}

And the following user.proto file:

message User {
    string id = 1;
    string name = 2;
    string email = 3;
}

I would like to have the possibility to dynamically create the protobuf message class (compile a .proto at runtime), so that if the json response gets enhanced with a field "phone" : "[phone]" I could just upload a new version of the protobuf file to contain string phone = 4 and get that field exposed in the protobuf response, without a service restart.

If I were to pull these classes from a hat, I would like to be able to write something along the following code.

import com.googlecode.protobuf.format.JsonFormat;
import com.googlecode.protobuf.Message;
import org.apache.commons.io.FileUtils;

...

public Message convertToProto(InputStream jsonInputStream){
    // get the latest user.proto file
    String userProtoFile = FileUtils.readFileToString("user.proto");

    Message userProtoMessage = com.acme.ProtobufUtils.compile(userProtoFile);
    Message.Builder builder = userProtoMessage.newBuilderForType(); 
    new JsonFormat().merge(jsonInputStream, Charset.forName("UTF-8"), builder);
    return builder.build();
}

Is there an existing com.acme.ProtobufUtils.compile(...) method? Or how to implement one? Running a protoc + load class seems overkill, but I'm willing to use it if no other option...

Hausfrau

Answer 1

You cannot compile the .proto file (at least not in Java), however you can pre-compile the .proto into a descriptor .desc

protoc --descriptor_set_out=user.desc user.proto

and then use the DynamicMessage's parser:

DynamicMessage.parseFrom(Descriptors.Descriptor type, byte[] data)

Source: google groups thread

Carmagnole

Just wanted to double check, if the byte[] data contains more fields than whats initially described in the .proto file would the parsing fail? – Woodcutter