Working with Protocol Buffers and internal data models

Question

I have an existing internal data model for a Picture, as follows:

package test.model;
public class Picture {

  private int height, width;
  private Format format;

  public enum Format {
    JPEG, BMP, GIF
  }

  // Constructor, getters and setters, hashCode, equals, toString etc.
}

I now want to serialize it using protocol buffers. I've written a Picture.proto file that mirrors the fields of the Picture class and compiled the code under the test.model.protobuf package with a classname of PictureProtoBuf:

package test.model.protobuf;

option java_package = "test.model.protobuf";
option java_outer_classname = "PictureProtoBuf";

message Picture {
  enum Format {
    JPEG = 1;
    BMP = 2;
    GIF = 3;
  }
  required uint32 width = 1;
  required uint32 height = 2;
  required Format format = 3;
}

Now I am now assuming that if I have a Picture that I want to serialize and send somewhere I have to create a PictureProtoBuf object and map all the fields across, like so:

Picture p = new Picture(100, 200, Picture.JPEG);
PictureProtoBuf.Picture.Builder output = PictureProtoBuf.Picture.newBuilder();
output.setHeight(p.getHeight());
output.setWidth(p.getWidth());

I'm coming unstuck when I have an enumeration in my data model. The ugly way that I'm using right now is:

output.setFormat(PictureProtoBuf.Picture.Format.valueOf(p.getFormat().name());

However, this is prone to breakage and relies on the enumeration name being consistent between my internal data model and the protocol buffer data model (which isn't a great assumption as enumeration names within .proto files need to be unique). I can see me having to hand-craft switch statements on enumerations if the .name() call from the internal model doesn't match the protobuf-generated enumeration name.

I guess my question is whether I'm going about this the right way? Am I supposed to scrap my internal data model (test.model.Picture) in favour of the protobuf-generated one (test.model.protobuf.PictureProtoBuf)? If so, how can I implement some of the niceties that I have done in my internal data model (e.g. hashCode(), equals(Object), toString(), etc.)?

Quire

I haven't tried it (purely since I'm primarily a .NET person), but I believe protostuff allows you to keep working with your existing model.. – Katleen

@MarcGravell - Thanks for your suggestion. Your hunch was correct; protostuff does exactly what I was after but retains protocol buffers on the back-end (haven't yet tested its compatibility with the Google protobuf library, though). – Quire

Answer 1

Although the existing answers are good, I decided to go a bit further with Marc Gravell's suggestion to look into protostuff.

You can use the protostuff runtime module along with the dynamic ObjectSchema to create schemas at runtime for your internal data model

My code now reduces to:

// Do this once
private static Schema<Picture> schema = RuntimeSchema.getSchema(Picture.class);
private static final LinkedBuffer buffer = LinkedBuffer.allocate(DEFAULT_BUFFER_SIZE);

// For each Picture you want to serialize...
Picture p = new Picture(100, 200, Picture.JPEG);
byte[] result = ProtobufIOUtil.toByteArray(p, schema, buffer);
buffer.clear();
return result;

This is a great improvement over the Google protobuf library (see my question) when you have lots and lots of attributes in your internal data model. There is also no speed penalty that I can detect (with my use cases, anyway!)

Quire

Answer 2

If you have control over your internal data model, you could modify test.model.Picture so that the enum values know their corresponding protobuf equivalent, probably passing in the correspondence to your enum constructors.

For example, using Guava's BiMap (bidirectional map with unique values), we get something like

enum ProtoEnum { // we don't control this
  ENUM1, ENUM2, ENUM3;
}

enum MyEnum {
  ONE(ProtoEnum.ENUM1), TWO(ProtoEnum.ENUM2), THREE(ProtoEnum.ENUM3);

  static final ImmutableBiMap<MyEnum, ProtoEnum> CORRESPONDENCE;

  static {
    ImmutableBiMap.Builder<ProtoEnum, MyEnum> builder = ImmutableBiMap.builder();
    for (MyEnum x : MyEnum.values()) {
      builder.put(x.corresponding, x);
    }
    CORRESPONDENCE = builder.build();
  }

  private final ProtoEnum corresponding;

  private MyEnum(ProtoEnum corresponding) {
    this.corresponding = corresponding;
  }
}

and then if we want to look up the MyEnum corresponding to a ProtoEnum, we just do MyEnum.CORRESPONDENCE.get(protoEnum), and to go the other way, we just do MyEnum.CORRESPONDENCE.inverse().get(myEnum) or myEnum.getCorresponding().

Violation

Thanks for your answer. I think I grasp the concept but I'm not sure how I would implement it. Would you mind stubbing out some code? – Quire

Answer 3

One way is to only keep the generated enum:

package test.model;
public class Picture {

  private int height, width;
  private PictureProtoBuf.Picture.Format format;

 // Constructor, getters and setters, hashCode, equals, toString etc.
}

I've used this a few times, it may or may not make sense in your case. Using the protobuf generated classes as you data model (or extending them to add functionality), is never recommended, though.

Fomalhaut