How to get Google Protobuf working in Matlab?

Question

So if one was to want to use Google protocol buffers in Matlab and using a Windows computer what would be the best way to do that since Matlab is not in the list of supported languages?

Hoopes

You should divide this post into a pair of question and answer. – Tobit

oh I'm sorry, I'm new to this and wanted to be helpful – Hoopes

This is very good answer, I'm just not sure what the question is. It would be a good idea to move what you have here to the answer section and just re-ask the question. – Example

Okay, I changed how the question answer looks, is this better? – Hoopes

Answer 1

As Cameron Lowell Palmer's answer suggests, the way to go is via Java.

Is I lost a couple of hours today on this problem, I would like to elaborate some more. I started with Cameron's answer but I had to do a couple of more steps. Essentially, I had to do all of the following:

1. Compile proto message for java, e.g. protoc --java_out=./ your_file.proto
2. Remember to install protobuf java runtime, e.g. in ubuntu: sudo apt install libprotobuf-java
3. Locate the protobuf java runtime on your system, e.g.: /usr/share/java/protobuf-3.6.1.jar; its path will be used later on. The name should always follow the pattern protobuf-version.jar or protobuf-java-version.jar, therefore locate protobuf- | grep jar$ should reduce the search space for you.
4. Add a short java file to the same directory, where your protobuf java files went (./x/y/z/MyProto.java). Use your package path instead of x/y/z. If you did not declare java package explicitly in the proto file, then protoc just used your filename as package name. Either way, you can probably check where the protoc generated files went by yourself. Contents of MyProto.java are listed below. Just replace YourProtoFileName and YourMessageName with your stuff. Note that this step is not optional, as this will not generate a simple helper class. For the life of me, Matlab would not let me use inner classes directly (in java, YourMessageName is the inner class of YourProtoFileName). But with the above helper, it was quite happy to generate them for me. Note that if you have more than one message defined in your proto file, you might need to expose more than one builder in this way. And if you only need to read protobuffers, then you might need to export just YourMessageName and not Builder.

package x.y.z;

import x.y.z.YourProtoFileName; // if you do not know it, do `ls x.y.z/*.java`

public class MyProto {
   public static YourProtoFileName.YourMessageName.Builder newBuilder() {
      return YourProtoFileName.YourMessageName.newBuilder();
   }
}

5. Check Matlab's version of java; in Matlab write version -java. In my case it was 1.8, while the default java installed on my system (java -version) is 11. I had to manually select java 1.8 for the next step, otherwise the whole thing did not work. Even worse, Matlab only produced a very nondescript error "No class x.y.z.YourProtoFileName." Thanks Matlab! You might need to install proper version first (sudo apt install openjdk-8-jdk) and then use update-java-alternatives or just locate javac for the appropriate java version on your system.
6. Compile both your java file and the one generated by protoc. Remember to point classpath at the jar file or protobuffer runtime that you found above. And remember to use your javac path. In a single command: /usr/lib/jvm/java-8-openjdk-amd64/bin/javac x.y.z/*.java -cp /usr/share/java/protobuf-3.6.1.jar. This will generate class files in ./x/y/z/.
7. Optional: pack the class files in a jar for easier distribution: jar cvf ./YourProtoFilename.jar x/y/z/*.class. Note that this command line works for me, since I've put all classes, including MyProto, in the same package. You might need to adapt it to your needs.
8. Start Matlab.

% make Matlab aware of your new classes
javaaddpath('./')
% tell Matlab where protobuf dependancy lives (use the path from step 3)
javaaddpath('/usr/share/java/protobuf-3.6.1.jar')

% test if the classes were found
methods('x.y.z.YourProtoFilename.YourMessageName')
% if methods are listed then you are good to go

% use the helper form step 4
b = x.y.z.MyProto.newBuilder();
% now you have a builder you can use to build your protobuf message

9. See Google's protobuf java primer to go on from here, as using java in Matlab is fairly straight forward: just write java statements. As long as they are simple statements :)

The same procedure works on octave too. With a bit different syntax for java inside octave. And octave was less picky about the java version in my case. YMMV

Ooze

Since I posted the above instructions I've learned it is indeed possible to use inner java classes in Matlab. But the more verbose and obscure syntax has to be used: result = javaMethod('methodName', 'x.y.z.OuterClass$InnerClass'); Therefore item 4 can be left out, and the last step of item 8 can be simplified to: b = javaMethod('newBuilder', 'x.y.z.YourProtoFileName$YourMessageName'); – Ooze

Answer 2

I haven't seen an answer on this and I thought the solution was a bit obscure so I am going to post a how to for matlab_out using the protoc.exe

A how to for google protocol buffer matlab out, this is using resources from the internet I will also include a zip file containing all this already done.

1. Unzip protobuf-‘version#’.zip (looks like: protobuf-#.#.#)

2. Open file protobuf-#.#.# -> src

3. Choose your favorite editor (Notepad++ is good) and change Makefile.am (do not include quotes)

a. Under “nobase_include_HEADERS =” and below “$(GZHEADERS)”

add the line “farsounder/protobuf/compiler/matlab/matlab_generator.h \” (Note the back slash)

b. Under “libprotoc_la_LIBADD = $(PTHREAD_LIBS) libprotobuf.la”

add lines
“farsounder/protobuf/compiler/matlab/matlab_generator.cc \”

“farsounder/protobuf/compiler/matlab/matlab_plugin.cc \”

c. Save the file and exit out

4. While still in the src directory go into ->google ->protobuf->compiler and change main.cc

a. Under “#include <google/protobuf/compiler/java/java_generator.h>”

add the line “#include <farsounder/protobuf/compiler/matlab/matlab_generator.h>”

b. In main function add the lines “// Proto2 Matlab farsounder::protobuf::compiler::matlab::MatlabGenerator matlab_generator; cli.RegisterGenerator("--matlab_out", &matlab_generator, "Generate Matlab M files.");”

c. Save the file and exit out

5. Unzip protobuf-matlab
6. Open file protobuf-matlab ->src and copy the farsounder directory to protobuf-#.#.# -> src directory
7. Go back to protobuf-#.#.# directory and into -> vsprojects and open protobuf.sln in Visual Studio, I believe any version should work
8. There should be a popup that wants to convert the solution file to a more updated version, go ahead and do that
9. If you do not see solutions explorer open it up using ctrl+alt+L
10. Open a new windows explorer and go into protobuf-#.#.# ->src->farsounder->protobuf->compiler->matlab, now in visual studio using the solutions explorer expand the libprotoc->Header Files, now in windows explorer copy the matlab_generator.h file and paste it into the Header Files directory
11. Still in Solutions explorer go into the Source Files directory and copy in the matlab_generator.cc
12. Contract libprotoc and right click on protoc and click on properties
13. Under Configuration Properties->Linker->General, Edit Enable Incremental Linking to be No
14. Under Configuration Properties->Linker->Input, Edit Additional Dependencies a. Add the lines “Release\libprotobuf.lib” and underneath “Release\libprotoc.lib” b. Click OK on the Additional Dependencies window and OK on the protoc Property Page
15. Change the build type to Release
16. Right-click on libprotobuf and select build, once completed right-click on libprotoc and select build
17. Once completed right-click on protoc and select build, it should provide you with an protoc.exe under protobuf-#.#.# ->vsprojects->Release, this will now allow you to create matlab .m files by using the matlab_out command

Help to find protobuf-#.#.# and the protobuf-matlab:

Follow these links:

https://code.google.com/p/protobuf-matlab/source/browse/
download the zip file

https://github.com/google/protobuf/releases

download the source code

Extra help, for using protoc.exe

1. Copy and paste the created protoc.exe to where your .proto file is
2. Run cmd and change the directory to where your protoc.exe and .proto file is
3. Run this command (lets pretend the name of my .proto file is “afunprotofile”) “protoc –matlab_out=./ -I./ afunprotofile.proto”

Hoopes

This instructions (with my edit) worked for me. I ignored the steps from number 7 to 17, and compiled it by running sh autogen.sh; sh configure; make – Hewet

Given that my edit has been rejected, step 3.b for me to work should say: Under "libprotoc_la_SOURCES = " – Hewet

For protobuf v3 the answer above no longer works. To use the current C++ API use github.com/BraunPhilipp/protobuf-matlab (forked from farsounders code). – Rating

@PhilippBraun do you mind telling how to build this source for windows environment? – Lunt

Answer 3

The easiest way is to use Java

The FarSounder code was nice, but it is quite old and unmaintained. The easiest way to produce Matlab compatible code is to just use the Java version of Protobuf. This solution should work on any platform that supports Matlab and Java.

The Steps

1. Create a .proto file
2. Process the file with the protoc compiler and output Java source
3. Using IntelliJ or other tool produce a JAR file that includes the dependencies
4. Add the JAR file to the Matlab classpath. edit('classpath.txt')
5. Restart Matlab

The protobuf runtime dependencies

I include these in a single JAR file output of the Protobuf and the two runtime libraries.

• protobuf-java-3.3.0.jar
• protobuf-java-util-3.3.0.jar

A sample

I wrote a simple Java wrapper class to hide the MyProtobuf.Builder return type from Matlab that I added to the JAR file

public class MyProtobufWrapperWrapper {
    public static MyProtobuf.Builder newBuilder()
    {
        return MyProtobuf.newBuilder();
    }
}

In Matlab

p = com.cameronpalmer.MyProtobufWrapper.newBuilder();
p.setIdentifier(java.util.UUID.randomUUID().toString());
p.setTimestampMilliseconds(timestamp);
p.setAngleRadians(0);
p.addAllChannelSamples(channel_vector);

planeWaveBuilt = p.build();
byteArray = planeWaveBuilt.toByteArray();

Laveta

Could you please explain what is planeWave in your code? – Microcyte

planeWave was just the name I used in my code, because that was what the data contained. – Laveta

Note: since matlab might include some protobuf of unknown version, user needs to add runtime libraries in classpath BEFORE default matlab protobuf library. Otherwise things might not work – Egret

@Egret github.com/grpc/grpc-java/issues/5258 this issue says Java-Matlab grpc integration is not possible now. Is it possible? ? – Lunt

@Lunt I was able to spin it up – Egret

@Egret im trying to call client from matlab R2021b, but getting the same issue. mediafire.com/file/txh8i91pz5p2fmq/java_sample.zip/file If possible could you please look into this, when you have time? i have attached the classpath, matlab test file as well. Im new to both java and Matlab and exploring this for a POC. Thanks – Lunt

@Egret #70593084 fyi – Lunt

Answer 4

@WPFUser,
We followed below steps to build protoc for matlab.

1) git clone https://github.com/protocolbuffers/protobuf.git to protobuf-main folder
2) git submodule update --init --recursive
3) git clone https://github.com/farsounder/protobuf-matlab.git - protobuf_matlab folder
4) Copied src folder from protobuf_matlab repo to protobuf-main/src
5) Updated cmake/libprotoc.cmake to include matlab_generator.cc and matlab_generator.h files
6) Updated cmake/extract_includes.bat.in to include matlab_generator.h file
7) \src\google\protobuf\compiler\main.cc to include matlab_generator

Diffract