How to get type contained by protobuf's RepeatedCompositeContainer or RepeatedScalarContainer in python3?

Question

I'm writing a Python app for serializing and sending protobuf3 messages. I'd like to make some sort of interactive UI that allows to choose a message and assign it on the fly. I've got a pretty big set of those messages, thus I don't want to make a get function for every message, but make one that will work with all of them.

To get all message fields, I can simply get all the message's attributes and choose those that are its fields, which is easy. Then, to know what type is the attribute, I use type(getattr(my_message, current_field)). And now there is the problem. Suppose those are my messages:

message myMess1 {
    //some fields
}

message myMess2 {
    string some_string = 1
    repeated myMess1 myMess1Field = 2
}

Now, there is no problem with assigning some_string field.

type(getattr(myMess2Instance, someStringFieldName)) returns string, so I know to feed it with string.

But what to do with the repeated myMess1 field? type(getattr(MyMess2Instance, myMess1FieldName)) actually returns google.protobuf.pyext._message.RepeatedCompositeContainer, which says nothing about what type is contained in it. How can I get it?

Terror

Answer 1

I had a similar Protobuf parsing issue, here is what I do "hope it helps":

Say we have this response message:

>>> msg
path {
}
path {
  value: "Arts & Entertainment"
}
path {
  value: "Comics & Animation"
}

Convert it to dictionary:

>>> from google.protobuf.json_format import MessageToDict
>>> d = MessageToDict(msg)
{'path': ['', 'Arts & Entertainment', 'Comics & Animation']}

Join the dictionary list of values as a string:

>>> " ".join(d["path"]).strip()
'Arts & Entertainment Comics & Animation'

Colley

Answer 2

This really perplexed me for a while since no documentation seems to be generated. The key is to see that RepeatedCompositeContainer behaves like a list. You can iterate on the contents, which in the example would have the myMess1 generated type. E.g:

for myMess1Element in myMess2Instance.myMess1FieldName:
    print(f"{myMess1Element.some_field=}")
    print(f"{myMess1Element.other_field=}")

If you somehow knew the number of repeats of that message ahead of time, you could index a particular one. E.g:

print(f"{myMess2Instance.myMess1FieldName[0]}")

It would be advisable to check the length though, since the message could be repeated any number of times (including zero).

Oshiro

Answer 3

One way to figure out the type contained by protobuf's RepeatedCompositeContainer is by using the add() function of RepeatedCompositeFieldContainer:

https://googleapis.dev/python/protobuf/latest/google/protobuf/internal/containers.html#google.protobuf.internal.containers.RepeatedCompositeFieldContainer.add

Based on the above documentation,

add(**kwargs: Any) → _T
Adds a new element at the end of the list and returns it. Keyword arguments may be used to initialize the element.

The key thing is the return_type -> notice it returns the type _T. This can give us an idea of what the type of elements will be within the RepeatedCompositeContainer.

So in your example, you can do something like this:

container = getattr(MyMess2Instance, myMess1FieldName)
elem_type = container.add()

Now the elem_type should be of the form myMess1.

Zinovievsk