golang protobuf remove omitempty tag from generated json tags

Question

I am using google grpc with a json proxy. for some reason i need to remove the omitempty tags from the struct generated in the *.pb.go files.

if i have a proto message like this

message Status {
  int32 code = 1;
  string message = 2;
}

The generated struct looks like this

type Status struct {
  Code int32 `protobuf:"varint,1,opt,name=code" json:"code,omitempty"`
  Message string `protobuf:"bytes,2,opt,name=message" json:"message,omitempty"`
}

But My need is to remove the omitempty tag from the generated structs. How can i do this?

Jaffa

do you need the proto-buf working well? proto-buf need the tags. – Elenore

the thing is @JiangYD , i am sending an 0 value with one of my int feild. but when i am sending an 0 value the grpc-gateway omits the value. i know there are other approach, but i would like to know how i can remove an omit empty tag. – Jaffa

@Sadlil I'm not sure I understand your problem. If you send a 0 value, it will be omitted and thus the field will be left unchanged - properly leaving with value 0 if you used a zero-value struct (which you should anyway). – Telegram

my problem is when i parsed the struct into json with a value 0 for Code. the value gets omitted from the json. but i need this value to be present. – Jaffa

@Sadlil if you explicitly state: struct.Field = proto.Uint64(0) it should not be empty. – Turk

Answer 1

If you are using grpc-gateway and you need the default values to be present during json marshaling, you may consider to add the following option when creating your servemux

    gwmux := runtime.NewServeMux(runtime.WithMarshalerOption(runtime.MIMEWildcard, &runtime.JSONPb{OrigName: true, EmitDefaults: true}))

Outside of grpc-gateway, if you want to marshal your protocol buffer message, use google.golang.org/protobuf/encoding/protojson (*) package instead of encoding/json

func sendProtoMessage(resp proto.Message, w http.ResponseWriter) {
    w.Header().Set("Content-Type", "application/json; charset=utf-8")
    m := protojson.Marshaler{EmitDefaults: true}
    m.Marshal(w, resp) // You should check for errors here
}

(*) google.golang.org/protobuf replaces the now deprecated github.com/golang/protobuf and its jsonpb package.

Rudie

Answer 2

A [more] portable solution:

Use sed to strip the tags after generating via protoc.

Example of what I actually use in my go:generate script after having generated the *.pb.go files:

ls *.pb.go | xargs -n1 -IX bash -c 'sed s/,omitempty// X > X.tmp && mv X{.tmp,}'

---

Note: sed -i (inline-replacement) is not used here because that flag isn't portable between standard OS-X and Linux.

Jodiejodo

This seems to be the simplest workaround for this issue – Alodi

Answer 3

I found that the omitempty json tag is hard-coded into the protoc-gen-go source around line 1778:

tag := fmt.Sprintf("protobuf:%s json:%q",
    g.goTag(message, field, wiretype), jsonName+",omitempty")

it will be easy changing the source and make a new protoc-gen-go binary yourself.

It's worth noting that this is likely inadvisable and not recommended for several reasons, particularly because you will then be responsible for ensuring that the hacked up binary always gets used if the protobufs need to be regenerated.

Elenore

Answer 4

You can try using gogo proto (https://github.com/gogo/protobuf) With the jsontag extension, your proto message would look like

message Status {
  int32 code = 1 [(gogoproto.jsontag) = "code"];
  string message = 2 [(gogoproto.jsontag) = "message"];
}

You can also add more tags, if you like.

Garman

Nice! In addition, gogo supports custom tags e.g. [ (gogoproto.moretags) = "datastore:\"-\"" ]. To switch to gogo, simply change protoc --go_out= to protoc --gogofast_out= (note: gogofast is also 5..10 times faster than google proto!) – Yonah

Answer 5

I am posting an update to DeeSilence's answer that works with the latest protobuf version (at the moment of writing).

import "google.golang.org/protobuf/encoding/protojson"

m := protojson.MarshalOptions{EmitUnpopulated: true}
resp, err := m.Marshal(w)

Ophthalmic

Answer 6

The Marshaler under the jsonpb package has a EmitDefaults field. Setting this to true, will just ignore the omitempty tag in struct.

https://godoc.org/github.com/golang/protobuf/jsonpb#JSONPBMarshaler

Icterus

Answer 7

You could use "sed" command to remove this text from files like following

sed -i "" -e "s/,omitempty//g" ./api/proto/*.go

where args:

1. -i "" is meaning that keep same name of files
2. -e "s/,omitempty//g" = format to replace like "s/SEARCH/INSERT/g"

Spiky

This answer from 3+ years ago already suggests using sed as a post-process: https://mcmap.net/q/383963/-golang-protobuf-remove-omitempty-tag-from-generated-json-tags – Stillmann

Answer 8

To expand on @colm.anseo's edit saying that the google.golang.org/protobuf/encoding/protojson is now deprecated and now google.golang.org/protobuf should be used, here is an example how to do it without modifying the proto.

import (
    "google.golang.org/protobuf/encoding/protojson"
)

func main() {
    someProto := SomeProto{}
    marshaler := protojson.MarshalOptions{EmitUnpopulated: true}
    output, _ := marshaler.Marshal(someProto)
    ... 
}

This is the only way to do it if you don't want to modify the generated struct / are importing the struct via buf and the bsr.

Evolutionist

Answer 9

you can copy the encoding/json package to your own folder for example my_json, and modify omitEmpty field to false, and use my_json.Marshal() to encode the struct to json string.

Hipparchus