protofile.proto: A file with this name is already in the pool

Question

Having the following structure:

- project1
  - project1.py
  - protofile_pb2.py
  - protofile_pb2_grpc.py
- project2
  - project2.py
  - protofile_pb2.py
  - protofile_pb2_grpc.py

project1.py:

import protofile_pb2.py
...

project2.py:

import protofile_pb2
import project1
...

When running project2.py, I get this error:

TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "protofile.proto":
protofile.proto: A file with this name is already in the pool.

Seanseana

Answer 1

You are using different versions of your generated protofile.proto. Make sure to regenerate the protobuf files and it should work fine.

Seanseana

just to add for a quickstart, regenerate the protobuf files with the protobuf compiler would look like this: python -m grpc_tools.protoc -I <folder of yourscript.proto> --python_out=<folder of yourscript_pb2.py> --grpc_python_out=<folder of yourscript_grpc_pb2.py> yourscript.proto – Stefanysteffane

Answer 2

According to this comment, it worked for me:

pip uninstall protobuf
pip install --no-binary protobuf protobuf

Seanseana

Answer 3

I faced similar issue while deploying google cloud functions in Python. My cloud function was using different versions of .proto files in my module (which was again a private module which I wanted to install using requirements.txt). The solution is to use pure python implementation instead of using binary implementation of protobuf. To solve this in cloud functions or in cloud build you can set cloud functions env variables to use python implementation of protocol buffers.

In your gcloud cloud function deploy command use following option:

<start of command> --set-env-vars=PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION='python' <your rest of the command>

or if you are in any other env/os simply do this

export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION='python' 

This solved my issue.

Goodale

Thank you! Your workaround seems to make things faster too when using sentencepiece_model_pb2 – Pewee

This worked for me, I don't know the culprit, but my logs seem to show it might have been ChromaDB or Newrelic. – Intercessory

Answer 4

For protobuf>=3.20.x

Install Protocol Buffers library for Python

pip install protobuf
set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python

Clean the caches and restart the kernel

Alternatives

Downgrade the protobuf package to 3.20.x or lower

pip install protobuf=3.11.3

Search for any duplicate occurrences of the sentencepiece_model.proto file in your project. Make sure there is only one copy of the file present.

Maladminister

Answer 5

I had same problem with 2 similar .proto files with one MessageName - and had error:

my_v1.proto:5:12: "MessageName" is already defined in file "my_v2.proto".

Error appeared because I've missed package line in .proto files:

syntax="proto3";

package filename1; # <-- this missed

message MessageName {<...>}

Of course packages in 2 files must be different

To check default pool error when compile I used :

protoc \
--proto_path=${PWD} \
--python_out=${PWD} \
${PWD}/my_v1.proto \
${PWD}/my_v2.proto

Easygoing