gsutil no longer works?

Question

I've been using gcloud and gsutil for a while but now suddenly for any gsutil command I run I get errors:

Traceback (most recent call last):
  File "/Users/julian/google-cloud-sdk/bin/bootstrapping/gsutil.py", line 12, in 
    import bootstrapping
  File "/Users/julian/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 22, in 
    from googlecloudsdk.core.credentials import store as c_store
  File "/Users/julian/google-cloud-sdk/lib/googlecloudsdk/core/credentials/store.py", line 27, in 
    from googlecloudsdk.core import http
  File "/Users/julian/google-cloud-sdk/lib/googlecloudsdk/core/http.py", line 31, in 
    from googlecloudsdk.core.resource import session_capturer
  File "/Users/julian/google-cloud-sdk/lib/googlecloudsdk/core/resource/session_capturer.py", line 32, in 
    from googlecloudsdk.core.resource import yaml_printer
  File "/Users/julian/google-cloud-sdk/lib/googlecloudsdk/core/resource/yaml_printer.py", line 17, in 
    from googlecloudsdk.core.resource import resource_printer_base
  File "/Users/julian/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_printer_base.py", line 38, in 
    from googlecloudsdk.core.resource import resource_projector
  File "/Users/julian/google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_projector.py", line 34, in 
    from google.protobuf import json_format as protobuf_encoding
ImportError: cannot import name json_format

I tried gcloud update and gcloud reinstall but still get same problem. Is there a conflict with the python installation? Any other ideas?

Onrush

What method are you using to install gcloud? – Mayhap

I followed the instructions at cloud.google.com/storage/docs/gsutil_install – Onrush

This also happened to me after a gcloud components update that bumped me to 180.0.1 – Boat

Answer 1

You can also fix this error by updating to the latest version of protobuf. Then you don't have to downgrade gcloud.

pip install protobuf --upgrade

Tartuffery

This didn't solve the problem for me. I still had to revert to 175.0.0. – Avram

This didn't work for me initially. However, I tried editing google-cloud-sdk/lib/googlecloudsdk/core/resource/resource_projector.py to see what was going in that file and found that making just a tiny change (adding a new line for example) was enough to make everything work again. I guess the system needs a nudge to use the new protobuf? – Onrush

Answer 2

The issue occurs when you have google protobuf package installed.

You can work around the problem either by using older version by running

gcloud components update --version 175.0.0

or by removing google protobuf package the python installation.

pip remove protobuf

If you do not want to do either of these, they can install a separate version of 2.7 python (without protobuf) and set CLOUDSDK_PYTHON=/path/to/python environment variable.

Also next release of cloud sdk should have this fix.

Propagation

pip remove protobuf not a recognized command. I assume you mean pip uninstall protobuf however this did not fix the problem. your first suggestion of reverting to version 175.0.0 does work - my version was 179.0.0 so I have gone backwards so not ideal but at least is a workaround for now. – Onrush

Can also confirm that version 175.0.0 fixes the issue. I was also seeing this issue in 180.0.1 before reverting – Boat

Just tried updating to v 180.0.1 but get same problem. Reverting to 175.0.0 fixes the problem. @Propagation So what gives? When you say the next release of the cloud should fix this problem what did you mean? Do you know something about what is going on at Google? Or were you just guessing? – Onrush

Answer 3

I had the same issue. I am using a mac. Looking into /usr/local/lib/python2.7/site-packages i found a homebrew protobuf link. I removed it with "rm homebrew-protobuf.pth" Then gsutil started working.

Wrathful

This worked for me! Thanks! :D mdfind -name homebrew-protobuf.pth found it here for me: /usr/local/Cellar/[email protected]/2.6.1/lib/python2.7/site-packages/homebrew-protobuf.pth after a brew uninstall --force protobuf.... – Glynas

Answer 4

For CentOS 7.5 (probably earlier as well) using the Google Cloud SDK rpm install, removing the protobuf-python package yum remove protobuf-python will solve this.

Longanimity

Answer 5

I wound up fixing this on my Mac by updating the homebrew install of protobuf:

$ brew upgrade protobuf
...
==> Upgrading 1 outdated package, with result:
protobuf 2.6.1 -> 3.6.0
...

See this issue on the gsutil GitHub repo.

Scampi