Why gcloud command is slow to start?

Question

Just typing gcloud for help take 5 secs.

$ gcloud
...
gcloud  0.30s user 0.13s system 7% cpu 5.508 total

$ gcloud version
Google Cloud SDK 128.0.0
alpha 2016.01.12
bq 2.0.24
bq-nix 2.0.24
core 2016.09.23
core-nix 2016.09.20
gcloud 
gsutil 4.21
gsutil-nix 4.21
kubectl 
kubectl-darwin-x86_64 1.3.7

$ uname -a
Darwin hiroshi-MacBook.local 16.0.0 Darwin Kernel Version 16.0.0: Mon Aug 29 17:56:20 PDT 2016; root:xnu-3789.1.32~3/RELEASE_X86_64 x86_64

Prostrate

Answer 1

EDIT 2017-03-31: Zachary said that gcloud 148.0.0 addressed this issue. So try gcloud components update. see https://stackoverflow.com/users/4922212/zachary-newman

tl;dr

It turns out that socket.gethostbyaddr(socket.gethostname()) is slow for .local hostname in macOS.

$ python -i
>>> socket.gethostname()
'hiroshi-MacBook.local'
>>> socket.gethostbyaddr(socket.gethostname()) # it takes about 5 seconds
('localhost', ['1.0.0.127.in-addr.arpa'], ['127.0.0.1'])

So, for a workaround, just added the hostname to the localhost line of /etc/hosts.

127.0.0.1     localhost hiroshi-Macbook.local                             

After that is return value is different, but it returns in an instant.

>>> socket.gethostbyaddr(socket.gethostname())
('localhost', ['1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa'], ['::1'])

How do I get there

Where gcloud command is:

$ which gcloud
/Users/hiroshi/google-cloud-sdk/bin/gcloud

Edit the endline of the shell script...

...
+ echo "$CLOUDSDK_PYTHON" $CLOUDSDK_PYTHON_ARGS "${CLOUDSDK_ROOT_DIR}/lib/gcloud.py" "$@"
"$CLOUDSDK_PYTHON" $CLOUDSDK_PYTHON_ARGS "${CLOUDSDK_ROOT_DIR}/lib/gcloud.py" "$@"

to echo where the gcloud.py is:

$ gcloud
python2.7 -S /Users/hiroshi/google-cloud-sdk/lib/gcloud.py

OK. Who take the 5 secs?

$ python2.7 -S -m cProfile -s time /Users/hiroshi/google-cloud-sdk/lib/gcloud.py
     173315 function calls (168167 primitive calls) in 5.451 seconds

Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    5.062    5.062    5.062    5.062 {_socket.gethostbyaddr}
...

_socket.gethostbyaddr is.

What is the argument of the function call and backtrace look like? I added some lines before main() of gcloud.py

import traceback
def mygethostbyaddr(addr):
  print addr
  traceback.print_stack()
  return addr
import socket
socket.gethostbyaddr = mygethostbyaddr

Execute gcloud again. I got it is my .local name of my machine.

$ gcloud
hiroshi-MacBook.local
  File "/Users/hiroshi/google-cloud-sdk/lib/gcloud.py", line 74, in <module>
    main()
  File "/Users/hiroshi/google-cloud-sdk/lib/gcloud.py", line 70, in main
    sys.exit(googlecloudsdk.gcloud_main.main())
  File "/Users/hiroshi/google-cloud-sdk/lib/googlecloudsdk/gcloud_main.py", line 121, in main
    metrics.Started(START_TIME)
  File "/Users/hiroshi/google-cloud-sdk/lib/googlecloudsdk/core/metrics.py", line 411, in Wrapper
    return func(*args, **kwds)
  File "/Users/hiroshi/google-cloud-sdk/lib/googlecloudsdk/core/metrics.py", line 554, in Started
    collector = _MetricsCollector.GetCollector()
  File "/Users/hiroshi/google-cloud-sdk/lib/googlecloudsdk/core/metrics.py", line 139, in GetCollector
    _MetricsCollector._instance = _MetricsCollector()
  File "/Users/hiroshi/google-cloud-sdk/lib/googlecloudsdk/core/metrics.py", line 197, in __init__
    hostname = socket.getfqdn()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 141, in getfqdn
    hostname, aliases, ipaddrs = gethostbyaddr(name)
  File "/Users/hiroshi/google-cloud-sdk/lib/gcloud.py", line 32, in mygethostbyaddr
    traceback.print_stack()

Prostrate

Dude, you rock! Thanks for saving me a bunch of time :) – Nevil

FYI we've made a change starting in Cloud SDK 148.0.0 that should speed things up. Thanks for bringing this to our attention! – Rondelet

This no longer seems to work When you run gcloud with no additional params, you get the error ERROR: (gcloud) Command name argument expected. – Nancinancie

Yeah, gcloud 382.0.0 emit the error. The problem was soleved though. – Prostrate

Answer 2

@hiroshi's answer solved the issue for me i.e. to run gcloud components update. However, since I had installed gcloud through their Cloud SDK using a package manager, I was stuck with the following error*:

ERROR: (gcloud.components.update) You cannot perform this action because the Google Cloud CLI component manager is disabled for this installation.

Hence, I had to explicitly mention the apt-get libraries to perform the update. The following command helped me to get it done in one go:

sudo apt-get update && sudo apt-get --only-upgrade install google-cloud-sdk-app-engine-go google-cloud-sdk-datastore-emulator google-cloud-sdk-datalab google-cloud-sdk-firestore-emulator google-cloud-sdk-kubectl-oidc google-cloud-sdk google-cloud-sdk-app-engine-python-extras google-cloud-sdk-cloud-build-local kubectl google-cloud-sdk-cbt google-cloud-sdk-minikube google-cloud-sdk-skaffold google-cloud-sdk-cloud-run-proxy google-cloud-sdk-pubsub-emulator google-cloud-sdk-config-connector google-cloud-sdk-gke-gcloud-auth-plugin google-cloud-sdk-kpt google-cloud-sdk-local-extract google-cloud-sdk-nomos google-cloud-sdk-app-engine-grpc google-cloud-sdk-bigtable-emulator google-cloud-sdk-app-engine-python google-cloud-sdk-terraform-validator google-cloud-sdk-anthos-auth google-cloud-sdk-spanner-emulator google-cloud-sdk-app-engine-java

*More details as to why the aforementioned error occurs and how to permanently solve it can be found here.

Semela