google compute engine tool gcloud is exceptionally slow

S

5

25

I tried downloading and using the gcloud bash tool to manage my accounts, however everything I do with the tool is exceptionally slow. It will take MINUTES to reply to a command that is typed.

Is there perhaps a firewall I need to open up on my router or something else to get this to work fast like it's supposed to? For example, the "Installing..." lines in this video https://youtu.be/4y4-xn4Vi04?t=1m21s -- you'll notice they are all complete in the tutorial within a few seconds. This takes over 10 minutes to complete on my machine.

I'm on a newer Macbook Air, and all other internet/etc works really fast. I'm on a decent speed internet connection from AT&T Uverse (30mbps/3mbps). All other browsing is fast and just fine, the only thing in the world I have a problem with is this gcloud tool.

Schmaltz answered 1/5, 2015 at 13:53 Comment(5)

Are others facing the same issue? I am facing this in on Mac OS X El Captain and cant seem to find a clear solution. Running traceroute www.googleapis.com reveals the delay on remote Google servers and not at the client or router's end. gcloud version is SDK 169.0.0 – Ctn 5/9, 2017 at 5:35

using gcloud on an f1-micro instance is slower than actually starting the whole instance. I really hope Google is going to exchange this extremely bulky tool against something FAST – Scud 24/11, 2017 at 1:8

I'm getting this recently (on Linux), and using strace have seen that it's when refreshing auth tokens with gcloud.py config config-helper - both with the gcloud & kubectl commands - it appears to be hanging during a DNS lookup of metadata.google.internal for me. Using Google's public DNS servers may help. – Dives 5/8, 2021 at 10:14

@AndrewRembrandt Had the exact same behaviour (on an M1 Mac), and traced it back to this same DNS lookup that was hanging. Your solution of adding google's public DNS as a secondary DNS server to my laptop fixed the issue ! – Edwyna 5/5, 2022 at 13:21

This is still an issue in 2023 for any devs that see this... – Woodrow 23/1, 2023 at 18:24

S

12

I traced this back to packet filters. When I deleted this rule #5 in my AT&T Uverse modem/router (Motorola NVG589), everything works ok. This was a default setting in the modem which caused all sorts of issues.

packet filters

Schmaltz answered 2/5, 2015 at 14:6 Comment(3)

this answer helped me tremendously.. ipv6 was enabled, had to login to the arris router admin and under firewall, disable block fragmented ip packets (enabled by default..) – Possessed 17/9, 2017 at 11:38

This help me as well on AT&T Small Business fiber. In fact none of the rules were enabled for us, but I still had this problem. Disabling the packet filter completely solved it. – Elmiraelmo 23/1, 2018 at 17:57

Thank you. I have their fiber service as well. I had to disable all the packet filters as well then reset my broadband connection. – Nicolanicolai 1/2, 2018 at 17:16

A

20

I just ran into a similar issue myself, though not as bad as minute long response times. What helped me was turning off the usage reporting.

Looking at some timings:

$ gcloud config set disable_usage_reporting False
$ time gcloud compute -h | tail -1
  real  0m7.058s
  user  0m0.464s
  sys   0m0.088s

A whopping 7 secs to access the help!

Fortunately, this improves greatly after disabling reporting:

$ gcloud config set disable_usage_reporting True
$ time gcloud compute -h | tail -1
  real  0m0.541s
  user  0m0.459s
  sys   0m0.080s

Much better!

Allonym answered 8/5, 2015 at 18:6 Comment(7)

What version of the SDK are you currently using? – Morceau 11/5, 2015 at 15:21

gcloud -v returns: "Google Cloud SDK 0.9.58" – Allonym 12/5, 2015 at 20:20

This answer still applies on Google Cloud SDK 137.0.0! – Josefjosefa 14/12, 2016 at 8:6

This is still an issue on Google Cloud SDK version 139.0.1 and Mac OS X Sierra. I think it's a different issue from the one noticed by Mark Shust. Commands went from taking about 10 seconds each to taking less than a second. – Ascarid 17/1, 2017 at 20:13

On windows 10, gcloud ssh into docker-optimized image, not as slow as reported about, but still extremely distracting. Disabling reporting helps only marginally. – Zobkiw 18/1, 2017 at 17:41

Looks like subsequent runs of the above command (without updating the config) reduces the time anyways (maybe some kind of caching). This what I got on two subsequent runs without any change in the config: First run: time gcloud compute -h | tail -1 real 0m3.031s user 0m0.756s sys 0m0.477s Second run: time gcloud compute -h | tail -1 real 0m0.585s user 0m0.426s sys 0m0.147s Also I didn't see any visible difference by disabling reporting on ver SDK 169.0.0 – Ctn 5/9, 2017 at 5:16

Still pretty slow with v310.0.0. 1.8 sec just for gcloud -v or gcloud config get-value product on my machine. – Verlie 23/10, 2020 at 14:54

S

12

I traced this back to packet filters. When I deleted this rule #5 in my AT&T Uverse modem/router (Motorola NVG589), everything works ok. This was a default setting in the modem which caused all sorts of issues.

packet filters

Schmaltz answered 2/5, 2015 at 14:6 Comment(3)

this answer helped me tremendously.. ipv6 was enabled, had to login to the arris router admin and under firewall, disable block fragmented ip packets (enabled by default..) – Possessed 17/9, 2017 at 11:38

This help me as well on AT&T Small Business fiber. In fact none of the rules were enabled for us, but I still had this problem. Disabling the packet filter completely solved it. – Elmiraelmo 23/1, 2018 at 17:57

Thank you. I have their fiber service as well. I had to disable all the packet filters as well then reset my broadband connection. – Nicolanicolai 1/2, 2018 at 17:16

F

3

Mark Shust's answer hinted to me that something might be going on with IPv6. Turning this support off in my router brought the gcloud times back to a couple of seconds from over 2 minutes for a gcloud compute machine-types list.

Note: I've also had to turn off IPv6 support in apt for a similar reason so this is likely a problem with my router or ISP, not the google cloud sdk.

Faulk answered 15/10, 2015 at 3:33 Comment(3)

I just disabled ipv6 on my router and it worked as well – Cohbert 18/3, 2022 at 17:51

In addition to my comment on the OP, it turns out my real issue was actually to do with the fact that IPv6 was mis-configured on a second router in my home network (it was requesting prefix delegation when it shouldn't have). As soon as I corrected the prefix issue, the gcloud sdk delays (when renewing auth tokens I suspect) went away. NB: Running strace -f -e trace=network -s 10000 gcloud <args> shows some interesting things that one would normally not expect - it appears that google has written it to detect if it's running within google's internal networks, which can be problematic – Dives 7/5, 2022 at 13:20

This also was an issue with my network; misconfigured ipv6; I disabled ipv6 from the adapter settings in windows. In my case gcloud was hanging forever, not just slow. – Leadin 18/8, 2023 at 6:38

O

0

Same behavior for me. Simple command hang on

gcloud compute instances list --log-http --verbosity debug

DEBUG: Starting new HTTPS connection (1): oauth2.googleapis.com:443

If I ping oauth2.googleapis.com PING oauth2.googleapis.com(sof02s34-in-x0a.1e100.net (2a00:1450:4017:80b::200a)) 56 data bytes you can see IPv6. In my case I have router Zyxel EX3301-T0

Disable LAN IPv6 Mode Setup and ping oauth2.googleapis.com PING oauth2.googleapis.com (172.217.17.138) 56(84) bytes of data. 64 bytes from 172.217.17.138 (172.217.17.138): icmp_seq=1 ttl=114 time=82.8 ms

Ordway answered 18/4, 2023 at 15:16 Comment(0)

J

0

In my case I using WSl, however my local computer using window already install gcloud cli and it making it so slow when using on WSL. So I resolve it by install again the gcloud cli on my WSL ubuntu follow the document. https://cloud.google.com/sdk/docs/install#deb

Jair answered 17/3 at 11:17 Comment(0)

Recommended topics

Hot tags