What is the maximum bandwidth gcp instance can use with single network interface?

Question

• I am uploading data to Google cloud storage bucket using n processes parallelly, egress data rate is ~16Gbps.
• For only downloading data from gcp storage bucket using n processes parallelly, ingress traffic is ~26Gbps.
• But when I execute upload and download at the same time with same no. of processes, ingress rate reduces to ~7Gbps and egress to ~11/12Gbps.
• Using different buckets for upload and download. Instance and buckets are in the same region. Using Windows Server 2012 r2 OS and configuration is n1-standard-32 (32 vCPUs, 120 GB memory). Upload and download api's are in C# REST api using httprequests. GCP documention doesn't mentioned regarding any bandwidth cap: https://cloud.google.com/vpc/docs/quota#per_instance.
• LinkSpeed for the adapter is 100 Gbps
• If only upload and only download takes full bandwidth, then I am assuming executing both at the same time should scale each to ~16Gbps.
• Is there any cap for overall traffic per gcp instance? what is the maximum bandwidth gcp instance can use with single network interface?

Intercessor

I'm not enough expert for explaining this, but it should have something in network congestion/collision and overhead ACK message when your sending and receiving data. Finally, the 16Gbps of download is in the best case, maybe your tests have been done at the wrong moment on the network! – Outmarch

Wojciech Bogacz's answer is good. Google caps bandwidth at 2 Gbps per core (ingress+egress) and either a 16 or 32 Gbps max depending on CPU architecture. These are theoretical numbers and not guarantees. Remember, you are on a virtualized platform. Items like link speed are logical and not physical. CPU slice allocation is very variable. For real numbers you need a baremetal instance so that you control everything (you are the only processes running on the entire machine). – Closemouthed

FYI a few days ago we increased egress transfer rate to 10gbps for 2/4-core machines: cloud.google.com/compute/docs/release-notes#november222019 – Aweless

Answer 1

GCP does not cap the ingress or egress traffic - but it all depends on what the machine can handle and how much network can take (it will vary depending on network conditions).

The documentation you linked to says that there's no cap but what it says is that

egress and ingress bandwidth depends on machine type

It also states (under "Maximum ingress data rate" in the "Notes" column that you should plan only 10Gbps for one machine:

For purposes of capacity planning, you should assume that each VM instance can handle no more than 10 Gbps of external Internet traffic.

You can look up the max bandwidth for various machine types in the documentation. You have N1 32cpu for which limit is 32Gbps (

32 Gbps for Skylake or later CPU platforms. 16 Gbps for all other platforms

Considering that you do achieved about 26 Gbps effectively I would say you're it's very good result - furthermore under non ideal network conditions.

There's more:

Network bandwidth is up to the specified limit. Actual performance depends on factors such as network congestion or protocol overhead.

Depending on the measuring method - in my opinion you almost reached the arbitrary bandwidth limit of a single GCP VM.

If I were you I would just stick with that number and plan accordingly. If you want more speed then you can write GCP support and ask.

Fussy

i use GCP for 2 yrs now. I cant get more than 1Gbps file upload to their SSH (web-based) console anyhow I try.. – Borderline

" I cant get more than 1Gbps file upload to their SSH (web-based) console anyhow I try." ssh has an application-level flow control mechanism which will preclude having high throughput over a high bandwidthXdelay product network. It is baked-into ssh. – Peking

Answer 2

There is one error at the beginning of Wojciech's answer. Google does apply an egress cap/throttle to a VM's traffic.

When you are running the various tests, have you been looking at the per-vCPU utilization in your VM? Not just the overall CPU utilization, but per-vCPU. You want to see if any individual vCPU(s) are saturating. Also, are these uploads/downloads from/to storage in the VM or simply memory? You may want to look at the utilization of your VM's storage while your running your tests.

You might also look to see what the depth of the vNIC receive queues happens to be and if it is already at the maximum.

Edit: If you have not already, you might consider enabling one of the private access to Google services options: https://cloud.google.com/vpc/docs/private-access-options

Peking