How does gRPC client-streaming flow control work in go?

Question

I would like to know, how flow control works in a client-streaming gRPC service in Go.

Specifically, I am interested in knowing when will a call to stream.SendMsg() function in the client-side block? According to the documentation :

SendMsg() blocks until :

• There is sufficient flow control to schedule m with the transport, or ...

So what is the specification for the flow control mechanism of the stream? For example, if the server-side code responsible for reading the messages from the stream, isn't reading the messages fast enough, at what point will calls to SendMsg() block?

Is there some kind of backpressure mechanism implemented for the server to tell the client that it is not ready to receive more data? In the meantime, where are all the messages that have been successfully sent before the backpressure signal, queued?

Boaster

Answer 1

gRPC flow control is based on http2 flow control: https://httpwg.org/specs/rfc7540.html#FlowControl

There will be backpressure. Messages are only successfully sent when there's enough flow control window for them, otherwise SendMsg() will block.

The signal from the receiving side is not to add backpressure, it's to release backpressure. It's like saying "now I'm ready to receive another 1MB of messages, send them".

Piton

Should you provide more information on how to set the flow control window of a gRPC server? Is it possible to increase the size of the window? – Pettifog

Answer 2

Here is another explanation of flow control

HTTP/2 flow control is a feature that prevents apps from being overwhelmed with data. When using flow control:

• Each HTTP/2 connection and request has an available buffer window. The buffer window is how much data the app can receive at once.
• Flow control activates if the buffer window is filled up. When activated, the sending app pauses sending more data.
• Once the receiving app has processed data, then space in the buffer window is available. The sending app resumes sending data.

Flow control can have a negative impact on performance when receiving large messages. If the buffer window is smaller than incoming message payloads or there's a latency between the client and server, then data can be sent in start/stop bursts.

Flow control performance issues can be fixed by increasing the buffer window size.

• client side: set the window through WithInitialWindowSize and WithInitialConnWindowSize of the dial option

  // WithInitialWindowSize returns a DialOption which sets the value for initial
  // window size on a stream. The lower bound for window size is 64K and any value
  // smaller than that will be ignored.
  func WithInitialWindowSize(s int32) DialOption {
      return newFuncDialOption(func(o *dialOptions) {
          o.copts.InitialWindowSize = s
      })
  }
  
  // WithInitialConnWindowSize returns a DialOption which sets the value for
  // initial window size on a connection. The lower bound for window size is 64K
  // and any value smaller than that will be ignored.
  func WithInitialConnWindowSize(s int32) DialOption {
      return newFuncDialOption(func(o *dialOptions) {
          o.copts.InitialConnWindowSize = s
      })
  }
  

• server side: set the window through InitialWindowSize and InitialConnWindowSize of server options

  // InitialWindowSize returns a ServerOption that sets window size for stream.
  // The lower bound for window size is 64K and any value smaller than that will be ignored.
  func InitialWindowSize(s int32) ServerOption {
      return newFuncServerOption(func(o *serverOptions) {
          o.initialWindowSize = s
      })
  }
  
  // InitialConnWindowSize returns a ServerOption that sets window size for a connection.
  // The lower bound for window size is 64K and any value smaller than that will be ignored.
  func InitialConnWindowSize(s int32) ServerOption {
      return newFuncServerOption(func(o *serverOptions) {
          o.initialConnWindowSize = s
      })
  }
  
  

Recommendations:

• If a gRPC service often receives messages larger than 96 KB, Kestrel's default stream window size, then consider increasing the connection and stream window size.
• The connection window size should always be equal to or greater than the stream window size. A stream is part of the connection, and the sender is limited by both.

For more information about how flow control works, see HTTP/2 Flow Control

Objectivism