keep track of the request queue in grpc server

Question

GRPC server does queue the requests and serve them based on the maxWorker configuration which is passed when the sever starts up. How do I print the metric- number of items in the queue .? Essentially, I would like to keep track of the numbers of requests in waiting state.

Alica

Answer 1

You can pass your own executor to serverBuilder.executor(). Note that you are then responsible for shutting down the executor service after the server has terminated.

Eat

I am using python on my server side. Ideally what I want is the number of messages currently sitting in the channel queue. – Alica

My answer was for grpc-java. I don't know the answer for grpc-python. – Eat

Answer 2

The solution for grpc-python is similar to grpc-java. You can pass your customized future executor to the server constructor, and monitor the submission of task yourself.

gRPC Python API: grpc.server(thread_pool, ...)

The executor class to extend: concurrent.futures.ThreadPoolExecutor

Levorotation