Python gRPC health check

Question

I'm trying to learn gRPC and implemented the same code as in the tutorial. Wondering how to add gRPC health check to it.

Stumbled upon this, but clueless on how to write a gRPC health check.

Middleclass

Answer 1

I found this after many hours of search. To health check gRPC server, you have to add healthCheckService to your existing server. So the existing server will have multiple services running on it.

Example of how to add multiple services in same server is exaplained here.

Sample server:

# pip install grpcio-health-checking
from grpc_health.v1 import health
from grpc_health.v1 import health_pb2
from grpc_health.v1 import health_pb2_grpc

server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
  
# your normal service, that the server is supposed to run
helloworld_pb2.add_GreeterServicer_to_server(_GreeterServicer(), server)
  
# health check service - add this service to server
health_pb2_grpc.add_HealthServicer_to_server(
    health.HealthServicer(), server)

server.add_insecure_port('[::]:50051')
server.start()

Lymn

Answer 2

The accompanying unit tests serve as a pretty good reference.

        def start_server(self, non_blocking=False, thread_pool=None):
            self._thread_pool = thread_pool
            self._servicer = health.HealthServicer(
                experimental_non_blocking=non_blocking,
                experimental_thread_pool=thread_pool)
            self._servicer.set('', health_pb2.HealthCheckResponse.SERVING)
            self._servicer.set(_SERVING_SERVICE,
                               health_pb2.HealthCheckResponse.SERVING)
            self._servicer.set(_UNKNOWN_SERVICE,
                               health_pb2.HealthCheckResponse.UNKNOWN)
            self._servicer.set(_NOT_SERVING_SERVICE,
                               health_pb2.HealthCheckResponse.NOT_SERVING)
            self._server = test_common.test_server()
            port = self._server.add_insecure_port('[::]:0')
            health_pb2_grpc.add_HealthServicer_to_server(
                self._servicer, self._server)
            self._server.start()

Keep a reference to the health servicer in the main servicer class for your application server. Then, call the set() method on it at at the appropriate times, e.g. when startup is finished or when it goes into a state in which it is unable to server requests. Do note however that this makes use of some experimental features in order to ensure that the attached health servicer does not cause application-level requests to be starved.

Macula

It seems like the only way to test or get the health status is a gRPC client call. Which sucks because I'm trying to do this in an env where the system wants a curl command to invoke on the docker container to check the status. If you have to manually set the status yourself (the health servicer isn't automatically checking the status of an associated servicer), what's the point of using any of this? – Kentiga

Firstly, this question asks about a gRPC health check. An HTTP health check is something different which can easily be added to an application that is also running a gRPC server. But doing that is outside of the scope of the gRPC library. If you're looking for a very cheap liveness check, many systems also support TCP-level liveness checks. – Macula

There are different levels of granularity with which you can do a health check. Reporting healthy whenever the process is up is not particularly robust, but it can be implemented by a single line setting the status to SERVING at start-up. If there's a period after the server starts during which the application needs to do preprocessing. Or if the application simply cannot operate when one of its downstream dependencies isn't functioning, being able to set that status yourself is vital. – Macula

Fair points. I found a grpcurl tool that I will likely use. Thanks for the insights! – Kentiga

Answer 3

I found this example really helpful: https://github.com/grpc/grpc/tree/master/examples/python/xds

Specifically, server.py provides an implementation of health checking as well as reflection.

Behavior