- tensorflow-gpu 1.10.0
- tensorflow-server 1.10.0
I have deployed a tensorflow server which serves several models.
The client code is like client.py this and I call the predict function.
channel = implementations.insecure_channel(host, port)
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
request = predict_pb2.PredictRequest()
def predict(data, shape, model_name, signature_name="predict"):
request.model_spec.name = model_name
request.model_spec.signature_name = signature_name
request.inputs['image'].CopyFrom(tf.contrib.util.make_tensor_proto(data, shape=shape))
result = stub.Predict(request, 10.0)
return result.outputs['prediction'].float_val[0]
I have about 100 clients with the same configuration.
And here is a sample code to call the predict function:
from client import predict
while True:
print(predict(data, shape, model_name))
# time.sleep some while
At first, when I run the client code, I can receieve the reponse correctly. But after several hours, the client crashed with the error
_Rendezvous of RPC that terminated with (StatusCode.UNAVAILABLE, Socket closed)
I have try to modify my client code to
def predict(data, shape, model_name, signature_name="predict"):
channel = implementations.insecure_channel(host, port)
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
request = predict_pb2.PredictRequest()
request.model_spec.name = model_name
request.model_spec.signature_name = signature_name
request.inputs['image'].CopyFrom(tf.contrib.util.make_tensor_proto(data, shape=shape))
result = stub.Predict(request, 10.0)
return result.outputs['prediction'].float_val[0]
that means I try to establish the connection with the tfs server each time the predict function is called. But this code also failed just like before.
So what should I do to deal with this situation?