How to retrieve float_val from a PredictResponse object?

Question

I am running a prediction on a tensorflow-serving model, and I get back this PredictResponse object as output:

Result:

outputs {
  key: "outputs"
  value {
    dtype: DT_FLOAT
    tensor_shape {
      dim {
        size: 1
      }
      dim {
        size: 20
      }
    }
    float_val: 0.000343723397236
    float_val: 0.999655127525
    float_val: 3.96821117632e-11
    float_val: 1.20521548297e-09
    float_val: 2.09611101809e-08
    float_val: 1.46216549979e-09
    float_val: 3.87274603497e-08
    float_val: 1.83520256769e-08
    float_val: 1.47733780764e-08
    float_val: 8.00914179422e-08
    float_val: 2.29388191997e-07
    float_val: 6.27798826258e-08
    float_val: 1.08802950649e-07
    float_val: 4.39628813353e-08
    float_val: 7.87182985462e-10
    float_val: 1.31638898893e-07
    float_val: 1.42612295306e-08
    float_val: 3.0768305237e-07
    float_val: 1.12661648899e-08
    float_val: 1.68554503688e-08
  }
}

I would like to get the out the float vals as a list. Or, alternatively, return the value/index of the argmax float_val!

This is generated by:

stub = prediction_service_pb2.beta_create_PredictionService_stub(channel) result = stub.Predict(request, 200.0)

Thanks for your help in advance.

Tallent

Answer 1

The answer is:

floats = result.outputs['outputs'].float_val

Tallent

Same problem but result.outputs['outputs'].float_val returns type <type 'google.protobuf.pyext._message.RepeatedScalarContainer'>. How did you convert this into list of floats? – Doy

The answer for TF 1.2+ is: result.result().outputs['outputs'].float_val – Tallent

Convert it to list and then numpy array if you'd like, floats = numpy.array(list(result.outputs['outputs'].float_val)) – Presentation

Answer 2

You generally want to recover a tensor, with a shape (not just a long list of floats). Here's how:

outputs_tensor_proto = result.outputs["outputs"]
shape = tf.TensorShape(outputs_tensor_proto.tensor_shape)
outputs = tf.constant(outputs_tensor_proto.float_val, shape=shape)

If you prefer to get a NumPy array, then just replace the last line:

outputs = np.array(outputs_tensor_proto.float_val).reshape(shape.as_list())

If you don't want to depend on the TensorFlow library at all, for some reason:

outputs_tensor_proto = result.outputs["outputs"]
shape = [dim.size for dim in outputs_tensor_proto.tensor_shape.dim]
outputs = np.array(outputs_tensor_proto.float_val).reshape(shape)

Dogface

This should be the accepted answer because it addresses the general-case scenario. – Rectal

I had to do .reshape((shape)) to get this to work. But I agree with jamix on this one. This should be the accepted answer. – Warton

Note, that the result may not be in ["outputs"]. In my case it was in ["score"]. But this was the right answer for me. – Kebab

Answer 3

If you would like to convert the entire PredictResponse to a numpy array (including its dimentions)

<script src="https://gist.github.com/eavidan/22ad044f909e5739ceca9ff9e6feaa43.js"></script>

Murmuration

Can you please help me how to get the predictions along with probabilities, I am getting the same output from TF-serving that is only probabilities.? – Copilot

Answer 4

This answer is for tensorflow-serving-api-python3 1.8.0

result.outputs['your key name'].float_val #key name in your case is outputs

This will return a repeated scalar container object. Which can be passed to python list() or np.array() etc

Veridical

Answer 5

result["outputs"].float_val should be a python list

Theirs

Hi Alexandre, Thank you for your response. But I got: Traceback (most recent call last): line 95, in main vals = result['outputs'].float_val TypeError: 'PredictResponse' object has no attribute '__getitem__' – Tallent

results.value[0].float_val then – Theirs

Does not work: AttributeError: 'PredictResponse' object has no attribute 'value' – Tallent