TensorFlow Estimator ServingInputReceiver features vs receiver_tensors: when and why?

Question

In a previous question the purpose and structure of the serving_input_receiver_fn is explored and in the answer:

def serving_input_receiver_fn():
  """For the sake of the example, let's assume your input to the network will be a 28x28 grayscale image that you'll then preprocess as needed"""
  input_images = tf.placeholder(dtype=tf.uint8,
                                         shape=[None, 28, 28, 1],
                                         name='input_images')
  # here you do all the operations you need on the images before they can be fed to the net (e.g., normalizing, reshaping, etc). Let's assume "images" is the resulting tensor.

  features = {'input_data' : images} # this is the dict that is then passed as "features" parameter to your model_fn
  receiver_tensors = {'input_data': input_images} # As far as I understand this is needed to map the input to a name you can retrieve later
  return tf.estimator.export.ServingInputReceiver(features, receiver_tensors)

the answer's author states (in regards to receiver_tensors):

As far as I understand this is needed to map the input to a name you can retrieve later

This distinction is unclear to me. In practice, (see this colab), the same dictionary can be passed to both features and receiver_tensors.

From the source code of @estimator_export('estimator.export.ServingInputReceiver') (or the ServingInputReceiver docs:

• features: A Tensor, SparseTensor, or dict of string to Tensor or SparseTensor, specifying the features to be passed to the model. Note: if features passed is not a dict, it will be wrapped in a dict with a single entry, using 'feature' as the key. Consequently, the model must accept a feature dict of the form {'feature': tensor}. You may use TensorServingInputReceiver if you want the tensor to be passed as is.
• receiver_tensors: A Tensor, SparseTensor, or dict of string to Tensor or SparseTensor, specifying input nodes where this receiver expects to be fed by default. Typically, this is a single placeholder expecting serialized tf.Example protos.

After reading, it is clear to me what the purposes of features is. features is a dictionary of inputs that I then send through the graph. Many common models have just a single input, but you can or course have more.

So then the statement regarding receiver_tensors which "Typically, this is a single placeholder expecting serialized tf.Example protos.", to me, suggests that receiver_tensors want a singular batched placeholder for (Sequence)Examples parsed from TF Records.

Why? If the TF Records is fully preprocessed, then this is redundant? if it is not fully pre-processed, why would one pass it? Should the keys in the features and receiver_tensors dictionaries be the same?

Can someone please provide me with a more concrete example of the difference and what goes where, as right now

input_tensors = tf.placeholder(tf.float32, <shape>, name="input_tensors")
features = receiver_tensors =  {'input_tensors': input_tensors}

works... (even if maybe it shouldn't...)

Rozella

Answer 1

The job of the serving input function is to convert the received raw features into the processed features which your model function accepts.

receiver_tensors : These are the input placeholders. This is opening in your graph where you will receive your raw input features.

After defining this placeholder you perform transformations on these receiver tensors to convert them into features which are model acceptable. Some of these transformations will include:

• Pre-processing received data.
• Parsing example from tfrecord. (In case you are providing tfrecord as input to serving function)

features : Once you transform receive tensors features are obtained which are directly fed to your model function during prediction.

In your case pre-processing is not required for the data which you are providing to your serving input function. Hence features = receiver_tensors is working.

Contravallation

Answer 2

As far as I understand, SWAPNIL's answer is correct. I would share a example of mine.

Suppose the input of graph is a placeholder of shape [None, 64]

inputs = tf.placeholder(dtype=tf.float32, shape=[None, 64])
prediction = ... # do some prediction

But what we get from upstream are arrays of 32 float numbers and we will need to process them into shape [None, 64], for example, simple repeat them.

def serving_fn():
    inputs = tf.placeholder(dtype=tf.float32, shape=[None, 32])  # this is raw input
    features = tf.concat([inputs, inputs], axis=1)  # this is how we get model input from raw input
    return tf.estimator.export.TensorServingInputReceiver(features, inputs)

Of course we can do this process outside, and feed the estimator data just as we define the inputs of graph. In this case, we concatenate the inputs in the upstream process, and the raw input would be of shape [None, 64] So the function would be

def serving_fn():
    inputs = tf.placeholder(dtype=tf.float32, shape=[None, 64])  # this is raw input
    features = inputs  # we simply feed the raw input to estimator
    return tf.estimator.export.TensorServingInputReceiver(features, inputs)

Springhead

Answer 3

If you do the preprocessing inside TensorServingInputReceiver than receiver_tensors and features would be different. features will be passed to the model after the preprocessing inside TensorServingInputReceiver has been made. receiver_tensors are the input for the TensorServingInputReceiver and they can be in a tf.Example format

Barquentine