tf.lookup.StaticHashTable with lists (of arbitrary sizes) as values

Question

I want to associate to each person's name a list of numbers.

keys = ["Fritz", "Franz", "Fred"]
values = [[1, 2, 3], [4, 5], [6, 7, 8, 9]]

If I run the following:

import tensorflow as tf
table = tf.lookup.StaticHashTable(tf.lookup.KeyValueTensorInitializer(keys, values), default_value=0)

, I get a ValueError: Can't convert non-rectangular Python sequence to Tensor. since the lists are not of the same size and hence cannot be converted to a tf.Tensor.

Is there another way to associate the values of a tensor to lists of arbitrary shape?

Thank you for your help :)

Cain

Answer 1

StaticHashTable - as of TF 2.3 - cannot return multi-dimensional values, let alone ragged ones. So despite padding the values, creating a hash table like this:

keys = ["Fritz", "Franz", "Fred"]
values = [[1, 2, 3, -1], [4, 5, -1, -1], [6, 7, 8, 9]]
table_init = tf.lookup.KeyValueTensorInitializer(keys, values)
table = tf.lookup.StaticHashTable(table_init, -1)

will throw the following error:

InvalidArgumentError: Expected shape [3] for value, got [3,4] [Op:LookupTableImportV2]

To circumvent this, you can use Dense hash tables although it is in experimental mode. Neither dense nor static hash tables provide support for ragged keys or values. So your best bet is to pad your values, and create a dense hash table. During lookup, you can rag them back. The overall code looks like this:

keys = ["Fritz", "Franz", "Fred"]
values = [[1, 2, 3, -1], [4, 5, -1, -1], [6, 7, 8, 9]]
table = tf.lookup.experimental.DenseHashTable(key_dtype=tf.string, value_dtype=tf.int64, empty_key="<EMPTY_SENTINEL>", deleted_key="<DELETE_SENTINEL>", default_value=[-1, -1, -1, -1])
table.insert(keys, values)

And during lookup:

>>> tf.RaggedTensor.from_tensor(table.lookup(['Franz', 'Emil']), padding=-1)
<tf.RaggedTensor [[4, 5], []]>

Stamford