Keras embedding layer with variable length in functional API

Question

I have the following sequential model that works with variable length inputs:

m = Sequential()
m.add(Embedding(len(chars), 4, name="embedding"))
m.add(Bidirectional(LSTM(16, unit_forget_bias=True, name="lstm")))
m.add(Dense(len(chars),name="dense"))
m.add(Activation("softmax"))
m.summary()

Gives the following summary:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
embedding (Embedding)        (None, None, 4)           204       
_________________________________________________________________
bidirectional_2 (Bidirection (None, 32)                2688      
_________________________________________________________________
dense (Dense)                (None, 51)                1683      
_________________________________________________________________
activation_2 (Activation)    (None, 51)                0         
=================================================================
Total params: 4,575
Trainable params: 4,575
Non-trainable params: 0

However when I try to implement the same model in functional API I don't know whatever I try as Input layer shape doesn't seem to be the same as the sequential model. Here is one of my tries:

charinput = Input(shape=(4,),name="input",dtype='int32')
embedding = Embedding(len(chars), 4, name="embedding")(charinput)
lstm = Bidirectional(LSTM(16, unit_forget_bias=True, name="lstm"))(embedding)
dense = Dense(len(chars),name="dense")(lstm)
output = Activation("softmax")(dense)

And here is the summary:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input (InputLayer)           (None, 4)                 0         
_________________________________________________________________
embedding (Embedding)        (None, 4, 4)              204       
_________________________________________________________________
bidirectional_1 (Bidirection (None, 32)                2688      
_________________________________________________________________
dense (Dense)                (None, 51)                1683      
_________________________________________________________________
activation_1 (Activation)    (None, 51)                0         
=================================================================
Total params: 4,575
Trainable params: 4,575
Non-trainable params: 0

Kroo

Perhaps the functional API doesn't handle input layer with variable dimensions? – Clouded

Answer 1

Use shape=(None,) in the input layer, in your case:

charinput = Input(shape=(None,),name="input",dtype='int32')

Beckwith

Answer 2

Try adding the argument input_length=None to the embeddinglayer.

Dobbins