How to write an LSTM in Keras without an Embedding layer?

Question

How do you write a simple sequence copy task in keras using the LSTM architecture without an Embedding layer? I already have the word vectors.

Kindly

Have you tried removing the embedding layer? – Fancie

Answer 1

If you say that you have word vectors, I guess you have a dictionary to map a word to its vector representation (calculated from word2vec, GloVe...).

Using this dictionary, you replace all words in your sequence by their corresponding vectors. You will also need to make all your sequences the same length, since LSTMs need all the input sequences to be of constant length. So you will need to determine a max_length value and trim all sequences that are longer and pad all sequences that are shorter with zeros (see Keras pad_sequences function).

Then you can pass the vectorized sequences directly to the LSTM layer of your neural network. Since the LSTM layer is the first layer of the network, you will need to define the input shape, which in your case is (max_length, embedding_dim, ).

This ways you skip the Embedding layer and use your own precomputed word vectors instead.

Petronilapetronilla

It is also possible to load pretrained embeddings into an Embedding layer with the weights argument, see this post. This way there is no need to fiddle with transformations later and one can pass in a sentence as list of integers to the model directly. – Darter

Answer 2

I had same problem after searching in Keras at "Stacked LSTM for sequence classification" part , I found following code might be useful: model = Sequential() model.add(LSTM(3NumberOfLSTM, return_sequences=True, input_shape=(YourSequenceLenght, YourWord2VecLenght)))

Kbp