What are hidden units in individual LSTM cells?

Question

I am new to Neural Networks and found the concept of "hidden units" in individual LSTM cells. I understood the concept of having "gates"(input/output/forget etc.), but what are hidden units? I also understand that this is different from the number of unrolled LSTM cells in each layer(which is equal to the number of timesteps).

Galicia

Answer 1

Even though its a old question i would like to answer this question. When i started learning LSTM even i couldn't understand Hidden Unit,Return Sequence ,Return State in LSTM . Check the above diagram that i drew which would help you understand it. It is the representation of 3 Hidden Unit LSTM Layer

• inp = Input(shape=(2,))
• x = Embedding(50000, 5)(inp)
• x = LSTM(3,return_sequences=True)(x)

For detailed information check my detailed blog on Medium https://medium.com/@raqueebilahi/

Miscarriage

Hi @raqueeb shaikh, where is the three Hidden Units in your diagram ? there is just tow LSTM cells for the tow times steps for the tow words {Good, Day} – Cattalo

hi @Cattalo number of LSTM cells i.e 2 in this case for {Good,Day} is different from what hidden units are. When setting the HyperParameter in LSTM we can select hidden unit (2,3,4,5 or what ever you like). In the above diagram if you consider just one LSTM Cell {Good} there are pairs of sigmod and tanh fuction which take weighted sum from the embedding so those are the hidden units i.e 3 in this case. If i had suppose 1000 hidden unit that those sigmoid and tanh would be 1000 for each gate. Do let me know if you understood it :-) or any further clarification is required . – Miscarriage